transformers
feat: add support for tensor parallel training workflow with accelerate
#34194

Merged

feat: add support for tensor parallel training workflow with accelerate #34194

SunMarc merged 8 commits into huggingface:main from kmehant:tp

kmehant force pushed 1 year ago

kmehant force pushed to f1f3b6bc 1 year ago

kmehant force pushed 1 year ago

kmehant changed the title ~~feat: add support for tensor parallel using Pytorch 2.0~~ feat: add support for tensor parallel using Pytorch 1 year ago

kmehant force pushed to 87bebaf2 1 year ago

kmehant force pushed from 87bebaf2 to 0a4fe4e3 1 year ago

kmehant force pushed from 0a4fe4e3 to e79f4d17 1 year ago

kmehant changed the title ~~feat: add support for tensor parallel using Pytorch~~ feat: add support for tensor parallel training workflow with accelerate 1 year ago

kmehant force pushed from e79f4d17 to d585ebd7 1 year ago

SunMarc approved these changes on 2025-01-29

SunMarc requested a review from

muellerzr 1 year ago

SunMarc requested a review from

ArthurZucker 1 year ago

kmehant force pushed from 19bffe21 to b8f2b40b 1 year ago

kmehant force pushed from b8f2b40b to 5c065ac9 1 year ago

kmehant force pushed from 5c065ac9 to 7e8a2c20 1 year ago

kmehant force pushed from 2e0b0cde to d073f052 1 year ago

kmehant force pushed from d073f052 to 177bd947 1 year ago

muellerzr approved these changes on 2025-02-12

kmehant force pushed from 177bd947 to b720640e 1 year ago

kmehant force pushed from 57d6a18b to cbc89dee 1 year ago

kmehant force pushed from cbc89dee to 52f184fb 1 year ago

ArthurZucker commented on 2025-02-12

kmehant force pushed from 52f184fb to 804a4f74 1 year ago

ArthurZucker approved these changes on 2025-02-13

kmehant force pushed from 99ff7f7a to f16d53d8 1 year ago

kmehant force pushed from f16d53d8 to 33f5ad26 1 year ago

kmehant force pushed from 33f5ad26 to ace502d2 1 year ago

kmehant force pushed from ace502d2 to 6bd51b15 1 year ago

feat: add support for tensor parallel flow using accelerate

a700b53a

fix: add tp degree to env variable

aae7ed90

fix: add version check for accelerate to allow TP

fd798335

docs: tensor parallelism

1846b36f

nit: rename plugin name

76cbddfc

fix: guard accelerate version before allow tp

a93b6f42

docs: add more docs and updates related to TP

65f87ddb

kmehant force pushed from 6bd51b15 to 65f87ddb 1 year ago

Merge branch 'main' into tp

f2820d4f

SunMarc merged c3ba5330 into main 1 year ago

Reviewers

ArthurZucker

SunMarc

muellerzr

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers feat: add support for tensor parallel training workflow with accelerate #34194 Merged

feat: add support for tensor parallel training workflow with accelerate #34194

transformers
feat: add support for tensor parallel training workflow with accelerate
#34194

Merged