transformers
feat: support indivisible shards for TP model loading and TPlizing.
#37220

Merged

feat: support indivisible shards for TP model loading and TPlizing. #37220

ArthurZucker merged 8 commits into huggingface:main from kmehant:error-out-tp-stride

github-actions marked this pull request as draft 1 year ago

kmehant force pushed from 57fd98d0 to 867dda55 1 year ago

kmehant force pushed from 58a42fe5 to 7ccc0715 1 year ago

kmehant marked this pull request as ready for review 1 year ago

github-actions requested a review from

MekkCyber 1 year ago

github-actions requested a review from

SunMarc 1 year ago

kmehant changed the title ~~Error out tp stride~~ feat: support indivisible shards for TP model loading and TPlizing. 1 year ago

ArthurZucker commented on 2025-04-08

kmehant force pushed from cd2a37fd to c8a1e65f 1 year ago

kmehant force pushed from c8a1e65f to 40ceafe4 1 year ago

ArthurZucker commented on 2025-04-09

kmehant force pushed from 40ceafe4 to 20710b5e 1 year ago

kmehant force pushed from 5b7894b1 to 09e103f0 1 year ago

kmehant force pushed from 09e103f0 to 5ea18b04 1 year ago

ArthurZucker commented on 2025-04-11

kmehant force pushed from 4e615007 to eb179253 1 year ago

kmehant force pushed from eb179253 to f8f2e6a6 1 year ago

kmehant force pushed from cb875dca to f5725e8d 1 year ago

kmehant force pushed from f5725e8d to cacf6ee5 1 year ago

kmehant force pushed from cacf6ee5 to d7fe1ac6 1 year ago

kmehant force pushed from d7fe1ac6 to 1363e837 1 year ago

kmehant force pushed from 1363e837 to 0cee2447 1 year ago

kmehant force pushed from 0cee2447 to 1b0d5123 1 year ago

kmehant force pushed from 1b0d5123 to 575f6cf2 1 year ago

SunMarc requested a review from

ArthurZucker 1 year ago

kmehant force pushed from 575f6cf2 to 4ecb49e8 1 year ago

kmehant force pushed from 4359d06b to 853b1101 1 year ago

kmehant force pushed from 853b1101 to 722a93d2 1 year ago

ArthurZucker commented on 2025-04-30

kmehant force pushed from 722a93d2 to 371487e7 1 year ago

kmehant force pushed from 2b5473fe to 8ca09f0c 1 year ago

SunMarc requested a review from

ArthurZucker 1 year ago

kmehant force pushed from 8ca09f0c to 36e6c887 1 year ago

kmehant force pushed from 36e6c887 to 5e630797 1 year ago

kmehant force pushed from 5e630797 to 21cf72f1 1 year ago

kmehant force pushed from 21cf72f1 to 25f186c4 1 year ago

kmehant force pushed from 25f186c4 to 0da69ae0 1 year ago

kmehant force pushed from 0da69ae0 to d3b063ce 1 year ago

kmehant force pushed from d3b063ce to 47a71f78 1 year ago

kmehant force pushed from 47a71f78 to 47c4cb95 1 year ago

kmehant force pushed from 47c4cb95 to 4fcd6801 1 year ago

kmehant force pushed from 4fcd6801 to e1895b16 1 year ago

feat: support uneven loading and sharding

abf88bd2

fix: allow for empty tensor computations

01a1dfb1

test: add llama1b test case

fc15bff9

due to q_proj colwise it has to be multi of 2

3c1d808e

refactor: use slice API

4ca1b8a3

refactor: use slice API

5544ad68

refactor: use slice API

5f86e6af

refactor: use slice API

6e06c9db

kmehant force pushed from e1895b16 to 6e06c9db 1 year ago

ArthurZucker approved these changes on 2025-07-01

ArthurZucker enabled auto-merge (squash) 1 year ago

ArthurZucker merged def96632 into main 1 year ago

Reviewers

ArthurZucker

MekkCyber

SunMarc

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers feat: support indivisible shards for TP model loading and TPlizing. #37220 Merged

feat: support indivisible shards for TP model loading and TPlizing. #37220

transformers
feat: support indivisible shards for TP model loading and TPlizing.
#37220

Merged