transformers
[CB] Scale the per-token memory estimate by the TP size
#48566
Merged

[CB] Scale the per-token memory estimate by the TP size #48566

ydshieh merged 23 commits into main from cb-tp-aware-pool
qgallouedec
qgallouedec CB: size the KV pool with the per-rank head count under tensor parall…
0e504cdd
qgallouedec Merge branch 'main' into cb-tp-aware-pool
d499f3fb
HuggingFaceDocBuilderDev
qgallouedec Merge branch 'main' into cb-tp-aware-pool
fa832beb
qgallouedec Merge branch 'main' into cb-tp-aware-pool
46f56b5d
qgallouedec Merge branch 'main' into cb-tp-aware-pool
b5b5460a
qgallouedec qgallouedec requested a review from remi-or remi-or 70 days ago
qgallouedec Merge branch 'main' into cb-tp-aware-pool
3662cd85
qgallouedec Merge remote-tracking branch 'upstream/main' into cb-tp-aware-pool
8cdd4421
qgallouedec Scale the per-token activation estimate by the TP size in the CB memo…
45fbe5dd
qgallouedec qgallouedec changed the title [CB] Size the KV pool with the per-rank head count under TP [CB] Scale the per-token memory estimate by the TP size 45 days ago
qgallouedec Merge branch 'main' into cb-tp-aware-pool
49e22201
github-actions
qgallouedec Merge branch 'main' into cb-tp-aware-pool
af40f05f
qgallouedec
qgallouedec qgallouedec requested a review from ArthurZucker ArthurZucker 17 days ago
qgallouedec Merge remote-tracking branch 'origin/main' into tfu-48566
d72cb23a
qgallouedec Merge branch 'main' into cb-tp-aware-pool
d814effd
qgallouedec Merge branch 'main' into cb-tp-aware-pool
1dde962d
qgallouedec Merge branch 'main' into cb-tp-aware-pool
d2c239f0
ArthurZucker
ArthurZucker approved these changes on 2026-11-19
remi-or Merge branch 'main' into cb-tp-aware-pool
41812626
remi-or More precise name + TODO
0b25b113
remi-or Nits
d5f93723
remi-or MAke new test run on CPU
ca59a4b3
remi-or Style
baa0cb5d
remi-or
remi-or Nitpicking
f3bfe72e
remi-or
remi-or approved these changes on 2026-11-19
qgallouedec Merge branch 'main' into cb-tp-aware-pool
8e2fb318
qgallouedec qgallouedec enabled auto-merge 1 day ago
remi-or Merge branch 'main' into cb-tp-aware-pool
0b28d834
qgallouedec Merge branch 'main' into cb-tp-aware-pool
92535779
ydshieh ydshieh merged d0da3b84 into main 5 hours ago
ydshieh ydshieh deleted the cb-tp-aware-pool branch 5 hours ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
Milestone