[CB] Scale the per-token memory estimate by the TP size #48566
CB: size the KV pool with the per-rank head count under tensor parall…
0e504cdd
Merge branch 'main' into cb-tp-aware-pool
d499f3fb
Merge branch 'main' into cb-tp-aware-pool
fa832beb
Merge branch 'main' into cb-tp-aware-pool
46f56b5d
Merge branch 'main' into cb-tp-aware-pool
b5b5460a
Merge branch 'main' into cb-tp-aware-pool
3662cd85
Merge remote-tracking branch 'upstream/main' into cb-tp-aware-pool
8cdd4421
Scale the per-token activation estimate by the TP size in the CB memo…
45fbe5dd
qgallouedec
changed the title [CB] Size the KV pool with the per-rank head count under TP [CB] Scale the per-token memory estimate by the TP size 45 days ago
Merge branch 'main' into cb-tp-aware-pool
49e22201
Merge branch 'main' into cb-tp-aware-pool
af40f05f
Merge remote-tracking branch 'origin/main' into tfu-48566
d72cb23a
Merge branch 'main' into cb-tp-aware-pool
d814effd
Merge branch 'main' into cb-tp-aware-pool
1dde962d
Merge branch 'main' into cb-tp-aware-pool
d2c239f0
Merge branch 'main' into cb-tp-aware-pool
41812626
More precise name + TODO
0b25b113
Nits
d5f93723
MAke new test run on CPU
ca59a4b3
Style
baa0cb5d
Nitpicking
f3bfe72e
remi-or
approved these changes
on 2026-11-19
Merge branch 'main' into cb-tp-aware-pool
8e2fb318
Merge branch 'main' into cb-tp-aware-pool
0b28d834
Merge branch 'main' into cb-tp-aware-pool
92535779
ydshieh
merged
d0da3b84
into main 5 hours ago
ydshieh
deleted the cb-tp-aware-pool branch 5 hours ago
Assignees
No one assigned
Login to write a write a comment.
Login via GitHub