transformers
[CB] Size the KV pool with the per-rank head count under TP
#48566
Open
Open
[CB] Size the KV pool with the per-rank head count under TP #48566
qgallouedec
wants to merge
6 commits into
main
from
cb-tp-aware-pool
CB: size the KV pool with the per-rank head count under tensor parall…
0e504cdd
Merge branch 'main' into cb-tp-aware-pool
d499f3fb
Merge branch 'main' into cb-tp-aware-pool
fa832beb
Merge branch 'main' into cb-tp-aware-pool
46f56b5d
Merge branch 'main' into cb-tp-aware-pool
b5b5460a
Merge branch 'main' into cb-tp-aware-pool
3662cd85
Login to write a write a comment.
Login via GitHub