[CB] Scale the per-token memory estimate by the TP size (#48566)
* CB: size the KV pool with the per-rank head count under tensor parallelism
PagedAttentionMemoryHandler estimated bytes per block and the activation peak with the unsharded head counts, while each rank only holds num_heads // tp_size of them, so under TP the pool came out tp x too small: Qwen3-32B at tp4 got 221k tokens and used 36 of 80 GiB per GPU, 14B at tp2 365k. The handler now takes the TP size when the KV projections are in the TP plan and divides both counts by it. Qwen3-32B tp4, 1024 short prompts, hub kernels: 6,523 -> 11,277 tok/s (pool 221k -> 884k tokens, 450 -> 672 concurrent requests); 14B tp2 365k -> 731k tokens.
* Scale the per-token activation estimate by the TP size in the CB memory handler
* More precise name + TODO
* Nits
* MAke new test run on CPU
* Style
* Nitpicking
---------
Co-authored-by: Rémi Ouazan <83456801+remi-or@users.noreply.github.com>
Co-authored-by: remi-or <remi.pierre_o@orange.fr>