DeepSpeed

739079b5 - Add kv_trim_threshold parameter to HybridEngineRollout

Add kv_trim_threshold parameter to HybridEngineRollout Controls when KV cache left-trim fires during continuous batching: - 0: never trim - 1: trim every step (most aggressive) - 16 (default): trim only when >=16 tokens of common padding accumulated