DeepSpeed
1abb504d - Move rollout engine to deepspeed/runtime/rollout/ as reusable module

Commit
40 days ago
Move rollout engine to deepspeed/runtime/rollout/ as reusable module - deepspeed/runtime/rollout/base.py: ABC + dataclasses (RolloutEngine, RolloutRequest, RolloutBatch, SamplingConfig) - deepspeed/runtime/rollout/hybrid_engine_rollout.py: full implementation with naive HF generate + continuous batching (shared prefix, early exit, KV left-trim) paths - examples/opsd/opsd/rollout/hybrid_engine.py: thin wrapper importing from deepspeed.runtime.rollout.HybridEngineRollout - Added smoke_hybrid_cb32.json and smoke_hybrid_n32.json test configs
Author
Parents
Loading