DeepSpeed

6093db0e - Fix HybridEngine OPT generation with modern cache interfaces (#8523)

Fix HybridEngine OPT generation with modern cache interfaces (#8523) ## Problem Related to #8197. HybridEngine injects legacy tuple-cache OPT layers into modern Cache-based decoders. Actual OPT generation fails on Transformers4.51.3 (`ValueError: too many values to unpack`) and5.14.1 (`AttributeError: tuple has no attribute size`). ## Change Keep OPT decoders exposing `cache_position` or `past_key_values` on the existing native-generation fallback, which logs that inference acceleration is unavailable. Legacy OPT injection remains enabled. This is a compatibility fallback, not modern-Cache kernel support or a performance optimization. It does not duplicate the merged profiling work. Add policy coverage, an offline two-rank training/generation regression, and a compatibility note. The regression updates tiny OPT weights10 times per rank, varies prompt length, and compares greedy outputs against an independent Hugging Face model loaded with each updated state. ## Validation Two NVIDIA L20 GPUs (physical1 and4), torch2.13.0+cu130: - Transformers4.51.3 and5.14.1: two-rank FP32 training/generation tests pass;10 updates per rank, exact output-token equality against native OPT after every update. - Transformers5.14.1: policy, two-GPU integration, rollout/profiling suite: **37 passed**,18 warnings,15.82s. - Transformers4.44.2: original native-kernel FP16 HybridEngine matrix on both ranks, each with8 shapes,112 measured calls,32 train/eval transitions. Both pass output/mask consistency and finite profiling checks; observed live-memory range0 bytes per rank. - Changed-file pre-commit hooks all pass after formatting. GPU tests ran the patch on8950dc33219042995faa96980c25468e2dfd4efd. The PR is based on1b892a75dfec96245491597bb496daf39d5c300d; the changed production file and pre-existing policy test were identical between those bases. No inference-v2 behavior is covered here. Reproduce the submitted regression with `pytest -q tests/unit/hybrid_engine/test_he_policy.py tests/unit/hybrid_engine/test_he_opt_cache.py tests/unit/runtime/rollout/test_hybrid_engine_rollout.py` on two GPUs with a modern Transformers installation. The accepted standalone two-rank runs execute the same test body through torchrun. ## Draft limitations This fallback provides native `engine.module.generate()` with data-parallel training; it does not provide inference TP, continuous-batching native-cache operations, CUDA Graphs, or ZeRO3 validation. No throughput improvement claim. Modern tests use FP32 and tiny random weights; the legacy acceleration matrix uses FP16. Compiler headers and architecture8.9 must be available for existing HybridEngine workspace operations. Earlier test-launch attempts hit a queue.py/std-library naming collision and a missing CUDA profiler include path; those environment failures are excluded from passing counts. The original OPT compatibility exceptions are retained as before-fix evidence. Signed-off-by: 0z5a <192209249+0z5a@users.noreply.github.com> Co-authored-by: 0z5a <192209249+0z5a@users.noreply.github.com> Co-authored-by: Ma, Guokai <guokai.ma@gmail.com>