DeepSpeed

1b892a75 - Read rope_theta from rope_parameters across Inference V2 (#8345)

Read rope_theta from rope_parameters across Inference V2 (#8345) Follow-up to #8341, which covers the same drift in the v1 kernel-injection policy. Separate engine, separate change; #8341 noted this exposure but did not touch it. ### What breaks transformers 5.0 folded the rotary settings into `config.rope_parameters` and dropped the `rope_theta` attribute. Eight V2 models still read the attribute. Seven of them read it directly and raise; `exaone4` reads it through a `getattr` default and quietly uses that default instead: | V2 model | on master (transformers 5.8.0) | with this PR | | --- | --- | --- | | `llama_v2` | `AttributeError: 'LlamaConfig' object has no attribute 'rope_theta'` | 10000.0 | | `mistral` | `AttributeError: 'MistralConfig' …` | 10000.0 | | `mixtral` | `AttributeError: 'MixtralConfig' …` | 1000000.0 | | `phi` | `AttributeError: 'PhiConfig' …` | 10000.0 | | `phi3` | `AttributeError: 'Phi3Config' …` | 10000.0 | | `qwen_v2` | `AttributeError: 'Qwen2Config' …` | 10000.0 | | `qwen_v2_moe` | `AttributeError: 'Qwen2MoeConfig' …` | 10000.0 | | `exaone4` | **silently 1000000.0** | 10000.0 | `exaone4` is the worse of the two. `Exaone4Config` carries `rope_parameters['rope_theta'] == 10000.0`, so the `getattr` default puts a 100x rotary base into `RotateHalfConfig` with nothing raised — wrong frequencies rather than a startup failure. `requirements-inf.txt` asks for `transformers>=4.32.1` with no upper bound, so 5.x is in range. ### The fix `exaone4_5` already reads both spellings — added with the model in #8121: ```python theta = rope_parameters.get("rope_theta", getattr(config, "rope_theta", None)) ``` Hoist the same lookup onto `DSTransformerModelBase` as a `rope_theta` property. `DSMoETransformerModelBase` extends it, so all eight models are covered by one place and each call site becomes `theta_base=self.rope_theta`. The attribute is tried first, so pre-5.0 installs take exactly the path they take today. When neither spelling carries a base it raises instead of guessing one — that is the only behaviour change beyond the fix, and it replaces `exaone4`'s silent 1e6. `exaone4_5` is left alone; its own helper also handles the nested per-layer `sliding_attention` dict, which is specific to that model. ### Test `tests/unit/inference/v2/model_implementations/test_rope_theta.py` — the three layouts, the precedence between them, the raise, plus a parametrization over the eight real configs from the installed transformers: ``` 13 passed ``` ``` tests/unit/inference/v2/model_implementations/ 19 passed yapf --diff / flake8 clean ``` I exercised the property and the configs, not a full V2 engine run against downloaded checkpoints. --------- Signed-off-by: alanhuangyoo <alanhuangyoo@gmail.com> Signed-off-by: Masahiro Tanaka <tanaka.masahiro@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com> Co-authored-by: Masahiro Tanaka <tanaka.masahiro@gmail.com>