DeepSpeed

6474bc55 - Read rope_theta from rope_parameters in the Llama injection policy (#8341)

Read rope_theta from rope_parameters in the Llama injection policy (#8341) Fixes #8340. ### What breaks `DS_LLAMAContainer.create_module` resolves `rope_theta` as: ```python if hasattr(self.policy.client_module.self_attn, 'config'): _config.rope_theta = self.policy.client_module.self_attn.config.rope_theta else: _config.rope_theta = self.policy.client_module.self_attn.rope_theta ``` transformers 5.0 folded the rotary settings into `config.rope_parameters` and dropped the attribute. `LlamaAttention` still has `.config`, so the first branch is taken and it raises. This is not specific to the DeepSeek checkpoint in the issue — on transformers >= 5.0 it reproduces with a default `LlamaConfig`. Checked on 5.8.0: ``` transformers 5.8.0 LlamaConfig().rope_theta -> AttributeError LlamaConfig().rope_parameters -> {'rope_theta': 10000.0, 'rope_type': 'default'} ``` and against a real module, both branches are dead: ``` self_attn has .config True self_attn has .rope_theta False self_attn.config has rope_theta False self_attn.config has rope_parameters True current code -> AttributeError: 'LlamaConfig' object has no attribute 'rope_theta' ``` ### The fix Try the old spellings first, then `rope_parameters['rope_theta']`, so pre-5.0 installs take exactly the path they take today and nothing changes for them. This is the same drift #7443 adapted to. The `num_heads` accessor it fixed alongside still resolves on 5.8 (`config.num_attention_heads` is intact), so `rope_theta` is the only one that moved again. `requirements-dev.txt` asks for `transformers>=4.51.3` with no upper bound, so 5.x is in range. Verified that `rope_theta` is the only accessor that moved: against a real `LlamaDecoderLayer` on 5.8.0, `get_hidden_heads()`, `attention()`, `mlp()` and `layernorm()` all resolve once this one is fixed. I exercised the policy and container layer, not a full `init_inference()` run against a downloaded checkpoint on GPU. ### Test `tests/unit/module_inject/test_llama_rope_theta.py` covers the three layouts plus the precedence between them and the not-found case — CPU only, no model download: ``` 6 passed ``` End to end against a real `LlamaAttention` on 5.8.0: ``` current code AttributeError: 'LlamaConfig' object has no attribute 'rope_theta' with this PR 10000.0 ``` ``` tests/unit/module_inject/ 43 passed yapf --diff / flake8 clean ``` I did not touch `inference/v2`. Six of its model implementations read `self._config.rope_theta` directly and `exaone4` is the only one using a `getattr` default, so they likely have the same exposure — but that is a different engine and a different change, and I have not reproduced it. --------- Signed-off-by: alanhuangyoo <alanhuangyoo@gmail.com> Co-authored-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com>