DeepSpeed
8fb2c9a2 - Give rollout cache fakes the per-layer config Transformers now reads (#8730)

Commit
48 days ago
Give rollout cache fakes the per-layer config Transformers now reads (#8730) ## Summary - `cpu-torch-latest` installs Transformers from git `main`. `StaticCache` now reads `config.per_layer_config` before choosing layer types. - The three continuous-generation tests used a `SimpleNamespace` cache config, so that read raised `AttributeError`. Real `PreTrainedConfig` objects already have this property. - The fakes now return a one-element `per_layer_config`, which Transformers treats as full attention. ## Test plan - [ ] `cpu-torch-latest` unit tests, especially the three continuous-generation cases - [ ] Local run could not reproduce the CI Transformers SHA; this machine's Transformers fails later on `Cache.update(cache_position=...)` Made with [Cursor](https://cursor.com) Signed-off-by: tunji-ruwase_snow <tunji.ruwase@snowflake.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Parents
Loading