transformers
7d5bc4f3 - Use new `per_layer_config` for Gemma 4 so that heterogeneous attention config is explicit (#47384)

Commit
122 days ago
Use new `per_layer_config` for Gemma 4 so that heterogeneous attention config is explicit (#47384) * Use new `per_layer_config` for Gemma4 so that heterogeneous attention config is explicit Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Remove the now legacy fields Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Fix docstrings Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Fix test harness Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Magic default Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Reinstate unused rope kwarg Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Make `per_layer_config` more flexible/useful Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Propagate with modular Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Less ugly solution to heterogeneous values in `validate_architecture` Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * ruff Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * update test Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Update test again Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Fix tests Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * make fix-repo Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * update other tests Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> --------- Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Author
Parents
Loading