Use new `per_layer_config` for Gemma 4 so that heterogeneous attention config is explicit (#47384)
* Use new `per_layer_config` for Gemma4 so that heterogeneous attention config is explicit
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* Remove the now legacy fields
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* Fix docstrings
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* Fix test harness
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* Magic default
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* Reinstate unused rope kwarg
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* Make `per_layer_config` more flexible/useful
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* Propagate with modular
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* Less ugly solution to heterogeneous values in `validate_architecture`
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* ruff
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* update test
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* Update test again
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* Fix tests
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* make fix-repo
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
* update other tests
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
---------
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>