transformers
e532b190 - Laguna: support per-element output gating (#46690)

Commit
26 days ago
Laguna: support per-element output gating (#46690) * Laguna: support per-element output gating Dispatch on config.gating: True/"per-head" keeps one gate per head (g_proj=num_heads), "per-element" sizes g_proj at num_heads*head_dim. Regenerate modeling/configuration and add gating shape tests. * Address review: consolidate gating test + add integration test Collapse the gating shape checks into one parameterized test_gating_variations, add an end-to-end per-element gate test, and a slow integration test asserting golden logits on a tiny published checkpoint (poolside/Laguna-tiny-per-element). * small nits --------- Co-authored-by: Vasqu <antonprogamer@gmail.com>
Author
Parents
Loading