transformers

f8a1e1fb - [`FA`] Fix head dim manipulation for more special models (#49862)

[`FA`] Fix head dim manipulation for more special models (#49862) * fix models head dim * style * revert these * adjust init values instead * minimize model sizes