transformers
[mistral] Support passing `head_dim` through config (and do not require `head_dim * num_heads == hidden_size`)
#32050
Merged
Merged
[mistral] Support passing `head_dim` through config (and do not require `head_dim * num_heads == hidden_size`) #32050
xenova
merged
4 commits into
main
from
mistral-head_dim
Allow `head_dim` to be set in Mistral config
5771d59c
Add docstring
6a963544
Do not require `head_dim * num_heads == hidden_size`
dd3e56f5
[run-slow] mistral
3f8e6e42
ArthurZucker
approved these changes
on 2024-07-18
amyeroberts
approved these changes
on 2024-07-18
Login to write a write a comment.
Login via GitHub