Add Mistral Large 4 (#50121)
* Mappings
* Modular converter tmp fix
* Modular first draft
* Fix Rope modular
* Indexer
* Fix pretrained and modular
* Fix dims
* Conversion mappings
* Nits
* Thread shared_kv_...
* Fix softmaxscaling
* Missing layer type
* Nope in indexer fix
* Missing cache
* Config update
* Conversion, start
* Inversed mapping
* Fix config
* Next step in converter
* Fix kv sgaring
* softcapping
* missing converter
* Add quantization conversion
* processor
* Conversion script
* Fix conversions for permuted rope
* Nits
* Conversion otkenizer ihappens before config
* Fixes
* Fix routing not in fp32
* Resolve some TODOs
* Re-wire indexer
* Reorganization
* Modular nits
* Style
* Fix
* Fix repo
* Doc
* Review 1/n
* Disable test of text model only
* Fix modular converter broken in rebase and run it
* Update to fit new cfg setup
* Remove norm_topk_prob and add the gate weights to the keep_in_fp32 list of modules
* Refactor conversion script
* Fix RoPE conversion bug
* Fix different name scales bug
* Review
* Fix for a problem that will never happen
* Tests, lacking integration tests
* Fix missing router logits
* Move logits softcapping o the text config
* Style
* Fix repo
* Init alpha to zeros
* revert cache -> crop should not be called of uninitialized
* use consolidated.safetensors natively, no new index
* add dtype to config for auto dtype
* fix
* do not shard the attn for now
* remove the hack, I fixed it upstream
* MAke alpha a buffer
* fix exporter tests
* Nits
* ignore
* style
* this was upstreamed
---------
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
