Add EXAONE-MoE implementations (#43080)
* Add EXAONE-MoE implementations
Co-authored-by: Junwon Hwang <nuclear1221@gmail.com>
Co-authored-by: Kibong Choi <rlqhd26@naver.com>
* Add documentations of EXAONE-MoE
* Fix EXAONE configs
* Change model prefix into ExaoneMoe
* Remove unnecessary classes and update EXAONE MoE config
* Fix EXAONE MoE modeling & config
* Update docs
* Add simple test for EXAONE MoE
* Fix docs & style
* Minor fix
* fixup
* Address PR feedback
* Address PR feedback
* Fix docstring
* Add RotaryEmbeddingConfigMixin to EXAONE MoE
* Update dummy model for test
* Fix test and config of EXAONE MoE
* some quick fixes
* fix
* more fixes
* update expectations
* update id
* style
---------
Co-authored-by: LG-AI-EXAONE <exaonemodels@lgresearch.ai>
Co-authored-by: Kibong Choi <rlqhd26@naver.com>
Co-authored-by: vasqu <antonprogamer@gmail.com>
Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
