HunYuan opensource (#39606)
* merge opensource_hunyuan
* add head_dim
* fix assertion error
* fix seen_tokens
* ready_for_upstream (merge request !17)
Squash merge branch 'ready_for_upstream' into 'main'
* fix configuration type&docstring
* fix style
* ready_for_upstream (merge request !18)
Squash merge branch 'ready_for_upstream' into 'main'
* add doc
* fix testcode
* fix configuration type&docstring
* rename base model
* remove assert
* update
* remove tiktoken
* update
* fix moe and code style (#3)
* update
* fix format
* update
* revert makefile
* fix moe config
* fix numel()
* remove prepare_inputs_for_generation
* fix kv_seq_len
* add docs/toctree
* remove unused paramter&add licence
* add licence
* remove unused paramter
* fix code
* dense modular
update import
fix
fix
use mistralmodel
fix qknorm
add sliding_window
make style
fix
dense done
hunyuan moe
fix import
fix modular
fixup
fixup
* update model path
* fix mlp_bias
* fix modular
* Fix modeling (#5)
* fix attention
* use llamamodel
* fix code
* Fix qk (#6)
* fix qk_norm
* fix
* fix modual
* Fix moe (#7)
* fix some moe code
* fix einsum
* try top1
* use top1
* Fix rotary (#8)
* fix rotary
* fix modeling
* fix modular
* fix testcode
* remove A13B unit test
* Fix moe v1 (#9)
fix moe & gate
* Fix gate norm (#10)
* add norm_topk_prob
* Fix testcase (#11)
* fix&skip test
* Fix testcase (#12)
* skip testcase
* Fix norm topk (#13)
* hardcode norm_topk_prob
* fix testcase
---------
Co-authored-by: pridejcyang <pridejcyang@tencent.com>
Co-authored-by: Mingji Han <mingjihan@tencent.com>
