model:add Zaya1 8B (Zyphra) (#32071)
* model:add Zaya1 8B (Zyphra)
- Added the support for Zaya.
- Fixed the SSM_CONV case, so the model runs on cuda without abort.
Please enter the commit message for your changes. Lines starting
* Made the following changes based on the bot reviewer feedback:
1) Added runtime asserts when loading wrong KV head size, unsupported conv kernel size.
2) Removed the static cast usage.
3) Fixed double load of tokenizer.
4) Updated res_scale_in to be input tensor.
* - Moved the res_scale_in back to layer 0 to fix the cuda bug
* i) Removed the EOS addition.Team agreed that updating Tokenizer is the right move.
ii) Fixed formatting
* Fixed the comment to make it clear
---------
Co-authored-by: varun <varun_subramanian@zyphra.com>