transformers
5b7dcb0d - MRoPE continued (#48594)

Commit
73 days ago
MRoPE continued (#48594) * add and fix * wait what, why would they even override if it is same as qwen?! * LOL why would we cat if rope is interleaved * fix last * axial but needs a fix * maybe last fix? * fix for the test, also this is more "standard" * skip * lost partial rotary factor * just noticed - kinda weird rope but oke * that still doesn't mul each value, gotta iterate * nit * keep float32 for rope, this should still be equivalent i hope * really didnt want it, but num precision matters more than i thought
Author
Parents
Loading