Fix ESM models buffers (#24576)

Commit

2 years ago

Fix ESM models buffers (#24576) * Fix ESM models buffers * Remove modifs * Tied weights keys are needed silly * quality