Clean up XCodec and other codecs (#40348)
* Clean up xcodec addition.
* Clean up config.
* Switch to fixtures test.
* Small stuff.
* Polish XCodec and standardize across codecs.
* Update src/transformers/models/xcodec/modeling_xcodec.py
Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
* Format and fix test.
* Update tol.
---------
Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
