[BNB] Throw `ValueError` when trying to cast or assign (#20409)
* `bnb` ValueError when tries to cast or assign
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* remove docstrings
* change error log
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>