Fix flaky external test (#3348)
* Fix flaky external test
* CHANGELOG
* Undo accidental diff
* Fix flaky external test
* CHANGELOG
* Undo accidental diff
* Remove from changelog
* Use bunny file
---------
Co-authored-by: Abubakar Abid <abubakar@huggingface.co>
