lighteval
b634749d - Fix DetailsLogger hashing with xxhash 4.0 (#1402)

Commit
32 days ago
Fix DetailsLogger hashing with xxhash 4.0 (#1402) * Encode strings before hashing with xxhash xxhash 4.0 no longer accepts str input to xxh64 (TypeError: Strings must be encoded before hashing), which breaks DetailsLogger and turns tests red on main. Encode to UTF-8 explicitly, matching xxhash 3.x's implicit behaviour so the resulting hashes are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Update Qwen2.5-VL-3B mmmu_pro reference for sample 16 With current dependencies the model answers D instead of B on sample 16 (gold is C), so extractive_match stays 0 and aggregate scores are unchanged. The drift was hidden until now because the slow tests crashed on xxhash. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Nathan Habib <nathan_habib@Nathans-MacBook-Pro.local> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Author
Parents
Loading