lighteval
669c3448 - tests: make harness metric comparison work with pytest 9.2

Commit
24 days ago
tests: make harness metric comparison work with pytest 9.2 pytest 9.2.0 raises TypeError when pytest.approx() is given nested data structures. Corpus-level metric inputs (mcc, loglikelihood_f1, multi_f1_numeric, target_perplexity, bits_per_byte) are compared as lists of dicts containing lists, so test_unit_harness_metrics failed for them (CI stopped on the first one, mcc, because of `pytest -x`). Add a small recursive approx_equal helper that walks dicts/lists and only applies pytest.approx to numeric leaves, with plain equality otherwise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author
Nathan Habib
Parents
Loading