lighteval
78dbee22 - fix rolling perplexity on the transformers backend (#1252)

Commit
45 days ago
fix rolling perplexity on the transformers backend (#1252) - score the query as the continuation when choices is None (the_pile/wikitext) - shift logits by one so tokens line up with their predictions - stop the sample cache dropping PERPLEXITY results - add a rolling-perplexity test and check the cached wrapper returns all docs
Author
Parents
Loading