dataset-viewer

4df09e52 - Parallelize tokenization to enable fast on-the-fly indexing (#3178)

Parallelize tokenization to enable fast on-the-fly indexing (#3178) * parallelize tokenization * mypy * debugging * improve admin auth * remove unused ingestion line * debugging * fix division by zero * revert debugging * mypy * bandit