dataset-viewer
Parallelize tokenization to enable fast on-the-fly indexing
#3178
Merged
Overview
Commits
10
Changes
View On GitHub

Parallelize tokenization to enable fast on-the-fly indexing #3178

lhoestq merged 10 commits into main from faster-on-the-fly-indexing
lhoestq
lhoestq parallelize tokenization
02d8e552
lhoestq mypy
3764ab8e
lhoestq debugging
43a4b371
lhoestq improve admin auth
f65392e2
lhoestq remove unused ingestion line
a3d70eb0
lhoestq debugging
df0b1391
lhoestq fix division by zero
3c0efce9
lhoestq revert debugging
81c32c68
lhoestq mypy
e4e10203
lhoestq bandit
888fc4fd
lhoestq lhoestq merged 4df09e52 into main 294 days ago
lhoestq lhoestq deleted the faster-on-the-fly-indexing branch 294 days ago
lhoestq
lhoestq commented on 2025-04-23

Login to write a write a comment.

Login via GitHub

Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum