transformers
2cc368c8 - Set all num_workers to 0 to eliminate all forked subprocesses

Commit
55 days ago
Set all num_workers to 0 to eliminate all forked subprocesses preprocessing_num_workers=1 still forks (nch went 3→2 not 0), and the CoW spike from even 1 fork at ~1GB parent PSS caused wav2vec2 to still peak at 5.8GB. The script default is None (no fork) — passing any integer forks that many workers. dataloader_num_workers=0 loads data inline in the main process with no persistent worker subprocesses. For tests running max_steps=2-10 on tiny fixture datasets there is no throughput benefit to workers. With all num_workers=0 no child processes are spawned → no CoW → PSS reflects only the actual model/optimizer/data footprint. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Author
Parents
Loading