Add a ResNet-18 / Tiny-ImageNet-200 example (#2696)
* Add ResNet-18 / Tiny-ImageNet training example
Self-contained example (own Project.toml) that trains a small-image
ResNet-18 variant on Tiny-ImageNet-200, streaming the dataset from the
HuggingFace Hub via HuggingFaceDatasets.jl and training with Flux.train!
on the GPU, with a @main script entry point.
Trained with train!(...; caching_allocator=false). By default train!
wraps each step in a cross-step allocation cache that keeps every
allocation of a step alive for reuse, so a step's peak memory becomes the
sum of its allocations (~2x the true working set); on the cold first step
the retained cuDNN algorithm-search workspaces balloon that past a 32 GB
GPU and OOM. That option (Flux #2695) isn't in a registered release yet,
so Project.toml points at this repo's Flux via [sources].
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* examples/resnet_tinyimagenet: rely on train! memory defaults
The new `train!` defaults (no caching allocator + adaptive paced GC) are
exactly right for this conv net, so drop the explicit memory keywords from the
training call and explain the default memory behavior instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* add pytorch example
* examples/resnet_tinyimagenet: add seed, LR schedule, and precision/regularization options
Add --seed, --weight-decay, --clip-norm (gradient L2 clipping via OptimiserChain), and --bfloat16 (BFloat16 training), plus a per-epoch cosine learning-rate schedule and per-epoch training time in the log. The @main entry point now returns nothing so the script exits 0 (returning the model made Julia's exit-code conversion throw).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* examples/resnet_tinyimagenet/pytorch: use device iterator in flux_bench
Switch the Flux benchmark to the DEVICE(train_loader) device iterator (matching the example), add an explicit exit() so the included example's @main does not launch a full training run at exit, and drop the now-redundant deviceiter_bench.jl.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* cleanup
* examples/resnet_tinyimagenet: remove bfloat16 support
Remove the --bfloat16 option and the BFloat16s dependency; bfloat16 training will be explored in a separate PR. The other options (seed, cosine LR schedule, weight decay, gradient clipping) are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
