[tests] Expand Reactant test coverage: layers, losses, optimisers, precision (#2750)
* [tests] Expand Reactant test coverage
Add tests on the Reactant device for:
- gradient parity with Zygote for layers not in TEST_MODELS: conv options,
pooling, upsampling, normalisation (also in test mode), recurrent layers
over sequences and cells, Attention, activations, containers, Dropout and
AlphaDropout in test mode;
- gradient parity for every loss, w.r.t. the model and input only;
- trainstep! trajectory parity with eager for the optimiser rules;
- f16/bf16 training and Float64 kept on the device.
Compile the Reactant gradient helpers at HIGHEST dot_general/convolution
precision, so that CUDA's TF32-like default doesn't eat into the tolerances.
Known upstream bugs are marked broken: FluxML/Optimisers.jl#234, #235 (CUDA
only), #236 and #237.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [tests] Merge and trim the Reactant tests; Optimisers 0.4.10 fixes
Optimisers 0.4.10 fixes RAdam, AMSGrad/AdaGrad setup on GPU, Rprop and
AccumGrad on Reactant, so their trajectory tests are no longer broken.
Merge the ten new files into four (layers, conv, losses, optimisers) and
chain several layers per gradient case, roughly halving the number of
compilations: each file and case pays Julia and XLA compilation, which
dominates their time on CI.
Compile the trajectory steps at full precision: on CUDA, TF32 rounding
flips signs that Rprop amplifies into a different trajectory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Link Reactant.jl#3504 from the onehot Embedding test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>