[Test] Print RNG of a failed testset and add option to set it (#56260)
Also, add a keyword option to `@testset` to let users override the seed
used there, to make testsets more replicable.
To give you a taster of what this PR
enables:
```
julia> using Random, Test
julia> @testset begin
@test rand() == 0
end;
test set: Test Failed at REPL[2]:2
Expression: rand() == 0
Evaluated: 0.559472630416976 == 0
Stacktrace:
[1] top-level scope
@ REPL[2]:2
[2] macro expansion
@ ~/repo/julia/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:1713 [inlined]
[3] macro expansion
@ REPL[2]:2 [inlined]
[4] macro expansion
@ ~/repo/julia/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:679 [inlined]
Test Summary: | Fail Total Time
test set | 1 1 0.9s
ERROR: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.
Random seed for this testset: Xoshiro(0x2e026445595ed28e, 0x07bb81ac4c54926d, 0x83d7d70843e8bad6, 0xdbef927d150af80b, 0xdbf91ddf2534f850)
julia> @testset rng=Xoshiro(0x2e026445595ed28e, 0x07bb81ac4c54926d, 0x83d7d70843e8bad6, 0xdbef927d150af80b, 0xdbf91ddf2534f850) begin
@test rand() == 0.559472630416976
end;
Test Summary: | Pass Total Time
test set | 1 1 0.0s
```
This also works with nested testsets, and testsets on for loops:
```
julia> @testset rng=Xoshiro(0xc380f460355639ee, 0xb39bc754b7d63bbf, 0x1551dbcfb5ed5668, 0x71ab5a18fec21a25, 0x649d0c1be1ca5436) "Outer" begin
@test rand() == 0.0004120194925605336
@testset rng=Xoshiro(0xee97f5b53f7cdc49, 0x480ac387b0527d3d, 0x614b416502a9e0f5, 0x5250cb36e4a4ceb1, 0xed6615c59e475fa0) "Inner: $(i)" for i in 1:10
@test rand() == 0.39321938407066637
end
end;
Test Summary: | Pass Total Time
Outer | 11 11 0.0s
```
Being able to see what was the seed inside a testset and being able to
set it afterwards should make replicating test failures which only
depend on the state of the RNG much easier to debug.
julia
6136893e - [Test] Print RNG of a failed testset and add option to set it (#56260)
