deno
df9398d7 - feat(test): add retry and repeats options to Deno.test (#35053)

Commit
88 days ago
feat(test): add retry and repeats options to Deno.test (#35053) Migrating a test suite from a third-party runner like Vitest to `deno test` is harder than it should be, partly because there's no built-in way to tolerate or surface flaky tests. This adds two per-test options that mirror Vitest's semantics: `retry` re-runs a failing test and passes if any attempt passes, while `repeats` runs the test several times and requires every run to pass. They compose, so each repetition may itself be retried. Matching `--retry` and `--repeats` flags set a default for the whole run; a test that sets its own option takes precedence, including an explicit `0` that opts the test out of a flag-provided default. The runner already drives one JS call per test from Rust, so this wraps that call in an attempt/repetition loop. Each attempt re-runs the beforeEach and afterEach hooks and captures a fresh sanitizer baseline, which keeps every attempt's leak check independent and makes a resource leak retryable like any other failure. A failed-but-retried attempt is surfaced through a new informational event so the reporter can print it, and a test that ultimately passes after a retry is counted as `flaky` in the summary line. Hook errors and uncaught errors are intentionally not retried. A couple of limitations worth calling out: there is no `deno.json` configuration for retry/repeats yet (CLI flag and per-test option only); and the `junit` and `tap` reporters don't yet treat retried attempts specially, so a retried test re-registers its steps under fresh ids and emits duplicate `<testcase>`/step entries in those reporters (the pretty and dot reporters drop the discarded attempts). When a test with steps passes on a later attempt, the final attempt's passing steps are counted but not re-printed individually (the earlier failed attempt's steps are still shown). None of these affect correctness of the pass/fail verdict or the flaky count. Closes https://github.com/denoland/deno/issues/19882
Author
Parents
Loading