Time based validation support (#21071)
* Add wall-clock val_check_interval with epoch alignment and timer reset
* Adjust checkpointing frequency when time-based validation is active
* Add tests for time based validation
* Update docs for time based validation through val_check_interval
* changelog
* doctest
* Parametrize val_check_interval test to include different types
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Nicki Skafte Detlefsen <skaftenicki@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
