pytorch-lightning
Add `save_on_exception` option to `ModelCheckpoint`
#20916
Merged
Merged
Add `save_on_exception` option to `ModelCheckpoint` #20916
Borda
merged
59 commits into
Lightning-AI:master
from
vsey:feat/ModelCheckpointException
add saving of checkpoint if an exception is raised
0f731674
import callback to checkpoint test file
136e59a9
add test for exception in training callbacks
e0dae535
split test for save checksave point on expection for expetions in tra…
2113acc2
add extra condition for checking if we should save on exception
7d750e69
add for saving checkpoint on exeption if the exception occurs in a va…
34e598ac
add test for save model chekpoint on exception for exception in train…
d4d933be
disable trainer prog bar for test of model checkpoint on exception
9f6063b8
model checkpoint on eception split trainer setup over two lines
02477d5b
remove trainling braket from shoukd_save_on_eception condition
e5b0498e
Merge branch 'master' into feat/ModelCheckpointException
8bc93e21
![pre-commit-ci[bot]](https://avatars.githubusercontent.com/in/68672?v=4&s=60)
[pre-commit.ci] auto fixes from pre-commit.com hooks
985c1e1e
Update save checkpoint on exception tests to use a shorter more preci…
f0502ecb
switch default on save on checkpoint on exception to false to don't i…
99af7ed1
checkpoint on exception put callback tests into a pytest prametrization
c092385d
change doc string to reflect new default value for save on exception …
904bd743
checkpoint on exception add test function for exception in callback
3a3204e3
Merge branch 'master' into feat/ModelCheckpointException
0b1eb775
add prefix option to generate checkpoint file name
467c57bb
add exception prefix to checkpoints saved on exception
8ba63819
add test to test prefix for checkpoint name
3076ea1a
add test for exceptions at diffrent position in a model
d78ea3e2
add description to on exception hook in model checkpoint
42bbac12
add test to check saving on exception in all relevalnt callback posit…
c4b80631
also print exception when saving checkpoint
2ca6dab5
test checkpointing on exception in varoius model steps
9e9e580e
remove deviders in test_model_checkpoint
d2f74e98
add test for run conditions for save checkpoint on exception
ac33670c
Borda
commented
on 2025-06-23
Borda
commented
on 2025-06-23
Borda
commented
on 2025-06-23
Update model_checkpoint on_exception run condition to follow common c…
0dc9f244
Merge branch 'master' into feat/ModelCheckpointException
2aa00add
vsey
changed the title
vsey
changed the title
add checkpoint path variable in test_on_exceptio_incallback and set w…
c62f0e13
remove exception prefix as this is already covert by last flag
f16b771b
use pytest parametrization for testing callback on exception
09ba24fb
add missing callback hooks for Test Troubled Callback and order them …
c921875d
add missing test hooks of lighning module to test save chekpoint on e…
fd3de65a
change datatype of on_exception hook from Exception to BaseException …
7dfdbc5a
Merge branch 'Lightning-AI:master' into feat/ModelCheckpointException
c2447cb1
Borda
changed the title
Merge branch 'master' into feat/ModelCheckpointException
f2cd64fd
change default prefix from empty string to Optinal to better convey m…
048ad860
Merge branch 'master' into feat/ModelCheckpointException
71baa1cc
Merge branch 'master' into feat/ModelCheckpointException
e81e291e
Merge branch 'master' into feat/ModelCheckpointException
28c61e1e
Merge branch 'master' into feat/ModelCheckpointException
6b67499a
Merge branch 'master' into feat/ModelCheckpointException
7e990cab
Added save_on_exception option for ModelCheckpoint to changelog
f0cf90b5
Fix tense of changelog entrie to be in line with rest of changelog
921c9ee1
change changelog entrie tense back
e388ee7a
Merge branch 'master' into feat/ModelCheckpointException
7c70d467
Merge branch 'master' into feat/ModelCheckpointException
97900b1b
Merge branch 'master' into feat/ModelCheckpointException
36f4bc92
Merge branch 'master' into feat/ModelCheckpointException
74271845
Merge branch 'master' into feat/ModelCheckpointException
03364787
Merge branch 'master' into feat/ModelCheckpointException
744cbbf9
[pre-commit.ci] auto fixes from pre-commit.com hooks
c2c7e93d
Borda
commented
on 2025-08-09
Borda
commented
on 2025-08-09
Apply suggestions from code review
37a5369f
Borda
commented
on 2025-08-09
Change the comment for the save_checkpoint_on_exception run condition…
43eae27e
[pre-commit.ci] auto fixes from pre-commit.com hooks
f4428e5d
split
c272633b
[pre-commit.ci] auto fixes from pre-commit.com hooks
2d3f833c
Borda
approved these changes
on 2025-08-11
Reviewers
Assignees
No one assigned
Labels
pl
Milestone
No milestone
Login to write a write a comment.
Login via GitHub