transformers
f2c9ee24 - Keep the cache under activation checkpointing when gradients are off (#50890)

Commit
2 days ago
Keep the cache under activation checkpointing when gradients are off (#50890) * Activation checkpointing: run the layer as is when gradients are off, cache included * Keep use_cache under activation checkpointing when gradients are off, as the layers keep the cache * Count only the layer calls with gradients on in the checkpointing test HrmText runs part of its layers under torch.no_grad(), which are no longer checkpointed.
Author
Parents
Loading