Support for pruning delays in Adagrad Optimizer (#34527)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/34527
Adding support for prune_delays and prune ratios in Adagrad optimizer.
Test Plan:
Tested via unit tests in masked_adagrad_optimizer_test. Added unit test for prune_delay versions of MaskedAdagrad
buck build caffe2/caffe2/fb/optimizers:masked_adagrad_optimizer_test; buck-out/gen/caffe2/caffe2/fb/optimizers/masked_adagrad_optimizer_test#binary.par
buck test caffe2/caffe2/fb/dper/layer_models/tests/split_1:sparse_nn_test -- 'test_pruning'
All Dper tests passed https://our.intern.facebook.com/intern/testinfra/testrun/7599824380741217
Reviewed By: chocjy
Differential Revision: D20313419
fbshipit-source-id: 5c2c8d4e0fc2ec538bcd6f145c6b87a2381f90f3
