Add option to dump activations on all ranks (#5455)
* Add option to dump activations on all ranks
* address review comments
* review comments
* Fix review comment
Co-authored-by: Aishwarya <aibhanda@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
