enhancements and fixes for FSDP and DeepSpeed (#532)
* checkpointing enhancements and fixes for FSDP and DeepSpeed
* resolving comments
1. Adding deprecation args and warnings in launcher for FSDP
2. Handling old configs to work with new launcher args wrt FSDP.
3. Reverting changes to public methods in `checkpointing.py` and handling it in `Accelerator`
4. Explicitly writing the defaults of various FSDP options in `dataclasses` for readability.
* fixes
1. FSDP wrapped model being added to the `_models`.
2. Not passing the env variables when args are None.
* resolving comments
* adding FSDP for all the collective operations
* adding deepspeed and fsdp tests
1. Removes mrpc datafiles and directly relies on HF datasets as it was throwing `file not found` error when running from within `tests` folder. Updating `moke_dataloaders` as a result.
2. adding `test_performance.py`, `test_memory.py` and `test_checkpointing.py` for multi-gpu FSDP and DeepSpeed tests
* reverting `mocked_dataloader` changes
* adding FSDP tests
* data files revert
* excluding fsdp tests from `tests_core`
* try 2
* adding time delay to avoid `torchrun` from crashing at times leading which causing flaky behaviour
* reducing the time of tests
* fixes
* fix
* fixes and reduce time further
* reduce time further and minor fixes
* adding a deepspeed basic e2e test for single gpu setup
