DeepSpeed

b61a2217 - Distributed testing (#6)

Distributed testing (#6) * Adds distributed_test decorator and some unit tests. * Setting NCCL backend. * Parametrizes test. * rank -> local_rank * Temporarily disable CUDA initialization.