PR #708 Dist testing backend fixes, etc.

add support for passing init_method to distributed backend

jeffra committed 5 years ago

make args.local_rank optional for engine, update docs

jeffra committed 5 years ago

int cast rank

jeffra committed 5 years ago

use fmt string

jeffra committed 5 years ago

remove args.local_rank assert and remove adding local rank on ds args tests

jeffra committed 5 years ago

refactor some tests related to local rank

jeffra committed 5 years ago

comment out some backward tests (based on discussion with @RezaYazdaniAminabadi)

jeffra committed 5 years ago

remove empty_grad=True, if zero-2 enabled this will cause a hang

jeffra committed 5 years ago

refactor to address comment

jeffra committed 5 years ago

DeepSpeed Dist testing backend fixes, etc. #708 Merged