DeepSpeed
5f0d177f - DeepNVMe GDS (#5852)

Commit

303 days ago

DeepNVMe GDS (#5852) PR for the GDS AIO code. --------- Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com> Co-authored-by: Logan Adams <loadams@microsoft.com> Co-authored-by: Ubuntu <deepspeed@H100-VM2.shlnn55tgwve1eacvp21ie45dg.jx.internal.cloudapp.net> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>

References

#5852 - DeepNVMe GDS

Author

jomayeri

Parents

c2e3a706

Files43

.github/workflows
- nv-pre-compile-ops.yml
csrc
- aio
  - py_lib
    - deepspeed_aio_op_desc.cpp
    - deepspeed_aio_op_desc.h
    - deepspeed_aio_thread.cpp
    - deepspeed_aio_thread.h
    - deepspeed_cpu_op.cpp
    - deepspeed_cpu_op.h
    - deepspeed_py_aio.cpp
    - deepspeed_py_aio.h
    - deepspeed_py_aio_handle.cpp
    - deepspeed_py_aio_handle.h
    - deepspeed_py_copy.cpp
    - deepspeed_py_copy.h
    - deepspeed_py_io_handle.cpp
    - deepspeed_py_io_handle.h
    - py_ds_aio.cpp
  - py_test
    - aio_bench_generate_param.py
    - aio_bench_perf_sweep.py
    - ds_aio_args.py
    - ds_aio_basic.py
    - ds_aio_handle.py
    - ds_aio_job.py
    - run_read_sweep.sh
    - run_write_sweep.sh
    - test_ds_aio.py
    - test_ds_aio_utils.py
    - validate_async_io.py
- gds
  - py_lib
    - deepspeed_gds_op.cpp
    - deepspeed_gds_op.h
    - deepspeed_gds_utils.h
    - deepspeed_py_gds_handle.cpp
    - deepspeed_py_gds_handle.h
    - py_ds_gds.cpp
  - py_test
    - validate_gds.py
deepspeed
- ops/gds
  - __init__.py
- runtime/swap_tensor
  - aio_config.py
  - constants.py
  - partitioned_param_swapper.py
op_builder
- async_io.py
- builder.py
- gds.py
tests/unit/ops/aio
- test_aio.py
- test_gds.py

DeepSpeed 5f0d177f - DeepNVMe GDS (#5852)

DeepSpeed
5f0d177f - DeepNVMe GDS (#5852)