Megatron-DeepSpeed
fix: doc_idx offset when merging indexed dataset files
#66
Merged

fix: doc_idx offset when merging indexed dataset files #66

adammoody
adammoody fix: doc_idx offset when merging indexed dataset files
ad626807
adammoody
thomasw21
thomasw21 approved these changes on 2021-08-16
thomasw21 thomasw21 merged c680954e into main 4 years ago
adammoody adammoody deleted the fixdocidx branch 4 years ago
adammoody

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
Milestone