transformers
4f2a6b91 - Keep reading the tensor parallel size from accelerate in `get_tp_size`

Commit
61 days ago
Keep reading the tensor parallel size from accelerate in `get_tp_size` `_get_num_items_in_batch` and the loss scale now go through `get_tp_size`, which only reads `model._tp_size`. Tensor parallelism configured through `TrainingArguments(parallelism_config=...)` rather than `DistributedConfig` leaves that unset, so both lost their `tp_size` divisor. Fall back to `parallelism_config.tp_size`, which is what they read before.
Author
Parents
Loading