llm-foundry
d39debbe - Fix Dtensor initialization (#1820)

Commit
271 days ago
Fix Dtensor initialization (#1820) Existing initialization utils can not handle DTensor slice properly, as DTensor slice could be a new DTensor (with a different sharding layout/placement) instead of a view to the original Tensor. Solution: Init on DTensor.full_tensor and then copy the shard of the full_tensor to original Dtensor.
Author
Parents
Loading