Trainer: average the gradients of parameters FSDP2 does not shard
PEFT adapters are attached after fully_shard, so they are plain replicated tensors that
FSDP2 never reduce-scatters, and the native-FSDP path skips the DDP wrap. Each rank kept the
gradient of its own batch: in fp32 the lora_B gradients were off by 2x against a single-GPU
reference while the loss matched exactly. A post-accumulate-grad hook averages them, which is
what DDP did for the same parameters.
(cherry picked from commit 213f2ce71cdd2eca576d6f26e134fdbdef05c03c)