Release each BF16 gradient once immediate_grad_update has accumulated it
With immediate_grad_update, BF16_Optimizer adds each parameter's BF16
gradient into its FP32 accumulation buffer as soon as autograd finishes it,
in a hook named accumulate_hp_grads_and_remove_lp, but it never removed the
BF16 gradient. #5328 zeroed it there and #6508 reverted that because
zeroing frees nothing. So every BF16 gradient stays alive beside the FP32
buffer through the end of backward and the gradient all-reduce, 6.7 GB per
rank for Qwen3-30B-A3B at EP16, where an allocator trace puts the step's
peak once the loss is chunked.
Reduction, clipping and the step read the FP32 buffer, so the hook now
drops the BF16 gradient after accumulating it, except under graph
harvesting, which replays kernels on fixed gradient addresses. A training
test checks that immediate accumulation gives bit-identical FP32
gradients and parameters to epilogue accumulation over two micro-batches,
with no BF16 gradient left after backward.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: yh0903 <helloyu0903@gmail.com>