PR #37842 Add z-loss to Bamba for v2

Add z-loss to Bamba for v2 #37842

Cyrilvallez merged 18 commits into huggingface:main from daviswer:bamba-add-zloss

Remove const

6989543b

Fix arg ref

f8878532

Sharded save

b4e73f72

Merge branch 'huggingface:main' into main

5e29bdf4

Add z_loss flag

11a4dee4

Add modeling zloss

570f9a3d

Demodularize clm forward for zloss

11406796

Also demodularize init for z_loss flag

aa4ec14f

fabianlim commented on 2025-04-28

vasqu commented on 2025-04-28

PR comments (mostly modularizing right)

5ebda1f2

Demodularize forward

26bd7d90

vasqu commented on 2025-05-07

Better name zloss and explain typematch

210613a5

vasqu commented on 2025-05-07

Fully propagate coeff name

f321778e

style fixes

aad9194f

vasqu commented on 2025-05-07

zloss default float

b1d3a735

daviswer marked this pull request as ready for review 1 year ago

Remove conflicting annotations

109e7727

Merge branch 'main' into bamba-add-zloss

1547ccdf

Cyrilvallez approved these changes on 2025-05-28

Cyrilvallez enabled auto-merge (squash) 1 year ago

Merge branch 'main' into bamba-add-zloss

f9e01335

disabled auto-merge 1 year ago
Manually disabled by user

Merge branch 'main' into bamba-add-zloss

ee01e0fc

Cyrilvallez merged a6f0e2b6 into main 1 year ago

daviswer deleted the bamba-add-zloss branch 1 year ago

Reviewers

Cyrilvallez

vasqu

fabianlim

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers Add z-loss to Bamba for v2 #37842 Merged

Add z-loss to Bamba for v2 #37842

transformers
Add z-loss to Bamba for v2
#37842

Merged