PR #326 Add option to normalize loss per target

Add option to normalize loss per target #326

Muennighoff merged 11 commits into t0loading from lossseq

Tmp lossseq

462efd99

Efficient loss normalization

992446c8

Reuse variable

616cfe86

Simplify division

900c8356

thomasw21 commented on 2022-08-10

Add norm_target_loss arg

7bc1dd20

Muennighoff changed the title ~~TMP: Lossseq~~ Add option to normalize loss per target 3 years ago

Muennighoff requested a review from

thomasw21 3 years ago

thomasw21 commented on 2022-08-16

Clarify loss on targets & remove kwarg

fce1a98e

Loss mask is already float

2e7554d7

Move norm to batch pipe

a6b26240

Muennighoff requested a review from

thomasw21 3 years ago

thomasw21 commented on 2022-08-17

Reshape loss mask

549f4993

Move view

d9a91feb

Merge branch 't0loading' into lossseq

456327c1

Muennighoff merged 1e77844c into t0loading 3 years ago

Muennighoff deleted the lossseq branch 3 years ago

Reviewers

thomasw21

Assignees

No one assigned

Labels

None yet

Milestone

No milestone