auto-round
fix critic bug for gradient_accumulate_steps!=1 and reduce cpu memory of lm-head tuning
#97

Merged

fix critic bug for gradient_accumulate_steps!=1 and reduce cpu memory of lm-head tuning #97

wenhuach21 merged 4 commits into main from fix_layer_output_mem_consumption_issue

fix layer output memory consumption issue

0ed4380e

[pre-commit.ci] auto fixes from pre-commit.com hooks

173b5eec

WeiweiZhang1 requested a review from

wenhuach21 1 year ago

wenhuach21 changed the title ~~fix layer output memory consumption issue~~ reduce cpu memory consumption of lm-head tuning 1 year ago

wenhuach21 commented on 2024-04-29

wenhuach21 changed the title ~~reduce cpu memory consumption of lm-head tuning~~ fix critic bug for gradient_accumulate_steps!=1 and reduce cpu memory consumption of lm-head tuning 1 year ago

wenhuach21 changed the title ~~fix critic bug for gradient_accumulate_steps!=1 and reduce cpu memory consumption of lm-head tuning~~ fix critic bug for gradient_accumulate_steps!=1 and reduce cpu memory of lm-head tuning 1 year ago

fixtypos

98764bd9

[pre-commit.ci] auto fixes from pre-commit.com hooks

54b0cb08

wenhuach21 approved these changes on 2024-04-29

wenhuach21 merged 7dd02eb4 into main 1 year ago

wenhuach21 deleted the fix_layer_output_mem_consumption_issue branch 1 year ago

Reviewers

wenhuach21

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

auto-round fix critic bug for gradient_accumulate_steps!=1 and reduce cpu memory of lm-head tuning #97 Merged

fix critic bug for gradient_accumulate_steps!=1 and reduce cpu memory of lm-head tuning #97

auto-round
fix critic bug for gradient_accumulate_steps!=1 and reduce cpu memory of lm-head tuning
#97

Merged