PR #1137 Initial kernel changes to support GaLore

Initial kernel changes to support GaLore #1137

matthewdouglas wants to merge 8 commits into bitsandbytes-foundation:main from matthewdouglas:galore

matthewdouglas commented on 2024-03-19

TimDettmers marked this pull request as ready for review 1 year ago

TimDettmers commented on 2024-07-18

Titus-von-Koeller force-pushed the main branch from 774d0656 to 9b726798 1 year ago

Titus-von-Koeller force-pushed the main branch from 9b726798 to 78007346 1 year ago

matthewdouglas added Enhancement

matthewdouglas assigned

matthewdouglas 1 year ago

matthewdouglas force pushed from 61189fcd to 91ea416f 1 year ago

Initial kernel changes for 2-state optimizers to support GaLore

a5f95521

Experimental implementation for bnb.optim.GaLoreAdamW8bit

57de0d62

Cleanup

759c78d5

Fix mistake

b2a01ec9

One more time

31854da9

Support eturn_outputs buffer option for 1-state optimizers

b1fb85b8

matthewdouglas force pushed from 91ea416f to b1fb85b8 1 year ago

fix wheel build on windows

0c87b70a

Introducte GaLoreWrappedParameter to decouple grad from param

7e3b5ff1

matthewdouglas added Optimizers

matthewdouglas closed this 130 days ago

Reviewers

TimDettmers

Assignees

matthewdouglas

Labels

Enhancement Optimizers

Milestone

No milestone