peft
intorduce AdaDoRA implementation using both DoRA and AdaLoRa together
#2969
Open
Open
intorduce AdaDoRA implementation using both DoRA and AdaLoRa together #2969
OrMullerHahitti
wants to merge
31 commits into
huggingface:main
from
OrMullerHahitti:main
Enable DoRA in AdaLoRA and add GLUE trainer
8232ea45
Implement AdaDoRA: Integrate DoRA with AdaLoRA's SVD decomposition
5d561913
REF: Update AdaDoraLinearLayer to allow gradient-based magnitude adap…
8f83cef9
Update src/peft/tuners/adalora/dora.py
79ca71ea
Update src/peft/tuners/adalora/variants.py
74d35c0d
style: apply formatting and remove unnecessary comment
788d84c5
refactor: address PR review comments
c57f1784
Remove dora_init as it is from a self expiriment (could add if intere…
b3cda10e
change name of example dir
e2854858
refactor: remove unused update_magnitude_after_pruning method
2bc5b65c
refactor: move variant resolution to AdaLoraLayer.update_layer
2ea5b710
refactor: get_weight_norm now receives pre-computed lora_weight
c596f7ba
Apply suggestion from @BenjaminBossan
33dffb01
Delete uv.lock
49c02024
revert comments back to upper case (where changed from original)
33415ad9
Merge branch 'main' of github.com:OrMullerHahitti/AdaDoRa
8784b710
feat: add reproducer scripts for AdaLoRA `p.grad` None issue
0b893b8f
refactor: integrate AdaDoRA into test
0d2dc186
optimize DoRA `lora_weight` computation by reusing the precomputed va…
5efce8ce
removing `lora_magnitude_vector` from parameter names.
bb53fdff
link papers
1aedd19a
link papers
ced5bb34
fix: adapt AdaDoRA to upstream's config-based API
c1815943
Merge branch 'main' of github.com:OrMullerHahitti/AdaDoRa
d8641e14
Merge branch 'main' into main
c2fcc73d
Merge remote-tracking branch 'upstream/main'
4add93a7
fix: resolve merge conflicts from upstream API refactoring
6345da1b
fix: resolve merge conflicts from upstream API refactoring
4c2d4eee
fix: remove remaining conflict markers
f7e96ad2
fix: remove all remaining conflict markers
93de4f6a
remove more trailing conflicts
da271117
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub