WoRA integration into PEFT #2872
sambhavnoobcoder
wants to merge
12 commits into
huggingface:main
from
sambhavnoobcoder:feature/wora-integration
Add WoRA core implementation with weighted direction and learnable al…
ae503aae
Update LoraLayer to support WoRA with alpha/beta parameter storage
e42daab5
Update Linear, Embedding, and Conv layer classes to support WoRA vari…
dca53dd4
Add use_wora parameter propagation in LoraModel layer creation
e8733fb5
Fix parameter ordering in update_layer methods for Python syntax comp…
a11c3384
Fix WoRA parameter registration and config validation for complete gr…
cc8fbc07
Enable gradient flow through WoRA alpha and beta parameters
633a8500
Set requires_grad=True for WoRA alpha and beta parameters
209752e4
Move wora_alpha and wora_beta to adapter_layer_names for trainability
7d9725b0
Rename wora_alpha/wora_beta to lora_wora_alpha/lora_wora_beta for pre…
01b18601
Add WoRA tests to test_lora_variants.py
9f2ec488
Add WoRA tests and fix gradient flow issues
a0dc6e0c
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub