peft
3b2ebf1b - FIX Bug that prevents BOFT from loading 2 adapters (#2068)

Commit
1 year ago
FIX Bug that prevents BOFT from loading 2 adapters (#2068) There was a bug in BOFT that made it impossible in some circumstances to load more than one adapter (creating more than 1 adapter was possible though). This was because a code path that adjusts boft_n_butterfly_factor was only visited when creating a fresh adapter, but not when updating with the 2nd adapter. This was fixed by moving this code path from the BOFT layer's __init__ method to update_layer. A test for loading multiple adapters was added. Since this was a gap in our test suite, this test will be applied to all appropriate PEFT methods, not only BOFT, but the others methods are all passing without needing further changes. For good measure, I also added BOFT to the test suite that checks multiple active adapters. These tests would have also passed without the fix in this PR, since these tests do not load multiple adapters but instead create them, which always worked. Still it's better to have these tests as well.
Parents
Loading