FIX Minimal target module optimization bug w/ IA³ (#2432)
Fixes #2429
During PEFT model initialization, we have an optimization/compression
step where we check the target_modules attribute and, if it's very long,
try to find a minimal subset that targets the same modules. If we find
it, we reduce the target_modules to that minimal set. This is done
mostly to prevent some cases (e.g. in diffusers) that result in hundreds
of target_modules being checked against thousands of module names,
slowing down initialization.
There is an issue with this when using IA³. There, we additionally have
the feedforward_modules attribute, which must be subset of
target_modules. When target_modules is shrunk, the subset check will
fail. This PR fixes this by simply skipping the compression step for
IA³.
It would be possible to adjust the logic to also shrink
feedforward_modules, but it's not quite as forward, since the latter may
not be identical to target_modules, so there would have to be extra
logic to account for that. At the end of the day, this is too much
effort for what's pretty much an edge case, so the simple solution is
implemented.
