optimum-intel
a7b766eb - Add load_in_4bit option for OVModelForCausalLM (#538)

Commit
1 year ago
Add load_in_4bit option for OVModelForCausalLM (#538) * Initial code for load_in_4_bit * Dataset does not work * Intermediate changes * Make it working with dataset * Style * Fixed small issue * Fixed failed tests * Style * Comment failed tests due to NNCF 2.8 * Commented failed tests until new NNCF release * Added tests for load_in_4bit * Added awq option. Included NNCF package into openvino extra. * Rolled back including nncf into openvino extra * Style * Fixed tests * Fixed issues with models larger than 1B. Added tests. * Style * Fixed issues. Applied comments. * Removed unnecessary exception * Applied more comments * Fixed issue * Make quantization_config a part of OVConfig in OVQuantizer * Fixed issue with Transformers * Fixed test * Changed the naming. Added additional tests * Fixed tests * Fixed tests * Applied more comments * Style
Author
Parents
Loading