transformers
Faster generation using AWQ + Fused modules
#27411
Merged
Overview
Commits
51
Changes
View On GitHub

Faster generation using AWQ + Fused modules #27411

younesbelkada merged 51 commits into huggingface:main from younesbelkada:awq-fused-modules
younesbelkada
younesbelkada v1 fusing modules
6c995f9f
younesbelkada add fused mlp support
85cc9c74
HuggingFaceDocBuilderDev
younesbelkada Merge remote-tracking branch 'upstream/main' into awq-fused-modules
b6cd5549
younesbelkada up
7ffbaa3e
younesbelkada fix CI
05b5f62e
younesbelkada block save_pretrained
8670aa20
younesbelkada fixup
9ee6b381
younesbelkada Merge remote-tracking branch 'upstream/main' into awq-fused-modules
f8d41775
younesbelkada small fix
1a8c9156
younesbelkada add new condition
b541b4dd
younesbelkada Merge remote-tracking branch 'upstream/main' into awq-fused-modules
2ea1f470
younesbelkada Merge branch 'awq-fused-modules' of https://github.com/younesbelkada/…
024b737d
younesbelkada add v1 docs
a7d74f80
younesbelkada add some comments
85e1e3b7
younesbelkada
younesbelkada younesbelkada requested a review from amyeroberts amyeroberts 2 years ago
younesbelkada Merge branch 'main' into awq-fused-modules
3e6ba9bf
amyeroberts
amyeroberts commented on 2023-11-16
younesbelkada Merge remote-tracking branch 'upstream/main' into awq-fused-modules
26194d01
younesbelkada style
f160a162
younesbelkada fix nit
14c820d3
younesbelkada adapt from suggestion
03d8dff6
younesbelkada add check
0a08551c
younesbelkada change arg names
234165f6
younesbelkada change variables name
03980d92
younesbelkada Update src/transformers/integrations/awq.py
8a68a232
younesbelkada style
21f68794
younesbelkada split up into 3 different private methods
cde53efe
younesbelkada more conditions
8517e325
younesbelkada more checks
b187c070
younesbelkada add fused tests for custom models
c3e32ab9
younesbelkada fix
d3c77538
younesbelkada fix tests
4113c45e
younesbelkada final update docs
0bd1b0ca
younesbelkada younesbelkada marked this pull request as ready for review 2 years ago
younesbelkada younesbelkada requested a review from amyeroberts amyeroberts 2 years ago
younesbelkada
younesbelkada final fixes
61db4309
younesbelkada
younesbelkada commented on 2023-11-22
younesbelkada fix importlib metadata
cd37d323
amyeroberts
amyeroberts approved these changes on 2023-11-24
SunMarc
SunMarc commented on 2023-11-28
SunMarc
SunMarc commented on 2023-11-28
younesbelkada Merge remote-tracking branch 'upstream/main' into awq-fused-modules
8f381edc
younesbelkada Merge branch 'awq-fused-modules' of https://github.com/younesbelkada/…
e80ad756
younesbelkada Update src/transformers/utils/quantization_config.py
b5c337cc
younesbelkada change it to `do_fuse`
3f98913d
younesbelkada nit
3bd0446a
younesbelkada Update src/transformers/utils/quantization_config.py
e1b3bfa4
younesbelkada Update src/transformers/utils/quantization_config.py
cb315465
younesbelkada Update src/transformers/utils/quantization_config.py
45875fd7
younesbelkada Merge branch 'awq-fused-modules' of https://github.com/younesbelkada/…
faaa255d
younesbelkada few fixes
c1ea9b2e
younesbelkada revert
d90eec75
younesbelkada fix test
e65687b7
younesbelkada fix copies
da78cf45
younesbelkada
younesbelkada younesbelkada requested a review from SunMarc SunMarc 2 years ago
SunMarc
SunMarc approved these changes on 2023-12-04
younesbelkada Merge remote-tracking branch 'upstream/main' into awq-fused-modules
2fcc465c
younesbelkada raise error if model is not quantized
06976877
younesbelkada add test
12aff7c3
younesbelkada use quantization_config.config when fusing
498fe55f
younesbelkada
younesbelkada commented on 2023-12-05
younesbelkada Update src/transformers/modeling_utils.py
196095ed
younesbelkada younesbelkada merged fdb85be4 into main 2 years ago
younesbelkada younesbelkada deleted the awq-fused-modules branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum