text-generation-inference
Add AWQ quantization inference support (#1019)
#1054
Merged
Overview
Commits
12
Changes
View On GitHub

Add AWQ quantization inference support (#1019) #1054

Narsil merged 12 commits into main from awq_support
Narsil
abhinavkulkarni Add AWQ quantization inference support (#1019)
c35f39cf
HuggingFaceDocBuilderDev
Narsil Finishing nits + integration test
8ee93076
Narsil
Narsil Add git to docker.
e08f3ac5
casper-hansen
Narsil Declare torch as build dep.
757cf178
Narsil Using kernel like Makefile instead.
27ecef51
Narsil Change deploy.
a8f870aa
Narsil
Narsil Make awq install optional + integration tests values.
02d4f62a
casper-hansen
Narsil
Narsil Update dockerfile with new build.
4a290742
Narsil Adding target list.
2d8c034d
Narsil Support TheBloke exported models.
cbf047b4
Narsil Fix and test sharded version.
97292ec2
Narsil Fix dockerfile.
e27438aa
Narsil Narsil merged c5de7cd8 into main 2 years ago
Narsil Narsil deleted the awq_support branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum