transformers
Refactor GGUF to speed-up inference
#47779
Merged
Overview
Commits
47
Changes
View On GitHub

Refactor GGUF to speed-up inference #47779

SunMarc merged 47 commits into main from poc-gguf
SunMarc
SunMarc gguf non quantized
783f1ef3
SunMarc add support for q_k
a7fbe8f5
SunMarc Merge branch 'main' into poc-gguf
9e57369f
LysandreJik
SunMarc kernels and config
e34177d7
SunMarc Merge remote-tracking branch 'origin/poc-gguf' into poc-gguf
ad4e7c2c
HuggingFaceDocBuilderDev
SunMarc better integration with kernels
5f48600a
SunMarc small update
0c556c4e
SunMarc Merge remote-tracking branch 'origin/poc-gguf' into poc-gguf
de3bf700
SunMarc fix
1a101a1d
SunMarc fix
1a3d470d
SunMarc SunMarc requested a review from ArthurZucker ArthurZucker 48 days ago
SunMarc support for ggml-attn
fa45d579
SunMarc update
a822ee02
SunMarc Merge remote-tracking branch 'upstream/main' into poc-gguf
20e97611
SunMarc fix
e946e7c2
SunMarc update
8413aa69
SunMarc fix
2abc18bc
SunMarc kernels gated delta
b963ea10
SunMarc Revert "kernels gated delta"
bf7d97c7
SunMarc kernels gated delta agin
6a7de5d3
BowenBao
ArthurZucker
ArthurZucker approved these changes on 2026-08-25
SunMarc
SunMarc input permutation
39104425
SunMarc Merge remote-tracking branch 'upstream/main' into poc-gguf
5d95dcfc
SunMarc style
4e3ebbfc
SunMarc update integration
31cec70a
SunMarc up
ef373059
SunMarc docs and bunch of fixes
3cd2049e
SunMarc tokenizer for qwen3.5
8320a1ac
SunMarc fix ci and doc
965029bb
SunMarc cleaner integration
fac572b1
SunMarc up
74dd50fb
SunMarc fix
4d2c2e08
SunMarc update
e6c7b422
SunMarc fix
6a5d9a1a
SunMarc up
b7267db9
SunMarc serve fix
5eb6df28
SunMarc doc
b155b5ea
SunMarc leftover stuff
73179955
SunMarc Merge remote-tracking branch 'origin/poc-gguf' into poc-gguf
0bb36abb
SunMarc Merge remote-tracking branch 'upstream/poc-gguf' into poc-gguf
8bd1b773
SunMarc Merge remote-tracking branch 'origin/poc-gguf' into poc-gguf
3e1ef578
SunMarc minor fix
70033acb
SunMarc Merge remote-tracking branch 'upstream/poc-gguf' into poc-gguf
f2f9370c
SunMarc norm ! 11% speed increase
fa74f127
SunMarc up
5bf39863
ArthurZucker
ArthurZucker approved these changes on 2026-09-01
SunMarc Merge remote-tracking branch 'upstream/main' into poc-gguf
728c5080
SunMarc correct path !
e234ff4c
SunMarc update
abdb5345
github-actions
SunMarc less verbose
f3cf4e72
SunMarc SunMarc enabled auto-merge 20 days ago
github-actions
SunMarc SunMarc merged f62dc9bf into main 20 days ago
SunMarc SunMarc deleted the poc-gguf branch 20 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum