llama.cpp
Quant: OCP FP8 E4M3 support
#27512
Open
Overview
Commits
16
Changes
View On GitHub

Quant: OCP FP8 E4M3 support #27512

ORippler wants to merge 16 commits into master from osimons/ocp_fp8
ORippler
github-actions github-actions added testing
github-actions github-actions added ggml
github-actions github-actions added conversion
0cc4m
michaelw9999
ORippler ORippler force pushed from 7ab744c7 to d21c273f 43 days ago
ORippler
ORippler ORippler force pushed from d21c273f to cef6c528 39 days ago
ORippler ORippler force pushed from cef6c528 to 060faf68 14 days ago
ORippler ORippler force pushed from 060faf68 to 340a287d 13 days ago
ORippler ORippler force pushed from 340a287d to 60dcb35a 13 days ago
ORippler ORippler force pushed from 60dcb35a to 91fe47f3 13 days ago
ORippler ORippler force pushed from 91fe47f3 to d904f345 12 days ago
github-actions github-actions added model
ORippler ORippler force pushed from d904f345 to b44ab5c3 12 days ago
ORippler ORippler force pushed from 479c902f to 574eb984 12 days ago
ORippler
ORippler commented on 2026-10-07
ORippler ORippler force pushed from 574eb984 to 8efdff03 4 days ago
ORippler ORippler marked this pull request as ready for review 4 days ago
ORippler ORippler requested a review from ggerganov ggerganov 4 days ago
ORippler ORippler requested a review from CISC CISC 4 days ago
ORippler WIP OCP FP8 E4M3 support
35b73bab
ORippler Fix python type-checker
51eb4f0e
ORippler Materialize input_scales also for FP8 when converting
0164f0d3
ORippler Fix conversion of https://huggingface.co/RedHatAI/Qwen3.8-27B-NVFP4
f084700f
ORippler Allow remaining FP8 tensors to be dequantized during conversion
4e08deae
ORippler Allow tensor-strategy for FP8-only compressed-tensor ckpts
84787540
ORippler Fix and test gguf-py behavior for FP8_E4M3
93564813
ORippler Disallow QKV-fusion for FP8 temporarily
0924a639
ORippler Fix conversion of fused qkvz & FP8 for Qwen3-Next
e5bd135a
ORippler Avoid fusing FP8 for --fuse-gate-up-exps as we may have different ws
4a1abd76
ORippler Emit warnings when not fusing FP8 during conversion
db9deb2c
ORippler More FP8 conversion fixes
06531400
ORippler Add supported for strided copies to/from FP8 in CPU
6434bb67
ORippler remove unneeded scratch buffer from FP8 cpu cpy/dup path
b12939ab
ORippler add explanatory comments and TODOs
9eda44cd
ORippler Add dedicated encoder and decoder tests for FP8
41dbb3b5
ORippler ORippler force pushed from 8efdff03 to 41dbb3b5 14 hours ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
model testing ggml conversion
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum