ggml-quants : ternary packing for TriLMs and BitNet b1.58 #8151
compilade
merged
33 commits into
master
from
compilade/bitnet-ternary
ggml-quants : 1.625 bpw ternary packing for BitNet 1.58b
bd807499
ggml-quants : faster 1.625 bpw AVX2 vec_dot
7ef4254a
ggml-quants : substract 1 when back in epi8
48b73b84
ggml-quants : Q2_2 now faster than Q4_K on with AVX2
ef1e345c
ggml-quants : cleanup Q1_3 code formatting
638ad52f
ggml-quants : ARM NEON vec_dot for q2_2 and q1_3
9465ec6e
ggml-quants : use ceiling division when quantizing q1_3
89dc3b25
convert-hf : simplify BitNet pre-quantization
961e2938
convert-hf : allow converting the weird BitNet 1.3B
09961499
compilade
changed the title
bitnet : replace 1.58b with b1.58, as in the paper
bfd2f21f
ggml-quants : fix build failure on Windows
ec50944b
compilade
commented
on 2024-06-29
ggml-quants : attempt to fix Arm 32-bit support
8fbd5930
ggerganov
commented
on 2024-07-07
ggerganov
commented
on 2024-07-07
ggml : add some informative comments in q1_3 vec_dot
dd3e62a7
Merge branch 'master' into compilade/bitnet-ternary
79a278e9
ggml : add TQ1_0 and TQ2_0 ternary quantization types
77b8f84a
ggml : even faster TQ2_0
560873f3
ggml : also faster TQ1_0
e9719576
ggml : fix build issues in certain environments
a6dd6994
ggml : add NEON vec_dot implementation for TQ1_0 and TQ2_0
5417089a
ggml : avoid directly using vmlal_high_s8, for 32-bit ARM compat
45719a24
ggml : remove q1_3 and q2_2
04eec581
compilade
changed the title
ggml-quants : rename fields of TQ1_0 and TQ2_0 structs for consistency
f034aa1b
ggml-quants : allow using vdotq_s32 in TQ2_0 vec_dot
96b3d411
Merge branch 'master' into compilade/bitnet-ternary
d911cd1f
gguf-py : Numpy (de)quantization for TQ1_0 and TQ2_0
3a0bf17d
convert : allow direct conversion to TQ1_0 and TQ2_0
895004f3
ggml-quants : allow using ARM dot product instructions for TQ1_0
69f77268
Merge branch 'master' into compilade/bitnet-ternary
82b24040
ggml-quants : deduplicate TQ1_0 and TQ2_0 __ARM_FEATURE_DOTPROD support
35cc5567
ggerganov
approved these changes
on 2024-08-15
Merge branch 'master' into compilade/bitnet-ternary
cb6d9962
Merge branch 'master' into compilade/bitnet-ternary
7f3a619c
ggml ; remove unused ggml_mul special case
8d616076
test-backend-ops : add TQ1_0 and TQ2_0 comments for later
75b3a096
Login to write a write a comment.
Login via GitHub