PR #951 Add Q8_0 quantization for intermediate results

Add Q8_0 quantization for intermediate results #951

ggerganov merged 7 commits into master from q8_0

ggerganov added help wanted

ggerganov added high priority

ggerganov added generation quality

ggerganov commented on 2023-04-13

ggerganov requested a review from

sw 2 years ago

sw commented on 2023-04-13

ggerganov assigned

ggerganov 2 years ago

ggerganov force pushed to 05bf3ab6 2 years ago

ggml : add Q8_0 quantization for intermediate results

3b894ec6

quantize-stats : fix test + add it to Makefile default

19e7a657

Q8: use int8_t, AVX/AVX2 optimizations

2c4f9b65

ggml : fix quantize_row_q8_0() ARM_NEON rounding

312a927f

minor : updates after rebase to latest master

3a111abd

quantize-stats : delete obsolete strings

01de5c54

ggerganov force pushed from 9056a24e to 01de5c54 2 years ago

sw commented on 2023-04-15

ggml : fix q4_1 dot func

60f27ed8

sw approved these changes on 2023-04-15

ggerganov merged e95b6554 into master 2 years ago

ggerganov deleted the q8_0 branch 2 years ago

mofosyne added Tensor Encoding Scheme

mofosyne added Review Complexity : High

Reviewers

Assignees

ggerganov

Labels

help wanted high priority generation quality Review Complexity : High Tensor Encoding Scheme

Milestone

No milestone