text-generation-inference
Add FP8 KVCache support
#2028
Closed
Overview
Commits
22
Changes
View On GitHub

Add FP8 KVCache support #2028

mht-sharma wants to merge 22 commits into main from fp8_kvcache
mht-sharma
add kvcache fp8 support
8c437a80
Narsil
Narsil commented on 2024-06-06
Narsil
Narsil commented on 2024-06-06
Narsil
fxmarty
fxmarty commented on 2024-06-14
mht-sharma
mht-sharma Merge branch 'main' into fp8_kvcache
084de990
mht-sharma rebase and update
81fd601c
mht-sharma fix
fb83e341
mht-sharma fixrs
f0d95b0f
mht-sharma add docs
8a0bb53e
mht-sharma
mht-sharma mht-sharma changed the title [WIP] Add kvcache fp8 support Add kvcache fp8 support 1 year ago
mht-sharma mht-sharma marked this pull request as ready for review 1 year ago
mht-sharma fix style
557e18e0
mht-sharma mht-sharma changed the title Add kvcache fp8 support Add FP8 KVCache support 1 year ago
mht-sharma update port
50806ffe
mht-sharma rename doc
001ec09d
HuggingFaceDocBuilderDev
mht-sharma update doc
3cc2f4e9
mht-sharma update launcher
e81c4cf8
mht-sharma
mht-sharma update heading
034686b1
mht-sharma revert makefile
3ae62304
mht-sharma mht-sharma requested a review from fxmarty fxmarty 1 year ago
mht-sharma mht-sharma requested a review from Narsil Narsil 1 year ago
fxmarty
fxmarty commented on 2024-06-24
mht-sharma remove example
f4714a8f
mht-sharma add torch dtype
a7909e6f
mht-sharma add AMMO example
1e6e7db0
mht-sharma updated doc
15b351b4
mht-sharma update launcher
5e38d353
mht-sharma updated doc
bf4db771
mht-sharma updated docs
f34560f7
mht-sharma fix formatting
6d6b0bdc
mht-sharma updated doc
0a5b19a3
danieldk danieldk assigned danieldk danieldk 1 year ago
yao-matrix
yao-matrix commented on 2024-07-29
Narsil
Narsil Narsil closed this 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum