text-generation-inference
feat(server): pre-allocate past key values for flash causal LM
#412

Merged

feat(server): pre-allocate past key values for flash causal LM #412

OlivierDehaene merged 8 commits into main from feat/faster_flash_cache

wip

5ff2dc91

working rw 7b

c9e74717

working

bfd6928c

fix

3fc87f93

add other models

c509e4e7

update commit

afdfe433

revert some changes

92a74ea0

OlivierDehaene force pushed from 0994ef7b to 92a74ea0 2 years ago

faster

4b9ebb0a

OlivierDehaene merged 5ce89059 into main 2 years ago

OlivierDehaene deleted the feat/faster_flash_cache branch 2 years ago

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Milestone

No milestone