huggingface/text-generation-inference

Pull Requests Commits

OlivierDehaene committed 2 years ago

47ac334a

revert to old version

OlivierDehaene committed 2 years ago

c01d9b9d

OlivierDehaene committed 2 years ago

c3e2b79a

increased initial delay

OlivierDehaene committed 2 years ago

f067673a

OlivierDehaene committed 2 years ago

31159266

OlivierDehaene committed 2 years ago

a874a52a

OlivierDehaene committed 2 years ago

6aa17d79

OlivierDehaene committed 2 years ago

645e0a38

OlivierDehaene committed 2 years ago

80d03723

feat(ci): push to AML registry (#56)

OlivierDehaene committed 2 years ago

Verified e114d874

feat(docs): Clarify installation steps (#54)

lewtun committed 2 years ago

Verified a0dca443

feat(router): refactor API and add openAPI schemas (#53)

OlivierDehaene committed 2 years ago

Verified 20c3c594

breaking(router): modify /generate API to only return generated text (#50)

OlivierDehaene committed 2 years ago

Verified b1482d90

feat(router): use background task to manage request queue (#52)

OlivierDehaene committed 2 years ago

Verified 7b870e1e

fix(server): allow greedy repetition penalty (#51)

OlivierDehaene committed 2 years ago

Verified df227ac2

feat(server): allow the server to use a local weight cache (#49)

OlivierDehaene committed 2 years ago

Verified 775115e3

feat(server): support repetition penalty (#47)

OlivierDehaene committed 2 years ago

Verified 313194f6

feat(server): allow gpt-neox models with odd vocab sizes to be sharded (#48)

OlivierDehaene committed 2 years ago

Verified 2ad895a6

feat(ci): Docker build and push (#46)

OlivierDehaene committed 2 years ago

Verified 404ed7a1

feat(server): Support GPT-Neox (#39)

OlivierDehaene committed 2 years ago

Verified f830706b

fix(server): fix quantization for sharded models (#45)

OlivierDehaene committed 2 years ago

Verified c6e8b944

feat: Add token streaming using ServerSideEvents support (#41)

OlivierDehaene committed 2 years ago

Verified 017a2a8c

fix(server): fix seeding with multiple shards (#44)

OlivierDehaene committed 2 years ago

Verified 54fec931

fix(server): fix seeding on gpu (#42)

OlivierDehaene committed 2 years ago

Verified 03bdf182

Revert "feat: Add token streaming using ServerSideEvents support" (#40)

OlivierDehaene committed 2 years ago

Verified 4f9ac67c

feat: Add token streaming using ServerSideEvents support (#36)

OlivierDehaene committed 2 years ago

Verified 7fbfbb0d

feat: Support sampling seeding (#37)

OlivierDehaene committed 2 years ago

Verified cd298bc5

feat(router): Remove second lock from batcher hot path (#27)

OlivierDehaene committed 2 years ago

Verified 1539d3cb

feat(bloom): use torch.nn.Linear and torch.nn.GELU (#33)

OlivierDehaene committed 2 years ago

Verified ce960be0

fix(dockerfile): fix docker build (#32)

OlivierDehaene committed 2 years ago

Verified 13e7044a

Older