PR #1539 Outlines guided generation

Outlines guided generation #1539

Narsil merged 34 commits into main from outlines-guided-generation

drbh force pushed from 105f103b to a0c8b9a1 2 years ago

drbh force pushed from a0c8b9a1 to d4de4021 2 years ago

drbh force pushed from 370e47f8 to cadb0a98 2 years ago

Narsil commented on 2024-02-12

drbh force pushed from cadb0a98 to fc689e04 2 years ago

drbh marked this pull request as ready for review 2 years ago

OlivierDehaene commented on 2024-02-13

OlivierDehaene commented on 2024-02-14

drbh force pushed from 0c9b22fe to 63b09177 2 years ago

feat: first draft constraining generation via outlines

b013cb4f

feat: support simple grammars via outlines

02455067

feat: support JSON schemas

066d3d48

fix: update all NextTokenChoosers

56e919e4

feat: support other models and add fsm caching

8fd2664a

feat: support grammars in batch

ff6e8d9e

fix: update grammar states after filter

a1c630d5

fix: prefer grammar as logit processor

f1d43f2d

fix: remove small unneeded changes

ffc22883

fix: adjust protos and router

8413cc45

fix: adjust docs

518ebfb1

fix: re update docs

565f246f

fix: resolve typos and docs

74a586b4

feat: add grammar tests and typo tweaks

f94fc831

feat: address syntactically comments

a28ba721

feat: remove states from proto and simplify logit processor

13e07b82

feat: improve grammar init

d0d7cd9e

feat: improve grammar advance logic to avoid blocking GPU

8f140190

feat: advance grammars in all models

91a114a4

feat: add --grammar-support cli flag and validation error

fe787d13

fix: update docs

8f1651f1

fix: add grammar_support to validation tests

95e577b9

feat: add grammar_support to test launcher

5ba1bacc

fix: adjust client and conftest for grammar

d39e45ab

feat: fix advance_grammar sig, add comment and move advance call

3df37fa9

feat: improve client typing and update tests

8b9430fb

feat: include grammar type in request, avoid alloc and improve proto …

d849641b

fix: adjust type typo

8b13c47d

fix: adjust next type typo

3974e4dc

fix: add missing grammar type typo

be783547

fix: remove unnecessary code, avoid copies and make deser safer

f0cdd9c8

drbh force pushed from 63b09177 to f0cdd9c8 2 years ago

feat: prefer disable grammar

63c52fb2

feat: support spectulative decoding grammar advances

c791187b

fix: add disable_grammar_support to docker_launcher args

12c7aae3

Narsil approved these changes on 2024-02-15

Narsil merged cef0553d into main 2 years ago

Narsil deleted the outlines-guided-generation branch 2 years ago

Reviewers

Narsil

OlivierDehaene

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

text-generation-inference Outlines guided generation #1539 Merged

Outlines guided generation #1539

text-generation-inference
Outlines guided generation
#1539

Merged