PR #2991 speculative : add grammar support

speculative : add grammar support

ggerganov committed 3 years ago

grammars : add json_arr.gbnf

ggerganov committed 3 years ago

grammar : add comments to new grammar file

ggerganov committed 3 years ago

grammar : remove one nested level

ggerganov committed 3 years ago

common : warm-up with 2 tokens - seems to work better

ggerganov committed 3 years ago

speculative : print draft token pieces

ggerganov committed 3 years ago

speculative : reuse grammar parser + better logs and comments

ggerganov committed 3 years ago

speculative : avoid grammar_mem

ggerganov committed 3 years ago

make : fix speculative build

ggerganov committed 3 years ago

llama.cpp speculative : add grammar support #2991 Merged