llama.cpp
`server`: fix tool-call of DeepSeek R1 Qwen, return reasoning_content (Command 7RB & DeepSeek R1) unless `--reasoning-format none`
#11607
Merged
Merged
`server`: fix tool-call of DeepSeek R1 Qwen, return reasoning_content (Command 7RB & DeepSeek R1) unless `--reasoning-format none` #11607
ochafik
merged
94 commits into
ggml-org:master
from
ochafik:r1-toolcall
minja: enhance backfill of templates w/o tools description (use examp…
d3b60b8a
pass vocab to common_chat_params_init
87de852b
DeepSeek R1: parse thoughts / return in separate field in API (non st…
130ca222
Avoid double bos w/ jinja
04d511b5
server/oai: ensure content is null when there are tool calls
28345877
update logs
c80cb309
rename tests
08716281
tool-call: allow `--jinja --chat-template chatml`
73d08d49
tool-call: fix command-r7b parsing when response is multiline
04be723b
tool-calls: add DeepSeek R1 Qwen 7B to server test_hello_world
ae9d5812
tell DS R1 not to overthink (weather test)
19bea4ec
add deepseek models to server tool call section in readme
5e6f2a21
tool-call: allow `--jinja --chat-template chatml`
1e9acd2d
Update test_tool_call.py
77ae97e7
minimize diffs
a76073cf
fix typo
cf83623a
fix double bos issue (drop bos/eos tokens from jinja template)
5d18d76b
fix bad merge
aa98e590
fix build / rm diff
2b3c4829
Merge branch 'jinja-chatml' into r1-toolcall
4cb0e1d8
add missing try catch around jinja parsing to default to chatml
b2dd4909
Merge branch 'jinja-chatml' into r1-toolcall
08271b55
tool-calls: r1: add missing <｜tool▁calls▁end｜> to grammar!
df3474e2
tweak delta logic
c397bd1f
tool-calls: accommodate variety of wrong tool call opening tags both …
569610ee
Simplify default chatml logic
d73448de
Merge branch 'jinja-chatml' into r1-toolcall
0be7f652
tool-calls: add deepseek r1 template + accommodate broken official te…
7dc271fb
rm unneeded vocab
c6214ee9
simpler hacky fixes for original broken template (+ fix minja example…
1c302e18
sync: minja https://github.com/google/minja/pull/46
108da907
Merge branch 'master' into r1-toolcall
bc6d910f
actually we want eos_token in the template to infer tool call example…
11c1f0c7
update to minja's new api
30ea3591
sync: minja
bbd45bf6
simplify hack to fix original template's backfill from minja
bff549de
tool-call: r1: add one more trigger approx "<｜tool calls begin｜>"
ce28224d
r1: fix inadvertent newline in grammar before <｜tool▁call▁end｜>
e84ee88f
tool-call: r1: fix grammar
18a11f43
move trigger_words init inside non-llguidance branch
9a6847c8
fix / test parsing of r1 parser
a682d121
Fix / test models/templates/llama-cpp-deepseek-r1.jinja
f0154a64
update test_calc_result
326e7002
fix test_calc_result
78b47bb0
fix spaces
86994db6
`sync`: minja
09caa634
Update test-chat.cpp
b1527292
fix mistral chat test: need empty tokens
56a14ddc
Update chat.cpp
f12e3507
Merge branch 'sync-minja-4' into r1-toolcall
d43e4f6c
server: check that content is null when we get tool_calls
812544ab
tool-call: ensure we don't return content when there are tool calls /…
d44eb95c
fix mistral expectation
b6e14a41
ensure deepseek r1 thoughts parsed even w/o tool calls
1f5ec598
fix test-chat
438ce0b8
ochafik
changed the title
ochafik
changed the title
Update chat.cpp
21f20715
Merge branch 'sync-minja-4' into r1-toolcall
b5b117fa
Fix r1 grammar since we made <｜tool▁calls▁begin｜> optional (triggerin…
0db98812
r1: revert making <｜tool▁calls▁begin｜> optional as somehow sampling t…
d1b66910
ochafik
changed the title
ochafik
changed the title 
return thoughts in reasoning_content field
39c1d816
ochafik
changed the title
update readme section about common model tool call formats
b2d17287
Merge branch 'master' into r1-toolcall
933f7a18
Update test_tool_call.py
5d60cebb
Merge branch 'master' into r1-toolcall
1f1f06aa
--think to force any model to return reasoning_content (or just parse…
9d7c3cc5
Merge branch 'r1-toolcall' of github.com:ochafik/llama.cpp into r1-to…
d20c2ce4
ochafik
changed the title
fix test_thoughts
f3e9f8b6
ochafik
changed the title
fix compiler warning about parens
3841a163
align Command R7B w/ --think / reasoning_content behaviour
e6d9b524
Update README.md
39b50c37
fix --think arg env
0917e0a8
disable some failing chatml tests
098629df
ochafik
changed the title
Update README.md
33efcb3c
use existing string_strip
994301da
revert tool example backfill change - command 7rb just needs the righ…
d1a06407
ochafik
changed the title
ngxson
commented
on 2025-02-06
Merge remote-tracking branch 'origin/master' into r1-toolcall
cc2c712c
Use --reasoning-format, remove forced thinking for now
c0f972bb
return reasoning_content before content
af638860
update model template / format mapping
a59fde29
ochafik
changed the title
fix test-chat
b829cab7
rm thoughts from generic parser
95cddfd8
sync: minja (https://github.com/google/minja/pull/52)
e598e7aa
tool-calls: allow r1 output to miss <think> opening tag (since latest…
91542ca2
sync: minja (https://github.com/ggerganov/llama.cpp/pull/11774)
8d82be90
rm wrong warning in command-r parser (when normal text)
30dcfaa5
update deepseek r1 templates (+ put update commands in ./scripts/get_…
e1bff8f6
fix server test_tool_calls.py
a29dc921
add models/templates/README.md
ea2f41e0
fix test_calc_result & test_thoughts
8409bf18
fix test-chat (update delta to latest r1 template change)
01db4291
Merge remote-tracking branch 'origin/master' into r1-toolcall
37a4bb25
prefer json::at to operator[] in chat.cpp
d52579a9
Merge remote-tracking branch 'origin/master' into r1-toolcall
47002452
Apply suggestions from code review
043cb99f
Login to write a write a comment.
Login via GitHub