llama.cpp
Tool call support (generic + native for Llama, Functionary, Hermes, Mistral, Firefunction, DeepSeek) w/ lazy grammars
#9639
Merged
Overview
Commits
375
Changes
View On GitHub
Commits
  • nits
    ochafik committed 1 year ago
  • `tool-call`: slow tool call integration tests
    ochafik committed 1 year ago
  • space nits
    ochafik committed 1 year ago
  • `tool_call`: test no tool call on a real model + rename scenarios
    ochafik committed 1 year ago
  • `tool-call`: script to prefetch models used in server tests
    ochafik committed 1 year ago
  • Update tool_call.feature
    ochafik committed 1 year ago
  • `tool-call`: add tests: tool_call=none, parallel_tool_calls=true
    ochafik committed 1 year ago
  • `tool-call`: remove duplicate script to fetch templates
    ochafik committed 1 year ago
  • `agent`: simplify syntax (default tools to local w/ default port)
    ochafik committed 1 year ago
  • `tool-call`: use Q4_K_M models
    ochafik committed 1 year ago
  • `tool-call`: update scripts/fetch_server_test_models.py
    ochafik committed 1 year ago
  • `tool-call`: test Hermes-3-Llama-3.1-8B
    Olivier Chafik committed 1 year ago
  • `tool-call`: use functionary-small-v3.2-Q8_0.gguf in test (Q4_K_M too dumb for function call)
    Olivier Chafik committed 1 year ago
  • `tool-call`: force printing of lazy grammar trigger tokens to regularize function call parsing
    Olivier Chafik committed 1 year ago
  • nits
    Olivier Chafik committed 1 year ago
  • `tool-call`: support tool_use variant in llama_chat_template_from_model + drop llama_get_chat_template
    ochafik committed 1 year ago
  • `tool-call`: fix missing initializer errors
    ochafik committed 1 year ago
  • `tool-call`: when slow server tests fail, hint to run `python scripts/fetch_server_test_models.py`
    ochafik committed 1 year ago
  • `tool-calls`: test Qwen2.5-7B-Instruct-Q4_K_M.gguf
    Olivier Chafik committed 1 year ago
  • Merge remote-tracking branch 'origin/master' into tool-call
    ochafik committed 1 year ago
  • Update llama-sampling.cpp
    ochafik committed 1 year ago
  • `tool-call`: greedy sampling in server tests + tweak prompt
    ochafik committed 1 year ago
  • `tool-call`: nemo tweak (accept raw sql again)
    ochafik committed 1 year ago
  • Update tool_call.feature
    ochafik committed 1 year ago
  • `tool-call`: behaviour-based detection of template features
    ochafik committed 1 year ago
  • `tool-call`: code_interpreter & system + tool call support for all jinja templates!
    ochafik committed 1 year ago
  • `tool-call`: don't use -fa w/ Mistral-Nemo (hard crashes?)
    ochafik committed 1 year ago
  • `tool-call`: add LLAMA_UPDATE_GOLDENS env for test-chat-template
    ochafik committed 1 year ago
  • `tool-call`: functionary-small-v3.2 test now green
    ochafik committed 1 year ago
  • Update README.md
    ochafik committed 1 year ago
  • nits
    ochafik committed 1 year ago
  • Update README.md
    ochafik committed 1 year ago
  • `tool-call`: fix qwen template test
    ochafik committed 1 year ago
  • `agent`: add missing tool name in response!
    ochafik committed 1 year ago
  • `agent`: memorize, search_memory (sqlite-vec + sqlite-lembed), fetch + docling (pdf -> markdown), sparql for dbpedia and wikidata
    ochafik committed 1 year ago
  • `minja`: don't explode upon referencing a field on an array (fixes Hermes tool use template)
    ochafik committed 1 year ago
  • Update README.md
    ochafik committed 1 year ago
  • agent: add --think "tool", default to local tools endpoint, support --temperature, fix --seed
    ochafik committed 1 year ago
  • Merge remote-tracking branch 'origin/master' into tool-call
    ochafik committed 1 year ago
  • agent: more robust squid config
    ochafik committed 1 year ago
  • agent: update readme
    ochafik committed 1 year ago
  • minja: remove tests (now in https://github.com/google/minja)
    ochafik committed 1 year ago
  • Update README.md
    ochafik committed 1 year ago
  • minja: sync @ https://github.com/google/minja/commit/916c181c0d4a6f96b153dc41d6dacd15d35fd3af
    ochafik committed 1 year ago
  • tool-call: add firefunction-v2 style
    ochafik committed 1 year ago
  • tool-calls: migrate tests to pytest
    ochafik committed 1 year ago
  • Merge remote-tracking branch 'origin/master' into tool-call
    ochafik committed 1 year ago
  • tool-calls: shorter name: grammar_triggers
    ochafik committed 1 year ago
  • Merge remote-tracking branch 'origin/master' into tool-call
    ochafik committed 1 year ago
  • tool-call: stabilize server tests
    ochafik committed 1 year ago
  • Merge remote-tracking branch 'origin/master' into tool-call
    ochafik committed 1 year ago
  • Update test-tool-call.cpp
    ochafik committed 1 year ago
  • Update minja.hpp https://github.com/google/minja/commit/202aa2f3de21b43edbe6cb016834f7743afa1bd0
    ochafik committed 1 year ago
  • rm trailing spaces
    ochafik committed 1 year ago
  • Update fetch_server_test_models.py
    ochafik committed 1 year ago
  • Fix tool-call server tests
    ochafik committed 1 year ago
  • Simplify tool call grammars when there's only 1 tool
    ochafik committed 1 year ago
  • Copy minja from https://github.com/google/minja/commit/58f0ca6dd74bcbfbd4e71229736640322b31c7f9
    ochafik committed 1 year ago
  • Add --jinja and --chat-template-file flags
    ochafik committed 1 year ago
  • Add missing <optional> include
    ochafik committed 1 year ago
  • Avoid print in get_hf_chat_template.py
    ochafik committed 1 year ago
  • No designated initializers yet
    ochafik committed 1 year ago
  • Try and work around msvc++ non-macro max resolution quirk
    ochafik committed 1 year ago
  • Update test_chat_completion.py
    ochafik committed 1 year ago
  • Merge remote-tracking branch 'origin/master' into jinja
    ochafik committed 1 year ago
  • Wire LLM_KV_TOKENIZER_CHAT_TEMPLATE_N in llama_model_chat_template
    ochafik committed 1 year ago
  • Refactor test-chat-template
    ochafik committed 1 year ago
  • Test templates w/ minja
    ochafik committed 1 year ago
  • Fix deprecation
    ochafik committed 1 year ago
  • Add --jinja to llama-run
    ochafik committed 1 year ago
  • Merge remote-tracking branch 'origin/master' into jinja
    ochafik committed 1 year ago
  • Update common_chat_format_example to use minja template wrapper
    ochafik committed 1 year ago
  • Test chat_template in e2e test
    ochafik committed 1 year ago
  • Update utils.py
    ochafik committed 1 year ago
  • Update test_chat_completion.py
    ochafik committed 1 year ago
  • Update run.cpp
    ochafik committed 1 year ago
  • Update arg.cpp
    ochafik committed 1 year ago
  • Merge branch 'jinja' into tool-call
    ochafik committed 1 year ago
  • Fix merge
    ochafik committed 1 year ago
  • Update test-chat-template.cpp
    Olivier Chafik committed 1 year ago
  • Merge remote-tracking branch 'origin/master' into tool-call
    Olivier Chafik committed 1 year ago
  • Update test-chat-template.cpp
    Olivier Chafik committed 1 year ago
  • Merge remote-tracking branch 'origin/master' into jinja
    Olivier Chafik committed 1 year ago
  • Refactor common_chat_* functions to accept minja template + use_jinja option
    ochafik committed 1 year ago
  • Refactor common_chat_* functions to accept minja template + use_jinja option
    ochafik committed 1 year ago
  • Merge remote-tracking branch 'origin/master' into jinja
    ochafik committed 1 year ago
  • Attempt to fix linkage of LLAMA_CHATML_TEMPLATE
    ochafik committed 1 year ago
  • Merge branch 'jinja' into tool-call
    ochafik committed 1 year ago
  • Revert LLAMA_CHATML_TEMPLATE refactor
    ochafik committed 1 year ago
  • Merge branch 'jinja' into tool-call
    ochafik committed 1 year ago
  • Fix fetch_server_test_models.py (avoid conv trap)
    ochafik committed 1 year ago
  • tools: greedy sampling in tests
    ochafik committed 1 year ago
  • tools: run tool call slow tests when SLOW_TESTS=1 (+ prefetch models)
    ochafik committed 1 year ago
  • Normalize newlines in test-chat-templates for windows tests
    ochafik committed 1 year ago
  • Forward decl minja::chat_template to avoid eager json dep
    ochafik committed 1 year ago
  • Flush stdout in chat template before potential crash
    ochafik committed 1 year ago
  • Fix copy elision warning
    ochafik committed 1 year ago
  • Merge branch 'jinja' into tool-call
    ochafik committed 1 year ago
  • Rm unused optional include
    ochafik committed 1 year ago
  • Add missing optional include to server.cpp
    ochafik committed 1 year ago
  • + more commits ...
FAQ Terms Privacy Refunds Impressum
Loading