llama.cpp
669912d9 - `tool-call`: fix Qwen 2.5 Coder support, add micro benchmarks, support trigger patterns for lazy grammars (#12034)

Commit
130 days ago
`tool-call`: fix Qwen 2.5 Coder support, add micro benchmarks, support trigger patterns for lazy grammars (#12034) * sampler: turn lazy grammar trigger words to regexes * add scripts/tool_bench.sh & .py * constrain llama json output regardless of function name if matches at beginning * update relaxed newline space rule in grammar tests * support add_generation_prompt query parameter (useful for /apply_template) * Update src/llama-grammar.cpp Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Author
Parents
  • File
    README.md
  • common
    • File
      chat.cpp
    • File
      common.cpp
    • File
      common.h
    • File
      json-schema-to-grammar.cpp
    • File
      json-schema-to-grammar.h
    • File
      sampling.cpp
  • examples
    • File
      json_schema_to_grammar.py
    • server
      • public_legacy
        • File
          json-schema-to-grammar.mjs
      • File
        server.cpp
      • tests
        • unit
          • File
            test_tool_call.py
        • File
          utils.py
      • File
        utils.hpp
  • include
    • File
      llama.h
  • models/templates
    • File
      README.md
  • File
    requirements.txt
  • requirements
    • File
      requirements-all.txt
    • File
      requirements-tool_bench.txt
  • scripts
    • File
      fetch_server_test_models.py
    • File
      tool_bench.py
    • File
      tool_bench.sh
  • src
    • File
      llama-grammar.cpp
    • File
      llama-grammar.h
    • File
      llama-sampling.cpp
  • tests
    • File
      test-chat.cpp
    • File
      test-json-schema-to-grammar.cpp