Chat Template support for function calling and RAG #30621
Rocketknight1
merged
69 commits into
main
from
new_chat_template_args
molbap
commented
on 2024-05-28
molbap
commented
on 2024-05-28
amyeroberts
approved these changes
on 2024-06-06
CISC
commented
on 2024-06-07
CISC
commented
on 2024-06-07
First draft, still missing automatic function conversion
3ad5bd4a
First draft of the automatic schema generator
59e2cb64
Lots of small fixes
8f7655db
the walrus has betrayed me
0b2ead3f
please stop committing your debug breakpoints
cb67fd2d
Lots of cleanup and edge cases, looking better now
41df7d11
Comments and bugfixes for the type hint parser
eec24865
More cleanup
cf2b8dab
Add tests, update schema generator
ad0984b6
Update tests, proper handling of return values
c9fb3de8
Small docstring change
10be3b19
More doc updates
d9e64540
More doc updates
80addcf9
Add json_schema decorator
d3d677bb
Clean up the TODOs and finish the docs
a7d241c6
self.maxDiff = None to see the whole diff for the nested list test
fad9ae20
add import for add_json_schema
d202bfe0
Quick test fix
11cf8f57
Fix something that was bugging me in the chat template docstring
8962c421
Less "anyOf" when unnecessary
6f4a897e
Support return types for the templates that need them
6462de21
Proper return type tests
a49b68ea
Switch to Google format docstrings
c8a021e1
Update chat templating docs to match new format
69b6d31b
Stop putting the return type in with the other parameters
7f20d445
Add Tuple support
c3cf8723
No more decorator - we just do it implicitly!
098780da
Add enum support to get_json_schema
0ad7fdee
Update docstring
90a3c5bd
Add copyright header
ab2e741d
Update src/transformers/tokenization_utils_base.py
b2563c21
Update docs/source/en/chat_templating.md
24c05893
Update src/transformers/utils/chat_template_utils.py
49f1e977
Update src/transformers/utils/chat_template_utils.py
1036c5a4
Add copyright header
cdbc9bc0
make fixup
dbb157f2
Fix indentation
6171a9f0
Reformat chat_template_utils
098d6e6a
Correct return value
0a92408b
Make regexes module-level
a4dc182d
Support more complex, multi-line arg docstrings
390596e1
Update error message for ...
5568cb31
Update ruff
ae443e20
Add document type validation
85cba4e6
Refactor docs
23692c3c
Refactor docs
6ebad3ad
Refactor docs
19a9624b
Make regexes module-level
a4dc182d
Support more complex, multi-line arg docstrings
390596e1
Update error message for ...
5568cb31
Update ruff
ae443e20
Add document type validation
85cba4e6
Refactor docs
23692c3c
Refactor docs
6ebad3ad
Clean up Tuple error
28b5bc24
Add an extra test for very complex defs and docstrings and clean ever…
880bc997
Document enum block
58f69db7
Quick test fixes
df3123e3
Stop supporting type hints in docstring to fix bugs and simplify the …
cfb11907
Update docs for the regex change
fc66acc7
Clean up enum regex
bb6ba180
Wrap functions in {"type": "function", "function": ...}
eda660ff
Update src/transformers/utils/chat_template_utils.py
50c00e4b
Temporary tool calling commit
7b396ec9
Add type hints to chat template utils, partially update docs (incompl…
f64acfd3
Code cleanup based on @molbap's suggestion
4e839cb0
Add comments to explain regexes
8d7ca8f2
Fix up type parsing for unions and lists
3fbd6823
Add custom exception types and adjust tests to look for them
76c33200
Update docs with a demo!
170e3675
Docs cleanup
1d4952a1
Pass content as string
9f71f6c6
Update tool call formatting
6c29d465
Update docs with new function format
0ab8c7f3
Update docs
2b3e2227
Update docs with a second tool to show the model choosing correctly
2b19b0a3
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub