-
Notifications
You must be signed in to change notification settings - Fork 9.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tool call support (Llama 3.x, Functionary v3, Hermes 2 Pro, Mistral Nemo, generic) w/ lazy grammars & minimalist Jinja engine #9639
base: master
Are you sure you want to change the base?
Commits on Sep 25, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5b6d504 - Browse repository at this point
Copy the full SHA 5b6d504View commit details -
Configuration menu - View commit details
-
Copy full SHA for eaca756 - Browse repository at this point
Copy the full SHA eaca756View commit details -
Configuration menu - View commit details
-
Copy full SHA for 26c175b - Browse repository at this point
Copy the full SHA 26c175bView commit details -
tool-call
: basic Functionary 3.2, Llama 3.1, Hermes 2 Pro grammar g……enerators + parsers
Configuration menu - View commit details
-
Copy full SHA for 3cfc21e - Browse repository at this point
Copy the full SHA 3cfc21eView commit details -
Configuration menu - View commit details
-
Copy full SHA for e309c6a - Browse repository at this point
Copy the full SHA e309c6aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 41103c0 - Browse repository at this point
Copy the full SHA 41103c0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4706bdb - Browse repository at this point
Copy the full SHA 4706bdbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8f25531 - Browse repository at this point
Copy the full SHA 8f25531View commit details -
Configuration menu - View commit details
-
Copy full SHA for 33ea20e - Browse repository at this point
Copy the full SHA 33ea20eView commit details -
Configuration menu - View commit details
-
Copy full SHA for d15dcfb - Browse repository at this point
Copy the full SHA d15dcfbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 97d0620 - Browse repository at this point
Copy the full SHA 97d0620View commit details -
Configuration menu - View commit details
-
Copy full SHA for e983c9d - Browse repository at this point
Copy the full SHA e983c9dView commit details
Commits on Sep 26, 2024
-
minja
: fix llama_chat_apply_template + adde use_jinja param to vali……date_model_chat_template
Configuration menu - View commit details
-
Copy full SHA for 45b243b - Browse repository at this point
Copy the full SHA 45b243bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9e366b3 - Browse repository at this point
Copy the full SHA 9e366b3View commit details -
Configuration menu - View commit details
-
Copy full SHA for a774093 - Browse repository at this point
Copy the full SHA a774093View commit details -
server
: catch errors in oaicompat_completion_params_parse instead o……f taking server down
Configuration menu - View commit details
-
Copy full SHA for d928ff4 - Browse repository at this point
Copy the full SHA d928ff4View commit details -
Configuration menu - View commit details
-
Copy full SHA for ab25e3f - Browse repository at this point
Copy the full SHA ab25e3fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1b62801 - Browse repository at this point
Copy the full SHA 1b62801View commit details -
Configuration menu - View commit details
-
Copy full SHA for 76d2938 - Browse repository at this point
Copy the full SHA 76d2938View commit details -
Configuration menu - View commit details
-
Copy full SHA for c124ab4 - Browse repository at this point
Copy the full SHA c124ab4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 595e11c - Browse repository at this point
Copy the full SHA 595e11cView commit details -
server
: catch errors in format_final_response_oaicompat instead of ……taking server down
Configuration menu - View commit details
-
Copy full SHA for 94377d7 - Browse repository at this point
Copy the full SHA 94377d7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 059babd - Browse repository at this point
Copy the full SHA 059babdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4cd82d6 - Browse repository at this point
Copy the full SHA 4cd82d6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2eb29bf - Browse repository at this point
Copy the full SHA 2eb29bfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5f5be9c - Browse repository at this point
Copy the full SHA 5f5be9cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8e4a9ba - Browse repository at this point
Copy the full SHA 8e4a9baView commit details -
tool-call
: test/fix functionary-medium-v3.1's template (can "look" ……like llama3.1 template)
Configuration menu - View commit details
-
Copy full SHA for 0c87013 - Browse repository at this point
Copy the full SHA 0c87013View commit details -
Configuration menu - View commit details
-
Copy full SHA for 749a21c - Browse repository at this point
Copy the full SHA 749a21cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3d2650c - Browse repository at this point
Copy the full SHA 3d2650cView commit details -
Configuration menu - View commit details
-
Copy full SHA for d7ec84f - Browse repository at this point
Copy the full SHA d7ec84fView commit details -
Configuration menu - View commit details
-
Copy full SHA for cf7bece - Browse repository at this point
Copy the full SHA cf7beceView commit details -
tool-call
: refactor llama_chat_template class + use in validate_mod……el_chat_template
Configuration menu - View commit details
-
Copy full SHA for 9cfe4d7 - Browse repository at this point
Copy the full SHA 9cfe4d7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 296331b - Browse repository at this point
Copy the full SHA 296331bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 50685f8 - Browse repository at this point
Copy the full SHA 50685f8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5840e10 - Browse repository at this point
Copy the full SHA 5840e10View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2926089 - Browse repository at this point
Copy the full SHA 2926089View commit details -
Configuration menu - View commit details
-
Copy full SHA for c88c932 - Browse repository at this point
Copy the full SHA c88c932View commit details -
Configuration menu - View commit details
-
Copy full SHA for 10f9fe8 - Browse repository at this point
Copy the full SHA 10f9fe8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8299fac - Browse repository at this point
Copy the full SHA 8299facView commit details
Commits on Sep 27, 2024
-
Configuration menu - View commit details
-
Copy full SHA for f9c1743 - Browse repository at this point
Copy the full SHA f9c1743View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1e5c0e7 - Browse repository at this point
Copy the full SHA 1e5c0e7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9295ca9 - Browse repository at this point
Copy the full SHA 9295ca9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 27cd07a - Browse repository at this point
Copy the full SHA 27cd07aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6610ecf - Browse repository at this point
Copy the full SHA 6610ecfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0abfa36 - Browse repository at this point
Copy the full SHA 0abfa36View commit details -
tool-call
: fix crash / test non-tool call case (added llama_sampler……_is_grammar_empty)
Configuration menu - View commit details
-
Copy full SHA for f62e688 - Browse repository at this point
Copy the full SHA f62e688View commit details -
Configuration menu - View commit details
-
Copy full SHA for e33b342 - Browse repository at this point
Copy the full SHA e33b342View commit details -
tool-call
: fix functionary-small-3.2 (first tool starts w/ name\n, ……subsequent are >>>name\n)
Configuration menu - View commit details
-
Copy full SHA for e62b5de - Browse repository at this point
Copy the full SHA e62b5deView commit details -
Configuration menu - View commit details
-
Copy full SHA for 86e4f99 - Browse repository at this point
Copy the full SHA 86e4f99View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2f25ee3 - Browse repository at this point
Copy the full SHA 2f25ee3View commit details -
minja
: fix identifiers parsing (when start w/ not/is/etc) and lstri……p_blocks corner case (needed by DeepSeek-V2.5
Configuration menu - View commit details
-
Copy full SHA for 0093a5e - Browse repository at this point
Copy the full SHA 0093a5eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 701b664 - Browse repository at this point
Copy the full SHA 701b664View commit details -
minja
: generate chat goldens w/ fixed date to support Llama-3.2-3B-……Instruct (uses strftime_now)
Configuration menu - View commit details
-
Copy full SHA for 887951b - Browse repository at this point
Copy the full SHA 887951bView commit details
Commits on Sep 28, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 0c85bc7 - Browse repository at this point
Copy the full SHA 0c85bc7View commit details -
tool-call
: let the tool call handler expand chat template, moving b……uiltin_tools down as extra_context
Configuration menu - View commit details
-
Copy full SHA for d983516 - Browse repository at this point
Copy the full SHA d983516View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8b2cf35 - Browse repository at this point
Copy the full SHA 8b2cf35View commit details -
tool-call
: more eager function call parsing for Functionary & Llama…… (give a chance to 3B model)
Configuration menu - View commit details
-
Copy full SHA for 7cef90c - Browse repository at this point
Copy the full SHA 7cef90cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 55cf337 - Browse repository at this point
Copy the full SHA 55cf337View commit details -
Configuration menu - View commit details
-
Copy full SHA for c657857 - Browse repository at this point
Copy the full SHA c657857View commit details -
chat-template
: enumerate files w/ C API rather than private using s……td::__fs::filesystem
Configuration menu - View commit details
-
Copy full SHA for 6e0053a - Browse repository at this point
Copy the full SHA 6e0053aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 05bbba9 - Browse repository at this point
Copy the full SHA 05bbba9View commit details -
Configuration menu - View commit details
-
Copy full SHA for ef2a020 - Browse repository at this point
Copy the full SHA ef2a020View commit details -
Configuration menu - View commit details
-
Copy full SHA for e6be59c - Browse repository at this point
Copy the full SHA e6be59cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9358d1f - Browse repository at this point
Copy the full SHA 9358d1fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1b32ac1 - Browse repository at this point
Copy the full SHA 1b32ac1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0ae1112 - Browse repository at this point
Copy the full SHA 0ae1112View commit details -
Configuration menu - View commit details
-
Copy full SHA for dbda025 - Browse repository at this point
Copy the full SHA dbda025View commit details -
Configuration menu - View commit details
-
Copy full SHA for b10ef04 - Browse repository at this point
Copy the full SHA b10ef04View commit details -
Configuration menu - View commit details
-
Copy full SHA for bc3e0c0 - Browse repository at this point
Copy the full SHA bc3e0c0View commit details -
Configuration menu - View commit details
-
Copy full SHA for a072f30 - Browse repository at this point
Copy the full SHA a072f30View commit details -
Configuration menu - View commit details
-
Copy full SHA for ad6719e - Browse repository at this point
Copy the full SHA ad6719eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 22493c8 - Browse repository at this point
Copy the full SHA 22493c8View commit details -
Configuration menu - View commit details
-
Copy full SHA for c87c121 - Browse repository at this point
Copy the full SHA c87c121View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8738d94 - Browse repository at this point
Copy the full SHA 8738d94View commit details -
Configuration menu - View commit details
-
Copy full SHA for cb7912e - Browse repository at this point
Copy the full SHA cb7912eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9ac4b04 - Browse repository at this point
Copy the full SHA 9ac4b04View commit details
Commits on Sep 30, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 277f385 - Browse repository at this point
Copy the full SHA 277f385View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0fc5ad7 - Browse repository at this point
Copy the full SHA 0fc5ad7View commit details -
Configuration menu - View commit details
-
Copy full SHA for d9451fd - Browse repository at this point
Copy the full SHA d9451fdView commit details
Commits on Oct 1, 2024
-
tool-call
: prepare possible externalization of minja + factor tool ……call style out of template
Configuration menu - View commit details
-
Copy full SHA for c36a196 - Browse repository at this point
Copy the full SHA c36a196View commit details -
Configuration menu - View commit details
-
Copy full SHA for c76b145 - Browse repository at this point
Copy the full SHA c76b145View commit details
Commits on Oct 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5b01402 - Browse repository at this point
Copy the full SHA 5b01402View commit details -
Configuration menu - View commit details
-
Copy full SHA for f3538e7 - Browse repository at this point
Copy the full SHA f3538e7View commit details -
tool-call
: promote getting chat templates w/ dedicated script rathe……r than rely on test resources
Configuration menu - View commit details
-
Copy full SHA for 9e502e8 - Browse repository at this point
Copy the full SHA 9e502e8View commit details -
Configuration menu - View commit details
-
Copy full SHA for b559d64 - Browse repository at this point
Copy the full SHA b559d64View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2428b73 - Browse repository at this point
Copy the full SHA 2428b73View commit details -
Configuration menu - View commit details
-
Copy full SHA for e2a9ab6 - Browse repository at this point
Copy the full SHA e2a9ab6View commit details -
agent
: remove *lots* of cruft from tool definitions derived from Fa……stAPI catalog (and remove wait* tools which can be implemented in Python anyway)
Configuration menu - View commit details
-
Copy full SHA for 6f2191d - Browse repository at this point
Copy the full SHA 6f2191dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 26e76f9 - Browse repository at this point
Copy the full SHA 26e76f9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6b4a454 - Browse repository at this point
Copy the full SHA 6b4a454View commit details -
agent
: drop fastify.py -> simpler serve_tools.py, and expose other ……tools to python interpreter
Configuration menu - View commit details
-
Copy full SHA for fa8df0c - Browse repository at this point
Copy the full SHA fa8df0cView commit details
Commits on Oct 3, 2024
-
antiprompts
: ensure partial match is at end of string (or else serv……er stops sending replies)
Configuration menu - View commit details
-
Copy full SHA for ece12b0 - Browse repository at this point
Copy the full SHA ece12b0View commit details -
tool-call
: adjust triggers to most common tool call variations from…… Llama-3.1-8B and Llama-3.2-3B
Configuration menu - View commit details
-
Copy full SHA for b4fc1e8 - Browse repository at this point
Copy the full SHA b4fc1e8View commit details -
Configuration menu - View commit details
-
Copy full SHA for da02397 - Browse repository at this point
Copy the full SHA da02397View commit details -
Configuration menu - View commit details
-
Copy full SHA for 366efc8 - Browse repository at this point
Copy the full SHA 366efc8View commit details -
agent
: tool tweaks (remove ansi escapes from python output, update ……env keys + provider docs)
Configuration menu - View commit details
-
Copy full SHA for 21a3c90 - Browse repository at this point
Copy the full SHA 21a3c90View commit details
Commits on Oct 4, 2024
-
Configuration menu - View commit details
-
Copy full SHA for a151ddc - Browse repository at this point
Copy the full SHA a151ddcView commit details
Commits on Oct 7, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 241acc2 - Browse repository at this point
Copy the full SHA 241acc2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3325069 - Browse repository at this point
Copy the full SHA 3325069View commit details
Commits on Oct 8, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e753f15 - Browse repository at this point
Copy the full SHA e753f15View commit details
Commits on Oct 22, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7576487 - Browse repository at this point
Copy the full SHA 7576487View commit details -
Configuration menu - View commit details
-
Copy full SHA for fa8462f - Browse repository at this point
Copy the full SHA fa8462fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9f5ab97 - Browse repository at this point
Copy the full SHA 9f5ab97View commit details -
Configuration menu - View commit details
-
Copy full SHA for b53362a - Browse repository at this point
Copy the full SHA b53362aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7f2429e - Browse repository at this point
Copy the full SHA 7f2429eView commit details -
Configuration menu - View commit details
-
Copy full SHA for db4bf93 - Browse repository at this point
Copy the full SHA db4bf93View commit details -
Configuration menu - View commit details
-
Copy full SHA for 351aecb - Browse repository at this point
Copy the full SHA 351aecbView commit details -
Configuration menu - View commit details
-
Copy full SHA for a4f12a4 - Browse repository at this point
Copy the full SHA a4f12a4View commit details -
Configuration menu - View commit details
-
Copy full SHA for fc80ad2 - Browse repository at this point
Copy the full SHA fc80ad2View commit details
Commits on Oct 23, 2024
-
tool-calls
: basic Nemo support, default parallel to true if templat……e mentions tool_call_id
Configuration menu - View commit details
-
Copy full SHA for 3e12b9b - Browse repository at this point
Copy the full SHA 3e12b9bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2b49440 - Browse repository at this point
Copy the full SHA 2b49440View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5f4aef1 - Browse repository at this point
Copy the full SHA 5f4aef1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4394e1c - Browse repository at this point
Copy the full SHA 4394e1cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 414f6f1 - Browse repository at this point
Copy the full SHA 414f6f1View commit details
Commits on Oct 24, 2024
-
agent
: isolate tools container + log its outgoing HTTP & HTTPS traf……fic w/ docker compose + self-signed squid proxy
Configuration menu - View commit details
-
Copy full SHA for 267e630 - Browse repository at this point
Copy the full SHA 267e630View commit details -
Configuration menu - View commit details
-
Copy full SHA for f5320af - Browse repository at this point
Copy the full SHA f5320afView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0f5d639 - Browse repository at this point
Copy the full SHA 0f5d639View commit details -
agent
: ditch aiohttp & define REQUESTS_CA_BUNDLE to fix http proxyi……ng / trust the self-signed cert from python
Configuration menu - View commit details
-
Copy full SHA for d338bfb - Browse repository at this point
Copy the full SHA d338bfbView commit details -
Configuration menu - View commit details
-
Copy full SHA for c2926e4 - Browse repository at this point
Copy the full SHA c2926e4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 03b8641 - Browse repository at this point
Copy the full SHA 03b8641View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0f4fc8c - Browse repository at this point
Copy the full SHA 0f4fc8cView commit details
Commits on Oct 25, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5c414a3 - Browse repository at this point
Copy the full SHA 5c414a3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 30bd00b - Browse repository at this point
Copy the full SHA 30bd00bView commit details