{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":766500302,"defaultBranch":"main","name":"gorilla","ownerLogin":"leixy76","currentUserCanPush":false,"isFork":true,"isEmpty":false,"createdAt":"2024-03-03T12:49:39.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/162015206?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1709470180.9806652","currentOid":""},"activityList":{"items":[{"before":"867b60d5ef5d688bf1eddb65d92c05a1dd415fd2","after":"7bef0009980152c0f3eb811a545d2e6209807b92","ref":"refs/heads/main","pushedAt":"2024-07-15T01:10:17.000Z","pushType":"push","commitsCount":10,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"[BFCL] Add Test Dataset to Repository (#504)\n\nPreviously, the test dataset was stored on HuggingFace, requiring users\r\nto clone this repository, download the dataset separately from\r\nHuggingFace, and then run the evaluation pipeline. This has caused many\r\ninconveniences and confusion within the community. Users often struggled\r\nwith the inconsistency of having the possible answers within the\r\nrepository while the test dataset was missing.\r\n\r\nPartially addresses #501.","shortMessageHtmlLink":"[BFCL] Add Test Dataset to Repository (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2393734635\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/504\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/504/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/504\">ShishirPatil#504</a>)"}},{"before":"e98fb6b72ac4df65f39e2671010061e7f26acc9a","after":"867b60d5ef5d688bf1eddb65d92c05a1dd415fd2","ref":"refs/heads/main","pushedAt":"2024-06-25T01:40:17.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"[BFCL] Add Claude 3.5 Sonnet Function Calling Infernece Inference  (#480)\n\nThis PR adds Claude 3.5 to leaderboard for both function calling and\r\nprompting. The PR will change the leaderboard.","shortMessageHtmlLink":"[BFCL] Add Claude 3.5 Sonnet Function Calling Infernece Inference (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2365828281\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/480\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/480/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/480\">Sh…</a>"}},{"before":"ba6463ddc397e4808982299964769496387913ce","after":"e98fb6b72ac4df65f39e2671010061e7f26acc9a","ref":"refs/heads/main","pushedAt":"2024-06-16T14:43:30.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"Bump pymysql from 1.1.0 to 1.1.1 in /goex (#453)\n\nBumps [pymysql](https://github.com/PyMySQL/PyMySQL) from 1.1.0 to 1.1.1.\r\n<details>\r\n<summary>Release notes</summary>\r\n<p><em>Sourced from <a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/releases\">pymysql's\r\nreleases</a>.</em></p>\r\n<blockquote>\r\n<h2>v1.1.1</h2>\r\n<blockquote>\r\n<p>[!WARNING]\r\nThis release fixes a vulnerability (CVE-2024-36039).\r\nAll users are recommended to update to this version.</p>\r\n<p>If you can not update soon, check the input value from untrusted\r\nsource has an expected type.\r\nOnly dict input from untrusted source can be an attack vector.</p>\r\n</blockquote>\r\n<h2>What's Changed</h2>\r\n<ul>\r\n<li>Prohibit dict parameter for <code>Cursor.execute()</code>. It didn't\r\nproduce valid SQL\r\nand might cause SQL injection. (CVE-2024-36039)</li>\r\n<li>Added ssl_key_password param by <a\r\nhref=\"https://github.com/svaskov\"><code>@​svaskov</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1145\">PyMySQL/PyMySQL#1145</a></li>\r\n</ul>\r\n<h2>Merged PRs</h2>\r\n<ul>\r\n<li>Add support for Python 3.12 by <a\r\nhref=\"https://github.com/hugovk\"><code>@​hugovk</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1134\">PyMySQL/PyMySQL#1134</a></li>\r\n<li>chore(deps): update actions/checkout action to v4 by <a\r\nhref=\"https://github.com/renovate\"><code>@​renovate</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1136\">PyMySQL/PyMySQL#1136</a></li>\r\n<li>Update codecov/codecov-action action to v4 by <a\r\nhref=\"https://github.com/renovate\"><code>@​renovate</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1137\">PyMySQL/PyMySQL#1137</a></li>\r\n<li>ci: use codecov@v3 by <a\r\nhref=\"https://github.com/methane\"><code>@​methane</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1142\">PyMySQL/PyMySQL#1142</a></li>\r\n<li>chore(deps): update dessant/lock-threads action to v5 by <a\r\nhref=\"https://github.com/renovate\"><code>@​renovate</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1141\">PyMySQL/PyMySQL#1141</a></li>\r\n<li>doc: use rtd theme by <a\r\nhref=\"https://github.com/methane\"><code>@​methane</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1143\">PyMySQL/PyMySQL#1143</a></li>\r\n<li>use Ruff as formatter by <a\r\nhref=\"https://github.com/methane\"><code>@​methane</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1144\">PyMySQL/PyMySQL#1144</a></li>\r\n<li>chore(deps): update dependency sphinx-rtd-theme to v2 by <a\r\nhref=\"https://github.com/renovate\"><code>@​renovate</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1147\">PyMySQL/PyMySQL#1147</a></li>\r\n<li>chore(deps): update actions/setup-python action to v5 by <a\r\nhref=\"https://github.com/renovate\"><code>@​renovate</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1152\">PyMySQL/PyMySQL#1152</a></li>\r\n<li>chore(deps): update github/codeql-action action to v3 by <a\r\nhref=\"https://github.com/renovate\"><code>@​renovate</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1154\">PyMySQL/PyMySQL#1154</a></li>\r\n<li>chore(deps): update codecov/codecov-action action to v4 by <a\r\nhref=\"https://github.com/renovate\"><code>@​renovate</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1158\">PyMySQL/PyMySQL#1158</a></li>\r\n<li>Support error packet without sqlstate by <a\r\nhref=\"https://github.com/methane\"><code>@​methane</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1160\">PyMySQL/PyMySQL#1160</a></li>\r\n<li>test json - mariadb without JSON type by <a\r\nhref=\"https://github.com/grooverdan\"><code>@​grooverdan</code></a> in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1165\">PyMySQL/PyMySQL#1165</a></li>\r\n</ul>\r\n<h2>New Contributors</h2>\r\n<ul>\r\n<li><a href=\"https://github.com/hugovk\"><code>@​hugovk</code></a> made\r\ntheir first contribution in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1134\">PyMySQL/PyMySQL#1134</a></li>\r\n<li><a href=\"https://github.com/svaskov\"><code>@​svaskov</code></a> made\r\ntheir first contribution in <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/pull/1145\">PyMySQL/PyMySQL#1145</a></li>\r\n</ul>\r\n<p><strong>Full Changelog</strong>: <a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/compare/v1.1.0...v1.1.1\">https://github.com/PyMySQL/PyMySQL/compare/v1.1.0...v1.1.1</a></p>\r\n</blockquote>\r\n</details>\r\n<details>\r\n<summary>Changelog</summary>\r\n<p><em>Sourced from <a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/blob/main/CHANGELOG.md\">pymysql's\r\nchangelog</a>.</em></p>\r\n<blockquote>\r\n<h2>v1.1.1</h2>\r\n<p>Release date: 2024-05-21</p>\r\n<blockquote>\r\n<p>[!WARNING]\r\nThis release fixes a vulnerability (CVE-2024-36039).\r\nAll users are recommended to update to this version.</p>\r\n<p>If you can not update soon, check the input value from\r\nuntrusted source has an expected type. Only dict input\r\nfrom untrusted source can be an attack vector.</p>\r\n</blockquote>\r\n<ul>\r\n<li>Prohibit dict parameter for <code>Cursor.execute()</code>. It didn't\r\nproduce valid SQL\r\nand might cause SQL injection. (CVE-2024-36039)</li>\r\n<li>Added ssl_key_password param. <a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/issues/1145\">#1145</a></li>\r\n</ul>\r\n</blockquote>\r\n</details>\r\n<details>\r\n<summary>Commits</summary>\r\n<ul>\r\n<li><a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/commit/2cab9ecc641e962565c6254a5091f90c47f59b35\"><code>2cab9ec</code></a>\r\nv1.1.1</li>\r\n<li><a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/commit/521e40050cb386a499f68f483fefd144c493053c\"><code>521e400</code></a>\r\nforbid dict parameter</li>\r\n<li><a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/commit/7f032a699d55340f05101deb4d7d4f63db4adc11\"><code>7f032a6</code></a>\r\nremove coveralls from requirements</li>\r\n<li><a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/commit/69f6c7439bee14784e0ea70ae107af6446cc0c67\"><code>69f6c74</code></a>\r\nruff format</li>\r\n<li><a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/commit/b4ed6884a1105df0a27f948f52b3e81d5585634f\"><code>b4ed688</code></a>\r\ntest json - mariadb without JSON type (<a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/issues/1165\">#1165</a>)</li>\r\n<li><a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/commit/bbd049f40db9c696574ce6f31669880042c56d79\"><code>bbd049f</code></a>\r\nSupport error packet without sqlstate (<a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/issues/1160\">#1160</a>)</li>\r\n<li><a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/commit/9694747ae619e88b792a8e0b4c08036572452584\"><code>9694747</code></a>\r\npyupgrade</li>\r\n<li><a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/commit/1f0b7856de4008e7e4c1e8c1b215d5d4dfaecd1a\"><code>1f0b785</code></a>\r\nchore(deps): update codecov/codecov-action action to v4 (<a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/issues/1158\">#1158</a>)</li>\r\n<li><a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/commit/1e28be81c24dde66f8acbf4c5e24f60d6b5e72e7\"><code>1e28be8</code></a>\r\nchore(deps): update github/codeql-action action to v3 (<a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/issues/1154\">#1154</a>)</li>\r\n<li><a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/commit/f13f054abcc18b39855a760a84be0a517f0da658\"><code>f13f054</code></a>\r\nchore(deps): update actions/setup-python action to v5 (<a\r\nhref=\"https://redirect.github.com/PyMySQL/PyMySQL/issues/1152\">#1152</a>)</li>\r\n<li>Additional commits viewable in <a\r\nhref=\"https://github.com/PyMySQL/PyMySQL/compare/v1.1.0...v1.1.1\">compare\r\nview</a></li>\r\n</ul>\r\n</details>\r\n<br />\r\n\r\n\r\n[![Dependabot compatibility\r\nscore](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pymysql&package-manager=pip&previous-version=1.1.0&new-version=1.1.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)\r\n\r\nDependabot will resolve any conflicts with this PR as long as you don't\r\nalter it yourself. You can also trigger a rebase manually by commenting\r\n`@dependabot rebase`.\r\n\r\n[//]: # (dependabot-automerge-start)\r\n[//]: # (dependabot-automerge-end)\r\n\r\n---\r\n\r\n<details>\r\n<summary>Dependabot commands and options</summary>\r\n<br />\r\n\r\nYou can trigger Dependabot actions by commenting on this PR:\r\n- `@dependabot rebase` will rebase this PR\r\n- `@dependabot recreate` will recreate this PR, overwriting any edits\r\nthat have been made to it\r\n- `@dependabot merge` will merge this PR after your CI passes on it\r\n- `@dependabot squash and merge` will squash and merge this PR after\r\nyour CI passes on it\r\n- `@dependabot cancel merge` will cancel a previously requested merge\r\nand block automerging\r\n- `@dependabot reopen` will reopen this PR if it is closed\r\n- `@dependabot close` will close this PR and stop Dependabot recreating\r\nit. You can achieve the same result by closing it manually\r\n- `@dependabot show <dependency name> ignore conditions` will show all\r\nof the ignore conditions of the specified dependency\r\n- `@dependabot ignore this major version` will close this PR and stop\r\nDependabot creating any more for this major version (unless you reopen\r\nthe PR or upgrade to it yourself)\r\n- `@dependabot ignore this minor version` will close this PR and stop\r\nDependabot creating any more for this minor version (unless you reopen\r\nthe PR or upgrade to it yourself)\r\n- `@dependabot ignore this dependency` will close this PR and stop\r\nDependabot creating any more for this dependency (unless you reopen the\r\nPR or upgrade to it yourself)\r\nYou can disable automated security fix PRs for this repo from the\r\n[Security Alerts\r\npage](https://github.com/ShishirPatil/gorilla/network/alerts).\r\n\r\n</details>\r\n\r\nSigned-off-by: dependabot[bot] <support@github.com>\r\nCo-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>","shortMessageHtmlLink":"Bump pymysql from 1.1.0 to 1.1.1 in /goex (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2326808947\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/453\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/453/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/453\">ShishirPatil#453</a>)"}},{"before":"5da8835a840bdfa9faad09aa7236cf3b51c7d659","after":"ba6463ddc397e4808982299964769496387913ce","ref":"refs/heads/main","pushedAt":"2024-05-26T08:50:55.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"[BFCL] update tree_sitter version in requirements.txt (#433)\n\nFollowing the instructions in the `BFCL` README and running the\r\n`eval_data_compilation.py` script results in this error:\r\n\r\n```\r\nAttributeError: type object 'tree_sitter.Language' has no attribute 'build_library'\r\n```\r\n\r\nbecause the `build_library` method was removed in a [recent\r\nrelease](https://github.com/tree-sitter/py-tree-sitter/releases/tag/v0.22.0)\r\nof py-tree-sitter. This PR updates the `requirements.txt` file so that\r\nthe instructions in the README will work for anyone following them now.","shortMessageHtmlLink":"[BFCL] update tree_sitter version in requirements.txt (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2304393571\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/433\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/433/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/433\">ShishirPatil#433</a>)"}},{"before":"624d371e085eefd23ae3f818107490ee8f59e8d9","after":"5da8835a840bdfa9faad09aa7236cf3b51c7d659","ref":"refs/heads/main","pushedAt":"2024-05-18T03:43:45.000Z","pushType":"push","commitsCount":10,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"BFCL May 14th Release (GPT-4o and Gemini) (#426)\n\nThis PR makes 3 models(4 entries) available for inference on BFCL: \r\n- gpt-4o-2024-05-13 (Function Calling Mode and Prompting Mode)\r\n- gemini-1.5-pro-preview-0514 (Function Calling Mode)\r\n- gemini-1.5-flash-preview-0514(Function Calling Mode)\r\n\r\nYou can start the evaluation by running `python\r\nopenfunctions_evaluation.py --model MODEL_NAME` and get score by running\r\n`python ./eval_runner.py --model MODEL_NAME`. For more detail, refer to\r\nReadme under the BFCL page.\r\n\r\nScore changes are reflected in #428 . \r\n\r\nThis PR also updated different models' pricing: \r\n- For Gemini, when prompts are less than 128K tokens, the new Gemini\r\nseries' prices are lowered by around\r\nhalf(https://ai.google.dev/pricing). All the BFCL test cases are less\r\nthan 128K tokens.\r\n- For Anthropic Models, the prices have decreased for claude-2.1 and\r\nClaude-instant-1.2 which have updated accordingly\r\n- For Mistral Models, the prices have been halved for Mistral-large and\r\nMistral-Small\r\n- For OpenAI Models, we have corrected GPT-3.5-turbo-0125 to the price\r\nit should have\r\n\r\n---------\r\n\r\nCo-authored-by: Huanzhi Mao <huanzhimao@gmail.com>","shortMessageHtmlLink":"BFCL May 14th Release (GPT-4o and Gemini) (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2296837512\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/426\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/426/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/426\">ShishirPatil#426</a>)"}},{"before":"1d8d51d0d091c33e730d38745745005c9bc7dfc0","after":"624d371e085eefd23ae3f818107490ee8f59e8d9","ref":"refs/heads/main","pushedAt":"2024-05-07T05:40:47.000Z","pushType":"push","commitsCount":8,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"RAFT Recovery Mode for interruptions  (#410)\n\nImplemented a \"safe\"/recovery mode that periodically saves chunks into\r\n\"checkpoint\" datasets while also keeping track of the current chunk\r\nnumber. In the case of an interruption or crash, the script resumes at\r\nthe saved chunk number. After all chunks have been processed, all\r\ncheckpoint datasets are concatenated and saved as one final dataset.\r\n\r\nAdded an argument allowing user to choose whether to run RAFT in safe or\r\nfast mode (defaults to safe).\r\n\r\nClose #394\r\n\r\n---------\r\n\r\nCo-authored-by: Kaihao Wen <kaiwen129@berkeley.edu>","shortMessageHtmlLink":"RAFT Recovery Mode for interruptions (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2278739789\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/410\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/410/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/410\">ShishirPatil#410</a>)"}},{"before":"16866336b74743706c6d1cd8c6aa650574176f3f","after":"1d8d51d0d091c33e730d38745745005c9bc7dfc0","ref":"refs/heads/main","pushedAt":"2024-04-25T02:31:45.000Z","pushType":"push","commitsCount":6,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"Add missing argument to `OSSHandler`'s `_format_prompt` function (#373)\n\n# Overview\r\n\r\nIn this PR, I added a missing argument to `OSSHandler`'s\r\n`_format_prompt` function.\r\n\r\nAlso my code formatter sorted the imports alphabetically which accounts\r\nfor all the other changes, including those in `handler_map.py`.\r\n\r\n# What Happened?\r\n\r\nIn every model handler, we define `_format_prompt` as taking in three\r\narguments (`prompt`, `function`, `test_category`). This is the standard.\r\nBut in `OSSHandler`, `_format_prompt` is defined as only taking in\r\n`prompt` and `function` and forgetting to take `test_category` as an\r\ninput.\r\n\r\nThis caused an error. So I simply added `test_category` to the\r\ndefinition of `_format_prompt`.","shortMessageHtmlLink":"Add missing argument to <code>OSSHandler</code>'s <code>_format_prompt</code> function (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2253987731\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/373\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/373/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/373\">Shi…</a>"}},{"before":"e0900c6a5578780dd37eb696b9039e4052b039c7","after":"16866336b74743706c6d1cd8c6aa650574176f3f","ref":"refs/heads/main","pushedAt":"2024-04-17T09:08:11.000Z","pushType":"push","commitsCount":8,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"Update requirements.txt (#343)\n\nRunning gorilla_falcon_cli.py withou einops package returns the\r\nfollowing error:\r\n\r\nImportError: This modeling file requires the following packages that\r\nwere not found in your environment: einops. Run `pip install einops`","shortMessageHtmlLink":"Update requirements.txt (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2238451240\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/343\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/343/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/343\">ShishirPatil#343</a>)"}},{"before":"e23476b7bce575743c51438a9377ec87cefe797a","after":"e0900c6a5578780dd37eb696b9039e4052b039c7","ref":"refs/heads/main","pushedAt":"2024-04-13T04:01:46.000Z","pushType":"push","commitsCount":6,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"Add Support for NousResearch/Hermes-2-Pro-Mistral-7B Function Calling (#327)\n\nThis PR introduces support for [\r\nNousResearch/Hermes-2-Pro-Mistral-7B](https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B).\r\nSpecifically, it includes:\r\n\r\n- The implementation of a function calling system prompt passing in\r\nOpenAI compatible JSON function descriptions, as detailed on the\r\nHuggingface model page.\r\n- The ability to parse raw Hermes output into a standard {\"name\": ...,\r\n\"arguments\": ...} dictionary format.\r\n- A refactoring of `oss_handler.py` to specify `test_category` during\r\nthe VLLM batch generation process. This enables special handling for\r\nfunction JSON parameter types in the java and javascript categories.\r\n\r\nThis PR will result in changes to the leaderboard by adding an entry for\r\none model. The PR that makes changes to the leaderboard webpage is\r\nlinked here [PR](https://github.com/ShishirPatil/gorilla/pull/328).\r\n\r\n---------\r\n\r\nCo-authored-by: CharlieJCJ <charliechengjieji@berkeley.edu>\r\nCo-authored-by: Huanzhi Mao <huanzhimao@gmail.com>","shortMessageHtmlLink":"Add Support for NousResearch/Hermes-2-Pro-Mistral-7B Function Calling (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2230486674\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/327\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/327/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/327\">…</a>"}},{"before":"82f8fc5e82173628df842625913ff7207b7c5970","after":"e23476b7bce575743c51438a9377ec87cefe797a","ref":"refs/heads/main","pushedAt":"2024-04-09T14:43:53.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"Correction in BFCL README instruction, fixed path in instructions (#329)\n\n- Corrected a legacy path to be consistent in\r\nberkeley-function-call-leaderboard folder README instruction.","shortMessageHtmlLink":"Correction in BFCL README instruction, fixed path in instructions (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2232219507\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/329\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/329/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/329\">Sh…</a>"}},{"before":"d8b3433e618c8af0db3ec9c71d7300007980b265","after":"82f8fc5e82173628df842625913ff7207b7c5970","ref":"refs/heads/main","pushedAt":"2024-04-05T14:16:19.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"Leaderboard April 3 release (#309)\n\nThis PR is for the leaderboard April 3 release:\r\n\r\n1. Bug fix for evaluation dataset possible answers, including those that\r\nare identified in #301.\r\n2. Implement **string standardization** for the AST evaluation pipeline,\r\ni.e. removing white spaces and a subset of punctuations (`,./-_*^`) to\r\nmake the AST evaluation more robust and accurate.\r\n3. Fix AST evaluation issue for type `tuple`. \r\n4. Fix AST evaluation issue for Java and JavaScript.\r\n5. Add 2 new models `meetkai/functionary-small-v2.4 (FC)`,\r\n`meetkai/functionary-medium-v2.4 (FC)` to the leaderboard.\r\n\r\nThis PR **DOES** change the leaderboard score. We will update the\r\nleaderboard website shortly, in a different PR.\r\n\r\n---------\r\n\r\nCo-authored-by: Charlie Cheng-Jie Ji <charliechengjieji@berkeley.edu>\r\nCo-authored-by: Fanjia Yan <fanjiayan@berkeley.edu>\r\n\r\n---------\r\n\r\nCo-authored-by: CharlieJCJ <charliechengjieji@berkeley.edu>","shortMessageHtmlLink":"Leaderboard April 3 release (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2224769619\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/309\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/309/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/309\">ShishirPatil#309</a>)"}},{"before":"c1b67eed241c419f5a9929993dcb4a117b6029b5","after":"d8b3433e618c8af0db3ec9c71d7300007980b265","ref":"refs/heads/main","pushedAt":"2024-04-03T14:35:30.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"Update README.md\n\nNews - Berkeley Function Calling Leaderboard v2 with fixes to data-inconsistencies, introducing cost and latency, and Claude-3 takes the top spot.","shortMessageHtmlLink":"Update README.md"}},{"before":"84ab3b6de0d03da567e76ff392207cff53805faa","after":"c1b67eed241c419f5a9929993dcb4a117b6029b5","ref":"refs/heads/main","pushedAt":"2024-03-26T08:22:00.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"[Fix] java, javascript parsers in openfunctions-v2 (#284)\n\n**Context**: Parser is used to convert model response str -> OpenAI\r\ncompatible `function_call`\r\n- Corrected edge cases with `new` keyword\r\n- Corrected `[Class.]method` optionally include Class in method. \r\n- Add recursive logic in `traverse_node` java parser, to special handle\r\nstrings\r\n- Corrected initialization with `ArrayList`, `Array` keywords\r\nformatting.\r\n- Additional comments, example usage and expected outputs included.\r\n\r\n**Example:** `PmsProductServiceImpl.updateNewStatus(ids=new\r\nArrayList<Long>(Arrays.asList(101L, 202L, 303L)), newStatus=2)` ->\r\n`{'function': {'name': 'PmsProductServiceImpl.updateNewStatus',\r\n'parameters': {'ids': 'new ArrayList<Long>(Arrays.asList(101L, 202L,\r\n303L))', 'newStatus': '2'}}}`","shortMessageHtmlLink":"[Fix] java, javascript parsers in openfunctions-v2 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2204175646\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/284\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/284/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/284\">ShishirPatil#284</a>)"}},{"before":"f32793233a950c9b16d716632d84129181d1e137","after":"84ab3b6de0d03da567e76ff392207cff53805faa","ref":"refs/heads/main","pushedAt":"2024-03-24T09:27:03.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"feat: more type annotations for the functions (#283)\n\nrelated to #282","shortMessageHtmlLink":"feat: more type annotations for the functions (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2203932579\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/283\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/283/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/283\">ShishirPatil#283</a>)"}},{"before":"b009a076a6c28955f55a4e2b12532cfd00b54104","after":"f32793233a950c9b16d716632d84129181d1e137","ref":"refs/heads/main","pushedAt":"2024-03-19T14:35:44.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"Adding Datadog API to API Zoo (#206)\n\nExhaustively added Datadog API to API Zoo","shortMessageHtmlLink":"Adding Datadog API to API Zoo (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2152621981\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/206\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/206/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/206\">ShishirPatil#206</a>)"}},{"before":"0684261601094673e5be5d5f1c9e0cacea478289","after":"b009a076a6c28955f55a4e2b12532cfd00b54104","ref":"refs/heads/main","pushedAt":"2024-03-16T04:01:41.000Z","pushType":"push","commitsCount":11,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"[Bug Fix] Restrict AST checker on Boolean Variable (#256)\n\nThis PR removes any AST checker answers that contain the string literals\r\n\"True\" and \"False\", thereby restricting the conditionals of the AST\r\nchecker to strictly boolean value.\r\n\r\nExample:\r\n\r\n`[true, \"true\"] -> [true]`\r\n\r\nThis PR will stack on top of #252\r\n\r\nWill this change the results of leaderboard? YES\r\n\r\nhttps://github.com/ShishirPatil/gorilla/pull/253 will updated the\r\nleaderboard with this fix.","shortMessageHtmlLink":"[Bug Fix] Restrict AST checker on Boolean Variable (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2180855032\" data-permission-text=\"Title is private\" data-url=\"https://github.com/ShishirPatil/gorilla/issues/256\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/ShishirPatil/gorilla/pull/256/hovercard\" href=\"https://github.com/ShishirPatil/gorilla/pull/256\">ShishirPatil#256</a>)"}},{"before":"4f1bf71d7b0923b41fa54352ed4be0fa8fca3298","after":"0684261601094673e5be5d5f1c9e0cacea478289","ref":"refs/heads/main","pushedAt":"2024-03-12T08:52:13.000Z","pushType":"push","commitsCount":8,"pusher":{"login":"leixy76","name":null,"path":"/leixy76","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/162015206?s=80&v=4"},"commit":{"message":"Update README.md for release-date and cosmetics","shortMessageHtmlLink":"Update README.md for release-date and cosmetics"}}],"hasNextPage":false,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEfzoZEwA","startCursor":null,"endCursor":null}},"title":"Activity · leixy76/gorilla"}