Skip to content

Commit

Permalink
Doc: Update Grammar Backend (#2545)
Browse files Browse the repository at this point in the history
Co-authored-by: Chayenne <[email protected]>
  • Loading branch information
shuaills and zhaochenyang20 authored Dec 23, 2024
1 parent 41b1db6 commit 25e5d58
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion docs/backend/openai_api_completions.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,14 @@
"metadata": {},
"source": [
"## Structured decoding (JSON, Regex)\n",
"You can specify a JSON schema or a regular expression to constrain the model output. The model output will be guaranteed to follow the given constraints.\n",
"You can define a JSON schema or regular expression to constrain the model's output. The model output will be guaranteed to follow the given constraints and this depends on the grammar backend.\n",
"\n",
"SGlang has two backends: outlines (default) and Xgrammar. Xgrammar enhances JSON decoding performance but does not support regular expressions. To use Xgrammar, add the `--grammar-backend xgrammar` when launching the server:\n",
"\n",
"```bash\n",
"python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct \\\n",
"--port 30000 --host 0.0.0.0 --grammar-backend xgrammar\n",
"```\n",
"\n",
"### JSON"
]
Expand Down

0 comments on commit 25e5d58

Please sign in to comment.