From 25e5d589e39b3b605296395e4f9c96ec42f09055 Mon Sep 17 00:00:00 2001 From: Shi Shuai <126407087+shuaills@users.noreply.github.com> Date: Mon, 23 Dec 2024 01:14:40 +0000 Subject: [PATCH] Doc: Update Grammar Backend (#2545) Co-authored-by: Chayenne --- docs/backend/openai_api_completions.ipynb | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/backend/openai_api_completions.ipynb b/docs/backend/openai_api_completions.ipynb index 067a046885..9340f953f1 100644 --- a/docs/backend/openai_api_completions.ipynb +++ b/docs/backend/openai_api_completions.ipynb @@ -221,7 +221,14 @@ "metadata": {}, "source": [ "## Structured decoding (JSON, Regex)\n", - "You can specify a JSON schema or a regular expression to constrain the model output. The model output will be guaranteed to follow the given constraints.\n", + "You can define a JSON schema or regular expression to constrain the model's output. The model output will be guaranteed to follow the given constraints and this depends on the grammar backend.\n", + "\n", + "SGlang has two backends: outlines (default) and Xgrammar. Xgrammar enhances JSON decoding performance but does not support regular expressions. To use Xgrammar, add the `--grammar-backend xgrammar` when launching the server:\n", + "\n", + "```bash\n", + "python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct \\\n", + "--port 30000 --host 0.0.0.0 --grammar-backend xgrammar\n", + "```\n", "\n", "### JSON" ]