diff --git a/docs/backend/openai_api_completions.ipynb b/docs/backend/openai_api_completions.ipynb index 0f88fd964b..8660da2f98 100644 --- a/docs/backend/openai_api_completions.ipynb +++ b/docs/backend/openai_api_completions.ipynb @@ -24,14 +24,7 @@ "source": [ "## Launch A Server\n", "\n", - "This code block is equivalent to executing \n", - "\n", - "```bash\n", - "python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct \\\n", - "--port 30000 --host 0.0.0.0\n", - "```\n", - "\n", - "in your terminal and wait for the server to be ready." + "Launch the server in your terminal and wait for it to initialize." ] }, { diff --git a/docs/backend/openai_api_embeddings.ipynb b/docs/backend/openai_api_embeddings.ipynb index 65b07c384d..67ce68bcf1 100644 --- a/docs/backend/openai_api_embeddings.ipynb +++ b/docs/backend/openai_api_embeddings.ipynb @@ -20,14 +20,7 @@ "source": [ "## Launch A Server\n", "\n", - "The following code is equivalent to running this in the shell:\n", - "\n", - "```bash\n", - "python -m sglang.launch_server --model-path Alibaba-NLP/gte-Qwen2-7B-instruct \\\n", - " --port 30000 --host 0.0.0.0 --is-embedding\n", - "```\n", - "\n", - "Remember to add `--is-embedding` to the command." + "Launch the server in your terminal and wait for it to initialize. Remember to add `--is-embedding` to the command." ] }, { diff --git a/docs/backend/openai_api_vision.ipynb b/docs/backend/openai_api_vision.ipynb index af17b44096..da8864c24c 100644 --- a/docs/backend/openai_api_vision.ipynb +++ b/docs/backend/openai_api_vision.ipynb @@ -22,13 +22,7 @@ "source": [ "## Launch A Server\n", "\n", - "This code block is equivalent to executing \n", - "\n", - "```bash\n", - "python3 -m sglang.launch_server --model-path meta-llama/Llama-3.2-11B-Vision-Instruct \\\n", - " --port 30000 --chat-template llama_3_vision\n", - "```\n", - "in your terminal and wait for the server to be ready.\n", + "Launch the server in your terminal and wait for it to initialize.\n", "\n", "Remember to add `--chat-template llama_3_vision` to specify the vision chat template, otherwise the server only supports text.\n", "We need to specify `--chat-template` for vision language models because the chat template provided in Hugging Face tokenizer only supports text." diff --git a/docs/references/contribution_guide.md b/docs/references/contribution_guide.md new file mode 100644 index 0000000000..182a73683a --- /dev/null +++ b/docs/references/contribution_guide.md @@ -0,0 +1,173 @@ +# Contribution Guide + +Welcome to **SGLang**! We appreciate your interest in contributing. This guide provides a concise overview of how to set up your environment, run tests, build documentation, and open a Pull Request (PR). Whether you’re fixing a small bug or developing a major feature, we encourage following these steps for a smooth contribution process. + +## 1. Setting Up & Building from Source + +### 1.1 Fork and Clone the Repository + +**Note**: SGLang does **not** accept PRs on the main repo. Please fork the repository under your GitHub account, then clone your fork locally. + +```bash +git clone https://github.com//sglang.git +cd sglang +``` + +### 1.2 Install Dependencies & Build + +Refer to [Install SGLang](https://sgl-project.github.io/start/install.html) documentation for more details on setting up the necessary dependencies. + +Install correct version of flashinfer according to your PyTorch and CUDA versions. +```bash +python -c "import torch; print('PyTorch Version:', torch.__version__); print('CUDA Version:', torch.version.cuda)" +``` + +Below is an example on PyTorch 2.4 with CUDA 12.4: +```bash +pip install --upgrade pip +pip install -e "python[all]" --find-links https://flashinfer.ai/whl/cu124/torch2.4/flashinfer/ + +# Install the lateset SGLang on your own fork repo +cd sglang/python +pip install . +``` + +## 2. Code Formatting with Pre-Commit + +We use [pre-commit](https://pre-commit.com/) to maintain consistent code style checks. Before pushing your changes, please run: + +```bash +pip3 install pre-commit +cd sglang +pre-commit run --all-files +``` + +- **`pre-commit run --all-files`** manually runs all configured checks, applying fixes if possible. If it fails the first time, re-run it to ensure lint errors are fully resolved. Make sure your code passes all checks **before** creating a Pull Request. +- **Do not commit** directly to the `main` branch. Always create a new branch (e.g., `feature/my-new-feature`), push your changes, and open a PR from that branch. + +## 3. Writing Documentation & Running Docs CI + +Most documentation files are located under the `docs/` folder. We prefer **Jupyter Notebooks** over Markdown so that all examples can be executed and validated by our docs CI pipeline. + +### 3.1 Docs Workflow + +Add or update your Jupyter notebooks in the appropriate subdirectories under `docs/`. If you add new files, remember to update `index.rst` (or relevant `.rst` files) accordingly. + +```bash +# 1) Compile all Jupyter notebooks +make compile + +# 2) Generate static HTML +make html + +# 3) Preview documentation locally +bash serve.sh +# Open your browser at the displayed port to view the docs + +# 4) Clean notebook outputs +pip install nbstripout +find . -name '*.ipynb' -exec nbstripout {} \; +# nbstripout removes notebook outputs so your PR stays clean + +# 5) Pre-commit checks and create a PR +pre-commit run --all-files +# After these checks pass, push your changes and open a PR on your branch +``` + + +If you need to run and shut up a SGLang server or engine, following these examples: + +1. Launch and close Sever: + +```python +#Launch Sever + +from sglang.utils import ( + execute_shell_command, + wait_for_server, + terminate_process, + print_highlight, +) + +server_process = execute_shell_command( + "python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct --port 30000 --host 0.0.0.0" +) + +wait_for_server("http://localhost:30000") + +# Terminate Sever + +terminate_process(server_process) +``` +2. Launch Engine and close Engine + +```python +# Launch Engine + +import sglang as sgl +import asyncio + +llm = sgl.Engine(model_path="meta-llama/Meta-Llama-3.1-8B-Instruct") + +# Terminalte Engine +llm.shutdown() +``` + + +## 4. Running Unit Tests & Adding to CI + +SGLang uses Python’s built-in [unittest](https://docs.python.org/3/library/unittest.html) framework. You can run tests either individually or in suites. + +### 4.1 Test Backend Runtime + +```bash +cd sglang/test/srt + +# Run a single file +python3 test_srt_endpoint.py + +# Run a single test +python3 -m unittest test_srt_endpoint.TestSRTEndpoint.test_simple_decode + +# Run a suite with multiple files +python3 run_suite.py --suite minimal +``` + +### 4.2 Test Frontend Language + +```bash +cd sglang/test/lang +export OPENAI_API_KEY=sk-***** + +# Run a single file +python3 test_openai_backend.py + +# Run a single test +python3 -m unittest test_openai_backend.TestOpenAIBackend.test_few_shot_qa + +# Run a suite with multiple files +python3 run_suite.py --suite minimal +``` + +### 4.3 Adding or Updating Tests in CI + +- Create new test files under `test/srt` or `test/lang` depending on the type of test. +- Ensure they are referenced in the respective `run_suite.py` (e.g., `test/srt/run_suite.py` or `test/lang/run_suite.py`) so they’re picked up in CI. +- In CI, all tests run automatically. You may modify the workflows in [`.github/workflows/`](https://github.com/sgl-project/sglang/tree/main/.github/workflows) to add custom test groups or extra checks. + +### 4.4 Writing Elegant Test Cases + +- Examine existing tests in [sglang/test](https://github.com/sgl-project/sglang/tree/main/test) for practical examples. +- Keep each test function focused on a single scenario or piece of functionality. +- Give tests descriptive names reflecting their purpose. +- Use robust assertions (e.g., assert, unittest methods) to validate outcomes. +- Clean up resources to avoid side effects and preserve test independence. + + +## 5. Tips for Newcomers + +If you want to contribute but don’t have a specific idea in mind, pick issues labeled [“good first issue” or “help wanted”](https://github.com/sgl-project/sglang/issues?q=is%3Aissue+label%3A%22good+first+issue%22%2C%22help+wanted%22). These tasks typically have lower complexity and provide an excellent introduction to the codebase. Also check out this [code walk-through](https://github.com/zhaochenyang20/Awesome-ML-SYS-Tutorial/tree/main/sglang/code-walk-through) for a deeper look into SGLang’s workflow. + +If you have any questions or want to start a discussion, please feel free to ask in our [Slack channel](https://join.slack.com/t/sgl-fru7574/shared_invite/zt-2um0ad92q-LkU19KQTxCGzlCgRiOiQEw). + +Thank you for your interest in SGLang—**happy coding**! diff --git a/docs/references/contributor_guide.md b/docs/references/contributor_guide.md deleted file mode 100644 index 15be5a703a..0000000000 --- a/docs/references/contributor_guide.md +++ /dev/null @@ -1,22 +0,0 @@ -# Contributor Guide - -## Build SGLang - -See [Install SGLang, Method 2: From Source section](../start/install.md). - -## Format Your Code -Use these commands to format your code and pass CI linting tests. - -``` -pip3 install pre-commit -cd sglang -pre-commit install -pre-commit run --all-files -``` - -## Add Unit Tests -Add unit tests under [sglang/test](https://github.com/sgl-project/sglang/tree/main/test). You can learn how to add and run tests from the README.md in that folder. - -## For Newcomers -If you want to contribute or learn but don't have a concrete idea yet, you can pick a task labeled as "good first issue" or "help wanted" from the list below. -[https://github.com/sgl-project/sglang/issues?q=is%3Aissue+label%3A%22good+first+issue%22%2C%22help+wanted%22](https://github.com/sgl-project/sglang/issues?q=is%3Aissue+label%3A%22good+first+issue%22%2C%22help+wanted%22)