Skip to content

Commit

Permalink
Merge branch 'main' into zhyncs/v0.4.1.post4
Browse files Browse the repository at this point in the history
  • Loading branch information
@zhyncs
zhyncs authored Jan 2, 2025
2 parents b0c7316 + dd2e2d2 commit ee6ebf7
Show file tree
Hide file tree
Showing 4 changed files with 86 additions and 155 deletions.
7 changes: 4 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,10 @@ The core features include:

## Getting Started
- [Install SGLang](https://sgl-project.github.io/start/install.html)
- [Send requests](https://sgl-project.github.io/start/send_request.html)
- [Backend: SGLang Runtime (SRT)](https://sgl-project.github.io/backend/backend.html)
- [Frontend: Structured Generation Language (SGLang)](https://sgl-project.github.io/frontend/frontend.html)
- [Quick Start](https://sgl-project.github.io/start/send_request.html)
- [Backend Tutorial](https://sgl-project.github.io/backend/openai_api_completions.html)
- [Frontend Tutorial](https://sgl-project.github.io/frontend/frontend.html)
- [Contribution Guide](https://sgl-project.github.io/references/contribution_guide.html)

## Benchmark and Performance
Learn more in our release blogs: [v0.2 blog](https://lmsys.org/blog/2024-07-25-sglang-llama3/), [v0.3 blog](https://lmsys.org/blog/2024-09-04-sglang-v0-3/), [v0.4 blog](https://lmsys.org/blog/2024-12-04-sglang-v0-4/)
Expand Down
82 changes: 62 additions & 20 deletions docs/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,35 +1,77 @@
# SGLang Documentation
This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang/tree/main/docs.
We recommend new contributors start from writing documentation, which helps you quickly understand SGLang codebase. Most documentation files are located under the `docs/` folder. We prefer **Jupyter Notebooks** over Markdown so that all examples can be executed and validated by our docs CI pipeline.

## Build the documentation website
## Docs Workflow

### Dependency
```
### Install Dependency

```bash
pip install -r requirements.txt
```

### Build
```
### Update Documentation

Update your Jupyter notebooks in the appropriate subdirectories under `docs/`. If you add new files, remember to update `index.rst` (or relevant `.rst` files) accordingly.

- **`pre-commit run --all-files`** manually runs all configured checks, applying fixes if possible. If it fails the first time, re-run it to ensure lint errors are fully resolved. Make sure your code passes all checks **before** creating a Pull Request.
- **Do not commit** directly to the `main` branch. Always create a new branch (e.g., `feature/my-new-feature`), push your changes, and open a PR from that branch.

```bash
# 1) Compile all Jupyter notebooks
make compile

# 2) Generate static HTML
make html
```

### Clean
To remove all generated files:
```
make clean
```
# 3) Preview documentation locally
# Open your browser at the displayed port to view the docs
bash serve.sh

### Serve (preview)
Run an HTTP server and visit http://localhost:8000 in your browser.
# 4) Clean notebook outputs
# nbstripout removes notebook outputs so your PR stays clean
pip install nbstripout
find . -name '*.ipynb' -exec nbstripout {} \;

# 5) Pre-commit checks and create a PR
# After these checks pass, push your changes and open a PR on your branch
pre-commit run --all-files
```
python3 -m http.server --d _build/html


If you need to run and shut down a SGLang server or engine, following these examples:

1. Launch and close Sever:

```python
#Launch Sever

from sglang.utils import (
execute_shell_command,
wait_for_server,
terminate_process,
print_highlight,
)

server_process = execute_shell_command(
"python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct --port 30000 --host 0.0.0.0"
)

wait_for_server("http://localhost:30000")

# Terminate Sever

terminate_process(server_process)
```
2. Launch Engine and close Engine

### Deploy
Clone [sgl-project.github.io](https://github.com/sgl-project/sgl-project.github.io) and make sure you have write access.
```python
# Launch Engine

```bash
export DOC_SITE_PATH=../../sgl-project.github.io # update this with your path
python3 deploy.py
import sglang as sgl
import asyncio

llm = sgl.Engine(model_path="meta-llama/Meta-Llama-3.1-8B-Instruct")

# Terminalte Engine
llm.shutdown()
```
137 changes: 5 additions & 132 deletions docs/references/contribution_guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,162 +6,35 @@ Welcome to **SGLang**! We appreciate your interest in contributing. This guide p

### Fork and Clone the Repository

**Note**: SGLang does **not** accept PRs on the main repo. Please fork the repository under your GitHub account, then clone your fork locally.
**Note**: New contributors do **not** have the write permission to push to SGLang. Please fork the repository under your GitHub account, then clone your fork locally.

```bash
git clone https://github.com/<your_user_name>/sglang.git
cd sglang
```

### Install Dependencies & Build

Refer to [Install SGLang](https://sgl-project.github.io/start/install.html) documentation for more details on setting up the necessary dependencies.

Install correct version of flashinfer according to your PyTorch and CUDA versions.
```bash
python -c "import torch; print('PyTorch Version:', torch.__version__); print('CUDA Version:', torch.version.cuda)"
```

Below is an example on PyTorch 2.4 with CUDA 12.4:
```bash
pip install --upgrade pip
pip install -e "python[all]" --find-links https://flashinfer.ai/whl/cu124/torch2.4/flashinfer/

# Install the lateset SGLang on your own fork repo
cd sglang/python
pip install .
```
Refer to [Install SGLang from Source](https://sgl-project.github.io/start/install.html#method-2-from-source) documentation for more details on setting up the necessary dependencies.

## Code Formatting with Pre-Commit

We use [pre-commit](https://pre-commit.com/) to maintain consistent code style checks. Before pushing your changes, please run:

```bash
pip3 install pre-commit
cd sglang
pre-commit run --all-files
```

- **`pre-commit run --all-files`** manually runs all configured checks, applying fixes if possible. If it fails the first time, re-run it to ensure lint errors are fully resolved. Make sure your code passes all checks **before** creating a Pull Request.
- **Do not commit** directly to the `main` branch. Always create a new branch (e.g., `feature/my-new-feature`), push your changes, and open a PR from that branch.

## Writing Documentation & Running Docs CI

Most documentation files are located under the `docs/` folder. We prefer **Jupyter Notebooks** over Markdown so that all examples can be executed and validated by our docs CI pipeline.

### Docs Workflow

Add or update your Jupyter notebooks in the appropriate subdirectories under `docs/`. If you add new files, remember to update `index.rst` (or relevant `.rst` files) accordingly.

```bash
# 1) Compile all Jupyter notebooks
make compile

# 2) Generate static HTML
make html

# 3) Preview documentation locally
bash serve.sh
# Open your browser at the displayed port to view the docs

# 4) Clean notebook outputs
pip install nbstripout
find . -name '*.ipynb' -exec nbstripout {} \;
# nbstripout removes notebook outputs so your PR stays clean

# 5) Pre-commit checks and create a PR
pre-commit run --all-files
# After these checks pass, push your changes and open a PR on your branch
```


If you need to run and shut up a SGLang server or engine, following these examples:

1. Launch and close Sever:

```python
#Launch Sever

from sglang.utils import (
execute_shell_command,
wait_for_server,
terminate_process,
print_highlight,
)

server_process = execute_shell_command(
"python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct --port 30000 --host 0.0.0.0"
)

wait_for_server("http://localhost:30000")

# Terminate Sever

terminate_process(server_process)
```
2. Launch Engine and close Engine

```python
# Launch Engine

import sglang as sgl
import asyncio

llm = sgl.Engine(model_path="meta-llama/Meta-Llama-3.1-8B-Instruct")

# Terminalte Engine
llm.shutdown()
```


## Running Unit Tests & Adding to CI

SGLang uses Python’s built-in [unittest](https://docs.python.org/3/library/unittest.html) framework. You can run tests either individually or in suites.

### Test Backend Runtime

```bash
cd sglang/test/srt

# Run a single file
python3 test_srt_endpoint.py
SGLang uses Python's built-in [unittest](https://docs.python.org/3/library/unittest.html) framework. For detailed instructions on running tests and adding them to CI, please refer to [test/README.md](https://github.com/sgl-project/sglang/tree/main/test/README.md).

# Run a single test
python3 -m unittest test_srt_endpoint.TestSRTEndpoint.test_simple_decode

# Run a suite with multiple files
python3 run_suite.py --suite minimal
```

### Test Frontend Language

```bash
cd sglang/test/lang
export OPENAI_API_KEY=sk-*****

# Run a single file
python3 test_openai_backend.py

# Run a single test
python3 -m unittest test_openai_backend.TestOpenAIBackend.test_few_shot_qa

# Run a suite with multiple files
python3 run_suite.py --suite minimal
```

### Adding or Updating Tests in CI

- Create new test files under `test/srt` or `test/lang` depending on the type of test.
- Ensure they are referenced in the respective `run_suite.py` (e.g., `test/srt/run_suite.py` or `test/lang/run_suite.py`) so they’re picked up in CI.
- In CI, all tests run automatically. You may modify the workflows in [`.github/workflows/`](https://github.com/sgl-project/sglang/tree/main/.github/workflows) to add custom test groups or extra checks.

### Writing Elegant Test Cases
## Writing Documentation & Running Docs CI

- Examine existing tests in [sglang/test](https://github.com/sgl-project/sglang/tree/main/test) for practical examples.
- Keep each test function focused on a single scenario or piece of functionality.
- Give tests descriptive names reflecting their purpose.
- Use robust assertions (e.g., assert, unittest methods) to validate outcomes.
- Clean up resources to avoid side effects and preserve test independence.
We recommend new contributors start from writing documentation, which helps you quickly understand SGLang codebase. For more details, please refer to [docs/README.md](https://github.com/sgl-project/sglang/tree/main/docs/README.md).


## Tips for Newcomers
Expand Down
15 changes: 15 additions & 0 deletions test/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,18 @@ python3 -m unittest test_openai_backend.TestOpenAIBackend.test_few_shot_qa
# Run a suite with multiple files
python3 run_suite.py --suite minimal
```

## Adding or Updating Tests in CI

- Create new test files under `test/srt` or `test/lang` depending on the type of test.
- Ensure they are referenced in the respective `run_suite.py` (e.g., `test/srt/run_suite.py` or `test/lang/run_suite.py`) so they’re picked up in CI.
- In CI, all tests run automatically. You may modify the workflows in [`.github/workflows/`](https://github.com/sgl-project/sglang/tree/main/.github/workflows) to add custom test groups or extra checks.


## Writing Elegant Test Cases

- Examine existing tests in [sglang/test](https://github.com/sgl-project/sglang/tree/main/test) for practical examples.
- Keep each test function focused on a single scenario or piece of functionality.
- Give tests descriptive names reflecting their purpose.
- Use robust assertions (e.g., assert, unittest methods) to validate outcomes.
- Clean up resources to avoid side effects and preserve test independence.

0 comments on commit ee6ebf7

Please sign in to comment.