Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Add Docs For Quantization #2531

Open
binhtranmcs opened this issue Dec 20, 2024 · 2 comments
Open

[Feature] Add Docs For Quantization #2531

binhtranmcs opened this issue Dec 20, 2024 · 2 comments
Assignees
@zhaochenyang20
Labels
good first issue Good for newcomers quant LLM Quantization

Comments

@binhtranmcs
Copy link

Quick question, what is the recommended way to do offline quantization? I cannot find any documents on this. Thanks in advance!
@zhaochenyang20
Copy link
Collaborator

https://docs.vllm.ai/en/v0.6.2/quantization/fp8.html

Check this please. We will also add docs for Quantization.

@zhaochenyang20 zhaochenyang20 self-assigned this Dec 22, 2024
@zhaochenyang20 zhaochenyang20 changed the title How to do quantization [Feature] Add Docs For Quantization Dec 22, 2024
@zhaochenyang20 zhaochenyang20 added good first issue Good for newcomers quant LLM Quantization labels Dec 22, 2024
@zhaochenyang20
Copy link
Collaborator

@JamesSand

I think this is a good start point for you to understand how SGLang works 😂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zhaochenyang20 zhaochenyang20

Labels
good first issue Good for newcomers quant LLM Quantization
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@binhtranmcs @zhaochenyang20