Skip to content

Issues: vllm-project/vllm

[Roadmap] vLLM Roadmap Q4 2024
#9006 opened Oct 1, 2024 by simon-mo
Open 26
vLLM's V1 Engine Architecture
#8779 opened Sep 24, 2024 by simon-mo
Open 10
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1,201 Open 4,607 Closed
1,201 Open 4,607 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Installation]: XPU dependencies not built against most recent oneAPI installation Installation problems
#11734 opened Jan 4, 2025 by janimo
1 task done
[Usage]: serving 'LLaVA-Next-Video-7B-Qwen2' usage How to use vllm
#11731 opened Jan 4, 2025 by Noctis-SC
1 task done
3
[Feature]: Does vLLM plan to support host multiple llm base models inside one server feature request
#11729 opened Jan 4, 2025 by ynwang007
1 task done
[Feature]: Publish an Arm image for GH200 feature request
#11728 opened Jan 3, 2025 by samos123
1 task done
[Bug]: PixtralHF inference broken since #11396 bug Something isn't working
#11726 opened Jan 3, 2025 by mgoin
1 task done
@DarkLight1337
1
[New Model]: unsloth/Llama-3.3-70B-Instruct-bnb-4bit new model Requests to new models
#11725 opened Jan 3, 2025 by Hyfred
1 task done
1
[Feature]: membind all NUMA nodes for all CPUs in list feature request
#11720 opened Jan 3, 2025 by hpcpony
1 task done
[Bug]: ValueError: Model architectures ['LlamaForCausalLM'] failed to be inspected bug Something isn't working
#11715 opened Jan 3, 2025 by npanpaliya
1 task done
1
[Bug]: Extremely slow inference speed when deploying with vLLM on 16 H100 GPUs according to instructions on DeepSeekV3 bug Something isn't working
#11705 opened Jan 3, 2025 by yonghenglh6
1 task done
1
[Bug]: Mismatch multi-modal placeholder of LLava-1.6-Mistral-7B bug Something isn't working
#11704 opened Jan 3, 2025 by jianghuyihei
1 task done
1
@DarkLight1337
1
[Bug]: 0.6.6.post1 crash in marlin_utils.py bug Something isn't working
#11703 opened Jan 3, 2025 by Flynn-Zh
1 task done
2
[Bug]: vLLM LoRA Crash when using Dynamic Loading bug Something isn't working
#11702 opened Jan 3, 2025 by haitwang-cloud
1 task done
1
5
[Feature]: The tool_choice option required is not yet supported but on the roadmap. feature request
#11700 opened Jan 3, 2025 by yumc2573
1 task done
[Bug]: vLLM is erroneously sending some information outputs into the error stream bug Something isn't working
#11686 opened Jan 2, 2025 by mrakgr
1 task done
2
[Bug]: Error while importing vllm since v0.6.6 bug Something isn't working
#11683 opened Jan 2, 2025 by kkimmk
1 task done
13
[Usage]: Trying to add codeshell 7b model, but garbled characters usage How to use vllm
#11681 opened Jan 2, 2025 by G1017
1 task done
1
[Bug]: The output size is not aligned with the quantized weight shape. This can be caused by too large tensor parallel size. bug Something isn't working
#11671 opened Jan 1, 2025 by ShelterWFF
1 task done
5
[Bug]: Continuous batching (OpenAI Server) with greedy search return different results bug Something isn't working
#11658 opened Dec 31, 2024 by thangld201
1 task done
2
[Bug]: I started a qwen2vl-7b video processing service using vllm (0.6.6), but encountered an error during inference bug Something isn't working
#11657 opened Dec 31, 2024 by hyyuananran
1 task done
6
[Feature]: Support Inflight quantization: load as 8bit quantization. feature request
#11655 opened Dec 31, 2024 by ShelterWFF
1 task done
9
[Bug]: NotImplementedError: No operator found for memory_efficient_attention_forward bug Something isn't working
#11653 opened Dec 31, 2024 by AnthonyX1an
1 task done
1
[Usage]: I would like to know how to transfer fps and max_pixels after starting a qwen2vl-7b service using vllm? usage How to use vllm
#11652 opened Dec 31, 2024 by hyyuananran
1 task done
3
[Usage]: multinode inference: assert value == world_size, f"Expected {world_size}, got {value}" AssertionError: Expected 16, got 1.0 usage How to use vllm
#11651 opened Dec 31, 2024 by mangomatrix
1 task done
[Performance]: V1 vs V0 with multi-steps performance Performance-related issues
#11649 opened Dec 31, 2024 by Desmond819
1 task done
[Bug]: I try to use vllm==0.6.5 for GLM4-9b-chat but error "/usr/bin/ld: cannot find -lcuda" bug Something isn't working
#11643 opened Dec 31, 2024 by Jimmy-L99
1 task done
3
ProTip! Type g i on any issue or pull request to go back to the issue listing page.