vLLM

All

13 repositories

vllm
Public
A high-throughput and memory-efficient inference and serving engine for LLMs
amd cuda inference pytorch transformer llama gpt rocm model-serving tpu
Python
•
Apache License 2.0
•5.8k•39k•1.3k•442•Updated Feb 20, 2025Feb 20, 2025
buildkite-ci
Public
HCL
•18•8•0•3•Updated Feb 20, 2025Feb 20, 2025
llm-compressor
Public
Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
sparsity compression quantization
Python
•
Apache License 2.0
•84•984•16•42•Updated Feb 20, 2025Feb 20, 2025
vllm-ascend
Public
Community maintained hardware plugin for vLLM on Ascend
inference transformer model-serving mlops ascend llm llmops llm-serving vllm
Python
•
Apache License 2.0
•27•92•22•15•Updated Feb 20, 2025Feb 20, 2025
production-stack
Public
Scale from single vLLM instance to distributed vLLM deployment without changing any application code.
Python
•
Apache License 2.0
•57•483•19•5•Updated Feb 20, 2025Feb 20, 2025
vllm-project.github.io
Public
SCSS
•
MIT License
•10•4•0•1•Updated Feb 19, 2025Feb 19, 2025
flash-attention
Public
Fast and memory-efficient exact attention
Python
•
BSD 3-Clause "New" or "Revised" License
•1.5k•44•0•10•Updated Feb 17, 2025Feb 17, 2025
vllm-project.github.io-static
Public archive
HTML
•
MIT License
•7•7•0•1•Updated Feb 7, 2025Feb 7, 2025
vllm-spyre
Public
Community maintained hardware plugin for vLLM on Spyre
Apache License 2.0
•0•2•0•0•Updated Jan 29, 2025Jan 29, 2025
vllm_allocator_adaptor
Public
An adaptor to allow Python allocator for PyTorch pluggable allocator
C++
•
Apache License 2.0
•1•2•0•0•Updated Jan 5, 2025Jan 5, 2025
media-kit
Public
vLLM Logo Assets
0•1•0•0•Updated Dec 12, 2024Dec 12, 2024
vllm-nccl
Public archive
Manages vllm-nccl dependency
Python
•
Apache License 2.0
•3•17•2•0•Updated Jun 3, 2024Jun 3, 2024
dashboard
Public
vLLM performance dashboard
Python
•
Apache License 2.0
•5•20•0•0•Updated Apr 26, 2024Apr 26, 2024