Repositories list

• vllm

  Public
  A high-throughput and memory-efficient inference and serving engine for LLMs

  amdcudainference+ 15pytorchtransformerllamagptrocmmodel-servingtpu+ 8

  Python

  •

  Apache License 2.0

  •5k•33k•1.2k•451•Updated Jan 4, 2025Jan 4, 2025

• llm-compressor

  Public
  Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

  sparsitycompressionquantization

  Python

  •

  Apache License 2.0

  •70•831•29•40•Updated Jan 4, 2025Jan 4, 2025

• flash-attention

  Public
  Fast and memory-efficient exact attention

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •1.4k•36•0•6•Updated Dec 30, 2024Dec 30, 2024

• buildkite-ci

  Public
  HCL

  •17•8•0•3•Updated Dec 18, 2024Dec 18, 2024

• vllm-project.github.io

  Public
  HTML

  •

  MIT License

  •6•4•0•1•Updated Dec 15, 2024Dec 15, 2024

• vllm-blog-source

  Public
  SCSS

  •

  MIT License

  •2•1•0•0•Updated Dec 15, 2024Dec 15, 2024

• media-kit

  Public
  vLLM Logo Assets

  0•0•0•0•Updated Dec 12, 2024Dec 12, 2024

• vllm-nccl

  Public archive
  Manages vllm-nccl dependency

  Python

  •

  Apache License 2.0

  •3•16•2•0•Updated Jun 3, 2024Jun 3, 2024

• dashboard

  Public
  vLLM performance dashboard

  Python

  •

  Apache License 2.0

  •4•20•0•0•Updated Apr 26, 2024Apr 26, 2024