Code

Semi structured v2#32

Code

Open

ilmarkov wants to merge 39 commits intomainneuralmagic/vllm:mainfrom semi_structured_v2neuralmagic/vllm:semi_structured_v2Copy head branch name to clipboard

+3,309-225

Commits

Commits on Nov 12, 2024

• [doc] fix location of runllm widget (vllm-project#10266)

  Show description for 96ae0ea
  youkaichao

  authored
  96ae0ea

• [doc] improve debugging doc (vllm-project#10270)

  Show description for 1808145
  youkaichao

  authored
  1808145

• Revert "[ci][build] limit cmake version" (vllm-project#10271)

  youkaichao

  authored
  377b74f

Commits on Nov 13, 2024

• [V1] Fix CI tests on V1 engine (vllm-project#10272)

  Show description for 112fa0b
  WoosukKwon

  authored
  112fa0b

• [core][distributed] use tcp store directly (vllm-project#10275)

  Show description for 0d4ea3f
  youkaichao

  authored
  0d4ea3f

• [V1] Support VLMs with fine-grained scheduling (vllm-project#9871)

  Show description for bbd3e86
  

  WoosukKwon

  and

  ywang96

  authored
  bbd3e86

• Bump to compressed-tensors v0.8.0 (vllm-project#10279)

  Show description for 56a955e
  dsikka

  authored
  56a955e

• [Doc] Fix typo in arg_utils.py (vllm-project#10264)

  Show description for 032fcf1
  xyang16

  authored
  032fcf1

• [Model] Add support for Qwen2-VL video embeddings input & multiple image embeddings input with varied resolutions (vllm-project#10221)

  Show description for 3945c82
  imkero

  authored
  3945c82

• [Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (vllm-project#9944)

  Show description for 1b886aa
  

  FurtherAI

  and

  FurtherAI

  authored
  1b886aa

• [Core] Flashinfer - Remove advance step size restriction (vllm-project#10282)

  pavanimajety

  authored
  b6dde33

• [Model][LoRA]LoRA support added for idefics3 (vllm-project#10281)

  Show description for d909acf
  B-201

  authored
  d909acf

• [V1] Add missing tokenizer options for Detokenizer (vllm-project#10288)

  Show description for bb7991a
  ywang96

  authored
  bb7991a

• semi_structured for fp16 and bf16 and int8

  ilmarkov

  committed
  78eea7b

• Fix A100 int8 tests

  ilmarkov

  committed
  331e9c5

• Add fp8 cusparseLt

  ilmarkov

  committed
  381a6b4

• wip

  ilmarkov

  committed
  b146a79

• Fix signatures

  ilmarkov

  committed
  0ac01cc

• Fix compilation and tests

  ilmarkov

  committed
  7472af2

• Update for older platforms

  ilmarkov

  committed
  3fe8bd4

• Add benchmarks

  ilmarkov

  committed
  e736027

• Fix typo

  ilmarkov

  committed
  ae66f77

• Added scaled_mm for fp8.

  Show description for 59ee24d
  ilmarkov

  committed
  59ee24d

• Add docstrings

  ilmarkov

  committed
  3367704

• Update for torch 2.5

  ilmarkov

  committed
  368beec

• Add handling contiguous dense input for int8 and fp8

  ilmarkov

  committed
  5be53f3

• Add fp8 cusparseLt

  ilmarkov

  committed
  f9546a8

• Fix compilation and tests

  ilmarkov

  committed
  2187236

• Add caching of cusparseLT meta

  ilmarkov

  committed
  f45a83b

• Cached cusparseLt

  ilmarkov

  committed
  b1aaea5

• Fix destroy function

  ilmarkov

  committed
  c36401c

• Prepare for reproduce

  ilmarkov

  committed
  9f6a469

• Fix cusparseLt caching

  ilmarkov

  committed
  2e56de9

• Make cached version default function

  ilmarkov

  committed
  9ad83cb

• Fixes and polishing after rebase

  ilmarkov

  committed
  1f6a05b

• Add output_dtype option, fix non-padded inputs case

  ilmarkov

  committed
  3a2c258

• Fix and polish

  ilmarkov

  committed
  31cf482

• Formatting

  ilmarkov

  committed
  68512d4

Commits on Nov 15, 2024

• Minor test and benchmarks updates

  ilmarkov

  committed
  72d6cd3