Code

[DRAFT] use cutlass for 24#33

Code

Draft

rahul-tuli wants to merge 53 commits intomainneuralmagic/vllm:mainfrom rahul-cleanupneuralmagic/vllm:rahul-cleanupCopy head branch name to clipboard

+6,347-615

Commits

Commits on Oct 22, 2024

• Add cutlass 2:4 infrastructure

  Faraz9877

  committed
  5d51361

Commits on Oct 28, 2024

• Update with test code

  Faraz9877

  committed
  17f5b96

Commits on Oct 30, 2024

• Clean up a bit; both fp8 and int8 working

  Faraz9877

  committed
  471a03c

• Add fp16 and bf16 support to sparse cutlass mm

  Faraz9877

  committed
  0b332fb

Commits on Nov 5, 2024

• semi_structured for fp16 and bf16 and int8

  ilmarkov

  committed
  da31648

• Fix A100 int8 tests

  ilmarkov

  committed
  e655f94

• Add fp8 cusparseLt

  ilmarkov

  committed
  5fc3c1c

• wip

  ilmarkov

  committed
  9cf36d6

• Fix signatures

  ilmarkov

  committed
  ad09e79

• Fix compilation and tests

  ilmarkov

  committed
  e75eabc

• Update for older platforms

  ilmarkov

  committed
  0306390

• Add benchmarks

  ilmarkov

  committed
  1021acb

• Fix typo

  ilmarkov

  committed
  19ce358

• Added scaled_mm for fp8.

  Show description for 959408c
  ilmarkov

  committed
  959408c

• Add docstrings

  ilmarkov

  committed
  117b87b

• Update for torch 2.5

  ilmarkov

  committed
  2c7e68e

• Add handling contiguous dense input for int8 and fp8

  ilmarkov

  committed
  922f4f8

• Add fp8 cusparseLt

  ilmarkov

  committed
  beca038

• Fix compilation and tests

  ilmarkov

  committed
  5d9cd25

• Add caching of cusparseLT meta

  ilmarkov

  committed
  39ad9d4

• Cached cusparseLt

  ilmarkov

  committed
  520eb62

• Fix destroy function

  ilmarkov

  committed
  20956e6

• Prepare for reproduce

  ilmarkov

  committed
  87c8088

• Fix cusparseLt caching

  ilmarkov

  committed
  4ea58b1

• Make cached version default function

  ilmarkov

  committed
  f0551ef

Commits on Nov 6, 2024

• Fixes and polishing after rebase

  ilmarkov

  committed
  d7476e8

• add sparse 2:4 weight loading suport

  

  dsikka

  authored and

  rahul-tuli

  committed
  681ea5e

• Some more changes!

  rahul-tuli

  committed
  ecf878f

• Cleanup

  rahul-tuli

  committed
  80952dc

• get uncompressed to work; update gemm to use contiguous; use alex's utils instead of our decompressor

  

  dsikka

  authored and

  rahul-tuli

  committed
  8462c9d

• patch

  

  dsikka

  authored and

  rahul-tuli

  committed
  0a3e506

• use our decompressor

  

  dsikka

  authored and

  rahul-tuli

  committed
  2e28972

• Some more work

  rahul-tuli

  committed
  28f0abb

Commits on Nov 7, 2024

• Use new scaled_T function

  rahul-tuli

  committed
  c7a97a8

Commits on Nov 8, 2024

• Add multiprocessing for kernel sweep benchmarking

  Faraz9877

  committed
  ccadad0

• Add multi-GPU

  Faraz9877

  committed
  807737c

Commits on Nov 14, 2024

• Add cutlass_scaled_sparse_mm op

  Faraz9877

  committed
  04c19a5

• Clean up

  Faraz9877

  committed
  2a85c5a

• Update code

  Faraz9877

  committed
  1b381c9

• Update code

  Faraz9877

  committed
  4e31076

• Clean up the benchmarking

  Faraz9877

  committed
  13fccf4

• Clean up the cutlass benchmarking

  Faraz9877

  committed
  b345cc8

• Fix cmake errors

  Faraz9877

  committed
  2d03e1d

• Fix the cmake TAG

  Faraz9877

  committed
  e9439cc

Commits on Nov 15, 2024

• Merge branch 'buildable' into rahul-quant-merged-rs

  robertgshaw2-neuralmagic

  committed
  4ba7c0f

• update

  robertgshaw2-neuralmagic

  committed
  f74ef37

• fixed

  robertgshaw2-neuralmagic

  committed
  f5bc9eb

• updated

  robertgshaw2-neuralmagic

  committed
  1316076

• updated

  robertgshaw2-neuralmagic

  committed
  4d2b12c

• updated, calling things properly

  robertgshaw2-neuralmagic

  committed
  fe30b53

• running end to end but not passing

  robertgshaw2-neuralmagic

  committed
  4c61b19

• updated

  robertgshaw2-neuralmagic

  committed
  86716f8

• Some cleanup

  rahul-tuli

  committed
  c796ac8