Skip to content

README.md

Latest commit

 

History

History
56 lines (42 loc) · 2.07 KB

README.md

File metadata and controls

56 lines (42 loc) · 2.07 KB
Raw

TileSparse

Arithmetic-Intensity-Aware Sparse Attention for Compute-Bound LLM Decoding

Chao Wang, Pengfei Zuo, Zhangyu Chen, Qihui Zhou, Tsung-Yi Ho, Ming-Chang Yang · ICML 2026

This is the reference implementation for the ICML 2026 paper of the same name. It is a research fork of vLLM that adds an arithmetic-intensity-aware sparse-attention backend for Multi-head Latent Attention (MLA) models such as DeepSeek-V2/V3.

Repository layout

All code lives in vllm/, a self-contained vLLM tree — the build, the TileSparse backend, and the evaluation harness all run from inside it. The git history is deliberately shallow so the full contribution is one diff:

  • the first commit is a pristine snapshot of vLLM upstream main at the fork point (commit 72506c98349d6bcd32b4e33eec7b5513453c1502);
  • the remaining commits layer the TileSparse backend, evaluation harness, and documentation on top.

Diffing against the base snapshot shows the complete increment.

Quick start

cd vllm
bash scripts/environment/init_venv.sh   # uv venv + precompiled vLLM + eval deps
source .venv/bin/activate
bash scripts/evaluations/longbench_dsv3.sh   # DeepSeek-V3-0324 sanity check (8×141 GB)

See vllm/README.md for the method overview, configuration reference, and the relationship to upstream vLLM, and vllm/evaluations/README.md for the full directory-to-figure reproduction map.

Citation

@inproceedings{wang2026tilesparse,
  title     = {TileSparse: Arithmetic-Intensity-Aware Sparse Attention for Compute-Bound {LLM} Decoding},
  author    = {Chao Wang and Pengfei Zuo and Zhangyu Chen and Qihui Zhou and Tsung-Yi Ho and Ming-Chang Yang},
  booktitle = {Forty-third International Conference on Machine Learning},
  year      = {2026},
  url       = {https://openreview.net/forum?id=lK2AzLDJal},
}

License

Apache-2.0, inherited from vLLM. Vendored evaluation code retains its original license (RULER: Apache-2.0; LongBench: MIT).