pagedattention

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

An Efficient and Versatile Inference Engine for Distributed LLM Serving

Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.

LLM inference kernels from scratch in Triton: KV cache, FlashAttention, PagedAttention, RMSNorm, RoPE, SwiGLU, and benchmarks.

A deterministic PyTorch autograd verification trap for catching silent KV-cache routing and block-alignment failures in vLLM and SGLang serving infrastructure.

Deadline-aware KV-cache scheduling for protecting decode-critical request-state under long-context LLM inference pressure.

From-scratch model of an LLM serving engine's systems core: paged KV-cache, continuous batching, preemption, and prefix caching — GPU-free, with reproducible benchmarks.

Empirical benchmarking harness mapping the boundary between compute saturation and PagedAttention KV-cache preemption cascades under streamed test-time compute scaling.

A minimal LLM inference engine implementing PagedAttention-style KV cache management on NanoGPT. Based on the "Efficient Memory Management for Large Language Model Serving with PagedAttention" paper.

Considerations about running AI Inference workloads on Kubernetes

French course summary of “Fast & Efficient LLM Inference with vLLM,” covering inference, quantization, LLM Compressor, PagedAttention, Continuous Batching, benchmarking, and evaluation.

LLM inference serving prototype with continuous batching, paged KV-cache management, benchmark telemetry, and C++/ONNX Runtime gRPC backend scaffold.

SGLang vs vLLM playground: FastAPI app benchmarks TTFT/throughput against your own servers, plus 2 notebooks