Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs
-
Updated
Jul 27, 2026 - HTML
#
roofline
Here are 33 public repositories matching this topic...
Modular C++ Toolkit for Performance Analysis and Logging. Profiling API and Tools for C, C++, CUDA, Fortran, and Python. The C++ template API is essentially a framework to creating tools: it is designed to provide a unifying interface for recording various performance measurements alongside data logging and interfaces to other tools.
python c performance cplusplus cross-platform cpp analysis mpi cuda hardware-counters performance-measurement gotcha modular-design cross-language papi roofline cupti memory-measurements instrumentation-api
-
Updated
Jul 31, 2024 - C++
Analyze LLM inference: FLOPs, memory, Roofline model. Supports GQA, MoE, MLA, RoPE, SwiGLU. 19 models × 20+ hardware platforms.
-
Updated
Apr 16, 2026 - Python
Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.
machine-learning deep-learning gpu cuda transformers self-hosting quantization roofline mlops inference-optimization kv-cache llm llmops llm-serving vllm llm-inference flash-attention speculative-decoding pagedattention sglang
-
Updated
Aug 11, 2026 - Python
Interactive theoretical Kimi-K3 inference roofline calculator for H200, B300, and GB300
-
Updated
Aug 11, 2026 - Python
Hand-written CUDA linear algebra library. A GEMM driven from a naive baseline to a compute-bound tensor-core kernel at ~90% of cuBLAS on an RTX 5070, profiled with Nsight Compute, plus GEMV, TRSM, CSR SpMV, cuSOLVER, a measured roofline, and NVML telemetry.
benchmarking hpc linear-algebra cuda cublas high-performance-computing cuda-kernels gpu-computing gemm ptx roofline tensor-cores blackwell nsight-compute wmma
-
Updated
Aug 12, 2026 - C++
Interactive 3D visualization of dense decoder-only LLM inference. Companion to the AI Inference Engineer 2026 course.
-
Updated
Jun 15, 2026 - TypeScript
Interactive macOS GPU benchmark suite for Apple Silicon - 25 Metal compute benchmarks (compute, memory, ML/tensor, ray tracing) with a full measurement harness and empirical roofline.
-
Updated
Jul 9, 2026 - Swift
Double-precision dense GEMM across sequential, OpenMP, MPI and CUDA on an i9-12900K, Tesla T4 and Tesla P100. The T4 loses to the CPU in FP64 (Turing runs it at 1:32); a P100 at 1:2 explains why. Raw data, Intel Advisor and Nsight profiles, notebooks and a 33-page report.
benchmark hpc gpu openmp mpi parallel-computing cuda high-performance-computing gemm roofline intel-advisor nsight-compute
-
Updated
Jul 15, 2026 - HTML
Executable validation harness for the OSTIR cache-resident quantized CPU inference thesis: grouped/hierarchical quantizers, hardware-counter instrumentation, and the S(h) residency-law fit, with experiments E1-E7 run against the monograph's own pass/fail gates.
performance-engineering benchmarking reproducible-research quantization memory-hierarchy roofline cpu-inference llm-inference
-
Updated
Jul 27, 2026 - Python
Three hand-written Triton kernels for LLM inference (fused RMSNorm plus residual, online softmax, INT4 g128 GEMV) benchmarked on NVIDIA Blackwell against PyTorch eager and torch.compile, with every raw CUDA-event sample, measured device ceiling, and Nsight Compute report committed and CI-verified.
portfolio performance-engineering benchmarking gpu cuda pytorch triton gpu-kernels roofline nsight-compute gpu-profiling kernel-optimization int4-quantization
-
Updated
Aug 12, 2026 - Python
Where does the next watt go? Choosing an AI-accelerator budget allocation when the workload mix is unknown -- max-expected vs minimax-regret over the whole forecast space. Normalized model, stdlib only.
computer-architecture performance-modeling roofline hardware-software-codesign ai-accelerator decision-under-uncertainty
-
Updated
Jul 25, 2026 - Python
GPU performance optimization labs covering Roofline Analysis, LLM Decode Optimization, and CUDA Graphs using PyTorch
-
Updated
Jul 30, 2026 - Jupyter Notebook
Pipeline-aware Roofline & Inference Sweep Model — wall-clock + TCO prediction for NPU architecture exploration, calibrated on Ascend 910B4 via msprof. Other architectures are under consideration.
-
Updated
Jul 1, 2026 - Python
Measurement-driven study of LLM training (FSDP2) and serving (vLLM) on 8×A100 — 16 reproducible incident write-ups that predict the bound, control the confound, and report the negative.
performance-engineering gpu pytorch distributed-training roofline mlops a100 kv-cache llm vllm fsdp llm-inference
-
Updated
Jul 23, 2026 - Python
AllReduce/AllGather scaling in ASTRA-sim across torus vs switch topologies on the analytical + ns-3 backends — latency-bound vs bandwidth-bound over message size and node count. Reproducible Docker/Chakra harness + write-up.
hpc torus ns-3 allreduce roofline network-simulation distributed-ml collective-communication ml-systems astra-sim allgather gpu-interconnect
-
Updated
Jun 10, 2026 - Python
A local research assistant on a hardware aware inference engine. Places work across CPU, integrated GPU and NPU from measured hardware characteristics, on Intel Lunar Lake, with nothing leaving the machine.
-
Updated
Aug 10, 2026 - Python
PyTorch GPU inference performance exercises: roofline analysis, decode-loop profiling, KV-cache optimization, torch.compile, and CUDA graphs.
performance gpu cuda inference pytorch profiling roofline kv-cache llm-inference h100 torch-compile cuda-graphs
-
Updated
Jun 26, 2026 - Jupyter Notebook
Improve this page
Add a description, image, and links to the roofline topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the roofline topic, visit your repo's landing page and select "manage topics."