Skip to content
Navigation Menu
Sign in
Appearance settings
Platform
AI CODE CREATION
GitHub Copilot
Write better code with AI
GitHub Copilot app
Direct agents from issue to merge
MCP Registry
Integrate external tools
DEVELOPER WORKFLOWS
Actions
Automate any workflow
Codespaces
Instant dev environments
Issues
Plan and track work
Code Review
Manage code changes
Code Quality
Enforce quality at merge
APPLICATION SECURITY
GitHub Advanced Security
Find and fix vulnerabilities
Code security
Secure your code as you build
Secret protection
Stop leaks before they start
EXPLORE
Why GitHub
Documentation
Blog
Changelog
Marketplace
View all features
Solutions
BY COMPANY SIZE
Enterprises
Small and medium teams
Startups
Nonprofits
BY USE CASE
App Modernization
DevSecOps
DevOps
CI/CD
View all use cases
BY INDUSTRY
Healthcare
Financial services
Manufacturing
Government
View all industries
View all solutions
Resources
EXPLORE BY TOPIC
AI
Software Development
DevOps
Security
View all topics
EXPLORE BY TYPE
Customer stories
Events & webinars
Ebooks & reports
Business insights
GitHub Skills
SUPPORT & SERVICES
Documentation
Customer support
Community forum
Trust center
Partners
View all resources
Open Source
COMMUNITY
GitHub Sponsors
Fund open source developers
PROGRAMS
Security Lab
Maintainer Community
GitHub Stars
Archive Program
REPOSITORIES
Topics
Trending
Collections
Enterprise
ENTERPRISE SOLUTIONS
Enterprise platform
AI-powered developer platform
AVAILABLE ADD-ONS
GitHub Advanced Security
Enterprise-grade security features
Copilot for Business
Enterprise-grade AI features
Premium Support
Enterprise-grade 24/7 support
Pricing
Search
/
Sign in
Sign up
Appearance settings
You signed in with another tab or window.
Reload
to refresh your session.
You signed out in another tab or window.
Reload
to refresh your session.
You switched accounts on another tab or window.
Reload
to refresh your session.
Dismiss alert
{{ message }}
FlashML-org
/
FreeToken
Public
Notifications
You must be signed in to change notification settings
Fork
1.3k
Star
12.8k
Code
Issues
179
Pull requests
129
Actions
Projects
Security and quality
0
Insights
Additional navigation options
Code
Issues
Pull requests
Actions
Projects
Security and quality
Insights
feat(kvcache): add nvfp4 kv quantization
- #408
#408
Open
ArqAlice
wants to merge 20 commits into
FlashML-org:main
FlashML-org/FreeToken:main
from
ArqAlice:feat/nvfp4-kv-quantization
ArqAlice/FreeToken:feat/nvfp4-kv-quantization
Copy head branch name to clipboard
Conversation
Commits
20
(20)
Checks
Files changed
Open
feat(kvcache): add nvfp4 kv quantization
#408
ArqAlice
wants to merge 20 commits into
FlashML-org:main
FlashML-org/FreeToken:main
from
ArqAlice:feat/nvfp4-kv-quantization
ArqAlice/FreeToken:feat/nvfp4-kv-quantization
Copy head branch name to clipboard
Commits
Commits on Sep 2, 2026
feat(kvcache): store the KV cache as fp8 e4m3 codes (--kv-cache-dtype fp8)
Show description for 03fb043
ArqAlice
committed
03fb043
View commit details
Copy full SHA for 03fb043
Browse repository at this point
Commits on Sep 4, 2026
fix(kernels): give the V tensor its own row pitch in the fp8 KV store
Show description for 811ccee
MT-z
and
claude
committed
811ccee
View commit details
Copy full SHA for 811ccee
Browse repository at this point
Merge pull request #1 from MT-z/fix/kv-fp8-vstore-pitch
Show description for 5b9efc5
ArqAlice
authored
5b9efc5
View commit details
Copy full SHA for 5b9efc5
Browse repository at this point
Merge branch 'FlashML-org:main' into feat/fp8-quantization
ArqAlice
authored
3e5bbdd
View commit details
Copy full SHA for 3e5bbdd
Browse repository at this point
Commits on Sep 5, 2026
test(kvcache): size the fp8 slot round-trip to the rows it indexes
Show description for 7f9a05a
MT-z
committed
7f9a05a
View commit details
Copy full SHA for 7f9a05a
Browse repository at this point
test(kernels): put the scale-one encoder's V tensor on the device
Show description for dabe93c
MT-z
committed
dabe93c
View commit details
Copy full SHA for dabe93c
Browse repository at this point
test(kvcache): give the layer-ids remap test a model deep enough for its ids
Show description for 5febeee
MT-z
committed
5febeee
View commit details
Copy full SHA for 5febeee
Browse repository at this point
test(kernels): give the triton-attention doubles the scale accessors the backend now reads
Show description for 0820ff4
MT-z
committed
0820ff4
View commit details
Copy full SHA for 0820ff4
Browse repository at this point
Commits on Sep 6, 2026
perf(kernel): apply the fp8 KV dequant scale after the dot, not to the tile
Show description for 05861fb
naerymdan
and
claude
committed
05861fb
View commit details
Copy full SHA for 05861fb
Browse repository at this point
perf(kernel): size the extend tile from the KV cache element size
Show description for 73ca76c
naerymdan
and
claude
committed
73ca76c
View commit details
Copy full SHA for 73ca76c
Browse repository at this point
Commits on Sep 7, 2026
Merge pull request #2 from MT-z/fix/fp8-tests-single-process
Show description for 33872fd
ArqAlice
authored
33872fd
View commit details
Copy full SHA for 33872fd
Browse repository at this point
feat(kvcache): add nvfp4 kv quantization
ArqAlice
committed
2f554c9
View commit details
Copy full SHA for 2f554c9
Browse repository at this point
test(kernels): stabilize fp8 extend attention regression
ArqAlice
committed
ca3675e
View commit details
Copy full SHA for ca3675e
Browse repository at this point
Merge commit 'ca3675ecde8d53385ddb32cf4d611c7230d0b897' into feat/nvfp4-kv-quantization
ArqAlice
committed
eae141d
View commit details
Copy full SHA for eae141d
Browse repository at this point
Merge pull request #3 from naerymdan/perf/kv-fp8-read-path
ArqAlice
authored
cfe82df
View commit details
Copy full SHA for cfe82df
Browse repository at this point
Merge commit 'cfe82df02b1d8999d86609aa44bf600ade2665d6' into feat/nvfp4-kv-quantization
ArqAlice
committed
3b84b80
View commit details
Copy full SHA for 3b84b80
Browse repository at this point
feat(kvcache): add fp8 support for dsa kv cache
ArqAlice
committed
9b103b0
View commit details
Copy full SHA for 9b103b0
Browse repository at this point
Merge commit '9b103b04f9c8a1544dbe857013dd129170defbd7' into feat/nvfp4-kv-quantization
ArqAlice
committed
7f4d788
View commit details
Copy full SHA for 7f4d788
Browse repository at this point
Commits on Sep 8, 2026
feat(kvcache): support nvfp4 latent dsa kv
ArqAlice
committed
04d4621
View commit details
Copy full SHA for 04d4621
Browse repository at this point
Commits on Sep 13, 2026
Merge branch 'FlashML-org:main' into feat/nvfp4-kv-quantization
ArqAlice
authored
1c7bb42
View commit details
Copy full SHA for 1c7bb42
Browse repository at this point
You can’t perform that action at this time.
