[KernelSwift算子优化] 程柯雷-Task02 Indexer-571900729 - #189
Open
Lfan-ke wants to merge 3 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
作品说明:沐曦 C500 上用 Clike 重写 KernelSwift 赛道二 Task02 Indexer,文件内同时含与参考一致的
Model与优化实现ModelNew,forward内通过 python 调用自实现算子。优化方案:把 einsum 之后的四段(relu×weights→sum、掩码、topk(128)、二次掩码,合计占参考实现的 91%)折成一个 kernel——一个 block 一行、只读因果掩码放行的那半数据、行内先按 128 一段各自排再树形归并只留 top-128;rotary 另写一个原地 kernel,一线程搬一个 uint4。
★本次更新:把 reduce+topk 的 11 次 kernel 启动合并成 1 次。原先按
valid的 2 的幂把行分段、shared 按段申请,代价是 S=2600 时 P 取遍 1..1024 要启动 11 次,而这张卡一次 kernel 启动就要 15 µs。但 kernel 内部本来就自己按valid重算 P,分段只决定 shared 大小与线程数:shared 给大了无害,线程数也不改变结果(所有循环都按threadIdx跨步,同步次数与blockDim无关),所以合成一次启动是逐位等价的。整题 2.151 → 1.968 ms,只测 reduce+topk 这一个 kernel 是 1.166 → 0.981 ms,交替 A/B 六对全胜。省下的不止 10 次启动开销:2 万个块的代价相差极大(s 小的行几乎不干活、s 大的要排 1024),合成一个 grid 后调度器能把轻重块混着填满,而分段启动每一段都是一道屏障。性能结果(MetaX C500 / MACA 3.7.0.38 / torch 2.8.0+metax3.7.0.7,官方
benchmarks/ks/auto_bench.py默认参数,同一次独占会话连跑五次,均 PASS accuracy):正确性:输出是 int64 索引,harness 用
torch.equal逐位比对、无容差,五次均 PASS。本作品为原创。