feat: experimental kernel porting tool by drbh · Pull Request #776 · huggingface/kernels

drbh · 2026-08-20T16:52:13Z

Warning

This is an experiment/draft. The recipe language, the op set, and the CLI are all subject to change without notice.

this pr adds kernel-port, an experimental tool for porting kernel repos into the kernel-builder layout by running a recipe instead of doing it by hand.

the idea is that porting an existing kernel to the kernel-builder is a set of deterministic rewrites/restructuring. a recipe is a list of those operations, which has the benefit of being able to be checked into the repo and provide a way to deterministically reproduce given a specific upstream commit.

one of the difficulties of maintaining a port is that the upstream repo can drift and currently keeping the port in sync is a manual process.

recipes are a way to codify the porting process, so that if the upstream repo drifts we can simply bump the commit in the recipe and re-run the porting process. if the upstream repo has changed in a way that breaks the porting process, the recipe will fail to apply and we can fix it before continuing.

a port is a port.kdl recipe plus an overlay dir of checked in files. same pins + same recipe gives a byte identical tree every time, and every op hard fails on drift rather than silently porting the wrong thing.

recipes are kdl 2.0 documents, one node per op:

recipe version=1

source repo="https://github.com/org/repo" commit="<40 char sha>"
prune keep="csrc/**,pkg/**"
move from="csrc" to="kernel"
move from="pkg" to="torch-ext/pkg"
relativize_imports in="torch-ext/pkg/**" package_root="torch-ext/pkg" changes=6
kernel name="pkg" backend="cuda" src="kernel/**"
manifest name="pkg" backends="cuda" torch_src="torch-ext/*.cpp"

there are 15 ops (source, vendor, prune, delete, move, overlay, replace, strip_suffix, expect, convert_import, remap_module, relativize_imports, ensure_init, kernel, manifest). python rewrites go through libcst so comments and formatting are preserved byte for byte. build.toml is always generated by the manifest op, never overlaid.

the pins are the whole point. count=, files= and changes= are literals that have to match exactly, so a new file upstream cannot be rewritten without someone looking at it:

error: recipe line 17: relativize_imports: expected exactly 3 change(s) but made 4 - upstream drifted; review the new rewrites and update changes=

you can try an op without a checkout at all, -e takes the recipe inline and --file path=content builds the input tree in memory

cargo run -p kernel-port -- -e 'relativize_imports in="pkg/**" package_root="pkg" changes=1' \
    --file 'pkg/__init__.py=from pkg.ops import hello'

[line   1] relativize_imports  rewrote 1 import(s) in 1 file(s)
M pkg/__init__.py

FILE: pkg/__init__.py
from .ops import hello

the readme has a cookbook with one runnable command per op, plus the full arg list and failure modes for each. every example in it was run and pasted, not written by hand.

***NEXT STEPS are to target a JIT and AOT kernel in the kernels-community and experiment using this tool

github-actions · 2026-08-21T02:06:30Z

Coverage report — `kernels/`

Measured on: Python 3.10 / Torch 2.13.0.
Other CI configurations are not included in this number.
Hardware-gated code paths (ROCm/XPU/NPU/Darwin/Windows) are excluded or unreachable on the Linux+CUDA runner.

Total coverage: 87.2% — threshold: 80% — ✅

Per-file breakdown

Name	Stmts	Miss	Cover	Missing
src/kernels/__init__.py	14	0	100%
src/kernels/_system.py	6	1	83%	10
src/kernels/_versions.py	130	14	89%	53, 59-60, 63-64, 102, 165-170, 199, 219
src/kernels/archs.py	56	1	98%	94
src/kernels/backends.py	213	62	71%	42, 46, 50-53, 70, 92, 110, 119, 123, 127-129, 150, 159, 163, 167-169, 190, 201, 203, 210-213, 226, 230, 234-254, 262, 285-305
src/kernels/compat.py	9	1	89%	5
src/kernels/deps.py	70	1	99%	56
src/kernels/hf_hub.py	63	2	97%	21, 23
src/kernels/importer.py	44	5	89%	80, 84, 87, 101-102
src/kernels/install.py	21	7	67%	76-100
src/kernels/layer/__init__.py	6	0	100%
src/kernels/layer/_interval_tree.py	103	4	96%	23, 52, 147, 150
src/kernels/layer/device.py	48	14	71%	42, 47-49, 91, 96-98, 101, 149, 152, 155-157
src/kernels/layer/func.py	85	6	93%	90, 115, 191, 311, 338, 368
src/kernels/layer/globals.py	5	0	100%
src/kernels/layer/kernelize.py	80	8	90%	258, 293, 301-302, 308, 312, 328-330
src/kernels/layer/layer.py	215	14	93%	182, 229, 256, 390, 470-471, 492, 500, 511, 540, 544, 557, 610, 640
src/kernels/layer/mode.py	14	0	100%
src/kernels/layer/repos.py	144	42	71%	27, 33, 36-43, 63-64, 70, 73-76, 90, 94, 103-104, 110, 113-116, 123-124, 130, 133-136, 143-144, 150, 153-156, 163-164, 170, 173-176, 257
src/kernels/load.py	71	2	97%	338, 378
src/kernels/locking.py	89	64	28%	35-83, 91-98, 102-125, 137, 152-159, 165-175, 179-186
src/kernels/python_deps.py	58	6	90%	59-60, 64-65, 101, 104
src/kernels/resolver.py	156	2	99%	220, 226
src/kernels/status.py	50	2	96%	25, 79
src/kernels/validate.py	88	5	94%	9, 100, 167, 190-191
src/kernels/variants.py	278	19	93%	65, 96, 117, 147, 256-257, 299-302, 304, 388-394, 400-406, 437-443, 455-461
src/kernels/verify.py	127	6	95%	46, 202-204, 318-319
TOTAL	2243	288	87%

Updated by the Test kernels workflow on commit 247bdfbde43ec5b9866263a2d1c89257952a187d.

sayakpaul · 2026-08-21T09:24:54Z

an overlay dir of checked in files

We might want to elaborate on what overlay means in this context.

recipe version=1

Is the manual specification for version inspired by how we do it in build.toml of kernels?

the pins are the whole point. count=, files= and changes= are literals that have to match exactly, so a new file upstream cannot be rewritten without someone looking at it:

How are those pins derived?

you can try an op without a checkout at all, -e takes the recipe inline and --file path=content builds the input tree in memory

Do the users have to specify changes= manually like that? Feels cumbersome no?

More comments

Yet another language 🥲
For the code snippet provided in the description, I think we would want to first showcase the high-level structure of the kernel and then it'd be much easier to understand how those ops relate to one another in the context of the builder layout.
I didn't find anything that registers the PyTorch ops as per the specifications from https://huggingface.co/docs/kernels/main/en/builder/writing-kernels#registering-torch-operators. Is that intentional? I think this should be an atomic op unless I am missing something.
Do we have the possibility that overlay directory can grow big? If so, wouldn't it be a problem to PR that upstream where we essentially want to just PR with the KDL file along with a lightweight overlay folder if needed?

drbh · 2026-08-21T17:20:04Z

@sayakpaul

We might want to elaborate on what overlay means in this context.

the docs on the readme may already fill this need https://github.com/huggingface/kernels/blob/b7062eed8821f6a1235680844ad4bbc4711d7a7c/kernel-port/README.md#overlay

copied for reference

Note

`overlay`

overlay from="<dir>"

Copy a directory of checked-in files over the workspace, overwriting what is there. from is relative to the recipe file. This is where the files with no upstream equivalent live: torch_binding.cpp, flake.nix, a CARD.md. Keep it small - anything derivable from upstream should be an op, not an overlay file, so that upstream drift is detected rather than papered over.

**the readme in the pr has a "cookbook" that explains what each operator in the recipe does and what args are optional, etc..

Is the manual specification for version inspired by how we do it in build.toml of kernels?

the version is manually specified when writing the recipe and is only used so if the port tool has any breaking changes we can bump in the future

How are those pins derived?

these are added by the users (either by manually counting) or more realisitcally through an iterative process. the tool is designed to be run iteratively making changes to the recipe and checking the output. the tool errors if the count=, files= and changes= does not match exactly, and reports the count of mutations the op would make. this ensure that every change is accounted for and tracked (and not silently applied when migrating to a newer commit)

example of how it errors

error: recipe line 45: expect: expected "torch-ext/einops/**" to match exactly 10 file(s), found 17 - the upstream file set drifted; update the port definition (torch-ext/einops/__init__.py, torch-ext/einops/_backends.py, torch-ext/einops/_torch_specific.py, torch-ext/einops/array_api.py, torch-ext/einops/einops.py, torch-ext/einops/experimental/__init__.py, torch-ext/einops/layers/__init__.py, torch-ext/einops/layers/_einmix.py, torch-ext/einops/layers/flax.py, torch-ext/einops/layers/keras.py, torch-ext/einops/layers/oneflow.py, torch-ext/einops/layers/paddle.py, torch-ext/einops/layers/tensorflow.py, torch-ext/einops/layers/torch.py, torch-ext/einops/packing.py, torch-ext/einops/parsing.py, torch-ext/einops/py.typed)

Do the users have to specify changes= manually like that? Feels cumbersome no?

the changes= argument is optional in all cases https://github.com/huggingface/kernels/blob/b7062eed8821f6a1235680844ad4bbc4711d7a7c/kernel-port/README.md#rewriting-python-imports, but are intentionally verbose since pins (like changes) are included to make the number of mutations explicit and record.

in the readme under conventions we include the following (copied for reference) https://github.com/huggingface/kernels/blob/b7062eed8821f6a1235680844ad4bbc4711d7a7c/kernel-port/README.md#cookbook

Note

Pins (count=, files=, changes=) are literal integers that must match exactly. They are the point: they turn "upstream changed" from a silent mis-port into a failed run.

so its recommended to include pins, however at the end of the day its fully optional

Yet another language 🥲

yea true, but at least its really tiny (only our 15 ops) and no control flow/etc so shouldn't be much effort to understand. I think the parts that can be confusing are concepts like pins but sufficient docs and good error messages should make this manageable

For the code snippet provided in the description, I think we would want to first showcase the high-level structure of the kernel and then it'd be much easier to understand how those ops relate to one another in the context of the builder layout.

added examples of a working port for einops for this reason in huggingface/kernels-community#1092 and how to update the pin in huggingface/kernels-community#1093

I didn't find anything that registers the PyTorch ops as per the specifications from https://huggingface.co/docs/kernels/main/en/builder/writing-kernels#registering-torch-operators. Is that intentional? I think this should be an atomic op unless I am missing something.

yes there are no recipe operations for torch ops specifically - this case is handled by the generic replace op, this way the exact string replacement for the ops is stored in the recipe (which makes them a bit longer), but the benefit of a generic op is that it works in all cases.

Do we have the possibility that overlay directory can grow big? If so, wouldn't it be a problem to PR that upstream where we essentially want to just PR with the KDL file along with a lightweight overlay folder if needed?

yes 100%, overlays will be big for kernels that are a combination of modified upstream and custom implementations we have in kernels-community. basically overlay is a way to inject files that follow kernel-builder conventions (no mutation needed). so in the case we are opening a pr for an external kernel, there is not really a case where we need to inject files (only the flake.nix) but if they dont already have a flake they can just add that to their root and we'll pull it in in the recipe.

tldr; in the pr case we'll just add two files port.kdl and flake.nix. the overlay is mostly a concept that caters to our situation where we have vendored kernels with non upstream additions.

sayakpaul · 2026-08-25T08:23:00Z

these are added by the users (either by manually counting) or more realisitcally through an iterative process. the tool is designed to be run iteratively making changes to the recipe and checking the output. the tool errors if the count=, files= and changes= does not match exactly, and reports the count of mutations the op would make. this ensure that every change is accounted for and tracked (and not silently applied when migrating to a newer commit)

Very good idea, IMO. But would we have a simple way to quickly test the correctness of its implementation? Not a blocker but I think we should strive for simplicity here.

so its recommended to include pins, however at the end of the day its fully optional

Maybe we need to distinguish between required pins and optional pins? count and filesfeel like they should be the required ones.

yes there are no recipe operations for torch ops specifically - this case is handled by the generic replace op, this way the exact string replacement for the ops is stored in the recipe (which makes them a bit longer), but the benefit of a generic op is that it works in all cases.

Cool, that works for me!

basically overlay is a way to inject files that follow kernel-builder conventions (no mutation needed). so in the case we are opening a pr for an external kernel, there is not really a case where we need to inject files (only the flake.nix) but if they dont already have a flake they can just add that to their root and we'll pull it in in the recipe.

Not sure if I fully understood it. Why would it differ for an external repo from kernels-community? Is it just because of the top-level flake?

tldr; in the pr case we'll just add two files port.kdl and flake.nix. the overlay is mostly a concept that caters to our situation where we have vendored kernels with non upstream additions.

Oh okay. I was under the impression that without the overlay folder (and with just the KDL recipe), it wouldn't build according to our compliance. So, the scope of overlay seemed a bit confusing to me. Possible to provide a bit more elaboration?

sayakpaul · 2026-08-25T08:41:25Z

  # Local packages/hooks.
  kernel-builder = final.callPackage ./pkgs/kernel-builder { inherit builderProvenance; };

+  kernel-port = final.callPackage ./pkgs/kernel-port { };


Any reasoning behind making it a part of our nix-builder? For future CI?

sayakpaul

Thanks just left a bunch of comments. I think we are headed in a good direction. Once this takes a bit more shape, we could think about how we wire this in the CI, etc.

sayakpaul

Left some comments. Thanks for the further updates!

Apart from the comments, I think we should log the dirty status in the port provenance.

sayakpaul · 2026-09-02T04:27:27Z

@@ -0,0 +1,992 @@
+// Python edits go through libcst, so comments, quoting and layout outside the


What is the formatting situation here? Do we follow it from upstream? In some of the port PRs on kernels-community, we saw a significant amount of formatting-related changes.

the python rewrites go through libcst so anything the tool touches keeps upstream formatting and comments byte for byte. the formatting diffs in the port prs come from overlay files and replace ops, where the content is whatever the recipe says, not from the rewriter

Fine for now but essentially, formatting related changes should be very minimal IMO at least for existing kernels. Otherwise, it's just a lot of reviewing time that could have been spent elsewhere.

sayakpaul

Thanks a lot for the iterations. Just one comment regarding the tests.

sayakpaul · 2026-09-21T03:19:00Z

@@ -0,0 +1,740 @@
+use crate::{ops, python, recipe, workspace::Workspace};


I didn't go through this fully, but on a high level, I would expect to see the tests covering both AOT and JIT-flavored kernels. If already done, no need for further modifications. If not, then let's do that.

I think tests for atomic ops that we will have in the tool are good but lightweight e2e tests also go a long way (the way we use the RELU example kernel in tests, for example).

really good point e2e tests for the two cases is really useful! thanks

just updated to include an example for both AOT and JIT kernels. each example has a upstream, expected dir and a port.kdl, then in CI we run the porting tool on upstream and check that its identical to the valid kernel format in expected

Automated hardening of the workflow files flagged on #776. > [!WARNING] > **This narrows what the workflow can reach.** Job permissions were declared in `.github/workflows/rust.yaml`. Each job now gets only the scopes its steps were read to need — if one of them does something this could not see, it will fail on the next run. The table below says which step drove each scope. Targets `kernel-port-tool`. Files changed, and what changed them: - `.github/workflows/rust.yaml` — job permissions Fixed by this PR: - **MEDIUM** `excessive-permissions` (zizmor) — .github/workflows/rust.yaml:1 - **MEDIUM** `excessive-permissions` (zizmor) — .github/workflows/rust.yaml:15 - **MEDIUM** `excessive-permissions` (zizmor) — .github/workflows/rust.yaml:35 - **MEDIUM** `excessive-permissions` (zizmor) — .github/workflows/rust.yaml:64 **This does not fix everything.** 3 further finding(s) (3 critical) need a decision this bot should not make for you. They are in the security channel with their locations — deliberately not repeated here, since this repository may be public and they are not fixed yet. ### Permissions `.github/workflows/rust.yaml` | job | granted | why | |---|---|---| | `fmt` | `contents: read` | Only actions/checkout plus cargo fmt checks, so read access to the repository contents is all that is needed. | | `clippy` | `contents: read` | actions/checkout drives contents: read; actions/cache and cargo clippy use no token permissions. | | `test` | `contents: read` | actions/checkout drives contents: read; actions/cache and cargo test require no additional scopes. | Anything not listed above keeps the permissions it had. To measure a job this could not read, add [`GitHubSecurityLab/actions-permissions/monitor`](https://github.com/GitHubSecurityLab/actions-permissions) to it and run the workflow — it reports the minimum the run actually used. Pinning changes come from `pinact` and are mechanical. Any other change was generated by Claude — read it before merging.  Co-authored-by: hf-security-analysis[bot] <265538906+hf-security-analysis[bot]@users.noreply.github.com>

sayakpaul

One optional comment which could be addressed in a follow-up.

sayakpaul · 2026-09-24T03:17:39Z

+        run: ( cd kernel-port && cargo clippy -- -D warnings )

  test:
+    permissions:


Running the tests on kernel-porter for every rust-related changes doesn't sound good to me. Maybe we should have a separate workflow for the porter and trigger it on changes specific to the porter?

good point, I'll follow up with improved ci in the next porter changes. also lets see how long the tests take to run, if its very small it might be more simple to keep in the rust workflow (will check when merged)

drbh temporarily deployed to testpypi August 20, 2026 17:34 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi August 21, 2026 14:16 — with GitHub Actions Inactive

sayakpaul mentioned this pull request Aug 25, 2026

AST-based utility for checking ops registration and relative imports #729

Closed

sayakpaul mentioned this pull request Aug 25, 2026

feat: prefer recipe ported einops huggingface/kernels-community#1092

Open

sayakpaul reviewed Aug 25, 2026

View reviewed changes

drbh temporarily deployed to testpypi August 26, 2026 23:51 — with GitHub Actions Inactive

drbh marked this pull request as ready for review August 26, 2026 23:55

drbh force-pushed the kernel-port-tool branch from 11b0596 to c99b104 Compare August 26, 2026 23:55

drbh temporarily deployed to testpypi August 26, 2026 23:55 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi August 27, 2026 18:51 — with GitHub Actions Inactive

sayakpaul reviewed Sep 2, 2026

View reviewed changes

drbh temporarily deployed to testpypi September 2, 2026 15:50 — with GitHub Actions Inactive

drbh force-pushed the kernel-port-tool branch from 2bb25b6 to c29a675 Compare September 2, 2026 15:54

drbh temporarily deployed to testpypi September 2, 2026 15:55 — with GitHub Actions Inactive

drbh force-pushed the kernel-port-tool branch from c29a675 to c44f051 Compare September 14, 2026 20:07

sayakpaul reviewed Sep 21, 2026

View reviewed changes

hf-security-analysis Bot added a commit that referenced this pull request Sep 23, 2026

fix(ci): harden workflow files flagged on #776

1acc525

hf-security-analysis Bot mentioned this pull request Sep 23, 2026

fix(ci): harden GitHub Actions workflows (#776) #844

Merged

sayakpaul approved these changes Sep 24, 2026

View reviewed changes

drbh added 5 commits September 24, 2026 09:38

feat: experimental kernel porting tool

0e71f07

fix: update nix crate dirs

c70794a

feat: split ops into individual files

c7821a7

feat: nixify porting tool

77c60e5

fix: unhide port-provenance

7530957

drbh added 6 commits September 24, 2026 09:38

feat: add ensure import

635f41a

feat: add kernelize import

97cbb94

feat: add a memo module to cache and improve port speed

5009a3a

fix: address typos

01d3925

fix: include kernel-port in kernel test Docker image

ebc665a

feat: add snapshot test for AOT and JIT ports

247bdfb

drbh force-pushed the kernel-port-tool branch from a0875ff to 247bdfb Compare September 24, 2026 13:38

drbh merged commit b01af47 into main Sep 24, 2026
49 of 50 checks passed

		@@ -0,0 +1,992 @@
		// Python edits go through libcst, so comments, quoting and layout outside the

		@@ -0,0 +1,740 @@
		use crate::{ops, python, recipe, workspace::Workspace};

Conversation

drbh commented Aug 20, 2026

Uh oh!

github-actions Bot commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage report — kernels/

Uh oh!

sayakpaul commented Aug 21, 2026

More comments

Uh oh!

drbh commented Aug 21, 2026

overlay

Uh oh!

sayakpaul commented Aug 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Aug 21, 2026 •

edited

Loading

Coverage report — `kernels/`

`overlay`