scripts: add export-kernel-tree.sh — render the carried series as a Linux-Kernel_MiSTer-style tree by mcfbytes · Pull Request #26 · mcfbytes/Buildroot_MiSTer

mcfbytes · 2026-07-16T15:27:27Z

Stacked on #25 (base is fix/kernel-patch-am-headers, not master). The export replays the series with git am, so it genuinely cannot work without #25's fix. GitHub will retarget this to master once #25 merges.

What

scripts/export-kernel-tree.sh renders the carried kernel series as a Linux-Kernel_MiSTer-style git tree: a pristine hash-verified tarball as one base commit, one commit per patch with authorship preserved, an in-tree defconfig, and an EXPORT.md.

$ scripts/export-kernel-tree.sh --output /tmp/export --fork-sync f0fb626ac

RESULT: PASS — exported Linux 6.18.38 + 31 patches
  branch   MiSTer-v6.18
  tag      mister-6.18.38
  commits  34 (1 base + 31 patches + defconfig + EXPORT.md)

Why this is even possible

The two formats are the same model. Linux-Kernel_MiSTer is a squashed tarball commit with an ordered series replayed on top; we are a pinned version + hash with an ordered series. The only difference is whether the base is stored as a hash or as 283MB of blobs.

That's verified, not assumed: that repo's v5.15.1 base is byte-for-byte pristine kernel.org v5.15.1. Its only delta is 11 absent files — all tools//Documentation//.kunitconfig dotfiles lost to .gitignore during a git add after extraction (selftests/arm64/tags/ dies to the kernel's tags ctags pattern). Zero content changes, zero additions. No MiSTer change hides in the base — which also means commit-level reconciliation is complete by construction.

Verified, not asserted

Claim	How it was checked
Deterministic	Two runs → identical HEAD SHAs. Regeneration after no change is a no-op, not a force-push of fresh SHAs.
Fails closed	Bumped the version with the tarball cached but no hash entry → refuses, exit 1.
Builds standalone	`make ARCH=arm MiSTer_defconfig` → `CONFIG_FB_MISTER=y`, `CONFIG_SND_MISTER_AUDIO=y`.
Cannot drift from the image	That `.config` is byte-identical to Buildroot's own `cp linux.config .config && make olddefconfig`.
Applies cleanly	31/31 on `master`; 32/32 including diag's `0038`.

Determinism comes from git am --committer-date-is-author-date plus a base date taken from the extracted Makefile's mtime — kernel.org tarballs come from git archive, so every file carries the tag's commit time, stable across machines unlike download time.

The 6.18.39 path

Edit two lines — BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE and linux.hash (from kernel.org's signed manifest) — and rerun. Nothing is hardcoded; the script reads the defconfig.

For comparison: the fork last took an upstream bump on 2021-11-07, onto 5.15.1. SUBLEVEL = 1 at HEAD today — 210 later 5.15.y point releases, none taken.

Two decisions worth reviewing

The defconfig is minimized, not expanded. The fork's arch/arm/configs/MiSTer_defconfig is a 4247-line full .config that bakes in CONFIG_CC_VERSION_TEXT — one machine's gcc build string — in a file whose whole purpose is portability. Ours is the kernel's own 495-line minimized form. Proven equivalent above.

The branch is an orphan. Attaching it to the 5.15 history requires a merge whose tree ignores its first parent, which makes git log list ~113 commits whose changes are absent from the tree — a reader sees "xone: update driver" and concludes xone is present, when it's a Buildroot package now. A log that lists absent changes is worse than an absent ancestor. MiSTer-v5.15 stays untouched and immutable; disposition per fork commit lives in MISTER-KERNEL-PATCH-RECON.md, which cites the superseding vanilla commit — something no git command can produce, since across this much context drift git patch-id matches nothing.

Honest cost: the two branches share no ancestor, so GitHub's compare view between them is meaningless.

Scope

The script never touches a remote. Publishing is an explicit fetch-into-a-fork plus push, spelled out in the generated EXPORT.md. No CI wiring here — regeneration is manual until we decide whether we publish this at all.

🤖 Generated with Claude Code

Copilot

Pull request overview

Adds a new helper script to materialize this repo’s “pinned upstream kernel tarball + ordered patch series” model into a standalone Linux-Kernel_MiSTer-style git tree (base tarball commit + one commit per patch + in-tree defconfig + EXPORT.md + tag), suitable for publishing from a fork without becoming a second source of truth.

Changes:

Introduces scripts/export-kernel-tree.sh to export a fresh kernel git repo from the pinned Buildroot defconfig inputs.
Verifies the kernel tarball against board/mister/de10nano/patches/linux/linux.hash before extraction/commit.
Replays the carried series with git am and adds arch/arm/configs/MiSTer_defconfig plus a generated EXPORT.md and tag.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

MiSTer-devel/Linux-Kernel_MiSTer keeps the kernel as a materialized git tree: a squashed tarball commit (`v5.15.1`) with MiSTer commits replayed on top. We keep it as {pinned version + hash} + an ordered patch series. Those are the same model — tarball base plus ordered series — differing only in whether the base is stored as a hash or as 283MB of blobs. (Verified: that repo's `v5.15.1` base is byte-for-byte pristine kernel.org v5.15.1. Its only delta is 11 absent files, all `tools/`/`Documentation/` dotfiles lost to `.gitignore` during a `git add` after extraction — no MiSTer change hides in the base.) So the two formats are convertible, and this renders one into the other: a pristine hash-verified tarball as one base commit, then one commit per carried patch with authorship preserved, then an in-tree defconfig and an EXPORT.md. It is a build output, not a second source of truth. Edits belong in the patch series; this regenerates from it. Design notes, all verified rather than assumed: - Deterministic. `git am --committer-date-is-author-date` plus a base-commit date taken from the extracted Makefile's mtime (kernel.org tarballs come from `git archive`, so every file carries the tag's commit time — stable across machines, unlike download time). Two runs produce identical SHAs, so regenerating after no change is a no-op rather than a force-push of fresh SHAs. This is what makes a 6.18.39 bump cheap: edit the version and the hash, rerun. - Fails closed. The tarball is checked against linux.hash and refuses to proceed without a matching entry — verified by bumping the version with the tarball cached but no hash present. - Ships arch/arm/configs/MiSTer_defconfig, the path the fork already uses, so the tree builds without Buildroot (`make ARCH=arm MiSTer_defconfig && make zImage`) — the thing `make linux` inside Buildroot cannot hand anyone. Verified that `make ARCH=arm MiSTer_defconfig` yields a .config byte-identical to Buildroot's own `cp linux.config .config && make olddefconfig`, so the exported tree cannot build a different kernel than the image. Written in the kernel's minimized form, unlike the fork's 4247-line expanded .config, which bakes CONFIG_CC_VERSION_TEXT — one machine's gcc build string — into a file whose purpose is portability. - Emits an ORPHAN branch. Attaching it to the 5.15 history needs a merge whose tree ignores its first parent, which makes `git log` list ~113 commits whose changes are absent from the tree: a reader sees "xone: update driver" and concludes xone is present, when it is a Buildroot package now. A log listing absent changes is worse than an absent ancestor. Disposition per fork commit lives in MISTER-KERNEL-PATCH-RECON.md, which cites the superseding vanilla commit — something no git command can produce, since across this much context drift `git patch-id` matches nothing. - Never touches a remote. Publishing is an explicit fetch-into-a-fork plus push, spelled out in the generated EXPORT.md. Verified end to end: exports Linux 6.18.38 + 31 patches, 34 commits, clean tree. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The orphan branch was wrong, and provably so: GitHub's compare API 404s with "No common ancestor", so an orphan branch cannot be PR'd anywhere at all. I had described that as the compare view being "meaningless"; it is stronger than that. The reason it was wrong is that I mis-read the fork's structure. Linux-Kernel_MiSTer is not one chain. Its tarball commits form a spine e12ed6c19 v5.13.12 -> 137491a75 v5.14 -> b6f2ca1c4 v5.14.5 -> aba1ef4c1 v5.15.1 and each MiSTer-vX.Y branch hangs off a spine point with the MiSTer series replayed on top (MiSTer-v5.15 = aba1ef4c1 + 112, MiSTer-v5.14 = 137491a75 + 63). They are siblings, not a line. Every spine commit is a pristine tarball with no MiSTer code. So --parent-repo/--parent parents the base commit on the newest spine point, which is simply what this project already does, four times. It gets all three properties at once, where each earlier design could only get two: - PR-able: merge-base with MiSTer-v5.15 is aba1ef4c1, so GitHub can compare. - Honest log: no MiSTer-5.15 commit appears, because they are siblings rather than ancestors. This was the entire objection the orphan existed to avoid, and parenting on the spine dissolves it — parenting on the branch TIP would have reintroduced it, listing ~112 commits whose changes this tree discards. - Meaningful base: its parent is a pristine tarball too, so the diff across it is the pure upstream 5.15.1 -> 6.18.38 delta. Verified: 80077 files, zero MiSTer-named paths on either side. Also fix `git add` -> `git add --force` for the base commit. The kernel ships .gitignore files matching paths it also tracks, so a plain add after a tarball extract silently drops them. This is not hypothetical — it is exactly why the fork's own v5.15.1 base is NOT byte-identical to kernel.org's v5.15.1 (11 files absent: Documentation/.yamllint, fs/*/.kunitconfig, selftests/bpf/test_progs.c, selftests/arm64/tags/* to the `tags` ctags pattern). Without --force we reproduced it, losing Documentation/.renames.txt. With --force the base tree hash is d13b0d25dbbc19af5884d0b780c74309c5d3fa1e — byte-identical to kernel.org's v6.18.38 tree, and independently checkable against their git. Our base is strictly more faithful than the one it extends. Verified: spine mode is deterministic (identical SHAs across runs), 38 commits (4 spine + base + 31 patches + defconfig + EXPORT.md), merge-base aba1ef4c1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…-tree Without this the exported tree builds a kernel with no Xbox (xone) and no 11ac WiFi, while MiSTer-v5.15 vendors both in-tree -- a silent feature regression for anyone who builds the export expecting what MiSTer ships. Vendors all four enabled kernel-module packages (xone + rtl8812au/8814au/8821au, 1882 files) at the paths the 5.15 branch uses, and emits build-mister-modules.sh to build them. NOT wired into Kconfig, deliberately. The Realtek Makefiles do this, ABOVE their own `ifneq ($(KERNELRELEASE),)` guard: export TopDIR ?= $(shell pwd) $(shell cp $(TopDIR)/autoconf_..._linux.h $(TopDIR)/include/autoconf.h) Parse-time filesystem mutation keyed off `pwd`. In-tree, `pwd` is the kernel root rather than the module dir, so TopDIR points at the wrong tree and the driver's generated autoconf.h silently never appears -- $(shell ...) eats the error. These 2594-line Makefiles assume they are never in-tree, across ~1900 files. Wiring them in would mean inventing hooks no upstream tests, then maintaining patches to upstream Makefiles forever. So they are built through the exact out-of-tree invocation Buildroot already uses -- upstream's own supported path, proven by our image builds -- with each recipe read from its .mk rather than reinvented. A driver bump is a pin change in the .mk plus a re-run; no rewiring. Three bugs found by actually building the result, not by reading it: - SILENT DRIVER OMISSION. The defconfig annotates package lines with trailing comments ("BR2_PACKAGE_RTL8812AU=y # RTL8812AU 11ac -- ..."), so anchoring the match on `=y$` hit only BR2_PACKAGE_XONE. The export vendored xone alone and dropped all three WiFi drivers -- exactly the regression this code exists to prevent, with the hole sitting upstream of the fail-closed MODULE_PATH check. Fixed, plus the identical latent bug in defconfig_value(), plus a post-export assertion that every enabled driver is really in the committed tree. - WRONG BUILD PRECONDITION. `modules_prepare` is not enough: external modules link against Module.symvers, which modpost writes during `make modules`, which needs vmlinux from the zImage build. Without it every kernel symbol reads as undefined ("ERROR: modpost: \"skb_pull\" [8812au.ko] undefined!") -- which blames the driver when nothing is wrong with it. The script now checks and says so. - VERMAGIC MISMATCH. This tree is a git repo ~35 commits past the v6.18.38 base, so setlocalversion correctly appends "+" -> 6.18.38+. Buildroot builds the same source from a tarball with no git, so its equally-patched kernel says plain 6.18.38. That "+" lands in vermagic and modprobe rejects every module on it. Passing LOCALVERSION= (set, empty) suppresses it. Verified: rebuilt 8812au.ko now reports `vermagic=6.18.38 SMP mod_unload ARMv7 p2v8`, byte-identical to Buildroot's -- so the export's modules and the image's are interchangeable. Verified end to end with the real ARM toolchain: all 4 drivers build (8812au.ko, 8814au.ko, 8821au.ko, 9 xone modules), 43 commits, 1932 files, deterministic across runs. Committed file counts match the tarballs exactly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Upstream created MiSTer-v6.18 (d9ac12a691) as its own vanilla 6.18.38 commit and asked for the changes as a PR against it. Notably it extends the spine at aba1ef4c1 (v5.15.1) -- independently the same structure --parent produces, which is a good sign the shape was right. But `--parent aba1ef4c1` builds OUR OWN base commit from the tarball, so our branch and theirs each carry a distinct "v6.18.38" commit over the same spine point. A PR would be diverged (ahead 39, behind 5) and merging would fuse two different bases. --onto replays straight onto an existing base instead: no base commit, no tarball needed for the kernel, and the result fast-forwards. Verified: 31/31 apply to d9ac12a691, 38 commits on top, their commit is an ancestor, so the PR is exactly our delta with nothing of theirs restated. Two guards, both from things that actually went wrong here: - The base's version is read back from its own Makefile and must equal the defconfig pin. Replaying a 6.18 series onto a 5.15 base otherwise fails deep in `git am` with conflicts that look like bad patches rather than a bad base. Verified: --onto aba1ef4c1 is refused with "is Linux 5.15.1, but this repo pins 6.18.38". - --onto is resolved to a SHA in the SOURCE repo before cloning. Ref names are ambiguous across a clone boundary, and this bit for real: `git clone` copies the source's LOCAL branches to origin/*, so `--onto origin/MiSTer-v6.18` resolved inside the clone to the source's own local MiSTer-v6.18 -- our tree, not theirs -- and replayed the series onto a tree that already had it. The version check could not catch it: both were 6.18.38. `git am` did, with "MiSTer_fb.c: already exists in index". Their base is missing Documentation/.renames.txt versus kernel.org's v6.18.38 (tree 4efcf6f42 vs d13b0d25d) -- the same .gitignore-eats-a-tracked-file bug that cost their v5.15.1 base 11 files, and that `git add --force` was added here to avoid. Cosmetic, no build impact, not ours to fix in a PR; raised upstream instead. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Two real findings from the PR #26 review, both reproduced before fixing: - `series=("$patch_dir"/*.patch)` without nullglob leaves the literal pattern in the array when nothing matches, so `((${#series[@]}))` sees length 1 and the "no patches" guard never fires. `git am` then fails on a path that does not exist, blaming a patch rather than the empty directory. Verified: an empty dir gave length 1 with element "<dir>/*.patch". Now scoped `shopt -s nullglob` around the glob; an empty patch dir dies with "no patches in <dir>". - The download path used `tarball="$(mktemp -d)/..."` with no cleanup, stranding a ~148MB kernel tarball per run, and bare `mktemp -d` is a GNU extension that errors on BSD/macOS. Now an explicit template plus an EXIT trap. --output is deliberately not cleaned: it is the deliverable and must survive a failure to be diagnosable. The review's third finding -- that `git add --all` respects .gitignore and needs -f -- was already fixed in this branch (the reviewer saw an earlier revision). That is the same bug that costs the upstream v5.15.1 base 11 files and its v6.18.38 base Documentation/.renames.txt; `git add --all --force` is why our tree hash matches kernel.org's exactly. The first cut of the trap made it worse, which is why this was tested rather than assumed: as an EXIT trap, cleanup()'s own return status becomes the script's exit status, so a bare `[[ -n $download_dir ]] && rm -rf ...` returned 1 on every run that used the dl/ cache -- the export printed PASS and exited 1, which would fail CI on success. Rewritten as an `if`. Verified after the change: export still exits 0, output SHA unchanged (9b485e0988), 31/31 patches apply. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings July 16, 2026 15:27

Copilot started reviewing on behalf of mcfbytes July 16, 2026 15:28 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

Comment thread scripts/export-kernel-tree.sh

Comment thread scripts/export-kernel-tree.sh Outdated

Comment thread scripts/export-kernel-tree.sh Outdated

mcfbytes and others added 4 commits July 16, 2026 12:05

mcfbytes force-pushed the fix/kernel-patch-am-headers branch from 3a2490b to 0e0efd1 Compare July 16, 2026 17:06

mcfbytes force-pushed the feat/export-kernel-tree branch from 90b8a8d to 38f1aed Compare July 16, 2026 17:06

mcfbytes merged commit 1d5050e into fix/kernel-patch-am-headers Jul 16, 2026
1 of 2 checks passed

mcfbytes deleted the feat/export-kernel-tree branch July 16, 2026 17:16

mcfbytes mentioned this pull request Jul 16, 2026

kernel-patches: make the carried series git am-able #25

Merged

Navigation Menu

scripts: add export-kernel-tree.sh — render the carried series as a Linux-Kernel_MiSTer-style tree - #26

scripts: add export-kernel-tree.sh — render the carried series as a Linux-Kernel_MiSTer-style tree#26
mcfbytes merged 5 commits into
fix/kernel-patch-am-headersfrom
feat/export-kernel-tree

mcfbytes commented Jul 16, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mcfbytes commented Jul 16, 2026

What

Why this is even possible

Verified, not asserted

The 6.18.39 path

Two decisions worth reviewing

Scope

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants