diff --git a/README.md b/README.md index 89c29e6b..945ab3af 100644 --- a/README.md +++ b/README.md @@ -793,7 +793,8 @@ broke and why, without grepping): - a sandboxed functional test of the timezone autodetect dhcpcd hook ([`scripts/test-timezone.sh`](scripts/test-timezone.sh)) — it turns a string off the network into a filesystem path, so every class of hostile answer is asserted rejected, - run twice: under the host shell and under the target's own BusyBox `ash` via `qemu-arm` + run three times: under the host shell, under the target's own `bash --posix` via `qemu-arm` + (what `/bin/sh` is on the box, as on stock), and under the target's BusyBox `ash` Alongside it, [`scripts/check-abi.sh`](scripts/check-abi.sh) runs the full SONAME/loader checklist from [`docs/abi-contract.md`](docs/abi-contract.md). The overlap on diff --git a/configs/fragments/de10nano-image.fragment b/configs/fragments/de10nano-image.fragment index 13ccda4a..5f2f5554 100644 --- a/configs/fragments/de10nano-image.fragment +++ b/configs/fragments/de10nano-image.fragment @@ -246,6 +246,12 @@ BR2_PACKAGE_WPA_SUPPLICANT_WPA3=y BR2_PACKAGE_WPA_SUPPLICANT_CLI=y BR2_PACKAGE_WPA_SUPPLICANT_PASSPHRASE=y BR2_PACKAGE_BASH=y +# WARNING: BR2_SYSTEM_BIN_SH_BASH depends on BR2_PACKAGE_BUSYBOX_SHOW_OTHERS +# (§5.15 above); without it kconfig drops this line silently and /bin/sh falls +# back to BusyBox ash. Stock parity: Buildroot's skeleton finalize hook makes +# /bin/sh -> bash AND sets root's login shell to /bin/bash from this one symbol +# (issue #144; docs §5.19). +BR2_SYSTEM_BIN_SH_BASH=y BR2_PACKAGE_DIALOG=y BR2_PACKAGE_WIRELESS_TOOLS=y BR2_PACKAGE_WIRELESS_TOOLS_IWCONFIG=y diff --git a/configs/fragments/golden.sha256 b/configs/fragments/golden.sha256 index 500a85a7..26f61e57 100644 --- a/configs/fragments/golden.sha256 +++ b/configs/fragments/golden.sha256 @@ -4,7 +4,7 @@ # scripts/check-config-fragments.sh --update-golden # and say in the commit message what changed and why. Columns: # -2026.05.2 de10nano 3cb8e3d9b32d701c4e3eefa1b9993da9b76a73a29a3b29f57924c10b807138a1 +2026.05.2 de10nano 251ae321011945e690a32f2fbb704b619681fd26d06acf578f7691e76286f992 2026.05.2 de10nano-kernel 5b372abe43c46c4eb5f3c88d51e48ff2eb6fa713b3c02f0f337c39d0f111dc69 2026.05.2 de25nano 2adb6b4029bee4467053403fdcf290042ccd2b554cc126de9a2f7a155b433b28 2026.05.2 rt e34a522a56eb088ac84b1e6db7350a22f12293d0d84086dbf1f405f68a58b7e5 diff --git a/docs/buildroot-config.md b/docs/buildroot-config.md index 0dcec471..a115bbdb 100644 --- a/docs/buildroot-config.md +++ b/docs/buildroot-config.md @@ -1129,6 +1129,29 @@ overlay's `70-persistent-net.rules` pre-up depends on it), `BR2_PACKAGE_IPROUTE2=y` (stock's `/usr/sbin/ip` — wifi.sh's link up/down fallback). +`BR2_SYSTEM_BIN_SH_BASH=y` — **`/bin/sh` is bash, and so is root's login +shell** (issue #144). Stock has `/bin/sh -> bash` and +`root:x:0:0:root:/root:/bin/bash`; until this symbol was added we shipped +Buildroot's defaults, BusyBox ash as `/bin/sh` and root on `/bin/sh`, with +nothing recording the difference. That matters in two places: a user script +in `/media/fat/Scripts` with a `#!/bin/sh` shebang and bash syntax runs on +stock and may not on ash (`docs/stock-reconciliation.md`'s `usr/bin/timidity` +row is the one case that was checked by hand — its `function` keyword happens +to be in ash's bash-compat set), and an interactive root session over ssh or +the console gets ash's line editing and no bash history. One symbol does both +halves: Buildroot's `SKELETON_INIT_COMMON_SET_BIN_SH` finalize hook runs +`ln -sf bash /bin/sh` **and** `sed '/^root:/s,[^/]*$,bash,' /etc/passwd` +(`package/skeleton-init-common/skeleton-init-common.mk`), so no post-build +edit is needed and the result is byte-for-byte stock's layout. The symbol +depends on `BR2_PACKAGE_BUSYBOX_SHOW_OTHERS` (§5.15, already on) — without +it kconfig silently drops the choice back to BusyBox, which is why the +fragment carries a WARNING next to it. BusyBox's own `ash` applet stays +built and listed in `/etc/shells`, exactly as on stock; only what `sh` +resolves to changes. Scripts invoked as `sh` get bash in POSIX mode +(`argv[0]` is `sh`), same as stock. `scripts/ci-tests.sh` asserts both +halves against `rootfs.tar` (P3.4 section) and runs the dhcpcd timezone hook +under the target's bash in POSIX mode as well as under ash. + ### 5.20 On-device text editors, ifupdown, BusyBox fragment, dhcpcd, ntp, cifs Editors (stock parity): stock ships `usr/bin/joe`, `usr/bin/nano` AND diff --git a/docs/decisions/0025-first-boot-timezone-autodetect.md b/docs/decisions/0025-first-boot-timezone-autodetect.md index f9e364bc..099d793c 100644 --- a/docs/decisions/0025-first-boot-timezone-autodetect.md +++ b/docs/decisions/0025-first-boot-timezone-autodetect.md @@ -158,8 +158,8 @@ to add traffic silently, hence the disclosure above, the FAQ entry, and the opt- `scripts/test-timezone.sh` — a sandboxed functional test (no build, no board, no network: paths rewritten into a temp dir, `curl` stubbed, and the hook sourced exactly as -`dhcpcd-run-hooks` sources it — under `$TZ_TEST_SH`, so the second CI leg genuinely parses -and runs it as the target's BusyBox `ash`). 16 cases / 60 assertions covering the +`dhcpcd-run-hooks` sources it — under `$TZ_TEST_SH`, so the target CI legs genuinely parse +and run it as the target's own shells). 16 cases / 60 assertions covering the happy path, the never-overwrite rule (including that an *empty* timezone file counts as unset, so a half-written card self-heals, and that a timezone set *while the lookup runs* is not clobbered), the once-and-only-once contract, the opt-out stamp, nine classes of @@ -170,10 +170,14 @@ which dhcpcd reasons fire it and which do not, `if_up` handled as data rather th `exit`ing dhcpcd's hook run, that it leaks nothing into the shell that sourced it, and that the sourcing shell returns while the lookup is still in flight. -`scripts/ci-tests.sh`'s Timezone section runs it **twice** — once under the host shell, -once under the target's own BusyBox `ash` via `qemu-arm`, since this is a boot-path script -and dash-accepts-it is not the same claim as ash-accepts-it — and additionally asserts the -script ships executable and that `curl`, its one runtime dependency, is still in the image. +`scripts/ci-tests.sh`'s Timezone section runs it **three times** — once under the host +shell, once under the target's own `bash --posix` via `qemu-arm`, once under the target's +BusyBox `ash` — since this is a boot-path script and dash-accepts-it is not the same claim +as the-box's-shell-accepts-it — and additionally asserts the script ships executable and +that `curl`, its one runtime dependency, is still in the image. (Addendum 2026-09-03: this +ADR originally said *twice*, host + ash, because `/bin/sh` was BusyBox ash at the time. +Issue #144 made `/bin/sh` bash as on stock, so the shell that actually sources this hook +on the box is bash in POSIX mode; the ash leg is kept as the stricter interpreter.) **Not yet verified on hardware**: the on-device first-boot path (real card, real DHCP) has not been exercised. That belongs to the next hardware validation pass. diff --git a/docs/init-parity.md b/docs/init-parity.md index 8a377558..16b8ce71 100644 --- a/docs/init-parity.md +++ b/docs/init-parity.md @@ -150,6 +150,7 @@ regardless, so the worst case is slow, never weak. | `/etc/network/interfaces` | **adapted (v9)** (1 additive deviation) | `lo` + `wlan0`/`wlan1` with the `wpa_supplicant -D nl80211,wext` pre-up hooks, all of stock's directives reproduced unchanged. No `eth0` stanza — matches stock exactly; wired ethernet is handled by dhcpcd's own default (manage-everything-not-explicitly-excluded) behavior, not ifupdown. **This row said "identical / byte-for-byte" until now and was stale:** P2.3 authored it byte-identical, but `4cf2fc7` (v9) added a header comment plus one `pre-up i=0; while [ $i -lt 20 ] && ! iw dev $IFACE info >/dev/null 2>&1; do sleep 1; i=$((i+1)); done` line per `wlan` stanza, because USB WiFi drivers that register `nl80211` asynchronously (mainline `rtw88`/`rtw89`) otherwise lose the race and `wpa_supplicant` fails with "interface not found" on a cold boot. **Revised since:** each stanza also gained a `pre-up [ -e /sys/class/net/$IFACE ]` device-presence guard ahead of that loop, because `S40network`'s `ifup -a` processes *every* `auto` stanza — so on the ordinary single-dongle box the `wlan1` stanza polled the full 20 s for a device that never appears, serialising `S41dhcpcd`/`S49ntp`/`S50sshd`/Main_MiSTer behind it on every WiFi boot (measured: 20 s). A device that is merely late is brought up by its own `add` uevent via `70-persistent-net.rules` → `etc/wifi-hotplug.sh`, which fires on udev's boot-time coldplug as well as on later insertion, so the guard costs no coverage. `diff` against `work/imgroot/etc/network/interfaces` now exits 1 with exactly 47 added lines (a 43-line header comment plus the guard and the loop in each of the two `wlan` stanzas) and zero removals. See `docs/wifi-parity.md` §1 and §9. | | `/etc/dhcpcd.conf` | **identical** | Byte-for-byte match. The package's own default differs meaningfully (`#hostname`/`#clientid` instead of stock's enabled `hostname`/`clientid`, `duid` instead of stock's `#duid`, and is missing the `option rapid_commit` block) — all reverted to stock via the overlay. | | `/etc/inittab` | **adapted** (3 documented deviations) | Full stock shape reproduced (`::sysinit:/media/fat/MiSTer &`, `/etc/resync &`, `rcS`, shutdown sequence) with: **(1)** the remount-rw sysinit line kept **commented out**, exactly as stock has it — Buildroot's own skeleton default inittab ships this line **uncommented**, which would remount `/` rw at every sysinit and defeat the whole read-only-root design (ADR 0011); confirmed this project's own finalize hook tries to uncomment it too (see below) and is overridden by the overlay running last. **(2)** the serial console runs `ttyS0::respawn:/sbin/agetty --nohostname -L ttyS0 115200 vt100` — util-linux `agetty`, matching stock (whose inittab also uses `agetty --nohostname`), now that `BR2_PACKAGE_UTIL_LINUX_AGETTY` is enabled (see `docs/util-linux-parity.md`). It still targets `ttyS0` explicitly rather than stock's `console` alias, because this board's actual cmdline is `console=ttyS0,115200` (`docs/boot-chain.md`). agetty treats a numeric positional argument as the baud rate, so `ttyS0 115200` and `115200 ttyS0` are equivalent; the port-first order is kept for continuity with the previous BusyBox `getty` line. (Earlier revisions used BusyBox `getty` because `BR2_PACKAGE_UTIL_LINUX_BINARIES` was not selected — that constraint no longer holds; the overlapping BusyBox `getty` applet is now disabled so `agetty` is the console.) **(3)** `gpm` is invoked at `/usr/sbin/gpm`, not stock's `/sbin/gpm` — originally **a real bug caught during acceptance verification**: gpm is a real package (not a BusyBox applet, which lands under `/sbin` regardless of usr-merge), and when this row was first written (P2.3) the rootfs was **not** usr-merged — `/sbin` was a real directory, not `-> /usr/sbin` the way stock's `work/imgroot/sbin` is — so `/sbin/gpm` genuinely did not exist in the image. An inittab `sysinit` line is an **absolute path**, so unlike `/etc/profile`'s `PATH` it gets no shell search at all; it would have silently failed to spawn on every single boot. Caught by exhaustively grep'ing every absolute path referenced anywhere in the overlay against the actually-built image (see the task report) — first build had this wrong; fixed and rebuilt before acceptance. **Since P2.9 v2** (`7be9ee5`) set `BR2_ROOTFS_MERGED_USR=y`, that premise no longer holds: this build is now usr-merged like stock (verified: `output/target/sbin -> usr/sbin`), so `/sbin/gpm` *would* resolve today. **The deviation is kept anyway, deliberately.** Buildroot builds gpm with `--prefix=/usr`, so the binary's real path is `/usr/sbin/gpm` in *either* layout (gpm's own file list: `gpm,./usr/sbin/gpm`) — stock's binary is at that same physical path, and stock's inittab only reaches it through the `/sbin -> usr/sbin` compat symlink that exists solely because stock is usr-merged. So our path is correct in both layouts and stock's is correct in only one; ours survives an unmerge, stock's would break on one. Given P2.3 already got bitten by exactly that (the unmerged build had no `/sbin/gpm` at all), pointing at the real path rather than a merge-dependent alias is the more durable choice, even though a future unmerge is unlikely. This is an **accepted permanent deviation from stock**, not a cleanup waiting to happen. **Also dropped — later RESTORED, guarded (T3, 2026-07-27):** `loadkeys /etc/kbd.map` and `setfont` were dropped by P2.3 because this BusyBox build has no `loadkeys` applet at all and `CONFIG_SETFONT` is explicitly not set (still true — re-verified in `output/build/busybox-1.38.0/.config`; busybox's `loadkmap` applet is not a substitute, it reads binary bkeymap, not stock's text keymap); keeping either line verbatim would just fail every boot. T3 vendored stock's `etc/kbd.map` (it blanks the F12/Mute/Vol± keycodes Main_MiSTer consumes via evdev) and restored both lines wrapped in `[ -x /usr/bin/... ]` guards: with the parallel T5 task's `BR2_PACKAGE_KBD` (the same package stock's own loadkeys/setfont came from) the lines do stock's exact job, without it they are silent no-ops instead of boot errors. See the inittab's own note 3 and `docs/stock-reconciliation.md` §3c. `gpm -m /dev/input/mice -t imps2` itself **is** kept (just at the corrected path) — `BR2_PACKAGE_GPM=y` was deliberately selected in P2.1 for this, and the binary is present. | +| `/etc/passwd` (root's shell) + `/bin/sh` | **identical (since issue #144)** | Stock: `root:x:0:0:root:/root:/bin/bash` and `/bin/sh -> bash`. Until 2026-09-03 this build shipped Buildroot's defaults — BusyBox ash as `/bin/sh` and root on `/bin/sh` — because no fragment set `BR2_SYSTEM_BIN_SH`, and no row here recorded it. Found while chasing #142 (WinSCP), which was a separate `/etc/profile` defect but whose error text ("BASH is recommended") pointed at the shell. Fixed with `BR2_SYSTEM_BIN_SH_BASH=y` in `de10nano-image.fragment`: Buildroot's skeleton finalize hook both re-links `/bin/sh` and rewrites root's passwd shell from that one symbol, so the result matches stock with no overlay or post-build edit. `docs/buildroot-config.md` §5.19 has the rationale; `scripts/ci-tests.sh` asserts both halves. | | `/etc/profile` | **adapted** (2 documented deviations) | Full stock content reproduced (PATH, `PS1='$(pwd)# '`, `EDITOR=/bin/vi`, `/etc/profile.d/*.sh` sourcing, `LC_ALL=en_US.UTF-8`, and critically the login-time `mount -o remount,rw /` — this is how `/` ever becomes writable at all, matching stock and ADR 0011's own description of the mechanism). **Deviation:** `PATH` gains an explicit `/bin:/sbin:` prefix stock's literal string doesn't have. Stock's rootfs is usr-merged (`work/imgroot`: `/bin -> usr/bin`, `/sbin -> usr/sbin`), so its `PATH="/usr/bin:/usr/sbin"` already covered `/bin`/`/sbin` for free. When this deviation was introduced (P2.3) *this* build was a plain (non-merged) skeleton layout, so omitting `/bin:/sbin` from `PATH` would have silently dropped most BusyBox applets from every interactive shell; changing `BR2_ROOTFS_MERGED_USR` was then out of P2.3's scope ("do NOT disturb ... the package set"), making the overlay `PATH` the correct-altitude fix. **Since P2.9 v2** (`7be9ee5`) set `BR2_ROOTFS_MERGED_USR=y` — the change that fixed the `/lib/security/pam_unix.so` SSH lockout — this build is usr-merged too, so the `/bin:/sbin:` prefix is now redundant. It is kept because it is harmless (the paths resolve to the same directories) and matches Buildroot's own skeleton default profile. The file's own header comment records the same history. **Deviation 2 (2026-09-03, issue #142):** stock's trailing bare `resize >/dev/null` is guarded as `[ "$PS1" ] && [ -z "$SSH_CONNECTION" ] && resize >/dev/null`. `resize` is the BusyBox applet: it writes a cursor-position query to stderr and reads the reply from stdin under a 3 s alarm. On the serial console that is the only way the shell learns the window size, so it stays. Over SSH it is redundant (sshd propagates the client's window size through the pty) and, for a client that opens the login shell **without** a pty, actively broken: WinSCP's SCP mode and its terminal window receive the escape bytes on stderr and lose their first command to `resize`'s `scanf`, which WinSCP reports as "Error skipping startup message. Your shell is probably incompatible with the application" (forum report: https://misterfpga.org/viewtopic.php?p=113698#p113698). SFTP, PuTTY and command-line scp were never affected. Stock has the identical defect — verified on `release_20250402`'s `linux.img` (same profile line, `resize -> busybox` 1.33.1) by running stock's own busybox under `qemu-arm` on a pty: identical bytes, next input line swallowed. `SSH_CONNECTION` rather than `SSH_TTY` because sshd sets the former for every session and the latter only when a pty is allocated. Verified on a board: no-pty login shell, pty login shell and `bash -l` are all clean; a simulated console login (SSH vars cleared) still runs `resize`. | | `/usr/lib/dhcpcd/dhcpcd-hooks/90-timezone` | **added (divergence, [ADR 0025](decisions/0025-first-boot-timezone-autodetect.md))** | Stock has no equivalent, and that is the gap it closes: `/etc/localtime` points at `/media/fat/linux/timezone`, which **does not exist on a fresh card**, so glibc falls back to UTC silently and permanently. The first time dhcpcd brings an interface up with an address, this asks `ip-api.com` for the zone of the box's public IP and copies `/usr/share/zoneinfo/posix/` to that path — the *same* provider, destination and file format as the community `Scripts_MiSTer/timezone.sh` "Automatic" mode, so the two are interchangeable. No new package (`curl` and tzdata were already in the image). **Note this adds no init script**: an earlier revision had an `S48timezone` as well, but on a DHCP box the lease has usually not landed by S48, so it was near-redundant with this hook — and it carried a `/proc/net/route` check whose IPv6 arm silently matched the kernel's own `ip6_null_entry`. A static-IP box configured only in `/etc/network/interfaces` never runs dhcpcd and so never autodetects; accepted deliberately, since setting a static address is already a by-hand act. Properties worth stating because they are the design: the guess is spent **once, and only when it was actually made** (gated on the timezone file *and* on a `timezone.autodetect` stamp written only when a provider answered *with a zone name*, so neither being offline nor a captive portal's HTTP 200 burns it), it **never overwrites** a timezone anyone has already set (re-checked immediately before the write, not only at the gate), and it **delays nothing** — the body is a backgrounded subshell, which also keeps it from leaking a single variable or function into dhcpcd's shell. Sourced, not executed (`dhcpcd-run-hooks`: `. "$hook"`), hence no shebang, no exec bit, and no `exit` — an `exit` here would end dhcpcd's whole hook run and take `20-resolv.conf`/`30-hostname` with it. Zone names arrive off the network, so they are validated against the shipped zoneinfo before being used as a path; `scripts/test-timezone.sh` asserts each rejection, and mutation-checks the two that could otherwise pass vacuously. | | `/usr/lib/dhcpcd/dhcpcd-hooks/91-ntp-kick` | **added (divergence)** | Stock has no equivalent. The gap: this board has no RTC, so a cold boot starts at the epoch, and `S49ntp` launches `ntpd -g` at a fixed point in `rcS` whether or not a network exists — which on the common path it does not, since WiFi association plus DHCP normally completes well after S49. ntpd survives that in one half and not the other. The **interface** half heals itself: ntpd is built with `HAVE_RTNETLINK` (`config.h:786`), so an address appearing is a netlink event and `ntp_io.c:1980` → `ntp_peer.c:763` `refresh_all_peerinterfaces` reattaches peers within ~3 s. The **DNS** half does not: for a `server ` line the peer is created *only* in the DNS callback (`ntp_config.c:4459-4463` → `peer_name_resolved`), and at S49 there is no `/etc/resolv.conf` — `20-resolv.conf` writes it in this very hook pass — so resolution fails and **no peers exist at all** for the rescan to reattach. The clock then waits on ntpd's DNS retry backoff and nothing else: `libntp/ntp_intres.c` `manage_dns_retry_interval` walks 2-3-4-6-8-12-16-24-32-48-64 s (`config.h:1313` leaves `IGNORE_DNS_ERRORS` undefined, so `DNSFLAGS` is 0 and `retmax` is 64, not 1024), putting attempts at t = 2, 5, 9, 15, 23, 35, 51, 75, 107, 155, 219, 283 … s after ntpd started. Measured cost of the next-slot wait: a few seconds on a wired box, 10-20 s on a typical WiFi boot, 30 s to a couple of minutes when the network turns up late. This restarts ntpd once, when an address actually arrives, so it re-resolves against the `resolv.conf` that now exists; `/etc/ntp.conf`'s `iburst` and `S49ntp`'s `-g` then give correct time within ~10-15 s. **Not `sntp`**: `BR2_PACKAGE_NTP_SNTP` would also install Buildroot's `S48sntp` (`package/ntp/ntp.mk:110-114`), which runs sntp at boot one script *before* ntpd with no network, and would have to be suppressed — not worth ~10 s. **Not `service_condcommand`** (which `50-ntp.conf` uses): dhcpcd's `detect_init` finds no systemctl/rc-service/invoke-rc.d/service/sv here and falls through to its `/etc/init.d` branch, which tests `[ -x /etc/init.d/ntpd ]` — ours is `S49ntp` — and `service_status` then runs `$x/$1 status`, a verb `S49ntp` has no case for; both fail silently. The hook calls the script directly. **Not a udev rule**: a `net` `add` uevent fires when the netdev is *created*, before association, lease, route or `resolv.conf`, which would reproduce the bug one layer down. Design properties: fires **at most once per boot** (`mkdir /run/ntp-kick` is the atomic test-and-set, on a tmpfs, so it re-arms each boot — two interfaces can `BOUND` at the same moment, and restarting ntpd more than once would discard its accumulated clock discipline for nothing); **never starts an ntpd that is not running**, so a by-hand or `/etc/default/ntpd` disable stands; and the stamp is claimed **last**, after every gate, so a pass that bails out leaves the one kick available — which is what makes the wired-box ordering case correct, where `S41dhcpcd` blocks until the first lease and so fires this hook *before* `S49ntp` has started ntpd at all. Backgrounded subshell (`S49ntp restart` contains a `sleep 1`), sourced not executed — no shebang, no exec bit, no `exit`. A static-IP box never runs dhcpcd and never gets kicked; accepted, the same gap ADR 0025 accepts, and ntpd's own backoff still converges there. **Acquisition reasons only** — `BOUND`/`REBOOT` and their `*6` forms, deliberately *not* `RENEW`/`REBIND` as `90-timezone` matches: dhcpcd picks those two only when `state->old` is non-NULL (`src/dhcp.c:2499-2513`), i.e. the address was already there. Matching them combines badly with the re-armable stamp — a renewal arriving hours into the session would restart an ntpd that had been synchronised the whole time, and the wired ordering case makes that reachable rather than theoretical, since it always bails out first and leaves the kick armed. Nothing is lost: WiFi associating late, a dongle plugged in later and an AP returning are all *new* leases, hence `BOUND`; and an address that merely changes under a `RENEW`/`REBIND` needs no kick, because ntpd holds peers by then and the `HAVE_RTNETLINK` path reattaches them in ~3 s. (Found in review of PR #147.) `scripts/test-ntp-kick.sh` asserts the behaviour in 38 sandboxed cases (run twice by `ci-tests.sh`: host shell, and the target's own BusyBox ash under qemu-arm), mutation-checked against dropping the once-per-boot gate, the liveness check, the narrowed reason set, the process-identity check and the pid-0 rejection. The liveness probe is `start-stop-daemon -K -t -q -p -x /usr/sbin/ntpd`, which checks identity as well: the pidfile cannot outlive a boot (tmpfs), but a crashed ntpd within one boot leaves one, and a long-running box can wrap `pid_max` and reuse that pid — restarting then would have `S49ntp`'s stock `stop()` (`start-stop-daemon -K -p` with no `-x`) SIGTERM an unrelated process, and this hook is the first automatic caller of that script. A pidfile containing `0` is rejected explicitly, because `kill -0 0` signals the process group and the probe itself was measured to succeed on it. Both behaviours measured on ours (BusyBox 1.38.0) and stock's (1.33.1). (Found in review of PR #147.) | diff --git a/docs/stock-reconciliation.md b/docs/stock-reconciliation.md index 070aed27..36d34cbe 100644 --- a/docs/stock-reconciliation.md +++ b/docs/stock-reconciliation.md @@ -224,7 +224,7 @@ marked CLOSED here. | `usr/sbin/vmode` | **B — CLOSED** | Vendored byte-identical. Writes `fb_cmd0/fb_cmd1` to `/dev/MiSTer_cmd` — a FIFO **Main creates** (`input.cpp:4051`), commands handled at `input.cpp:6236` — and polls `/sys/module/MiSTer_fb/parameters/res_count`, which exists because `CONFIG_FB_MISTER=y` and `MiSTer_fb.c:38` declares `module_param(res_count, uint, 0444)`. bash + busybox `usleep` both present. | | `usr/bin/vhd_mount` | **B — CLOSED** | Vendored byte-identical. Mechanism verified end-to-end on *our* stack: busybox `losetup` attaches, the cmdline's `loop.max_part=8` (`docs/boot-chain.md:155`) makes `/dev/loop1p1` appear, busybox `mount` mounts it on `/media/rootfs` — which must pre-exist on a read-only `/`, so the overlay now ships `media/rootfs/` (`.gitkeep` idiom, same as `/media/fat`). Stock's image has the same dir (`work/imgroot/media/rootfs`). | | `usr/bin/m3u_play` | **B — CLOSED** | Vendored byte-identical. mp3 branch works (`mpg123` shipped); the vgm/vgz branches reference `vgmplay`, deliberately not shipped (below) — on a vgm playlist the script prints `vgmplay: not found`, exactly the graceful-degradation stock had for absent optional players. | -| `usr/bin/timidity` | **B — CLOSED (recon error corrected)** | The first reconciliation pass classified this "audio player (mixed)" as if it were TiMidity++ needing a new package. **It is a 22-line `/bin/sh` wrapper** (read from `addon.tar`): fluidsynth (ALSA-seq, `/media/fat/linux/soundfonts/SC-55.sf2`) + `aplaymidi --port 128 $MC_EXT_SELECTED`. Both callees shipped; `$MC_EXT_SELECTED` is exported by mc on every ext.ini action (`mc-4.8.33 src/filemanager/ext.c:205`), so it also works with mc 4.8.33's stock `[midi]` handler (whose `sound.sh` execs `timidity` — i.e. this wrapper). Its `function ctrl_c()` bashism is legal in our `/bin/sh`: busybox 1.38 ash with `CONFIG_ASH_BASH_COMPAT=y` compiles the `function` keyword in (`shell/ash.c:223,9054`; `.config:1148`). Vendored byte-identical. | +| `usr/bin/timidity` | **B — CLOSED (recon error corrected)** | The first reconciliation pass classified this "audio player (mixed)" as if it were TiMidity++ needing a new package. **It is a 22-line `/bin/sh` wrapper** (read from `addon.tar`): fluidsynth (ALSA-seq, `/media/fat/linux/soundfonts/SC-55.sf2`) + `aplaymidi --port 128 $MC_EXT_SELECTED`. Both callees shipped; `$MC_EXT_SELECTED` is exported by mc on every ext.ini action (`mc-4.8.33 src/filemanager/ext.c:205`), so it also works with mc 4.8.33's stock `[midi]` handler (whose `sound.sh` execs `timidity` — i.e. this wrapper). Its `function ctrl_c()` bashism is legal in our `/bin/sh`: since issue #144 `/bin/sh` is bash, as on stock (`BR2_SYSTEM_BIN_SH_BASH=y`, `docs/buildroot-config.md` §5.19), and bash accepts `function` in POSIX mode — this file was written for bash all along, and the earlier note that busybox 1.38 ash's `CONFIG_ASH_BASH_COMPAT=y` happened to tolerate it described the accommodation, not the design. Vendored byte-identical. | | `usr/bin/vgmplay`, `usr/bin/VGMPlay.ini` | **C — DECLINED, documented** | Real gap, deliberate decline. VGMPlay (vgmrips/vgmplay, GPL) has no Buildroot package; writing one here could not be build-verified (T3 runs under a no-build constraint), and an untested C package is a worse outcome than an honest absence — it risks breaking `make all` for a niche feature (VGM chiptune playback inside mc). Cost of absence: mc's vgm handler and `m3u_play`'s vgm branch print `not found`; nothing else references it. Revisit as its own small task if VGM playback is ever asked for: pin a release tarball, `Makefile`-type package, install `VGMPlay.ini` beside the binary (it looks for its ini next to `argv[0]`). | | `usr/bin/memtool` | **A — CLOSED** | Not a sourceless blob after all: strings on the stock ELF are pengutronix memtool's exact usage text, and `addon.tar`'s `usr/bin/md`/`mw` are symlinks → `memtool` (argv[0] dispatch — `memtool.c:475` in the pinned 2018.03.0 tarball switches on `basename(argv[0])`). `BR2_PACKAGE_MEMTOOL=y` (Buildroot's own package, same upstream, 2018.03.0 — upstream's last release, confirmed against pengutronix's release directory). The package installs only the `memtool` binary, so stock's `md`/`mw` symlinks are reproduced in the overlay. | | `usr/bin/fpga` | **D — INFEASIBLE, precisely bounded** | Stripped ARM ELF; **no public source found**: GitHub code search for its distinctive strings finds only `Main_MiSTer/fpga_io.cpp` (which shares the literal `"FPGA: Unaligned data, realign to 32bit boundary."`, `fpga_io.cpp:352`) and u-boot's `drivers/fpga/socfpga.c`; no MiSTer-devel repo builds a standalone `fpga` binary. It is a dev-era peek/poke + RBF loader (`Usage(1): %s { address } [ data ]`, `Usage(2): %s { rbf_file }`, mmaps `/dev/mem`). Shipping the blob violates rule G6; recreating it from `fpga_io.cpp` would be new, untestable systems code. **Intent is covered**: core loading = `echo load_core > /dev/MiSTer_cmd` (Main, `input.cpp:6238-6242` — and mc's Enter-on-`.rbf` now does exactly that); peek/poke = `memtool md`/`mw` (above) or busybox `devmem`. Same documentation standard as `docs/firmware-parity.md`'s no-upstream-source firmware. | diff --git a/scripts/ci-tests.sh b/scripts/ci-tests.sh index 4cad99be..28f59fe3 100755 --- a/scripts/ci-tests.sh +++ b/scripts/ci-tests.sh @@ -712,6 +712,29 @@ else fail "iwgetid present (symlink to iwconfig)" "not in rootfs.tar" fi +# /bin/sh and root's login shell are bash, as on stock (issue #144, +# docs/buildroot-config.md §5.19). One symbol (BR2_SYSTEM_BIN_SH_BASH) drives +# both through Buildroot's skeleton finalize hook, and kconfig drops that +# symbol SILENTLY if BR2_PACKAGE_BUSYBOX_SHOW_OTHERS ever goes away -- the +# build stays green and /bin/sh quietly becomes BusyBox ash again. Usr-merged +# rootfs: the tar records /bin/sh only as ./usr/bin/sh. +sh_line=$(tar tvf "$ROOTFS_TAR" -- "./usr/bin/sh" 2>/dev/null | head -1) +case "$sh_line" in +*'-> bash') + pass "/bin/sh -> bash (stock parity, BR2_SYSTEM_BIN_SH_BASH)" ;; +'') + fail "/bin/sh -> bash (stock parity, BR2_SYSTEM_BIN_SH_BASH)" "usr/bin/sh not in rootfs.tar" ;; +*) + fail "/bin/sh -> bash (stock parity, BR2_SYSTEM_BIN_SH_BASH)" "is: $sh_line" ;; +esac +root_line=$(tar xOf "$ROOTFS_TAR" "./etc/passwd" 2>/dev/null | grep -m1 '^root:') +case "$root_line" in +root:*:/bin/bash) + pass "root's login shell is /bin/bash (stock parity)" ;; +*) + fail "root's login shell is /bin/bash (stock parity)" "passwd: ${root_line:-no root line}" ;; +esac + # ============================================================================= section "T2 — WiFi hotplug (70-persistent-net.rules, docs/wifi-parity.md §9)" # ============================================================================= @@ -1747,22 +1770,41 @@ else "one or more cases failed -- see output above" fi -# ...and again under the shell that will ACTUALLY run it on the box. The host's -# /bin/sh (dash, on the CI runner) is a good POSIX proxy for BusyBox ash, but it -# is not the same interpreter, and this is a boot-path script: a construct dash -# accepts and ash does not would fail on hardware and nowhere else. +# ...and again under the target's own interpreters. The host's /bin/sh (dash, +# on the CI runner) is a good POSIX proxy, but it is not what runs on the box, +# and this is a boot-path script: a construct dash accepts and the real shell +# does not would fail on hardware and nowhere else. On the box /bin/sh is bash +# (issue #144, stock parity), invoked through the `sh` symlink so it runs in +# POSIX mode -- `bash --posix` is the same thing spelled explicitly. BusyBox +# ash is still shipped and still a valid /etc/shells entry, so keep that run +# too: it is the stricter interpreter, and the hook must not depend on bash. if [ -z "$QEMU_ARM" ]; then + skip "test-timezone.sh under the target's own bash --posix (/bin/sh)" "qemu-arm not found on PATH" skip "test-timezone.sh under the target's own BusyBox ash" "qemu-arm not found on PATH" -elif [ ! -x "$TARGET/bin/busybox" ]; then - skip "test-timezone.sh under the target's own BusyBox ash" "$TARGET/bin/busybox not present" else - printf -- '--- test-timezone.sh: same cases, target BusyBox ash under qemu-arm ---\n' - if TZ_TEST_SH="$QEMU_ARM -L $TARGET $TARGET/bin/busybox sh" \ - "$ROOT/scripts/test-timezone.sh"; then - pass "test-timezone.sh under the target's own BusyBox ash" + if [ ! -x "$TARGET/usr/bin/bash" ]; then + skip "test-timezone.sh under the target's own bash --posix (/bin/sh)" "$TARGET/usr/bin/bash not present" else - fail "test-timezone.sh under the target's own BusyBox ash" \ - "passes on the host shell but not on BusyBox ash -- see output above" + printf -- '--- test-timezone.sh: same cases, target bash --posix (what /bin/sh is on the box) under qemu-arm ---\n' + if TZ_TEST_SH="$QEMU_ARM -L $TARGET $TARGET/usr/bin/bash --posix" \ + "$ROOT/scripts/test-timezone.sh"; then + pass "test-timezone.sh under the target's own bash --posix (/bin/sh)" + else + fail "test-timezone.sh under the target's own bash --posix (/bin/sh)" \ + "passes on the host shell but not on the target's bash -- see output above" + fi + fi + if [ ! -x "$TARGET/bin/busybox" ]; then + skip "test-timezone.sh under the target's own BusyBox ash" "$TARGET/bin/busybox not present" + else + printf -- '--- test-timezone.sh: same cases, target BusyBox ash under qemu-arm ---\n' + if TZ_TEST_SH="$QEMU_ARM -L $TARGET $TARGET/bin/busybox sh" \ + "$ROOT/scripts/test-timezone.sh"; then + pass "test-timezone.sh under the target's own BusyBox ash" + else + fail "test-timezone.sh under the target's own BusyBox ash" \ + "passes on the host shell but not on BusyBox ash -- see output above" + fi fi fi diff --git a/scripts/test-initramfs.sh b/scripts/test-initramfs.sh index edaf0212..29441a18 100755 --- a/scripts/test-initramfs.sh +++ b/scripts/test-initramfs.sh @@ -190,8 +190,22 @@ ensure_qemu_kernel() { log "configuring: multi_v7_defconfig + $KERNEL_FRAGMENT" make -C "$KERNEL_SRC" O="$KBUILD" ARCH=arm CROSS_COMPILE="$CROSS_COMPILE" \ multi_v7_defconfig - "$KERNEL_SRC/scripts/kconfig/merge_config.sh" -O "$KBUILD" \ - "$KBUILD/.config" "$KERNEL_FRAGMENT" >&2 + # merge_config.sh finishes with a BARE `make ... alldefconfig` in the + # CURRENT directory -- it has no -C. Run from this repo's root (the + # normal way to invoke this script) that `make` hits the wrapper + # Makefile, which forwards `alldefconfig` to Buildroot, which fails + # ("Can't read seed configuration"), and merge_config.sh exits before + # writing the merged .config. Every fragment symbol the base defconfig + # already had looks fine; every one it lacked (CONFIG_EXFAT_FS, + # CONFIG_FAT_DEFAULT_UTF8) is silently missing, and the three exFAT + # cases fail with "mount: No such device" for a reason that looks + # nothing like this. So: cd into the kernel tree, give its make the + # ARCH it needs, and refuse to continue if the merge fails. The + # fragment-survival check after olddefconfig below is the backstop. + (cd "$KERNEL_SRC" && ARCH=arm CROSS_COMPILE="$CROSS_COMPILE" \ + scripts/kconfig/merge_config.sh -O "$KBUILD" \ + "$KBUILD/.config" "$KERNEL_FRAGMENT" >&2) \ + || die "merge_config.sh failed for $KERNEL_FRAGMENT" elif [ ! -d "$KERNEL_SRC" ]; then die "$KBUILD exists but its source tree $KERNEL_SRC does not." \ "Remove $KBUILD (or set TEST_INITRAMFS_KBUILD to a fresh path) and re-run." @@ -217,6 +231,23 @@ ensure_qemu_kernel() { make -C "$KERNEL_SRC" O="$KBUILD" ARCH=arm CROSS_COMPILE="$CROSS_COMPILE" \ olddefconfig >&2 + # Every `CONFIG_X=y` the fragment asks for must be in the resolved config, + # on a fresh bootstrap AND on a cached $KBUILD (a cache made by a run whose + # merge silently failed -- see above -- keeps its broken .config forever, + # because the merge only happens on bootstrap). Fail with the fix spelled + # out rather than let the exFAT cases fail on a symptom. + missing="" + while IFS= read -r sym; do + grep -qx "$sym" "$KBUILD/.config" || missing="$missing $sym" + done <<-EOF + $(grep -E '^CONFIG_[A-Z0-9_]+=y$' "$KERNEL_FRAGMENT") + EOF + [ -z "$missing" ] || die \ + "QEMU test kernel .config lacks fragment symbol(s):$missing" \ + "(from $KERNEL_FRAGMENT). If $KBUILD is a cache from before the" \ + "merge_config.sh cwd fix, remove it (rm -rf $KBUILD) and re-run to" \ + "bootstrap a correct one." + log "building QEMU test kernel zImage (embedding $(basename "$CPIO"))" make -C "$KERNEL_SRC" O="$KBUILD" ARCH=arm CROSS_COMPILE="$CROSS_COMPILE" \ -j"$(nproc)" zImage >&2 diff --git a/scripts/test-timezone.sh b/scripts/test-timezone.sh index 1b385b72..e763c6d5 100755 --- a/scripts/test-timezone.sh +++ b/scripts/test-timezone.sh @@ -32,14 +32,17 @@ # # Usage: scripts/test-timezone.sh # Exit 0 iff every case passed. Wired into scripts/ci-tests.sh's Timezone -# section, which runs it twice: once under the host shell, and once under the -# target's own BusyBox ash via qemu-arm (TZ_TEST_SH, below) -- the shell that -# will actually run this on the box. The host's /bin/sh is usually dash, which -# is a good POSIX proxy but is not the same interpreter. +# section, which runs it three times: under the host shell, under the target's +# own bash --posix via qemu-arm (TZ_TEST_SH, below) -- /bin/sh on the box is +# bash as on stock (issue #144), invoked as `sh` so it runs in POSIX mode -- +# and under the target's BusyBox ash, the stricter interpreter, which stays +# shipped as /bin/ash. The host's /bin/sh is usually dash, which is a good +# POSIX proxy but is not the same interpreter as either. # # Env: # TZ_TEST_SH shell to run the script under (default: sh). May be a command # with arguments, e.g. +# TZ_TEST_SH="qemu-arm -L output/target output/target/usr/bin/bash --posix" # TZ_TEST_SH="qemu-arm -L output/target output/target/bin/busybox sh" # shellcheck disable=SC2030,SC2031 @@ -166,8 +169,8 @@ reset() { rm -f "$TZFILE" "$TZSTAMP" "$TZFILE.tmp"; rmdir "$LOCKDIR" 2>/dev/null # fire [reason] [if_up] -- one dhcpcd address event, sourced the way # dhcpcd-run-hooks sources it, INSIDE ${TEST_SH[@]}. Running it under the -# harness's own shell would make TZ_TEST_SH decorative and the "target BusyBox -# ash" CI leg a no-op that prints PASS. +# harness's own shell would make TZ_TEST_SH decorative and the target-shell CI +# legs (bash --posix, BusyBox ash) no-ops that print PASS. # # Defaults to the event that matters: BOUND with the interface up. Pass the # literal string "unset" as if_up to leave it undefined -- a case dhcpcd never