diff --git a/.agents/docs/2026-09-17-issue-660-glibc-line-binding-analysis.md b/.agents/docs/2026-09-17-issue-660-glibc-line-binding-analysis.md new file mode 100644 index 00000000..959f9a74 --- /dev/null +++ b/.agents/docs/2026-09-17-issue-660-glibc-line-binding-analysis.md @@ -0,0 +1,231 @@ +--- +subject: triage +status: landed +--- + +# Issue #660 分析:`glibc@2.44` 绑定在 2.44.3 发布后解析失败 + +- mcpp-community/mcpp#660(Sunrisepeak,2026-09-16,open) +- 触发事件:openxlings/xim-pkgindex#852(glibc 2.44.3,`latest` 前移),它是 openxlings/xlings#605 的根治修复 +- 分析日期:2026-09-17;本机 x86_64,host glibc 2.39;被测对象是发布物 `xim-x-mcpp/2026.9.15.1`(与使用方 CI 同版本) + +## 0. 结论 + +1. **issue 对机制的描述不成立。** mcpp 并不要求目录名恰好是 `2.44`。从 2026.8.27.x 起, + `payload_dir_for_version`(`src/xlings/xlings.cppm:1183`)的规则是:有同名目录就用它; + 否则**恰好一个**按分量细化的目录(`2.44.x`)也算答案;**两个及以上细化目录则拒绝**。 + #660 的真实触发条件是: + + > 绑定是一条版本线(`glibc@2.44`),store 里没有 `2.44` 目录,却有 **两个** `2.44.x` + > (缓存恢复的 `2.44.2` + 新下载的 `2.44.3`),并且 subos 视图里没有 libc, + > 于是 fixup 走到按版本查目录的那条分支。 + + 报错信息打印的是 `/2.44`,读起来像「必须精确匹配」,报告人正是据此得出了错误结论。 + 实际走到的是「两个细化目录,拒绝选择」这条分支(`post_install.cppm:422-429`)。 + +2. **「每台干净机器都失败」不成立,「开发机不复现」的解释也不对。** 只有 `2.44.3` 的干净 + store 能正常通过(本地实测 C3)。会不会失败取决于 store 的历史:只要 store 里有上一个修订 + 版且没有精确的 `2.44`,就会失败,无论这台机器是 CI 还是开发机。 + +3. **CI 为什么会得到一条版本线作为绑定。** 使用方 CI 的 mcpp home 实际运行的是 + xlings **2026.8.17.2**(日志:`Note vendored xlings 2026.8.17.2 is older than the pinned + 2026.9.14.1, but no newer source is available (keeping it ...)`)。这个版本低于 mcpp 注释里 + 写明的下限 2026.8.27.2;低于下限时,subos 的运行时取编译期常量 `glibc@2.44`。mcpp 把这个下限 + 称为「floor, not a preference」,但遇到旧 xlings 时只打印一条 Note 就继续执行。 + 新版 xlings 按索引解析,记录的是精确版本(本地实测:`glibc@2.44.3`,`runtime_source: index`)。 + +4. **#852 本身没有问题,它只是触发者。** 补丁语义正确,验证充分。但每发布一个打包修订版 + (`2.44.N`),凡是绑定为版本线、store 里又留着上一个修订版的环境,都会多出一个细化目录, + 从而踩中 mcpp 的歧义拒绝。2.44.2 发布时没有暴露,是因为当时各处 store 里还有精确的 `2.44`, + 精确匹配优先。 + +5. **使用方的绕过办法有效,但理由和评论里写的不同。** 在 `[xlings.workspace]` 里钉 + `xim:glibc@2.44` 能过,是因为精确目录优先;与「必须精确匹配」无关。更直接的办法见 §8。 + +## 1. 三个对象的关系 + +``` +xlings#605 node 的原生模块(sharp → libvips-cpp.so,DT_RUNPATH=$ORIGIN)找不到 libresolv.so.2 + ├─ xim-pkgindex#851 node 配方追加 NEEDED(缓解措施,只覆盖 node) + └─ xim-pkgindex#852 glibc 2.44.3:ld.so 把自身所在目录当默认库目录(根治);latest → 2.44.3 + │ + └─ mcpp#660 store 中 {2.44.2, 2.44.3} 并存 + 绑定为版本线 glibc@2.44 ⇒ fixup 拒绝 +``` + +## 2. mcpp 侧的机制(代码) + +`ensure_post_install_fixup`(`src/toolchain/post_install.cppm`,2026.9.15.1 与 main 相同) +在 Linux 上有两条分支: + +| 条件 | 分支 | 查找方式 | +|---|---|---| +| `resolve_runtime_binding` 在 subos 视图 `lib64/`/`lib/` 找到 `libc.so.6` | A | 取视图链接的**物理**目录,`runtimeId` 改写成物理版本(`binding.cppm:414-420`) | +| 视图里没有 libc(`libraryDirs` 为空) | B | `find_sandbox_glibc_lib` → `select_glibc_payload_lib` → `payload_dir_for_version(root, "2.44")` | + +分支 B 的解析规则(`xlings.cppm:1183-1223`): + +- 如果 `root/2.44` 是目录,直接返回它(精确匹配优先) +- 否则逐项扫描:分量数更多且前缀分量相等的目录算作细化;**第二个细化目录出现时返回 nullopt** +- 单测已经钉住了这条规则:`TwoRefinementsAreRefusedRatherThanChosenBetween`、 + `PayloadDirForVersion.TwoRefinementsAreNotAnAnswer`(`tests/unit/test_post_install.cpp:268,302`) + +还有两处与此相关的缺陷: + +- **报错文本有误导性**:`post_install.cppm:426-429` 打印的是 `glibcRoot / version`,既没有说明 + 这是「歧义」,也没有列出候选目录。#660 的全部误读都源于这一行。 +- **过时的注释**:`post_install.cppm:379-381` 仍写着 "glibc@2.44 means the `2.44` directory + and no other",与下面的实现矛盾。 +- **第三个按声明版本拼目录的调用点**:`binding.cppm:264` 的 `exact_declared_glibc_payload` + 只认精确目录,没有使用共享的解析器。CI 日志里每次构建都出现的 + `runtime binding glibc@2.44 has no loader path or library directory yet ... a second build + resolves it` 就来自这里。对版本线绑定来说,「第二次构建会解决」永远不会发生。 + `payload_dir_for_version` 的头注释写着「它有两个调用者,而它们曾分别出过错」, + 这里是尚未收拢的第三个。 + +## 3. 为什么 CI 得到的是版本线绑定,而且视图里没有 libc + +**绑定。** 使用方的 `install-mcpp.sh` 用 `xlings install mcpp@2026.9.15.1 -g` 安装 mcpp。 +xlings 包形态的 mcpp 不使用自包含 home,因此 home 是 `~/.mcpp`。 +`acquire_xlings_binary`(`src/fallback/xlings_binary.cppm`)的升级来源只有两个: +`MCPP_VENDORED_XLINGS` 与 `which xlings`。CI 系统上的 xlings 是 `v2026.8.17.2`, +所以 home 一直停留在 2026.8.17.2。然而同一个 mcpp 包里自带 +`registry/bin/xlings`,版本是 **2026.9.14.1**(本地实测),升级逻辑没有把它算作来源。 +2026.8.17.2 的 `manifest.cppm:55` 是 `DEFAULT_RUNTIME = "glibc@2.44"` 常量。 + +报告人在评论里说「用 xlings 2026.9.16.1 新建 subos 仍然记录 `glibc@2.44`」。这是另一条路径: +在隔离的 `XLINGS_HOME` 里没有同步索引,新版 xlings 回落到 `DEFAULT_RUNTIME_FALLBACK = "glibc@2.44"` +(此时 `runtime_source` 为 `fallback`,但评论贴出的 JSON 里没有这个字段)。 +两条路径结果相同,原因不同。 + +**视图。** 使用方的 `actions/cache` 只缓存 `~/.mcpp/registry/data/xpkgs`,不包含 xlings 的 +版本数据库和 `registry/subos`。每次运行时,xlings 看不到恢复出来的 payload 已经安装, +于是重新下载(16:01 那次绿色运行的日志里,在同一个缓存 key 下仍然出现了 `Downloading xim:glibc@2.44.2`)。 +至于视图为什么一直没有 libc,我的推断是:`mcpp test` 的 `[xlings.workspace]` 按项目模式供给, +glibc 被链接进了项目的 subos,而没有进 registry 的 default 视图。这一点**没有单独测量**; +但日志里的 `no loader path or library directory yet` 证明 fixup 当时确实走的是分支 B。 + +## 4. CI 证据:同一个缓存 key,只有索引变了,结果由绿转红 + +| run | 时间 | 缓存 key | 供给时下载的 glibc | 结果 | +|---|---|---|---|---| +| 35119235698 | 16:01 | `…-9ebd58…` | `xim:glibc@2.44.2` | 绿 | +| (xim-pkgindex#852 合入,17:13) | | | | | +| 35138626160 | 19:07 | `…-9ebd58…`(同一个) | `xim:glibc@2.44.3` | 红,27 秒后报 #660 错误 | +| 35147931933 | 20:41 | `…-eaa867…` | 恢复后打印:store 里只有 `2.44.2`;随后下载 `2.44.3` | 未钉 glibc 的 job 全红;钉了 `xim:glibc@2.44` 的 job 绿 | + +同一个缓存在 16:01 能通过、在 19:07 失败,说明 store 里没有精确的 `2.44` +(如果有,精确匹配优先,两次都会通过);而 20:41 的打印直接显示 store 里是 `2.44.2`。 +「缓存 `2.44.2` + 新下载 `2.44.3`」正是歧义的构成。 + +## 5. 本地复现(A/B,发布物 mcpp 2026.9.15.1) + +临时 `MCPP_HOME`,`MCPP_VENDORED_XLINGS=<包>/registry/bin/xlings`(2026.9.14.1), +每一步都执行 `mcpp toolchain install gcc@16.1.0`,触发 fixup。 + +| 步骤 | 绑定 | 视图里有 libc | store | 结果 | +|---|---|---|---|---| +| A | `glibc@2.44.3`(index) | 有 | 2.44.3 | 通过 | +| B1 | `glibc@2.44`(`xlings subos runtime`) | 有(链到 2.44.3) | 2.44.3 | 通过(分支 A) | +| B2 | 同上 | 有 | 2.44.2, 2.44.3 | 通过(分支 A,不看 store) | +| C2 | `glibc@2.44` | **无** | 2.44.2, 2.44.3 | **失败,与 #660 报错逐字相同** | +| C3 | `glibc@2.44` | 无 | 2.44.3 | 通过(唯一细化) | +| C4 | `glibc@2.44` | 无 | 2.44.2, 2.44.3 | **失败**(恢复 2.44.2 后再次复现) | +| C1 | `glibc@2.44` | 无 | 2.44 空壳, 2.44.2, 2.44.3 | 失败:`payload '…/2.44' is stale/incomplete` | + +C2/C3/C4 是判据:只改 store 里是否存在第二个细化目录,结果随之翻转。 + +B 系列还显示了一个附带现象:`xlings subos runtime glibc@2.44` 改写了声明,但视图仍然链接到 +2.44.3,因此 mcpp 在分支 A 中把物理身份当作 `glibc@2.44.3`。声明与视图不一致, +xlings 对此只给出一条 warn。 + +C1 中的 `2.44` 是第一次 `xlings install xim:glibc@2.44` 产生的空壳:目录里只有 `.xpkg.lua` +和 `.xpkg-install.json`,没有 payload。在同一个 home 里第二次安装就得到了完整 payload, +**原因未查明,也没有复现出第二次**。报告人第一条评论里看到的「钉 2.44 后 stale/incomplete」 +可能就是这个现象,而不是他后来撤回时给出的 `lib64` 符号链接解释。 + +## 6. xim-pkgindex#852 审查 + +**正确性。** 补丁在 `_dl_init_paths` 的系统目录初始化之后,只替换 `dirs[0]` 的 `dirname`/`dirnamelen`, +并同步更新 `max_dirnamelen`(`open_path` 按它分配 hwcaps 子目录的缓冲区)。它不改 cache、 +不改其余编译期目录,也不改 secure 模式下 `is_trusted_path_normalize` 使用的 `SYSTEM_DIRS`。 +两种进入方式分别处理:PT_INTERP 取 `l_libname->name`,直接调用取 `/proc/self/exe`。 +行为断言放在 `build-glibc.sh` 里,覆盖了三件事:只有 RUNPATH 的程序能启动,`--list` 从加载器目录解析, +宿主独有的 `libz.so.1` 仍不可达。补丁不生效时断言会失败,PR 里有验证记录。 +这是合适的判据,因为 `strings` 看不出差异。 + +**「目录里只有 glibc 自己的对象」的前提。** 这个前提要求 PT_INTERP 指向 payload 目录,而不是 +subos 视图(视图里汇集了所有包的库)。本机实测:xlings 的 node/python 和 mcpp 的 gcc,PT_INTERP +都是 `xpkgs/xim-x-glibc//lib{,64}/ld-linux-x86-64.so.2`,前提成立。如果将来有程序把 PT_INTERP +指向视图,这个补丁就会把整个视图变成默认目录。这个约束目前只写在补丁注释里,没有任何检查在执行它。 + +**残余风险(低)。** 实现依赖 rtld 内部的初始化顺序:`_dl_init_paths` 执行时 +`_dl_rtld_map.l_name` 是否已经设置,决定了走哪条分支。升级 glibc 时,build 脚本里的行为断言会拦住回归。 + +**跨仓影响(#660 的来源)。** PR 正文说「已有的 subos 仍绑定原来的 glibc」,只考虑了 xlings 侧。 +mcpp 的细化规则把「同一条版本线下出现新修订版」变成了一个会导致失败的事件。 +这个组合的判据应当是「先 A 后 B」:在已经装有 2.44.2 的 store 上安装 2.44.3,再做一次 mcpp 工具链解析。 +这项测试没有做。 + +**附带发现:2.44 发布物带着构建机的临时 RUNPATH。** 索引里 `2.44` 的归档 +(sha256 `0105292f…`,与索引一致)中有 15 个 ELF(`bin/iconv`、`locale`、`localedef`、`getent`、 +`makedb`、`sbin/zic`、`iconvconfig`、`libexec/getconf/*` 等)的 RUNPATH 是 +`/tmp/claude-1000/-home-speak-workspace-github-openxlings-xlings/…/scratchpad/realhome/…`, +安装后也没有被改写。`2.44.2` 和 `2.44.3` 中为 0 个。影响很小,但 `2.44` 恰好是 xlings 的 +`DEFAULT_RUNTIME_FALLBACK`。如果多用户主机上有人抢先创建了这个 `/tmp` 路径,就能向这些工具注入库。 + +## 7. 修复建议 + +### mcpp(#660 的归属仓) + +1. **让安装与绑定出自同一个答案。** `lifecycle.cppm:937` 与 `prepare` 的首次安装都用不带版本的 + `xim:glibc`(即 latest),而 fixup 读的是 subos 的声明。两处回答的是同一个问题。 + 应当按绑定的版本安装(`xim:glibc@`);如果绑定来源是 `fallback` + 或旧 xlings 写入的常量,就先用索引重新绑定(`xlings subos runtime `),再安装。 + **不建议**把歧义规则改成「取最高版本」,那只是换一种方式猜。 +2. **把 xlings 下限变成真正的下限。** 在 `candidate_source_version` 中加入可执行文件旁的 + `../registry/bin/xlings`(xlings 包形态的 mcpp 自带这个文件,版本正好等于 pin)。仍然低于下限时, + 至少要对依赖下限的操作(运行时绑定)给出错误或明确警告,而不是一条 Note。 +3. **报错写出事实。** 区分「没有候选」和「有 N 个候选,拒绝选择」,并列出候选目录名; + 同时更新 `post_install.cppm:379` 的过时注释。 +4. **收拢第三个调用点。** 让 `exact_declared_glibc_payload`(`binding.cppm:264`)走 + `payload_dir_for_version`,并审视「a second build resolves it」这句提示对版本线绑定是否成立。 +5. **判据。** 加一条 e2e:store 预置 `2.44.2`,绑定 `glibc@2.44`,视图无 libc;安装 `2.44.3` 后解析工具链。 + 修复前应为红,修复后为绿。已有单测只覆盖纯函数,覆盖不到「安装与绑定不一致」这个组合。 + +### xlings + +1. 回落绑定(`runtime_source: fallback`)在索引第一次可用时应自动重新解析,或者由 `self doctor` 报告; + 否则它会持续充当一条版本线。 +2. `subos runtime ` 改写声明后,应重新链接视图,或者拒绝在视图与声明不一致的状态下完成 + (§5 B 系列)。 +3. 查明 C1 中 `install xim:glibc@2.44` 只留下元数据空壳的原因(观测到一次)。 + +### xim-pkgindex + +1. 在 `glibc.lua` 的修订版策略注释中写明:发布新修订版会让绑定为版本线的 store 出现第二个细化目录; + 在 mcpp 的修复发布之前,这是一个跨仓事件。 +2. 考虑是否需要处理 `2.44` 归档里泄漏的临时 RUNPATH(条目只追加不修改,因此只能在注释里说明, + 或者让 xlings 回落常量不再指向它)。 + +### 使用方的即时绕过 + +以下任一即可,不需要 feature 开关: + +- 让 home 里的 xlings 达到 pin:CI 的 `XLINGS_VERSION` 升到 ≥ 2026.9.14.1,或者设置 + `MCPP_VENDORED_XLINGS=$(dirname $(readlink -f $(which mcpp)))/../registry/bin/xlings`。 + 然后确认 `registry/subos/default/.xlings.json` 里是 `runtime_source: index`,且 runtime 是精确版本。 +- 或者让缓存里不再同时存在两个修订版:修改缓存 key 的前缀使旧缓存失效,或者在恢复后删除 + `xim-x-glibc/2.44.2`。 + +评论里的第二个诉求(`cfg(host_os)`)本身是一个合理的需求,应当单独开 issue 评估。 +但 #660 不应以它为前提:工具链的 glibc 属于 mcpp 的契约,项目不应该需要去声明它。 + +## 更正(2026-09-17,后续实测) + +§0 第 3 点与 §7「使用方的即时绕过」中「让 home 里的 xlings 达到 pin 即可」**不成立**。 + +- 在满足下限的 xlings(2026.9.14.1)下,如果工具链由缓存命中而声明的运行时版本不在缓存中,fixup 同样失败(`RuntimeBinding glibc@2.44.3 requires payload …/2.44.3`)。 +- 更深一层的原因:mcpp 在 `lifecycle.cppm:937` / `prepare.cppm:3720` 安装 `xim:glibc` 的循环从未生效。`resolve_xpkg_path` 拒绝不带版本的目标,而调用方忽略了这个错误。 +- 修订后的根因与方案见 `2026-09-17-runtime-binding-multi-repo-plan.md`(v3)。 + +有效的即时绕过只剩一条:让缓存中不同时存在两个修订版,或让缓存包含当前声明的精确版本。 diff --git a/.agents/docs/2026-09-17-runtime-binding-multi-repo-plan.md b/.agents/docs/2026-09-17-runtime-binding-multi-repo-plan.md new file mode 100644 index 00000000..bb2adbd7 --- /dev/null +++ b/.agents/docs/2026-09-17-runtime-binding-multi-repo-plan.md @@ -0,0 +1,213 @@ +--- +subject: triage +status: landed +--- + +# 运行时绑定方案 v3:让 mcpp 真正安装它所声明的运行时 + +- 日期:2026-09-17(v3,取代同日的 v1 与 v2) +- 前置分析:`.agents/docs/2026-09-17-issue-660-glibc-line-binding-analysis.md`(其中 §7「使用方即时绕过」的第一条已被本文 §1 的实测推翻,见该文件末尾的更正) +- 范围:mcpp 一个 PR(必需);xim-pkgindex 一个 PR(判据门);xlings **不需要 PR**,只开一个独立 issue +- 状态:待 review + +--- + +## 0. 讨论记录:对 v2 三个问题的回答 + +### Q1 `xlings subos describe --json` 是做什么的,是否必要? + +v2 引入它有两个用途:(a) 把旧 subos 的运行时记录迁移成「版本 + payload 位置」的完整契约;(b) 作为 mcpp 查询这份契约的机器接口。 + +**结论:不必要,撤回。** + +- 机器接口:xlings 已有 NDJSON 的 `xlings interface `(目前 21 个 capability),mcpp 已经通过它调用 `install_packages`。如果将来确实需要查询,应当新增一个 **interface capability**,而不是新增 CLI 动词;CLI 是面向人的界面。 +- 迁移:v3 不需要迁移(见 Q3)。 + +### Q2 D2(离线 + 从未链接 + 旧契约 → 报错)的影响面 + +v3 中这条变为:「离线,且 subos 声明的那个运行时版本不在 store 中」时报错;报错会写明需要预先获取的精确坐标 `xim:glibc@`。三个条件必须**同时**成立才会触发: + +1. 处于离线状态(`--offline`、`MCPP_OFFLINE`,或网络不可达); +2. 声明的版本目录不存在。现实中只有两种来源: + - 旧 xlings(< 2026.8.27.2)建的 home,声明的是常量 `glibc@2.44`,store 里却只有 `2.44.x`; + - CI 缓存恢复了工具链,但缓存里的 glibc 不是当前声明的版本; +3. 视图中没有链接 libc(视图已链接时,身份取自链接,不查 store)。 + +影响的变化:在 v3 之前,这类 home 靠「store 里恰好只有一个 `2.44.x`」才能通过;一旦出现两个就是 #660。所以 v3 不是把能用的变成不能用,而是把**时而通过、时而失败**变成**确定报错并给出命令**。联网环境(包括普通 CI)不会触发这条:第一次运行就会把声明的版本安装好,之后永久满足条件 2。受影响的只有「从旧 home 或旧缓存直接进入离线运行」的空隔离环境,修复方法是把报错中的坐标加入它们的离线缓存。 + +### Q3 v2 的「运行时新契约」要解决什么,为什么现在撤回? + +**v2 想解决的问题**:mcpp 需要知道 subos 的 glibc payload 在哪里,而它是**推导**出来的:要么读视图里的链接,要么按版本号扫描 `xpkgs/xim-x-glibc/`。推导在版本线和多个修订版并存时会失败(#660)。v2 的思路是让 xlings 把 payload 位置直接写进 `subos_info`,mcpp 只读不推导。 + +**撤回的原因**:继续实测后发现,真正缺的不是「位置信息」,而是 **payload 从来没有被保证安装**: + +- xlings 在下限版本之上已经具备全部所需的语义: + - `self init` 按索引解析,写入具体版本(实测 E2:`glibc@2.44.3`,`runtime_source: index`); + - 安装时的优先级是「声明 > 激活 > 索引」:不带版本的 `xim:glibc` 或 `glibc@>=2.39` 依赖都会解析到 subos 声明的版本(xlings `xim/commands.cpp`,`subos_version_of_`); + - store 的布局 `xim-x-glibc/` 与视图链接,本身就是位置信息。 +- 缺口在 mcpp:它「安装声明的运行时」的那段代码**从来没有生效过**(§1 F2),payload 只是作为工具链依赖的**副作用**被装进来。工具链由缓存命中时,这个副作用就不会发生。 + +只要 mcpp 显式安装它声明的运行时,位置就恒等于 `xim-x-glibc/<声明版本>`,不需要新增字段,也不需要迁移。新契约成了冗余的第二份记录,撤回。 + +--- + +## 1. 根因(v3 修正版,带新证据) + +| # | 事实 | 证据 | +|---|---|---| +| F1 | mcpp 的「确保 sysroot payload」循环传入的是不带版本的 `xim:glibc` / `xim:linux-headers` | `lifecycle.cppm:937`,`prepare.cppm:3720` | +| F2 | `Fetcher::resolve_xpkg_path` 遇到不带版本的目标直接返回错误 `expected @`;调用方只在 debug 级别记录结果并忽略。**因此这段安装从未执行过** | `package_fetcher.cppm:1020-1025`;`mcpp -v` 实测:`installing dep: xim:glibc` 之后没有 fetcher 的 `resolve:` 行,而 `xim:gcc@16.1.0` 有 | +| F3 | 声明的运行时 payload 只能作为工具链依赖被 xlings 装进来。工具链带着 `.mcpp_ok` 命中(缓存恢复、全局拷贝)时,mcpp 不会调用 xlings,运行时也就不会被安装 | 实测 F-new 第 1 步 | +| F4 | 下限以上的 xlings:声明具体、按声明安装。下限以下(2026.8.17.2):声明是常量 `glibc@2.44`,依赖解析到 latest | 实测 E1 / E2;CI 日志 | +| F5 | mcpp 对 xlings 下限只打印 Note。xlings 包形态的 mcpp 自带满足下限的 xlings,但获取逻辑从不使用它 | `fallback/xlings_binary.cppm` | +| F6 | `payload_dir_for_version` 的细化规则,是 mcpp 为下限以下的 xlings 写的兼容层,并且掩盖了 F2 | 该函数头注释;#660 | + +**实测矩阵**(发布物 mcpp 2026.9.15.1;每组都是新建 home) + +| 组 | xlings | store 预置(模拟缓存) | 视图有无 libc | 结果 | +|---|---|---|---|---| +| E1 | 2026.8.17.2 | glibc 2.44.2 | 安装后链接到 2.44.3 | 绿(分支 A 掩盖了问题) | +| E2 | 2026.9.14.1 | glibc 2.44.2 | 安装后链接到 2.44.3 | 绿 | +| F-new 第 1 步 | 2026.9.14.1 | glibc 2.44.2 + **已有 gcc** | 无 | **红**:`RuntimeBinding glibc@2.44.3 requires payload …/2.44.3` | +| F-new 第 3 步 | 2026.9.14.1 | 同上,再由项目模式安装 glibc(2.44.3) | 链接到 2.44.3 | 绿 | +| 分析报告 C2/C4 | 2026.9.14.1 | `{2.44.2, 2.44.3}`,声明为版本线 | 无 | 红(#660 原文) | + +F-new 第 1 步说明:**仅升级 xlings 不能修复**;只要工具链是缓存命中而声明的运行时不在缓存里,新 xlings 也会失败。报错形式与 #660 不同,但是同一个根因。 + +**一句话根因**:mcpp 声明了运行时,却从不安装它;它依赖一个会被缓存绕过的副作用,再用目录扫描去猜副作用留下了什么。 + +--- + +## 2. 原则 + +1. **声明的东西由声明方保证存在。** mcpp 选定了 subos 的运行时,就负责让 xlings 安装那一个精确版本。 +2. **解析交给 xlings,查找只做精确匹配。** 版本语义(声明 > 激活 > 索引)只在 xlings 中存在一份;mcpp 按 `xim-x-glibc/<声明版本>` 精确读取,不做细化匹配,也不扫描。 +3. **下限就是下限。** 低于下限的 xlings 没有第 1 条所依赖的语义,因此必须拒绝,而不是提示后继续。 +4. **失败不能被吞掉。** 安装失败要作为错误返回,不能只写 debug 日志。 + +--- + +## 3. mcpp PR(必需) + +### 3.1 按声明安装运行时(修复 F1–F3) + +- 删除两处死代码循环(`lifecycle.cppm:937`、`prepare.cppm:3720`)。 +- 新增一个函数 `ensure_declared_runtime(cfg, binding)`: + - 触发条件:仅当 `runtimeId` 不是 hosted 运行时,并且 `xim-x-/` 中没有 loader; + - 动作:调用 `fetcher.resolve_xpkg_path("xim:" + runtimeId, autoInstall=true)`,例如 `xim:glibc@2.44.3`;这走的是现有的 `xlings interface install_packages` 通道; + - 失败时返回错误:`declared runtime glibc@2.44.3 is not installed and could not be installed: <原因>`;离线时附加 `pre-fetch xim:glibc@2.44.3`。 +- **调用位置**:在两个入口计算 runtime binding 快照的地方(`prepare` 与 `toolchain install`),解析之后发现 payload 缺失时调用一次,然后重新解析。五处 fixup 调用点都消费这份快照,因此不需要各自处理。 +- **正常路径零开销**:payload 已存在时不启动任何子进程。 +- `xim:linux-headers` 不纳入:它一直由工具链依赖带入(死循环从未安装过它),保持现状就等于不改变行为。已列入 §6 风险。 + +### 3.2 精确查找,删除兼容层(修复 F6) + +- `select_glibc_payload_lib` 改为只认 `root/`(恢复该函数注释中「glibc@2.44 means the `2.44` directory」的原始语义)。 +- 删除 `paths::payload_dir_for_version` 的细化分支。`probe.cppm` 的调用点同样改为精确查找,因为 3.1 保证了精确目录存在。 +- 更新对应单测:`ARequestResolvesToItsOneRefinement`、`TwoRefinements*` 改为断言精确匹配语义。 + +### 3.3 xlings 下限真正生效(修复 F4、F5) + +- 获取来源的顺序:发布物自带的 `/../registry/bin/xlings`(自包含 tarball 与 xlings 包两种形态)→ `MCPP_VENDORED_XLINGS` → `which xlings`。「只升不降」规则不变。 +- 获取之后仍低于 `kXlingsVersion`:返回错误,并给出获取命令。发布物形态必定带有满足下限的 xlings,因此这只可能发生在开发构建上。 +- 不需要提升 pin:2026.9.16.1 已经具备所需语义(declared 优先级从 2026.8.27.5 起就有)。 + +### 3.4 判据(e2e,全部使用发布形态) + +| # | 场景 | 期望 | +|---|---|---| +| 1 | F-new 第 1 步的形态:store 中有 gcc + glibc 2.44.2,新 xlings 声明 2.44.3 | 绿;store 中新增 `2.44.3`;fixup 使用它 | +| 2 | #660 的形态:声明版本线 `glibc@2.44`(旧 manifest 夹具),store `{2.44.2, 2.44.3}`,视图无 libc | 绿;xlings 按声明安装精确的 `2.44` | +| 3 | 与 1 相同,但加 `--offline` | 红;报错中含 `xim:glibc@2.44.3` | +| 4 | PATH 上有 xlings 2026.8.17.2,使用 xlings 包形态的 mcpp | home 中的 xlings 等于 pin | +| 5 | 正常 home 连续构建两次 | 第二次构建不启动 xlings 子进程(trace 断言) | +| 6 | macOS / Windows | 不调用 `ensure_declared_runtime`(trace 断言) | +| 7 | 修复前后各跑一次 1 与 2 | 修复前为红(确认判据真的在检查) | + +**代码量**:删除两个死循环、一个细化分支及相关单测;新增一个函数和获取来源顺序的一处改动。净删除。 + +--- + +## 4. xim-pkgindex PR(判据门,必需) + +运行时包(`exports.runtime` 非空)的 `latest` 发生变化时,CI 执行: + +1. 用已发布的 mcpp latest 建 home,安装工具链; +2. **模拟缓存**:只保留 `xpkgs/` 下的工具链与上一个运行时修订版,删除 `subos/`; +3. 发布新修订版(以 `latest` 指向它的索引运行),再次构建并运行一个 C++ 程序。 + +这个形态正是 #660 和 F-new 第 1 步共同的缺口。门必须覆盖「工具链由缓存命中」这一步,只做全新安装测不到问题。 + +合入顺序:先把门指向 mcpp 2026.9.15.1,在当前索引上确认为红;mcpp PR 发布后再改为 required。 + +--- + +## 5. xlings:不开 PR,只开独立 issue + +v3 所依赖的 xlings 语义在下限版本中都已存在。以下是观测到、但**不在 #660 路径上**的问题,各自开 issue: + +- `subos runtime glibc@2.44`:写入的声明是 `2.44`,激活的却是 `2.44.3`(分析报告 §5 B0)。推测原因是安装时读取的是**改写前**的声明;未验证。 +- 第一次 `install xim:glibc@2.44` 只留下元数据空壳(C1,仅观测到一次)。 +- `self init` 声明了运行时但不安装它,而 `subos new` 会安装:同一件事有两个创建入口,行为不一致。v3 的 3.1 已在 mcpp 侧覆盖;xlings 是否统一,由 xlings 自己决定。 + +--- + +## 6. 兼容性、无感升级与跨平台 + +| 情形 | 升级到 v3 的 mcpp 之后 | 可见变化 | +|---|---|---| +| 正常 home(声明版本已安装) | 无任何额外动作 | 无 | +| 旧 home,声明为常量 `glibc@2.44`,store 只有 2.44.x | 安装精确的 2.44,fixup 指向它;运行时契约变化,触发一次重建 | 一次下载(40 MB)加一次重建 | +| CI 缓存只含 `xpkgs`,声明的新修订版不在缓存中 | 安装声明版本 | 与今天的依赖下载量相当 | +| home 中的 xlings 低于下限(xlings 包形态的 mcpp) | 自动换成包内自带的 xlings | 一行 `Updating vendored xlings` | +| 离线,且声明的版本不在 store(Q2) | 报错,给出坐标 | 从时好时坏变为确定报错 | +| macOS / Windows / hosted 运行时 | 不进入 3.1 | 无 | +| Linux aarch64 | subos 不声明 glibc payload(glibc.lua 只提供 x86_64)→ 不进入 3.1 | 无;判据 6 扩展到这个平台 | +| 交叉构建 | 仍由 `needs_linux_sysroot_payloads(target)` 与声明共同决定 | 无 | + +**关于「旧 home 会装上 2.44 旧发布物」**:这是 xlings「声明优先」语义的直接结果,v3 刻意不在 mcpp 中改写。用户可以用 `xlings subos runtime glibc@2.44.3` 显式切换。 + +**风险** + +- `linux-headers` 与 glibc 是同一种形态(同样依赖副作用)。目前没有观测到失败,因为它只有一个版本。发布第二个版本前,需要按同样的方式评估。 +- 3.2 删除细化匹配后,凡是**没有经过** 3.1 的读者都会立刻暴露问题。需要用 grep 穷举 `xim-x-glibc` 的全部读取点,确认只有 `post_install` 和 `probe` 两处,且都在快照之后。 + +--- + +## 7. 发布顺序 + +``` +mcpp PR → release → (bot) 索引 bump +xim-pkgindex 门:先指向旧 mcpp 确认为红 → mcpp 发布后改为 required +使用方:删除 host-glibc feature 与 CI 中的 XLINGS_VERSION(自行清理) +``` + +两个 PR 互不阻塞;xlings 不在发布链上。 + +--- + +## 8. 需要 review 决定的问题 + +- **D1 3.2 是否立即删除细化匹配,还是保留一个版本作为过渡?** 建议立即删除。保留它就会继续掩盖 3.1 没有覆盖到的路径,正如它过去掩盖了 F2。 +- **D2 旧 home 的常量声明 `glibc@2.44`:遵从声明安装 2.44,还是在 mcpp 中把它当作未决定、改用 index?** 建议遵从声明。「未决定」的判断属于 xlings;mcpp 自行重新解释,会重新引入第二个解析者。 +- **D3 `linux-headers` 是否一并纳入 3.1?** 建议不纳入。它没有被声明,没有「声明的版本」可以安装;硬写版本只会把副作用换成猜测。 + +--- + +## 9. 实施记录(mcpp 2026.9.17.2) + +实现与 §3 的差异,以及原因: + +- **安装是惰性的。** `ensure_declared_runtime` 不在解析 binding 时执行,而在 `prepare` 中第一次调用需要 + C 运行时的 fixup 之前执行(`provide_runtime_payload`),且只对 `post_install_fixup_kind` 非空的 + 工具链执行。否则使用系统编译器的构建也会下载 glibc。`toolchain install` 在解析 binding 后、 + fixup 之前执行同一函数。 +- **安装失败不在这一步报错。** 失败写入 verbose 日志;需要 payload 的 fixup 报告缺失,并写出坐标 + `xim:glibc@`。这样使用 musl、PE 或系统工具链的构建在离线时不受影响。 +- **xlings 下限没有改为硬错误。** §3.3 的前提是「低于下限的 xlings 缺少 §3.1 依赖的语义」。实现中 + §3.1 按精确坐标安装,不依赖 xlings 的「声明优先」解析,所以下限以下的 xlings 同样得到正确结果。 + 硬错误只会让 PATH 上 xlings 较旧、且不使用发布布局的安装方式(例如只依赖系统 xlings 的打包渠道) + 无法运行。保留的改动是把随发布的 xlings 加入获取来源,并排在 PATH 之前。 +- **删除 `needs_linux_sysroot_payloads`。** 它唯一的调用点是那两个从未生效的循环。 +- **判据。** e2e 737 在已发布的 2026.9.17.1 上为红(报 #660 原文),在本次构建上为绿;e2e 687 的 + C 腿同样在已发布的二进制上为红、在本次构建上为绿。 diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 7e1392a5..2026b401 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -293 records. +295 records. ## By subject @@ -62,6 +62,8 @@ Records that declare one. Everything else is listed by date below. ### triage +- [运行时绑定方案 v3:让 mcpp 真正安装它所声明的运行时](2026-09-17-runtime-binding-multi-repo-plan.md) — landed +- [Issue #660 分析:`glibc@2.44` 绑定在 2.44.3 发布后解析失败](2026-09-17-issue-660-glibc-line-binding-analysis.md) — landed - [#655 implemented: one reading of a compile-flag element, the plan, and the ledger](2026-09-17-655-implementation-plan.md) — landed - [#655: the `arguments` array is not the command the build runs](2026-09-17-655-arguments-are-not-the-command-the-build-runs.md) — landed - [#646 to #649 implemented: the plan, its review from eight angles, and the ledger](2026-09-16-646-649-implementation-plan.md) — landed @@ -78,6 +80,8 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 +- [运行时绑定方案 v3:让 mcpp 真正安装它所声明的运行时](2026-09-17-runtime-binding-multi-repo-plan.md) — landed +- [Issue #660 分析:`glibc@2.44` 绑定在 2.44.3 发布后解析失败](2026-09-17-issue-660-glibc-line-binding-analysis.md) — landed - [#655 implemented: one reading of a compile-flag element, the plan, and the ledger](2026-09-17-655-implementation-plan.md) — landed - [#655: the `arguments` array is not the command the build runs](2026-09-17-655-arguments-are-not-the-command-the-build-runs.md) — landed - [#646 to #649 implemented: the plan, its review from eight angles, and the ledger](2026-09-16-646-649-implementation-plan.md) — landed diff --git a/CHANGELOG.md b/CHANGELOG.md index c2ea86d6..bf56acc0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,32 @@ ## [Unreleased] +### 声明的 C 运行时由 mcpp 安装,查找只做精确匹配:#660(2026.9.17.2) + +xim-pkgindex#852 发布 glibc 2.44.3 之后,CI 缓存只恢复 `registry/data/xpkgs` 的环境报 +`toolchain post-install fixup: selected RuntimeBinding glibc@... requires payload ...`。根因在 mcpp: +默认 SubOS 按精确版本声明 glibc,工具链 fixup 按该版本的 payload 目录打补丁,但没有任何步骤安装 +这个 payload。`lifecycle` 与首次运行中本应安装它的两个循环传入不带版本的 `xim:glibc`, +`resolve_xpkg_path` 拒绝了这种目标,而错误只写入 debug 日志。payload 因此只在 xlings 把它作为 +工具链依赖安装时才存在;从缓存恢复的工具链不会重新安装,于是缓存中若是另一个修订版,声明的 +payload 就不存在。按版本线接受 `2.44.x` 的目录扫描掩盖了这一点,直到同时出现两个修订版。 +分析与方案:`.agents/docs/2026-09-17-issue-660-glibc-line-binding-analysis.md`、 +`.agents/docs/2026-09-17-runtime-binding-multi-repo-plan.md`。 + +- **声明的运行时在第一次需要它的 fixup 之前安装。** `ensure_declared_runtime` 通过 xlings 安装 + `xim:glibc@<声明版本>` 并重新解析 binding。它只在 binding 找不到 payload、宿主为 Linux、提供者为 + `glibc`、且工具链是 `gcc` 或 `llvm` payload 时执行;常态下不启动任何进程,使用系统、musl 或 PE + 工具链的构建不下载任何东西。(单测 `DeclaredRuntime.*`,e2e 737) +- **查找只做精确匹配。** `glibc@` 只对应 `xim-x-glibc/`;删除 `payload_dir_for_version` 的 + 版本线细化,相邻修订版不再作为替代。缺失时报错写出需要提供的坐标 `xim:glibc@`, + 与「目录存在但没有 loader」的报错区分。(单测 `GlibcPayload.*`) +- **随 mcpp 发布的 xlings 是获取来源。** 以 xlings 包形式安装的 mcpp 使用 `~/.mcpp`, + 此前只从 `MCPP_VENDORED_XLINGS` 与 PATH 获取 xlings,PATH 上的 xlings 低于 pin 时保留旧版本 + 并打印一条 Note。现在 `/registry/bin/xlings` 排在 PATH 之前。(e2e 687 C) +- 删除 `needs_linux_sysroot_payloads`:它的两个调用点就是上述从未生效的循环。 +- **升级影响。** 声明的 payload 已安装的 home 没有变化。缺少该 payload 的 home 在下一次需要它 + 的构建中下载一次(约 40 MB);离线时报错并给出坐标。 + ### 编译 flag 列表元素的一种读法,以及与构建一致的编译数据库:#655(2026.9.17.1) `mcpp emit build-database` 与 `compile_commands.json` 的 `arguments` 此前把 flag 文本经一种 diff --git a/docs/91-toolchain-internals.md b/docs/91-toolchain-internals.md index f7b615cd..41c91288 100644 --- a/docs/91-toolchain-internals.md +++ b/docs/91-toolchain-internals.md @@ -476,6 +476,18 @@ Post-install alignment follows the same identity rule: `glibc@2.44` resolves only `/xim-x-glibc/2.44/{lib64,lib}`. A missing/stale exact payload is an error; another installed version is never a fallback. +The exact payload is installed by mcpp before the first fixup that patches +against it (`ensure_declared_runtime`, mcpp#660). The step runs only when the +binding locates no payload, only on Linux, only for the `glibc` provider, and +only for a toolchain whose fixup consumes a C runtime (`gcc` and `llvm` +payloads), so a build with a system, musl or PE toolchain downloads nothing. It +installs `xim:glibc@` through xlings and re-resolves the +binding. Before this step existed, the payload was present only when xlings had +installed it as a dependency of a toolchain; a toolchain restored from a CI +cache is not reinstalled, so a cache holding another glibc revision left the +declared payload absent. When the install is impossible, for example offline, +the fixup error names the coordinate `xim:glibc@` to provide. + ### 6.2 The permitted locations of a runtime search path (`runtime_env_contract.cppm`) There are two ways to tell a loader where to look, and they differ by blast diff --git a/docs/zh/91-toolchain-internals.md b/docs/zh/91-toolchain-internals.md index ec0f477f..f935c804 100644 --- a/docs/zh/91-toolchain-internals.md +++ b/docs/zh/91-toolchain-internals.md @@ -387,6 +387,14 @@ verdict 以 `.mcpp-runtime-verdicts.json` 存在 `build.ninja` 旁,键包含产 `/xim-x-glibc/2.44/{lib64,lib}`。精确 payload 缺失/陈旧就是错误,其他已安装版本 永远不是回退项。 +精确 payload 由 mcpp 在第一次需要它的 fixup 之前安装(`ensure_declared_runtime`, +mcpp#660)。这一步只在 binding 找不到 payload、宿主为 Linux、提供者为 `glibc`,且工具链的 +fixup 需要 C 运行时(`gcc` 与 `llvm` payload)时执行;因此使用系统、musl 或 PE 工具链的构建 +不会下载任何东西。它通过 xlings 安装 `xim:glibc@<声明的版本>`,然后重新解析 binding。 +在这一步出现之前,payload 只有在 xlings 把它作为某个工具链的依赖安装时才存在;从 CI 缓存 +恢复的工具链不会重新安装,于是缓存中若是另一个 glibc 修订版,声明的 payload 就不存在。 +无法安装时(例如离线),fixup 的报错会写出需要预先提供的坐标 `xim:glibc@<版本>`。 + ### 6.2 运行时搜索路径的允许位置(`runtime_env_contract.cppm`) 告诉 loader「去哪找」有两条通道,差别不在便利性,而在**波及范围**: diff --git a/mcpp.toml b/mcpp.toml index d6b414bb..ad1b2108 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.17.1" +version = "2026.9.17.2" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index f0532680..aa8b74ef 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.17.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.17.2"; } // namespace mcpp diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 9f17eae7..7694d184 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -2507,10 +2507,31 @@ prepare_build(bool print_fingerprint, return bin.string(); }(); - const auto runtimePayload = runtimeBindingSnapshot.libc.value_or(""); - const auto runtimeLibDir = runtimeBindingSnapshot.libraryDirs.empty() + auto runtimePayload = runtimeBindingSnapshot.libc.value_or(""); + auto runtimeLibDir = runtimeBindingSnapshot.libraryDirs.empty() ? std::filesystem::path{} : runtimeBindingSnapshot.libraryDirs.front(); + // THE DECLARED RUNTIME PAYLOAD IS PROVIDED BEFORE THE FIRST FIXUP THAT + // CONSUMES IT (mcpp#660), and not earlier: a build whose toolchain needs + // no C runtime payload must not download one. At most once per build. An + // inherited binding is not exempt, because the parent build may have used + // a toolchain that needed no payload. The two values derived above are + // refreshed with the binding, because detection and the fingerprint read + // them after the fixups. + bool runtimePayloadProvided = false; + auto provide_runtime_payload = [&](const mcpp::toolchain::XimToolchainPackage& pkg) { + if (runtimePayloadProvided) return; + if (mcpp::toolchain::post_install_fixup_kind(pkg).empty()) return; + runtimePayloadProvided = true; + auto cfgP = get_cfg(); + if (!cfgP) return; + if (!mcpp::toolchain::ensure_declared_runtime(**cfgP, runtimeBindingSnapshot)) + return; + runtimePayload = runtimeBindingSnapshot.libc.value_or(""); + runtimeLibDir = runtimeBindingSnapshot.libraryDirs.empty() + ? std::filesystem::path{} : runtimeBindingSnapshot.libraryDirs.front(); + }; + // mcpp#427: a toolchain fixup that could not run is a DEGRADATION, not a // failure — the build continues without it. But it has to be said, or the // eventual `stdlib.h: No such file or directory` arrives with no way to @@ -3520,6 +3541,7 @@ prepare_build(bool print_fingerprint, // manifest [toolchain] path previously ran none, so a freshly // auto-installed payload kept its stale install-time cfg / // unpatched runtime libs. + provide_runtime_payload(pkg); if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( **cfg, payload->root, pkg, runtimeBindingSnapshot.runtimeId, runtimeLibDir); !fixed) @@ -3713,14 +3735,6 @@ prepare_build(bool print_fingerprint, mcpp::fetcher::Fetcher fetcher(**cfg); mcpp::fetcher::InstallProgressHandler progress; - // The glibc default toolchain needs the sysroot payloads (C library + - // kernel headers). One derivation, shared with `toolchain install` — - // see registry.cppm for what the two spellings used to disagree about. - if (mcpp::toolchain::needs_linux_sysroot_payloads(defaultParsed->target)) { - for (auto dep : {"xim:glibc", "xim:linux-headers"}) { - (void)fetcher.resolve_xpkg_path(dep, /*autoInstall=*/true, &progress); - } - } auto payload = fetcher.resolve_xpkg_path(defaultPkg.target(), /*autoInstall=*/true, &progress); if (!payload) { @@ -3746,6 +3760,7 @@ prepare_build(bool print_fingerprint, // `mcpp toolchain install` performs — without it a fresh sandbox // gcc cannot find the C library (stdlib.h: No such file or // directory) and a fresh llvm keeps its stale install-time cfg. + provide_runtime_payload(defaultPkg); if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( **cfg, payload->root, defaultPkg, runtimeBindingSnapshot.runtimeId, runtimeLibDir); !fixed) @@ -4317,6 +4332,7 @@ prepare_build(bool print_fingerprint, gnuPkg.target(), mcpp::toolchain::payload_frontend_dir(payloadR->root, gnuPkg).string())); } + provide_runtime_payload(gnuPkg); if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( **cfgR, payloadR->root, gnuPkg, runtimeBindingSnapshot.runtimeId, runtimeLibDir); !fixed) @@ -4539,6 +4555,7 @@ prepare_build(bool print_fingerprint, pkg.target(), mcpp::toolchain::payload_frontend_dir(payload->root, pkg).string())); } + provide_runtime_payload(pkg); if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( **cfgH, payload->root, pkg, runtimeBindingSnapshot.runtimeId, runtimeLibDir); !fixed) diff --git a/src/fallback/xlings_binary.cppm b/src/fallback/xlings_binary.cppm index 0964d103..9a11548b 100644 --- a/src/fallback/xlings_binary.cppm +++ b/src/fallback/xlings_binary.cppm @@ -2,8 +2,9 @@ // // Tries multiple strategies to obtain the xlings binary: // 1. MCPP_VENDORED_XLINGS env var (explicit override) -// 2. system `which xlings` -// 3. Fail with user-facing instructions +// 2. the xlings released with this mcpp, `/registry/bin/xlings` +// 3. system `which xlings` +// 4. Fail with user-facing instructions module; #include @@ -22,10 +23,23 @@ export namespace mcpp::fallback { // The version already vendored at destBin, or empty when it cannot be read. std::string vendored_xlings_version(const std::filesystem::path& bin); +// The xlings that was released together with the running mcpp, when this mcpp +// runs from its release layout `/bin/mcpp` and the file exists. Empty +// otherwise, and empty when it is `destBin` itself (a self-contained home is +// that layout, so its vendored binary is already the released one). +// +// WHY IT IS A SOURCE (mcpp#660). A release pins an xlings version and ships +// that binary beside the mcpp executable. A home that is not the release +// directory (an mcpp installed as an xlings package uses `~/.mcpp`) consulted +// only the override variable and the PATH, so on a machine whose system +// xlings was older than the pin it kept the older one and printed a note. The +// released binary is the one that satisfies the pin by construction. +std::filesystem::path released_xlings_source(const std::filesystem::path& destBin); + // The version the acquisition chain WOULD install, without installing it. // Empty when nothing is available. Replacing a vendored binary is only an // improvement when this is newer than what is already there. -std::string candidate_source_version(); +std::string candidate_source_version(const std::filesystem::path& destBin = {}); // True when `have` is strictly older than `want`, comparing dot-separated // numeric components. Anything unparseable answers false -- a version this @@ -67,7 +81,7 @@ acquire_xlings_binary(const std::filesystem::path& destBin, bool quiet = false, // re-acquired, which replaced 2026.8.2.1 with the system's 0.4.51 -- // older still, and equally missing the feature the check exists to // restore. Look before leaping. - auto candidate = candidate_source_version(); + auto candidate = candidate_source_version(destBin); if (candidate.empty() || !version_is_older(have, candidate)) { // stderr, not stdout. This is a remark about the environment, // not output of the command that happens to be running -- and @@ -122,7 +136,25 @@ acquire_xlings_binary(const std::filesystem::path& destBin, bool quiet = false, } } - // 2. Copy from system (`which xlings`) + // 2. The xlings released with this mcpp. Ahead of the system copy, which + // may be any version (see released_xlings_source). + if (auto released = released_xlings_source(destBin); !released.empty()) { + std::filesystem::copy_file(released, destBin, + std::filesystem::copy_options::overwrite_existing, ec); + if (!ec) { + std::filesystem::permissions(destBin, + std::filesystem::perms::owner_exec + | std::filesystem::perms::group_exec + | std::filesystem::perms::others_exec, + std::filesystem::perm_options::add, ec); + if (!quiet) print_status("Bundled", + std::format("xlings (released with this mcpp: {})", released.string())); + return destBin; + } + ec.clear(); + } + + // 3. Copy from system (`which xlings`) auto xlings_name = std::string("xlings") + std::string(mcpp::platform::exe_suffix); auto sysXlings = mcpp::platform::fs::which(xlings_name); if (sysXlings) { @@ -215,12 +247,26 @@ bool version_is_older(std::string_view have, std::string_view want) { } -std::string candidate_source_version() { +std::filesystem::path released_xlings_source(const std::filesystem::path& destBin) { + auto exe = mcpp::platform::fs::self_exe_path(); + if (exe.empty() || exe.parent_path().filename() != "bin") return {}; + auto released = exe.parent_path().parent_path() / "registry" / "bin" + / (std::string("xlings") + std::string(mcpp::platform::exe_suffix)); + std::error_code ec; + if (!std::filesystem::is_regular_file(released, ec)) return {}; + if (!destBin.empty() && std::filesystem::equivalent(released, destBin, ec)) + return {}; + return released; +} + +std::string candidate_source_version(const std::filesystem::path& destBin) { if (const char* e = std::getenv("MCPP_VENDORED_XLINGS"); e && *e) { std::error_code ec; if (std::filesystem::exists(std::filesystem::path(e), ec)) return vendored_xlings_version(std::filesystem::path(e)); } + if (auto released = released_xlings_source(destBin); !released.empty()) + return vendored_xlings_version(released); if (auto sys = mcpp::platform::fs::which( std::string("xlings") + std::string(mcpp::platform::exe_suffix))) return vendored_xlings_version(*sys); diff --git a/src/toolchain/lifecycle.cppm b/src/toolchain/lifecycle.cppm index 29e32f0a..ed30f1dd 100644 --- a/src/toolchain/lifecycle.cppm +++ b/src/toolchain/lifecycle.cppm @@ -840,6 +840,57 @@ int attach_target_arg(mcpp::toolchain::ToolchainSpec& spec, return 0; } +// THE RUNTIME A SUBOS DECLARES IS INSTALLED BY THE READER OF THE DECLARATION +// (mcpp#660). +// +// A SubOS declares its C runtime by exact version (`glibc@2.44.3`), and the +// toolchain fixup patches against the payload directory of that version. The +// declaration was read; the payload was never installed on purpose. The two +// install loops meant to do it passed `xim:glibc` without a version, which +// `Fetcher::resolve_xpkg_path` rejects, and the rejection was logged at debug +// level and discarded. The payload therefore arrived only as a side effect of +// xlings installing a toolchain whose recipe depends on glibc. A toolchain +// restored from a cache is not installed again, so the side effect did not +// happen and the fixup found no payload for the declared version. A directory +// scan that accepted any `2.44.x` for `2.44` concealed this until two +// revisions were present at once. +// +// This function installs the declared payload by its exact coordinate and +// re-resolves `binding` in place. It chooses nothing: the version is the +// SubOS's, and xlings performs the install. It does nothing when the binding +// already locates a payload, off Linux, and for a provider other than glibc, +// so a home in its normal state starts no process. +// +// A failed install is reported at verbose level and is not an error here. The +// consumer that needs the payload, the fixup, reports its absence with the +// exact coordinate, and a caller whose toolchain needs no C runtime payload +// is unaffected. Returns whether `binding` changed. +export bool ensure_declared_runtime(const mcpp::config::GlobalConfig& cfg, + mcpp::platform::runtime::RuntimeBinding& binding) { + namespace rt = mcpp::platform::runtime; + if constexpr (!mcpp::platform::is_linux) return false; + if (!binding.libraryDirs.empty()) return false; + if (rt::runtime_provider(binding.runtimeId) != "glibc") return false; + + const auto target = "xim:" + binding.runtimeId; + mcpp::log::verbose("toolchain", std::format( + "declared runtime {} has no installed payload; installing {}", + binding.runtimeId, target)); + mcpp::fetcher::Fetcher fetcher(cfg); + mcpp::fetcher::InstallProgressHandler progress; + if (auto installed = fetcher.resolve_xpkg_path(target, /*autoInstall=*/true, &progress); + !installed) { + mcpp::log::verbose("toolchain", std::format( + "installing the declared runtime {} failed: {}", + target, installed.error().message)); + return false; + } + auto again = rt::resolve_runtime_binding(binding.selection, {}, cfg.xlingsHome()); + if (!again) return false; + binding = std::move(*again); + return true; +} + // `mcpp toolchain install [--target ]` — install + fixups. export int toolchain_install(const mcpp::config::GlobalConfig& cfg, const std::string& pos0, const std::string& pos1, @@ -928,20 +979,6 @@ export int toolchain_install(const mcpp::config::GlobalConfig& cfg, mcpp::fetcher::Fetcher fetcher(cfg); mcpp::fetcher::InstallProgressHandler progress; - // Ensure sysroot dependencies (glibc, linux-headers) are installed: - // the C library and kernel headers a glibc-targeting compile needs. - // The rule itself lives in registry.cppm — prepare's first-run - // install asks the same question and used to answer it in its own - // words, with one term missing. - if (mcpp::toolchain::needs_linux_sysroot_payloads(spec->target)) { - for (auto dep : {"xim:glibc", "xim:linux-headers"}) { - mcpp::log::verbose("toolchain", std::format("installing dep: {}", dep)); - auto depPayload = fetcher.resolve_xpkg_path(dep, /*autoInstall=*/true, &progress); - mcpp::log::debug("toolchain", std::format("dep {} result: {}", - dep, depPayload ? "ok" : depPayload.error().message)); - } - } - mcpp::log::verbose("toolchain", std::format("installing main: {}", pkg.target())); // A previous `remove` may have parked a held payload beside this one; // by now whatever held it has exited, so free the bytes before adding @@ -1037,6 +1074,8 @@ export int toolchain_install(const mcpp::config::GlobalConfig& cfg, std::filesystem::path runtimeLibDir; if (auto rb = mcpp::platform::runtime::resolve_runtime_binding( mcpp::xlings::runtime::RuntimeSelection{}, {}, cfg.xlingsHome())) { + if (!mcpp::toolchain::post_install_fixup_kind(pkg).empty()) + ensure_declared_runtime(cfg, *rb); runtimeId = rb->runtimeId; if (!rb->libraryDirs.empty()) runtimeLibDir = rb->libraryDirs.front(); } // a binding that cannot be resolved degrades below, it does not stop diff --git a/src/toolchain/post_install.cppm b/src/toolchain/post_install.cppm index 8e0f49ab..ccb6a1b0 100644 --- a/src/toolchain/post_install.cppm +++ b/src/toolchain/post_install.cppm @@ -376,10 +376,18 @@ export void fixup_clang_cfg(const std::filesystem::path& payloadRoot, } } -// Resolve one glibc payload from the RuntimeBinding identity. This is a -// semantic exact lookup, never a directory-order choice: glibc@2.44 means the -// `2.44` directory and no other. Exported so the #392 regression stays pinned -// by a pure unit test without installing a toolchain. +// Resolve one glibc payload from the RuntimeBinding identity. The lookup is +// exact: `glibc@2.44.3` means the `2.44.3` directory and no other, which is +// the name xlings gives the payload of that version. Exported so the rule is +// pinned by a pure unit test without installing a toolchain. +// +// THERE IS NO FALLBACK TO A NEIGHBOURING VERSION. A scan that accepted any +// `2.44.x` for `2.44` existed to tolerate a payload installed as a side effect +// of some other install; it concealed that the declared payload itself was +// never installed, and it refused as soon as two revisions were present +// (mcpp#660). The declared payload is now installed by +// `ensure_declared_runtime` before any fixup, so an absent directory is a +// fact to report, with the coordinate that provides it. export std::expected select_glibc_payload_lib(const std::filesystem::path& glibcRoot, std::string_view runtimeId) { @@ -410,30 +418,21 @@ select_glibc_payload_lib(const std::filesystem::path& glibcRoot, runtimeId)); } - // ONE RESOLVER, TWO CALLERS. `payload_dir_for_version` also answers - // `probe`'s compile-side discovery; see its own header for why a request - // and a resolution are two vocabularies, and why a unique component-wise - // refinement is an answer while a directory-order pick is not. - // - // THE FIRST VERSION OF THIS FIX SPELLED IT HERE, and that left the other - // caller: the toolchain then installed and the compile line came out - // without the glibc include directory, which reads as - // `features.h: No such file` from inside libstdc++'s own headers. - auto payload = mcpp::xlings::paths::payload_dir_for_version( - glibcRoot, version); - if (!payload) { + const auto payload = glibcRoot / std::string(version); + std::error_code ec; + if (!std::filesystem::is_directory(payload, ec)) { return std::unexpected(std::format( - "selected RuntimeBinding {} requires payload '{}', but no installed " - "payload is its resolution; mcpp will not fall back to another " - "directory entry", - runtimeId, (glibcRoot / std::string(version)).string())); + "the declared runtime {} is not installed: '{}' does not exist. " + "It is installed on first use when the network is available; for an " + "offline build, provide `xim:{}` in the xlings store beforehand", + runtimeId, payload.string(), runtimeId)); } - auto lib = payload_lib_dir_with_loader(*payload); + auto lib = payload_lib_dir_with_loader(payload); if (lib.empty()) { return std::unexpected(std::format( "selected RuntimeBinding {} payload '{}' is stale/incomplete: no " "dynamic loader was found under lib64/ or lib/", - runtimeId, payload->string())); + runtimeId, payload.string())); } return lib; } @@ -545,18 +544,28 @@ export struct FixupOutcome { std::string skippedReason; // non-empty ⇒ degraded, and why }; +// WHICH TOOLCHAIN PAYLOADS THE FIXUP PATCHES AGAINST A C RUNTIME, stated once. +// +// Empty for every payload the fixup leaves alone. On Windows it is always +// empty: a PE toolchain has nothing to patch. Callers that must provide the +// runtime payload before a fixup ask this rather than repeating the dispatch, +// so a toolchain that needs no C runtime payload never causes one to be +// installed. +export std::string_view post_install_fixup_kind(const XimToolchainPackage& pkg) { + if constexpr (mcpp::platform::is_windows) return {}; + if (pkg.needsGccPostInstallFixup) return "gcc"; + if (pkg.ximName == "llvm") return "llvm"; + return {}; +} + export std::expected ensure_post_install_fixup(const mcpp::config::GlobalConfig& cfg, const std::filesystem::path& payloadRoot, const XimToolchainPackage& pkg, std::string_view runtimeId = {}, const std::filesystem::path& selectedRuntimeLibDir = {}) { - std::string kind; - if (pkg.needsGccPostInstallFixup) kind = "gcc"; - else if (pkg.ximName == "llvm") kind = "llvm"; - else return FixupOutcome{}; - if constexpr (mcpp::platform::is_windows) - return FixupOutcome{}; // PE world: no fixups + const std::string kind{post_install_fixup_kind(pkg)}; + if (kind.empty()) return FixupOutcome{}; // Ownership guard: payloads inherited via symlink from another MCPP_HOME // are not ours to patch — their owner already ran the fixup, and patching diff --git a/src/toolchain/probe.cppm b/src/toolchain/probe.cppm index 44ab9de6..b9379d72 100644 --- a/src/toolchain/probe.cppm +++ b/src/toolchain/probe.cppm @@ -400,28 +400,29 @@ payload_root_for_binding(const std::filesystem::path& compilerBin, const auto name = binding.substr(0, at); const auto version = std::string(binding.substr(at + 1)); - // THE DIRECTORY IS NAMED AFTER WHAT THE REQUEST RESOLVED TO, NOT AFTER - // THE REQUEST. `payload_dir_for_version` is the one answer to that, shared - // with the toolchain post-install fixup — see its header. + // THE DIRECTORY IS NAMED BY THE EXACT VERSION. The binding carries the + // version whose payload `ensure_declared_runtime` installed, so no nearby + // version is accepted in its place (mcpp#660). // - // THIS SITE'S FAILURE DOES NOT NAME A VERSION. The fixup at least says - // which payload it wanted; here the include directory is simply never - // added, and what the user reads comes from inside libstdc++: + // THIS SITE'S FAILURE DOES NOT NAME A VERSION. The fixup says which + // payload it wanted; here the include directory is simply never added, + // and what the user reads comes from inside libstdc++: // // bits/os_defines.h:39: fatal error: features.h: No such file - // - // Measured 2026-08-27 on openkal-musl's CI after the fixup alone was fixed. + const auto exact = [&](const std::filesystem::path& xpkgs) + -> std::optional { + auto root = xpkgs / std::format("xim-x-{}", name) / version; + std::error_code ec; + if (std::filesystem::is_directory(root, ec)) return root; + return std::nullopt; + }; // Compiler siblings: <...>/xpkgs/xim-x-/ if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { - if (auto root = mcpp::xlings::paths::payload_dir_for_version( - *xpkgs / std::format("xim-x-{}", name), version)) - return *root; + if (auto root = exact(*xpkgs)) return *root; } // Active home. if (auto xpkgs = mcpp::xlings::paths::active_home_xpkgs()) { - if (auto root = mcpp::xlings::paths::payload_dir_for_version( - *xpkgs / std::format("xim-x-{}", name), version)) - return *root; + if (auto root = exact(*xpkgs)) return *root; } return std::nullopt; } diff --git a/src/toolchain/registry.cppm b/src/toolchain/registry.cppm index 1314f41a..07fa32b0 100644 --- a/src/toolchain/registry.cppm +++ b/src/toolchain/registry.cppm @@ -381,26 +381,6 @@ resolve_managed_msvc(const mcpp::xlings::Env& env, const XimToolchainPackage& pkg, bool identifyVersion = true); -// Does installing a toolchain FOR THIS TARGET additionally need the Linux -// sysroot payloads (`xim:glibc` + `xim:linux-headers`)? -// -// THE SINGLE DERIVATION. It was two, and they were not equivalent while a -// comment on one of them said "mirrors the guard on the other": -// -// lifecycle !musl && !pe && !windows-host && !macos-host -// prepare !macos-host && !windows-host && !musl -// -// The PE term was missing from the second. It happens to be unreachable today -// (first-run never selects a PE target on Linux), which is what let the -// divergence sit there — a latent difference between two spellings of one -// rule is exactly the state that becomes a bug the moment either side moves. -// -// Decided by the TARGET, not the payload name: musl targets are -// self-contained, PE targets (native MinGW and the Linux-hosted cross alike) -// bring their own CRT, and a non-Linux host never needs a Linux sysroot at -// all. -bool needs_linux_sysroot_payloads(const triple::Triple& target); - // Can THIS host serve that target — is there an installable payload for the // (host, target) pair? Empty target = host target, always serviceable. // @@ -1220,11 +1200,6 @@ bool is_system_toolchain(const ToolchainSpec& spec) { && (spec.version.empty() || spec.version == "system"); } -bool needs_linux_sysroot_payloads(const triple::Triple& target) { - if constexpr (!mcpp::platform::is_linux) return false; - return !target.is_musl() && !target.is_pe(); -} - bool host_can_serve(const triple::Triple& target) { if (target.empty()) return true; // host target diff --git a/src/xlings/xlings.cppm b/src/xlings/xlings.cppm index c6c120d7..fe3162f3 100644 --- a/src/xlings/xlings.cppm +++ b/src/xlings/xlings.cppm @@ -230,35 +230,6 @@ namespace paths { find_home_tool(std::string_view tool, std::string_view requiredRelPath = {}); - // WHICH INSTALLED PAYLOAD DIRECTORY ANSWERS A REQUESTED VERSION. - // - // A request and a resolution are two vocabularies for one fact. A - // RuntimeBinding carries the DECLARED version (`glibc@2.44`); xlings names - // the payload directory after what that request RESOLVED to (`2.44.2`). - // They coincide until the index moves a package within a series, and then - // every lookup that spells the directory by the declared version stops - // finding it. - // - // STATED ONCE HERE BECAUSE IT HAS TWO CALLERS AND THEY FAILED - // SEPARATELY. `post_install`'s toolchain fixup and `probe`'s compile-side - // payload discovery each spelled it themselves; fixing one left the other, - // and the second failure did not name a version at all — it read - // - // bits/os_defines.h:39: fatal error: features.h: No such file - // - // because the glibc include directory had simply not been added. - // - // A REFINEMENT, NOT A DIRECTORY-ORDER PICK. `2.44.2` is what the request - // `2.44` resolved to: its version COMPONENTS begin with the requested ones. - // `2.4` does not answer `2.44`, because the comparison is per component and - // not per character. And when two payloads both refine the request, this - // returns nothing — "the resolution of this request" has to be one payload - // to be an answer at all, and picking by directory order is the guess every - // caller here refuses to make. - std::optional - payload_dir_for_version(const std::filesystem::path& packageRoot, - std::string_view version); - // index data root: env.home / "data" std::filesystem::path index_data(const Env& env); @@ -1179,52 +1150,6 @@ find_sibling_package(const std::filesystem::path& compilerBin, return std::nullopt; } -std::optional -payload_dir_for_version(const std::filesystem::path& packageRoot, - std::string_view version) { - // AN EMPTY REQUEST IS NOT A REQUEST FOR EVERYTHING. `packageRoot / ""` - // is `packageRoot` itself, which IS a directory — so without this the - // exact-match branch below would hand back the package root and every - // caller would treat that container as a payload. Both callers today - // reject an empty version before arriving here; this is the invariant - // stated where it holds rather than at each of them. - if (version.empty()) return std::nullopt; - - std::error_code ec; - auto exact = packageRoot / std::string(version); - if (std::filesystem::is_directory(exact, ec)) return exact; - - auto components = [](std::string_view v) { - std::vector parts; - std::size_t start = 0; - while (start <= v.size()) { - auto dot = v.find('.', start); - if (dot == std::string_view::npos) { - parts.emplace_back(v.substr(start)); - break; - } - parts.emplace_back(v.substr(start, dot - start)); - start = dot + 1; - } - return parts; - }; - const auto wanted = components(version); - - std::optional only; - std::error_code dec; - for (auto it = std::filesystem::directory_iterator(packageRoot, dec); - !dec && it != std::filesystem::directory_iterator{}; - it.increment(dec)) { - if (!it->is_directory(dec)) continue; - const auto have = components(it->path().filename().string()); - if (have.size() <= wanted.size()) continue; - if (!std::equal(wanted.begin(), wanted.end(), have.begin())) continue; - if (only) return std::nullopt; // two refinements are not an answer - only = it->path(); - } - return only; -} - std::filesystem::path index_data(const Env& env) { return env.home / "data"; } diff --git a/tests/e2e/687_the_vendored_xlings_probe_is_an_argument_vector.sh b/tests/e2e/687_the_vendored_xlings_probe_is_an_argument_vector.sh index c5aa6b00..6e0bd5bd 100755 --- a/tests/e2e/687_the_vendored_xlings_probe_is_an_argument_vector.sh +++ b/tests/e2e/687_the_vendored_xlings_probe_is_an_argument_vector.sh @@ -18,7 +18,13 @@ # any dated xlings. Not `subos/default/bin/ninja`: that is an xlings shim, one # multicall binary that answers as xlings once it is named `xlings`, and for # the same reason the real xlings is kept under its own name. -# A and B discriminate on Windows; elsewhere they are the control legs. +# C. Without MCPP_VENDORED_XLINGS, an mcpp running from its release layout +# (`/bin/mcpp` beside `/registry/bin/xlings`) replaces an +# older vendored binary with the released one (mcpp#660). Before, the only +# sources were the override variable and the PATH, so a home outside the +# release directory kept whatever older xlings the PATH offered. +# A and B discriminate on Windows; elsewhere they are the control legs. C +# discriminates everywhere. set -e TMP=$(mktemp -d) @@ -76,3 +82,25 @@ grep -q "vendored xlings $older -> " third.err \ "$VENDORED" --version 2>/dev/null | grep -q '^xlings ' \ || fail "B: after the replacement the vendored binary is not xlings" third.err echo "ok: B, a vendored binary older than the pin was replaced" + +# ── C ────────────────────────────────────────────────────────────────────── +rm -f "$VENDORED" +cp "$NINJA" "$VENDORED" +chmod +x "$VENDORED" 2>/dev/null || true +mkdir -p "$TMP/release/bin" "$TMP/release/registry/bin" +cp "$MCPP" "$TMP/release/bin/mcpp$EXE" +cp "$TMP/real/xlings$EXE" "$TMP/release/registry/bin/xlings$EXE" +chmod +x "$TMP/release/bin/mcpp$EXE" "$TMP/release/registry/bin/xlings$EXE" 2>/dev/null || true + +# The PATH is reduced to the system directories, so an xlings on the PATH +# cannot be the source that makes C pass. +C_PATH="/usr/bin:/bin" +if PATH="$C_PATH" command -v xlings > /dev/null 2>&1; then + fail "C: an xlings is reachable on $C_PATH, so the criterion cannot tell its sources apart" +fi +env -u MCPP_VENDORED_XLINGS PATH="$C_PATH" "$TMP/release/bin/mcpp$EXE" self env > fourth.out 2> fourth.err || true +grep -q "vendored xlings $older -> " fourth.err \ + || fail "C: the released xlings did not replace a vendored xlings answering $older" fourth.out fourth.err +"$VENDORED" --version 2>/dev/null | grep -q '^xlings ' \ + || fail "C: after the replacement the vendored binary is not xlings" fourth.err +echo "ok: C, the xlings released with this mcpp replaced an older vendored one" diff --git a/tests/e2e/737_the_declared_runtime_payload_is_installed.sh b/tests/e2e/737_the_declared_runtime_payload_is_installed.sh new file mode 100755 index 00000000..9ac161dd --- /dev/null +++ b/tests/e2e/737_the_declared_runtime_payload_is_installed.sh @@ -0,0 +1,124 @@ +#!/usr/bin/env bash +# requires: gcc +# 737 -- the C runtime a SubOS declares is installed before the toolchain fixup +# consumes it (mcpp#660). +# +# THE DEFECT. The default SubOS declares its glibc by exact version, and the +# gcc fixup patches against the payload directory of that version. Nothing +# installed that payload on purpose: the two loops meant to do it passed +# `xim:glibc` without a version, which the fetcher rejects, and the rejection +# was discarded. The payload arrived only as a dependency of a toolchain that +# xlings installed. A toolchain restored from a CI cache is not installed again, +# so a home whose cache held a different glibc revision failed with +# +# error: toolchain post-install fixup: selected RuntimeBinding glibc@ +# requires payload '.../xim-x-glibc/', ... +# +# THE SHAPE. A CI cache keeps `registry/data/xpkgs` and nothing else: no xlings +# version database, no SubOS. This test installs gcc once, reduces the home to +# that shape, and renames the declared glibc payload so that the store holds a +# different directory for glibc, as a cache from before an index revision does. +# The neighbour's name is not a dotted refinement of the declared version, so +# the directory scan this change removes could not have accepted it either; +# the test is red on the previous implementation. +# +# Criteria: +# A. Offline, the build fails and the message names the exact coordinate +# `xim:glibc@` that provides the payload. +# B. Online, the build succeeds, the declared payload directory now exists +# with a loader, and the program runs. +# C. `mcpp toolchain install` in the same shape succeeds as well (the second +# entry point that runs the fixup). +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +export MCPP_HOME="$TMP/mcpp-home" +export MCPP_NO_AUTO_INSTALL=1 + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +set_mirror() { + if [[ -n "${MCPP_E2E_TOOLCHAIN_MIRROR:-}" ]]; then + "$MCPP" self config --mirror "$MCPP_E2E_TOOLCHAIN_MIRROR" > /dev/null + fi +} + +declared_runtime() { + sed -n 's/.*"runtime": *"\(glibc@[^"]*\)".*/\1/p' \ + "$MCPP_HOME/registry/subos/default/.xlings.json" | head -1 +} + +# Reduce the home to what a CI cache restores, and move the declared glibc +# payload aside under a name that is not a version of it. +cache_shape() { + local keep="$TMP/xpkgs-kept" + mv "$MCPP_HOME/registry/data/xpkgs" "$keep" + rm -rf "$MCPP_HOME" + mkdir -p "$MCPP_HOME/registry/data" + mv "$keep" "$MCPP_HOME/registry/data/xpkgs" + local glibc="$MCPP_HOME/registry/data/xpkgs/xim-x-glibc" + [[ -d "$glibc/$1" ]] || fail "cache shape: no payload for $1 under $glibc" + mv "$glibc/$1" "$glibc/$1-cached-revision" +} + +mkdir -p "$TMP/proj/src" +cat > "$TMP/proj/mcpp.toml" <<'EOF' +[package] +name = "runtimeprobe" +version = "0.1.0" +EOF +cat > "$TMP/proj/src/main.cpp" <<'EOF' +import std; +int main() { std::println("runtime ok"); return 0; } +EOF + +# ── Install once, in a fresh home ──────────────────────────────────────────── +set_mirror +"$MCPP" toolchain install gcc 16.1.0 > "$TMP/install1.log" 2>&1 \ + || fail "initial toolchain install" "$TMP/install1.log" +DECLARED=$(declared_runtime) +[[ -n "$DECLARED" ]] || fail "the default SubOS declares no glibc runtime" \ + "$MCPP_HOME/registry/subos/default/.xlings.json" +VERSION=${DECLARED#glibc@} +echo "declared runtime: $DECLARED" + +# ── A: offline, in the cache shape ─────────────────────────────────────────── +cache_shape "$VERSION" +set_mirror +"$MCPP" toolchain default gcc@16.1.0 > "$TMP/default.log" 2>&1 \ + || fail "toolchain default in the cache shape" "$TMP/default.log" +[[ "$(declared_runtime)" == "$DECLARED" ]] \ + || fail "the re-initialised SubOS declares $(declared_runtime), not $DECLARED" +[[ ! -e "$MCPP_HOME/registry/data/xpkgs/xim-x-glibc/$VERSION" ]] \ + || fail "A: the declared payload exists before the build; the shape is wrong" + +cd "$TMP/proj" +rc=0 +MCPP_OFFLINE=1 "$MCPP" build > "$TMP/offline.log" 2>&1 || rc=$? +[[ $rc -ne 0 ]] || fail "A: an offline build without the declared payload succeeded" "$TMP/offline.log" +grep -q "xim:$DECLARED" "$TMP/offline.log" \ + || fail "A: the offline failure does not name xim:$DECLARED" "$TMP/offline.log" +echo "ok: A, offline build names xim:$DECLARED" + +# ── B: online, the build installs the declared payload ─────────────────────── +rm -rf target +"$MCPP" build > "$TMP/online.log" 2>&1 || fail "B: online build" "$TMP/online.log" +payload="$MCPP_HOME/registry/data/xpkgs/xim-x-glibc/$VERSION" +ls "$payload"/lib64/ld-linux-* "$payload"/lib/ld-linux-* > /dev/null 2>&1 \ + || fail "B: no loader under $payload after the build" "$TMP/online.log" +out=$("$MCPP" run 2>&1) || fail "B: run" "$TMP/online.log" +[[ "$out" == *"runtime ok"* ]] || fail "B: unexpected run output: $out" +echo "ok: B, online build installed $DECLARED and the program runs" + +# ── C: toolchain install, in the cache shape ──────────────────────────────── +cd "$TMP" +cache_shape "$VERSION" +set_mirror +"$MCPP" toolchain install gcc 16.1.0 > "$TMP/install2.log" 2>&1 \ + || fail "C: toolchain install in the cache shape" "$TMP/install2.log" +ls "$payload"/lib64/ld-linux-* "$payload"/lib/ld-linux-* > /dev/null 2>&1 \ + || fail "C: no loader under $payload after toolchain install" "$TMP/install2.log" +echo "ok: C, toolchain install installed $DECLARED" + +echo "OK" diff --git a/tests/unit/test_post_install.cpp b/tests/unit/test_post_install.cpp index 3dfa0f1f..bd5fd96e 100644 --- a/tests/unit/test_post_install.cpp +++ b/tests/unit/test_post_install.cpp @@ -193,20 +193,23 @@ TEST(PostInstallFixup, APackageWithNoFixupReportsNothingToReport) { << result->skippedReason; } -// THE VERSION THAT WAS ASKED FOR AND THE VERSION THAT WAS INSTALLED ARE -// TWO VOCABULARIES FOR ONE FACT. +// THE DECLARED VERSION NAMES ITS PAYLOAD DIRECTORY EXACTLY (mcpp#660). // -// A RuntimeBinding carries the DECLARED identity; xlings names the payload -// directory after what the request RESOLVED to. The two come apart the moment -// the index moves a package within a series — `xim:glibc@2.44` resolving to -// `2.44.2`. +// The lookup used to accept a unique `2.44.x` for a binding of `2.44` and to +// refuse two of them. That tolerance existed because the declared payload was +// never installed on purpose; it arrived as a side effect of another install, +// under whatever version that install resolved. It concealed the missing +// install, and it failed as soon as a cache held one revision and an install +// added the next: // -// Measured 2026-08-27 on every CI machine with a cold cache, on `main` as -// readily as on any branch: +// error: toolchain post-install fixup: selected RuntimeBinding glibc@2.44 +// requires payload '.../xim-x-glibc/2.44', but no installed payload +// is its resolution; mcpp will not fall back to another directory +// entry +// $ ls .../xim-x-glibc/ -> 2.44.2 2.44.3 // -// error: selected RuntimeBinding glibc@2.44 requires payload -// '…/xim-x-glibc/2.44', but it is not installed -// $ ls …/xim-x-glibc/ → 2.44.2 +// `ensure_declared_runtime` now installs the declared version, so the lookup +// is exact and a neighbouring version is never a substitute. namespace { // A payload is "installed" for this purpose when it has a lib dir with a @@ -235,83 +238,31 @@ struct GlibcRootFixture { } // namespace -TEST(GlibcPayload, TheExactVersionIsPreferred) { +TEST(GlibcPayload, TheExactVersionIsSelected) { GlibcRootFixture fx{"mcpp_glibc_exact"}; - auto want = make_glibc_payload(fx.root, "2.44"); + auto want = make_glibc_payload(fx.root, "2.44.3"); make_glibc_payload(fx.root, "2.44.2"); - auto got = mcpp::toolchain::select_glibc_payload_lib(fx.root, "glibc@2.44"); + auto got = mcpp::toolchain::select_glibc_payload_lib(fx.root, "glibc@2.44.3"); ASSERT_TRUE(got.has_value()) << got.error(); EXPECT_EQ(*got, want); } -TEST(GlibcPayload, ARequestResolvesToItsOneRefinement) { - GlibcRootFixture fx{"mcpp_glibc_refine"}; - auto only = make_glibc_payload(fx.root, "2.44.2"); - auto got = mcpp::toolchain::select_glibc_payload_lib(fx.root, "glibc@2.44"); - ASSERT_TRUE(got.has_value()) << got.error(); - EXPECT_EQ(*got, only); -} - -// PER COMPONENT, NOT PER CHARACTER. `2.4` is not a request that `2.44` -// answers — a prefix match on the string would say it is, and would then hand -// a build the wrong C library without saying anything. -TEST(GlibcPayload, AStringPrefixIsNotARefinement) { - GlibcRootFixture fx{"mcpp_glibc_strprefix"}; - make_glibc_payload(fx.root, "2.44"); - auto got = mcpp::toolchain::select_glibc_payload_lib(fx.root, "glibc@2.4"); - EXPECT_FALSE(got.has_value()); -} - -// AND THE REFUSAL STILL STANDS WHEN THERE IS NO ONE ANSWER. "The resolution -// of this request" has to be a single payload to be an answer at all; two -// refinements are not a menu to pick from. -TEST(GlibcPayload, TwoRefinementsAreRefusedRatherThanChosenBetween) { - GlibcRootFixture fx{"mcpp_glibc_ambiguous"}; - make_glibc_payload(fx.root, "2.44.1"); +// THE CASE OF mcpp#660. Two revisions of the line are present and the declared +// version is neither; the answer is "not installed", naming the coordinate +// that provides it, and never one of the two. +TEST(GlibcPayload, ANeighbouringRevisionIsNotASubstitute) { + GlibcRootFixture fx{"mcpp_glibc_neighbour"}; make_glibc_payload(fx.root, "2.44.2"); + make_glibc_payload(fx.root, "2.44.3"); auto got = mcpp::toolchain::select_glibc_payload_lib(fx.root, "glibc@2.44"); - EXPECT_FALSE(got.has_value()); -} - -// THE RESOLVER ITSELF, because it has TWO callers and they failed -// separately. The first version of this fix lived inside the toolchain fixup; -// `probe`'s compile-side payload discovery spelled the same lookup its own way -// and kept missing — and ITS failure names no version at all: -// -// bits/os_defines.h:39: fatal error: features.h: No such file -// -// because the glibc include directory is simply never added. Asserting on the -// shared function is what makes both call sites covered by one test. -TEST(PayloadDirForVersion, ExactWinsOverRefinement) { - GlibcRootFixture fx{"mcpp_pdfv_exact"}; - std::filesystem::create_directories(fx.root / "2.44"); - std::filesystem::create_directories(fx.root / "2.44.2"); - auto got = mcpp::xlings::paths::payload_dir_for_version(fx.root, "2.44"); - ASSERT_TRUE(got.has_value()); - EXPECT_EQ(got->filename(), "2.44"); -} - -TEST(PayloadDirForVersion, OneRefinementIsTheAnswer) { - GlibcRootFixture fx{"mcpp_pdfv_one"}; - std::filesystem::create_directories(fx.root / "2.44.2"); - auto got = mcpp::xlings::paths::payload_dir_for_version(fx.root, "2.44"); - ASSERT_TRUE(got.has_value()); - EXPECT_EQ(got->filename(), "2.44.2"); -} - -TEST(PayloadDirForVersion, TwoRefinementsAreNotAnAnswer) { - GlibcRootFixture fx{"mcpp_pdfv_two"}; - std::filesystem::create_directories(fx.root / "2.44.1"); - std::filesystem::create_directories(fx.root / "2.44.2"); - EXPECT_FALSE(mcpp::xlings::paths::payload_dir_for_version(fx.root, "2.44")); -} - -// PER COMPONENT, NOT PER CHARACTER — a string prefix would hand a build the -// wrong C library and say nothing. -TEST(PayloadDirForVersion, AStringPrefixIsNotARefinement) { - GlibcRootFixture fx{"mcpp_pdfv_strprefix"}; - std::filesystem::create_directories(fx.root / "2.44"); - EXPECT_FALSE(mcpp::xlings::paths::payload_dir_for_version(fx.root, "2.4")); + ASSERT_FALSE(got.has_value()); + EXPECT_NE(got.error().find("xim:glibc@2.44"), std::string::npos) << got.error(); + EXPECT_NE(got.error().find("not installed"), std::string::npos) << got.error(); + + // One neighbour is not a substitute either: the old tolerance accepted it. + GlibcRootFixture one{"mcpp_glibc_one_neighbour"}; + make_glibc_payload(one.root, "2.44.2"); + EXPECT_FALSE(mcpp::toolchain::select_glibc_payload_lib(one.root, "glibc@2.44")); } TEST(GlibcPayload, NothingInstalledIsStillRefused) { @@ -320,22 +271,12 @@ TEST(GlibcPayload, NothingInstalledIsStillRefused) { EXPECT_FALSE(got.has_value()); } -// AN EMPTY REQUEST IS NOT A REQUEST FOR EVERYTHING. -// -// `packageRoot / ""` is `packageRoot`, and that IS a directory — so the -// exact-match branch would hand the CONTAINER back and every caller would -// treat it as a payload. Both callers reject an empty version before arriving, -// which is exactly why nothing would have caught it here. -TEST(PayloadDirForVersion, AnEmptyRequestIsNotTheContainer) { - auto root = std::filesystem::temp_directory_path() - / "mcpp_payload_empty_request"; - std::filesystem::remove_all(root); - std::filesystem::create_directories(root / "2.44.2"); - - EXPECT_FALSE(mcpp::xlings::paths::payload_dir_for_version(root, "")); - // The denominator: a real request against the same tree still resolves, - // so an empty result above is the guard and not an unreadable directory. - EXPECT_TRUE(mcpp::xlings::paths::payload_dir_for_version(root, "2.44")); - - std::filesystem::remove_all(root); +// A DIRECTORY WITHOUT A LOADER IS REPORTED AS INCOMPLETE, NOT AS ABSENT. The +// two call for different repairs, so the messages must not coincide. +TEST(GlibcPayload, APayloadWithoutALoaderIsIncomplete) { + GlibcRootFixture fx{"mcpp_glibc_husk"}; + std::filesystem::create_directories(fx.root / "2.44.3"); + auto got = mcpp::toolchain::select_glibc_payload_lib(fx.root, "glibc@2.44.3"); + ASSERT_FALSE(got.has_value()); + EXPECT_NE(got.error().find("stale/incomplete"), std::string::npos) << got.error(); } diff --git a/tests/unit/test_toolchain_lifecycle.cpp b/tests/unit/test_toolchain_lifecycle.cpp index fa762720..dcb150be 100644 --- a/tests/unit/test_toolchain_lifecycle.cpp +++ b/tests/unit/test_toolchain_lifecycle.cpp @@ -1,7 +1,12 @@ #include import std; +import mcpp.config; +import mcpp.platform; +import mcpp.runtime.binding; import mcpp.toolchain.lifecycle; +import mcpp.toolchain.post_install; +import mcpp.toolchain.registry; using namespace mcpp::toolchain; @@ -137,3 +142,74 @@ TEST(ToolchainRemove, TheSweepAlsoClearsAFileLessSkeleton) { std::error_code ec; std::filesystem::remove_all(pkgRoot, ec); } + +// ─── The declared C runtime payload (mcpp#660) ───────────────────────────── +// +// `ensure_declared_runtime` installs the payload a SubOS declares. The install +// itself needs xlings and the network and is covered end to end (e2e 700); +// what is pinned here is when it must NOT act, because each of these cases +// runs on every build and must start no process and change nothing. + +namespace { + +mcpp::config::GlobalConfig unreachable_cfg() { + // A registry that does not exist. None of the cases below may consult it. + mcpp::config::GlobalConfig cfg; + cfg.registryDir = std::filesystem::temp_directory_path() + / "mcpp-declared-runtime-unreachable-registry"; + return cfg; +} + +} // namespace + +TEST(DeclaredRuntime, ABindingThatLocatesItsPayloadIsLeftAlone) { + auto cfg = unreachable_cfg(); + mcpp::platform::runtime::RuntimeBinding binding; + binding.runtimeId = "glibc@2.44.3"; + binding.libraryDirs = {"/store/xim-x-glibc/2.44.3/lib64"}; + const auto before = binding.libraryDirs; + EXPECT_FALSE(ensure_declared_runtime(cfg, binding)); + EXPECT_EQ(binding.libraryDirs, before); + EXPECT_EQ(binding.runtimeId, "glibc@2.44.3"); +} + +// A provider that binds no payload (`ucrt` is an OS component) and an absent +// identity are not requests to install anything. +TEST(DeclaredRuntime, OnlyAGlibcPayloadIsInstalled) { + auto cfg = unreachable_cfg(); + for (auto id : {"", "ucrt@10.0.26100.0", "macos_sdk@15.0", "glibc"}) { + mcpp::platform::runtime::RuntimeBinding binding; + binding.runtimeId = id; + EXPECT_FALSE(ensure_declared_runtime(cfg, binding)) << id; + EXPECT_TRUE(binding.libraryDirs.empty()) << id; + } +} + +// Off Linux there is no glibc payload to provide, whatever the identity says. +TEST(DeclaredRuntime, NothingIsInstalledOffLinux) { + if constexpr (mcpp::platform::is_linux) + GTEST_SKIP() << "the Linux path is exercised by e2e 700"; + auto cfg = unreachable_cfg(); + mcpp::platform::runtime::RuntimeBinding binding; + binding.runtimeId = "glibc@2.44.3"; + EXPECT_FALSE(ensure_declared_runtime(cfg, binding)); +} + +// The dispatch that decides whether a toolchain needs the payload at all. A +// toolchain whose fixup does not patch against a C runtime must not cause a +// download. +TEST(DeclaredRuntime, OnlyToolchainsTheFixupPatchesNeedThePayload) { + XimToolchainPackage gcc; gcc.ximName = "gcc"; gcc.needsGccPostInstallFixup = true; + XimToolchainPackage llvm; llvm.ximName = "llvm"; + XimToolchainPackage musl; musl.ximName = "musl-gcc"; + XimToolchainPackage msvc; msvc.ximName = "msvc"; + if constexpr (mcpp::platform::is_windows) { + for (auto const* p : {&gcc, &llvm, &musl, &msvc}) + EXPECT_TRUE(post_install_fixup_kind(*p).empty()) << p->ximName; + } else { + EXPECT_EQ(post_install_fixup_kind(gcc), "gcc"); + EXPECT_EQ(post_install_fixup_kind(llvm), "llvm"); + EXPECT_TRUE(post_install_fixup_kind(musl).empty()); + EXPECT_TRUE(post_install_fixup_kind(msvc).empty()); + } +} diff --git a/tests/unit/test_toolchain_registry.cpp b/tests/unit/test_toolchain_registry.cpp index 089dddf2..8d4056d4 100644 --- a/tests/unit/test_toolchain_registry.cpp +++ b/tests/unit/test_toolchain_registry.cpp @@ -336,29 +336,6 @@ TEST(ToolchainOrigin, NonMsvcSystemIsRejectedWhereItIsReadAndOffersTheAlternativ } } -// The Linux sysroot payloads (`xim:glibc` + `xim:linux-headers`) had two -// derivations, and a comment on one claimed it mirrored the other. It did -// not: the PE term was missing from the second. -TEST(ToolchainSysrootDeps, OneDerivationForTheGlibcSysrootPayloads) { - mcpp::toolchain::triple::Triple host{}; // empty = host - mcpp::toolchain::triple::Triple musl{std::string(mcpp::platform::host_arch), "linux", "musl"}; - mcpp::toolchain::triple::Triple mingw{"x86_64", "windows", "gnu"}; - - if constexpr (mcpp::platform::is_linux) { - EXPECT_TRUE(needs_linux_sysroot_payloads(host)); - // Self-contained: a musl payload carries its own C library. - EXPECT_FALSE(needs_linux_sysroot_payloads(musl)); - // THE TERM THAT WAS MISSING. A PE target brings its own CRT, whether - // it is a native MinGW or the Linux-hosted cross, so a Linux sysroot - // is not part of installing one. - EXPECT_FALSE(needs_linux_sysroot_payloads(mingw)); - } else { - // No Linux sysroot exists to want. - for (auto const& t : {host, musl, mingw}) - EXPECT_FALSE(needs_linux_sysroot_payloads(t)); - } -} - // ─── An SDK payload is chosen by the TARGET, and knows its own layout ────── // // `to_xim_package` returned the generic llvm payload for every `Family::Llvm`