From 9188fefb925f25e4592cf14c4f1b5fb63ed537f5 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 05:49:05 +0800 Subject: [PATCH 01/20] docs(design): the #646-#649 triage record, its probes, and the implementation plan --- .../2026-09-16-646-649-four-issues-by-home.md | 1346 +++++++++++++++++ .../2026-09-16-646-649-implementation-plan.md | 410 +++++ .agents/docs/2026-09-16-646-649-probes.sh | 778 ++++++++++ .agents/docs/README.md | 6 +- 4 files changed, 2539 insertions(+), 1 deletion(-) create mode 100644 .agents/docs/2026-09-16-646-649-four-issues-by-home.md create mode 100644 .agents/docs/2026-09-16-646-649-implementation-plan.md create mode 100755 .agents/docs/2026-09-16-646-649-probes.sh diff --git a/.agents/docs/2026-09-16-646-649-four-issues-by-home.md b/.agents/docs/2026-09-16-646-649-four-issues-by-home.md new file mode 100644 index 00000000..d0a3e512 --- /dev/null +++ b/.agents/docs/2026-09-16-646-649-four-issues-by-home.md @@ -0,0 +1,1346 @@ +--- +subject: triage +status: active +--- + +# Four issues from a framework and an editor: #646, #647, #648 and #649, read against the engine and routed by home + +**Status:** revision 1, for review. Nothing is implemented. + +Engine code was read at `2fc7b5b0` (origin/main, after mcpp 2026.9.15.2). +mcpp-plugins, openxlings/xlings (`3cd8061`) and mcpplibs/mcpp-index were read +at their current heads. A statement marked *measured* was run on a Linux +x86_64 host with the released mcpp 2026.9.15.2, `gcc@16.1.0`, `llvm@22.1.8` +and `xim:android-ndk@30.0.16248370`; the probes are +`2026-09-16-646-649-probes.sh` (groups a to d), and §11 quotes their +`READING` lines. A statement marked *read* names a file and line and was not +executed. Nothing in this revision ran on macOS or Windows; §9 lists the +premises that need a temporary pull request before their decisions are taken. + +The four issues hold 23 items. Measuring them found twelve defects that no +issue reports, two of which are more severe than anything the issues list: the +default llvm build of a program over a C++ shared library aborts on Linux +(F3a), and every plan of a project with a bare `compat.*` dependency starts a +network index refresh once the two-minute debounce has passed (T). + +§0 is the ledger. §1 lists where the measured state differs from the issues, +§2 the defects beyond them. §3 states the rules the decisions share. §4 to §8 +take the items by subject. §9 collects the decisions for the reviewer and the +measurements still owed, §10 the order of work, §11 the readings, §12 what +each project can do today, §13 the self-review. + +## 0. The ledger + +"Home" follows the routing rule of the #634 record: **usage** (a spelling or +an existing mechanism), **project**, **official plugin** (mcpp-plugins), +**ecosystem data** (xim-pkgindex, xlings, mcpp-index), **engine**. A defect +stays in the engine whatever its size; the rule filters features, not +defects. "Beyond" marks a finding no issue reports. + +| # | real | general | home | decision (§) | evidence | +|---|---|---|---|---|---| +| F3a | defect, beyond | yes | engine | on ELF, every C++ image a plan loads into one process shares one C++ runtime; a program over a plan-built C++ shared library takes the shared-library contract (§4.1) | measured: llvm default aborts with `std::bad_cast`, exit 134; gcc default runs with 900 interposed libstdc++ symbols | +| F3b | defect (false report) | yes | engine | `STB_GNU_UNIQUE` is vague linkage; a duplicate whose two definitions come from one plan object is not a conflict (§4.2) | measured: 2 (llvm) and 8 (gcc) findings under a uniform contract; `--strict` exits 1 | +| F1 | defect | yes | engine | a static package reachable from exactly one shared image is linked into that image; one reachable from two images is refused with the `linkage = "shared"` remedy (§4.3) | measured: `-z defs` link fails, a foreign `dlopen` fails, ELF interposition hides both | +| F2 | probable defect | yes | engine, measurement first | measure on macos-15; the F3a rule per format decides the change (§4.4) | read: libc++ `typeinfo`; not measured on the payload origin | +| E10 | defect in the record, and a default to decide | yes | engine | step 1: the record and docs state the static CRT the row delivers, and a `cxx_runtime` the row cannot deliver is diagnosed; step 2 (own record, after windows-2022): whether the row adopts `/MD` (§4.5) | measured: the driver's `-defaultlib:libcmt`; read: no CRT flag outside the `msvc` dialect | +| E1 | gap | yes | engine | the root build program reads a graph file (`mcpp::graph_file()`): packages in dependency order with manifest directory, kind, features, link form and verbatim `[package.metadata]`; its digest joins the re-run key (§5.1) | measured: the program sees direct dependencies only; a metadata edit replays the cached run | +| E2 | feature | Apple rows | official plugin (`rules-swift`) | one package's Swift compiles through `action` objects, `link_flag` and a generated header; cross-package Swift modules wait for an engine interface channel (§5.2) | read; no `swiftc` on the host | +| E3 | defect | yes (macOS host, every non-Apple hosted row; latent on Windows) | engine | choose the link branch by the target's object format, in a pure function with a host-by-row unit test that runs on Linux (§5.3) | read line by line; the issue's CI log | +| E5 | defect | yes | engine; plugin workaround retires | strip the program on every row and every shared library the graph built; strip staged toolchain runtime copies; report what was done; expose the decision to build programs (§5.4) | measured: desktop `libdep.so`, Android program, `libdep.so` and `libc++_shared.so` unstripped under "stripped" | +| E9 | gap, and an inconsistency | yes | engine | `mcpp pack --message-format json` prints one `mcpp.pack` envelope; `pack` accepts `--release` and `--dev` with `build`'s precedence (§5.5) | measured: three `unknown option` refusals | +| E4.1 | defect | yes | engine | the forward validator accepts a key declared in any dependency table of the manifest, on any row (§6.1) | measured: warning and `--strict` exit 2 while the forward applies | +| E4.2 | documentation defect, and a silent drop beyond | yes | docs; engine (one check) | the docs say to restate the source; a restatement whose source differs is refused (§6.2) | measured: the restated form works; a different path is ignored under `--strict` | +| E4.3 | defect | yes | engine | one helper names a provider for a consumer; `dep_dir`, `dep_linkage` and `dep_bin` use it (§6.3) | measured: `qualified=[]` | +| E6 | defect | yes | engine | a package with no library target contributes nothing to a consumer's target graph; its programs come from the tool sub-build alone; package cycles are checked at resolution (§6.4) | measured: builds with `--cache=local`; the tool's own dependency is linked into the application | +| E7 | gap | yes | engine | a git dependency whose identity differs from the root manifest's is looked up among the repository root's `[workspace] members`; no `subdir` key (§6.5) | measured: member-only is refused; both keys drop the `tools` request | +| E8 | defect (narrower than stated), and a gap | yes | engine | a `dep/feature` token is a root forward and never a macro; a plain undeclared name without `[features]` keeps its documented meaning (§6.6) | measured: `-DMCPP_FEATURE_SPIKE_FW_INSTALLER` | +| T | defect, beyond | yes | engine; the xlings manifest | the refresh decision walks the resolver's deprecated bare-name rung before it calls a miss (§7.0) | measured on openxlings/xlings | +| A1 | gap | yes | engine | refusal code `offline-download-required` at every offline refusal site; envelope code `MCPP_OFFLINE_DOWNLOAD_REQUIRED` naming the first missing item (§7.1) | read: five prose-only sites | +| A2 | defect | yes | engine | the saved stdout descriptor is close-on-exec (Windows: not inheritable) (§7.2) | measured: a build program holds the caller's pipe as fd 3 | +| A3 | defect | yes | engine; xlings for its own connection timeout | xlings runs under the owned launcher; refresh has a total deadline, install an inactivity deadline (§7.3) | read | +| A4 | defect against docs/50 | yes | engine | a per-run record of observed effects; a launched network child records `network` (§7.4) | read | +| A5 | defect (docs and code disagree) | yes | engine | the three refreshes that bypass `mcpp.pm.index_refresh` go through `decide_for_miss` (§7.5) | read | +| A6 | ecosystem-data defect, beyond; then a default | yes | mcpp-index, then engine | the index artifact becomes byte-reproducible and is republished; then the default `artifact` becomes a region object (§7.6) | measured: GitCode's asset digest differs from the pointer | +| P1 | feature | yes | official plugin | `dist-apple` `options::omit_keys`, restricted to the defaulted keys (§8) | read | +| P2 | feature | yes | official plugin | `dist-web` `options::page`, default `index.html` (§8) | read | + +The mcppls-side measures of #648 (offline by default, its own process group, +a cached last document) belong to the caller and are correct as stated (§7.7). + +## 1. Where the measured state differs from the issues + +1. **F3's 882 symbols are a split runtime, not the module initialiser.** The + default ELF contracts give the program `self-contained` and the shared + library `toolchain-coupled` (`resolution.json`, reading a). Under gcc the + program's static libstdc++ is exported and interposes the library's + `libstdc++.so.6` (900 conflicts, the run succeeds). Under llvm the same + shape aborts when run (exit 134), even when the program itself does not + import `std`. Only under a uniform contract do the initialiser findings the + issue describes remain (§4.1, §4.2). +2. **F1 is described as intended by the code.** `symbol_provision.cppm:36-43` + and the #519 design call the arrangement benign and state that PE and + Mach-O "structurally do not have this problem". That holds only on ELF, and + only for a consumer that linked the static package itself: `-z defs` + refuses the library, and a program that did not link `x` cannot `dlopen` + it (reading a, F1c and F1d). +3. **E10's record is false, and no value of `cxx_runtime` reaches the row.** + The PE contract table treats clang on the MSVC ABI like `cl.exe` and + records `host-coupled` (`distribution.cppm:615-686`), while the driver + links `libcmt` (reading a, E10) because no CRT flag is emitted outside the + `msvc` dialect (`flags.cppm:888-895`). +4. **E1: dependency build programs run dependents first**, in discovery order + (`prepare.cppm:9551-9640`); `[package.metadata.*]` is accepted silently + today, even under `--strict`; and the engine cannot resolve paths inside + metadata it does not interpret. +5. **E3 is wider than Android.** The macOS host branch serves every non-Apple + hosted row whose C library comes from a payload (wasm32-emscripten + included, read), the Windows non-MSVC branch has the same shape + (`flags.cppm:1568-1570`, latent: the NDK publishes no Windows archive), and + no Linux test can see it because the branch is chosen by `if constexpr` + and the Android e2e scripts are gated `# requires: gcc`. +6. **E5: the Android row strips nothing because it never reaches the strip + step.** `run()` dispatches a shared-object program to + `run_shared_program` (`pack.cppm:1733`), which does not call + `strip_program` and ignores `--debug-symbols`. The word "stripped" is the + decision, printed before the outcome (`pipeline.cppm:499-502`). +7. **E4.2: the restated form already works**, and docs/30 states it + correctly (`docs/30:1387-1392`). The defect is docs/05 and its translation, + plus a restatement with a different source that is ignored without a word. +8. **E6: the cycle is in the consumer's own target graph, not in a key.** The + tool package's `[dependencies]` are walked into the consumer graph + (`prepare.cppm:7680-7686`), so the same manifest builds with + `--cache=local` (the only walk that detects cycles runs for the global + cache, `:12790`), and a tool's own library dependency is linked into the + application (reading c, E6c). Keying builds on features does not remove + that edge. +9. **E7: the both-keys spelling drops the member's `tools` request silently** + (`recordDependencyEdge` returns early on a second declaration, + `prepare.cppm:6304-6313`). +10. **E8: accepting any plain name without a `[features]` table is + documented** (docs/06:40-42, pure macro usage). The defect is that a + `dep/feature` token, which can only be a forward, becomes the macro + `MCPP_FEATURE_SPIKE_FW_INSTALLER`. +11. **#648: the refresh was started by a disagreement, not by a stale index** + (§7.0). The pipe leak also reaches children started through + `posix_spawn`, not only `popen` (§7.2). The CN mirror the issue asks for + is deployed, and its artifact does not match its pointer (§7.6). +12. **A1 cannot name every missing item.** Planning stops at the first + refusal; the diagnostic names the first (§7.1). +13. **A3: mcpp already owns its children.** The process group, the signal + guard and the Windows job object were built for ninja; the xlings calls + are the remaining exception (§7.3). + +## 2. Defects beyond the issues + +| # | defect | severity | § | +|---|---|---|---| +| F3a | the default ELF contracts put two C++ runtimes in one process; llvm aborts | a working-looking build that does not run | §4.1 | +| T | the refresh decision disagrees with the resolver on bare `compat.*` names; a network refresh every two minutes | every build of such a project, online | §7.0 | +| A6 | GitCode's index artifact digest differs from the pointer | the CN route falls back to GitHub | §7.6 | +| X1 | a second declaration of one dependency by one consumer loses its `tools`, `features`, `host-module` and `reexport` (`prepare.cppm:6304-6313`) | silent, under `--strict` | §6.5 | +| X2 | package-edge cycles are detected only by the global-cache key walk | the same manifest passes with `--cache=local` | §6.4 | +| X3 | a tool cycle is refused after four nested sub-builds, with a repeated prefix and no edge named (`tool_store.cppm:62`) | diagnostic | §6.4 | +| X4 | `[package]` accepts unknown keys silently, under `--strict` too; `[build]`, `[targets]` and `[test]` do not | a misspelt key is ignored | §5.1 | +| X5 | `run` gives `--release` precedence over `--profile`; `build` does the reverse (`cmd_build.cppm:477-479`, `:109-114`) | one command line, two profiles | §5.5 | +| X6 | a git dependency's compile banner prints `v` with no version (`execute.cppm:886-891`) | cosmetic | §6.5 | +| X7 | `mcpp why deps --features` is `unknown option` | a feature build's graph is visible only in `resolution.json` | §6.6 | +| X8 | a forward to a dependency declared for another row fails `--strict` | a portable manifest cannot be strict | §6.1 | +| X9 | the only qualified-`dep_bin` test uses the legacy dotted name (`tests/e2e/187:170`) | E4.3 was untested | §6.3 | + +## 3. The rules the decisions share + +The items are many; the rules they violate are few. Each decision below is an +application of one of these, and a reviewer who rejects a rule rejects its +decisions together. + +- **R1. One process, one C++ runtime; one static package, one image.** A + type's identity, a locale facet's id and a static package's state exist + once per process only if one image defines them. F3a, F2 and E10 are the + runtime instance; F1 is the general instance. #641's refusal + `shared-library-cxx-runtime` is the precedent: it enforced the rule for a + graph runtime and left the payload's defaults and ordinary static packages + outside it. +- **R2. One question, one derivation.** Every "second copy" in this batch + answers a question another function already answers: the refresh decision + and the resolver (T), the refresh policy and three bypasses (A5), the + forward validator and the injector (E4.1), the tool variable and + `fillDepDirs` (E4.3), the host branch and the target format (E3), the CLI + feature parser and the manifest forward (E8), `run` and `build` profile + precedence (X5). The fix in each case deletes a derivation rather than + correcting it, because a corrected copy drifts again. +- **R3. A child mcpp starts is owned, bounded, and inherits only what it + needs.** The launcher that owns ninja and build programs already exists. + A2 and A3 move the remaining children onto it and close the one descriptor + that escapes it. +- **R4. What mcpp reports is what happened.** The envelope's `effects` (A4), + its error codes (A1), pack's status line (E5), pack's artifacts (E9), the + runtime record (E10) and the symbol-provision check (F3b) each state + something the build did not do, or omit something it did. docs/50 already + states the rule for `effects`; this batch applies it to the rest. +- **R5. The engine states facts; plugins build products.** A graph with + metadata (E1), a strip decision (E5) and the existing `action` and + `link_flag` channels (E2) are facts and mechanisms; merging resources, + packaging an APK, compiling Swift and writing an Info.plist are products + (E1's consumers, E2, P1, P2). +- **R6. Published data keeps loading on older clients.** A manifest shape + that older clients refuse cannot be introduced to express something the + current grammar already expresses (E4.2), and a new key that older clients + ignore must not change what is built when ignored (E7's rejected `subdir`). + E1's metadata table is admissible under this rule precisely because older + clients already ignore it. + +## 4. #646 and #649 E10: images, runtimes and placement + +### 4.1 F3a: the default ELF contracts put two C++ runtimes in one process + +**Measured (group a).** A program over a `kind = "shared"` C++ library, both +importing `std`, with `std::format` in the library: + +``` +F3 gcc contracts in resolution.json: {"distributable": "self-contained", ..., "shared-library": "toolchain-coupled", ...} +F3 gcc NEEDED program: [liblib.so] [libm.so.6] [libgcc_s.so.1] [libc.so.6] NEEDED liblib.so: [libstdc++.so.6] ... +F3 gcc run: lib-3 exit=0 bin/app conflicts=900 +F3 llvm run: libc++abi: terminating due to uncaught exception of type std::bad_cast: std::bad_cast exit=134 +F3 llvm-no-println run: ... std::bad_cast exit=134 +F3 llvm-tc (cxx_runtime = "toolchain-coupled") run: lib-3 exit=0 +F3 llvm-static-lib (kind = "lib") run: lib-3 exit=0 +``` + +The lead reproduced the llvm abort independently, through `mcpp run` as well. + +**Mechanism (read).** `default_contract` returns `toolchain-coupled` for an +ELF shared library since #414, which stopped shared libraries from embedding +libstdc++ and hijacking the program's runtime, and `self-contained` for an ELF +program. Neither default is wrong alone; together they place a static C++ +runtime in the executable and a shared one in the library. The executable +exports the runtime symbols the library references, so the library binds +part of its runtime to the program's copy and part to its own. The `_Unwind_*` +symbols among the 46 llvm conflicts are the partial interposition already +recorded for the unwinder in 2026-08; the `bad_cast` is probably a locale +facet looked up by an id that exists twice, which was not isolated. + +- **Kind:** defect, general to every ELF project whose program depends on a + C++ shared library it builds. It is the ELF instance of F2. +- **Home:** engine (`distribution.cppm`). +- **Decision (R1).** When a plan's root image links a C++ shared library that + the plan builds, the contracts of its programs and tests are derived from + the shared-library contract (`toolchain-coupled`) instead of the + per-role default. An explicit `cxx_runtime = "self-contained"` on the + program in such a graph is refused before compiling, with a new refusal + code beside `shared-library-cxx-runtime`, naming the library and the split. + Unlike #641 M3, no refusal is needed for the default: a mechanism that + delivers one runtime exists. +- **Cost.** A gcc program over a C++ shared library gains `NEEDED + libstdc++.so.6`. The process already requires that file, through the + library's own `NEEDED`, so no deployment gains a requirement; the change is + where the program binds. +- **Scope.** The first criterion covers shared libraries the plan builds. A + prebuilt C++ shared library from the store is the same class and is + detectable from its `DT_NEEDED`; it is named here and not decided, because + its detection runs after linking rather than at planning. +- **Criterion.** The llvm default shape runs (today exit 134); the gcc default + shape reports no libstdc++ conflicts; `cxx_runtime = "self-contained"` on + the program is refused with the code. + +### 4.2 F3b: the symbol-provision check reports what the loader unifies by design + +**Measured (group a).** Under a uniform contract the llvm shape reports two +conflicts (`_ZGIW3std`, `_ZGIW3stdW6compat`) and the gcc shape eight (the +initialiser and seven `STB_GNU_UNIQUE` objects such as +`__from_chars_alnum_to_val_table::value`). Each finding is +`diag::degraded` (`ninja_backend.cppm:3283-3297`), so `--strict` exits 1 on +every leg. The initialiser in `std.o` is one global function whose body is a +return (gcc) or a frame push, pop and return (clang): no guard, no state. + +- **Kind:** defect (a false report, R4), general. +- **Home:** engine (`symbol_provision.cppm`, `src/runtime/elf.cppm:742`). +- **Decision.** (1) `STB_GNU_UNIQUE` counts as vague linkage, as `STB_WEAK` + already does. (2) A duplicate is not reported when both definitions come + from one object of this plan linked into both images; the decision is by + provenance, not by a `_ZGIW` name pattern, so a real duplicate with an + initialiser-shaped name is still reported. Today that object set is + `std.o` and `std.compat.o`. +- **Rejected: one owner of `std.o`.** An image without the definition cannot + link on Mach-O or PE and cannot be loaded by a host that did not link the + owner (F1c and F1d prove the same for any symbol); sibling shared libraries + have no common owner; and the object holds nothing worth owning. +- **Criterion.** Under a uniform contract, no `build/symbol-provision` finding + and `--strict` exits 0; a static zlib in the program over a shared `libz` + still reports. + +### 4.3 F1: a static dependency of a shared package goes into the program + +**Read.** A dependency-owned shared unit receives its own package's objects +and its direct shared dependencies (`plan.cppm:1914-1929`); the root's units +receive every compile unit of every non-shared package +(`plan.cppm:1994-2000`, `2100-2115`). + +**Measured (group a).** + +``` +F1a libfw.so inputs: obj/mcpplibs_fw/src/fw.o undefined x_answer=1 program exports x_answer=1 run exit=0 +F1c libfw.so relinked with -z defs: exit=1 undefined reference to `x_answer' +F1d foreign host dlopen(libfw.so, RTLD_NOW): exit=1 ... undefined symbol: x_answer +F1b root also depends on x: x.o linked into program=1 libfw.so=0; run counter=2 +F1f libfw.so relinked with x.o as well: libfw defines x_counter=1; run counter=2 +F1e root-owned shared image over x: libfw.so defines x_answer=1 undefined=0 +``` + +F1c is the Linux form of the Mach-O `-undefined error` and PE link claims; +F1d is the Linux form of Android's `System.loadLibrary("fw")` before the +application, and of any consumer outside mcpp. F1f shows that ELF also hides +the state split a duplicate would cause elsewhere: the interposed copy wins +and the counter reads as shared. + +- **Kind:** defect, general (every format but ELF-with-this-consumer). +- **Home:** engine (`make_plan`). Placement is plan logic; no cheaper home + exists. +- **Decision (R1).** For each shared image, its static closure is the set of + static packages reachable from its package without crossing another shared + package; `directPackageDeps` (`plan.cppm:1800-1827`) already holds the + edges. A static package in exactly one closure is linked into that image and + removed from the root's object set. A static package in two or more closures + (two images, or an image and the program) is the "one library, one provider, + one form" conflict and is refused before compiling, with a refusal code + naming the package, the images, and the remedy that `linkage_form` already + implements: `linkage = "shared"` on that package, per dependency or as its + default. Silent duplication is rejected because on Mach-O and PE each image + would keep its own state, which F1f shows ELF would hide. +- **Consequence.** The comment at `symbol_provision.cppm:36-43` is rewritten; + the arrangement it calls benign is removed. +- **Criterion.** On Linux, M3b's `libfw.so` links under `-Wl,-z,defs` and + loads from a foreign host; F1b is refused naming `x` and builds after + `linkage = "shared"` on `x`; on macos-15, M3b's dylib links (temporary + pull request, §9.2). + +### 4.4 F2: the payload's Mach-O shared default + +**Read.** On macOS the deployment floor is never empty (`macos.cppm:91, +105-111`), so programs and dylibs alike are `self-contained` and embed +`-Wl,-load_hidden,libc++.a` and `libc++abi.a` +(`distribution.cppm:214-251, 572-593`); `toolchain-coupled` is refused on +Mach-O (#202). The payload's `include/c++/v1/typeinfo:146-187` states that on +arm64 Apple types are "considered different types" across linked image +boundaries under the non-unique RTTI bit, and x86_64 Apple compares unique +type information by address. The inference is that a `std::runtime_error` +thrown in a dylib is not caught by its class in the program, and that any +libc++ object compared by address, such as an `std::error_code` category, +compares unequal across images. + +- **Kind:** probable defect, general; working builds depend on the default. +- **Home:** engine, after a measurement. +- **Decision.** No change before the measurement. The measurement (§9.2) runs + the #641 M3 app/fw pair without `llvm.libcxx` on macos-15 against the + released tarball, in three legs: the default; `cxx_runtime = "host-coupled"` + project-wide (both images on `/usr/lib/libc++.1.dylib`); and the default + program over a `{ shared = "host-coupled" }` library. The reading names + whether `std::runtime_error`, the library's own exception type and + `std::errc` comparisons survive the boundary. If the default leg splits, + R1 applies per format: on Mach-O the only runtime shared across images is + the system's, so a graph with a C++ dylib couples every image to it, subject + to the floor, and an explicit `self-contained` program receives a + diagnostic. Because working builds change, the first release warns, as the + rule-E precedent did. +- **PE.** The `msvc` dialect defaults to `/MD` and shares one CRT across DLLs + (`distribution.cppm:615-620`). The llvm row gives every image a static CRT + (§4.5); MSVC exception handling matches catch types by decorated name + (read), so the type split probably does not occur there, while per-image + CRT state (`FILE*`, `errno`, locale, `atexit`) does. The windows-2022 leg of + the same fixture answers it. + +### 4.5 E10: the llvm row's runtime on `x86_64-windows-msvc` + +**Measured (group a, the driver's decision is host-independent).** + +``` +E10 compile, no flag: dependent-lib= defines= +E10 link, no flag: -defaultlib:libcmt -defaultlib:oldnames +E10 -fms-runtime-lib=dll: compile --dependent-lib=msvcrt --dependent-lib=oldnames link -defaultlib:libcmt -defaultlib:oldnames +``` + +**Read.** The CRT flag is emitted only for `isMsvcDialect` +(`flags.cppm:888-895`); the clang std module path never takes +`msvc_crt_flag` (`stdmod.cppm:330-346`), whose comment says non-MSVC dialects +yield `""` while `msvc_crt_flag(gnu, true)` returns `"-static"` +(`dialect.cppm:67, 134-135, 211`). The PE contract table records +`host-coupled` for the row (`distribution.cppm:615-686`), and runtime staging +is guarded on `CompilerId::MSVC` (`flags.cppm:1370`). + +The issue's three questions, answered: + +1. The static runtime is not a decision. It is clang's link-time default + (`-defaultlib:libcmt`), and without `_DLL` the MSVC STL headers select the + static C++ library, which matches the imports HuxerUI read. The record says + otherwise. +2. No value selects the dynamic runtime. `host-coupled` and + `toolchain-coupled` are recorded, not delivered, with no diagnostic. +3. Every llvm-row DLL carries its own CRT and STL, so the row is in F2's class + for CRT state; whether exception identity also splits is the windows-2022 + measurement. + +- **Kind:** defect (R4: the record and the artifact disagree), and a default + that deserves a decision. General. +- **Home:** engine. +- **Decision, in two steps.** Step 1 changes no artifact: for clang on the + MSVC ABI the record states `self-contained` unless a runtime flag is + emitted; an explicit `host-coupled` or `toolchain-coupled` receives a + degraded diagnostic saying the row does not deliver it; the `msvc_crt_flag` + comment is corrected; docs/20 states the row's model. Step 2, in its own + record after the windows-2022 measurement, decides whether the llvm row + adopts the `cl.exe` model (`/MD` by default through `-fms-runtime-lib`, the + same `msvc_wants_static_crt` derivation, the std module included, the + redistributable staging extended to the row). Step 2 is CMake parity and + changes every existing llvm-row artifact, which is why it is not taken here. +- **Criterion (step 1).** `resolution.json` for the default llvm row on + windows-2022 records `self-contained`, and `cxx_runtime = "host-coupled"` + prints the diagnostic. + +## 5. #647 E1 to E3 and #649 E5, E9: what build programs see, and what pack does + +### 5.1 E1: the resolved graph and package metadata + +**Read.** The root build program runs at step L3 +(`prepare.cppm:10710-10808`), after resolution, feature activation, every +dependency's build program and link-form resolution, so the whole graph is +known when it runs. `fillDepDirs` publishes only the consumer's visible +provisions: direct dependencies and reexports, under the bare and the +qualified name (`:6016-6055`). `resolution.json` receives a `graph` section +(`:13479-13505`, the #634 X decision) only at the end of `prepare`, and a +build program never sees it. A build program re-runs when its program, its +compiler, its contract environment (`contract_hash`, +`build_program.cppm:721-727, 971-972`) or a declared input changes; nothing +about a transitive package is in that key. + +**Measured (group b).** + +``` +E1-root-env: SPIKE deps: MCPP_DEP_A_DIR MCPP_DEP_SPIKE_A_DIR MCPP_DEP_A_LINKAGE MCPP_DEP_SPIKE_A_LINKAGE +E1-program-order: SPIKE order a SPIKE order b +E1-metadata-edit: build.mcpp up to date (cached)x3 replayed=yes +E1-strict-metadata-table: exit=0 warnings=0 +E1-unknown-package-key-strict: exit=0 mentions=0 +``` + +- **Kind:** gap, general: the capability CMake target walks and + `cargo metadata` provide. +- **Home:** engine. A plugin cannot enumerate transitive packages: registry + and git roots are invisible to it, and `dep_dir` is scoped to provisions on + purpose. #634 A8 deferred this "until a need transitive deploy cannot + answer"; merging every library's resources into one package at build time + is such a need, because deploy stages files and does not declare a build + action over them. +- **Decision (R5).** + 1. The root build program receives `mcpp::graph_file()` + (`MCPP_GRAPH_FILE`), an absolute path to a JSON document written before + L3. It holds the per-package objects of `resolution.json`'s `graph`, + extended with `manifest_dir`, active `features`, target kinds and the + resolved link form, in topological order (dependencies first, ties in + discovery order). A file rather than variables, because a graph with + metadata exceeds what `MAX_ARG_STRLEN` and the Windows environment + block allow; the precedent for a path a program reads is + `pack_stage_dir()`. Root only, as `dep_linkage` is: the root holds the + final facts. The JSON reader `mcpp-plugins/dist/apk.cppm:1027-1135` + moves into a shared plugin module so rules read the file with one + parser. + 2. `[package.metadata.]` is copied verbatim into that package's + entry as `metadata`. The engine does not interpret it; a consumer + resolves paths against `manifest_dir`. Older clients already ignore the + table (X4), so publishing it breaks none of them (R6). + 3. The document's digest joins the contract environment, so a change of any + graph package's identity, features or metadata re-runs the root program, + and an edit to a package's sources does not. + 4. The open question (a dependency's program emitting values for its + dependents) is deferred: manifest metadata serves both stated needs, and + such a channel first requires dependency programs to run dependencies + first, which they do not (E1-program-order). +- **X4.** When `[package]` gains a known-key check, `metadata` is in its list; + the check itself is a separate, compatibility-sensitive change and warns + before it refuses. +- **Criterion.** The issue's fixture: the root program prints `spike.b` after + `spike.a` with `b`'s resolved metadata; editing `b`'s metadata changes the + run, editing `b/src/b.cpp` does not. + +### 5.2 E2: Swift sources + +**Read.** The inferred glob has no `.swift`, and nothing in mcpp or +mcpp-plugins mentions it. `rules-metal` is the precedent for a language that +lives in a rule package: its feature declares +`device_extensions = [".metal"]` (`mcpp-plugins/mcpp.toml:91-95`), the rule +receives its sources through `mcpp::device_sources()` and locates tools with +`xcrun --sdk --find`. The engine already offers what one package needs: +an `action` whose role `object` joins the image link, a `source` action with +companion outputs for a generated header, `link_flag` (LinkGlobal, +`directives.cppm:320-340`), and `include_dir` (PackagePrivate by design, +`:28-34`). + +- **Kind:** feature, Apple rows. +- **Home:** official plugin, `rules-swift`. The issue's CI check (one + package, `@_cdecl` exports, Swift calling C through a bridging header) needs + no engine change: one whole-module `swiftc -emit-object -target ` + action, a generated `-Swift.h` through a `source` action and `include_dir`, + and `link_flag` for the Swift library search paths and + `-Wl,-rpath,/usr/lib/swift`. +- **What a plugin cannot do.** A Swift `import` of another package's module, + and a C++ consumer in another package reading a generated header, both need + an interface directory that dependents can see. Compile-interface directives + are PackagePrivate on purpose, and dependency programs run dependents first + (§5.1). That engine capability is named here and not designed; it is E1's + deferred question in another form. +- **Unmeasured (macos-15).** Whether `ld64.lld` under `-fuse-ld=lld` honours + the `LC_LINKER_OPTION` autolink entries of swiftc objects + (`swiftCompatibility*`, `swiftCore`), and the `aarch64-ios-sim` link. + +### 5.3 E3: an Android row does not link on a macOS host + +**Read, line by line.** `crossTarget` is `--target=` +(`flags.cppm:553-555`), and an own-sysroot row carries it only in +`link_toolchain_flags` (`:729-754`). The host branch is chosen by +`if constexpr`: Windows at `:1561`, macOS at `:1637`. The macOS branch +assembles `f.ld` from `full_static`, `b_flag`, `apple_cross_ld`, `macos_sdk`, +`version_min`, `-fuse-ld=lld` and the user and extra flags (`:1753-1756`), and +never appends `link_toolchain_flags`; `apple_cross_ld` is set only for an +Apple SDK row (`:1748-1752`); `macos_sdk` falls back to the host SDK +(`:1707-1711`); `version_min` is added whenever the row is not an Apple +cross (`:1683-1685`). An Android link therefore has no `--target`, which +selects `ld64.lld`, and carries the macOS `-isysroot` and +`-mmacosx-version-min`. The issue's cause is exact. Freestanding rows and +graph-supplied C-library rows replace the line after the host branch +(`:1841`, `:1958`) and are not affected. + +- **Kind:** defect (R2: the host answers a question that belongs to the + target), general. +- **Home:** engine. +- **Decision.** The link branch is chosen by host and target object format, + not by host alone: the Apple SDK branch applies to Mach-O targets, the + Windows `lld-link` branch to PE targets (MSVC keeps `link.exe`), and every + other target takes the generic branch that consumes + `link_toolchain_flags`. The choice is a pure function, for example + `link_shape(host_os, triple, dialect)`, with a unit test over host by row + that runs on Linux (macOS host and `x86_64-linux-android` gives the target + driver; macOS host and `aarch64-ios-sim` gives the Apple SDK; Windows host + and `x86_64-linux-android` gives the target driver). The generic branch is + already compiled on every host, since a discarded `if constexpr` branch in + non-template code is still checked. +- **Criterion.** The unit test above; on macos-15, + `mcpp build --target x86_64-linux-android` produces a file `file` reports + as ELF, and mcpp-plugins restores its macOS `dist-apk` steps. + +### 5.4 E5: pack strips the program of some rows and nothing the graph built + +**Measured (group b).** + +``` +E5-desktop-status: Packing hostapp v0.1.0 (vendored, stripped) +E5-desktop: bin/hostapp bytes=14280 symtab=0 debug=0 +E5-desktop: lib/libdep.so bytes=16592 symtab=1 debug=7 +E5-android-status: Packing app v0.1.0 (vendored, stripped) +E5-android: lib/libapp.so bytes=5904 symtab=1 debug=1 +E5-android: lib/libc++_shared.so bytes=9091400 symtab=1 debug=6 +E5-android: lib/libdep.so bytes=5496 symtab=1 debug=1 +``` + +**Read.** `strip_program` strips the staged program with the `Executable` +shape (`pack.cppm:818-837`) and is called from the PE and ELF paths +(`:1503`, `:2079`); `bundle_libs` copies closure libraries, graph-built ones +included, unchanged; the Android row's dispatch skips the strip step entirely +(§1 item 6). The Mach-O row does not strip on purpose (`:1774`). The NDK's +`llvm-strip` is already resolved by `binutils_tool` +(`registry.cppm:1446-1450`). + +- **Kind:** defect (a dropped row, and a status line that reports a decision + as an outcome, R4), general. +- **Home:** engine; `dist-apk` 0.11.1's own strip becomes unnecessary. +- **Decision.** + 1. The program is stripped on every row that strips, inside + `run_shared_program` too, per leg with that leg's tools, using the + `SharedLibrary` shape when the program is a shared object. + 2. Every shared library this graph built is stripped in its staged copy + with `--strip-unneeded`, which keeps `.dynsym`. `pack::Plan` receives the + absolute outputs of `plan.linkUnits` whose kind is `SharedLibrary`. The + comment's rule ("a bundled `.so` is somebody else's file") stays true for + store and host files; the dh_strip rule it quotes, "a package strips + what it built", covers these. + 3. A staged copy of the toolchain's runtime (`libc++_shared.so`) is stripped + with `--strip-unneeded`. The comment's reason, not changing a shared + payload's bytes, does not apply to a copy; Gradle strips it. `--no-strip` + covers every case. + 4. The status line reports what was done: a row that stripped nothing does + not print "stripped". + 5. Build programs read the decision through `mcpp::pack_strip()` and + `mcpp::pack_debug_symbols_dir()`, in the contract environment beside + `pack_format()` and `pack_stage_dir()` (`hostprogram.cppm:544-562`), for + the AAR and Maven libraries `dist-apk` stages itself. +- **Criterion.** The issue's three checks; they fail today per the readings. + +### 5.5 E9: a machine-readable pack report, and the profile switches + +**Measured (group b).** + +``` +E9-pack --release: exit=2 [error: unknown option: --release] +E9-pack --dev: exit=2 [error: unknown option: --dev] +E9-pack --message-format json: exit=2 [error: unknown option: --message-format] +E9-human-lines-on-stdout: bytes=273 +E9-run-precedence(--profile dev --release): Finished release +E9-build-precedence(--profile dev --release): Finished dev +``` + +**Read.** `PackOutcome` already holds absolute artifact paths and moves the +outputs another action consumed into `intermediate` +(`pipeline.cppm:34-46, 693-719`); only `shorten_path`'s form is printed. +docs/50 §3 reserves `--format json` for machine output, while `pack`'s +`--format` names the package format; `test --message-format json` is an NDJSON +event stream; `emit build-database` sends its narration to stderr. + +- **Kind:** gap and inconsistency, general. +- **Home:** engine; only the engine holds `PackOutcome`. +- **Decision.** `mcpp pack --message-format json` prints one enveloped + `mcpp.pack` document on stdout and sends every human line to stderr, as + `emit build-database` does. `data.artifacts[]` holds + `{path, type: "file" | "directory", format, targets[]}` with absolute paths; + legs appear as `targets` of the artifact they produced, and intermediate + outputs are omitted; `data.stage` holds the stage directory, manifest and + closure. `--protocol-version` advertises the kind. docs/50 §3 gains one + sentence: a command whose `--format` names its product asks for machine + output with `--message-format json`. The shape follows the kind: a stream + for `test`, whose events arrive over time, and one document for `pack`, + which has one result. `pack` accepts `--release` and `--dev`; the pack + default stays `release`. +- **X5.** `run` adopts `build`'s precedence (`--profile` over the + shorthands), one derivation for the three commands (R2). +- **Criterion.** The issue's two checks, and `mcpp run --profile dev + --release` finishing `dev`. + +## 6. #647 E4 and #649 E6 to E8: features, tools and git sources + +### 6.1 E4.1: a forward along a build-dependency edge is applied and reported as undeclared + +**Read.** The validator looks the key up in `dependencies` and +`devDependencies` (`prepare.cppm:6677-6678`), while `injectForwards` applies +forwards to `buildDependencies` for the root (`:6718-6728`) and for +transitive packages (`:7690-7698`). Active `[feature-deps]` are merged into +`dependencies` before validation (`:6693`, `:7611`), and conditional tables +are folded into the same maps (`:497-506`). + +**Measured (group c).** + +``` +E4.1-build: 2 forward warnings; tool kt built: 1; rc=0 +E4.1-strict: error: feature 'kotlin' of 'fw' forwards to dependency 'spike.rules' ... rc=2 +E4.1-control-without-feature: tool kt built: 0; rc=0 +E4.1b-forward-to-other-row-dep: error: feature 'kotlin' of 'fw' forwards to dependency 'spike.win' ... rc=2 +``` + +- **Kind:** defect (R2), general; X8 is its conditional-row twin. +- **Home:** engine. +- **Decision.** The validator asks the question the forward language + defines: is the key declared in any dependency table of this manifest, + on any row and under any feature? `conditionalConfigs` survives the merge + (`types.cppm:1774`), so the check can read every row. A declared key that + is not in the graph on this row does nothing and says nothing; a key + declared nowhere keeps the #243 warning and its `--strict` error. +- **Criterion.** E4.1 and E4.1b build under `--strict` without a warning; a + forward to a key declared nowhere still exits 2. + +### 6.2 E4.2: `[feature-deps]` and a source-less `tools` entry + +**Read.** docs/05 (`docs/05-dependencies.md:344-346`, `zh/05:304`) says a +feature-deps entry "may add `tools`" to a dependency declared elsewhere. The +parser tells a dependency table from a namespace table by the presence of a +source key (`toml.cppm:1383-1389`) and refuses an options-only table with a +message that prescribes restating the source (`:1400-1440`). The merge +already implements the intent of #359: `tools` and `features` are unioned, +`host-module` and `reexport` are or-ed, and identity fields are dropped +(`prepare.cppm:6603-6631`). docs/30 states the restated form. + +**Measured (group c).** + +``` +E4.2-sourceless: error: ... must be a string, inline dep table, or nested table ... rc=2 +E4.2-restated: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/installer@.../bin/installer] qualified=[]; rc=0 +E4.2-restated-without-feature: SPIKE short=[] qualified=[]; rc=0 +E4.2-restated-different-path: (--strict) SPIKE short=[.../installer...]; rc=0 +``` + +- **Kind:** documentation defect; plus a silent drop (a restatement whose + source differs from the declaration in effect is ignored). +- **Home:** docs; engine for the check. +- **Decision (R6).** The documentation changes, not the parser. A + source-less table would change the parser's discriminator, and a published + package that used it would fail to load on every older client, which is the + hazard the parser's own comment describes (`toml.cppm:1370-1382`). + Restating the source is already the grammar for conditional dependencies + (#634 A1) and is what the refusal prescribes. `mergeActiveFeatureDeps` + refuses a restatement whose path, git source or version differs from the + declaration in effect on that row, after the conditional fold. +- **Criterion.** E4.2-restated-different-path exits 2 naming both sources; + docs/05 and its translation say to restate the source. +- **Read, not measured.** Feature-deps merge into `dependencies` + (`prepare.cppm:6609`), so a feature cannot scope a build-only dependency; + restating a `[build-dependencies]` key under a feature probably moves it to + the target axis. This is recorded for the implementation, not decided. + +### 6.3 E4.3: the qualified `dep_bin` spelling is missing + +**Read.** Tool variables are published at `prepare.cppm:9226-9233`, from +`manifest.package.name`, whose short form is taken after the last dot. For a +manifest with `namespace = "spike"` and `name = "installer"`, the name has no +dot, the qualified and short spellings coincide, and only +`MCPP_DEP_INSTALLER_BIN_*` is emitted (`:9272-9289`). #642 fixed the same +shape for `dep_dir` and `dep_linkage` in `fillDepDirs` (`:6036-6044`, through +`qualified_package_name`, `plan.cppm:435`); the tool site is a second +publication point that fix did not reach. The only test uses the legacy +dotted name (X9). + +- **Kind:** defect (R2), general. +- **Home:** engine. +- **Decision.** One helper returns the names under which a provider is + published to a consumer (qualified, canonical, and bare when bound), and + both `fillDepDirs` and the tool record use it. `bareBindingsFor` + (`:5838-5848`) reads `package.name` as well; two packages with one name in + different namespaces probably collide on the bare variable (read), which + the helper's test covers. +- **Criterion.** e2e 187 gains a `namespace =` plus `name =` package, and + `dep_bin("myns.tp", ...)` answers. + +### 6.4 E6: a feature-gated tool that depends on the package declaring it + +**Read.** The error is raised by the build-cache key walk +(`prepare.cppm:12861-12930`), which runs only for the global cache +(`:12790`). The cycle is in the consumer's graph: `fw-installer` is merged +into `fw.dependencies` (`:6609`), the worklist walks the tool package's own +`[dependencies]` into the consumer's graph (`:7680-7686`), and `spike.fw` +meets the resolved `fw`, recording `fw-installer -> fw` +(`:7266-7270`). The tool itself is built by a sub-build "with its own +resolution" (`:8876-8881`) and succeeds. + +**Measured (group c).** + +``` +E6-issue-fixture: SPIKE short=[.../fw-installer] qualified=[] | error: dependency cycle through package 'fw' while computing its build-cache key; rc=2 +E6-tool-runs: fw-installer ran 42 +E6-cache-local: rc=0; app runs: exit=0 +E6c-tool-dependency-in-consumer: app link line: obj/main.o obj/spike_z/src/z.o obj/spike_fw/src/fw.o; app defines z symbol: 1; rc=0 +E6b-tool-activates-declaring-feature: ... tool provisioning nested more than 4 levels deep — this is almost certainly a cycle.; rc=2 +``` + +E6c is the wider defect: when the tool depends on a library `z`, `z.o` is +linked into the application, and the tool's `main.cpp` is compiled in the +consumer's build. + +- **Kind:** defect, general. +- **Home:** engine. +- **Decision.** A package that declares no library target has nothing to link + and contributes nothing to a consumer's target graph: its `[dependencies]` + and `[build-dependencies]` are not walked into that graph and its sources + are not compiled there. Its programs come from the tool sub-build, which + resolves them on its own; that is the model the sub-build's comment already + states, and the shape of Cargo's artifact dependencies and Bazel's exec + configuration. A package with no `[targets]` (whose inferred library + compiles into the consumer) keeps today's behaviour. This removes the cycle + edge, the link leak and the tool's participation in the consumer's version + resolution, with no change to any key. +- **Rejected: keying builds on (package, features, toolchain).** The tool + store's key already carries features, host triple and compiler identity + (`tool_store.cppm`), and splitting target-graph nodes by feature set would + contradict one identity per graph with features unioned (the + one-package-one-version rule). +- **X2 and X3.** Package-edge cycles are checked at resolution, in one place, + with a message naming the edges, so `--cache=local` and the global cache + agree. A repeated (package source, tool) in the tool chain is refused at + its first repetition, naming the feature edge, instead of after four + nested sub-builds. +- **Criterion.** The issue's fixture exits 0 with the default cache, `dep_bin` + answers, and the tool prints `fw-installer ran 42`; E6c's application link + line has no `spike_z`; E6b is refused once, naming the edge. +- **Blast radius to measure.** Before implementation, the manifests of the + e2e corpus, the examples, mcpp-index and mcpp-plugins are scanned for a + dependency on a package whose targets are all programs and which is not + requested for `tools`; any such edge is a consumer that relied on the leak. +- **A modelling note, not a decision.** `tools` builds for the host. + HuxerUI's installer interface is a program for the Windows target, so the + channel serves it only when host and target coincide, which holds for a + Windows host packaging a Windows setup. + +### 6.5 E7: a git dependency resolves only the repository's root package + +**Read.** A git dependency's root is the clone root +(`prepare.cppm:7376-7455`); its identity is url plus ref (`sourceRefOf`, +`:6736-6744`), so a second key over the same source takes the first key's +identity (#634 A2, `:6800-6829`). Declaring the member alone fails the name +check (`:7536-7549`), and membership is tested against the consumer's +workspace, not the repository's (`:7477-7483`). + +**Measured (group c).** + +``` +E7-both-keys: 1 adoption warning(s) | SPIKE short=[] | Compiling spike.fw v | rc=0 +E7-member-only: error: dependency 'spike.fw-installer' resolved to package 'fw' (mismatch with declared name 'fw-installer'); rc=2 +E7-member-by-path: SPIKE short=[.../fw-installer...] | rc=0 +EDUP-dependencies-and-build-dependencies: tool built: 0 | SPIKE short=[] | rc=0 +``` + +- **Kind:** gap, general to multi-package repositories; X1 and X6 beside it. +- **Home:** engine. +- **Decision (R6).** A git dependency whose key names an identity other than + the root manifest's is looked up among the repository root's + `[workspace] members`, at the same commit. The source identity becomes + (url, commit, member path); the member inherits the root's + `[workspace.package]`; a member's `path` edge that stays inside the clone + resolves as the same git source at that commit, so `spike.fw = { path = + ".." }` inside the member and `spike.fw = { git = ... }` in the application + name one package. The #634 A2 adoption applies only when no member declares + the key's identity. Cargo resolves a git dependency by package name within + the repository in the same way. +- **Rejected: a `subdir` key.** An older client ignores an unknown key and + would build the root package, which is a silent wrong build, not a + degradation; and a path is a second statement of identity that can disagree + with the key. +- **X1.** A second declaration of one dependency by one consumer merges into + the existing edge additively, as `mergeActiveFeatureDeps` already does, + instead of returning early. +- **X6.** The banner prints the commit for a git source, or the manifest's + version. +- **Criterion.** E7-member-only and the both-keys form build the member's tool + with no identity warning; EDUP builds the tool. + +### 6.6 E8: `--features` and a dependency's feature + +**Read.** `parse_feature_request` splits on commas and spaces +(`prepare.cppm:650-661`). The unknown-name check applies only when the root +declares `[features]` (`:8531-8548`), and docs/06:40-42 documents that a +package without the table "accepts any request (pure macro usage)"; each name +becomes `-DMCPP_FEATURE_` (`:8348`). + +**Measured (group c).** + +``` +E8-dep-feature-no-table: SPIKE short=[] qualified=[] | tool built: 0; rc=0 +E8-dep-feature-no-table-macro: -DMCPP_FEATURE_SPIKE_FW_INSTALLER +E8-unknown-no-table: rc=0; macros: -DMCPP_FEATURE_NOTHING_HERE +E8-dep-feature-with-table-strict: error: --features requests 'spike.fw/installer' which [features] does not declare; rc=2 +E8-manifest-forward-control: SPIKE short=[.../fw-installer...] | tool built: 1; rc=0 +``` + +- **Kind:** a defect narrower than the issue states (a token that can only be + a forward becomes a macro), and a gap (Cargo accepts `--features + dep/feature`). +- **Home:** engine. Today's usage-side answer is the root forward line of + the control reading. +- **Decision (R2).** A token containing `/` is split with + `split_feature_forward_token` and applied through `injectForwards` and + `validateForwards` as a forward of the root, over the table set of §6.1. + One that names no dependency is warned about, and refused under `--strict`, + whether or not the root declares `[features]`; it never becomes a macro. + Plain names keep their documented meaning. `pack`, `run` and `test` inherit + the behaviour through the shared `overrides.features`. +- **X7.** `mcpp why deps` accepts `--features`, so the graph of a feature + build is visible where the graph of a default build is. +- **Criterion.** E8-dep-feature-no-table builds the tool and `dep_bin` + answers, with no `MCPP_FEATURE_SPIKE_FW_INSTALLER`; `--strict --features + nope/x` exits 2. +- **Open.** Whether a forward, in a manifest or on the command line, may name + a dependency by the qualified identity after A2 adoption as well as by its + key. + +## 7. #648: an editor that plans in the background + +The issue's six asks are three defects of one kind (children that are not +owned or bounded), two contract gaps of the machine interface, and one +ecosystem-data item. The trigger of the field hang is a seventh finding that +the issue does not name, and it is the only one that sends a project to the +network on every build. + +### 7.0 T: what started the refresh on openxlings/xlings + +The issue establishes that an offline plan of the same project is complete +and byte-identical, so the refresh was not needed. It does not establish which +decision asked for it. Planning openxlings/xlings with `MCPP_OFFLINE=1 -v` +prints the refresh decision per dependency, and the decision for `ftxui` is +the suppressed kind, which `decide_for_dependency` assigns only after it has +decided to refresh (`index_refresh.cppm:237-280`). The same run then resolves +the dependency: + +``` +T decision ftxui@6.1.9: offline mode +T resolver warning: dependency 'ftxui' resolved to 'compat.ftxui' through the deprecated bare-name search; namespace omission means `mcpplibs` only. +``` + +The two functions answer one question with two ladders. The refresh decision +looks up the canonical coordinate `(mcpplibs, ftxui)`, which is conclusive +because `mcpplibs` is authoritative, and calls the miss a `DescriptorMiss` +(`index_refresh.cppm:221-249`; the reason is inferred, since the offline +opt-out overwrites it). The resolver, after the same miss, tries +`mcpp::pm::legacy_bare_candidates` for a version selector whose namespace was +omitted and reaches `compat.ftxui` (`prepare.cppm:5113-5160`); its comment +states the reach of that rung: every manifest written before it spells such +dependencies bare. The debounce is 120 seconds (`xlings.cppm:522`), so online +every plan or build of such a project that starts more than two minutes after +the last successful refresh runs a network `xlings update`, editor or not. An +editor that plans on every save meets it several times an hour; it made the +unbounded wait visible. + +- **Kind:** defect (R2), general to every manifest with a bare `compat.*` + dependency; also a usage issue in the xlings manifest, whose warning names + the spelling. +- **Home:** engine; openxlings/xlings. +- **Decision.** The refresh decision consults `legacy_bare_candidates` under + the resolver's condition (`spec.isVersion() && spec.namespaceOmitted`) + before it calls a miss, so both walk one ladder for as long as the + deprecated rung exists. The rung's warning announces its removal in 2026.9; + the decision must not depend on that schedule. openxlings/xlings writes + `compat.ftxui = "6.1.9"`. +- **Criterion.** A fixture whose dependency resolves only through the + deprecated rung, planned offline with `-v`, prints no suppressed decision + for it; planned online against a stub xlings that records its argv, it + starts no `update`. + +### 7.1 A1: an offline plan that needs a download has no code + +**Read.** Every offline refusal is prose: the toolchain payload +(`prepare.cppm:3452-3475`), `[xlings]` provisioning (`:1541`), a dependency +download (`:5468-5474`), the fetcher's install (`package_fetcher.cppm:1114-1119`) +and a git dependency (`prepare.cppm:7308-7321`). `emit build-database` maps +every planning failure to `MCPP_BUILD_DATABASE_PLAN_FAILED` +(`cmd_build.cppm:375`). An absent index under `--offline` is not refused at +all: `update_index_unguarded` reports success (`xlings.cppm:1923-1926`) and the +resolver later fails on a missing descriptor. + +- **Kind:** gap in the machine interface (R4), general. +- **Home:** engine; no client can separate the two failures without matching + the message, which docs/50 forbids. +- **Decision.** One refusal code, `offline-download-required`, recorded in + the per-run refusal sink (`src/build/refusal.cppm`, set immediately before + the `return`) at each site above and at the absent-index case, with the + subject's kind, name, version and index. `emit build-database` then uses + `MCPP_OFFLINE_DOWNLOAD_REQUIRED` and one diagnostic naming the subject and + the command (`mcpp build` without `--offline`, or `mcpp index update`); + `mcpp why toolchain` reports the token. Planning stops at the first + refusal, so the diagnostic names the first missing item. The issue asks for + one per missing item; that would require resolution to continue past a + refusal, which no other refusal does, and the client's remedy (one online + build) is the same. +- **Criterion.** The issue's e2e case, and a home without an index planned + offline yields the code. + +### 7.2 A2: children inherit the caller's pipe + +**Measured (group d).** + +``` +A2 reader stdin=pipe:[429071926] +A2 child fd 3 -> pipe:[429071926] +``` + +A build program started by `emit build-database` holds, as descriptor 3, the +pipe the caller reads. The leak is not specific to `popen`: the owned +launcher's `posix_spawnp` (`process.cppm:648-655`) passes it on too, because +the saved descriptor is created by `dup(1)` without close-on-exec +(`terminal.cppm:76-85`). A build program, hook or xlings process that outlives +mcpp keeps the caller from reading end-of-file. + +- **Kind:** defect (R3), general to every caller that reads mcpp through a + pipe. +- **Home:** engine (`mcpp.platform.terminal`). +- **Decision.** Save with `fcntl(1, F_DUPFD_CLOEXEC, 3)`; on Windows clear + `HANDLE_FLAG_INHERIT` on the saved handle. The issue's second measure, + closing every descriptor above 2 in each child, is not available through + `posix_spawn` on musl, which the Linux release links, so it cannot be the + rule; the rule is that a descriptor mcpp keeps open across a spawn is + close-on-exec. The survey found one such descriptor: `FileLock` already + opens with `O_CLOEXEC` (`fs.cppm:314`), and every other open or stream is + scoped to a function that starts no child. +- **Criterion.** Reading A2 shows no descriptor naming the reader's pipe; with + a build program that leaves a sleeping grandchild, a reader of the envelope + sees end-of-file within one second of mcpp's exit. + +### 7.3 A3: xlings runs unbounded and outside mcpp's ownership + +**Read.** `run_exec` and the deadline runners give a child its own process +group, register the group with a signal guard that kills it when mcpp is +terminated, and use a job object on Windows (`process.cppm:631-667`, +`unix/bounded_process.cppm:414-470`); this was built after an orphaned ninja +outlived its sandbox. The xlings calls are the exception: `call`, +`update_index_unguarded`, `install_direct` and the bootstrap go through +`popen` or `std::system` (`xlings.cppm:1421, 1620, 1692-1724, 1955`), with no +deadline and no group. The refresh retries three times on a non-zero exit +(`:1950-1967`), which does nothing for a stall. + +- **Kind:** defect (R3), general: a terminal build on a black-holed network + hangs the same way. +- **Home:** engine for the bound; xlings for the missing connection timeout in + `xlings update`, which the reporter files there. +- **Decision.** + 1. xlings invocations move to the owned launcher, argv-based, with the + streaming sink the deadline runner already accepts. Ownership and the + move happen together: a child in its own group without the guard would + no longer receive a terminal's Ctrl-C, which would be a regression. + 2. The index refresh has a total deadline, `[index] refresh_timeout` + (seconds). A timed-out refresh is a failed refresh, is not retried, and + the existing rule then holds: warn, resolve from local data, fail only if + the local data cannot answer (`prepare.cppm:4465-4472`). + 3. An install has an inactivity deadline. The interface protocol emits a + heartbeat after five seconds without output + (`xlings/src/interface.cpp:249-263`), so no line for the deadline means + the xlings process is wedged, while a slow download that progresses is + never killed. A stalled connection inside a live xlings is bounded by + xlings' downloader. + The default values are measured before they are fixed (§9.3). +- **Criterion.** The issue's two e2e cases with the stub named through + `[xlings] binary`, and a Ctrl-C in a pseudo-terminal during a stubbed + install that leaves no stub process. + +### 7.4 A4: the envelope omits the network access that happened + +docs/50 §2 defines the envelope's `effects` as "what running the command did", +and §4 defines the `--protocol-version` table as what a command may do. `emit +build-database` builds its list from constants and one observation +(`cmd_build.cppm:427-428`), so a run that refreshed the index reports no +`network`. + +- **Kind:** defect against docs/50 (R4), general to enveloped commands. +- **Home:** engine. +- **Decision.** A per-run record of observed effects, written where a network + child is launched (the index refresh, an install, a git remote operation, + the bootstrap) and read by every enveloped command. An attempt that failed + or timed out counts: the effect is the access. Under `--offline` those sites + return before launching, so the record cannot contain `network`. +- **Criterion.** The issue's. + +### 7.5 A5: `auto_refresh = false` does not govern three refreshes + +**Read.** `mcpp.pm.index_refresh` exists so one function decides whether a +run touches the network for an index; its header records that five +independent derivations had drifted (#315). Three callers bypass it: the +fetcher's pre-install refresh (`package_fetcher.cppm:1127`, through +`ensure_official_package_index_fresh`, which checks neither the policy nor the +module's debounce), the fetcher's refresh before a retry (`:1205`), and the +first sync of a project's custom index (`prepare.cppm:4644-4662`). docs/05 +states that `auto_refresh = false` means "never refresh the index +automatically". + +- **Kind:** defect (R2), general. +- **Home:** engine. +- **Decision.** The three callers go through `decide_for_miss`, which already + applies `--offline`, `auto_refresh` and the debounce. The issue's + alternative, documenting that only `--offline` is complete, would keep the + second derivation. A first sync is not exempted: the decision module already + lets `auto_refresh` suppress an absent index, and a failure that names + `mcpp index update` is the documented behaviour. +- **Criterion.** The issue's, and the custom-index case. + +### 7.6 A6: the CN mirror exists, and its artifact does not match the pointer + +**Measured (group d).** `publish_mcpp_index.sh` already publishes to +`xlings-res/mcpp-index` on GitHub and GitCode, and the GitCode pointer names +the same version as GitHub's. The asset behind it differs: + +``` +A6 pointer version=7649883 sha=78535f362b9e... +A6 github sha=78535f362b9e... bytes=553573 first-entry=... 2026-09-16 04:41 ./ +A6 gitcode sha=d89135a348c9... bytes=553557 first-entry=... 2026-09-16 03:00 ./ +``` + +The trees are identical; the entries carry different modification times. The +script ran twice for one version, GitHub's upload replaced its asset with +`--clobber`, GitCode cannot replace an asset, and both pointers were +rewritten with the second digest. The tar invocation fixes names and owners +but not times (`publish_mcpp_index.sh:35`). A CN-routed client today would +reject GitCode's bytes and fall through to GitHub, the connection that hung. + +- **Kind:** ecosystem-data defect first; then a small engine default. +- **Home:** mcpp-index (publish script), then engine (`config.cppm`). +- **Decision, in order.** + 1. The artifact becomes byte-reproducible (`--mtime` from the source + commit, `gzip -n`), and a version whose GitCode asset exists with a + different digest is published under a new name instead of keeping bytes + that no pointer describes. + 2. The engine's default `artifact` becomes the region object + `{"GLOBAL": ..., "CN": ...}` that xlings parses since #377, in the + configuration template, the in-memory default and the existing + `.xlings.json` text migration (#269). +- **Criterion.** Two runs of the script on one commit produce one digest, and + group d's three digests agree; with `mirror = CN`, `mcpp index update` + fetches no `github.com` URL in xlings' verbose download log and succeeds. + +### 7.7 What belongs to the caller + +mcppls running offline by default, owning its children through a process +group, bounding its read after mcpp exits and caching the last good document +are the caller's decisions and are correct independently of mcpp. The engine +items make them unnecessary as defences, not wrong. + +## 8. Plugin items + +- **P1, `dist-apple`.** `defaulted_plist_keys()` holds `UIDeviceFamily`, + `LSRequiresIPhoneOS` and `NSHighResolutionCapable` + (`mcpp-plugins/dist/apple.cppm:631-634`), and a project entry can only + replace a default (`:700`, `:568-586`); a property list has no null to + express removal. `options::omit_keys` lists defaulted keys to leave out; + any other key is refused by name, as derived keys already are, and naming a + key in both `info_plist` and `omit_keys` is refused. +- **P2, `dist-web`.** The page name is fixed in the `mcpp.dist.web.index` + action (`dist/web.cppm:363-368`) and in `tests/web-consumer/check-web-plan.sh:42`. + `options::page` names it, default `index.html`, a bare `*.html` name with no + separator; the plan test gains a named-page leg. +- **`dist-apk`'s strip** (0.11.1) is retired once E5 is released and the + plugin reads `mcpp::pack_strip()`. +- **`rules-swift`** is §5.2. + +## 9. Decisions for the reviewer, and what is still owed + +### 9.1 Decisions + +| # | decision | alternative not taken | why | +|---|---|---|---| +| D1 | F3a: on ELF, programs over a plan-built C++ shared library take the shared-library contract | refuse the default shape | a mechanism that delivers one runtime exists; the process already requires the shared runtime | +| D2 | F1: a static package in two shared-image closures is refused with the `linkage = "shared"` remedy | link a copy into each image | each image would keep its own state on Mach-O and PE, and ELF would hide it (F1f) | +| D3 | E6: a package with no library target contributes nothing to a consumer's target graph | key builds by (package, features, toolchain) | the edge itself is wrong (E6c); feature-split nodes contradict one identity per graph | +| D4 | E1: a root-only graph file with verbatim metadata | a function family; dependency programs emitting values | one document serves both needs; dependency programs run dependents first | +| D5 | E5: staged toolchain runtime copies are stripped by default | leave them as the payload ships them | the reason for leaving store files alone does not apply to a copy; parity with Gradle | +| D6 | E9: `--message-format json` on `pack` prints one envelope | an NDJSON stream as on `test` | pack has one result; the flag names machine output, the kind names its shape | +| D7 | A3: a total deadline for the refresh and an inactivity deadline for installs, defaults measured | one total deadline for both | a total bound kills slow but progressing toolchain downloads | +| D8 | E4.2: the documentation changes | the parser accepts a source-less table | older clients could not load a package that used it | +| D9 | E7: members selected by identity | a `subdir` key | an older client ignoring `subdir` builds the wrong package silently | +| D10 | E10 and F2: step 1 corrects the record now; any change of default waits for its measurement and its own record | change the defaults here | working builds depend on both defaults | + +### 9.2 Measurements owed, by platform + +One temporary pull request carries a measurement workflow only, as #635 did. + +| platform | fixture | reading | decides | +|---|---|---|---| +| macos-15 | #641 M3 app/fw without `llvm.libcxx`, three legs | exception and `errc` identity across the dylib boundary; `otool -L` | F2, D10 | +| macos-15 | #646 M3b | the dylib link with and without F1 placement | F1 criterion | +| macos-15 | an Android `kind = "app"` | `file` on the linked object | E3 criterion | +| macos-15 | a `@_cdecl` Swift function, C callback | the autolink entries under `ld64.lld`; `aarch64-ios-sim` | E2 | +| windows-2022 | the llvm row default and `-fms-runtime-lib=dll` spelled through flags | `llvm-objdump -p` imports; lld warnings | E10 step 2 | +| windows-2022 | the F2 fixture across an llvm-row DLL | exception identity, CRT state | F2 PE half | + +### 9.3 Values owed + +The A3 defaults: a cold `xlings update` of the default indexes from a +GitHub-hosted runner and from a CN host, on both mirrors, with the 95th +percentile of ten runs; the refresh deadline is a small multiple of it. The +install inactivity deadline is a multiple of the heartbeat interval. + +## 10. Order of work + +The engine items group into three batches by what they risk. Within a batch +the items are independent unless stated. + +**Batch 1: defects whose fix does not change what a working build does when +it runs** (one release; Linux-testable except the E3 criterion). E5 changes +the bytes of packed libraries and nothing they do. + +- R3: A2, A3. +- R2: T, A5, E4.1 with X8, E4.3, E8 with X7, X5. +- R4: A1, A4, F3b, E10 step 1, E5 (including the status line), E9. +- E3 (the pure function and its unit test; the macOS criterion rides the + temporary pull request). +- X1, X3, X6; E4.2's refusal of a differing restatement; E4.2's + documentation. + +**Batch 2: decisions that change what some builds produce** (one release, +each with its own e2e and a CHANGELOG entry naming the change). + +- F3a (D1), then F1 (D2), which share the image-closure computation. +- E6 (D3) with X2, after the blast-radius scan of §6.4. +- E7 (D9). +- E1 (D4). + +**Batch 3: measured first, in their own records.** F2 and E10 step 2 (D10). + +**Outside the engine.** + +- mcpp-index: A6 step 1, before the engine's A6 step 2 (which joins batch 1 + once the digests agree). +- openxlings/xlings: `compat.ftxui`, now; it removes T's trigger for that + project before any release. +- mcpp-plugins: P1, P2 now; `rules-swift` after its measurement; `dist-apk`'s + strip retired after E5 is released. + +The release chain follows the established order: mcpp release, mirror, index, +then the plugin pins that need the new engine surface (`pack_strip`, +`graph_file`). + +## 11. Readings + +Each block is quoted from the group's `READING` lines, with the home +directory written as `~` and scratch paths shortened. + +**Group a (#646, E10).** + +``` +F1a M3b exit=0 +F1a libfw.so inputs: obj/mcpplibs_fw/src/fw.o +F1a libfw.so undefined x_answer=1 program defines x_answer=1 program exports x_answer=1 +F1a run exit=0 +F1c libfw.so relinked with -z defs: exit=1 undefined reference to `x_answer' +F1d foreign host dlopen(libfw.so, RTLD_NOW): exit=1 output: dlopen failed: .../bin/libfw.so: undefined symbol: x_answer +F1b root also depends on x: exit=0 +F1b x.o linked into: program=1 libfw.so=0 +F1b run exit=0 counter=2 +F1f libfw.so relinked with x.o: exit=0 libfw defines x_counter=1 run: counter=2 (1 shared counter => 2; split => 1) +F1e root-owned shared image over x: exit=0 libfw.so defines x_answer=1 undefined=0 +F3 gcc run: lib-3 exit=0 +F3 gcc NEEDED program: [liblib.so] [libm.so.6] [libgcc_s.so.1] [libc.so.6] [ld-linux-x86-64.so.2] +F3 gcc NEEDED liblib.so: [libstdc++.so.6] [libm.so.6] [libgcc_s.so.1] [libc.so.6] +F3 gcc contracts in resolution.json: {"distributable": "self-contained", "intermediate": "self-contained", "shared-library": "toolchain-coupled", "test": "self-contained"} +F3 gcc bin/app status=conflict exported=4571 conflicts=900 defined-in-std.o=1 first=['_ZGIW3std', '_ZGTtNKSt11logic_error4whatEv', ...] +F3 gcc --strict after touch exit=1 mentions=1 +F3 llvm run: libc++abi: terminating due to uncaught exception of type std::bad_cast: std::bad_cast exit=134 +F3 llvm NEEDED liblib.so: [libc++.so.1] [libc++abi.so.1] [libunwind.so.1] [libm.so.6] [libc.so.6] +F3 llvm bin/app status=conflict exported=145 conflicts=46 defined-in-std.o=1 first=['_Unwind_DeleteException', '_Unwind_ForcedUnwind', ...] +F3 llvm-static-lib run: lib-3 exit=0 +F3 llvm-no-println run: libc++abi: terminating due to uncaught exception of type std::bad_cast: std::bad_cast exit=134 +F3 llvm-tc run: lib-3 exit=0 +F3 llvm-tc bin/app status=conflict exported=84 conflicts=2 defined-in-std.o=1 first=['_ZGIW3std', '_ZGIW3stdW6compat'] +F3 llvm-tc --strict after touch exit=1 mentions=1 +F3 gcc-tc run: lib-3 exit=0 +F3 gcc-tc bin/app status=conflict exported=656 conflicts=8 defined-in-std.o=1 first=['_ZGIW3std', '_ZNSt8__detail31__from_chars_alnum_to_val_tableILb0EE5valueE', ...] +F3 gcc-tc --strict after touch exit=1 mentions=1 +F3 std.o global definitions=1 strong=1 sample=['_ZGIW3std'] (gcc and llvm) +E10 compile, no flag: dependent-lib= defines= +E10 link, no flag: -defaultlib:libcmt -defaultlib:oldnames +E10 -fms-runtime-lib=dll: compile --dependent-lib=msvcrt --dependent-lib=oldnames link -defaultlib:libcmt -defaultlib:oldnames +E10 -fms-runtime-lib=static: compile --dependent-lib=libcmt --dependent-lib=oldnames link -defaultlib:libcmt -defaultlib:oldnames +``` + +The lead's independent run of the F3 llvm default shape: + +``` +F3A build exit=0 +libc++abi: terminating due to uncaught exception of type std::bad_cast: std::bad_cast +F3A run exit=134 +F3A mcpp-run exit=134 +warning: app: 46 symbols in this image are also provided by a library it loads. +``` + +**Group b (E1, E5, E9).** + +``` +E1-strict-metadata-table: exit=0 warnings=0 +E1-root-env: SPIKE deps: MCPP_DEP_A_DIR MCPP_DEP_SPIKE_A_DIR MCPP_DEP_A_LINKAGE MCPP_DEP_SPIKE_A_LINKAGE +E1-program-order: SPIKE order a SPIKE order b +E1-graph-record: [('mcpplibs.app@0.1.0', ['package','requested_by','root']), ('spike.a@0.1.0', ['link','package','requested_by','root']), ('spike.b@0.2.0', [...])] +E1-metadata-edit: build.mcpp up to date (cached)x3 replayed=yes +E1-unknown-package-key-strict: exit=0 mentions=0 +E5-desktop-status: Packing hostapp v0.1.0 (vendored, stripped) +E5-desktop: bin/hostapp bytes=14280 symtab=0 debug=0 +E5-desktop: lib/libdep.so bytes=16592 symtab=1 debug=7 +E5-android-status: Packing app v0.1.0 (vendored, stripped) +E5-android: lib/libapp.so bytes=5904 symtab=1 debug=1 +E5-android: lib/libc++_shared.so bytes=9091400 symtab=1 debug=6 +E5-android: lib/libdep.so bytes=5496 symtab=1 debug=1 +E9-pack --release: exit=2 [error: unknown option: --release] +E9-pack --dev: exit=2 [error: unknown option: --dev] +E9-pack --message-format json: exit=2 [error: unknown option: --message-format] +E9-human-lines-on-stdout: bytes=273 +E9-run-precedence(--profile dev --release): Finished release +E9-build-precedence(--profile dev --release): Finished dev +E2-host: no swiftc on this host; E2 needs macos-15 +``` + +**Group c (E4, E6, E7, E8).** + +``` +E4.1-build: 2 forward warnings; tool kt built: 1; rc=0 +E4.1-strict: error: feature 'kotlin' of 'fw' forwards to dependency 'spike.rules' (as 'spike.rules/kotlin') which is not declared in ; rc=2 +E4.1-control-without-feature: tool kt built: 0; rc=0 +E4.1b-forward-to-other-row-dep: error: feature 'kotlin' of 'fw' forwards to dependency 'spike.win' (as 'spike.win/x') which is not declared in [dependencies] or [feature-de; rc=2 +E4.2-sourceless: error: .../e42/app/mcpp.toml: error: [f; rc=2 +E4.2-restated: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/installer@0.1.0+path.c728166a178c2a19/38e9d63bb5f6464d/bin/installer] qualified=[]; rc=0 +E4.2-restated-without-feature: SPIKE short=[] qualified=[]; rc=0 +E4.2-restated-different-path: warnings=1 tool-from: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/installer@0.1.0+path.c728166a178c2a19/38e9d63bb5f6464d/bin/installer] qualified=[]; rc=0 +E6-issue-fixture: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/fw-installer@0.1.0+path.2c270cde13410c30/46f3d4215af669cd/bin/fw-installer] qualified=[] | error: dependency cycle through package 'fw' while computing its build-cache key; rc=2 +E6-tool-runs: fw-installer ran 42 +E6-without-feature: ; rc=0 +E6-cache-local: ; rc=0; app runs: exit=0 +E6-cache-local-link: obj/main.o obj/spike_fw/src/fw.o +E6b-tool-activates-declaring-feature: 4 host-tool lines | error: building host tool 'fw-installer:fw-installer' failed: (x4) ... tool provisioning nested more than 4 levels deep — this is almost certainly a cycle.; rc=2 +E6d-build-dependency-tool-edge: | error: building host tool 'fw-installer:fw-installer' failed: (x4) ... nested more than 4 levels deep ...; rc=2 +E7-both-keys: 1 adoption warning(s) | SPIKE short=[] | Compiling spike.fw v | ; rc=0 +E7-member-only: error: dependency 'spike.fw-installer' resolved to package 'fw' (mismatch with declared name 'fw-installer'); rc=2 +E7-member-by-path: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/fw-installer@0.1.0+path.2db8369d57c7c650/47cd46c89a4debd7/bin/fw-installe | ; rc=0 +EDUP-dependencies-and-build-dependencies: tool built: 0 | SPIKE short=[] | ; rc=0 +E8-dep-feature-no-table: SPIKE short=[] qualified=[] | tool built: 0; rc=0 +E8-dep-feature-no-table-macro: -DMCPP_FEATURE_SPIKE_FW_INSTALLER +E8-unknown-no-table: ; rc=0; macros: -DMCPP_FEATURE_NOTHING_HERE +E8-dep-feature-with-table: warning: --features requests 'spike.fw/installer' which [features] does not declare | tool built: 0; rc=0 +E8-dep-feature-with-table-strict: error: --features requests 'spike.fw/installer' which [features] does not declare; rc=2 +E8-manifest-forward-control: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/fw-installer@0.1.0+path.5020fbc16166960c/21a7db9275c93a63/bin/f | tool built: 1; rc=0 +E6c-tool-dependency-in-consumer: app link line: obj/main.o obj/spike_z/src/z.o obj/spike_fw/src/fw.o; app defines z symbol: 1; rc=0 +``` + +**Group d (#648).** + +``` +A2 reader stdin=pipe:[429071926] +A2 child fd 0 -> socket:[430147394] +A2 child fd 1 -> pipe:[430149989] +A2 child fd 2 -> pipe:[430149989] +A2 child fd 3 -> pipe:[429071926] +A2 child fd 4 -> .../a2/fds.txt +T exit=0 +T decision ftxui@6.1.9: offline mode +T resolver warning: dependency 'ftxui' resolved to 'compat.ftxui' through the deprecated bare-name search; namespace omission means `mcpplibs` only. Write the exact package: +T resolver warning: dependency 'gtest' resolved to 'compat.gtest' through the deprecated bare-name search; namespace omission means `mcpplibs` only. Write the exact package: +A6 pointer version=7649883 sha=78535f362b9e4803e2036a5fd64730751074240001c262f204e07460dda6bc42 +A6 github sha=78535f362b9e4803e2036a5fd64730751074240001c262f204e07460dda6bc42 bytes=553573 +A6 gitcode sha=d89135a348c99c20baeba211f4bbaf2892b210fce0114ebee954880835c32a60 bytes=553557 +A6 github first-entry=drwxr-xr-x 0/0 0 2026-09-16 04:41 ./ +A6 gitcode first-entry=drwxr-xr-x 0/0 0 2026-09-16 03:00 ./ +``` + +The T reading planned openxlings/xlings at `59068d6`, the local checkout; its +`[dependencies]` spell `ftxui = "6.1.9"` as the current head `3cd8061` does +(`mcpp.toml:92`). `gtest` resolves through the same rung and prints no +decision, because the refresh loop walks `[dependencies]` only +(`prepare.cppm:4455`) and `gtest` is a dev-dependency. The A6 +digests are of the day's artifacts and change with the next publish. + +## 12. What each project can do today + +- **HuxerUI.** E4.2: restate the source in `[feature-deps]`. E6: keep the + per-application installer package until D3 is released. E7: pin the tool by + path within a checkout, or by a published version. E8: keep the root + forward line. E1: keep hand-written library paths. E5: `dist-apk` 0.11.1 + strips. E3: build Android on Linux. E10: the specification's allowed + difference stands. +- **mcppls.** Run offline by default, own the process group and bound the + read after mcpp exits, as planned. Until A2 is released, a child that mcpp + started in its own group can hold the pipe after the caller's group is + killed, so the bounded read is the part that cannot be dropped. +- **openxlings/xlings.** Write `compat.ftxui = "6.1.9"`; the refresh every two + minutes stops at once. + +## 13. Self-review + +1. **Unmeasured premises that carry decisions.** F2's split on the payload + origin; E10's step 2; the Mach-O and PE halves of F1; E3's criterion; E2's + autolink. Each is in §9.2, and no decision that changes an artifact is + taken on them. +2. **T's reason is inferred.** The offline opt-out overwrites the reason the + decision computed. The conclusion that the decision wanted a refresh does + not depend on it (the suppressed reason is assigned only after + `shouldRefresh` is set); which of `DescriptorMiss` or `IndexAbsent` fired + does, and the bare-name warning of the same run makes the first the only + consistent one. The criterion of §7.0 measures it directly. +3. **D3's blast radius is not yet known.** A consumer that depended on a + program-only package's dependencies leaking into its link would break; + §6.4 names the scan that precedes implementation. +4. **D1's scope stops at plan-built libraries.** A prebuilt C++ shared library + from the store is the same class and is left undecided. +5. **A2's second measure was dismissed on musl's `posix_spawn`.** macOS offers + `POSIX_SPAWN_CLOEXEC_DEFAULT`; the rule chosen is the one that holds on + every host, and the platform-specific closing can be added where it + exists without changing the rule. +6. **The readings are from one host.** Group b's Android leg used an NDK + already installed on this host; groups a to d ran against a home shared by + other processes, and no reading depends on an index refresh. +7. **Two forks' conclusions were checked against the code by the lead** before + entering this record: E3's branch assembly, E5's dispatch, E1's program + order, E4.3's publication site, E8's documentation and the X1 early return; + and F3a was reproduced independently. diff --git a/.agents/docs/2026-09-16-646-649-implementation-plan.md b/.agents/docs/2026-09-16-646-649-implementation-plan.md new file mode 100644 index 00000000..64844061 --- /dev/null +++ b/.agents/docs/2026-09-16-646-649-implementation-plan.md @@ -0,0 +1,410 @@ +--- +subject: triage +status: active +--- + +# #646 to #649 implemented: the plan, its review from eight angles, and the ledger + +**Status:** active. This record turns the decisions of +`2026-09-16-646-649-four-issues-by-home.md` ("the triage record") into work: +one pull request per repository, the order in which they merge and release, +the criterion each task is held to, and a ledger whose rows close only with a +reading. The triage record's decisions D1 to D10 are adopted; §1 states the +refinements this review makes before code is written, and a refinement found +while implementing is appended to §1.9. + +Base commits: mcpp `2fc7b5b0` (2026.9.15.2), mcpp-plugins `d6bee6a` (0.11.1), +mcpplibs/mcpp-index, openxlings/xlings `3cd8061`, openxlings/xim-pkgindex at +their heads on 2026-09-16. The engine release is 2026.9.16.1; the plugins +release is 0.12.0. + +## 0. The ledger + +Status is `todo`, `doing`, `branch` (implemented on the pull request's branch, +with its reading), `done` (merged or published, with its reading) or `dropped` +(with the reason). Owner `lead` is the integrating session; `W1` to `W3` are +the parallel work trees of §8. Triage sections are cited as `T§`. + +### 0.1 Engine: mcpp-community/mcpp, one pull request (`feat/646-649`) + +| id | task (triage §) | owner | depends on | status | +|---|---|---|---|---| +| L1 | the refresh decision walks the resolver's deprecated bare-name rung before calling a miss (T§7.0) | lead | - | todo | +| L2 | the saved standard output is close-on-exec, and not inheritable on Windows (T§7.2) | lead | - | todo | +| L3 | xlings runs under the owned launcher; a total deadline for the index refresh, an inactivity deadline for installs (T§7.3) | lead | L2 | todo | +| L4 | a per-run record of observed effects; `network` when a network child is launched (T§7.4) | lead | L3 | todo | +| L5 | the three refreshes that bypass `mcpp.pm.index_refresh` go through `decide_for_miss` (T§7.5) | lead | L1 | todo | +| L6 | the refusal code `offline-download-required` at every offline refusal site; `MCPP_OFFLINE_DOWNLOAD_REQUIRED` in the envelope (T§7.1) | lead | - | todo | +| L7 | the default `artifact` of the mcpplibs index is the region object; the existing `.xlings.json` migration carries it (T§7.6 step 2) | lead | X1 | todo | +| R1 | ELF: programs and tests over a plan-built C++ shared library take the shared-library runtime contract; an explicit `self-contained` program there is refused (T§4.1, D1) | W1 | - | todo | +| R2 | symbol provision: `STB_GNU_UNIQUE` is vague linkage; a duplicate whose definitions come from one plan object is not reported (T§4.2) | W1 | - | todo | +| R3 | a static package reachable from one shared image only is linked into that image; one reachable from two images is refused where the link or load cannot succeed and diagnosed elsewhere (T§4.3, D2, §1.5) | W1 | - | todo | +| R4 | clang on the MSVC ABI records the runtime it delivers; an undeliverable `cxx_runtime` is diagnosed; docs/20 states the row's model (T§4.5 step 1) | W1 | - | todo | +| R5 | measurement legs: exception identity across a Mach-O dylib under the payload default (macos-15), and across an llvm-row DLL (windows-2022), each printing its reading (T§4.4, §9.2) | W1 | - | todo | +| G1 | the forward validator accepts a key declared in any dependency table on any row (T§6.1, X8) | W2 | - | todo | +| G2 | a `[feature-deps]` restatement whose source differs is refused; docs/05 says to restate the source (T§6.2, D8) | W2 | - | todo | +| G3 | one helper names a provider for a consumer; `dep_bin` gains the qualified spelling (T§6.3) | W2 | - | todo | +| G4 | a package with no library target contributes nothing to a consumer's target graph; package cycles are checked at resolution; a repeated tool is refused at its first repetition (T§6.4, D3, X2, X3) | W2 | - | todo | +| G5 | a git dependency selects a repository member by identity; a second declaration merges additively; the git banner names the commit (T§6.5, D9, X1, X6) | W2 | G4 | todo | +| G6 | `--features dep/feature` is a root forward; `why deps --features` (T§6.6, X7) | W2 | G1 | todo | +| S1 | `mcpp::graph_file()` for the root build program, with `[package.metadata]` and a digest in the re-run key (T§5.1, D4) | W3 | - | todo | +| S2 | the link branch is chosen by host and target object format, as a pure function with a host-by-row unit test (T§5.3) | W3 | - | todo | +| S3 | pack strips the program on every stripping row, every graph-built shared library and staged runtime copies; the status line reports what was done; `mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()` (T§5.4, D5) | W3 | - | todo | +| S4 | `mcpp pack --message-format json` prints one `mcpp.pack` envelope; `pack --release/--dev`; `run` takes `build`'s profile precedence (T§5.5, D6, X5) | W3 | - | todo | +| S5 | `[package]` warns about an unknown key, `metadata` included in its known set (X4) | W3 | S1 | todo | +| C1 | CI: new llvm-dependent e2e scripts run on the hermetic job with their PASS lines asserted; the macOS and Windows measurement legs print their readings to the job summary | lead | R1-R5, S2 | todo | +| C2 | user documentation with its Chinese mirror, docs/50 codes and kinds, CHANGELOG, version 2026.9.16.1 | lead, W1-W3 | all | todo | +| C3 | the two records closed with their readings | lead | all | todo | + +### 0.2 Plugins: mcpp-community/mcpp-plugins, one pull request (`feat/646-649`, 0.12.0) + +| id | task | owner | depends on | status | +|---|---|---|---|---| +| P1 | `dist-apple` `options::omit_keys` over the defaulted keys (T§8) | lead | - | todo | +| P2 | `dist-web` `options::page`, default `index.html` (T§8) | lead | - | todo | +| P3 | `dist-apk` follows `mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()` when the engine provides them, and keeps its own strip otherwise | lead | S3 released | todo | +| P4 | `rules-swift`: one package's Swift sources compile through an `object` action, a generated header through a `source` action, and the runtime link flags through `link_flag`; a macos-15 consumer runs (T§5.2) | lead | - | todo | +| P5 | `dist-apk` and `dist-apple` collect library contributions from `[package.metadata.dist-apk]` and `[package.metadata.dist-apple]` through `mcpp::graph_file()` when present | lead | S1 released | todo | + +### 0.3 Index, payloads and projects + +| id | task | owner | depends on | status | +|---|---|---|---|---| +| X1 | mcpplibs/mcpp-index: the artifact is byte-reproducible, and a version whose GitCode asset differs is republished under a new name (T§7.6 step 1) | lead | - | todo | +| X2 | openxlings/xim-pkgindex: the release bot's `mcpp` bump merged | lead | release | todo | +| X3 | mcpplibs/mcpp-index: the plugins descriptor 0.12.0; the index CI's `MCPP_VERSION` moves to 2026.9.16.1 | lead | X2, P1-P5 | todo | +| X4 | openxlings/xlings: `compat.ftxui` and `compat.gtest` spelled with their namespace (T§7.0) | lead | - | todo | + +### 0.4 Verification + +| id | task | owner | depends on | status | +|---|---|---|---|---| +| V1 | a fresh SubOS sandbox with CN mirrors for xlings and mcpp runs §7's scenarios against the published engine, plugins and index | lead | X2, X3 | todo | +| V2 | replies on #646, #647, #648 and #649 naming the releases and what each project writes | lead | V1 | todo | + +## 1. The review + +Each decision was held to eight questions. The answers either confirm the +triage record's decision or refine it; a refinement is stated where the +decision is. + +### 1.1 Architecture + +- Every engine change lands in the module that already owns the question: + runtime contracts in `distribution.cppm`, placement in `make_plan`, the + refresh policy in `mcpp.pm.index_refresh`, child ownership in + `mcpp.platform.process`, forwards in the feature pass of `prepare`, + build-program facts in `hostprogram.cppm`, pack policy in `src/pack`. No + task adds a module. +- The two new facts a build program reads (`graph_file`, `pack_strip`) extend + the accessor family that exists (`dep_dir`, `pack_stage_dir`), reach the + program through the contract environment, and therefore take part in the + re-run key by construction. +- The refusal codes added here (`offline-download-required`, + `program-cxx-runtime-split`, `static-package-in-two-images`, + `package-cycle`) join the existing sink; `mcpp why toolchain` and the + envelope read them without a second classification. + +### 1.2 Stability + +- L3 changes how every xlings child is started. The streaming sink, the exit + code and the NDJSON parsing stay as they are; only the launcher changes. The + deadline paths are exercised by a stub xlings named through + `[xlings] binary`, on Linux in CI and on Windows through the e2e shard. +- R3 moves objects between images. Its unit test enumerates the closures of + six graph shapes (a chain, a diamond through one image, a diamond through two + images, a root-owned image, an image over an image, a header-only package) + before any link line changes. +- G4 removes edges from the consumer graph. Before it is implemented, the + manifests of the e2e corpus, the examples, mcpp-index and mcpp-plugins are + scanned for a dependency on a program-only package that is not requested for + `tools` (§1.9 records the result). + +### 1.3 Simplicity + +- A2 is one call. A5 deletes a derivation. L1 adds one condition to a lookup + that exists. G3 deletes a duplicated derivation. E3's fix is a pure function + replacing an `if constexpr` choice. +- The deadline design uses the launcher's existing total deadline for the + refresh; only the install path needs an inactivity bound, which is one more + parameter of the same launcher, not a second launcher. +- D6 adds a kind to the envelope table rather than a new output mechanism. + +### 1.4 User experience + +- The status line of `pack` states what was done (S3); the envelope states + what happened (L4, L6); a refusal names its remedy (R1, R3, G2, G4); a + deadline names itself and the configuration key that sets it (L3). +- **Refinement U1 (L3).** A timed-out refresh prints one warning naming the + deadline and `[index] refresh_timeout`, and the build continues from local + data; it is never silent and never fatal on its own. +- **Refinement U2 (G6).** `--features dep/feature` is accepted by `build`, + `run`, `test`, `pack` and `emit build-database` alike, because they share + one overrides structure; the help text of each says so. + +### 1.5 Compatibility and seamless upgrade + +Each change that alters what a working build produces is listed with why an +upgrade is not a cliff. + +| task | what changes for a working build | why it is not a cliff | +|---|---|---| +| R1 | a gcc program over a C++ shared library it builds gains `NEEDED libstdc++.so.6` | the process already loads that file through the library; `pack` already bundles it | +| R3 | on ELF, a static package reachable only through a dependency's shared image moves from the program into the image | the program never referred to it; its symbols resolved to the same single copy | +| R3 | a static package reachable from two images | **Refinement C1.** It is refused only where the build cannot work today (Mach-O and PE, whose links fail, and the Android `app` row, whose loader binds the image before the program). On other ELF rows the build proceeds as it does today, with a degraded diagnostic naming `linkage = "shared"`; `--strict` fails it. Refusing a build that runs today on Linux would be an upgrade cliff for no gain on that row. | +| R4 | the recorded contract of an llvm-row PE artifact changes from `host-coupled` to `self-contained` | the artifact is byte-identical; the record now matches it | +| G4 | a program-only package's dependencies leave the consumer's link | §1.2's scan decides whether any consumer relied on it | +| S3 | packed shared libraries lose their symbol tables | `--no-strip` restores them; debug sections go to `--debug-symbols` as the program's do | +| S4 | `mcpp run --profile dev --release` builds `dev` | the same line already builds `dev` under `mcpp build` | +| L5 | `auto_refresh = false` blocks the pre-install refresh | the documentation already states that behaviour | +| L7 | a CN machine fetches the mcpplibs index from GitCode | only after X1 makes the GitCode artifact match its pointer; a mismatch falls back to GitHub, which is today's route | + +No manifest key is removed and no manifest that loads today stops loading. +The new manifest surface is `[package.metadata.*]` (already ignored by every +client) and one configuration key, `[index] refresh_timeout`, whose absence +means the default. + +### 1.6 Cross-platform + +| task | Linux | macOS | Windows | +|---|---|---|---| +| L2 | `F_DUPFD_CLOEXEC` | `F_DUPFD_CLOEXEC` | `HANDLE_FLAG_INHERIT` cleared | +| L3 | process group, signal guard | process group, signal guard | job object with kill-on-close | +| R1 | the rule | not applicable (Mach-O is R5's measurement) | not applicable | +| R3 | ELF: diagnosed; Android app row: refused | refused | refused | +| R4 | unit test of the record | - | e2e on the llvm row | +| R5 | - | e2e reading | e2e reading | +| S2 | unit test over host by row | e2e: Android row links | unit test | +| S3 | ELF, Android | Mach-O unchanged (not stripped by design) | PE, MinGW DWARF | + +### 1.7 Consistency + +- Profile precedence, feature tokens, dependency names and runtime contracts + each get one derivation (R2 of the triage record). The review confirmed that + no second copy of these remains after the tasks: `run`, `build`, `test` and + `pack` read one profile resolver; manifest forwards and CLI forwards share + the validator; `fillDepDirs` and the tool publication share the name helper. +- **Refinement K1 (S4).** docs/50 §3 gains the rule for a command whose + `--format` names its product, and `test`'s existing `--message-format json` + is cited as the precedent, so the two spellings are one convention. + +### 1.8 Test coverage + +- Every task has a criterion that fails on 2026.9.15.2: an e2e script whose + first line of output names the release it was measured against, or a unit + test of a pure function. +- e2e scripts that need llvm run on the hermetic job and their PASS lines are + asserted (the #641 lesson: `# requires: llvm` alone never runs on a shard). +- Measurement legs (R5) print `READING` lines into the job summary and do not + fail on the value they read; they fail only if the probe could not run. +- The sandbox verification (§7) runs its script once against 2026.9.15.2 first: + the change-detecting scenarios must fail there and pass on 2026.9.16.1, and + the regression guards must pass on both. + +### 1.9 Refinements found while implementing + +(appended during implementation) + +## 2. Engine tasks + +### 2.1 Lead: #648 (L1 to L7) + +- **L1.** `decide_for_dependency` calls `legacy_bare_candidates` under + `spec.isVersion() && spec.namespaceOmitted` after a conclusive miss, and + treats a hit as `None`. Unit: `IndexRefresh.LegacyBareRung`. e2e 730: a + fixture whose bare dependency resolves through the rung, planned offline with + `-v`, prints no suppressed decision. +- **L2.** `StdoutToStderr` saves with `F_DUPFD_CLOEXEC` (Windows: the saved + CRT descriptor's handle is made non-inheritable). e2e 731: reading A2's + build program lists no descriptor naming the reader's pipe. +- **L3.** `call`, `update_index_unguarded`, `install_direct`, the bootstrap + and `config` invocations run through the owned launcher with argv, the + environment the `ScopedInvocationEnv` states, and the working directory as + a parameter. The refresh is bounded by `[index] refresh_timeout` (default + 120 seconds); an install by an inactivity bound of 300 seconds without an + NDJSON line. A timeout is not retried. e2e 732: a stub xlings that sleeps; + `emit build-database` with local data returns within the deadline plus a + margin and succeeds; after SIGTERM to mcpp no stub process remains. +- **L4.** `mcpp::wire` keeps a per-run set of observed effects; the launcher + of a network child adds `network`. Every enveloped command merges it into + its `effects`. e2e 733 (with 732's stub): a refresh run's envelope lists + `network`; an offline run's does not. +- **L5.** The fetcher's pre-install refresh, its refresh before a retry, and + the first custom-index sync call `decide_for_miss` and apply its decision. + e2e 734: `auto_refresh = false` with a missing package runs no `update` in + the stub's argv log and names `mcpp index update`. +- **L6.** `refusal::Code::OfflineDownloadRequired` at the five offline sites and + the absent-index case; `emit build-database` maps it to + `MCPP_OFFLINE_DOWNLOAD_REQUIRED`. e2e 735. +- **L7.** The configuration template, the in-memory default and the + `.xlings.json` migration write `artifact` as + `{"GLOBAL": "https://github.com/xlings-res/mcpp-index", "CN": "https://gitcode.com/xlings-res/mcpp-index"}`. + Unit: the migration is idempotent and preserves unrelated state. + +### 2.2 W1: images and runtimes (R1 to R5) + +- **R1.** In the contract resolution, when the root image of a plan links a + `SharedLibrary` unit of the plan whose objects are C++, the `Distributable` + and `Test` roles on an ELF target take the `SharedLibrary` role's contract. + An explicit `cxx_runtime = "self-contained"` (or `{ program = ... }`) in + that graph is refused with `program-cxx-runtime-split`. e2e 700 (llvm and + gcc legs): the default shape runs; the explicit statement is refused. +- **R2.** `STB_GNU_UNIQUE` joins `STB_WEAK` as vague linkage; the check + receives, per image, the plan objects linked into it and suppresses a + duplicate defined by one object in both. Unit tests on both rules. e2e 701: + a uniform-contract program over a C++ shared library reports no + `build/symbol-provision` finding and `--strict` exits 0; a program with a + static zlib over a shared libz still reports. +- **R3.** `make_plan` computes each shared image's static closure over + `directPackageDeps`; a static package in exactly one closure is linked into + that image and removed from the root's objects; one in several closures is + refused (`static-package-in-two-images`) on Mach-O, PE and the Android app + row, and diagnosed (degraded) on other ELF rows. Unit: the six shapes of + §1.2. e2e 702: M3b's library links under `-Wl,-z,defs` and loads through a + foreign `dlopen`; the two-closure shape builds with the diagnostic on Linux + and without it after `linkage = "shared"`. +- **R4.** The PE contract table records `self-contained` for clang on the MSVC + ABI unless a runtime flag is emitted; an explicit `host-coupled` or + `toolchain-coupled` on that row is diagnosed; the `msvc_crt_flag` comment is + corrected; docs/20 states the row's model. Unit test of the table. e2e 703 + (Windows shard): `resolution.json` records `self-contained`. +- **R5.** e2e 704 (macOS shard): the #641 M3 app/fw pair without + `llvm.libcxx`, three legs, prints whether `std::runtime_error`, the + library's own exception and an `std::errc` comparison cross the dylib + boundary. e2e 705 (Windows shard): the same fixture over an llvm-row DLL. + Both print `READING` lines; the decision of T§4.4 is taken from them in + §1.9. + +### 2.3 W2: features, tools and git sources (G1 to G6) + +- **G1.** The forward validator reads the dependency tables of every row and + feature (`conditionalConfigs`) and the build-dependency table. e2e 710. +- **G2.** `mergeActiveFeatureDeps` refuses a restatement whose path, git + source or version differs from the declaration in effect on the row. docs/05 + and its mirror say to restate the source. e2e 711. +- **G3.** `published_names(provider, consumer)` is used by `fillDepDirs` and + the tool publication. e2e 187 gains a `namespace =` plus `name =` leg. +- **G4.** Step 0 is the scan of §1.2. A package whose targets are all + programs is not walked into a consumer's target graph and compiles nothing + there; its tools come from the sub-build. Package-edge cycles are detected + in the resolution pass with a `package-cycle` refusal naming the edges; the + tool chain refuses a repeated (source, tool) at its first repetition. e2e 712 + (the issue's fixture with the default cache; E6c's link line; E6b refused + once). +- **G5.** A git dependency whose identity is not the root manifest's is + looked up among the root's `[workspace] members` at the same commit; a + member's in-clone path edges resolve as the same git source; a second + declaration of one dependency by one consumer merges `tools`, `features`, + `host-module` and `reexport` into the edge; the banner prints the short + commit. e2e 713. +- **G6.** A `/` token in `--features` is split and applied as a root forward; + one naming no dependency warns (error under `--strict`) whether or not the + root declares `[features]`, and never becomes a macro. `mcpp why deps` + accepts `--features`. e2e 714. + +### 2.4 W3: build programs, the link line and pack (S1 to S5) + +- **S1.** Before the root build program, `prepare` writes + `/.mcpp/graph.json` (the `resolution.json` `graph` objects plus + `manifest_dir`, `features`, `targets` kinds, `link`, `metadata`) in + dependency order, and publishes its path as `MCPP_GRAPH_FILE` + (`mcpp::graph_file()`); its digest joins the contract environment. The + manifest parser keeps `[package.metadata]` as a raw table. e2e 720. +- **S2.** `link_shape(host, target_format, dialect)` selects the branch; + `tests/unit/test_build_flags.cpp` gains host-by-row cases for macOS and + Windows hosts. e2e 721 (macOS shard): `mcpp build --target + x86_64-linux-android` links an ELF object. +- **S3.** `pack::Plan` carries the graph's `SharedLibrary` outputs; + `run_shared_program` strips per leg; `bundle_libs`, `stage_closure` and the + multi-leg path strip graph-built libraries and toolchain runtime copies with + `--strip-unneeded`; the status line prints what was done; + `MCPP_PACK_STRIP` and `MCPP_PACK_DEBUG_SYMBOLS_DIR`. e2e 722 (Linux desktop + and, where the NDK is present, Android). +- **S4.** `pack` accepts `--release`, `--dev` and `--message-format json`; + the `mcpp.pack` kind is in `kKinds` and `--protocol-version`; `run` resolves + the profile with `build`'s function. e2e 723. +- **S5.** `[package]` reports an unknown key the way `[build]` does, with + `metadata` known. Unit test in the manifest suite. + +## 3. Plugin tasks + +One pull request on mcpp-plugins, version 0.12.0, after the engine release +(P3 and P5 need the released accessors; P1, P2 and P4 do not, and are written +first). Its CI runs the consumers against the pinned engine and, before the +release, against `mcpp_ref=feat/646-649`. + +## 4. Index and project tasks + +- **X1** is independent of the engine and merges first; the next index publish + republishes the artifact, and the group d probe of the triage record then + reads three equal digests. +- **X2** is the release bot's pull request. +- **X3** follows X2 and the plugins tag. +- **X4** is a two-line manifest change in openxlings/xlings. + +## 5. Order + +1. X1 and X4 open at once (independent of everything). +2. W1, W2, W3 and the lead's L1-L6 in parallel; the lead merges each tree into + `feat/646-649` as it completes, builds, and runs the unit tests and the new + e2e scripts. +3. L7, C1 and C2 on the integrated branch; one pull request; CI green, + including the measurement legs; §1.9 records R5's readings and the decision + they imply. +4. Self-review of the diff and of the ecosystem; merge. +5. Release 2026.9.16.1; GitCode assets uploaded locally as each archive + appears; X2. +6. P1-P5 on the plugins branch; its CI against the released engine; tag 0.12.0, + mirrored to GitCode; X3. +7. V1; the bootstrap pin; V2; C3. + +## 6. Release chain + +`gh workflow run release.yml --ref main` from a clean `main` whose HEAD is the +merge commit, with nothing pushed to `main` until the tag exists. Each archive +and its `.sha256` go to `xlings-res/mcpp` on GitCode with the local tool as soon +as the build job puts it on the GitHub release; an asset counts as done when a +GET returns 200 with the upstream size and the bytes compare equal. +`publish-ecosystem` then opens the index pull request, whose four hashes are +compared with the downloaded archives by script. The release is complete when +the published xim index artifact (not git main) names 2026.9.16.1 as `latest`. + +## 7. Sandbox verification (V1) + +A new SubOS `v646`, entered with `xlings subos use v646 --sandbox --cmd`, +after `xlings config --mirror CN` and `mcpp self config --mirror CN` inside it. +The engine is addressed by its store path and exact version. The script runs +first against 2026.9.15.2 (change detectors fail, guards pass), then against +2026.9.16.1 (all pass). Scenarios, each with a `READING` line: + +1. F3a: an llvm program over a C++ shared library runs. +2. F1: a shared package over a static package links under `-z defs`. +3. E6: a feature-gated tool that depends on its declaring package builds and + runs; `dep_bin` answers under both spellings. +4. E8: `--features /` builds the dependency's tool. +5. E1: the root build program reads the graph file and a dependency's + metadata. +6. E5: `mcpp pack --format tar` ships a stripped graph-built `libdep.so`. +7. E9: `mcpp pack --release --message-format json` prints a record whose path + exists. +8. #648: offline `emit build-database` with a removed package yields + `MCPP_OFFLINE_DOWNLOAD_REQUIRED`; a build program lists no inherited pipe; + a bare `compat.*` dependency produces no refresh decision. +9. A6: `mcpp index update` with `mirror = CN` succeeds from GitCode. +10. plugins 0.12.0 from the index: `dist-web` with `page` names the page. + +## 8. Parallel work trees + +| tree | branch | owns | may touch minimally | +|---|---|---|---| +| lead `mcpp-646` | `feat/646-649` | `modules/platform/src/{terminal,process}.cppm`, `modules/platform/src/{unix,windows}/bounded_process.cppm`, `src/xlings/xlings.cppm`, `src/pm/index_refresh.cppm`, `src/pm/package_fetcher.cppm`, `src/config.cppm`, the refresh, custom-index and offline-refusal blocks of `prepare.cppm`, the `emit build-database` function of `cmd_build.cppm`, the effects half of `src/wire.cppm`, workflows, CHANGELOG, version files | `src/build/refusal.cppm` (one code) | +| W1 `mcpp-646-runtime` | `feat/646-649-runtime` | `src/build/distribution.cppm`, the link-unit assembly of `src/build/plan.cppm`, `src/build/symbol_provision.cppm`, `src/runtime/elf.cppm`, `src/build/runtime_validation.cppm`, `src/toolchain/dialect.cppm`, the CRT block of `flags.cppm`, docs/20, docs/22 | `src/build/refusal.cppm` (two codes), the contract call site in `prepare.cppm` | +| W2 `mcpp-646-graph` | `feat/646-649-graph` | the feature, forward, worklist, edge, git-source, tool-publication and cache-key-walk blocks of `prepare.cppm`, `src/build/execute.cppm` (banner), `src/build/tool_store.cppm`, the `why deps` option table in `src/cli.cppm`, docs/05 (features and git), docs/06, docs/07, the `dep_bin` section of docs/30 | `modules/manifest` (feature-deps check), `src/build/refusal.cppm` (one code) | +| W3 `mcpp-646-surface` | `feat/646-649-surface` | the root build program block and `resolution.json` writer of `prepare.cppm`, `src/build/hostprogram.cppm`, `src/build/build_program.cppm`, the link-branch block of `flags.cppm`, `src/pack/*`, the pack option table of `src/cli.cppm`, the `run` profile resolution of `cmd_build.cppm`, the kinds table of `src/wire.cppm`, `tests/unit/test_build_flags.cpp`, docs/10, the accessor sections of docs/30, the `mcpp.pack` section of docs/50 | `modules/manifest` (`[package]` keys) | + +e2e numbers: W1 700-709, W2 710-719, W3 720-729, lead 730-739. A tree removes +its `target/` after its branch is merged. + +## 9. Closure + +(written when the rows of §0 are closed) diff --git a/.agents/docs/2026-09-16-646-649-probes.sh b/.agents/docs/2026-09-16-646-649-probes.sh new file mode 100755 index 00000000..c00fe64f --- /dev/null +++ b/.agents/docs/2026-09-16-646-649-probes.sh @@ -0,0 +1,778 @@ +#!/usr/bin/env bash +# Probes for .agents/docs/2026-09-16-646-649-four-issues-by-home.md. +# +# Four independent groups. Each is written out as a standalone script and run +# with its own options and variables. Every result is a line starting +# with READING, and the record quotes those lines. +# +# a #646 F1, F3 and the driver half of #649 E10 (needs gcc@16.1.0, llvm@22.1.8) +# b #647 E1 and #649 E5, E9 (the Android leg needs xim:android-ndk installed) +# c #647 E4 and #649 E6, E7, E8 (host only) +# d #648: the inherited pipe (A2), the refresh trigger (T), the mirror digests (A6) +# +# Usage: 2026-09-16-646-649-probes.sh [a|b|c|d|all] [work dir] +# Engine: the released mcpp 2026.9.15.2 (override with M=/path/to/mcpp); home ~/.mcpp. +# Group d's T reading plans a checkout of openxlings/xlings (XL=). +set -u +WHICH=${1:-all} +WORK=${2:-$(mktemp -d)} +mkdir -p "$WORK" && WORK=$(cd "$WORK" && pwd) +export M=${M:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/2026.9.15.2/bin/mcpp} +export MCPP="$M" + +group_a() { mkdir -p "$WORK/a"; cat > "$WORK/a.sh" <<'__PROBES_646_649_GROUP_A__' +# Group a: probes for #646 F1/F3 (Linux x86_64) and the driver half of #649 E10. +# Engine: the released mcpp 2026.9.15.2 (xlings package; home ~/.mcpp). +# Every result is a line starting with READING. +# Usage: probes.sh [work dir] (default: ./work next to this script) +set -u +HERE=$(cd "$(dirname "$0")" && pwd) +WORK=${1:-$HERE/work} +mkdir -p "$WORK" && WORK=$(cd "$WORK" && pwd) +M=${MCPP:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/2026.9.15.2/bin/mcpp} +NINJA=$(find "$HOME/.mcpp/registry/data/xpkgs/xim-x-ninja" -name ninja -type f | head -1) +LLVM=$HOME/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8 +GCCBIN=$HOME/.mcpp/registry/data/xpkgs/xim-x-gcc/16.1.0/bin +unset XLINGS_ACTIVE_SUBOS +reading() { echo "READING $*"; } +# Replay the engine's own link of bin/libfw.so from the build directory, with +# extra inputs ($1) and extra flags ($2). Ninja deletes response files after a +# link, so the file is rewritten from the edge's inputs, as probe M5 did. +relink_fw() { + local cmd ins + cmd=$("$NINJA" -t commands bin/libfw.so | tail -1) + ins=$(grep '^build bin/libfw.so' build.ninja | sed 's/^build bin\/libfw.so : c[a-z_]* //; s/ || .*//; s/ | .*//') + case "$cmd" in + *libfw.so.rsp*) printf '%s %s\n' "$ins" "$1" > bin/libfw.so.rsp; eval "$cmd $2" ;; + *) eval "${cmd/ -o / $1 -o } $2" ;; + esac +} +"$M" --version | sed 's/^/READING engine: /' + +# ---------------------------------------------------------------- F1 -------- +# F1a: M3b verbatim (x static C, fw shared C over x, root C++ over fw). +f1_fixture() { # $1 dir, $2 extra root deps line (may be empty), $3 main body + mkdir -p "$1/app/src" "$1/fw/src" "$1/x/src" + printf '[package]\nname = "x"\nversion = "0.1.0"\n[build]\nsources = ["src/*.c"]\n[targets.x]\nkind = "lib"\n' > "$1/x/mcpp.toml" + printf 'int x_counter = 0;\nint x_answer(void) { return 41; }\nint x_bump(void) { return ++x_counter; }\n' > "$1/x/src/x.c" + printf '[package]\nname = "fw"\nversion = "0.1.0"\n[build]\nsources = ["src/*.c"]\n[targets.fw]\nkind = "shared"\n[dependencies.x]\npath = "../x"\n' > "$1/fw/mcpp.toml" + printf 'extern int x_answer(void);\nextern int x_bump(void);\nint fw_answer(void) { return x_answer() + 1; }\nint fw_bump(void) { return x_bump(); }\n' > "$1/fw/src/fw.c" + printf '[package]\nname = "app"\nversion = "0.1.0"\n[toolchain]\ndefault = "gcc@16.1.0"\n[dependencies.fw]\npath = "../fw"\n%s' "$2" > "$1/app/mcpp.toml" + printf '%s\n' "$3" > "$1/app/src/main.cpp" +} +f1_fixture "$WORK/f1a" "" 'extern "C" int fw_answer(void); +int main() { return fw_answer() == 42 ? 0 : 1; }' +cd "$WORK/f1a/app" && timeout 900 "$M" build > build.log 2>&1 +reading "F1a M3b exit=$?" +d=$(ls -d target/*/*/ | head -1) +so=$d/bin/libfw.so; bin=$d/bin/app +reading "F1a libfw.so inputs: $(grep '^build bin/libfw.so' "$d/build.ninja" | sed 's/^build bin\/libfw.so : c[a-z_]* //; s/ |.*//')" +reading "F1a libfw.so undefined x_answer=$(nm -D --undefined-only "$so" | grep -c ' x_answer$') program defines x_answer=$(nm "$bin" | grep -c ' T x_answer$') program exports x_answer=$(nm -D --defined-only "$bin" | grep -c ' x_answer$')" +"$bin"; reading "F1a run exit=$?" + +# F1c: the same library linked the way Mach-O and PE link (every reference +# resolved at link time): ELF's -z defs. +cd "$d" +relink_fw "" "-Wl,-z,defs" > f1c.log 2>&1 +reading "F1c libfw.so relinked with -z defs: exit=$? $(grep -m1 -o 'undefined reference to .x_answer.\|undefined symbol: x_answer' f1c.log)" +relink_fw "" "" > /dev/null 2>&1 # restore + +# F1d: a host that did not link x (a foreign program, or Android's +# System.loadLibrary("fw") before "app") loads the library with RTLD_NOW. +cat > "$WORK/f1a/host.c" <<'C' +#include +#include +int main(int argc, char** argv) { + void* h = dlopen(argv[1], RTLD_NOW | RTLD_LOCAL); + if (!h) { printf("dlopen failed: %s\n", dlerror()); return 1; } + printf("dlopen ok\n"); return 0; +} +C +"$GCCBIN/gcc" -o "$WORK/f1a/host" "$WORK/f1a/host.c" -ldl > /dev/null 2>&1 +out=$("$WORK/f1a/host" "$PWD/bin/libfw.so" 2>&1); rc=$? +reading "F1d foreign host dlopen(libfw.so, RTLD_NOW): exit=$rc output: $out" + +# F1b: x reachable from both the program and the shared image. +f1_fixture "$WORK/f1b" '[dependencies.x] +path = "../x" +' '#include +extern "C" int fw_answer(void); extern "C" int fw_bump(void); +extern "C" int x_bump(void); extern "C" int x_counter; +int main() { x_bump(); fw_bump(); std::printf("counter=%d\n", x_counter); return fw_answer() == 42 ? 0 : 1; }' +cd "$WORK/f1b/app" && timeout 900 "$M" build > build.log 2>&1 +reading "F1b root also depends on x: exit=$?" +d=$(ls -d target/*/*/ | head -1) +reading "F1b x.o linked into: program=$(grep '^build bin/app ' "$d/build.ninja" | grep -c 'x\.c\.o\|/x\.o') libfw.so=$(grep '^build bin/libfw.so' "$d/build.ninja" | grep -c 'x\.c\.o\|/x\.o')" +out=$("$d/bin/app"); reading "F1b run exit=$? $out" + +# F1f: what "the static package goes into the image" produces on ELF when the +# program links it as well: relink libfw.so with x's objects added. +cd "$d" +xo=$(grep -o '[^ ]*x\.c\.o\|[^ ]*/x\.o' build.ninja | head -1) +relink_fw "$xo" "" > f1f.log 2>&1 +reading "F1f libfw.so relinked with x.o: exit=$? libfw defines x_counter=$(nm -D --defined-only bin/libfw.so | grep -c ' x_counter$') run: $(./bin/app) (1 shared counter => 2; split => 1)" + +# F1e: the root itself is the shared image over x (root-owned unit). +mkdir -p "$WORK/f1e" && cp -r "$WORK/f1a/x" "$WORK/f1e/x" && mkdir -p "$WORK/f1e/fw/src" +printf '[package]\nname = "fw"\nversion = "0.1.0"\n[toolchain]\ndefault = "gcc@16.1.0"\n[build]\nsources = ["src/*.c"]\n[targets.fw]\nkind = "shared"\n[dependencies.x]\npath = "../x"\n' > "$WORK/f1e/fw/mcpp.toml" +cp "$WORK/f1a/fw/src/fw.c" "$WORK/f1e/fw/src/fw.c" +cd "$WORK/f1e/fw" && timeout 900 "$M" build > build.log 2>&1 +so=$(ls target/*/*/bin/libfw.so | head -1) +reading "F1e root-owned shared image over x: exit=$? libfw.so defines x_answer=$(nm -D --defined-only "$so" | grep -c ' x_answer$') undefined=$(nm -D --undefined-only "$so" | grep -c ' x_answer$')" + +# ---------------------------------------------------------------- F3 -------- +F3_DEFAULT_MAIN='import std; +import lib; +int main() { std::println("{}", lib_greet(3)); }' +f3() { # $1 id, $2 toolchain, [$3 root manifest tail], [$4 lib kind], [$5 main body] + local w="$WORK/f3-$1" + mkdir -p "$w/app/src" "$w/lib/src" + printf '[package]\nname = "lib"\nversion = "0.1.0"\n[targets.lib]\nkind = "%s"\n' "${4:-shared}" > "$w/lib/mcpp.toml" + printf 'export module lib;\nimport std;\nexport [[gnu::visibility("default")]] std::string lib_greet(int n);\n' > "$w/lib/src/lib.cppm" + printf 'module lib;\nimport std;\nstd::string lib_greet(int n) { return std::format("lib-{}", n); }\n' > "$w/lib/src/lib.cpp" + printf '[package]\nname = "app"\nversion = "0.1.0"\n[toolchain]\ndefault = "%s"\n[dependencies.lib]\npath = "../lib"\n%s' "$2" "${3:-}" > "$w/app/mcpp.toml" + local body=${5:-$F3_DEFAULT_MAIN} + printf '%s\n' "$body" > "$w/app/src/main.cpp" + cd "$w/app" + timeout 1800 "$M" build > build.log 2>&1; local rc=$? + reading "F3 $1 build exit=$rc symbol-provision mentions=$(grep -c 'symbol-provision\|also provided by a library it loads' build.log)" + local d; d=$(ls -d target/*/*/ 2>/dev/null | head -1) + [ -n "$d" ] || return + reading "F3 $1 run: $("$d/bin/app" 2>&1) exit=$?" + reading "F3 $1 NEEDED program: $(readelf -d "$d/bin/app" | grep -o 'Shared library: \[[^]]*\]' | tr '\n' ' ')" + [ -f "$d/bin/liblib.so" ] || { reading "F3 $1 no liblib.so (static form)"; return; } + reading "F3 $1 NEEDED liblib.so: $(readelf -d "$d/bin/liblib.so" | grep -o 'Shared library: \[[^]]*\]' | tr '\n' ' ')" + reading "F3 $1 contracts in resolution.json: $(python3 -c 'import json,sys,glob; d=json.load(open(sys.argv[1])); print(json.dumps(d["runtime"]["cxx_runtime_by_role"]))' "$d/resolution.json" 2>&1)" + reading "F3 $1 liblib.so link inputs: $(grep '^build bin/liblib.so' "$d/build.ninja" | sed 's/^build bin\/liblib.so : c[a-z_]* //; s/ |.*//')" + python3 - "$d/.mcpp-runtime-verdicts.json" "$d/obj/std.o" "$1" <<'PY' +import json, subprocess, sys +doc = json.load(open(sys.argv[1])) +def walk(o): + if isinstance(o, dict): + if 'conflicts' in o and 'path' in o: yield o + for v in o.values(): yield from walk(v) + elif isinstance(o, list): + for v in o: yield from walk(v) +stdo = set() +try: + out = subprocess.run(['nm', '--defined-only', '-g', sys.argv[2]], capture_output=True, text=True).stdout + stdo = {l.split()[-1] for l in out.splitlines() if l.strip()} + strong = sum(1 for l in out.splitlines() if l.split()[-2] in ('T','D','B','R')) + print(f"READING F3 {sys.argv[3]} std.o global definitions={len(stdo)} strong={strong} sample={sorted(stdo)[:3]}") +except Exception as e: + print(f"READING F3 {sys.argv[3]} std.o unreadable: {e}") +for e in walk(doc): + c = e.get('conflicts') or [] + names = [x.get('symbol') for x in c] + fromstd = sum(1 for n in names if n in stdo) + by = sorted({p for x in c for p in x.get('also_provided_by', [])}) + print(f"READING F3 {sys.argv[3]} {e['path']} status={e.get('status')} exported={e.get('exported')} conflicts={len(c)} defined-in-std.o={fromstd} first={names[:4]} providers={by}") +PY + timeout 600 "$M" build --strict > strict.log 2>&1 + reading "F3 $1 --strict exit=$? (fast path may skip the check: $(grep -c 'symbol-provision' strict.log) mentions)" + touch src/main.cpp; timeout 600 "$M" build --strict > strict2.log 2>&1 + reading "F3 $1 --strict after touch exit=$? mentions=$(grep -c 'symbol-provision\|also provided by' strict2.log)" +} +f3 gcc "gcc@16.1.0" +f3 llvm "llvm@22.1.8" +# Controls: which part of the default shape the findings and the abort belong to. +f3 llvm-static-lib "llvm@22.1.8" "" "lib" +f3 llvm-no-println "llvm@22.1.8" "" "shared" 'import lib; +#include +int main() { std::puts(lib_greet(3).c_str()); }' +f3 llvm-tc "llvm@22.1.8" '[build] +cxx_runtime = "toolchain-coupled" +' +f3 gcc-tc "gcc@16.1.0" '[build] +cxx_runtime = "toolchain-coupled" +' + +# ---------------------------------------------------------------- E10 ------- +# The GNU-style clang driver on the MSVC triple, with no runtime flag: which +# CRT does it select at compile and at link? (Driver logic is host-independent.) +cd "$WORK" && printf 'int main(){return 0;}\n' > e10.cpp +c=$("$LLVM/bin/clang++" --target=x86_64-pc-windows-msvc -### -c e10.cpp 2>&1 | tail -1) +reading "E10 compile, no flag: dependent-lib=$(echo "$c" | grep -o -- '--dependent-lib=[a-z]*' | tr '\n' ' ') defines=$(echo "$c" | grep -o -- '"-D" "_[A-Z]*"' | tr '\n' ' ')" +l=$("$LLVM/bin/clang++" --target=x86_64-pc-windows-msvc -fuse-ld=lld -### e10.o -o e10.exe 2>&1 | tail -1) +reading "E10 link, no flag: $(echo "$l" | grep -o -- '-defaultlib:[a-z]*' | tr '\n' ' ')" +for v in dll static; do + c=$("$LLVM/bin/clang++" --target=x86_64-pc-windows-msvc -fms-runtime-lib=$v -### -c e10.cpp 2>&1 | tail -1) + l=$("$LLVM/bin/clang++" --target=x86_64-pc-windows-msvc -fms-runtime-lib=$v -fuse-ld=lld -### e10.o -o e10.exe 2>&1 | tail -1) + reading "E10 -fms-runtime-lib=$v: compile $(echo "$c" | grep -o -- '--dependent-lib=[a-z]*\|"-D" "_[A-Z]*"' | tr '\n' ' ') link $(echo "$l" | grep -o -- '-defaultlib:[a-z]*' | tr '\n' ' ')" +done +__PROBES_646_649_GROUP_A__ + bash "$WORK/a.sh" "$WORK/a" "$WORK/b.sh" <<'__PROBES_646_649_GROUP_B__' +# Group b: probes for #647 E1 and #649 E5/E9, against the released mcpp 2026.9.15.2 +# on a Linux x86_64 host. Each reading is printed as `READING : ...`. +# Usage: bash probes.sh [workdir] (the Android leg needs xim:android-ndk installed) +set -uo pipefail +M=${M:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/2026.9.15.2/bin/mcpp} +W=${1:-$(mktemp -d)} +mkdir -p "$W"; cd "$W" +reading() { echo "READING $*"; } +reading "version: $($M --version)" + +# ── E1: app -> a -> b (path deps); b declares [package.metadata.demo] ────── +mkdir -p e1/app/src e1/a/src e1/b/src +cat > e1/b/mcpp.toml <<'EOF' +[package] +name = "b" +namespace = "spike" +version = "0.2.0" +standard = "c++20" + +[package.metadata.demo] +resources = "res" + +[targets.b] +kind = "lib" + +[build] +sources = ["src/*.cpp"] +EOF +echo 'int b_answer() { return 40; }' > e1/b/src/b.cpp +cat > e1/a/mcpp.toml <<'EOF' +[package] +name = "a" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.b = { path = "../b" } + +[targets.a] +kind = "lib" + +[build] +sources = ["src/*.cpp"] +EOF +echo 'int b_answer(); int a_answer() { return b_answer() + 2; }' > e1/a/src/a.cpp +cat > e1/app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.a = { path = "../a" } + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +printf 'int a_answer();\nint main() { return a_answer() == 42 ? 0 : 1; }\n' > e1/app/src/main.cpp +cat > e1/app/build.mcpp <<'EOF' +import std; +import mcpp; +extern char** environ; +int main() { + std::string s = "SPIKE deps:"; + for (char** e = environ; *e; ++e) { + std::string_view v(*e); + if (v.starts_with("MCPP_DEP_")) { s += " "; s += v.substr(0, v.find('=')); } + } + mcpp::warning(s.c_str()); + std::string c = "SPIKE run " + std::to_string(std::chrono::system_clock::now().time_since_epoch().count()); + mcpp::warning(c.c_str()); + return 0; +} +EOF +for p in a b; do cat > e1/$p/build.mcpp <&1); rc=$? +reading "E1-strict-metadata-table: exit=$rc warnings=$(grep -c '^warning: \(schema\|\[package\]\)' <<<"$out")" +reading "E1-root-env: $(grep -o 'SPIKE deps:.*' <<<"$out")" +reading "E1-program-order: $(grep -o 'SPIKE order [ab]' <<<"$out" | tr '\n' ' ')" +f=$(find target -name resolution.json | head -1) +reading "E1-graph-record: $(python3 -c "import json;j=json.load(open('$f'));print([ (p['package']['canonical'], sorted(p.keys())) for p in j['graph']['packages']])")" +first=$(grep -o 'SPIKE run [0-9]*' <<<"$out") +sleep 1; sed -i 's/resources = "res"/resources = "res2"/' ../b/mcpp.toml +out=$($M build 2>&1) +reading "E1-metadata-edit: $(grep -o 'build.mcpp [a-z ()]*' <<<"$out" | tr '\n' ';') replayed=$([ "$(grep -o 'SPIKE run [0-9]*' <<<"$out")" = "$first" ] && echo yes || echo no)" +sed -i 's/^standard = "c++20"/standard = "c++20"\nbogus_key = 1/' mcpp.toml +out=$($M build --strict 2>&1); rc=$? +reading "E1-unknown-package-key-strict: exit=$rc mentions=$(grep -c bogus_key <<<"$out")" +cd "$W" + +# ── E5 / E9: shared dependency, Android app row and Linux desktop row ──────── +mkdir -p e5/dep/src e5/app/src e5/hostapp/src +cat > e5/dep/mcpp.toml <<'EOF' +[package] +name = "dep" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[targets.dep] +kind = "shared" + +[build] +sources = ["src/*.cpp"] +EOF +printf '#include \n[[gnu::visibility("default")]] int dep_answer() { return static_cast(std::to_string(42).size()) + 40; }\n' > e5/dep/src/dep.cpp +cat > e5/app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.dep = { path = "../dep" } + +[targets.app] +kind = "app" +main = "src/main.cpp" + +[target.x86_64-linux-android] +min_api_level = 23 +EOF +printf '#include \nint dep_answer();\n[[gnu::visibility("default")]] int app_entry() { return dep_answer() + static_cast(std::to_string(1).size()); }\nint main() { return app_entry() == 43 ? 0 : 1; }\n' > e5/app/src/main.cpp +cat > e5/hostapp/mcpp.toml <<'EOF' +[package] +name = "hostapp" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.dep = { path = "../dep" } + +[targets.hostapp] +kind = "bin" +main = "src/main.cpp" +EOF +printf 'int dep_answer();\nint main() { return dep_answer() == 42 ? 0 : 1; }\n' > e5/hostapp/src/main.cpp + +sections() { echo "bytes=$(stat -c %s "$1") symtab=$(readelf -S -W "$1" | grep -c ' .symtab') debug=$(readelf -S -W "$1" | grep -c ' .debug_')"; } + +cd e5/hostapp +line=$($M pack --format tar 2>&1 | grep -E 'Packing') +reading "E5-desktop-status: $line" +rm -rf x && mkdir x && tar -xzf target/dist/hostapp-0.1.0-x86_64-linux-gnu.tar.gz -C x +for f in x/*/bin/hostapp x/*/lib/libdep.so; do reading "E5-desktop: ${f#x/*/} $(sections "$f")"; done +for a in "--release" "--dev" "--message-format json"; do + err=$($M pack $a --format tar 2>&1 >/dev/null | tail -1); $M pack $a --format tar >/dev/null 2>&1; rc=$? + reading "E9-pack $a: exit=$rc [$err]" +done +reading "E9-human-lines-on-stdout: bytes=$($M pack --format tar 2>/dev/null | wc -c)" +reading "E9-run-precedence(--profile dev --release): $($M run --profile dev --release 2>&1 | grep -o 'Finished [a-z]*')" +reading "E9-build-precedence(--profile dev --release): $($M build --profile dev --release 2>&1 | grep -o 'Finished [a-z]*')" +cd "$W" + +if ls -d "$HOME"/.mcpp/registry/data/xpkgs/xim-x-android-ndk/* >/dev/null 2>&1; then + cd e5/app + line=$(MCPP_OFFLINE=1 $M pack --target x86_64-linux-android --format tar 2>&1 | grep -E 'Packing') + reading "E5-android-status: $line" + rm -rf x && mkdir x && tar -xzf target/dist/app-0.1.0-x86_64-linux-android.tar.gz -C x + for f in x/*/lib/*.so; do reading "E5-android: ${f#x/*/} $(sections "$f")"; done + cd "$W" +else + reading "E5-android: skipped (xim:android-ndk not installed)" +fi +command -v swiftc >/dev/null || reading "E2-host: no swiftc on this host; E2 needs macos-15" +__PROBES_646_649_GROUP_B__ + bash "$WORK/b.sh" "$WORK/b" "$WORK/c.sh" <<'__PROBES_646_649_GROUP_C__' +# Group c: probes for #647 E4 and #649 E6/E7/E8, against the released mcpp 2026.9.15.2. +# Usage: probes.sh [e41|e42|e43|e6|e6b|e6c|e7|e8|all] +set +e +M=${M:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/2026.9.15.2/bin/mcpp} +ROOT=${ROOT:-$(cd "$(dirname "$0")" && pwd)/work} +reading() { echo "READING $1: $2"; } +pkg() { # dir ns name kind [extra toml] + mkdir -p "$1/src" + { echo '[package]'; echo "name = \"$3\""; [ -n "$2" ] && echo "namespace = \"$2\""; + echo 'version = "0.1.0"'; echo 'standard = "c++20"'; echo; echo "[targets.$3]"; echo "kind = \"$4\""; + [ "$4" = bin ] && echo 'main = "src/main.cpp"'; [ "$4" = lib ] && { echo; echo '[build]'; echo 'sources = ["src/*.cpp"]'; } + echo; printf '%s\n' "${5:-}"; } > "$1/mcpp.toml" +} +run() { # label dir args... + local label=$1 dir=$2; shift 2 + out=$(cd "$dir" && timeout 900 "$M" "$@" 2>&1); rc=$? + printf '%s\n' "$out" > "$ROOT/$label.log" + echo "rc=$rc" >> "$ROOT/$label.log" +} +mkdir -p "$ROOT" +$M --version + +e41() { # a forward along [build-dependencies] edges, two levels, host only + local W=$ROOT/e41; rm -rf "$W"; mkdir -p "$W" + pkg "$W/kt" spike kt bin; echo 'int main(){return 0;}' > "$W/kt/src/main.cpp" + pkg "$W/leaf" spike leaf lib '[features] +kt = [] + +[feature-deps.kt] +spike.kt = { path = "../kt", tools = ["kt"] }' + echo 'int leaf_answer(){return 1;}' > "$W/leaf/src/leaf.cpp" + pkg "$W/rules" spike rules lib '[build-dependencies] +spike.leaf = { path = "../leaf" } + +[features] +kotlin = ["spike.leaf/kt"]' + echo 'int rules_answer(){return 1;}' > "$W/rules/src/rules.cpp" + pkg "$W/fw" spike fw lib '[build-dependencies] +spike.rules = { path = "../rules" } + +[features] +kotlin = ["spike.rules/kotlin"]' + echo 'int fw_answer(){return 42;}' > "$W/fw/src/fw.cpp" + pkg "$W/app" "" app bin '[dependencies] +spike.fw = { path = "../fw", features = ["kotlin"] }' + printf 'int fw_answer();\nint main(){return fw_answer()==42?0:1;}\n' > "$W/app/src/main.cpp" + run e41-build "$W/app" build + reading E4.1-build "$(grep -c 'forwards to dependency' $ROOT/e41-build.log) forward warnings; tool kt built: $(grep -c 'Building host tool kt' $ROOT/e41-build.log); $(tail -1 $ROOT/e41-build.log)" + grep 'forwards to dependency' $ROOT/e41-build.log | sed 's/^/ /' + run e41-strict "$W/app" build --strict + reading E4.1-strict "$(grep -m1 -o 'error: .*' $ROOT/e41-strict.log | cut -c1-120); $(tail -1 $ROOT/e41-strict.log)" + sed -i 's/features = \["kotlin"\]/features = []/' "$W/app/mcpp.toml"; rm -rf "$W/app/target" + run e41-off "$W/app" build + reading E4.1-control-without-feature "tool kt built: $(grep -c 'Building host tool kt' $ROOT/e41-off.log); $(tail -1 $ROOT/e41-off.log)" + rm -rf "$W"/*/target +} + +e42() { # [feature-deps.] adding tools to an unconditionally declared dependency + local W=$ROOT/e42; rm -rf "$W"; mkdir -p "$W" + pkg "$W/installer" spike installer bin; echo '#include +int main(){std::puts("installer ran");return 0;}' > "$W/installer/src/main.cpp" + # the installer package also needs a library so that the unconditional edge means something + pkg "$W/app" "" app bin '[dependencies] +spike.installer = { path = "../installer" } + +[features] +installer = [] + +[feature-deps.installer] +spike.installer = { tools = ["installer"] }' + echo 'int main(){return 0;}' > "$W/app/src/main.cpp" + run e42-sourceless "$W/app" build --features installer + reading E4.2-sourceless "$(grep -m1 -o 'error: .*' $ROOT/e42-sourceless.log | cut -c1-160); $(tail -1 $ROOT/e42-sourceless.log)" + # restated with the same source: the documented intent, spelled as the parser requires + sed -i 's|spike.installer = { tools = \["installer"\] }|spike.installer = { path = "../installer", tools = ["installer"] }|' "$W/app/mcpp.toml" + cat > "$W/app/build.mcpp" <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("installer", "installer"); + const char* b = mcpp::dep_bin("spike.installer", "installer"); + mcpp::warning((std::string("SPIKE short=[") + (a ? a : "") + "] qualified=[" + (b ? b : "") + "]").c_str()); + return 0; +} +EOF + run e42-restated "$W/app" build --features installer + reading E4.2-restated "$(grep -o 'SPIKE.*' $ROOT/e42-restated.log | sed "s|$HOME|~|g" | cut -c1-200); $(tail -1 $ROOT/e42-restated.log)" + rm -rf "$W/app/target" + run e42-restated-off "$W/app" build + reading E4.2-restated-without-feature "$(grep -o 'SPIKE.*' $ROOT/e42-restated-off.log); $(tail -1 $ROOT/e42-restated-off.log)" + # a restatement naming a DIFFERENT source than the unconditional declaration + mkdir -p "$W/other"; cp -r "$W/installer/." "$W/other/" + sed -i 's|spike.installer = { path = "../installer", tools|spike.installer = { path = "../other", tools|' "$W/app/mcpp.toml" + rm -rf "$W/app/target" + run e42-mismatch "$W/app" build --features installer --strict + reading E4.2-restated-different-path "warnings=$(grep -c '^warning' $ROOT/e42-mismatch.log) tool-from: $(grep -o 'Building host tool.*' $ROOT/e42-mismatch.log | head -1) $(grep -o 'SPIKE.*' $ROOT/e42-mismatch.log | sed "s|$HOME|~|g" | cut -c1-200); $(tail -1 $ROOT/e42-mismatch.log)" + rm -rf "$W"/*/target +} + + +e41b() { # a forward to a dependency declared for another row, and to one behind an inactive feature + local W=$ROOT/e41b; rm -rf "$W"; mkdir -p "$W" + pkg "$W/win" spike win lib '[features] +x = []' + echo 'int win_answer(){return 1;}' > "$W/win/src/win.cpp" + pkg "$W/fw" spike fw lib '[target.'"'"'cfg(os = "windows")'"'"'.dependencies] +spike.win = { path = "../win" } + +[features] +kotlin = ["spike.win/x"]' + echo 'int fw_answer(){return 42;}' > "$W/fw/src/fw.cpp" + pkg "$W/app" "" app bin '[dependencies] +spike.fw = { path = "../fw", features = ["kotlin"] }' + printf 'int fw_answer();\nint main(){return fw_answer()==42?0:1;}\n' > "$W/app/src/main.cpp" + run e41b-other-row "$W/app" build --strict + reading E4.1b-forward-to-other-row-dep "$(grep -m1 -o 'error: .*' $ROOT/e41b-other-row.log | cut -c1-140); $(tail -1 $ROOT/e41b-other-row.log)" + rm -rf "$W"/*/target +} + + +e6_fixture() { # W installer-edge-toml tool-dep-toml + local W=$1; rm -rf "$W"; mkdir -p "$W" + pkg "$W/fw" spike fw lib "[features] +installer = [] + +$2" + echo 'int fw_answer() { return 42; }' > "$W/fw/src/fw.cpp" + pkg "$W/fw/tool" spike fw-installer bin "$3" + printf 'int fw_answer();\n#include \nint main() { std::printf("fw-installer ran %%d\\n", fw_answer()); return 0; }\n' > "$W/fw/tool/src/main.cpp" + pkg "$W/app" "" app bin '[dependencies] +spike.fw = { path = "../fw" } + +[features] +windows-installer = ["spike.fw/installer"]' + printf 'int fw_answer();\nint main() { return fw_answer() == 42 ? 0 : 1; }\n' > "$W/app/src/main.cpp" + cat > "$W/app/build.mcpp" <<'EOF2' +import std; +import mcpp; + +int main() { + const char* a = mcpp::dep_bin("fw-installer", "fw-installer"); + const char* b = mcpp::dep_bin("spike.fw-installer", "fw-installer"); + const std::string message = std::string("SPIKE short=[") + (a ? a : "") + "] qualified=[" + (b ? b : "") + "]"; + mcpp::warning(message.c_str()); + return 0; +} +EOF2 +} +e6() { # the issue's fixture + local W=$ROOT/e6 + e6_fixture "$W" "[target.'cfg(os = \"linux\")'.feature-deps.installer] +spike.fw-installer = { path = \"tool\", tools = [\"fw-installer\"], reexport = true }" '[dependencies] +spike.fw = { path = ".." }' + run e6-build "$W/app" build --features windows-installer + reading E6-issue-fixture "$(grep -o 'SPIKE.*' $ROOT/e6-build.log | sed "s|$HOME|~|g" | cut -c1-160) | $(grep -m1 -o 'error: .*' $ROOT/e6-build.log); $(tail -1 $ROOT/e6-build.log)" + local tool=$(grep -o 'short=\[[^]]*' $ROOT/e6-build.log | sed 's/short=\[//') + [ -n "$tool" ] && reading E6-tool-runs "$($tool)" + run e6-nofeature "$W/app" build + reading E6-without-feature "$(grep -m1 -o 'error: .*' $ROOT/e6-nofeature.log); $(tail -1 $ROOT/e6-nofeature.log)" + # the resolved graph: which edges exist + run e6-local "$W/app" build --features windows-installer --cache=local + local exe=$(find "$W/app/target" -type f -perm -u+x -name app | head -1) + reading E6-cache-local "$(grep -m1 -o 'error: .*' $ROOT/e6-local.log); $(tail -1 $ROOT/e6-local.log); app runs: $([ -n "$exe" ] && { "$exe"; echo exit=$?; })" + reading E6-cache-local-link "$(grep -h '^build bin/app' $(find "$W/app/target" -name build.ninja) | sed 's|.*cxx_link||')" + rm -rf "$W"/*/target "$W"/fw/tool/target +} +e6b() { # the tool graph REALLY activates the declaring feature + local W=$ROOT/e6b + e6_fixture "$W" "[target.'cfg(os = \"linux\")'.feature-deps.installer] +spike.fw-installer = { path = \"tool\", tools = [\"fw-installer\"], reexport = true }" '[dependencies] +spike.fw = { path = "..", features = ["installer"] }' + run e6b-build "$W/app" build --features windows-installer + reading E6b-tool-activates-declaring-feature "$(grep -c 'Building host tool' $ROOT/e6b-build.log) host-tool lines | $(grep -m1 -o 'error: .*' $ROOT/e6b-build.log | cut -c1-200); $(tail -1 $ROOT/e6b-build.log)" + grep -A3 'error:' $ROOT/e6b-build.log | head -6 | sed "s|$HOME|~|g; s/^/ /" + rm -rf "$W"/*/target "$W"/fw/tool/target +} +e6c() { # does a tool-only dependency's own library reach the consumer's link? + local W=$ROOT/e6c + e6_fixture "$W" "[feature-deps.installer] +spike.fw-installer = { path = \"tool\", tools = [\"fw-installer\"], reexport = true }" '[dependencies] +spike.z = { path = "../../z" }' + pkg "$W/z" spike z lib '' + echo 'int z_only_in_the_tool() { return 7; }' > "$W/z/src/z.cpp" + printf 'int z_only_in_the_tool();\n#include \nint main() { std::printf("tool %%d\\n", z_only_in_the_tool()); return 0; }\n' > "$W/fw/tool/src/main.cpp" + run e6c-build "$W/app" build --features windows-installer --verbose + local exe=$(find "$W/app/target" -type f -name app -perm -u+x | head -1) + reading E6c-tool-dependency-in-consumer "app link line:$(grep -h '^build bin/app' $(find "$W/app/target" -name build.ninja) | sed 's|.*cxx_link||'); app defines z symbol: $(nm -C "$exe" 2>/dev/null | grep -c z_only_in_the_tool); $(tail -1 $ROOT/e6c-build.log)" + rm -rf "$W"/*/target "$W"/fw/tool/target +} +e6d() { # the same cycle without a feature: an unconditional [build-dependencies] tool edge + local W=$ROOT/e6d + e6_fixture "$W" '[build-dependencies] +spike.fw-installer = { path = "tool", tools = ["fw-installer"], reexport = true }' '[dependencies] +spike.fw = { path = ".." }' + run e6d-build "$W/app" build + reading E6d-build-dependency-tool-edge "$(grep -o 'SPIKE.*' $ROOT/e6d-build.log | sed "s|$HOME|~|g" | cut -c1-120) | $(grep -m1 -o 'error: .*' $ROOT/e6d-build.log); $(tail -1 $ROOT/e6d-build.log)" + rm -rf "$W"/*/target "$W"/fw/tool/target +} + + +e7() { # a git dependency naming a workspace member of the repository + local W=$ROOT/e7; rm -rf "$W"; mkdir -p "$W" + pkg "$W/repo" spike fw lib '[workspace] +members = ["tool"]' + echo 'int fw_answer() { return 42; }' > "$W/repo/src/fw.cpp" + pkg "$W/repo/tool" spike fw-installer bin '[dependencies] +spike.fw = { path = ".." }' + printf 'int fw_answer();\n#include \nint main() { std::printf("fw-installer ran %%d\\n", fw_answer()); return 0; }\n' > "$W/repo/tool/src/main.cpp" + (cd "$W/repo" && git init -q && git add -A && git -c user.email=p@p -c user.name=p commit -qm init) + local rev=$(git -C "$W/repo" rev-parse HEAD) + local url="file://$W/repo" + pkg "$W/app" "" app bin "[dependencies] +spike.fw = { git = \"$url\", rev = \"$rev\" } + +[features] +windows-installer = [] + +[target.'cfg(os = \"linux\")'.feature-deps.windows-installer] +spike.fw-installer = { git = \"$url\", rev = \"$rev\", tools = [\"fw-installer\"] }" + printf 'int fw_answer();\nint main() { return fw_answer() == 42 ? 0 : 1; }\n' > "$W/app/src/main.cpp" + cat > "$W/app/build.mcpp" <<'EOF2' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("fw-installer", "fw-installer"); + mcpp::warning((std::string("SPIKE short=[") + (a ? a : "") + "]").c_str()); + return 0; +} +EOF2 + run e7-both "$W/app" build --features windows-installer + reading E7-both-keys "$(grep -c 'that identity is used' $ROOT/e7-both.log) adoption warning(s) | $(grep -o 'SPIKE.*' $ROOT/e7-both.log) | $(grep -o 'Compiling spike.fw.*' $ROOT/e7-both.log) | $(grep -m1 -o 'error: .*' $ROOT/e7-both.log | cut -c1-160); $(tail -1 $ROOT/e7-both.log)" + # the member alone, with no key naming the root package + rm -rf "$W/app/target" + sed -i '/^spike.fw = { git/d' "$W/app/mcpp.toml" + sed -i 's/^int fw_answer();$//; s/return fw_answer() == 42 ? 0 : 1;/return 0;/' "$W/app/src/main.cpp" + run e7-member-only "$W/app" build --features windows-installer + reading E7-member-only "$(grep -m1 -o 'error: .*' $ROOT/e7-member-only.log | sed "s|$HOME|~|g" | cut -c1-200); $(tail -1 $ROOT/e7-member-only.log)" + # control: the same member by path from a checkout works + rm -rf "$W/app/target" + sed -i "s|spike.fw-installer = { git = \"$url\", rev = \"$rev\",|spike.fw-installer = { path = \"../repo/tool\",|" "$W/app/mcpp.toml" + run e7-member-by-path "$W/app" build --features windows-installer + reading E7-member-by-path "$(grep -o 'SPIKE.*' $ROOT/e7-member-by-path.log | sed "s|$HOME|~|g" | cut -c1-120) | $(grep -m1 -o 'error: .*' $ROOT/e7-member-by-path.log); $(tail -1 $ROOT/e7-member-by-path.log)" + rm -rf "$W"/app/target "$W"/repo/target "$W"/repo/tool/target +} + +edup() { # one consumer naming one package in [dependencies] and [build-dependencies] with different requests + local W=$ROOT/edup; rm -rf "$W"; mkdir -p "$W" + mkdir -p "$W/installer/src" + cat > "$W/installer/mcpp.toml" <<'EOF2' +[package] +name = "installer" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[targets.installer-lib] +kind = "lib" + +[targets.installer] +kind = "bin" +main = "src/main.cpp" + +[build] +sources = ["src/lib.cpp"] +EOF2 + echo 'int installer_answer() { return 42; }' > "$W/installer/src/lib.cpp" + echo 'int main() { return 0; }' > "$W/installer/src/main.cpp" + pkg "$W/app" "" app bin '[dependencies] +spike.installer = { path = "../installer" } + +[build-dependencies] +spike.installer = { path = "../installer", tools = ["installer"] }' + printf 'int installer_answer();\nint main() { return installer_answer() == 42 ? 0 : 1; }\n' > "$W/app/src/main.cpp" + cat > "$W/app/build.mcpp" <<'EOF2' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("installer", "installer"); + mcpp::warning((std::string("SPIKE short=[") + (a ? a : "") + "]").c_str()); + return 0; +} +EOF2 + run edup-two-tables "$W/app" build --strict + reading EDUP-dependencies-and-build-dependencies "tool built: $(grep -c 'Building host tool' $ROOT/edup-two-tables.log) | $(grep -o 'SPIKE.*' $ROOT/edup-two-tables.log | sed "s|$HOME|~|g" | cut -c1-100) | $(grep -m1 -o 'error: .*' $ROOT/edup-two-tables.log); $(tail -1 $ROOT/edup-two-tables.log)" + rm -rf "$W"/*/target +} + +e8() { # --features with a dependency's feature, with and without a root [features] table + local W=$ROOT/e8 + e6_fixture "$W" "[target.'cfg(os = \"linux\")'.feature-deps.installer] +spike.fw-installer = { path = \"tool\", tools = [\"fw-installer\"], reexport = true }" '' + printf '#include \nint main() { std::puts("fw-installer ran"); return 0; }\n' > "$W/fw/tool/src/main.cpp" + sed -i '/^\[features\]$/,$d' "$W/app/mcpp.toml" # no [features] table in the application + run e8-dep-feature-no-table "$W/app" build --strict --features spike.fw/installer + reading E8-dep-feature-no-table "$(grep -o 'SPIKE.*' $ROOT/e8-dep-feature-no-table.log) | tool built: $(grep -c 'Building host tool' $ROOT/e8-dep-feature-no-table.log); $(tail -1 $ROOT/e8-dep-feature-no-table.log)" + reading E8-dep-feature-no-table-macro "$(grep -o -- '-DMCPP_FEATURE_[A-Z_]*' $W/app/compile_commands.json | sort -u | tr '\n' ' ')" + run e8-unknown-no-table "$W/app" build --strict --features nothing-here + reading E8-unknown-no-table "$(grep -m1 -o 'error: .*' $ROOT/e8-unknown-no-table.log); $(tail -1 $ROOT/e8-unknown-no-table.log); macros: $(grep -o -- '-DMCPP_FEATURE_[A-Z_]*' $W/app/compile_commands.json | sort -u | tr '\n' ' ')" + printf '\n[features]\nwindows-installer = []\n' >> "$W/app/mcpp.toml" + run e8-dep-feature-table "$W/app" build --features spike.fw/installer + reading E8-dep-feature-with-table "$(grep -m1 -o 'warning: --features.*' $ROOT/e8-dep-feature-table.log) | tool built: $(grep -c 'Building host tool' $ROOT/e8-dep-feature-table.log); $(tail -1 $ROOT/e8-dep-feature-table.log)" + run e8-dep-feature-table-strict "$W/app" build --strict --features spike.fw/installer + reading E8-dep-feature-with-table-strict "$(grep -m1 -o 'error: .*' $ROOT/e8-dep-feature-table-strict.log); $(tail -1 $ROOT/e8-dep-feature-table-strict.log)" + # control: the manifest forward the CLI form would replace + sed -i 's/^windows-installer = \[\]$/windows-installer = ["spike.fw\/installer"]/' "$W/app/mcpp.toml" + run e8-manifest-forward "$W/app" build --strict --features windows-installer + reading E8-manifest-forward-control "$(grep -o 'SPIKE.*' $ROOT/e8-manifest-forward.log | sed "s|$HOME|~|g" | cut -c1-110) | tool built: $(grep -c 'Building host tool' $ROOT/e8-manifest-forward.log); $(tail -1 $ROOT/e8-manifest-forward.log)" + rm -rf "$W"/*/target "$W"/fw/tool/target +} +case "${1:-all}" in + all) e41; e41b; e42; e6; e6b; e6c; e6d; e7; edup; e8 ;; + *) for f in "$@"; do "$f"; done ;; +esac +__PROBES_646_649_GROUP_C__ + ROOT="$WORK/c" bash "$WORK/c.sh" all "$WORK/d.sh" <<'__PROBES_646_649_GROUP_D__' +# Group d: probes for #648. Each prints READING lines. Linux x86_64, mcpp 2026.9.15.2. +set +e +M=${M:-~/.xlings/data/xpkgs/xim-x-mcpp/2026.9.15.2/bin/mcpp} +W=${W:-$(mktemp -d)} + +# ── A2: the descriptors a child started during planning inherits ────────── +# A build program lists /proc/self/fd; the reader prints the inode of the pipe +# it reads mcpp's stdout from. A child fd naming the same inode is the leak. +mkdir -p "$W/a2/src" && cd "$W/a2" || exit 1 +cat > mcpp.toml <<'T' +[package] +name = "a2probe" +version = "0.1.0" +standard = "c++23" + +[targets.a2probe] +kind = "bin" +main = "src/main.cpp" +T +echo 'int main() { return 0; }' > src/main.cpp +cat > build.mcpp < " + << std::filesystem::read_symlink(e.path(), ec).string() << "\n"; + } + return 0; +} +T +MCPP_OFFLINE=1 "$M" emit build-database --format json 2>/dev/null \ + | { echo "READING A2 reader stdin=$(readlink /proc/self/fd/0)"; cat >/dev/null; } +sed 's/^/READING A2 child fd /' "$W/a2/fds.txt" + +# ── T: what triggered the refresh on openxlings/xlings ──────────────────── +# Offline, the refresh decision is still computed; a dependency reported as +# "offline mode" is one the decision wanted to refresh. The same run resolves it. +XL=${XL:-$HOME/workspace/github/openxlings/xlings} +if [ -f "$XL/mcpp.toml" ]; then + (cd "$XL" && MCPP_OFFLINE=1 timeout 180 "$M" emit build-database --format json -v \ + 2>"$W/xl.err" >"$W/xl.json"; echo "READING T exit=$?") + grep -a "index: .*offline mode" "$W/xl.err" | sed 's/\x1b\[[0-9;]*m//g; s/^.*index: /READING T decision /' + grep -a "resolved to .* through the deprecated bare-name search" "$W/xl.err" | sed 's/^/READING T resolver /' +fi + +# ── A6: the GitCode mirror of the mcpp-index artifact against its pointer ── +ptr=$(curl -sSL https://raw.githubusercontent.com/xlings-res/mcpp-index/main/mcpp-index-pointers.json) +ver=$(printf '%s' "$ptr" | python3 -c 'import json,sys; print(json.load(sys.stdin)["indexes"]["mcpp"]["index_version"])') +sha=$(printf '%s' "$ptr" | python3 -c 'import json,sys; print(json.load(sys.stdin)["indexes"]["mcpp"]["artifact"]["sha256"])') +echo "READING A6 pointer version=$ver sha=$sha" +for host in github gitcode; do + url="https://$host.com/xlings-res/mcpp-index/releases/download/v$ver/mcpp-index-$ver.tar.gz" + curl -sSL -o "$W/$host.tgz" "$url" + echo "READING A6 $host sha=$(sha256sum "$W/$host.tgz" | cut -d' ' -f1) bytes=$(wc -c < "$W/$host.tgz")" + echo "READING A6 $host first-entry=$(tar -tvzf "$W/$host.tgz" | head -1)" +done +__PROBES_646_649_GROUP_D__ + W="$WORK/d" bash "$WORK/d.sh" &2; exit 2 ;; +esac diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 056f8a57..c04f94f6 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -289 records. +291 records. ## By subject @@ -62,6 +62,8 @@ Records that declare one. Everything else is listed by date below. ### triage +- [#646 to #649 implemented: the plan, its review from eight angles, and the ledger](2026-09-16-646-649-implementation-plan.md) — active +- [Four issues from a framework and an editor: #646, #647, #648 and #649, read against the engine and routed by home](2026-09-16-646-649-four-issues-by-home.md) — active - [Link forms, standard levels and a path limit: the asks of #641 and #642, read against the code](2026-09-15-641-642-link-forms-standards-and-paths.md) — landed - [#641 and #642 implemented: the plan, its review from eight angles, and the ledger](2026-09-15-641-642-implementation-plan.md) — landed - [#634 implemented across five repositories: the plan, its review, and the ledger that tracks it](2026-09-14-634-implementation-plan.md) — landed @@ -74,6 +76,8 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 +- [#646 to #649 implemented: the plan, its review from eight angles, and the ledger](2026-09-16-646-649-implementation-plan.md) — active +- [Four issues from a framework and an editor: #646, #647, #648 and #649, read against the engine and routed by home](2026-09-16-646-649-four-issues-by-home.md) — active - [Link forms, standard levels and a path limit: the asks of #641 and #642, read against the code](2026-09-15-641-642-link-forms-standards-and-paths.md) — landed - [#641 and #642 implemented: the plan, its review from eight angles, and the ledger](2026-09-15-641-642-implementation-plan.md) — landed - [The build database of #636, and two defects on the way to the latest xlings](2026-09-14-636-build-database-and-the-latest-xlings.md) — active From ed52f782f692397e887002499184bd18cbfebe90 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:01:47 +0800 Subject: [PATCH 02/20] The refresh decision walks the resolver's bare-name rung; the saved stdout is not inherited (#648 L1, L2) --- modules/platform/src/terminal.cppm | 34 ++++++++++++++++--- src/pm/index_refresh.cppm | 18 ++++++++++ tests/unit/test_pm_index_refresh.cpp | 50 ++++++++++++++++++++++++++++ 3 files changed, 98 insertions(+), 4 deletions(-) diff --git a/modules/platform/src/terminal.cppm b/modules/platform/src/terminal.cppm index 8d5f6203..b8a7fe24 100644 --- a/modules/platform/src/terminal.cppm +++ b/modules/platform/src/terminal.cppm @@ -12,9 +12,17 @@ module; #include #endif #if defined(_WIN32) -#include // _dup, _dup2, _close +#include // _dup, _dup2, _close, _get_osfhandle +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include // SetHandleInformation, HANDLE_FLAG_INHERIT #else -#include // dup, dup2, close +#include // dup2, close +#include // fcntl, F_DUPFD_CLOEXEC #endif export module mcpp.platform.terminal; @@ -39,6 +47,15 @@ std::size_t cols(); // a document (`mcpp emit build-database`) plans under one of these and prints // the document after it is gone: people still see the progress, on stderr, and // the document arrives alone. +// +// THE SAVED DESCRIPTOR IS NOT INHERITED. It is the caller's pipe, and a child +// started during the redirection (a build program, an xlings refresh) that +// inherited it would keep the caller from reading end-of-file for as long as +// the child lives, however long after mcpp itself exited. Measured +// (mcpp-community/mcpp#648): a build program started by +// `emit build-database` held the caller's pipe as its descriptor 3. The copy +// is therefore close-on-exec on POSIX and not inheritable on Windows, so a +// child receives only descriptors 0 to 2, and 1 is standard error here. class StdoutToStderr { public: StdoutToStderr(); @@ -77,9 +94,18 @@ StdoutToStderr::StdoutToStderr() { std::fflush(stdout); #if defined(_WIN32) saved_ = ::_dup(1); - if (saved_ >= 0) ::_dup2(2, 1); + if (saved_ >= 0) { + // `_dup` duplicates the handle as inheritable, and CreateProcess with + // handle inheritance (every `_popen` and `system`) passes it on. + const auto h = reinterpret_cast(::_get_osfhandle(saved_)); + if (h != INVALID_HANDLE_VALUE) + ::SetHandleInformation(h, HANDLE_FLAG_INHERIT, 0); + ::_dup2(2, 1); + } #else - saved_ = ::dup(1); + // Not `dup`: its copy survives exec. Descriptor 3 or above, as `dup` + // would have chosen, so nothing else about the redirection moves. + saved_ = ::fcntl(1, F_DUPFD_CLOEXEC, 3); if (saved_ >= 0) ::dup2(2, 1); #endif } diff --git a/src/pm/index_refresh.cppm b/src/pm/index_refresh.cppm index 591b5d81..ff51e8b2 100644 --- a/src/pm/index_refresh.cppm +++ b/src/pm/index_refresh.cppm @@ -41,6 +41,7 @@ import mcpp.log; import mcpp.platform; import mcpp.platform.axis; import mcpp.pm.dep_spec; +import mcpp.pm.dependency_selector; // legacy_bare_candidates import mcpp.pm.index_contract; import mcpp.pm.index_route; import mcpp.pm.resolver; @@ -223,6 +224,23 @@ RefreshDecision decide_for_dependency(const IndexRoute& route, // 3. INV-3: a miss only counts when the index could have refuted it. auto found = lookup_descriptor(route, coords); + + // 3b. THE RESOLVER'S LADDER, NOT A SHORTER ONE. After the exact coordinate + // misses, the resolver tries the deprecated bare-name rung for a + // version selector whose namespace was omitted (`ftxui = "6.1.9"` + // reaches `compat.ftxui`, prepare.cppm). A decision that stopped at the + // exact coordinate called that dependency missing while the resolver + // found it on disk, so every build of such a manifest started a network + // refresh once the debounce had passed (mcpp-community/mcpp#648: an + // editor planning in the background waited on one for 11 minutes). The + // same condition as the resolver's, for as long as the rung exists. + if (!found.hit && found.error.empty() && found.conclusive + && spec.isVersion() && spec.namespaceOmitted && !coords.empty()) { + auto legacy = lookup_descriptor( + route, mcpp::pm::legacy_bare_candidates(coords.front())); + if (legacy.hit) found = std::move(legacy); + } + if (!found.error.empty()) { d.reason = RefreshReason::SuppressedMalformedDescriptor; return d; diff --git a/tests/unit/test_pm_index_refresh.cpp b/tests/unit/test_pm_index_refresh.cpp index a4a912a9..c7604be1 100644 --- a/tests/unit/test_pm_index_refresh.cpp +++ b/tests/unit/test_pm_index_refresh.cpp @@ -62,6 +62,27 @@ package = {{ )", name, versions, versions, versions); } + // A descriptor under another namespace, laid out as the index lays out + // `compat.*` packages: pkgs//..lua. + void publish_namespaced(std::string_view ns, std::string_view name, + std::string_view versions) { + auto dir = index_dir() / "pkgs" / std::string(1, ns.front()); + std::filesystem::create_directories(dir); + std::ofstream(dir / std::format("{}.{}.lua", ns, name)) << std::format(R"( +package = {{ + spec = "1", + namespace = "{}", + name = "{}", + type = "package", + xpm = {{ + linux = {{ {} }}, + macosx = {{ {} }}, + windows = {{ {} }}, + }}, +}} +)", ns, name, versions, versions, versions); + } + // Age the refresh marker by writing it and back-dating it. void mark_refreshed(std::chrono::seconds ago = std::chrono::seconds{0}) { auto marker = index_dir() / ".mcpp-index-updated"; @@ -147,6 +168,35 @@ TEST(PmIndexRefresh, OfflineSuppressesEvenAGenuineMiss) { EXPECT_FALSE(refreshes(route, spec, reg.env(), offline)); } +// mcpp-community/mcpp#648. `ftxui = "6.1.9"` omits the namespace, so the +// exact coordinate is (mcpplibs, ftxui), which the index does not carry; the +// resolver then reaches compat.ftxui through the deprecated bare-name rung and +// the build succeeds from disk. The decision must walk the same rung, or every +// build of such a manifest refreshes the index once the debounce has passed. +TEST(PmIndexRefresh, BareNameResolvedThroughTheLegacyRungIsNotAMiss) { + FakeRegistry reg("legacyrung"); + reg.publish_namespaced("compat", "ftxui", R"(["6.1.9"] = { url = "u", sha256 = "s" },)"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto bare = version_dep("mcpplibs", "ftxui", "6.1.9"); + bare.namespaceOmitted = true; + EXPECT_EQ(decide(route, bare, reg.env()), RefreshReason::None); + EXPECT_FALSE(refreshes(route, bare, reg.env())); + + // A constraint is checked against the descriptor the rung found. + auto bareRange = version_dep("mcpplibs", "ftxui", "^7"); + bareRange.namespaceOmitted = true; + EXPECT_EQ(decide(route, bareRange, reg.env()), RefreshReason::VersionMiss); + + // The namespace written out states an identity: the rung does not apply, + // exactly as in the resolver, and the miss stands. + auto stated = version_dep("mcpplibs", "ftxui", "6.1.9"); + EXPECT_EQ(decide(route, stated, reg.env()), RefreshReason::DescriptorMiss); +} + TEST(PmIndexRefresh, OfflineDoesNotMisreportAResolvableDependency) { // The opt-outs are applied AFTER the local analysis, so a dependency that // resolved fine is reported as such even offline. Short-circuiting on the From 5c8b851ee77234c3cd235f555b713df64309f3de Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:08:34 +0800 Subject: [PATCH 03/20] Refreshes go through one policy; an offline plan that needs a download has its own code (#648 L5, L6) The fetcher's pre-install and retry refreshes and a project's first custom-index sync now take the refresh policy's decision, so [index] auto_refresh = false governs them. The policy half of mcpp.pm.index_refresh moves to mcpp.pm.refresh_policy, which the fetcher can import. Every offline refusal records offline-download-required, and emit build-database reports it as MCPP_OFFLINE_DOWNLOAD_REQUIRED. --- src/build/prepare.cppm | 32 +++++- src/build/refusal.cppm | 9 ++ src/cli/cmd_build.cppm | 16 ++- src/pm/index_refresh.cppm | 177 +----------------------------- src/pm/package_fetcher.cppm | 30 +++-- src/pm/refresh_policy.cppm | 212 ++++++++++++++++++++++++++++++++++++ src/xlings/xlings.cppm | 53 ++------- 7 files changed, 298 insertions(+), 231 deletions(-) create mode 100644 src/pm/refresh_policy.cppm diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index d3888aac..94e636f5 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -1560,6 +1560,7 @@ provision_xlings_addresses(const mcpp::config::GlobalConfig& cfg, mcpp::platform::env::offline_mode() ? "drop --offline / unset MCPP_OFFLINE" : "unset MCPP_NO_AUTO_INSTALL"; + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "{} are declared but not provisioned, " "and auto-install is off.\n" @@ -3472,6 +3473,7 @@ prepare_build(bool print_fingerprint, // will actually work there instead. if (mcpp::platform::is_windows && !msvc_usable_either_origin()) { + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "no toolchain configured (and no Visual Studio found).\n" " run one of:\n" @@ -3482,6 +3484,7 @@ prepare_build(bool print_fingerprint, pins::kFirstRunWinGnu, pins::kFirstRunWinGnuTarget, release)); } if constexpr (mcpp::platform::is_macos || mcpp::platform::is_windows) { + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "no toolchain configured.\n" " run one of:\n" @@ -3490,6 +3493,7 @@ prepare_build(bool print_fingerprint, " {}", pins::kSuggestLlvm, pins::kFirstRunMac, release)); } else { + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "no toolchain configured.\n" " run one of:\n" @@ -4650,7 +4654,19 @@ prepare_build(bool print_fingerprint, needsRemoteUpdate = true; break; } - if (needsRemoteUpdate) { + // A first sync is a refresh of an index that has no local + // copy yet, and `[index] auto_refresh = false` means that no + // refresh happens implicitly (docs/05). The policy's answer is + // taken rather than restated (#648 A5); offline, the sync is a + // no-op as before and resolution reports what is missing. + const auto refreshPolicy = mcpp::pm::policy_for(**cfg2); + if (needsRemoteUpdate && !refreshPolicy.offline && !refreshPolicy.autoRefresh) { + return std::unexpected(std::string( + "the project's custom index repositories have never been synced, " + "and [index] auto_refresh = false forbids syncing them implicitly\n" + " run `mcpp index update` once, then build again")); + } + if (needsRemoteUpdate && !refreshPolicy.offline) { mcpp::ui::status("Fetching", "custom index repos (first use)"); auto projEnv = mcpp::config::make_project_xlings_env(**cfg2, *root); int rc = mcpp::xlings::update_index(projEnv, /*quiet=*/true); @@ -5238,6 +5254,18 @@ prepare_build(bool print_fingerprint, mcpp::pm::staleness_note( mcpp::config::make_xlings_env(**cfgA))); } + // Offline with no local copy of the index at all, the miss says + // nothing about the package: nothing has been downloaded to look + // in. That is a download the run needs, not a wrong selector. + if (mcpp::platform::env::offline_mode()) { + if (auto cfgO = get_cfg(); + cfgO && !mcpp::xlings::default_index_status( + mcpp::config::make_xlings_env(**cfgO), 0).present) { + hint += "\n offline: the package index has never been fetched; " + "run `mcpp index update` without --offline"; + refusal::record(refusal::Code::OfflineDownloadRequired); + } + } return std::unexpected(with_index_cause(std::format( "dependency '{}': no package found for exact selector" "\n tried: {}{}", @@ -5466,6 +5494,7 @@ prepare_build(bool print_fingerprint, // down. (The toolchain payload path has its own gate; this is the // dependency path, which does not go through resolve_xpkg_path.) if (mcpp::platform::env::offline_mode()) { + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "offline mode: dependency '{}' v{} is not installed and " "cannot be downloaded\n" @@ -7311,6 +7340,7 @@ prepare_build(bool print_fingerprint, auto refuse_offline = [&](std::string_view need, std::string_view why, std::string_view verb) { + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "offline mode: git dependency '{}' needs {} of '{}'\n" " {}\n" diff --git a/src/build/refusal.cppm b/src/build/refusal.cppm index e5d2f8d0..30fc9c50 100644 --- a/src/build/refusal.cppm +++ b/src/build/refusal.cppm @@ -105,6 +105,13 @@ enum class Code { // Distinct from LayerOrdering, which is about the layers not stacking: here // they stack, and one image would link with no C++ runtime at all (#641). SharedLibraryCxxRuntime, + // The plan needs something that has to be downloaded -- a toolchain, a + // package, a git revision, the package index -- and the run is offline + // (`--offline`, `MCPP_OFFLINE`, or the older `MCPP_NO_AUTO_INSTALL`). + // Distinct from every other code: nothing is wrong with the project, and + // one online run fixes it. An editor that plans offline by default tells + // the two apart by this code (#648). + OfflineDownloadRequired, Other, // a refusal that has not been given a code yet }; @@ -137,6 +144,8 @@ constexpr std::string_view name(Code c) { case Code::ToolVersionConflict: return "tool-version-conflict"; case Code::SharedLibraryCxxRuntime: return "shared-library-cxx-runtime"; + case Code::OfflineDownloadRequired: + return "offline-download-required"; case Code::Other: return "other"; } return "other"; diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index d7df3682..c3d544bf 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -21,6 +21,7 @@ import mcpp.build.schedule.detach_codegen; import mcpp.build.test_targets; import mcpp.build.build_database; import mcpp.build.build_program; +import mcpp.build.refusal; // offline-download-required (#648 A1) import mcpp.dyndep; import mcpp.home; import mcpp.hooks; @@ -372,7 +373,20 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) testDiscovery.push_back(std::move(discovery)); } } - if (planError) return failed("MCPP_BUILD_DATABASE_PLAN_FAILED", *planError); + // An offline plan that needs a download is not a defect of the project, and + // a client that plans offline by default (an editor) has to tell the two + // apart without reading the message (#648 A1). The code is taken only while + // the run is offline, so a refusal recorded on a path that recovered cannot + // relabel an unrelated failure. + if (planError) { + const bool offline = mcpp::platform::env::offline_mode() + || mcpp::platform::env::no_auto_install(); + if (mcpp::build::refusal::take() + == mcpp::build::refusal::Code::OfflineDownloadRequired + && offline) + return failed("MCPP_OFFLINE_DOWNLOAD_REQUIRED", *planError); + return failed("MCPP_BUILD_DATABASE_PLAN_FAILED", *planError); + } // The lock this planning produced, against the project's. The project's is // never written; a difference is reported. diff --git a/src/pm/index_refresh.cppm b/src/pm/index_refresh.cppm index ff51e8b2..e3c2efc8 100644 --- a/src/pm/index_refresh.cppm +++ b/src/pm/index_refresh.cppm @@ -44,48 +44,13 @@ import mcpp.pm.dep_spec; import mcpp.pm.dependency_selector; // legacy_bare_candidates import mcpp.pm.index_contract; import mcpp.pm.index_route; +export import mcpp.pm.refresh_policy; // the policy half, shared with the fetcher import mcpp.pm.resolver; import mcpp.ui; import mcpp.xlings; export namespace mcpp::pm { -// Why a refresh was, or was not, triggered. Every value is user-visible under -// `-v`, and the Suppressed* ones are the interesting half when diagnosing -// "why did/didn't mcpp go to the network". -enum class RefreshReason { - None, // resolvable locally — the steady state, zero network - IndexAbsent, // no local index at all (cold start) - DescriptorMiss, // package unknown locally, and the index can say so - VersionMiss, // package known, constraint unsatisfiable locally - SuppressedOffline, // --offline / MCPP_OFFLINE - SuppressedDisabled, // [index] auto_refresh = false - SuppressedDebounce, // refreshed moments ago; upstream simply lacks it - SuppressedInconclusive, // a miss here proves nothing (see header) - SuppressedIndexUnusable,// the index that would answer is too new to read - SuppressedMalformedDescriptor, // local bytes exist but violate identity -}; - -struct RefreshPolicy { - bool offline = false; - bool autoRefresh = true; - // Shared with the xim install gate — one constant, one rationale, in the - // leaf module both layers can see (mcpp::xlings). - std::int64_t debounceSeconds = mcpp::xlings::kIndexRefreshDebounceSeconds; -}; - -struct RefreshDecision { - bool shouldRefresh = false; - RefreshReason reason = RefreshReason::None; - std::string subject; // "mcpplibs:fmt@^1.3" — for logs and errors -}; - -// Human tail for the status line: "package index — ". -std::string_view reason_text(RefreshReason r); - -// flag > env > config. `offlineFlag` is the parsed `--offline`. -RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg); - // Pure: no network, no filesystem writes, no side effects. The judgement table // is the contract — `tests/unit/test_pm_index_refresh.cpp` locks it row by row. RefreshDecision decide_for_dependency(const IndexRoute& route, @@ -100,39 +65,12 @@ RefreshDecision decide_for_dependency(const IndexRoute& route, // nothing for the others, so callers use this to skip the sync entirely. bool routes_to_builtin(const IndexRoute& route, const DependencySpec& spec); -// For a caller that has ALREADY established a conclusive miss through -// `lookup_descriptor` (that is `mcpp add`): only the policy half of the -// judgement is left to make. Keeps the opt-outs in one place rather than -// re-tested at each call site. -RefreshDecision decide_for_miss(const RefreshPolicy& policy, - const mcpp::xlings::Env& env, - std::string_view subject); - -// Run the sync a decision asked for. At most ONE per process: a build whose -// deps miss for the same reason should pay one sync, not one per dep. -// Returns an error only when the sync itself failed; callers decide whether -// that is fatal (it is not, if the build can still resolve locally). -std::expected apply(const RefreshDecision& d, - const mcpp::xlings::Env& env); - -// Explicit user intent (`mcpp update`): ignores debounce and the once-per- -// process guard. Still refuses when offline, and says so. -std::expected force_refresh(const mcpp::xlings::Env& env); - -// One line of advisory context for a resolution failure: what the index is and -// how old it is. Never a gate — only ever appended to an error the user is -// already seeing. -std::string staleness_note(const mcpp::xlings::Env& env); - } // namespace mcpp::pm namespace mcpp::pm { namespace { -// One sync per process, however many dependencies ask for one. -bool g_refreshed_this_process = false; - // Echo back what the USER wrote (the `[dependencies]` key), not the resolved // coordinate. `xim.nasm` is parsed into the candidate `(mcpplibs.xim, nasm)`, // and reporting that spelling in a diagnostic sends the reader looking for a @@ -170,41 +108,8 @@ std::vector coords_of(std::string_view depKey, } }; } -std::string age_phrase(std::int64_t s) { - if (s < 0) return "never refreshed"; - if (s < 90) return std::format("refreshed {}s ago", s); - if (s < 5400) return std::format("refreshed {}m ago", s / 60); - if (s < 172800) return std::format("refreshed {}h ago", s / 3600); - return std::format("refreshed {}d ago", s / 86400); -} - } // namespace -std::string_view reason_text(RefreshReason r) { - switch (r) { - case RefreshReason::None: return "resolvable locally"; - case RefreshReason::IndexAbsent: return "no local package index"; - case RefreshReason::DescriptorMiss: return "not found locally"; - case RefreshReason::VersionMiss: return "not satisfiable locally"; - case RefreshReason::SuppressedOffline: return "offline mode"; - case RefreshReason::SuppressedDisabled: return "[index] auto_refresh = false"; - case RefreshReason::SuppressedDebounce: return "index was just refreshed"; - case RefreshReason::SuppressedInconclusive: return "no index can refute this"; - case RefreshReason::SuppressedIndexUnusable: - return "an index requires a newer mcpp — refreshing cannot help"; - case RefreshReason::SuppressedMalformedDescriptor: - return "descriptor identity is malformed — refreshing cannot help"; - } - return ""; -} - -RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg) { - RefreshPolicy p; - p.offline = mcpp::platform::env::offline_mode(); - p.autoRefresh = cfg.indexAutoRefresh; - return p; -} - RefreshDecision decide_for_dependency(const IndexRoute& route, std::string_view depKey, const DependencySpec& spec, @@ -317,84 +222,4 @@ bool routes_to_builtin(const IndexRoute& route, const DependencySpec& spec) { return coords_route_to_builtin(route, coords_of(spec.shortName, spec)); } -RefreshDecision decide_for_miss(const RefreshPolicy& policy, - const mcpp::xlings::Env& env, - std::string_view subject) { - RefreshDecision d; - d.subject = std::string(subject); - // See decide(): a refresh cannot fix an index this binary cannot read. - if (mcpp::pm::any_index_unusable()) { - d.reason = RefreshReason::SuppressedIndexUnusable; - return d; - } - if (policy.offline) { d.reason = RefreshReason::SuppressedOffline; return d; } - if (!policy.autoRefresh) { d.reason = RefreshReason::SuppressedDisabled; return d; } - // Same debounce as the build path: two `mcpp add` typos in a row should not - // buy two multi-repo syncs, for the same reason a build with two missing - // packages does not. - auto status = mcpp::xlings::default_index_status(env, policy.debounceSeconds); - if (status.present && status.ageSeconds >= 0 - && status.ageSeconds < policy.debounceSeconds) { - d.reason = RefreshReason::SuppressedDebounce; - return d; - } - d.reason = RefreshReason::DescriptorMiss; - d.shouldRefresh = true; - return d; -} - -namespace { - -std::expected run_sync(const mcpp::xlings::Env& env, - std::string_view banner) { - auto before = mcpp::xlings::default_index_status(env, 0).rev; - mcpp::ui::status("Refreshing", banner); - int rc = mcpp::xlings::update_index(env, /*quiet=*/true); - if (rc != 0) - return std::unexpected(std::format("package index refresh failed (rc {})", rc)); - auto after = mcpp::xlings::default_index_status(env, 0).rev; - if (before && after && *before != *after) - mcpp::ui::status("Updated", std::format("package index {} → {}", *before, *after)); - else if (after) - mcpp::log::verbose("index", std::format("package index still at {}", *after)); - return {}; -} - -} // namespace - -std::expected apply(const RefreshDecision& d, - const mcpp::xlings::Env& env) -{ - if (!d.shouldRefresh) { - mcpp::log::verbose("index", std::format( - "skip refresh for {}: {}", d.subject, reason_text(d.reason))); - return {}; - } - if (g_refreshed_this_process) { - mcpp::log::verbose("index", std::format( - "skip refresh for {}: already refreshed in this run", d.subject)); - return {}; - } - g_refreshed_this_process = true; - return run_sync(env, std::format("package index — `{}` {} (one-time)", - d.subject, reason_text(d.reason))); -} - -std::expected force_refresh(const mcpp::xlings::Env& env) { - if (mcpp::platform::env::offline_mode()) - return std::unexpected( - "offline mode is on — cannot refresh the package index " - "(unset MCPP_OFFLINE or drop --offline)"); - g_refreshed_this_process = true; - return run_sync(env, "package index (requested)"); -} - -std::string staleness_note(const mcpp::xlings::Env& env) { - auto st = mcpp::xlings::default_index_status(env, 0); - if (!st.present) return "no local package index"; - return st.rev - ? std::format("local index {} ({})", *st.rev, age_phrase(st.ageSeconds)) - : std::format("local index {}", age_phrase(st.ageSeconds)); -} - } // namespace mcpp::pm diff --git a/src/pm/package_fetcher.cppm b/src/pm/package_fetcher.cppm index 7cee61e2..e6286e62 100644 --- a/src/pm/package_fetcher.cppm +++ b/src/pm/package_fetcher.cppm @@ -20,6 +20,8 @@ import mcpp.manifest; // xpkg_lua_identity_matches — descriptor identit import mcpp.pm.compat; import mcpp.pm.dep_spec; import mcpp.pm.index_contract; // index.toml floor check (E0006) +import mcpp.pm.refresh_policy; // decide_for_miss / apply (#648 A5) +import mcpp.build.refusal; // offline-download-required (#648 A1) import mcpp.ui; import mcpp.pm.index_spec; import mcpp.xlings; @@ -1112,6 +1114,8 @@ Fetcher::resolve_xpkg_path(std::string_view target, // download attempt is refused, and it names the package so the user // knows what to fetch rather than seeing a generic network error. if (mcpp::platform::env::offline_mode()) { + mcpp::build::refusal::record( + mcpp::build::refusal::Code::OfflineDownloadRequired); return std::unexpected(CallError{ std::format( "offline mode: `{}@{}` is not installed and cannot be downloaded\n" " run without --offline (or unset MCPP_OFFLINE) to fetch it", @@ -1119,13 +1123,18 @@ Fetcher::resolve_xpkg_path(std::string_view target, } if (parsed.indexName == "xim") { mcpp::xlings::Env xlEnv{ cfg_.xlingsBinary, cfg_.xlingsHome() }; - // quiet=false: this only ever prints when a dependency is missing - // from the local index and we refresh once to fetch it — a rare, - // intentional event worth surfacing so a one-time network pause - // doesn't look like a silent hang. Steady-state builds (deps - // present) return early without a word. - mcpp::xlings::ensure_official_package_index_fresh( - xlEnv, parsed.packageName, cfg_.searchTtlSeconds, /*quiet=*/false); + // A descriptor missing from the local xim index may be one the + // remote index has. Whether that miss refreshes is the refresh + // policy's decision, so `[index] auto_refresh = false` and the + // debounce hold here as on every other path (#648 A5). The status + // line `apply` prints makes the one-time network pause visible. + if (!mcpp::xlings::official_package_present(xlEnv, parsed.packageName)) { + auto decision = mcpp::pm::decide_for_miss( + mcpp::pm::policy_for(cfg_), xlEnv, + std::format("xim:{}", parsed.packageName)); + if (auto r = mcpp::pm::apply(decision, xlEnv); !r) + mcpp::ui::warning(r.error()); + } } std::vector targets { @@ -1200,9 +1209,12 @@ Fetcher::resolve_xpkg_path(std::string_view target, if (parsed.indexName == "xim") { mcpp::xlings::Env refreshEnv{ cfg_.xlingsBinary, cfg_.xlingsHome() }; mcpp::log::verbose("fetcher", - std::format("install failed for {}; refreshing index before retry", + std::format("install failed for {}; asking the refresh policy before retry", targets[0])); - mcpp::xlings::update_index(refreshEnv, /*quiet=*/true); + auto decision = mcpp::pm::decide_for_miss( + mcpp::pm::policy_for(cfg_), refreshEnv, targets[0]); + if (auto r = mcpp::pm::apply(decision, refreshEnv); !r) + mcpp::log::verbose("fetcher", r.error()); } mcpp::log::verbose("fetcher", diff --git a/src/pm/refresh_policy.cppm b/src/pm/refresh_policy.cppm new file mode 100644 index 00000000..00b06c9f --- /dev/null +++ b/src/pm/refresh_policy.cppm @@ -0,0 +1,212 @@ +// mcpp.pm.refresh_policy -- the policy half of "may this run refresh a package +// index now?", in a module the package fetcher can import. +// +// WHY THIS IS NOT PART OF mcpp.pm.index_refresh. That module answers the whole +// question for a dependency, and to do so it reads descriptors through +// mcpp.pm.index_route, which imports the fetcher. The fetcher could therefore +// not ask it, and it asked a second derivation instead: +// `ensure_official_package_index_fresh` before an install, and a bare +// `update_index` before a retry. Neither consulted `[index] auto_refresh`, +// which docs/05 documents as "never refresh the index automatically" +// (mcpp-community/mcpp#648 A5). The opt-outs, the debounce, the one-sync-per- +// process guard and the sync itself live here, and every caller that refreshes +// an index goes through `decide_for_miss` or `decide_for_dependency` and then +// `apply`. + +export module mcpp.pm.refresh_policy; + +import std; +import mcpp.config; +import mcpp.log; +import mcpp.platform; +import mcpp.pm.index_contract; +import mcpp.ui; +import mcpp.xlings; + +export namespace mcpp::pm { + +// Why a refresh was, or was not, triggered. Every value is user-visible under +// `-v`, and the Suppressed* ones are the interesting half when diagnosing +// "why did/didn't mcpp go to the network". +enum class RefreshReason { + None, // resolvable locally — the steady state, zero network + IndexAbsent, // no local index at all (cold start) + DescriptorMiss, // package unknown locally, and the index can say so + VersionMiss, // package known, constraint unsatisfiable locally + SuppressedOffline, // --offline / MCPP_OFFLINE + SuppressedDisabled, // [index] auto_refresh = false + SuppressedDebounce, // refreshed moments ago; upstream simply lacks it + SuppressedInconclusive, // a miss here proves nothing (see header) + SuppressedIndexUnusable,// the index that would answer is too new to read + SuppressedMalformedDescriptor, // local bytes exist but violate identity +}; + +struct RefreshPolicy { + bool offline = false; + bool autoRefresh = true; + // Shared with the xim install gate — one constant, one rationale, in the + // leaf module both layers can see (mcpp::xlings). + std::int64_t debounceSeconds = mcpp::xlings::kIndexRefreshDebounceSeconds; +}; + +struct RefreshDecision { + bool shouldRefresh = false; + RefreshReason reason = RefreshReason::None; + std::string subject; // "mcpplibs:fmt@^1.3" — for logs and errors +}; + +// Human tail for the status line: "package index — ". +std::string_view reason_text(RefreshReason r); + +// flag > env > config. `offlineFlag` is the parsed `--offline`. +RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg); + +// For a caller that has ALREADY established a conclusive miss through +// `lookup_descriptor` (that is `mcpp add`): only the policy half of the +// judgement is left to make. Keeps the opt-outs in one place rather than +// re-tested at each call site. +RefreshDecision decide_for_miss(const RefreshPolicy& policy, + const mcpp::xlings::Env& env, + std::string_view subject); + +// Run the sync a decision asked for. At most ONE per process: a build whose +// deps miss for the same reason should pay one sync, not one per dep. +// Returns an error only when the sync itself failed; callers decide whether +// that is fatal (it is not, if the build can still resolve locally). +std::expected apply(const RefreshDecision& d, + const mcpp::xlings::Env& env); + +// Explicit user intent (`mcpp update`): ignores debounce and the once-per- +// process guard. Still refuses when offline, and says so. +std::expected force_refresh(const mcpp::xlings::Env& env); + +// One line of advisory context for a resolution failure: what the index is and +// how old it is. Never a gate — only ever appended to an error the user is +// already seeing. +std::string staleness_note(const mcpp::xlings::Env& env); + +} // namespace mcpp::pm + +namespace mcpp::pm { + +namespace { + +// One sync per process, however many dependencies ask for one. +bool g_refreshed_this_process = false; + +std::string age_phrase(std::int64_t s) { + if (s < 0) return "never refreshed"; + if (s < 90) return std::format("refreshed {}s ago", s); + if (s < 5400) return std::format("refreshed {}m ago", s / 60); + if (s < 172800) return std::format("refreshed {}h ago", s / 3600); + return std::format("refreshed {}d ago", s / 86400); +} + +} // namespace + +std::string_view reason_text(RefreshReason r) { + switch (r) { + case RefreshReason::None: return "resolvable locally"; + case RefreshReason::IndexAbsent: return "no local package index"; + case RefreshReason::DescriptorMiss: return "not found locally"; + case RefreshReason::VersionMiss: return "not satisfiable locally"; + case RefreshReason::SuppressedOffline: return "offline mode"; + case RefreshReason::SuppressedDisabled: return "[index] auto_refresh = false"; + case RefreshReason::SuppressedDebounce: return "index was just refreshed"; + case RefreshReason::SuppressedInconclusive: return "no index can refute this"; + case RefreshReason::SuppressedIndexUnusable: + return "an index requires a newer mcpp — refreshing cannot help"; + case RefreshReason::SuppressedMalformedDescriptor: + return "descriptor identity is malformed — refreshing cannot help"; + } + return ""; +} + +RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg) { + RefreshPolicy p; + p.offline = mcpp::platform::env::offline_mode(); + p.autoRefresh = cfg.indexAutoRefresh; + return p; +} + +RefreshDecision decide_for_miss(const RefreshPolicy& policy, + const mcpp::xlings::Env& env, + std::string_view subject) { + RefreshDecision d; + d.subject = std::string(subject); + // See decide(): a refresh cannot fix an index this binary cannot read. + if (mcpp::pm::any_index_unusable()) { + d.reason = RefreshReason::SuppressedIndexUnusable; + return d; + } + if (policy.offline) { d.reason = RefreshReason::SuppressedOffline; return d; } + if (!policy.autoRefresh) { d.reason = RefreshReason::SuppressedDisabled; return d; } + // Same debounce as the build path: two `mcpp add` typos in a row should not + // buy two multi-repo syncs, for the same reason a build with two missing + // packages does not. + auto status = mcpp::xlings::default_index_status(env, policy.debounceSeconds); + if (status.present && status.ageSeconds >= 0 + && status.ageSeconds < policy.debounceSeconds) { + d.reason = RefreshReason::SuppressedDebounce; + return d; + } + d.reason = RefreshReason::DescriptorMiss; + d.shouldRefresh = true; + return d; +} + +namespace { + +std::expected run_sync(const mcpp::xlings::Env& env, + std::string_view banner) { + auto before = mcpp::xlings::default_index_status(env, 0).rev; + mcpp::ui::status("Refreshing", banner); + int rc = mcpp::xlings::update_index(env, /*quiet=*/true); + if (rc != 0) + return std::unexpected(std::format("package index refresh failed (rc {})", rc)); + auto after = mcpp::xlings::default_index_status(env, 0).rev; + if (before && after && *before != *after) + mcpp::ui::status("Updated", std::format("package index {} → {}", *before, *after)); + else if (after) + mcpp::log::verbose("index", std::format("package index still at {}", *after)); + return {}; +} + +} // namespace + +std::expected apply(const RefreshDecision& d, + const mcpp::xlings::Env& env) +{ + if (!d.shouldRefresh) { + mcpp::log::verbose("index", std::format( + "skip refresh for {}: {}", d.subject, reason_text(d.reason))); + return {}; + } + if (g_refreshed_this_process) { + mcpp::log::verbose("index", std::format( + "skip refresh for {}: already refreshed in this run", d.subject)); + return {}; + } + g_refreshed_this_process = true; + return run_sync(env, std::format("package index — `{}` {} (one-time)", + d.subject, reason_text(d.reason))); +} + +std::expected force_refresh(const mcpp::xlings::Env& env) { + if (mcpp::platform::env::offline_mode()) + return std::unexpected( + "offline mode is on — cannot refresh the package index " + "(unset MCPP_OFFLINE or drop --offline)"); + g_refreshed_this_process = true; + return run_sync(env, "package index (requested)"); +} + +std::string staleness_note(const mcpp::xlings::Env& env) { + auto st = mcpp::xlings::default_index_status(env, 0); + if (!st.present) return "no local package index"; + return st.rev + ? std::format("local index {} ({})", *st.rev, age_phrase(st.ageSeconds)) + : std::format("local index {}", age_phrase(st.ageSeconds)); +} + +} // namespace mcpp::pm diff --git a/src/xlings/xlings.cppm b/src/xlings/xlings.cppm index 20eca1e7..663dbd42 100644 --- a/src/xlings/xlings.cppm +++ b/src/xlings/xlings.cppm @@ -547,14 +547,12 @@ int update_index(const Env& env, bool quiet = false); // when no update is needed. void ensure_index_fresh(const Env& env, std::int64_t ttlSeconds, bool quiet = false); -// Ensure xlings' official xim index is present and fresh. -void ensure_official_index_fresh(const Env& env, std::int64_t ttlSeconds, bool quiet = false); - -// Ensure a specific package file exists in xlings' official xim index. -void ensure_official_package_index_fresh(const Env& env, - std::string_view packageName, - std::int64_t ttlSeconds, - bool quiet = false); +// Whether xlings' official xim index on disk carries a descriptor for +// `packageName`. Offline and read-only. Whether a miss may refresh the index is +// not decided here: that is mcpp.pm.refresh_policy's `decide_for_miss`, so +// `[index] auto_refresh`, `--offline` and the debounce apply to it as they do +// to every other refresh (mcpp-community/mcpp#648 A5). +bool official_package_present(const Env& env, std::string_view packageName); // ─── Index status (read-only, offline) ────────────────────────────── // Snapshot of a local index directory — computed without touching the @@ -1978,43 +1976,10 @@ void ensure_index_fresh(const Env& env, std::int64_t ttlSeconds, bool quiet) { update_index(env, /*quiet=*/true); } -void ensure_official_index_fresh(const Env& env, std::int64_t ttlSeconds, bool quiet) { - if (is_official_index_fresh(env, ttlSeconds)) return; - if (!quiet) - print_status("Updating", "package index (auto-refresh)"); - update_index(env, /*quiet=*/true); -} - -void ensure_official_package_index_fresh(const Env& env, - std::string_view packageName, - [[maybe_unused]] std::int64_t ttlSeconds, - bool quiet) { - // Offline-first, miss-triggered. We do NOT auto-update just because a TTL - // expired — that runs a network `xlings update` (git-syncs several index - // repos) that stalls for minutes on slow/blocked networks (the Termux - // first-run / build hang). But fully offline is too strict: if a requested - // dependency is NOT in the local index, we DO refresh once to discover it. - // - // present locally → use as-is, zero network (the common build case). - // missing locally → refresh once to try to fetch it. - // - // Routine, deps-already-present refresh stays the user's explicit - // `mcpp index update` / `xlings update`. +bool official_package_present(const Env& env, std::string_view packageName) { auto pkg = official_package_file(env, packageName); - if (!pkg.empty() && std::filesystem::exists(pkg)) return; - - // The package is missing locally. Refresh once — but guard against a build - // that resolves several genuinely-absent packages re-running the heavy - // `xlings update` per package: if the index was refreshed moments ago and - // the package is STILL missing, upstream simply lacks it; re-pulling won't - // help. (A package added upstream before this run lands in that one pull.) - if (is_official_index_fresh(env, kIndexRefreshDebounceSeconds)) return; - - if (!quiet) - print_status("Refreshing", - std::format("package index — `{}` not found locally (one-time)", - packageName)); - update_index(env, /*quiet=*/quiet); + std::error_code ec; + return !pkg.empty() && std::filesystem::exists(pkg, ec); } } // namespace mcpp::xlings From d69ca44e88dcbf651e483252415e9db9318accc5 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:15:28 +0800 Subject: [PATCH 04/20] Symbol provision: GNU_UNIQUE is vague linkage, and a definition both images take from one object is not a conflict (#646 F3) --- src/build/runtime_validation.cppm | 104 +++++++++++++++++++++++++++ src/build/symbol_provision.cppm | 68 +++++++++++++++--- src/runtime/elf.cppm | 103 ++++++++++++++++++++++++-- tests/unit/test_symbol_provision.cpp | 43 +++++++++++ 4 files changed, 303 insertions(+), 15 deletions(-) diff --git a/src/build/runtime_validation.cppm b/src/build/runtime_validation.cppm index e7871a43..d96d2015 100644 --- a/src/build/runtime_validation.cppm +++ b/src/build/runtime_validation.cppm @@ -955,6 +955,98 @@ check_symbol_provision(const mcpp::build::BuildPlan& plan, return closureCache.emplace(object, std::move(names)).first->second; }; + // The objects a link unit links, as absolute paths. The staged `std` + // module objects are added to every unit: the emitter appends them to + // each C++ image that imports `std` and records them nowhere else, and an + // image that did not link them cannot define the names they define, so + // listing them for every unit attributes nothing that is not there. + auto objects_of = [&](const mcpp::build::LinkUnit& lu) { + std::set out; + for (auto const& o : lu.objects) + out.insert((o.is_absolute() ? o : plan.outputDir / o).lexically_normal()); + for (auto name : {"std.o", "std.compat.o"}) + out.insert((plan.outputDir / "obj" / name).lexically_normal()); + return out; + }; + std::map> objectSymbolCache; + auto object_defines = [&](const std::filesystem::path& object) + -> const std::vector& { + auto it = objectSymbolCache.find(object); + if (it != objectSymbolCache.end()) return it->second; + std::vector names; + std::error_code ec; + if (std::filesystem::exists(object, ec)) + if (auto symbols = mcpp::platform::elf::defined_object_symbols(object)) + names = std::move(*symbols); + return objectSymbolCache.emplace(object, std::move(names)).first->second; + }; + // The installation the toolchain's compiler belongs to, canonical, for + // the second rule below: `/bin/`, whose C++ runtime + // library and module source both live under ``. + std::filesystem::path toolchainRoot; + { + std::error_code ec; + auto compiler = std::filesystem::weakly_canonical(plan.toolchain.binaryPath, ec); + if (!ec && compiler.has_parent_path()) + toolchainRoot = compiler.parent_path().parent_path(); + } + auto in_toolchain = [&](const std::filesystem::path& file) { + if (toolchainRoot.empty()) return false; + std::error_code ec; + auto canonical = std::filesystem::weakly_canonical(file, ec); + if (ec) return false; + auto rel = canonical.lexically_relative(toolchainRoot); + return !rel.empty() && *rel.begin() != ".."; + }; + // For each provider in `resolution` that is a shared library of this + // plan: the names defined by objects linked into both it and `image`. + // + // AND ONE PROVIDER THAT IS NOT A LINK UNIT: the toolchain's own C++ + // runtime. The staged `std` module object is compiled from the module + // source that runtime ships, and GCC 16's `libstdc++.so.6` exports the + // same initialisers (`_ZGIW3std@@GLIBCXX_3.4.35`, measured), so a program + // coupled to that runtime defines them twice by construction. The names + // excused are the ones the std objects define, and only against a library + // that belongs to the toolchain's own installation. + auto shared_plan_definitions = + [&](const mcpp::build::LinkUnit& image, + const mcpp::platform::elf::RuntimeResolution& resolution) { + std::map> out; + const auto imageObjects = objects_of(image); + for (auto const& object : resolution.objects) { + if (in_toolchain(object.artifact)) { + auto& names = out[object.artifact.string()]; + for (auto name : {"std.o", "std.compat.o"}) + for (auto const& defined : + object_defines((plan.outputDir / "obj" / name).lexically_normal())) + names.insert(defined); + continue; + } + const mcpp::build::LinkUnit* provider = nullptr; + for (auto const& lu : plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::SharedLibrary) continue; + std::vector names{plan.outputDir / lu.output}; + for (auto const& alias : lu.runtimeAliases) + names.push_back(plan.outputDir / alias); + for (auto const& candidate : names) { + std::error_code ec; + if (std::filesystem::equivalent(candidate, object.artifact, ec)) { + provider = &lu; + break; + } + } + if (provider) break; + } + if (!provider) continue; + auto& names = out[object.artifact.string()]; + for (auto const& o : objects_of(*provider)) { + if (!imageObjects.contains(o)) continue; + for (auto const& name : object_defines(o)) names.insert(name); + } + } + return out; + }; + for (auto const& [artifact, oldStamp] : before) { auto now = stamp(artifact); if (!now.exists) continue; @@ -1079,6 +1171,18 @@ check_symbol_provision(const mcpp::build::BuildPlan& plan, for (auto const& conflict : all) if (conflict.isWeak) ++report.sharedWeak; std::erase_if(all, [](auto const& c) { return c.isWeak; }); + // WHAT THIS BUILD PUT INTO BOTH IMAGES ON PURPOSE (#646 F3). + // + // A provider that is itself a link unit of this plan may define a + // name from an object this image links as well. The `std` module's + // object is linked into every C++ image that imports `std`, so its + // empty initialiser is defined twice in a program over a C++ shared + // library by construction; reporting it made `--strict` unusable for + // that shape. Attributed by the objects both units link, never by + // the shape of the name. + if (unit && !all.empty()) + report.sharedPlanDefinitions = sp::drop_shared_plan_definitions( + all, shared_plan_definitions(*unit, resolution)); report.conflicts = std::move(all); report.status = report.conflicts.empty() ? sp::Status::Clean : sp::Status::Conflict; diff --git a/src/build/symbol_provision.cppm b/src/build/symbol_provision.cppm index c5c6bdf3..4cf48da4 100644 --- a/src/build/symbol_provision.cppm +++ b/src/build/symbol_provision.cppm @@ -33,14 +33,22 @@ // mcpp binary (measured: 0 of 217 dynamic symbols). Stage two runs only when // stage one is not, and it is the stage that makes the report TRUE. // -// STAGE TWO IS NOT OPTIONAL, and the reason is mcpp's own doing. A -// `kind = "shared"` dependency's link unit receives only ITS OWN objects -// (mcpp.build.plan), so a static package underneath it lands in the CONSUMER's -// executable instead, and the shared library binds back to it at run time. -// That is the shape above — arranged by mcpp, with exactly one copy of the -// code in the process, and completely benign. Reporting stage one alone would -// warn about a correct build that the user cannot do anything about, which is -// precisely the noise `mcpp.build.distribution` refuses to emit. +// STAGE TWO IS NOT OPTIONAL. An executable's dynamic symbol table also holds +// what the linker exported because a shared object references it, and one of +// those shapes is mcpp's own: a static package that several images reach is +// linked into the program, and the shared libraries bind back to it at run +// time. Reporting stage one alone would name that as a finding. +// +// THE ARRANGEMENT IS NOT BENIGN, AND IT IS NO LONGER THE DEFAULT (#646 F1). +// This comment used to call it completely benign, and that held only on ELF +// and only for a program that links the package: the same library refuses +// `-Wl,-z,defs`, a host that did not link the package cannot `dlopen` it, +// Mach-O and PE resolve at link time, and Android loads the library first. A +// static package reachable from ONE shared image is now linked into that image +// (mcpp.build.plan, `place_static_packages`). One reachable from several is +// refused where it cannot work and reported by prepare on ELF, where the +// build still binds to the program's copy; this check then reports what the +// linker exported, which is a true statement about that build. // // Design: .agents/docs/2026-08-28-issue519-dependency-linkage-form.md §2. @@ -106,6 +114,11 @@ struct Report { // to be told which ones this check decided about, or "clean" reads as // "did not look". std::size_t sharedWeak = 0; + // Definitions this image and a library it loads both take from ONE object + // of this build, counted and not listed. The module initialiser in + // `std.o` is the case that exists: every C++ image that imports `std` + // links it, and the function is empty. See `drop_shared_plan_definitions`. + std::size_t sharedPlanDefinitions = 0; // Why, for the two non-answers. Empty for Clean and Conflict. std::string reason; @@ -134,6 +147,22 @@ exported_definitions(const mcpp::platform::elf::DynamicSymbols& symbols); std::vector conflicting_exports(std::span exports, std::span closure); +// Remove what this build put into both images on purpose. +// +// `sharedDefinitions` maps a provider's label to the names defined by the +// objects this build links into BOTH the image being checked and that +// provider. A provider is removed from a conflict whose name it defines only +// through such an object, a conflict left with no provider is dropped, and the +// number of removed (symbol, provider) pairs is returned. +// +// PROVENANCE, NOT A NAME PATTERN. The case this exists for is the `std` +// module's initialiser (`_ZGIW3std`), which every C++ image importing `std` +// links from the same `std.o`; matching `_ZGIW` instead would also excuse a +// real duplicate whose name happens to have that shape (#646 F3). +std::size_t drop_shared_plan_definitions( + std::vector& conflicts, + const std::map>& sharedDefinitions); + // The report an image with no dynamic symbol table gets, and the one an image // whose author asked for exports gets. Named constructors rather than raw // struct literals so every non-answer carries its reason. @@ -206,6 +235,23 @@ std::vector conflicting_exports(std::span exports, return out; } +std::size_t drop_shared_plan_definitions( + std::vector& conflicts, + const std::map>& sharedDefinitions) { + std::size_t removed = 0; + for (auto& conflict : conflicts) { + std::erase_if(conflict.alsoProvidedBy, [&](const std::string& label) { + auto it = sharedDefinitions.find(label); + const bool shared = it != sharedDefinitions.end() + && it->second.contains(conflict.name); + if (shared) ++removed; + return shared; + }); + } + std::erase_if(conflicts, [](const Conflict& c) { return c.alsoProvidedBy.empty(); }); + return removed; +} + Report not_applicable(std::string reason) { return Report{ .status = Status::NotApplicable, .reason = std::move(reason) }; } @@ -246,6 +292,12 @@ std::string Report::explain(std::string_view artifact) const { " finding: the C++ ABI emits one per image and the loader keeps one.)\n", sharedWeak, sharedWeak == 1 ? "" : "s", sharedWeak == 1 ? "is" : "are"); + if (sharedPlanDefinitions > 0) + body += std::format( + " ({} definition{} this build links into both images from one object\n" + " {} NOT part of this finding either.)\n", + sharedPlanDefinitions, sharedPlanDefinitions == 1 ? "" : "s", + sharedPlanDefinitions == 1 ? "is" : "are"); // WHY it matters, then what to do — IN THE ORDER THAT ACTUALLY WORKS. // diff --git a/src/runtime/elf.cppm b/src/runtime/elf.cppm index 8deaef93..e8b4a064 100644 --- a/src/runtime/elf.cppm +++ b/src/runtime/elf.cppm @@ -137,16 +137,38 @@ struct DynamicSymbol { // dynamic symbol table therefore has exactly one cause — the linker // exported it so that some shared object's reference would bind to it. bool isFunc = false; - // STB_WEAK. A vague-linkage definition -- a template instantiation, an - // inline function, a vtable -- which the C++ ABI emits into every - // translation unit that needs it and expects the loader to unify across - // the process. That is the intended behaviour, not a leak, so a caller - // asking "is this image providing something twice" has to be able to tell - // it from a strong definition that displaces a library's own. + // A vague-linkage definition -- a template instantiation, an inline + // function, a vtable, or the static data member of such a template -- + // which the C++ ABI emits into every translation unit that needs it and + // expects the loader to unify across the process. That is the intended + // behaviour, not a leak, so a caller asking "is this image providing + // something twice" has to be able to tell it from a strong definition that + // displaces a library's own. + // + // TWO BINDINGS SAY IT, NOT ONE. `STB_WEAK` is the portable spelling; + // `STB_GNU_UNIQUE` is what GCC emits for the static data of an inline + // entity so that the loader keeps exactly one copy even across RTLD_LOCAL + // (libstdc++'s `__from_chars_alnum_to_val_table::value`, measured). + // Reading only the first reported seven such objects as a program + // displacing libstdc++ (#646 F3). See `is_vague_linkage_binding`. bool isWeak = false; std::uint64_t value = 0; // st_value; the key a copy relocation matches }; +// Is a symbol of this binding a vague-linkage definition the loader unifies? +// `STB_WEAK` (2) and `STB_GNU_UNIQUE` (10); every other binding is either +// strong (`STB_GLOBAL`) or not visible to the loader. +constexpr bool is_vague_linkage_binding(unsigned char binding) { + return binding == 2 || binding == 10; +} + +// The names an ELF RELOCATABLE object (`.o`) defines with external linkage: +// GLOBAL, WEAK and GNU_UNIQUE entries of `.symtab` whose section index is not +// SHN_UNDEF. Used to attribute a definition to the object that contributed it, +// which a linked image no longer records. +std::expected, std::string> +defined_object_symbols(const std::filesystem::path& object); + // What an ELF object's DYNAMIC symbol table says it provides. // // Deliberately not folded into ElfRuntimeFacts: every caller of @@ -739,7 +761,7 @@ inspect_dynamic_symbols(const std::filesystem::path& object) { out.defined.push_back(DynamicSymbol{ .name = std::move(*name), .isFunc = (type == detail::kSttFunc || type == detail::kSttGnuIfunc), - .isWeak = (bind == detail::kStbWeak), + .isWeak = is_vague_linkage_binding(bind), .value = *value, }); } @@ -770,6 +792,73 @@ inspect_dynamic_symbols(const std::filesystem::path& object) { return out; } +std::expected, std::string> +defined_object_symbols(const std::filesystem::path& object) { + detail::Reader reader; + std::ifstream input(object, std::ios::binary); + if (!input) return std::unexpected(std::format( + "cannot open ELF object '{}'", object.string())); + reader.bytes.assign(std::istreambuf_iterator(input), {}); + + if (reader.bytes.size() < 0x40 + || reader.bytes[0] != 0x7f || reader.bytes[1] != 'E' + || reader.bytes[2] != 'L' || reader.bytes[3] != 'F') + return std::unexpected(std::format( + "object '{}' is not ELF", object.string())); + if (reader.bytes[4] != 2 || reader.bytes[5] != 1) + return std::unexpected(std::format( + "object '{}' is not ELF64 little-endian", object.string())); + + // Section headers: e_shoff at 0x28, e_shentsize at 0x3A, e_shnum at 0x3C. + auto shoff = reader.u64(0x28); + auto shentsize = reader.u16(0x3A); + auto shnum = reader.u16(0x3C); + if (!shoff || !shentsize || !shnum || *shentsize < 0x40 || *shnum > 65000 + || !reader.range(*shoff, static_cast(*shentsize) * *shnum)) + return std::unexpected(std::format( + "object '{}' has a truncated section table", object.string())); + + constexpr std::uint32_t kShtSymtab = 2; + std::vector names; + for (std::uint16_t i = 0; i < *shnum; ++i) { + const auto at = *shoff + static_cast(i) * *shentsize; + auto type = reader.u32(at + 0x04); + if (!type || *type != kShtSymtab) continue; + auto offset = reader.u64(at + 0x18); + auto size = reader.u64(at + 0x20); + auto link = reader.u32(at + 0x28); + auto entsize = reader.u64(at + 0x38); + if (!offset || !size || !link || !entsize + || *entsize != detail::kSymEntrySize || *link >= *shnum + || !reader.range(*offset, *size)) + return std::unexpected(std::format( + "object '{}' has a malformed symbol table", object.string())); + const auto strAt = *shoff + static_cast(*link) * *shentsize; + auto strOffset = reader.u64(strAt + 0x18); + auto strSize = reader.u64(strAt + 0x20); + if (!strOffset || !strSize || !reader.range(*strOffset, *strSize)) + return std::unexpected(std::format( + "object '{}' has a malformed string table", object.string())); + for (std::uint64_t off = 0; off + detail::kSymEntrySize <= *size; + off += detail::kSymEntrySize) { + const auto sym = *offset + off; + auto nameOffset = reader.u32(sym); + auto shndx = reader.u16(sym + 6); + if (!nameOffset || !shndx) break; + if (*shndx == 0) continue; // SHN_UNDEF + const unsigned char bind = + static_cast(reader.bytes[sym + 4] >> 4); + if (bind == detail::kStbLocal) continue; + if (*nameOffset >= *strSize) continue; + auto name = reader.cstr(*strOffset + *nameOffset, *strSize - *nameOffset); + if (name && !name->empty()) names.push_back(std::move(*name)); + } + } + std::ranges::sort(names); + names.erase(std::ranges::unique(names).begin(), names.end()); + return names; +} + std::expected inspect_elf_runtime(const std::filesystem::path& artifact) { detail::Reader reader; diff --git a/tests/unit/test_symbol_provision.cpp b/tests/unit/test_symbol_provision.cpp index eefd60bd..a0db4eeb 100644 --- a/tests/unit/test_symbol_provision.cpp +++ b/tests/unit/test_symbol_provision.cpp @@ -250,3 +250,46 @@ TEST(SymbolProvision, AConflictRemembersWhetherItsDefinitionIsWeak) { EXPECT_TRUE(conflicts[0].isWeak); EXPECT_FALSE(conflicts[1].isWeak); } + +// #646 F3. GCC emits the static data of an inline entity with STB_GNU_UNIQUE +// (10) so that the loader keeps one copy across RTLD_LOCAL. It is vague linkage +// exactly as STB_WEAK is; reading only the latter reported seven libstdc++ +// objects as a program displacing the library. +TEST(SymbolProvision, TheUniqueBindingIsVagueLinkageAsTheWeakOneIs) { + EXPECT_TRUE(elf::is_vague_linkage_binding(2)); // STB_WEAK + EXPECT_TRUE(elf::is_vague_linkage_binding(10)); // STB_GNU_UNIQUE + EXPECT_FALSE(elf::is_vague_linkage_binding(1)); // STB_GLOBAL + EXPECT_FALSE(elf::is_vague_linkage_binding(0)); // STB_LOCAL +} + +// #646 F3. The std module's initialiser is linked into every C++ image that +// imports `std`, from one object. A provider that defines a name only through +// an object this build linked into both images is removed from that conflict. +TEST(SymbolProvision, ADefinitionBothImagesTakeFromOneObjectIsNotAConflict) { + std::vector conflicts{ + { .name = "_ZGIW3std", .isFunc = true, + .alsoProvidedBy = {"bin/liblib.so"} }, + { .name = "inflate", .isFunc = true, + .alsoProvidedBy = {"bin/liblib.so", "/usr/lib/libz.so.1"} }, + }; + const std::map> shared{ + { "bin/liblib.so", {"_ZGIW3std"} }, + }; + EXPECT_EQ(sp::drop_shared_plan_definitions(conflicts, shared), 1u); + ASSERT_EQ(conflicts.size(), 1u); + EXPECT_EQ(conflicts[0].name, "inflate"); + // The library still defines `inflate` from its OWN object, so it stays a + // provider: attribution is by object, never by the library as a whole. + EXPECT_EQ(conflicts[0].alsoProvidedBy, + (std::vector{"bin/liblib.so", "/usr/lib/libz.so.1"})); +} + +// The rule is provenance, not a name pattern: an initialiser-shaped name that +// no shared object defines is still a finding. +TEST(SymbolProvision, AnInitialiserShapedNameWithoutSharedProvenanceIsStillReported) { + std::vector conflicts{ + { .name = "_ZGIW5other", .isFunc = true, .alsoProvidedBy = {"bin/libother.so"} }, + }; + EXPECT_EQ(sp::drop_shared_plan_definitions(conflicts, {}), 0u); + EXPECT_EQ(conflicts.size(), 1u); +} From 199d47884f18034567665c32527d4cb22c95a63f Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:15:28 +0800 Subject: [PATCH 05/20] One process, one C++ runtime; one static package, one image; the llvm row's MSVC runtime record (#646 F1, F3a; #649 E10) --- src/build/distribution.cppm | 116 ++++++++++++++++ src/build/flags.cppm | 67 +++++----- src/build/plan.cppm | 193 +++++++++++++++++++++++++++ src/build/prepare.cppm | 123 ++++++++++++++++- src/build/refusal.cppm | 16 +++ tests/unit/test_distribution.cpp | 117 ++++++++++++++++ tests/unit/test_static_placement.cpp | 110 +++++++++++++++ 7 files changed, 708 insertions(+), 34 deletions(-) create mode 100644 tests/unit/test_static_placement.cpp diff --git a/src/build/distribution.cppm b/src/build/distribution.cppm index fc094c7b..6819ded9 100644 --- a/src/build/distribution.cppm +++ b/src/build/distribution.cppm @@ -250,6 +250,44 @@ Contract default_contract(Role r, Format f) { return Contract::SelfContained; } +// What a manifest states about the C++ runtime, read once for every role. +struct ContractStatement { + std::string_view cxxRuntime; // `cxx_runtime = "..."` or its `default` + std::string_view cxxRuntimeTests; // `cxx_runtime = { tests = "..." }` + std::string_view cxxRuntimeShared; // `cxx_runtime = { shared = "..." }` + bool staticStdlib = true; +}; + +// Which images of a build load a C++ shared library the build itself makes: +// its programs, and its test programs. +struct CxxSharedLoad { + bool program = false; + bool tests = false; +}; + +// The contract each role holds, and whether a human stated it. +// +// A stated contract is what the manifest said; an unstated one is a default, +// and a default is the only thing `role_contracts` is allowed to move. +struct RoleContracts { + Contract program = Contract::SelfContained; + Contract tests = Contract::SelfContained; + Contract intermediate = Contract::SelfContained; + Contract shared = Contract::SelfContained; + bool programStated = false; + bool testsStated = false; + bool sharedStated = false; +}; + +// THE ONE DERIVATION of every role's contract, for the flag assembly and for +// the refusal that reads the same answer before anything compiles. +RoleContracts role_contracts(const ContractStatement& s, Format f, CxxSharedLoad load); + +// The role whose STATED contract splits the process's C++ runtime: on ELF, a +// self-contained program or test that loads a C++ shared library coupled to a +// shared runtime. Empty when no role does. +std::optional runtime_split(const RoleContracts& c, Format f, CxxSharedLoad load); + // The contract a manifest STATES for shared libraries, or nothing when it // states none and the role's default applies. // @@ -269,6 +307,57 @@ std::optional stated_shared_library_contract(std::string_view cxxRunti staticStdlib ? default_contract(Role::Distributable, f) : Contract::HostCoupled); } +RoleContracts role_contracts(const ContractStatement& s, Format f, CxxSharedLoad load) { + RoleContracts c; + c.programStated = !s.cxxRuntime.empty() || !s.staticStdlib; + c.program = parse_contract(s.cxxRuntime).value_or( + s.staticStdlib ? default_contract(Role::Distributable, f) : Contract::HostCoupled); + c.intermediate = c.program; + c.testsStated = c.programStated || !s.cxxRuntimeTests.empty(); + c.tests = parse_contract(s.cxxRuntimeTests).value_or(c.program); + c.sharedStated = c.programStated || !s.cxxRuntimeShared.empty(); + c.shared = stated_shared_library_contract(s.cxxRuntime, s.cxxRuntimeShared, + s.staticStdlib, f) + .value_or(default_contract(Role::SharedLibrary, f)); + + // ONE PROCESS, ONE C++ RUNTIME (#646 F3a). + // + // The ELF defaults are right one at a time and wrong together. A program + // is self-contained and a shared library couples to the toolchain's + // runtime, so a program that LOADS such a library holds a static runtime + // and a shared one. The executable exports the runtime symbols the library + // references, the library binds some of them to the program's copy and + // keeps the rest, and the two halves disagree about shared state. + // Measured on Linux x86_64 with llvm@22.1.8: the program aborted with + // `std::bad_cast` (exit 134) as soon as the library formatted a string; + // with gcc@16.1.0 it ran with 900 libstdc++ symbols interposed. + // + // So a role nobody stated takes the shared library's contract when its + // image loads a C++ shared library this build makes. The process already + // needs that runtime through the library's own NEEDED entry, so no + // deployment gains a requirement; what changes is that the program binds + // to the same copy. A stated contract is never changed here; one that + // splits the runtime is refused by the caller (`runtime_split`). + if (f == Format::Elf) { + if (load.program && !c.programStated) c.program = c.shared; + if (load.tests && !c.testsStated) c.tests = c.shared; + } + return c; +} + +std::optional runtime_split(const RoleContracts& c, Format f, CxxSharedLoad load) { + if (f != Format::Elf) return std::nullopt; + // A shared library that embeds a hidden private copy keeps its runtime to + // itself (`hide_static_cxx_runtime`); that is the documented private-copy + // arrangement, not a split. A coupled library shares the process's. + if (c.shared == Contract::SelfContained) return std::nullopt; + if (load.program && c.programStated && c.program == Contract::SelfContained) + return Role::Distributable; + if (load.tests && c.testsStated && c.tests == Contract::SelfContained) + return Role::Test; + return std::nullopt; +} + // ---------------------------------------------------------------- Layer 3 struct MechanismInput { @@ -294,6 +383,13 @@ struct MechanismInput { // decoration. Derived by `msvc_wants_static_crt`, which is also what // emits the flag. bool msvcStaticCrt = false; + // MSVC STL only: does mcpp pass a CRT model (`/MT` or `/MD`) to this + // compiler? True for cl.exe. FALSE FOR CLANG ON THE MSVC ABI: that driver + // speaks the GNU dialect, mcpp emits no runtime flag for it, and clang + // then links the static CRT (`-defaultlib:libcmt`, measured with the + // 22.1.8 driver). The table must report the model the compiler was given, + // not the model `cl.exe` would have been given (#649 E10). + bool msvcCrtModelEmitted = true; // Toolchain capability id: "libstdc++", "libc++", or an MSVC STL spelling. std::string_view stdlibId; Format format = Format::Elf; @@ -625,6 +721,26 @@ Mechanism resolve(const MechanismInput& in) { // // No unit flags: the model is a COMPILE flag on every TU, not // something added to the link line. + // + // CLANG ON THE MSVC ABI IS GIVEN NO MODEL, so the table records + // the one its driver chooses. The rows below were written for + // cl.exe, and for this row they recorded `host-coupled` beside an + // artifact that imports no vcruntime DLL at all (#649 E10). The + // artifact is left as it is; the record, and an explicit request + // the row does not deliver, now say what it is. + if (!in.msvcCrtModelEmitted) { + m.effective = Contract::SelfContained; + if (in.requested != Contract::SelfContained && in.explicitRequest) { + m.degraded = true; + m.diagnostic = std::format( + "cxx_runtime = \"{}\" is not delivered for clang on the " + "MSVC ABI: mcpp passes this driver no CRT model, and clang " + "links the static CRT (libcmt) by default. Use msvc@system " + "for the dynamic CRT; using self-contained", + to_string(in.requested)); + } + return m; + } m.effective = in.msvcStaticCrt ? Contract::SelfContained : Contract::HostCoupled; if (in.requested == Contract::SelfContained && !in.msvcStaticCrt) { diff --git a/src/build/flags.cppm b/src/build/flags.cppm index ca7e9750..c5fc1547 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -1057,28 +1057,27 @@ CompileFlags compute_flags(const BuildPlan& plan) { // its documented meaning has always been exactly self-contained vs the // dynamic system runtime. An explicit `cxx_runtime` wins. // - // The role defaults come from `dist::default_contract` rather than - // being spelled again here. They were spelled again here, and that - // second derivation is why `default_contract` sat with no caller while - // this file quietly disagreed with it about shared libraries. - const dist::Contract base = - dist::parse_contract(bc.cxxRuntime).value_or( - bc.staticStdlib - ? dist::default_contract(dist::Role::Distributable, format) - : dist::Contract::HostCoupled); - const dist::Contract testsContract = - dist::parse_contract(bc.cxxRuntimeTests).value_or(base); - // A project-wide statement (`cxx_runtime = "…"` or `static_stdlib = - // false`) applies to shared libraries too — a human said what the - // whole project promises. Only when nobody said anything does the - // role's own default apply, which is the case that changes on ELF. - // The statement is read by `stated_shared_library_contract`, which the - // refusal of a C++ shared library over a graph runtime reads too (#641). - const bool projectWideExplicit = !bc.cxxRuntime.empty() || !bc.staticStdlib; - const dist::Contract sharedContract = - dist::stated_shared_library_contract(bc.cxxRuntime, bc.cxxRuntimeShared, - bc.staticStdlib, format) - .value_or(dist::default_contract(dist::Role::SharedLibrary, format)); + // Every role's contract comes from `dist::role_contracts`, the one + // derivation the refusal of a split runtime reads as well. The role + // defaults were spelled here once, and that second derivation is why + // `default_contract` sat with no caller while this file quietly + // disagreed with it about shared libraries. A project-wide statement + // (`cxx_runtime = "..."` or `static_stdlib = false`) applies to shared + // libraries too; only when nobody said anything does a role's own + // default apply, and on ELF a program or test that loads a C++ shared + // library of this build takes that library's contract (#646 F3a). + const dist::CxxSharedLoad cxxSharedLoad{ + .program = mcpp::build::image_loads_cxx_shared_library(plan, LinkUnit::Binary), + .tests = mcpp::build::image_loads_cxx_shared_library(plan, LinkUnit::TestBinary), + }; + const dist::RoleContracts contracts = dist::role_contracts( + dist::ContractStatement{ + .cxxRuntime = bc.cxxRuntime, + .cxxRuntimeTests = bc.cxxRuntimeTests, + .cxxRuntimeShared = bc.cxxRuntimeShared, + .staticStdlib = bc.staticStdlib, + }, + format, cxxSharedLoad); // Archive lookup, directory half. LLVM lays these out either directly // under lib/ (the macOS packages) or under lib// (the @@ -1134,6 +1133,9 @@ CompileFlags compute_flags(const BuildPlan& plan) { // ignoring a role override. mi.msvcStaticCrt = mcpp::toolchain::msvc_wants_static_crt( bc.linkage, bc.cxxRuntime); + // The model is emitted only for the `msvc` dialect (above); clang on + // the MSVC ABI receives none and links the static CRT (#649 E10). + mi.msvcCrtModelEmitted = isMsvcDialect; mi.mingw = isMingwTc; mi.macosFloor = !macosDeploymentTarget.empty(); // READ from the one value prepare resolved. The SDK being located for @@ -1233,9 +1235,10 @@ CompileFlags compute_flags(const BuildPlan& plan) { }; const bool wantsArchives = - (base == dist::Contract::SelfContained - || testsContract == dist::Contract::SelfContained - || sharedContract == dist::Contract::SelfContained) + (contracts.program == dist::Contract::SelfContained + || contracts.tests == dist::Contract::SelfContained + || contracts.intermediate == dist::Contract::SelfContained + || contracts.shared == dist::Contract::SelfContained) && caps.stdlib_id == "libc++"; // THE ARCHIVES A LINKER SCRIPT OPENS, by file name. // @@ -1317,9 +1320,9 @@ CompileFlags compute_flags(const BuildPlan& plan) { // "Explicit" = a human wrote it down. `static_stdlib = false` counts: // nobody sets a flag to its default to get non-default behavior. - const bool explicitBase = projectWideExplicit; - const bool explicitTests = explicitBase || !bc.cxxRuntimeTests.empty(); - const bool explicitShared = explicitBase || !bc.cxxRuntimeShared.empty(); + const bool explicitBase = contracts.programStated; + const bool explicitTests = contracts.testsStated; + const bool explicitShared = contracts.sharedStated; // Report a role's degradation only if this build HAS that role. // @@ -1337,10 +1340,10 @@ CompileFlags compute_flags(const BuildPlan& plan) { bool wantsToolchainRuntime = false; for (auto [role, requested, wasAsked] : { - std::tuple{dist::Role::Distributable, base, explicitBase}, - std::tuple{dist::Role::Test, testsContract, explicitTests}, - std::tuple{dist::Role::Intermediate, base, explicitBase}, - std::tuple{dist::Role::SharedLibrary, sharedContract, explicitShared}}) { + std::tuple{dist::Role::Distributable, contracts.program, explicitBase}, + std::tuple{dist::Role::Test, contracts.tests, explicitTests}, + std::tuple{dist::Role::Intermediate, contracts.intermediate, explicitBase}, + std::tuple{dist::Role::SharedLibrary, contracts.shared, explicitShared}}) { mi.role = role; mi.requested = requested; mi.explicitRequest = wasAsked; diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 94d9844d..43263072 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -178,6 +178,46 @@ struct ResolvedRuntimeContract { ResolvedRuntimeContract resolve_runtime_contract( const std::vector& packages); +// A static package this build could not link into exactly one image (#646 F1). +// +// `images` names the shared libraries of the plan whose static closure +// reaches the package (by target name); `program` says whether the root's own +// images reach it as well. `make_plan` leaves such a package where it always +// was, in the root's images, and records it here: whether that is refused or +// reported depends on the target format, which prepare decides. +struct StaticPlacementConflict { + std::string package; // qualified package name + std::vector images; // shared library target names + bool program = false; +}; + +// Where each static package of a graph is linked. +// +// A shared image's STATIC CLOSURE is the set of static packages reachable from +// its package without crossing another shared package. A static package in +// exactly one closure, which the root does not reach itself, belongs in that +// image; one in several closures, or in one and the root's, has no single +// image to live in (`conflicts`). Packages in no closure stay with the root. +// +// Indices are package indices; 0 is the root. `sharedImages` maps each shared +// package to the number of images it produces; `boundaries` are packages that +// are neither placed nor traversed (a distribution package's library is +// already built). +struct StaticPlacement { + std::map intoImage; // static package -> shared package + struct Conflict { + std::size_t package = 0; + std::vector images; // shared packages reaching it + bool root = false; + }; + std::vector conflicts; +}; + +StaticPlacement place_static_packages( + const std::map>& edges, + const std::map& sharedImages, + const std::set& boundaries); + struct BuildPlan { mcpp::manifest::Manifest manifest; mcpp::toolchain::Toolchain toolchain; @@ -215,6 +255,10 @@ struct BuildPlan { // the failure is `relocation R_X86_64_32S ... can not be used when making // a shared object` on a file nobody edited. bool needsPic = false; + // Static packages that no single image could take (#646 F1); see + // `StaticPlacementConflict`. Empty for every graph without a shared + // library over a static package that something else reaches too. + std::vector staticPlacementConflicts; std::string scheduleTag = "none"; // Whether `--accel` / `--no-accel` selected this graph's device variant // over `[build] accel`. The variant is in the fingerprint, so the two @@ -428,6 +472,14 @@ package_link_objects(const BuildPlan& plan, std::string_view packageName); // library without a C++ runtime reads the same answer (#641). bool link_unit_holds_cxx(const BuildPlan& plan, const LinkUnit& lu); +// Does an image of `kind` load a shared library of this plan that holds C++, +// directly or through another shared library of the plan? Answered from the +// link edges the plan declares (a consumer lists each shared library it links +// among its implicit inputs), so it is the process the build will produce and +// not a guess about it. The C++ runtime contract of programs and tests reads +// this (#646 F3a). +bool image_loads_cxx_shared_library(const BuildPlan& plan, LinkUnit::Kind kind); + } // namespace mcpp::build namespace mcpp::build { @@ -453,6 +505,69 @@ bool link_unit_holds_cxx(const BuildPlan& plan, const LinkUnit& lu) { }); } +StaticPlacement place_static_packages( + const std::map>& edges, + const std::map& sharedImages, + const std::set& boundaries) { + auto closure = [&](std::size_t start) { + std::set out; + std::vector stack{start}; + while (!stack.empty()) { + const auto at = stack.back(); + stack.pop_back(); + auto it = edges.find(at); + if (it == edges.end()) continue; + for (auto next : it->second) { + if (next == 0 || next == start) continue; + if (sharedImages.contains(next) || boundaries.contains(next)) continue; + if (out.insert(next).second) stack.push_back(next); + } + } + return out; + }; + const auto rootClosure = closure(0); + std::map> owners; + for (auto const& [shared, count] : sharedImages) { + (void)count; + for (auto s : closure(shared)) owners[s].push_back(shared); + } + StaticPlacement out; + for (auto const& [s, images] : owners) { + std::size_t imageCount = 0; + for (auto p : images) imageCount += sharedImages.at(p); + const bool root = rootClosure.contains(s); + if (!root && images.size() == 1 && imageCount == 1) + out.intoImage.emplace(s, images.front()); + else + out.conflicts.push_back({ .package = s, .images = images, .root = root }); + } + return out; +} + +bool image_loads_cxx_shared_library(const BuildPlan& plan, LinkUnit::Kind kind) { + std::map sharedByOutput; + for (auto const& lu : plan.linkUnits) + if (lu.kind == LinkUnit::SharedLibrary) + sharedByOutput.emplace(lu.output.lexically_normal().generic_string(), &lu); + if (sharedByOutput.empty()) return false; + for (auto const& lu : plan.linkUnits) { + if (lu.kind != kind) continue; + std::vector stack{&lu}; + std::set seen; + while (!stack.empty()) { + const LinkUnit* at = stack.back(); + stack.pop_back(); + for (auto const& input : at->implicitInputs) { + auto it = sharedByOutput.find(input.lexically_normal().generic_string()); + if (it == sharedByOutput.end() || !seen.insert(it->second).second) continue; + if (link_unit_holds_cxx(plan, *it->second)) return true; + stack.push_back(it->second); + } + } + } + return false; +} + namespace { std::string sanitize_for_path(std::string_view module_name) { @@ -1714,6 +1829,10 @@ make_plan(const mcpp::manifest::Manifest& manifest, std::set sharedDepPackages; std::map> sharedTargetsByPackage; std::map> packageIndexByName; + // #646 F1: static packages linked into a dependency's shared image instead + // of the root's, by name and by the shared package that takes them. + std::set placedInImage; + std::map> staticsByImagePackage; for (std::size_t i = 0; i < packages.size(); ++i) { auto const& p = packages[i]; packageIndexByName[qualified_package_name(p.manifest)] = i; @@ -1827,6 +1946,69 @@ make_plan(const mcpp::manifest::Manifest& manifest, } } + // WHERE A STATIC PACKAGE UNDER A SHARED LIBRARY IS LINKED (#646 F1). + // + // A dependency's shared library used to be linked from its own package's + // objects only, and every static package went into the root's images. A + // static package reachable only through the library then lived in the + // program, and the library bound to the program's copy at run time. ELF + // allows that, and only for a consumer that linked the package itself: + // measured on Linux, the same library refused `-Wl,-z,defs`, and a + // program that did not link the package could not `dlopen` it + // (`undefined symbol: x_answer`). Mach-O and PE resolve at link time, and + // Android's Java host loads the library before the application. + // + // So a static package in exactly one shared image's closure, which the + // root does not reach itself, is linked into that image. One that several + // images reach has no single home and is recorded; prepare refuses it + // where the link or load cannot succeed and reports it on ELF, where the + // build proceeds as it always did. The root's edges include its + // dev-dependencies: a test program that links a package directly keeps it. + { + auto placementEdges = directPackageDeps; + for (auto const& [depName, spec] : manifest.devDependencies) { + for (auto const& candidate : dependency_name_candidates(depName, spec)) { + auto it = packageIndexByName.find(candidate); + if (it == packageIndexByName.end() || it->second == 0) continue; + auto& deps = placementEdges[0]; + if (std::ranges::find(deps, it->second) == deps.end()) + deps.push_back(it->second); + break; + } + } + std::map sharedImages; + for (auto const& [i, targets] : sharedTargetsByPackage) + sharedImages[i] = targets.size(); + // Two kinds of package are neither placed nor walked through. A + // distribution package's library is already built. A package that + // provides a target layer (a C library, a compiler runtime, a C++ + // runtime: `provides = ["mcpp:..."]`) is beneath every image, and + // where its objects go is the runtime contract's decision, which + // #641 made (`cxx_runtime = { shared = "self-contained" }`). + std::set boundaries; + for (std::size_t i = 1; i < packages.size(); ++i) { + auto const& pm = packages[i].manifest; + const bool layer = std::ranges::any_of(pm.provides, [](const std::string& p) { + return p.starts_with("mcpp:") || p == "hosted-standard-library"; + }); + if (layer || mcpp::pack::is_distribution_package(pm)) boundaries.insert(i); + } + auto placement = place_static_packages(placementEdges, sharedImages, boundaries); + for (auto const& [staticIndex, sharedIndex] : placement.intoImage) { + placedInImage.insert(qualified_package_name(packages[staticIndex].manifest)); + staticsByImagePackage[sharedIndex].push_back(staticIndex); + } + for (auto const& conflict : placement.conflicts) { + StaticPlacementConflict c; + c.package = qualified_package_name(packages[conflict.package].manifest); + c.program = conflict.root; + for (auto sharedIndex : conflict.images) + for (auto targetIndex : sharedTargetsByPackage[sharedIndex]) + c.images.push_back(sharedDepTargets[targetIndex].target.name); + plan.staticPlacementConflicts.push_back(std::move(c)); + } + } + auto append_direct_shared_deps = [&](LinkUnit& lu, std::size_t packageIndex) { auto depsIt = directPackageDeps.find(packageIndex); if (depsIt == directPackageDeps.end()) return; @@ -1886,6 +2068,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, linkedPackages.insert(0); for (auto& cu : plan.compileUnits) { if (sharedDepPackages.contains(cu.packageName)) continue; + if (placedInImage.contains(cu.packageName)) continue; auto it = packageIndexByName.find(cu.packageName); if (it == packageIndexByName.end()) continue; linkedPackages.insert(it->second); @@ -1927,6 +2110,14 @@ make_plan(const mcpp::manifest::Manifest& manifest, lu.loaderTagFlag = loader_tag_flag(lu.kind); append_package_objects(lu, dep.packageName); append_direct_shared_deps(lu, dep.packageIndex); + if (auto it = staticsByImagePackage.find(dep.packageIndex); + it != staticsByImagePackage.end()) { + for (auto staticIndex : it->second) { + append_package_objects( + lu, qualified_package_name(packages[staticIndex].manifest)); + append_direct_shared_deps(lu, staticIndex); + } + } plan.linkUnits.push_back(std::move(lu)); } @@ -1993,6 +2184,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, // For binary target, also include main.cpp's object if main is present. for (auto& cu : plan.compileUnits) { if (sharedDepPackages.contains(cu.packageName)) continue; + if (placedInImage.contains(cu.packageName)) continue; if (mcpp::links_unconditionally(cu.kind)) { lu.objects.push_back(cu.object); } @@ -2107,6 +2299,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, // is exclusive to that binary). for (auto& cu : plan.compileUnits) { if (sharedDepPackages.contains(cu.packageName)) continue; + if (placedInImage.contains(cu.packageName)) continue; if (!is_implementation_source(cu.kind)) continue; if (lu.entryMain && cu.source == *lu.entryMain) continue; // own entry: already added above if (entryFilesAcrossTargets.contains(cu.source)) continue; // foreign entry: skip diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index d3888aac..1c84d8b4 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -11589,8 +11589,10 @@ prepare_build(bool print_fingerprint, // construction the fingerprint promised (stdFlagAndDialect above). // #422: the CRT model reaches the std module too. Derived from the // SAME expression the project's TUs use (flags.cppm), through the one - // helper, so the two cannot drift. Non-MSVC dialects yield "" and the - // command is unchanged. + // helper, so the two cannot drift. A GNU dialect yields "-static" or "" + // here, and the gcc and clang std module builders do not read it, so + // their commands are unchanged; clang on the MSVC ABI is given no CRT + // model at all (see `MechanismInput::msvcCrtModelEmitted`). const auto& stdDialect = mcpp::toolchain::dialect_for(*tc); const auto stdCrt = mcpp::toolchain::msvc_crt_flag( stdDialect, mcpp::toolchain::msvc_wants_static_crt( @@ -12030,6 +12032,123 @@ prepare_build(bool print_fingerprint, } } + // ONE PROCESS, ONE C++ RUNTIME; ONE STATIC PACKAGE, ONE IMAGE (#646). + // + // Both are decided by `make_plan` and the contract table; this is where a + // decision that cannot be delivered stops the build before it compiles. + { + namespace dist = mcpp::build::dist; + auto const& bc = ctx.plan.manifest.buildConfig; + const auto format = dist::format_for( + tc->targetTriple, + mcpp::platform::is_windows ? dist::Format::Pe + : mcpp::platform::is_macos ? dist::Format::MachO + : dist::Format::Elf); + const dist::CxxSharedLoad load{ + .program = mcpp::build::image_loads_cxx_shared_library( + ctx.plan, mcpp::build::LinkUnit::Binary), + .tests = mcpp::build::image_loads_cxx_shared_library( + ctx.plan, mcpp::build::LinkUnit::TestBinary), + }; + const auto contracts = dist::role_contracts( + dist::ContractStatement{ + .cxxRuntime = bc.cxxRuntime, + .cxxRuntimeTests = bc.cxxRuntimeTests, + .cxxRuntimeShared = bc.cxxRuntimeShared, + .staticStdlib = bc.staticStdlib, + }, + format, load); + // F3a. A stated self-contained program over a coupled C++ shared + // library of this build: the program would carry a static C++ runtime + // and the library would load a shared one. The unstated case needs no + // refusal, because `role_contracts` then gives the program the + // library's contract. + if (auto role = dist::runtime_split(contracts, format, load)) { + std::string libraries; + for (auto const& lu : ctx.plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::SharedLibrary) continue; + if (!mcpp::build::link_unit_holds_cxx(ctx.plan, lu)) continue; + libraries += (libraries.empty() ? "'" : ", '") + lu.targetName + "'"; + } + const bool tests = *role == dist::Role::Test; + refusal::record(refusal::Code::ProgramCxxRuntimeSplit); + return std::unexpected(std::format( + "this build's {} state a self-contained C++ runtime and load the C++ " + "shared library {}, which is linked against the {} C++ runtime.\n" + " The process would hold two C++ runtimes: the program exports the " + "runtime symbols the\n" + " library references, the library binds some of them there and keeps " + "the rest, and the two\n" + " halves disagree about shared state (measured: a string formatted in " + "the library aborts\n" + " with std::bad_cast).\n" + " Remove the self-contained statement for {} (the `{}` value of " + "[build] cxx_runtime), and\n" + " they take the shared library's contract, or give the shared library " + "a private copy of the\n" + " runtime:\n" + "\n" + " [build]\n" + " cxx_runtime = {{ shared = \"self-contained\" }}", + tests ? "tests" : "programs", + libraries.empty() ? std::string("'(unnamed)'") : libraries, + dist::to_string(contracts.shared), + tests ? "tests" : "programs", tests ? "tests" : "default")); + } + + // F1. A static package that several images reach. Refused where the + // build cannot work (Mach-O and PE resolve every reference at link + // time; Android's Java host loads an application's shared library + // before anything that could supply the package), reported on other + // ELF rows, where the library binds to the program's copy at run time + // as it always has. + if (!ctx.plan.staticPlacementConflicts.empty()) { + const bool applicationRow = std::ranges::any_of(ctx.plan.linkUnits, + [](auto const& lu) { + return lu.kind == mcpp::build::LinkUnit::SharedLibrary + && !lu.dependencyOwned && lu.entryMain.has_value(); + }); + const bool refuse = format == dist::Format::MachO + || format == dist::Format::Pe || applicationRow; + std::string listing; + for (auto const& c : ctx.plan.staticPlacementConflicts) { + std::string reachers; + if (c.program) reachers = "the program"; + for (auto const& image : c.images) + reachers += (reachers.empty() ? "'" : ", '") + image + "'"; + listing += std::format(" '{}' is reached by {}\n", c.package, reachers); + } + const std::string first = ctx.plan.staticPlacementConflicts.front().package; + const std::string remedy = std::format( + " Link the package shared, so that every image loads one copy: on its " + "edge in [dependencies],\n" + "\n" + " {} = {{ ..., linkage = \"shared\" }}\n" + "\n" + " or as the package's own default, in its manifest:\n" + "\n" + " [targets.]\n" + " linkage = \"shared\"", first); + if (refuse) { + refusal::record(refusal::Code::StaticPackageInTwoImages); + return std::unexpected(std::format( + "a static package is linked into more than one image of this build, " + "and on this target\n" + " an image cannot use another image's copy:\n{}{}", + listing, remedy)); + } + mcpp::diag::degraded("build/static-placement", + std::format("a static package is reachable from more than one image of " + "this build and is linked into the program only:\n{}", + listing), + "the shared libraries bind to the program's copy at run time, which only " + "an ELF process whose program links the package can do; the same graph " + "is refused on Mach-O, PE and the Android application row", + std::format("give the package the shared form, e.g. {} = {{ ..., linkage " + "= \"shared\" }}", first)); + } + } + // The module graph outlives the plan for one consumer: `mcpp pack`, which // has to know which units are INTERFACE (published as source) and which // are implementation (published only as an object). The plan flattens that diff --git a/src/build/refusal.cppm b/src/build/refusal.cppm index e5d2f8d0..7954393a 100644 --- a/src/build/refusal.cppm +++ b/src/build/refusal.cppm @@ -105,6 +105,18 @@ enum class Code { // Distinct from LayerOrdering, which is about the layers not stacking: here // they stack, and one image would link with no C++ runtime at all (#641). SharedLibraryCxxRuntime, + // A program or test whose manifest states a self-contained C++ runtime + // loads a C++ shared library of the same build that couples to a shared + // runtime, so the process would hold two C++ runtimes (#646 F3a). Distinct + // from SharedLibraryCxxRuntime, where the library has none: here both + // images have one and they are different copies. + ProgramCxxRuntimeSplit, + // A static package is reachable from two images of one build (two shared + // libraries, or a shared library and the program) on a format whose link + // or load cannot share one copy between them (#646 F1). Distinct from + // ExclusiveCapability, which is about two implementations of an interface: + // here it is one package that has no single image to live in. + StaticPackageInTwoImages, Other, // a refusal that has not been given a code yet }; @@ -137,6 +149,10 @@ constexpr std::string_view name(Code c) { case Code::ToolVersionConflict: return "tool-version-conflict"; case Code::SharedLibraryCxxRuntime: return "shared-library-cxx-runtime"; + case Code::ProgramCxxRuntimeSplit: + return "program-cxx-runtime-split"; + case Code::StaticPackageInTwoImages: + return "static-package-in-two-images"; case Code::Other: return "other"; } return "other"; diff --git a/tests/unit/test_distribution.cpp b/tests/unit/test_distribution.cpp index c4210eb0..34d96f25 100644 --- a/tests/unit/test_distribution.cpp +++ b/tests/unit/test_distribution.cpp @@ -817,3 +817,120 @@ TEST(Distribution, WasmIsSelfContainedByConstructionAndSaysNothing) { // second answer to a question the driver has already answered. EXPECT_TRUE(m.unitFlags.empty()) << m.unitFlags; } + +// --------------------------------------------------------------------------- +// #646 F3a -- one process, one C++ runtime. +// +// The ELF defaults give a program `self-contained` and a shared library +// `toolchain-coupled`. Each is right alone; a program that LOADS such a library +// held two C++ runtimes, and with llvm@22.1.8 it aborted with std::bad_cast. +// An unstated program or test contract takes the shared library's when its +// image loads a C++ shared library of the build. +TEST(Distribution, AProgramThatLoadsACxxSharedLibraryTakesItsContractOnElf) { + const dist::ContractStatement nothing{}; + auto c = dist::role_contracts(nothing, dist::Format::Elf, + {.program = true, .tests = true}); + EXPECT_EQ(c.shared, dist::Contract::ToolchainCoupled); + EXPECT_EQ(c.program, dist::Contract::ToolchainCoupled); + EXPECT_EQ(c.tests, dist::Contract::ToolchainCoupled); + // The intermediate role links no runtime and keeps the program default. + EXPECT_EQ(c.intermediate, dist::Contract::SelfContained); + EXPECT_FALSE(c.programStated); + EXPECT_FALSE(dist::runtime_split(c, dist::Format::Elf, + {.program = true, .tests = true}).has_value()); +} + +// Nothing moves for a program that loads no C++ shared library: the default +// that makes a shipped binary portable is untouched. +TEST(Distribution, AProgramThatLoadsNoCxxSharedLibraryKeepsItsDefault) { + auto c = dist::role_contracts({}, dist::Format::Elf, {}); + EXPECT_EQ(c.program, dist::Contract::SelfContained); + EXPECT_EQ(c.tests, dist::Contract::SelfContained); + auto onlyTests = dist::role_contracts({}, dist::Format::Elf, {.tests = true}); + EXPECT_EQ(onlyTests.program, dist::Contract::SelfContained); + EXPECT_EQ(onlyTests.tests, dist::Contract::ToolchainCoupled); +} + +// Mach-O and PE keep their table: Mach-O's shared default is already +// self-contained (the F2 question is measured separately), and PE resolves +// imports per DLL. +TEST(Distribution, TheFollowingRuleIsElfOnly) { + for (auto f : {dist::Format::MachO, dist::Format::Pe, dist::Format::Wasm}) { + auto c = dist::role_contracts({}, f, {.program = true, .tests = true}); + EXPECT_EQ(c.program, dist::Contract::SelfContained); + EXPECT_FALSE(dist::runtime_split(c, f, {.program = true}).has_value()); + } +} + +// A STATED contract is never moved. A stated self-contained program over a +// coupled C++ shared library is the split, and is named for the caller to +// refuse; a stated self-contained project (the library self-contained too) is +// the documented private-copy arrangement and is not. +TEST(Distribution, AStatedSelfContainedProgramOverACoupledLibraryIsASplit) { + const dist::ContractStatement split{ + .cxxRuntime = "self-contained", .cxxRuntimeShared = "toolchain-coupled"}; + auto c = dist::role_contracts(split, dist::Format::Elf, {.program = true}); + EXPECT_EQ(c.program, dist::Contract::SelfContained); + EXPECT_TRUE(c.programStated); + auto role = dist::runtime_split(c, dist::Format::Elf, {.program = true}); + ASSERT_TRUE(role.has_value()); + EXPECT_EQ(*role, dist::Role::Distributable); + + const dist::ContractStatement whole{.cxxRuntime = "self-contained"}; + auto w = dist::role_contracts(whole, dist::Format::Elf, {.program = true}); + EXPECT_EQ(w.shared, dist::Contract::SelfContained); + EXPECT_FALSE(dist::runtime_split(w, dist::Format::Elf, {.program = true}).has_value()); + + const dist::ContractStatement tests{.cxxRuntimeTests = "self-contained"}; + auto t = dist::role_contracts(tests, dist::Format::Elf, {.tests = true}); + auto testRole = dist::runtime_split(t, dist::Format::Elf, {.tests = true}); + ASSERT_TRUE(testRole.has_value()); + EXPECT_EQ(*testRole, dist::Role::Test); +} + +// `static_stdlib = false` is a statement for every role: all of them are +// host-coupled and nothing splits. +TEST(Distribution, TheOlderSpellingStatesEveryRole) { + const dist::ContractStatement host{.staticStdlib = false}; + auto c = dist::role_contracts(host, dist::Format::Elf, {.program = true, .tests = true}); + EXPECT_EQ(c.program, dist::Contract::HostCoupled); + EXPECT_EQ(c.shared, dist::Contract::HostCoupled); + EXPECT_FALSE(dist::runtime_split(c, dist::Format::Elf, {.program = true}).has_value()); +} + +// --------------------------------------------------------------------------- +// #649 E10 -- clang on the MSVC ABI is given no CRT model, and its driver links +// the static CRT (`-defaultlib:libcmt`). The table recorded `host-coupled` +// beside an artifact that imports no vcruntime DLL. It now records what the +// row delivers, and an explicit request the row does not deliver says so. +TEST(Distribution, ClangOnTheMsvcAbiRecordsTheStaticCrtItsDriverLinks) { + dist::MechanismInput in; + in.format = dist::Format::Pe; + in.stdlibId = "msvc"; + in.msvcCrtModelEmitted = false; + + for (auto requested : {dist::Contract::SelfContained, + dist::Contract::HostCoupled, + dist::Contract::ToolchainCoupled}) { + in.requested = requested; + in.explicitRequest = false; + auto quiet = dist::resolve(in); + EXPECT_EQ(quiet.effective, dist::Contract::SelfContained); + EXPECT_FALSE(quiet.degraded); + EXPECT_FALSE(quiet.deployToolchainRuntime); + EXPECT_TRUE(quiet.unitFlags.empty()); + } + + in.explicitRequest = true; + in.requested = dist::Contract::HostCoupled; + auto undelivered = dist::resolve(in); + EXPECT_EQ(undelivered.effective, dist::Contract::SelfContained); + EXPECT_TRUE(undelivered.degraded); + EXPECT_NE(undelivered.diagnostic.find("not delivered"), std::string::npos) + << undelivered.diagnostic; + EXPECT_NE(undelivered.diagnostic.find("libcmt"), std::string::npos) + << undelivered.diagnostic; + + in.requested = dist::Contract::SelfContained; + EXPECT_FALSE(dist::resolve(in).degraded); +} diff --git a/tests/unit/test_static_placement.cpp b/tests/unit/test_static_placement.cpp new file mode 100644 index 00000000..a457f8b1 --- /dev/null +++ b/tests/unit/test_static_placement.cpp @@ -0,0 +1,110 @@ +// #646 F1 -- where a static package under a shared library is linked. +// +// A dependency's shared library used to take only its own package's objects, +// so a static package beneath it went into the program and the library bound +// to the program's copy at run time. That works on ELF alone, and only for a +// program that links the package. `place_static_packages` decides the image +// each static package belongs in; these are the graph shapes the plan record +// enumerates (§1.2 of the implementation plan), asserted before any link line +// depends on them. +// +// Package 0 is the root. `shared` maps each shared package to its image count. + +#include + +import std; +import mcpp.build.plan; + +using mcpp::build::place_static_packages; + +namespace { + +using Edges = std::map>; + +} // namespace + +// root -> fw (shared) -> x: x belongs in fw. +TEST(StaticPlacement, AStaticPackageUnderOneSharedLibraryGoesIntoIt) { + Edges edges{{0, {1}}, {1, {2}}}; + auto p = place_static_packages(edges, {{1, 1}}, {}); + ASSERT_EQ(p.intoImage.size(), 1u); + EXPECT_EQ(p.intoImage.at(2), 1u); + EXPECT_TRUE(p.conflicts.empty()); +} + +// root -> fw -> a -> x, fw -> x: the whole static closure goes into fw. +TEST(StaticPlacement, ADiamondInsideOneImageStaysInThatImage) { + Edges edges{{0, {1}}, {1, {2, 3}}, {2, {3}}}; + auto p = place_static_packages(edges, {{1, 1}}, {}); + EXPECT_EQ(p.intoImage.size(), 2u); + EXPECT_EQ(p.intoImage.at(2), 1u); + EXPECT_EQ(p.intoImage.at(3), 1u); + EXPECT_TRUE(p.conflicts.empty()); +} + +// root -> fw1 -> x, root -> fw2 -> x: x has no single image. +TEST(StaticPlacement, TwoImagesReachingOnePackageIsAConflict) { + Edges edges{{0, {1, 2}}, {1, {3}}, {2, {3}}}; + auto p = place_static_packages(edges, {{1, 1}, {2, 1}}, {}); + EXPECT_TRUE(p.intoImage.empty()); + ASSERT_EQ(p.conflicts.size(), 1u); + EXPECT_EQ(p.conflicts[0].package, 3u); + EXPECT_EQ(p.conflicts[0].images, (std::vector{1, 2})); + EXPECT_FALSE(p.conflicts[0].root); +} + +// root -> fw -> x and root -> x: the program and the library both need it. +TEST(StaticPlacement, ThePackageTheRootAlsoLinksIsAConflictNamingTheRoot) { + Edges edges{{0, {1, 2}}, {1, {2}}}; + auto p = place_static_packages(edges, {{1, 1}}, {}); + EXPECT_TRUE(p.intoImage.empty()); + ASSERT_EQ(p.conflicts.size(), 1u); + EXPECT_EQ(p.conflicts[0].package, 2u); + EXPECT_TRUE(p.conflicts[0].root); +} + +// root (itself the shared image) -> x: nothing moves; the root's images +// already take every static package they reach. +TEST(StaticPlacement, ARootOwnedImageKeepsItsStaticPackages) { + Edges edges{{0, {1}}}; + auto p = place_static_packages(edges, {}, {}); + EXPECT_TRUE(p.intoImage.empty()); + EXPECT_TRUE(p.conflicts.empty()); +} + +// root -> fw1 -> fw2 -> x: a closure stops at the next shared package, so x +// belongs to fw2 and not to fw1. +TEST(StaticPlacement, AClosureStopsAtTheNextSharedLibrary) { + Edges edges{{0, {1}}, {1, {2}}, {2, {3}}}; + auto p = place_static_packages(edges, {{1, 1}, {2, 1}}, {}); + ASSERT_EQ(p.intoImage.size(), 1u); + EXPECT_EQ(p.intoImage.at(3), 2u); + EXPECT_TRUE(p.conflicts.empty()); +} + +// A package with two shared targets is two images: its static closure has no +// single home. +TEST(StaticPlacement, OnePackageWithTwoSharedTargetsIsTwoImages) { + Edges edges{{0, {1}}, {1, {2}}}; + auto p = place_static_packages(edges, {{1, 2}}, {}); + EXPECT_TRUE(p.intoImage.empty()); + ASSERT_EQ(p.conflicts.size(), 1u); + EXPECT_EQ(p.conflicts[0].package, 2u); +} + +// A boundary (a distribution package, or a package providing a target layer) +// is neither placed nor walked through. +TEST(StaticPlacement, ABoundaryIsNeitherPlacedNorTraversed) { + Edges edges{{0, {1}}, {1, {2}}, {2, {3}}}; + auto p = place_static_packages(edges, {{1, 1}}, {2}); + EXPECT_TRUE(p.intoImage.empty()); + EXPECT_TRUE(p.conflicts.empty()); +} + +// A header-only package placed into an image contributes no objects, and the +// placement is still well defined. +TEST(StaticPlacement, APackageWithNoObjectsIsPlacedLikeAnyOther) { + Edges edges{{0, {1}}, {1, {2}}}; + auto p = place_static_packages(edges, {{1, 1}}, {}); + EXPECT_EQ(p.intoImage.at(2), 1u); +} From 9f336b082c0eb5fed876184db298106e75cec941 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:20:54 +0800 Subject: [PATCH 06/20] Features, tools and git sources: forwards over every table, one naming derivation, packages of programs, git members (#647 E4, #649 E6-E8) - The forward validator reads every dependency table of its manifest, on every row, including [build-dependencies] (#647 E4.1). - A [feature-deps] restatement naming another source than the declaration in effect is refused, naming both (#647 E4.2). - One derivation names a provider for dep_dir, dep_linkage and dep_bin, so a namespace + name package's tool is published under its qualified name (#647 E4.3). - A package whose declared targets are all programs contributes nothing to a consumer's graph; package cycles are refused where the graph is resolved (package-cycle); a tool requested by its own sub-build is refused at the first repetition (#649 E6). - A git dependency selects the repository member that declares its key; a member's in-clone path edge names the same git source; a second declaration of one dependency merges its requests into the edge; the git banner names its reference (#649 E7). - --features / is a forward of the root and never a macro; mcpp why accepts --features (#649 E8). --- modules/manifest/src/toml.cppm | 2 + modules/manifest/src/types.cppm | 6 + src/build/execute.cppm | 12 + src/build/prepare.cppm | 497 ++++++++++++++++++++++++++++---- src/build/refusal.cppm | 6 + src/cli.cppm | 10 +- src/cli/cmd_self.cppm | 3 +- src/doctor.cppm | 10 +- 8 files changed, 490 insertions(+), 56 deletions(-) diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 4d310e3b..64e1a6fa 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -3799,6 +3799,7 @@ void apply_defaults_and_infer(Manifest& m, const std::filesystem::path& root) { t.kind = Target::Binary; t.main = "src/main.cpp"; m.targets.push_back(std::move(t)); + m.targetsInferred = true; m.inferredNotes.push_back( std::format("target {} (bin from src/main.cpp)", m.package.name)); } else if (hasModuleInterface) { @@ -3806,6 +3807,7 @@ void apply_defaults_and_infer(Manifest& m, const std::filesystem::path& root) { t.name = m.package.name; t.kind = Target::Library; m.targets.push_back(std::move(t)); + m.targetsInferred = true; m.inferredNotes.push_back( std::format("target {} (lib from {} in src/)", m.package.name, moduleInterfaceExt)); } diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index b5959772..23aaf861 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1978,6 +1978,12 @@ struct Manifest { bool usesModules = true; // refined by scanner bool usesImportStd = true; // refined by scanner std::vector inferredNotes; // for `Inferred ...` banner + // `targets` was inferred from the source tree (no `[targets]` table), not + // declared. A consumer treats a package whose DECLARED targets are all + // programs as a tool provider that contributes nothing to its own graph + // (#649 E6); a package that declared nothing keeps the older reading, so an + // inferred `bin` from `src/main.cpp` is not taken as that statement. + bool targetsInferred = false; // Non-fatal schema warnings collected during parse (e.g. unsupported keys // under [targets.]). The caller (prepare_build) prints these and, under diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 27e6c4f4..fb3ddb90 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -886,8 +886,20 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, // result over in ctx.resolvedVersions; fall back to the spec only for // deps that never went through resolution (git, or an exact pin). auto rit = ctx.resolvedVersions.find(name); + // A git dependency has no version to announce: `spec.version` is empty + // for it, and the banner used to read "Compiling spike.fw v" (#649 E7). + // It names the reference the manifest wrote instead, shortening a + // commit to the length `git` itself abbreviates to. + auto gitReference = [&] { + std::string ref = spec.gitRev; + if (spec.gitRefKind == "rev" && ref.size() > 12) ref.resize(12); + return std::format("(git {} {})", + spec.gitRefKind.empty() ? "rev" : spec.gitRefKind, ref); + }; std::string ver = spec.isPath() ? "(path)" + : spec.isGit() + ? gitReference() : std::string("v") + (rit != ctx.resolvedVersions.end() ? rit->second : spec.version); auto it = cachedUnits.find(name); diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index d3888aac..68b05e01 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -647,7 +647,7 @@ std::vector feature_closure(const mcpp::manifest::Manifest& pm, } // --features value → tokens (comma/space separated). -std::vector parse_feature_request(std::string_view s) { +std::vector feature_request_tokens(std::string_view s) { std::vector out; for (std::size_t p = 0; p < s.size();) { auto c = s.find_first_of(", ", p); @@ -659,6 +659,33 @@ std::vector parse_feature_request(std::string_view s) { return out; } +// The root's own features among the --features tokens. +// +// A TOKEN CONTAINING `/` IS NOT A FEATURE OF THE ROOT (#649 E8). It can only +// mean "open this feature of that dependency", which is what the same token +// means inside `[features]`, so it is taken out here and applied as a forward +// of the root (`feature_forward_request`). It used to stay in this list, where +// a root without `[features]` turned it into `-DMCPP_FEATURE_SPIKE_FW_INSTALLER` +// and a root with the table reported it as an undeclared feature; neither +// opened the dependency's feature. +std::vector parse_feature_request(std::string_view s) { + std::vector out; + for (auto& tok : feature_request_tokens(s)) + if (tok.find('/') == std::string::npos) out.push_back(std::move(tok)); + return out; +} + +// The `/` tokens of --features, in the keyspace of a +// `[features]` forward. A token with an empty half is kept whole and named by +// the caller, rather than being dropped as the manifest parser drops it: on a +// command line the user typed it just now. +std::vector feature_forward_request_tokens(std::string_view s) { + std::vector out; + for (auto& tok : feature_request_tokens(s)) + if (tok.find('/') != std::string::npos) out.push_back(std::move(tok)); + return out; +} + bool is_std_module(std::string_view name) { return name == "std" || name == "std.compat"; } @@ -1059,6 +1086,10 @@ export struct BuildOverrides { int tool_depth = 0; // The request chain, for that diagnostic. "root → grpc:grpc_cpp_plugin → …" std::string tool_chain; + // The (package source, tool) pairs being built by the enclosing sub-builds, + // outermost first. A request for one of them is the tool's own build asking + // for itself, refused at its first repetition (#649 E6). + std::vector tool_chain_sources; // Use THIS manifest instead of reading `/mcpp.toml`. // // Required for a `compat`-style registry package (Form B), which ships no @@ -4850,6 +4881,48 @@ prepare_build(bool print_fingerprint, std::map identityBySource; std::map declaringManifest; std::set> adoptionsReported; + // A GIT DEPENDENCY NAMES A REPOSITORY, AND THE KEY NAMES WHICH PACKAGE OF + // IT (#649 E7). The root manifest's package is one; each `[workspace] + // members` entry of that manifest is another. Before this a git source + // always yielded the root package, so a repository holding a framework and + // its tools could be pinned by revision for the framework only. The clone + // of every git source resolved so far is kept with the reference it was + // resolved from, so a later key over the same source, and a member's + // `path` edge that stays inside the clone, find it. + struct GitClone { + std::filesystem::path root; + std::string url, refKind, ref; + }; + std::map gitCloneBySource; + // The member of the repository at `cloneRoot` whose manifest declares + // `want`, when the root manifest's package is not `want` itself. + auto gitMemberDeclaring = [&](const std::filesystem::path& cloneRoot, + const ResolvedKey& want) + -> std::optional { + auto declares = [&](const mcpp::manifest::Manifest& mm) { + auto rn = mcpp::pm::compat::resolve_package_name( + mm.package.name, mm.package.namespace_); + // A manifest that names no namespace takes the key's, as a path + // dependency's does (#634 A2), so only the short name is compared. + const bool nsDeclared = !mm.package.namespace_.empty() || rn.usedLegacySplit; + return rn.shortName == want.shortName + && (!nsDeclared || rn.namespace_ == want.ns); + }; + std::error_code ec; + if (!std::filesystem::exists(cloneRoot / "mcpp.toml", ec)) return std::nullopt; + auto rootManifest = mcpp::manifest::load(cloneRoot / "mcpp.toml"); + if (!rootManifest || declares(*rootManifest) + || !rootManifest->workspace.present) + return std::nullopt; + for (auto const& member : rootManifest->workspace.members) { + const auto path = cloneRoot / member / "mcpp.toml"; + if (!std::filesystem::exists(path, ec)) continue; + auto mm = mcpp::manifest::load(path, {.insideWorkspace = true}); + if (mm && declares(*mm)) + return std::filesystem::path(member).lexically_normal().generic_string(); + } + return std::nullopt; + }; auto qualifiedKey = [](const ResolvedKey& k) { return k.ns.empty() ? k.shortName : std::format("{}.{}", k.ns, k.shortName); }; @@ -5847,7 +5920,42 @@ prepare_build(bool print_fingerprint, } return prov::bind_bare_names(fqns); }; + // THE NAMES UNDER WHICH ONE PROVIDER IS PUBLISHED TO ONE CONSUMER, derived + // once for every channel (#647 E4.3). The manifest's `name`, the qualified + // `namespace.name` when the manifest writes the two apart, and the bare + // tail where the namespace ladder binds it to this provider for this + // consumer. `dep_dir`/`dep_linkage` and `dep_bin` used to derive this list + // separately; #642 added the qualified spelling to the first and the second + // kept publishing `MCPP_DEP_INSTALLER_BIN_*` alone for a package written + // `namespace = "spike"`, `name = "installer"`, so + // `dep_bin("spike.installer", ...)` read nothing. + auto publishedNamesFor = + [&](std::size_t provider, + const std::map& bind) { + std::vector out; + auto const& manifest = packages[provider].manifest; + auto const& canon = manifest.package.name; + out.push_back(canon); + if (auto qualified = mcpp::build::qualified_package_name(manifest); + qualified != canon) + out.push_back(std::move(qualified)); + if (auto tail = prov::tail_of(canon); tail != canon) { + auto it = bind.find(tail); + if (it != bind.end() && it->second.owner == canon) + out.push_back(std::move(tail)); + } + return out; + }; + // A package whose DECLARED targets are all programs (#649 E6). See the + // worklist, where such a package is not walked into a consumer's graph. + auto isProgramOnlyPackage = [](const mcpp::manifest::Manifest& pm) { + if (pm.targetsInferred || pm.targets.empty()) return false; + return std::ranges::none_of(pm.targets, [](const mcpp::manifest::Target& t) { + return t.kind == mcpp::manifest::Target::Library + || t.kind == mcpp::manifest::Target::SharedLibrary; + }); + }; auto parseVisibility = [](std::string_view visibility) { if (visibility == "private") return mcpp::modgraph::DependencyVisibility::Private; @@ -6030,25 +6138,13 @@ prepare_build(bool print_fingerprint, if (linkForms) if (auto f = linkForms->find(pr.provider); f != linkForms->end()) form = &f->second; - e.depDirs.emplace_back(canon, depPkg.root); - if (form) e.depLinkages.emplace_back(canon, *form); - // AND THE SPELLING THE CONSUMER'S MANIFEST USES. A package that - // writes `namespace = "ns"` and `name = "fw"` is a dependency named - // `ns.fw`, and `dep_dir("ns.fw")` read nothing while the reference - // above promised the canonical spelling (#642: the framework's rule - // asks `dep_linkage("huxerui.huxerui")`). The qualified name is - // unique in the graph, so it needs no binding check. - if (auto qualified = mcpp::build::qualified_package_name(depPkg.manifest); - qualified != canon) { - e.depDirs.emplace_back(qualified, depPkg.root); - if (form) e.depLinkages.emplace_back(qualified, *form); - } - auto tail = prov::tail_of(canon); - if (tail == canon) continue; - auto it = bind.find(tail); - if (it != bind.end() && it->second.owner == canon) { - e.depDirs.emplace_back(tail, depPkg.root); - if (form) e.depLinkages.emplace_back(tail, *form); + // Every spelling of `publishedNamesFor`: the manifest's name, the + // qualified name a manifest writing `namespace = "ns"` and + // `name = "fw"` is addressed by (#642: the framework's rule asks + // `dep_linkage("huxerui.huxerui")`), and the bound tail. + for (auto const& n : publishedNamesFor(pr.provider, bind)) { + e.depDirs.emplace_back(n, depPkg.root); + if (form) e.depLinkages.emplace_back(n, *form); } } }; @@ -6309,6 +6405,30 @@ prepare_build(bool print_fingerprint, // way round would let a `[build-dependencies]` line quietly drop a // library the target needs. if (!buildOnly) it->buildOnly = false; + // AND THE SECOND DECLARATION'S REQUESTS ARE KEPT (#649 E7). Both + // declarations name one edge, so what each asks of the dependency + // is asked of that edge: this used to return here and lose the + // second one's `tools`, `features`, `host-module` and `reexport` + // without a word, under `--strict` too. The rule is the one + // `mergeActiveFeatureDeps` already applies to a feature's + // restatement: additive fields union, `default-features` stays on + // unless every declaration opts out. + for (auto const& t : spec.tools) + if (std::ranges::find(it->requestedTools, t) == it->requestedTools.end()) + it->requestedTools.push_back(t); + for (auto const& f : spec.features) + if (std::ranges::find(it->requestedFeatures, f) + == it->requestedFeatures.end()) + it->requestedFeatures.push_back(f); + it->defaultFeatures = it->defaultFeatures || spec.defaultFeatures; + if (spec.hostModule) it->hostModule = true; + else if (dependencyPackageIndex < packages.size()) + for (auto const& f : spec.features) + if (packages[dependencyPackageIndex].manifest.featureRuleModule.contains(f)) { + it->hostModule = true; + break; + } + it->reexport = it->reexport || spec.reexport; return; } // A REQUESTED FEATURE THAT IS A BUILD RULE IMPLIES `host-module`. @@ -6349,6 +6469,10 @@ prepare_build(bool print_fingerprint, } auto& consumer = packages[edge.consumerPackageIndex]; auto const& dependency = packages[edge.dependencyPackageIndex]; + // A package of programs publishes no usage requirements to its + // consumers (#649 E6): nothing of it is compiled or linked here. + if (edge.dependencyPackageIndex > 0 + && isProgramOnlyPackage(dependency.manifest)) continue; if (edge.visibility == mcpp::modgraph::DependencyVisibility::Private || edge.visibility == mcpp::modgraph::DependencyVisibility::Public) { @@ -6599,16 +6723,52 @@ prepare_build(bool print_fingerprint, // when that feature is active. `seedDefault` carries consumer-side // `default-features = false` (#242): when a consumer opts out of this dep's // default set, feature-deps behind the default pseudo-feature stay dormant. + // + // A RESTATEMENT NAMES THE SAME SOURCE OR IS REFUSED (#647 E4.2). The grammar + // asks a `[feature-deps]` entry to restate its dependency's source, and the + // merge below takes only the additive fields from it, so a restatement + // that names another path, repository or version was dropped without a + // word, under `--strict` too: the tool came from the declaration in effect + // while the manifest said it came from somewhere else. The comparison runs + // against `dependencies` after the conditional fold, so a row's replacement + // (#634 A1) is the declaration a restatement is held to. + auto dependencySourceOf = [](const mcpp::manifest::DependencySpec& s) { + if (s.inheritWorkspace) return std::string("workspace = true"); + if (s.isPath()) { + auto norm = std::filesystem::path(s.path).lexically_normal().generic_string(); + while (norm.size() > 1 && norm.back() == '/') norm.pop_back(); + return std::format("path = \"{}\"", norm); + } + if (s.isGit()) + return std::format("git = \"{}\", {} = \"{}\"", s.git, + s.gitRefKind.empty() ? "rev" : s.gitRefKind, s.gitRev); + return std::format("version = \"{}\"", s.version); + }; auto mergeActiveFeatureDeps = [&](mcpp::manifest::Manifest& pm, const std::vector& requested, - bool seedDefault = true) { - if (pm.featureDeps.empty()) return; + bool seedDefault = true) + -> std::expected { + if (pm.featureDeps.empty()) return {}; for (auto& f : activateFeatures(pm, requested, seedDefault)) { auto it = pm.featureDeps.find(f); if (it == pm.featureDeps.end()) continue; for (auto& [k, spec] : it->second) { auto [pos, fresh] = pm.dependencies.try_emplace(k, spec); if (fresh) continue; + if (!pos->second.inheritWorkspace && !spec.inheritWorkspace) { + const auto inEffect = dependencySourceOf(pos->second); + const auto restated = dependencySourceOf(spec); + if (inEffect != restated) + return std::unexpected(std::format( + "[feature-deps.{}] of '{}' restates the dependency '{}' " + "with {}, while the declaration in effect on this row " + "names {}.\n" + " One dependency has one source, so the " + "restatement would be ignored.\n" + " fix: restate the same source ({}), or declare " + "'{}' only under the feature.", + f, pm.package.name, k, restated, inEffect, inEffect, k)); + } // #359: the key already exists unconditionally, and dropping // the feature's spec here loses REQUESTS the feature exists to // make. gRPC is the shape: it depends on compat.protobuf @@ -6635,6 +6795,7 @@ prepare_build(bool print_fingerprint, dst.reexport = dst.reexport || spec.reexport; } } + return {}; }; // #243: dep/feat forwarding. When a resolved package's feature F is active, @@ -6662,10 +6823,36 @@ prepare_build(bool print_fingerprint, } }; // #243: a forward whose active feature targets a dependency that is not - // declared (in [dependencies], [dev-dependencies], or an active - // [feature-deps] already folded into `dependencies`) is a manifest bug — - // name it instead of silently dropping. Only active features' forwards are - // checked (lazy, like the unknown-requested-feature gate at ~2875). + // declared is a manifest bug — name it instead of silently dropping. Only + // active features' forwards are checked (lazy, like the + // unknown-requested-feature gate at ~2875). + // + // THE VALIDATOR ASKS WHAT THE FORWARD LANGUAGE DEFINES: IS THE KEY DECLARED + // IN ANY DEPENDENCY TABLE OF THIS MANIFEST, ON ANY ROW, UNDER ANY FEATURE + // (#647 E4.1). It used to look in `dependencies` and `devDependencies` + // only, while `injectForwards` applies a forward to the build-dependency + // edge as well, so a forward along `[build-dependencies]` was applied and + // reported as undeclared in the same run, and `--strict` refused a build + // whose forward had worked. A key declared only for another row, or only + // under an inactive feature, is declared: on this row the forward reaches + // no edge and does nothing, which is what a portable manifest means by it. + auto declaresDependencyKey = [](const mcpp::manifest::Manifest& pm, + const std::string& key) { + auto inFeatureDeps = [&](const auto& byFeature) { + for (auto const& [f, deps] : byFeature) + if (deps.contains(key)) return true; + return false; + }; + if (pm.dependencies.contains(key) || pm.devDependencies.contains(key) + || pm.buildDependencies.contains(key) || inFeatureDeps(pm.featureDeps)) + return true; + for (auto const& cc : pm.conditionalConfigs) + if (cc.dependencies.contains(key) || cc.devDependencies.contains(key) + || cc.buildDependencies.contains(key) + || inFeatureDeps(cc.featureDeps)) + return true; + return false; + }; auto validateForwards = [&](const mcpp::manifest::Manifest& parent, const std::vector& parentActive, std::string_view parentName) @@ -6674,12 +6861,12 @@ prepare_build(bool print_fingerprint, auto it = parent.featureForwards.find(f); if (it == parent.featureForwards.end()) continue; for (auto const& [depKey, depFeat] : it->second) { - if (parent.dependencies.contains(depKey) - || parent.devDependencies.contains(depKey)) continue; + if (declaresDependencyKey(parent, depKey)) continue; auto msg = std::format( "feature '{}' of '{}' forwards to dependency '{}' (as " - "'{}/{}') which is not declared in [dependencies] or " - "[feature-deps]", f, parentName, depKey, depKey, depFeat); + "'{}/{}') which no dependency table declares ([dependencies], " + "[build-dependencies], [dev-dependencies] or [feature-deps], " + "on any row)", f, parentName, depKey, depKey, depFeat); if (overrides.strict) return std::unexpected(msg); mcpp::diag::warning("features/forwarding", msg); } @@ -6690,25 +6877,60 @@ prepare_build(bool print_fingerprint, // Pull the root package's active feature-deps into its dependency set before // seeding, so `mcpp build --features X` resolves X's optional deps. std::vector rootReq = parse_feature_request(overrides.features); - mergeActiveFeatureDeps(*m, rootReq); + if (auto fm = mergeActiveFeatureDeps(*m, rootReq); !fm) + return std::unexpected(fm.error()); // #243: the root's active features may forward features to its direct deps. std::vector rootActive = feature_closure(*m, rootReq, true); if (auto fe = validateForwards(*m, rootActive, m->package.name); !fe) return std::unexpected(fe.error()); activeFeaturesByPackage.assign(1, rootActive); + // `--features /` (#649 E8): a forward of the root, + // applied to the edges exactly as a `[features]` forward is and checked + // against the same tables. Named whether or not the root declares + // `[features]`: the token cannot be a macro of the root, so there is no + // "pure macro usage" to preserve for it. + std::vector> cliForwards; + for (auto const& tok : feature_forward_request_tokens(overrides.features)) { + auto fwd = mcpp::pm::split_feature_forward_token(tok); + std::string msg; + if (!fwd) + msg = std::format("--features requests '{}', which names neither a " + "feature nor `/`", tok); + else if (!declaresDependencyKey(*m, fwd->first)) + msg = std::format("--features requests '{}', and no dependency table " + "of '{}' declares '{}'", tok, m->package.name, + fwd->first); + if (!msg.empty()) { + if (overrides.strict) return std::unexpected(msg); + mcpp::diag::warning("features/request", msg); + continue; + } + cliForwards.push_back(std::move(*fwd)); + } + auto injectCliForwards = [&](const std::string& childKey, + mcpp::manifest::DependencySpec& childSpec) { + for (auto const& [depKey, depFeat] : cliForwards) + if (depKey == childKey + && std::ranges::find(childSpec.features, depFeat) + == childSpec.features.end()) + childSpec.features.push_back(depFeat); + }; + // Seed the worklist from the main manifest. Dev-deps only when the // caller wants them; they're never propagated transitively. const std::string mainPkgLabel = m->package.name; for (auto& [n, s] : m->dependencies) { auto req = s; injectForwards(*m, rootActive, n, req); + injectCliForwards(n, req); worklist.push_back({n, req, mainPkgLabel, req.version, kMainConsumer, {}}); } if (includeDevDeps) { for (auto& [n, s] : m->devDependencies) { auto req = s; injectForwards(*m, rootActive, n, req); + injectCliForwards(n, req); worklist.push_back({n, req, mainPkgLabel + " (dev-dep)", req.version, kMainConsumer, {}, /*devOnly=*/true}); } @@ -6722,6 +6944,7 @@ prepare_build(bool print_fingerprint, for (auto& [n, s] : m->buildDependencies) { auto req = s; injectForwards(*m, rootActive, n, req); + injectCliForwards(n, req); worklist.push_back({n, req, mainPkgLabel + " (build-dep)", req.version, kMainConsumer, {}, /*devOnly=*/false, /*buildOnly=*/true}); @@ -6779,6 +7002,41 @@ prepare_build(bool print_fingerprint, } } + // A `path` edge that stays inside a git clone this graph already + // resolved names the same git source at the same commit (#649 E7): + // a member's `spike.fw = { path = ".." }` reaches the repository the + // application pinned by revision, and is that package rather than a + // second, path-sourced declaration of it. Only the clone root itself + // and its `[workspace] members` are mapped; any other directory keeps + // being an ordinary path. + if (spec.isPath() && !gitCloneBySource.empty()) { + std::filesystem::path p = spec.path; + auto base = item.resolveRoot.empty() ? *root : item.resolveRoot; + if (p.is_relative()) p = base / p; + std::error_code ec; + auto canon = std::filesystem::weakly_canonical(p, ec); + if (ec) canon = p.lexically_normal(); + for (auto const& [src, clone] : gitCloneBySource) { + auto rel = canon.lexically_relative(clone.root).generic_string(); + if (rel.empty() || rel.starts_with("..")) continue; + bool mapped = rel == "."; + if (!mapped) { + if (auto rm = mcpp::manifest::load(clone.root / "mcpp.toml"); + rm && rm->workspace.present) + for (auto const& member : rm->workspace.members) + if (std::filesystem::path(member).lexically_normal() + .generic_string() == rel) + mapped = true; + } + if (!mapped) continue; + spec.path.clear(); + spec.git = clone.url; + spec.gitRefKind = clone.refKind; + spec.gitRev = clone.ref; + break; + } + } + // Pin SemVer constraint before dedup/fetch. if (auto r = resolveSemver(spec, name); !r) { return std::unexpected(r.error()); @@ -6795,14 +7053,26 @@ prepare_build(bool print_fingerprint, // The commit a `git` dependency resolved to, carried out of the clone // branch below for the cache identity. std::string sourceCommit; + // The repository member a `git` dependency selected; empty for the + // repository's root package (#649 E7). + std::string gitMember; + std::filesystem::path gitMemberCloneRoot; // A second key over a source that is already resolved takes the // identity resolved there; its manifest is not loaded again. if (sourceKind != "version") { const auto source = sourceRefOf(sourceKind, spec, item.resolveRoot, item.originalConstraint); + // A key naming another package of the same repository is that + // member, not a second key over the root's identity (#649 E7). + bool namesMember = false; + if (sourceKind == "git") + if (auto clone = gitCloneBySource.find(source); + clone != gitCloneBySource.end()) + namesMember = gitMemberDeclaring(clone->second.root, key).has_value(); if (auto bySource = identityBySource.find(source); - bySource != identityBySource.end() && !(bySource->second == key)) { + !namesMember && bySource != identityBySource.end() + && !(bySource->second == key)) { const auto& existing = resolved.at(bySource->second); const auto& declaring = declaringManifest.at(bySource->second); if (!declaring.namespaceDeclared) { @@ -7449,6 +7719,14 @@ prepare_build(bool print_fingerprint, } sourceCommit = resolvedGitRev; dep_root = gitRoot; + gitCloneBySource.try_emplace( + sourceRefOf("git", spec, item.resolveRoot, item.originalConstraint), + GitClone{ gitRoot, spec.git, spec.gitRefKind, spec.gitRev }); + if (auto member = gitMemberDeclaring(gitRoot, key)) { + gitMember = *member; + gitMemberCloneRoot = gitRoot; + dep_root = gitRoot / *member; + } } // (version-source: dep_root + manifest are loaded together via // loadVersionDep below since the index entry drives both.) @@ -7484,7 +7762,7 @@ prepare_build(bool print_fingerprint, *wsManifest, runtimeWorkspaceRoot, dep_root); auto dm = mcpp::manifest::load( dep_root / "mcpp.toml", - {.insideWorkspace = depIsMember}); + {.insideWorkspace = depIsMember || !gitMember.empty()}); if (!dm) { return std::unexpected(std::format( "dependency '{}' (at '{}'): {}", @@ -7501,6 +7779,16 @@ prepare_build(bool print_fingerprint, if (auto bad = mcpp::project::workspace_inheritance_error( *dep_manifest, dep_root)) return std::unexpected(*bad); + } else if (!gitMember.empty()) { + // A repository member inherits its repository's + // `[workspace.package]`, as it does when the repository is + // built from its own checkout. + if (auto rm = mcpp::manifest::load(gitMemberCloneRoot / "mcpp.toml")) { + mcpp::project::inherit_workspace_package(*dep_manifest, *rm); + if (auto bad = mcpp::project::workspace_inheritance_error( + *dep_manifest, dep_root)) + return std::unexpected(*bad); + } } // #229: path/git-dep half of the L1 cfg funnel — mirrors the // loadVersionDep call site above (loadFrom's L1 cfg merge, ~1740 @@ -7608,9 +7896,22 @@ prepare_build(bool print_fingerprint, // dependency set before its children are pushed, so a dep's feature can // transitively pull a provider. `spec.features` = features the consumer // requested for this dep. - mergeActiveFeatureDeps(*dep_manifest, spec.features, spec.defaultFeatures); - - auto linkFlagsAdded = propagateLinkFlags(dep_root, *dep_manifest); + if (auto fm = mergeActiveFeatureDeps(*dep_manifest, spec.features, + spec.defaultFeatures); !fm) + return std::unexpected(fm.error()); + + // A PACKAGE OF PROGRAMS HAS NOTHING TO LINK (#649 E6). Its tools are + // built by the tool sub-build, which resolves the package as its own + // root; in this graph it is a provider of tools and of its directory, + // and nothing more. Walking its dependencies here put a tool's own + // library into the application's link (a tool depending on `z` gave the + // application `z.o`), compiled its sources in the consumer's build, and + // made a tool that depends on the package declaring it a cycle of the + // consumer's graph although the two builds never meet. + const bool depProgramOnly = isProgramOnlyPackage(*dep_manifest); + auto linkFlagsAdded = depProgramOnly + ? std::vector{} + : propagateLinkFlags(dep_root, *dep_manifest); // Move the manifest into stable storage so we can later look it up // by depIndex (the SemVer merger needs to overwrite the slot). @@ -7636,7 +7937,8 @@ prepare_build(bool print_fingerprint, // (ns, name) hit the fast path (skip / merge / conflict). if (sourceKind != "version") { identityBySource.emplace( - sourceRefOf(sourceKind, spec, item.resolveRoot, item.originalConstraint), + sourceRefOf(sourceKind, spec, item.resolveRoot, item.originalConstraint) + + (gitMember.empty() ? std::string{} : "#member=" + gitMember), key); declaringManifest[key] = DeclaringManifest{ manifestPath, namespaceDeclared }; } @@ -7656,6 +7958,8 @@ prepare_build(bool print_fingerprint, // Recurse: the dep's own [dependencies] become new worklist items. // dev-dependencies are intentionally NOT walked — those are // private to the dep's test runs, not part of its public ABI. + // A package of programs is not walked at all; see `depProgramOnly`. + if (depProgramOnly) continue; const std::string thisDepLabel = std::format( "{}{}{}@{}", key.ns, @@ -7702,6 +8006,52 @@ prepare_build(bool print_fingerprint, } } + // ONE PLACE DETECTS A CYCLE OF PACKAGES, AND IT IS HERE, WHERE THE GRAPH + // IS RESOLVED (#649 E6). The build-cache key walk was the only reader that + // noticed, and it runs for the global cache only, so the same manifest was + // refused by default and built under `--cache=local`. Every edge counts, + // build-only ones included, as the key walk counts them. + { + std::vector state(packages.size(), 0); // 0 new / 1 on stack / 2 done + std::vector stack, cycle; + auto visit = [&](auto&& self, std::size_t u) -> bool { + state[u] = 1; + stack.push_back(u); + for (auto const& e : dependencyEdges) { + if (e.consumerPackageIndex != u) continue; + const auto v = e.dependencyPackageIndex; + if (v >= packages.size()) continue; + if (state[v] == 1) { + cycle.assign(std::ranges::find(stack, v), stack.end()); + cycle.push_back(v); + return true; + } + if (state[v] == 0 && self(self, v)) return true; + } + stack.pop_back(); + state[u] = 2; + return false; + }; + for (std::size_t i = 0; i < packages.size() && cycle.empty(); ++i) + if (state[i] == 0) (void)visit(visit, i); + if (!cycle.empty()) { + std::string path; + for (auto p : cycle) { + if (!path.empty()) path += " -> "; + path += std::format("'{}'", + mcpp::build::qualified_package_name(packages[p].manifest)); + } + refusal::record(refusal::Code::PackageCycle); + return std::unexpected(std::format( + "dependency cycle: {}.\n" + " A package cannot reach itself through its own dependencies.\n" + " fix: remove one of these edges. A program that depends on the " + "package requesting it builds without a cycle when its package " + "declares only `kind = \"bin\"` targets: it is then built by its " + "own tool sub-build.", path)); + } + } + computeUsageRequirements(); // ─── The toolchain, resolved now that the graph exists ────────────────── @@ -9269,26 +9619,25 @@ prepare_build(bool print_fingerprint, binList.empty() ? std::string("none") : binList)); } - auto var = mcpp::build::tool_store::env_var_name(depName, toolName); - auto varShort = - mcpp::build::tool_store::env_var_name(depShort, toolName); - // #359: every consumer that can SEE this tool gets it, not // just the one whose edge asked for it. The bare spelling // is emitted only where the namespace ladder binds the tail // to this package — otherwise two libraries re-exporting a // same-tailed tool would decide the winner by append order. + // The spellings are `publishedNamesFor`'s, so a tool is + // addressed by exactly the names its directory is. const prov::Provision want{ prov::Kind::Tool, depIdx, toolName }; auto record = [&](const std::filesystem::path& p) { for (std::size_t c = 0; c < provisionGraph.visible.size(); ++c) { if (!provisionGraph.visible[c].contains(want)) continue; auto& v = toolEnvByConsumer[c]; - v.emplace_back(var, p.string()); - if (varShort == var) continue; - auto bind = bareBindingsFor(c); - auto it = bind.find(depShort); - if (it != bind.end() && it->second.owner == depName) - v.emplace_back(varShort, p.string()); + std::vector vars; + for (auto const& n : publishedNamesFor(depIdx, bareBindingsFor(c))) { + auto var = mcpp::build::tool_store::env_var_name(n, toolName); + if (std::ranges::find(vars, var) != vars.end()) continue; + vars.push_back(var); + v.emplace_back(std::move(var), p.string()); + } } }; @@ -9309,6 +9658,42 @@ prepare_build(bool print_fingerprint, continue; } + // A TOOL WHOSE OWN BUILD REQUESTS IT AGAIN IS REFUSED AT + // THE FIRST REPETITION (#649 E6). The depth bound below + // caught it only after four nested sub-builds, with the + // same prefix repeated four times and no word about which + // edge asked. The edge is the one whose request reached + // this package in THIS graph. + const std::string toolSource = std::format( + "{}|{}", depPkg.root.lexically_normal().generic_string(), toolName); + if (std::ranges::find(overrides.tool_chain_sources, toolSource) + != overrides.tool_chain_sources.end()) { + std::string askedBy; + for (auto const& edge : dependencyEdges) { + if (edge.dependencyPackageIndex != depIdx) continue; + if (std::ranges::find(edge.requestedTools, toolName) + == edge.requestedTools.end()) continue; + if (edge.consumerPackageIndex < packages.size()) { + askedBy = mcpp::build::qualified_package_name( + packages[edge.consumerPackageIndex].manifest); + break; + } + } + return std::unexpected(std::format( + "the host tool '{}:{}' is requested by its own build: " + "{} -> {}:{}.\n" + " The request comes from '{}', which the tool's " + "sub-build resolves with the feature or dependency that " + "asks for the tool.\n" + " fix: the tool's own graph must not activate " + "that request (a feature it does not enable, or a " + "`[target..feature-deps]` row it does not match).", + depName, toolName, + overrides.tool_chain.empty() ? "root" : overrides.tool_chain, + depName, toolName, + askedBy.empty() ? std::string("a package of its graph") : askedBy)); + } + // Build it. The feature set is the tool package's own // defaults PLUS the target's required_features — in a tool // sub-build the target is what was ASKED FOR, so its @@ -9411,6 +9796,8 @@ prepare_build(bool print_fingerprint, sub.profile = "release"; sub.cache_mode = overrides.cache_mode; sub.tool_depth = overrides.tool_depth + 1; + sub.tool_chain_sources = overrides.tool_chain_sources; + sub.tool_chain_sources.push_back(toolSource); // The PRISTINE manifest the resolver produced for this // package — `packages[depIdx].manifest` is a copy that // feature activation has already mutated, and re-activating @@ -9558,6 +9945,9 @@ prepare_build(bool print_fingerprint, // written to. for (std::size_t i = 1; i < packages.size(); ++i) { auto& pkg = packages[i]; + // A package of programs runs its build program in its own tool + // sub-build, where its sources are compiled (#649 E6). + if (isProgramOnlyPackage(pkg.manifest)) continue; std::error_code bpEc; if (!std::filesystem::exists(pkg.root / "build.mcpp", bpEc) && pkg.manifest.buildConfig.ruleModules.empty()) continue; @@ -11032,16 +11422,25 @@ prepare_build(bool print_fingerprint, // Modgraph: regex scanner by default; opt-in to compiler-driven P1689 // scanner via env var MCPP_SCANNER=p1689 (see docs/27). + // + // A dependency whose declared targets are all programs compiles nothing in + // this build (#649 E6): its programs come from the tool sub-build, which + // scans it as its own root. The root itself is always scanned. + std::vector scannedPackages; + scannedPackages.reserve(packages.size()); + for (std::size_t i = 0; i < packages.size(); ++i) + if (i == 0 || !isProgramOnlyPackage(packages[i].manifest)) + scannedPackages.push_back(packages[i]); auto scan = [&] { const char* sel = std::getenv("MCPP_SCANNER"); if (sel && std::string_view(sel) == "p1689") { auto tmp = std::filesystem::temp_directory_path() / std::format("mcpp_p1689_{}", std::random_device{}()); std::filesystem::create_directories(tmp); - return mcpp::modgraph::scan_packages_p1689(packages, *tc, tmp, + return mcpp::modgraph::scan_packages_p1689(scannedPackages, *tc, tmp, stdFlagAndDialect); } - return mcpp::modgraph::scan_packages(packages); + return mcpp::modgraph::scan_packages(scannedPackages); }(); if (!scan.errors.empty()) { std::string msg = "scanner errors:\n"; diff --git a/src/build/refusal.cppm b/src/build/refusal.cppm index e5d2f8d0..c313ed58 100644 --- a/src/build/refusal.cppm +++ b/src/build/refusal.cppm @@ -105,6 +105,11 @@ enum class Code { // Distinct from LayerOrdering, which is about the layers not stacking: here // they stack, and one image would link with no C++ runtime at all (#641). SharedLibraryCxxRuntime, + // The dependency graph contains a cycle of packages. Detected where the + // graph is resolved, so every cache mode refuses it alike (#649 E6); the + // build-cache key walk used to be the only place that noticed, and a + // `--cache=local` build of the same graph went ahead. + PackageCycle, Other, // a refusal that has not been given a code yet }; @@ -137,6 +142,7 @@ constexpr std::string_view name(Code c) { case Code::ToolVersionConflict: return "tool-version-conflict"; case Code::SharedLibraryCxxRuntime: return "shared-library-cxx-runtime"; + case Code::PackageCycle: return "package-cycle"; case Code::Other: return "other"; } return "other"; diff --git a/src/cli.cppm b/src/cli.cppm index 92583126..5ff64d70 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -381,7 +381,7 @@ int run(int argc, char** argv) { .option(cl::Option("release").help("Shorthand for --profile release")) .option(cl::Option("dev").help("Shorthand for --profile dev (-O0 -g)")) .option(cl::Option("features").takes_value().value_name("LIST") - .help("Activate root-package features (comma-separated)")) + .help("Activate root-package features, and a dependency's as / (comma-separated)")) .option(cl::Option("cap").takes_value().value_name("LIST") .help("Pin capability providers (e.g. blas=openblas,lapack=mkl)")) .option(cl::Option("strict") @@ -511,7 +511,7 @@ int run(int argc, char** argv) { .option(cl::Option("profile").takes_value().value_name("NAME") .help("Build profile for the test build: dev (default) | release | dist | <[profile.*] name>")) .option(cl::Option("features").takes_value().value_name("LIST") - .help("Activate root-package features for the test build (comma-separated)")) + .help("Activate root-package features, and a dependency's as /, for the test build (comma-separated)")) .option(cl::Option("cap").takes_value().value_name("LIST") .help("Pin capability providers (e.g. blas=openblas,lapack=mkl)")) .option(cl::Option("strict") @@ -548,6 +548,8 @@ int run(int argc, char** argv) { .help("Ask about this target instead of the project's default")) .option(cl::Option("toolchain").takes_value() .help("Ask about this toolchain, e.g. llvm@22.1.8")) + .option(cl::Option("features").takes_value().value_name("LIST") + .help("Resolve with these features, as `build --features` does (root features and /)")) .option(cl::Option("format").takes_value().value_name("json") .help("Machine-readable output (enveloped; see docs/50-machine-output.md)")) .action(wrap_rc(cmd_why))) @@ -623,7 +625,7 @@ int run(int argc, char** argv) { .option(cl::Option("toolchain").takes_value().value_name("SPEC") .help("Build with this toolchain for one invocation, e.g. llvm@22.1.8")) .option(cl::Option("features").takes_value().value_name("LIST") - .help("Activate root-package features for every build pass of the pack (comma-separated)")) + .help("Activate root-package features, and a dependency's as /, for every build pass of the pack (comma-separated)")) .option(cl::Option("no-strip") .help("Ship the artifacts as built (default: strip debug info)")) .option(cl::Option("debug-symbols").takes_value().value_name("DIR") @@ -678,7 +680,7 @@ int run(int argc, char** argv) { .option(cl::Option("release").help("Shorthand for --profile release")) .option(cl::Option("dev").help("Shorthand for --profile dev")) .option(cl::Option("features").takes_value().value_name("LIST") - .help("Activate root-package features (comma-separated)")) + .help("Activate root-package features, and a dependency's as / (comma-separated)")) .option(cl::Option("cap").takes_value().value_name("LIST") .help("Pin capability providers (e.g. blas=openblas,lapack=mkl)")) .option(cl::Option("strict") diff --git a/src/cli/cmd_self.cppm b/src/cli/cmd_self.cppm index f118c370..9ff92c8e 100644 --- a/src/cli/cmd_self.cppm +++ b/src/cli/cmd_self.cppm @@ -107,7 +107,8 @@ export int cmd_why(const mcpplibs::cmdline::ParsedArgs& parsed) { parsed.option_or_empty("target").value(), parsed.option_or_empty("toolchain").value()); } - return mcpp::doctor::why_report(parsed.positional(0)); + return mcpp::doctor::why_report(parsed.positional(0), + parsed.option_or_empty("features").value()); } // Also called directly by the dispatcher for the legacy `--explain CODE` form. diff --git a/src/doctor.cppm b/src/doctor.cppm index 7cd87e48..7bd2848a 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -1012,7 +1012,7 @@ export int why_toolchain_json(std::string_view target, std::string_view tcSpec) return 0; } -export int why_report(const std::string& topic) { +export int why_report(const std::string& topic, const std::string& features) { const bool all = topic.empty() || topic == "all"; // The dedicated runtime view is a pure interpreter of the build's stored @@ -1020,7 +1020,13 @@ export int why_report(const std::string& topic) { // artifact re-parse is allowed on this path. if (topic == "runtime") return print_stored_runtime_resolution(); - auto ctx = mcpp::build::prepare_build(/*print_fingerprint=*/false); + // `--features` reaches the resolution this report reads (#649 E8): the + // graph of a feature build is a different graph, and before this it was + // visible only in the `resolution.json` of that build. + mcpp::build::BuildOverrides ov; + ov.features = features; + auto ctx = mcpp::build::prepare_build(/*print_fingerprint=*/false, + /*includeDevDeps=*/false, {}, ov); if (!ctx) { std::println(stderr, "error: {}", ctx.error()); return 2; } auto& tc = ctx->tc; From d1bcca76a49fb4f35da872a044578f7a48504b17 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:20:54 +0800 Subject: [PATCH 07/20] Tests for forwards, feature-deps restatements, tool packages, git members and dependency feature tokens (e2e 710-714, 187) --- tests/e2e/187_dep_host_tool.sh | 50 +++++ ..._forward_reaches_every_dependency_table.sh | 167 +++++++++++++++ ...ature_deps_restatement_names_one_source.sh | 112 ++++++++++ ...age_contributes_nothing_to_its_consumer.sh | 200 ++++++++++++++++++ ..._dependency_selects_a_repository_member.sh | 172 +++++++++++++++ ...714_features_names_a_dependency_feature.sh | 121 +++++++++++ tests/unit/test_manifest.cpp | 32 +++ 7 files changed, 854 insertions(+) create mode 100755 tests/e2e/710_a_forward_reaches_every_dependency_table.sh create mode 100755 tests/e2e/711_a_feature_deps_restatement_names_one_source.sh create mode 100755 tests/e2e/712_a_tool_package_contributes_nothing_to_its_consumer.sh create mode 100755 tests/e2e/713_a_git_dependency_selects_a_repository_member.sh create mode 100755 tests/e2e/714_features_names_a_dependency_feature.sh diff --git a/tests/e2e/187_dep_host_tool.sh b/tests/e2e/187_dep_host_tool.sh index 9ebb2368..af9b9995 100755 --- a/tests/e2e/187_dep_host_tool.sh +++ b/tests/e2e/187_dep_host_tool.sh @@ -221,4 +221,54 @@ cd nsapp out="$("$MCPP" run 2>&1 | grep '^G=' | tail -1)" [[ "$out" == "G=5" ]] || { echo "FAIL: namespaced tool did not generate: $out"; exit 1; } +# ── a package that writes `namespace` and `name` apart (#647 E4.3) ───────── +# `myns.tp` above is the legacy dotted name, whose `package.name` is already the +# qualified spelling. A manifest writing `namespace = "myns2"`, `name = "tp2"` +# has `package.name == "tp2"`, and the tool used to be published under that +# spelling alone, so `dep_bin("myns2.tp2", ...)` read nothing while +# `dep_dir("myns2.tp2")` answered. +cd "$TMP" +mkdir -p ns2/src +cat > ns2/mcpp.toml <<'EOF' +[package] +name = "tp2" +namespace = "myns2" +version = "0.1.0" + +[targets.gen] +kind = "bin" +main = "src/gen.cpp" +EOF +cp ns/src/gen.cpp ns2/src/gen.cpp +mkdir -p ns2app/src +cat > ns2app/mcpp.toml <<'EOF' +[package] +name = "ns2app" +version = "0.1.0" + +[dependencies] +myns2.tp2 = { path = "../ns2", tools = ["gen"] } +EOF +printf '#include \nint gv();\nint main(){std::printf("G2=%%d\\n",gv());}\n' > ns2app/src/main.cpp +cat > ns2app/build.mcpp <<'EOF' +#include +#include +#include +import mcpp; +int main() { + const char* full = mcpp::dep_bin("myns2.tp2", "gen"); + const char* brief = mcpp::dep_bin("tp2", "gen"); + if (!*full) { std::fprintf(stderr, "qualified spelling did not resolve\n"); return 1; } + if (!*brief) { std::fprintf(stderr, "name spelling did not resolve\n"); return 1; } + std::string out = std::string(mcpp::out_dir()) + "/g.cpp"; + std::string cmd = std::string("\"") + full + "\" \"" + out + "\""; + if (std::system(cmd.c_str()) != 0) return 1; + mcpp::generated(out.c_str()); +} +EOF +cd ns2app +"$MCPP" build > b7.log 2>&1 || { cat b7.log; echo "FAIL: a namespace + name package's tool was not addressable by its qualified name"; exit 1; } +out="$("$MCPP" run 2>&1 | grep '^G2=' | tail -1)" +[[ "$out" == "G2=5" ]] || { echo "FAIL: the namespace + name tool did not generate: $out"; exit 1; } + echo "OK" diff --git a/tests/e2e/710_a_forward_reaches_every_dependency_table.sh b/tests/e2e/710_a_forward_reaches_every_dependency_table.sh new file mode 100755 index 00000000..2d391a70 --- /dev/null +++ b/tests/e2e/710_a_forward_reaches_every_dependency_table.sh @@ -0,0 +1,167 @@ +#!/usr/bin/env bash +# requires: gcc +# 710 -- a feature forward is validated against every dependency table of its +# manifest (#647 E4.1). +# +# `injectForwards` applies a `[features]` forward to `[build-dependencies]` +# edges, while the validator used to look the key up in `[dependencies]` and +# `[dev-dependencies]` only. A two-level forward along build-dependency edges +# therefore opened the leaf's feature (its tool was built) and printed +# "not declared" for each level in the same run, and `--strict` refused the +# build whose forward had worked. A key declared only for another row is also +# declared: on this row the forward reaches no edge. +# +# Readings: the two-level forward builds the tool under `--strict` with no +# forward warning; a forward to a dependency declared for another row builds +# under `--strict`; a forward to a key no table declares still warns, and +# `--strict` refuses it. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p kt/src leaf/src rules/src fw/src app/src win/src +cat > kt/mcpp.toml <<'EOF' +[package] +name = "kt" +namespace = "spike" +version = "0.1.0" + +[targets.kt] +kind = "bin" +main = "src/main.cpp" +EOF +echo 'int main() { return 0; }' > kt/src/main.cpp + +cat > leaf/mcpp.toml <<'EOF' +[package] +name = "leaf" +namespace = "spike" +version = "0.1.0" + +[targets.leaf] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[features] +kt = [] + +[feature-deps.kt] +spike.kt = { path = "../kt", tools = ["kt"] } +EOF +echo 'int leaf_answer() { return 1; }' > leaf/src/leaf.cpp + +cat > rules/mcpp.toml <<'EOF' +[package] +name = "rules" +namespace = "spike" +version = "0.1.0" + +[targets.rules] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[build-dependencies] +spike.leaf = { path = "../leaf" } + +[features] +kotlin = ["spike.leaf/kt"] +EOF +echo 'int rules_answer() { return 1; }' > rules/src/rules.cpp + +cat > win/mcpp.toml <<'EOF' +[package] +name = "win" +namespace = "spike" +version = "0.1.0" + +[targets.win] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[features] +x = [] +EOF +echo 'int win_answer() { return 1; }' > win/src/win.cpp + +cat > fw/mcpp.toml <<'EOF' +[package] +name = "fw" +namespace = "spike" +version = "0.1.0" + +[targets.fw] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[build-dependencies] +spike.rules = { path = "../rules" } + +[target.'cfg(os = "windows")'.dependencies] +spike.win = { path = "../win" } + +[features] +kotlin = ["spike.rules/kotlin", "spike.win/x"] +EOF +echo 'int fw_answer() { return 42; }' > fw/src/fw.cpp + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw = { path = "../fw", features = ["kotlin"] } +EOF +cat > app/src/main.cpp <<'EOF' +int fw_answer(); +int main() { return fw_answer() == 42 ? 0 : 1; } +EOF + +# ── 1. two levels along [build-dependencies], strict, no forward warning ── +cd "$TMP/app" +"$MCPP" build --strict > b1.log 2>&1 || fail "the two-level build-dependency forward was refused under --strict" b1.log +grep -q "forwards to dependency" b1.log \ + && fail "a forward along a build-dependency edge was reported as undeclared" b1.log +grep -q "Building host tool kt" b1.log \ + || fail "the forwarded feature did not reach the leaf (its tool was not built)" b1.log + +# ── 2. a forward to a dependency declared only for another row ─────────── +# `spike.win` is declared for Windows only (this script runs where gcc is the +# host compiler, which is never a Windows row). The forward names it, reaches +# no edge here, and must be accepted: the manifest is portable, not wrong. +grep -q "spike.win" b1.log && fail "a row-declared dependency's forward was reported" b1.log + +# ── 3. a key no table declares is still named, and --strict refuses it ─── +sed -i.bak 's|kotlin = \["spike.rules/kotlin", "spike.win/x"\]|kotlin = ["spike.rules/kotlin", "spike.nowhere/x"]|' "$TMP/fw/mcpp.toml" +rm -rf target +"$MCPP" build > b3.log 2>&1 || fail "an undeclared forward failed a non-strict build" b3.log +grep -q "forwards to dependency 'spike.nowhere'" b3.log \ + || fail "a forward to an undeclared key was not named" b3.log +if "$MCPP" build --strict > b4.log 2>&1; then + fail "--strict accepted a forward to a key no table declares" b4.log +fi +grep -q "spike.nowhere" b4.log || fail "the --strict refusal does not name the key" b4.log + +echo "PASS: 710 a forward is validated against every dependency table" diff --git a/tests/e2e/711_a_feature_deps_restatement_names_one_source.sh b/tests/e2e/711_a_feature_deps_restatement_names_one_source.sh new file mode 100755 index 00000000..bdb924d8 --- /dev/null +++ b/tests/e2e/711_a_feature_deps_restatement_names_one_source.sh @@ -0,0 +1,112 @@ +#!/usr/bin/env bash +# requires: gcc +# 711 -- a `[feature-deps.]` entry that adds `tools` to a dependency declared +# unconditionally restates that dependency's source, and a restatement naming +# another source is refused (#647 E4.2). A build program addresses the tool of a +# package written `namespace = "spike"`, `name = "installer"` by the qualified +# name as well as the bare one (#647 E4.3). +# +# The grammar requires the source in a dependency table, so a source-less entry +# is refused by the parser, whose message prescribes restating it. The merge +# took only the additive fields of a restatement: one naming another path was +# ignored without a word, under `--strict` too. And the tool variable was +# published under `package.name` alone, so `dep_bin("spike.installer", ...)` +# read nothing while `dep_dir("spike.installer")` answered. +# +# Readings: the source-less entry is refused and names the remedy; the +# restatement with the same source builds the tool only under the feature, and +# the build program sees both spellings; a restatement naming another path is +# refused under `--strict` naming both sources. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p installer/src other/src app/src +for d in installer other; do +cat > $d/mcpp.toml <<'EOF' +[package] +name = "installer" +namespace = "spike" +version = "0.1.0" + +[targets.installer] +kind = "bin" +main = "src/main.cpp" +EOF +cat > $d/src/main.cpp <<'EOF' +#include +int main() { std::puts("installer ran"); return 0; } +EOF +done + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.installer = { path = "../installer" } + +[features] +installer = [] + +[feature-deps.installer] +spike.installer = { tools = ["installer"] } +EOF +echo 'int main() { return 0; }' > app/src/main.cpp +cat > app/build.mcpp <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("installer", "installer"); + const char* b = mcpp::dep_bin("spike.installer", "installer"); + mcpp::warning((std::string("DEPBIN short=[") + (a ? a : "") + "] qualified=[" + + (b ? b : "") + "]").c_str()); + return 0; +} +EOF + +cd "$TMP/app" + +# ── 1. a source-less entry is refused, and the refusal names the remedy ── +if "$MCPP" build --features installer > b1.log 2>&1; then + fail "a source-less [feature-deps] entry was accepted" b1.log +fi +grep -q "source" b1.log || fail "the refusal does not say to restate the source" b1.log + +# ── 2. the same source restated: the tool, both spellings, only under the feature +sed -i.bak 's|spike.installer = { tools = \["installer"\] }|spike.installer = { path = "../installer", tools = ["installer"] }|' mcpp.toml +"$MCPP" build --strict --features installer > b2.log 2>&1 \ + || fail "the restated feature-deps entry was refused under --strict" b2.log +grep -q 'DEPBIN short=\[[^]]' b2.log || fail "dep_bin(\"installer\") read nothing" b2.log +grep -q 'qualified=\[[^]]' b2.log \ + || fail "dep_bin(\"spike.installer\") read nothing for a namespace + name package" b2.log +rm -rf target +"$MCPP" build --strict > b3.log 2>&1 || fail "the build without the feature failed" b3.log +grep -q 'DEPBIN short=\[\] qualified=\[\]' b3.log \ + || fail "the tool was provided without its feature" b3.log + +# ── 3. a restatement naming another source is refused, naming both ─────── +sed -i.bak 's|spike.installer = { path = "../installer", tools|spike.installer = { path = "../other", tools|' mcpp.toml +rm -rf target +if "$MCPP" build --strict --features installer > b4.log 2>&1; then + fail "a restatement naming another path was accepted" b4.log +fi +grep -q '../other' b4.log && grep -q '../installer' b4.log \ + || fail "the refusal does not name both sources" b4.log + +echo "PASS: 711 a feature-deps restatement names one source; dep_bin answers both spellings" diff --git a/tests/e2e/712_a_tool_package_contributes_nothing_to_its_consumer.sh b/tests/e2e/712_a_tool_package_contributes_nothing_to_its_consumer.sh new file mode 100755 index 00000000..8a59f1aa --- /dev/null +++ b/tests/e2e/712_a_tool_package_contributes_nothing_to_its_consumer.sh @@ -0,0 +1,200 @@ +#!/usr/bin/env bash +# requires: gcc +# 712 -- a package whose declared targets are all programs contributes nothing +# to a consumer's graph; its tools come from the tool sub-build (#649 E6). +# +# An SDK provides a program behind a feature, and the program depends on the +# SDK. The tool sub-build resolves the program's package as its own root, so +# the program and the SDK it links are built there. The consumer's graph used +# to walk the program package's `[dependencies]` as well, which made the +# declaration a cycle of the consumer's graph ("dependency cycle through +# package 'fw' while computing its build-cache key", default cache only) and +# linked a tool's own library into the application. +# +# Readings: the SDK fixture builds with the default cache, the tool runs, and +# the build program reads its path under both spellings; a tool's own library +# is not linked into the application; a cycle among libraries is refused with +# the same message under both cache modes; a tool whose own build activates +# the feature that requests it is refused once, naming the repetition. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +# fixture +fixture() { + local W=$1 + mkdir -p "$W/fw/src" "$W/fw/tool/src" "$W/app/src" "$W/z/src" + cat > "$W/fw/mcpp.toml" <<'EOF' +[package] +name = "fw" +namespace = "spike" +version = "0.1.0" + +[targets.fw] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[features] +installer = [] + +[feature-deps.installer] +spike.fw-installer = { path = "tool", tools = ["fw-installer"], reexport = true } +EOF + echo 'int fw_answer() { return 42; }' > "$W/fw/src/fw.cpp" + cat > "$W/fw/tool/mcpp.toml" < "$W/z/mcpp.toml" <<'EOF' +[package] +name = "z" +namespace = "spike" +version = "0.1.0" + +[targets.z] +kind = "lib" + +[build] +sources = ["src/*.cpp"] +EOF + echo 'int z_only_in_the_tool() { return 7; }' > "$W/z/src/z.cpp" + cat > "$W/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw = { path = "../fw" } + +[features] +windows-installer = ["spike.fw/installer"] +EOF + cat > "$W/app/src/main.cpp" <<'EOF' +int fw_answer(); +int main() { return fw_answer() == 42 ? 0 : 1; } +EOF + cat > "$W/app/build.mcpp" <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("fw-installer", "fw-installer"); + const char* b = mcpp::dep_bin("spike.fw-installer", "fw-installer"); + mcpp::warning((std::string("DEPBIN short=[") + (a ? a : "") + "] qualified=[" + + (b ? b : "") + "]").c_str()); + return 0; +} +EOF +} + +# ── 1. the SDK shape: the tool depends on the package declaring it ──────── +fixture "$TMP/s1" 'spike.fw = { path = ".." }' +cat > "$TMP/s1/fw/tool/src/main.cpp" <<'EOF' +#include +int fw_answer(); +int main() { std::printf("fw-installer ran %d\n", fw_answer()); return 0; } +EOF +cd "$TMP/s1/app" +"$MCPP" build --features windows-installer > b1.log 2>&1 \ + || fail "a feature tool depending on its declaring package did not build" b1.log +tool=$(grep -o 'DEPBIN short=\[[^]]*' b1.log | sed 's/DEPBIN short=\[//') +[ -n "$tool" ] || fail "dep_bin(\"fw-installer\") read nothing" b1.log +grep -q 'qualified=\[[^]]' b1.log || fail "dep_bin(\"spike.fw-installer\") read nothing" b1.log +out=$("$tool") || fail "the tool did not run: $out" +[ "$out" = "fw-installer ran 42" ] || fail "the tool printed '$out'" +"$MCPP" run > r1.log 2>&1 || fail "the application does not run" r1.log + +# ── 2. a tool's own library does not reach the application ────────────── +fixture "$TMP/s2" 'spike.z = { path = "../../z" }' +cat > "$TMP/s2/fw/tool/src/main.cpp" <<'EOF' +#include +int z_only_in_the_tool(); +int main() { std::printf("tool %d\n", z_only_in_the_tool()); return 0; } +EOF +cd "$TMP/s2/app" +"$MCPP" build --features windows-installer > b2.log 2>&1 || fail "the tool-with-a-library build failed" b2.log +ninja_file=$(find target -name build.ninja | head -1) +grep '^build bin/app' "$ninja_file" | grep -q 'spike_z' \ + && fail "the tool's own library was linked into the application" "$ninja_file" +grep -q 'spike_fw-installer' "$ninja_file" \ + && fail "the tool package's sources were compiled in the consumer's build" "$ninja_file" + +# ── 3. a cycle among libraries is refused under both cache modes ───────── +mkdir -p "$TMP/s3/a/src" "$TMP/s3/b/src" "$TMP/s3/app/src" +for p in a b; do + q=$([ "$p" = a ] && echo b || echo a) + cat > "$TMP/s3/$p/mcpp.toml" < "$TMP/s3/$p/src/$p.cpp" +done +cat > "$TMP/s3/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.a = { path = "../a" } +EOF +echo 'int main() { return 0; }' > "$TMP/s3/app/src/main.cpp" +cd "$TMP/s3/app" +for mode in "" "--cache=local"; do + if "$MCPP" build $mode > b3.log 2>&1; then + fail "a cycle of packages was accepted (${mode:-default cache})" b3.log + fi + grep -q "dependency cycle: 'spike.a' -> 'spike.b' -> 'spike.a'" b3.log \ + || fail "the cycle refusal does not name its edges (${mode:-default cache})" b3.log +done + +# ── 4. a tool whose own build requests it again is refused once ────────── +fixture "$TMP/s4" 'spike.fw = { path = "..", features = ["installer"] }' +echo 'int main() { return 0; }' > "$TMP/s4/fw/tool/src/main.cpp" +cd "$TMP/s4/app" +if "$MCPP" build --features windows-installer > b4.log 2>&1; then + fail "a tool requesting itself through its own build was accepted" b4.log +fi +grep -q "is requested by its own build" b4.log || fail "the repetition is not named" b4.log +[ "$(grep -o 'building host tool' b4.log | wc -l | tr -d ' ')" -le 1 ] \ + || fail "the repetition was refused only after nested sub-builds" b4.log + +echo "PASS: 712 a package of programs contributes nothing to its consumer's graph" diff --git a/tests/e2e/713_a_git_dependency_selects_a_repository_member.sh b/tests/e2e/713_a_git_dependency_selects_a_repository_member.sh new file mode 100755 index 00000000..695dbfe6 --- /dev/null +++ b/tests/e2e/713_a_git_dependency_selects_a_repository_member.sh @@ -0,0 +1,172 @@ +#!/usr/bin/env bash +# requires: gcc +# 713 -- a git dependency whose key names a package other than the repository's +# root package selects the `[workspace] members` entry that declares it +# (#649 E7). A second declaration of one dependency by one consumer keeps its +# requests, and the compile banner of a git dependency names its reference. +# +# A framework repository holds the framework, its rules and its tools, and a +# consumer pins it by revision before it is published. The git source always +# yielded the root package, so a key naming a member was either refused as a +# name mismatch or, beside a key naming the root, adopted the root's identity +# with a warning and lost its `tools` request. The same early return dropped +# the requests of a dependency named in `[dependencies]` and again in +# `[build-dependencies]`. +# +# Readings: the member alone builds its tool; the root and the member by the +# same revision build without an identity warning and with the member's tool; +# the banner prints the reference instead of an empty version; a dependency +# declared in both tables gets the tool the second declaration requests. +set -e + +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +command -v git >/dev/null || { echo "SKIP: git is not installed"; exit 0; } + +mkdir -p repo/src repo/tool/src +cat > repo/mcpp.toml <<'EOF' +[package] +name = "fw" +namespace = "spike" +version = "0.1.0" + +[targets.fw] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[workspace] +members = ["tool"] +EOF +echo 'int fw_answer() { return 42; }' > repo/src/fw.cpp +cat > repo/tool/mcpp.toml <<'EOF' +[package] +name = "fw-installer" +namespace = "spike" +version = "0.1.0" + +[targets.fw-installer] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw = { path = ".." } +EOF +cat > repo/tool/src/main.cpp <<'EOF' +#include +int fw_answer(); +int main() { std::printf("fw-installer ran %d\n", fw_answer()); return 0; } +EOF +( + cd repo + git init -q + git -c user.email=e2e@mcpp.dev -c user.name=e2e add -A + git -c user.email=e2e@mcpp.dev -c user.name=e2e commit -qm fixture +) +REV=$(git -C repo rev-parse HEAD) +REPO_HOST=$(host_path "$TMP/repo") + +write_app() { + mkdir -p "$TMP/$1/src" + cat > "$TMP/$1/mcpp.toml" + echo 'int main() { return 0; }' > "$TMP/$1/src/main.cpp" + cat > "$TMP/$1/build.mcpp" <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("spike.fw-installer", "fw-installer"); + mcpp::warning((std::string("DEPBIN tool=[") + (a ? a : "") + "]").c_str()); + return 0; +} +EOF +} + +# ── 1. the member alone ───────────────────────────────────────────────── +write_app member < b1.log 2>&1 || fail "a git dependency naming a repository member was refused" b1.log +tool=$(grep -o 'DEPBIN tool=\[[^]]*' b1.log | sed 's/DEPBIN tool=\[//') +[ -n "$tool" ] || fail "the member's tool was not provided" b1.log +[ "$("$tool")" = "fw-installer ran 42" ] || fail "the member's tool did not run" + +# ── 2. the root and the member by the same revision ───────────────────── +write_app both < "$TMP/both/src/main.cpp" <<'EOF' +int fw_answer(); +int main() { return fw_answer() == 42 ? 0 : 1; } +EOF +cd "$TMP/both" +"$MCPP" build > b2.log 2>&1 || fail "the root and a member by one revision did not build" b2.log +grep -q "that identity is used" b2.log \ + && fail "the member's key adopted the root's identity" b2.log +grep -q 'DEPBIN tool=\[[^]]' b2.log || fail "the member's tools request was lost" b2.log +grep -q "Compiling spike.fw v$" b2.log && fail "the git banner printed an empty version" b2.log +grep -q "spike.fw (git rev ${REV:0:12})" b2.log \ + || fail "the git banner does not name the reference" b2.log +"$MCPP" run > r2.log 2>&1 || fail "the application over the git root does not run" r2.log + +# ── 3. one dependency in [dependencies] and in [build-dependencies] ───── +mkdir -p "$TMP/local/src" +cp -r "$TMP/repo/tool/." "$TMP/local/" +sed -i.bak 's|spike.fw = { path = ".." }||' "$TMP/local/mcpp.toml" +cat > "$TMP/local/src/main.cpp" <<'EOF' +#include +int main() { std::puts("local tool"); return 0; } +EOF +write_app twice <<'EOF' +[package] +name = "twice" +version = "0.1.0" + +[targets.twice] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw-installer = { path = "../local" } + +[build-dependencies] +spike.fw-installer = { path = "../local", tools = ["fw-installer"] } +EOF +cd "$TMP/twice" +"$MCPP" build > b3.log 2>&1 || fail "a dependency declared in two tables did not build" b3.log +grep -q 'DEPBIN tool=\[[^]]' b3.log \ + || fail "the second declaration's tools request was dropped" b3.log + +echo "PASS: 713 a git dependency selects a repository member; a second declaration keeps its requests" diff --git a/tests/e2e/714_features_names_a_dependency_feature.sh b/tests/e2e/714_features_names_a_dependency_feature.sh new file mode 100755 index 00000000..4a9df96e --- /dev/null +++ b/tests/e2e/714_features_names_a_dependency_feature.sh @@ -0,0 +1,121 @@ +#!/usr/bin/env bash +# requires: gcc +# 714 -- `--features /` opens a feature of a dependency, as +# the same token does inside `[features]` (#649 E8). +# +# The command line accepted root features only. With no `[features]` table the +# token became the macro `-DMCPP_FEATURE_SPIKE_FW_INSTALLER` and opened nothing; +# with the table it was reported as an undeclared feature of the root. A plain +# name without a `[features]` table keeps its documented meaning, a macro. +# +# Readings: without a `[features]` table the dependency's tool is built and no +# macro is made of the token; with the table the build is clean under +# `--strict`; a token naming no dependency is refused under `--strict`; and +# `mcpp why deps --features` reports the graph of the feature build. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p fw/src fw/tool/src app/src +cat > fw/mcpp.toml <<'EOF' +[package] +name = "fw" +namespace = "spike" +version = "0.1.0" + +[targets.fw] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[features] +installer = [] + +[feature-deps.installer] +spike.fw-installer = { path = "tool", tools = ["fw-installer"], reexport = true } +EOF +echo 'int fw_answer() { return 42; }' > fw/src/fw.cpp +cat > fw/tool/mcpp.toml <<'EOF' +[package] +name = "fw-installer" +namespace = "spike" +version = "0.1.0" + +[targets.fw-installer] +kind = "bin" +main = "src/main.cpp" +EOF +echo 'int main() { return 0; }' > fw/tool/src/main.cpp + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw = { path = "../fw" } +EOF +cat > app/src/main.cpp <<'EOF' +int fw_answer(); +int main() { return fw_answer() == 42 ? 0 : 1; } +EOF +cat > app/build.mcpp <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("spike.fw-installer", "fw-installer"); + mcpp::warning((std::string("DEPBIN tool=[") + (a ? a : "") + "]").c_str()); + return 0; +} +EOF +cd "$TMP/app" + +# ── 1. no [features] table: the dependency's feature opens, no macro ───── +"$MCPP" build --features spike.fw/installer > b1.log 2>&1 || fail "the dependency feature build failed" b1.log +grep -q 'DEPBIN tool=\[[^]]' b1.log || fail "--features spike.fw/installer did not open the feature" b1.log +ninja_file=$(find target -name build.ninja | head -1) +grep -q 'MCPP_FEATURE_SPIKE_FW_INSTALLER' "$ninja_file" \ + && fail "the dependency token became a macro of the root" "$ninja_file" + +# A plain undeclared name without a [features] table is documented as a macro. +rm -rf target +"$MCPP" build --strict --features nothing-here > b2.log 2>&1 \ + || fail "a plain name without a [features] table was refused" b2.log + +# ── 2. a token naming no dependency is refused under --strict ─────────── +if "$MCPP" build --strict --features nope/x > b3.log 2>&1; then + fail "--features nope/x was accepted under --strict" b3.log +fi +grep -q "nope" b3.log || fail "the refusal does not name the token" b3.log + +# ── 3. with a [features] table the token is not reported as undeclared ── +cat >> mcpp.toml <<'EOF' + +[features] +windows-installer = [] +EOF +rm -rf target +"$MCPP" build --strict --features spike.fw/installer > b4.log 2>&1 \ + || fail "the dependency token was refused beside a [features] table" b4.log +grep -q 'DEPBIN tool=\[[^]]' b4.log || fail "the feature did not open beside a [features] table" b4.log + +# ── 4. `why deps` reads the graph of the feature build ────────────────── +"$MCPP" why deps --features spike.fw/installer > w1.log 2>&1 || fail "why deps --features failed" w1.log +grep -q 'spike.fw-installer' w1.log || fail "why deps does not show the feature's dependency" w1.log + +echo "PASS: 714 --features opens a dependency's feature" diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 144ed77d..a1ac3a65 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -5686,3 +5686,35 @@ kind = "shared" EXPECT_EQ(m.targets[0].kind, mcpp::manifest::Target::Library); EXPECT_EQ(m.targets[0].linkageDefault, "shared"); } + +// #649 E6: a consumer reads "every declared target is a program" as "a tool +// provider that contributes nothing to my graph". A target list the loader +// INFERRED from the tree is not that statement, so the loader says which of +// the two it produced. +TEST(Manifest, TargetsInferredIsDistinguishedFromDeclared) { + auto dir = std::filesystem::temp_directory_path() + / std::format("mcpp_targets_inferred_{}", std::random_device{}()); + std::filesystem::create_directories(dir / "src"); + { + std::ofstream(dir / "src" / "main.cpp") << "int main() {}\n"; + std::ofstream(dir / "mcpp.toml") << "[package]\nname = \"inferred\"\nversion = \"0.1.0\"\n"; + } + auto inferred = mcpp::manifest::load(dir / "mcpp.toml"); + ASSERT_TRUE(inferred) << (inferred ? "" : inferred.error().message); + ASSERT_EQ(inferred->targets.size(), 1u); + EXPECT_EQ(inferred->targets[0].kind, mcpp::manifest::Target::Binary); + EXPECT_TRUE(inferred->targetsInferred); + + { + std::ofstream(dir / "mcpp.toml") + << "[package]\nname = \"declared\"\nversion = \"0.1.0\"\n\n" + "[targets.declared]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n"; + } + auto declared = mcpp::manifest::load(dir / "mcpp.toml"); + ASSERT_TRUE(declared) << (declared ? "" : declared.error().message); + ASSERT_EQ(declared->targets.size(), 1u); + EXPECT_FALSE(declared->targetsInferred); + + std::error_code ec; + std::filesystem::remove_all(dir, ec); +} From 95fad451fd76b06e292fc9355290a5ef55bfa543 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:20:54 +0800 Subject: [PATCH 08/20] docs: forwards and dependency feature tokens, feature-deps restatements, packages of programs, git members, one naming derivation, package-cycle --- docs/05-dependencies.md | 70 +++++++++++++++++++++++-- docs/06-features-and-capabilities.md | 27 ++++++++++ docs/07-workspace.md | 6 +++ docs/30-build-mcpp.md | 7 +++ docs/50-machine-output.md | 1 + docs/zh/05-dependencies.md | 53 +++++++++++++++++-- docs/zh/06-features-and-capabilities.md | 21 ++++++++ docs/zh/07-workspace.md | 5 ++ docs/zh/30-build-mcpp.md | 5 ++ docs/zh/50-machine-output.md | 1 + 10 files changed, 189 insertions(+), 7 deletions(-) diff --git a/docs/05-dependencies.md b/docs/05-dependencies.md index 3ca04056..07bd1691 100644 --- a/docs/05-dependencies.md +++ b/docs/05-dependencies.md @@ -167,6 +167,34 @@ source: the build is refused before scanning, naming both, and the fix is a `namespace` in that manifest or one key in both places. A `version` dependency is unaffected; its identity is the key. +### A package of a git repository (mcpp 2026.9.16.1+) + +A `git` dependency names a repository, and its key names which package of the +repository is meant. The root manifest's package is one; each entry of that +manifest's `[workspace] members` is another. A key whose identity is not the +root package's selects the member whose manifest declares it, at the same +commit: + +```toml +# repo/mcpp.toml declares spike.fw and [workspace] members = ["tool"]; +# repo/tool/mcpp.toml declares spike.fw-installer +[dependencies] +spike.fw = { git = "https://example.org/fw.git", rev = "cc3c74c5" } +spike.fw-installer = { git = "https://example.org/fw.git", rev = "cc3c74c5", tools = ["fw-installer"] } +``` + +- The member inherits the repository's `[workspace.package]`, as it does when + the repository is built from its own checkout. +- A member's `path` edge that stays inside the clone, such as + `spike.fw = { path = ".." }`, names the same git source at the same commit, + so it is the package the root's key resolved rather than a second, + path-sourced declaration of it. +- A key that names neither the root package nor a member keeps the rule of the + section above: the root manifest's identity is used, with the warning. + +A key selects by identity, and no `subdir` key exists: an older client would +ignore such a key and build the root package without a word. + ### Namespace resolution rules Every package has a two-part identity: a **namespace** and a **name**. Every @@ -335,15 +363,49 @@ protobuf = { version = "35.1", tools = ["protoc"] } express — a package whose library must not reach the target while its tool or its rule is still wanted. Naming one package in both tables is not an error: the ordinary declaration wins, because a `[build-dependencies]` line must not -quietly drop a library the target needs. +quietly drop a library the target needs, and what each declaration requests +(`tools`, `features`, `host-module`, `reexport`) is requested of the one edge +(mcpp 2026.9.16.1+; before that release the second declaration's requests were +dropped). + +**A package of programs contributes only its programs (mcpp 2026.9.16.1+).** A +dependency whose declared `[targets]` are all programs (`bin`, `app`, `test`) +has nothing to link. Its tools are built by the tool sub-build, which resolves +the package as its own root, and in the consumer's graph it provides its tools +and its directory and nothing else: its own dependencies are not resolved +there, its sources are not compiled there, and its `ldflags` do not reach the +consumer's link. A program may therefore depend on the package that requests +it, which is how an SDK provides a program built against itself. A package +that declares no `[targets]` table is unaffected, even when a `src/main.cpp` +infers a program for it. + +A cycle among packages is refused where the graph is resolved, naming its +edges, under every cache mode; a tool whose own sub-build requests it again is +refused at that first repetition, naming the chain. Unlike `[dev-dependencies]`, these **are** walked transitively: a build dependency's own dependencies are what make it work, and they inherit its build-only nature. -A feature scopes build-time requests without a second declaration site — -`[feature-deps.]` may add `tools` to a dependency already declared -unconditionally, so "only when needed" needs no separate table. +A feature scopes build-time requests without a second declaration site. A +`[feature-deps.]` entry may restate a dependency already declared +unconditionally, with the same source, and add `tools` to it, so "only when +needed" needs no separate table: + +```toml +[dependencies] +spike.installer = { path = "../installer" } + +[feature-deps.installer] +spike.installer = { path = "../installer", tools = ["installer"] } +``` + +The restatement names its source because every dependency table does: an entry +without `path`, `git`, `version` or `workspace` is read as a namespace table +and refused, and the refusal says to restate the source. `tools`, `features`, +`host-module` and `reexport` of the restatement are added to the declaration in +effect on the row. A restatement that names another source is refused, naming +both sources (mcpp 2026.9.16.1+); before that release it was ignored. > The section has been parsed since early versions and, until 2026.8.29.1, read > by nothing that made a decision: writing it produced a manifest that loaded, diff --git a/docs/06-features-and-capabilities.md b/docs/06-features-and-capabilities.md index ba15baa5..6871b01c 100644 --- a/docs/06-features-and-capabilities.md +++ b/docs/06-features-and-capabilities.md @@ -41,6 +41,33 @@ extra = [] requesting an undeclared feature produces a warning; an error under `--strict`. A package that does not declare `[features]` accepts any request (pure macro usage). +### A feature of a dependency + +A token `/` opens a feature of a dependency. The dependency +is named by the key its consumer's manifest writes (`spike.fw`, `compat.opencv`), +and the token is additive: it opens more of the dependency, never pulls the +dependency in. + +```toml +[dependencies] +spike.fw = { path = "../fw" } + +[features] +windows-installer = ["spike.fw/installer"] # the same as `forward = [...]` in the table form +``` + +- **On the command line** (mcpp 2026.9.16.1+), `mcpp build --features spike.fw/installer` + opens the same feature for one command, as a forward of the root does; + `run`, `test`, `pack`, `emit build-database` and `why deps` accept the token + alike. It is never a feature of the root and never becomes a macro. +- **Validation** reads every dependency table of the manifest that writes the + forward: `[dependencies]`, `[build-dependencies]`, `[dev-dependencies]` and + `[feature-deps.]`, on every row. A key declared only for another row or + under an inactive feature is declared; on this row the forward reaches no edge + and does nothing. A key no table declares is reported, and the report is an + error under `--strict`. On the command line the same check runs whether or not + the root declares `[features]`. + ### Table form — a feature that contributes more than implied features A `[features]` entry may be written as a **table** instead of an array, letting the diff --git a/docs/07-workspace.md b/docs/07-workspace.md index f1020e0b..c8dcf220 100644 --- a/docs/07-workspace.md +++ b/docs/07-workspace.md @@ -372,6 +372,12 @@ myproject/ Each member's build artifacts live under its own `target/` subdirectory. +A project outside the workspace reaches a member of a git-hosted workspace by +the member's identity: `myproject.http = { git = "...", rev = "..." }` selects +`libs/http` among the root manifest's `members`, at the same commit, and the +member inherits `[workspace.package]` as it does here (mcpp 2026.9.16.1+; see +[05 — Dependencies](05-dependencies.md)). + ## 7. Relationship to C++ Modules Workspaces work in concert with the C++23 module mechanism: diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 798a51fd..0108b7dc 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -1399,6 +1399,13 @@ int main() { return grpcgen::generate_all() ? 0 : 1; } bare spelling is bound to `mcpplibs.`, else `compat.`, else an unnamespaced ``, else the single remaining candidate — and when it is contested mcpp says so instead of picking silently. +- **A tool is addressed by exactly the names its directory is.** One derivation + names a provider for `dep_dir`, `dep_linkage` and `dep_bin`: the manifest's + `name`, the qualified `namespace.name`, and the bound tail. A package that + writes `namespace = "spike"` and `name = "installer"` apart is therefore + `dep_bin("spike.installer", ...)` as well as `dep_bin("installer", ...)` + (mcpp 2026.9.16.1+; before that release its tool was published under + `installer` alone). #### Older mcpp reading a manifest that uses this diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 1f82c647..ca6c6327 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -395,6 +395,7 @@ a program classifying the outcome reads `reason`: | `host-module-missing` | `build.mcpp` imports a module no dependency supplies as a host module | | `tool-version-conflict` | two declarations name one xlings package at versions that cannot both hold | | `shared-library-cxx-runtime` | a dependency's C++ shared library in a graph whose C++ runtime is a package, with no private copy stated | +| `package-cycle` | the dependency graph contains a cycle of packages; the message names its edges | | `other` | a refusal whose branch has not been given a token yet | **Exit 0 whenever the question was answered, including "refused".** "Would diff --git a/docs/zh/05-dependencies.md b/docs/zh/05-dependencies.md index c8363d79..a95ac877 100644 --- a/docs/zh/05-dependencies.md +++ b/docs/zh/05-dependencies.md @@ -150,6 +150,27 @@ warning: 'huxdemo.comp@path' declares the dependency 'fw', which names mcpplibs. 点名二者;修正方式是在该清单中声明 `namespace`,或两处写同一个键。`version` 依赖 不受影响,它的身份就是键。 +### git 仓库中的一个包(mcpp 2026.9.16.1+) + +`git` 依赖指向一个仓库,键说明指的是仓库里的哪一个包。根清单的包是其一;根清单 +`[workspace] members` 的每一项是另一个。键的身份不是根包时,选中清单声明该身份的那个 +member,提交相同: + +```toml +# repo/mcpp.toml 声明 spike.fw 与 [workspace] members = ["tool"]; +# repo/tool/mcpp.toml 声明 spike.fw-installer +[dependencies] +spike.fw = { git = "https://example.org/fw.git", rev = "cc3c74c5" } +spike.fw-installer = { git = "https://example.org/fw.git", rev = "cc3c74c5", tools = ["fw-installer"] } +``` + +- member 继承仓库的 `[workspace.package]`,与从仓库自己的检出构建时相同。 +- member 中留在克隆目录之内的 `path` 边(例如 `spike.fw = { path = ".." }`)指向同一 + git 源的同一提交,因此它就是根的键解析到的那个包,而不是第二条以 path 为源的声明。 +- 既不指根包也不指任何 member 的键沿用上一节的规则:使用根清单的身份,并给出警告。 + +键按身份选择,不存在 `subdir` 键:较旧的客户端会忽略这样的键,并不声不响地构建根包。 + ### 命名空间解析规则 每个包的身份是**命名空间 + 名字**二元组。每个 selector 都只规范化成一个身份: @@ -296,13 +317,39 @@ protobuf = { version = "35.1", tools = ["protoc"] } `[build-dependencies]` 用于第一个轴无法表达的那种组合 —— 某个包的库不得进入目标, 而它的工具或规则仍然需要。同一个包同时出现在两张表里不是错误:普通声明胜出,因为 -一行 `[build-dependencies]` 不应该悄悄拿掉目标真正需要的库。 +一行 `[build-dependencies]` 不应该悄悄拿掉目标真正需要的库;两条声明各自的请求 +(`tools`、`features`、`host-module`、`reexport`)都作用于这一条边(mcpp 2026.9.16.1+; +此前第二条声明的请求被丢弃)。 + +**只含程序的包只贡献它的程序(mcpp 2026.9.16.1+)。** 声明的 `[targets]` 全部是程序 +(`bin`、`app`、`test`)的依赖没有可链接的东西。它的工具由工具子构建构建,子构建把这个包 +当作自己的根来解析;在消费方的图里它只提供工具与目录:它自己的依赖不在那里解析,它的源码 +不在那里编译,它的 `ldflags` 不进入消费方的链接。因此一个程序可以依赖请求它的那个包, +SDK 正是这样提供一个针对自身构建的程序。没有 `[targets]` 表的包不受影响,即使 +`src/main.cpp` 为它推断出一个程序。 + +包之间的环在解析依赖图的地方被拒绝,并列出环上的边,与缓存模式无关;工具子构建再次请求 +正在构建的同一个工具时,在第一次重复处被拒绝,并给出请求链。 与 `[dev-dependencies]` 不同,这些依赖**会**被传递遍历:一个构建期依赖自己的依赖正是 让它能工作的东西,并且继承它「只服务构建」的性质。 -feature 可以为构建期请求划定范围而无需第二个声明处 —— `[feature-deps.]` 可以给 -一条已经无条件声明的依赖追加 `tools`,所以「按需才要」不需要另开一张表。 +feature 可以为构建期请求划定范围而无需第二个声明处。`[feature-deps.]` 的条目可以 +以相同的源重述一条已经无条件声明的依赖,并为它追加 `tools`,所以「按需才要」不需要另开 +一张表: + +```toml +[dependencies] +spike.installer = { path = "../installer" } + +[feature-deps.installer] +spike.installer = { path = "../installer", tools = ["installer"] } +``` + +重述要写出源,因为每张依赖表都如此:没有 `path`、`git`、`version` 或 `workspace` 的条目 +被当作命名空间表读取并被拒绝,拒绝消息会说明要重述源。重述中的 `tools`、`features`、 +`host-module` 与 `reexport` 加到该行生效的声明上。重述写了另一个源时被拒绝,并列出两个源 +(mcpp 2026.9.16.1+);此前它被忽略。 > 这个段很早就能被解析,而直到 2026.8.29.1 之前没有任何做决定的代码读它:写下它得到的是 > 一份能加载的清单、零诊断、零效果。 diff --git a/docs/zh/06-features-and-capabilities.md b/docs/zh/06-features-and-capabilities.md index 21360dc2..862a87d3 100644 --- a/docs/zh/06-features-and-capabilities.md +++ b/docs/zh/06-features-and-capabilities.md @@ -35,6 +35,27 @@ extra = [] - **严格校验**:目标包声明了 `[features]` 表时,请求一个未声明的 feature 产生 警告,在 `--strict` 下是错误。不声明 `[features]` 的包接受任意请求(纯宏用法)。 +### 依赖的 feature + +`<依赖>/` 形式的记号打开某个依赖的 feature。依赖以消费方清单所写的键命名 +(`spike.fw`、`compat.opencv`),记号是可加的:它只打开依赖更多的部分,从不把依赖拉进来。 + +```toml +[dependencies] +spike.fw = { path = "../fw" } + +[features] +windows-installer = ["spike.fw/installer"] # 与表形式中的 `forward = [...]` 相同 +``` + +- **命令行上**(mcpp 2026.9.16.1+),`mcpp build --features spike.fw/installer` 为一条命令 + 打开同一个 feature,与根的转发相同;`run`、`test`、`pack`、`emit build-database` 与 + `why deps` 同样接受该记号。它从不是根的 feature,也从不变成宏。 +- **校验**读取写下该转发的清单的每一张依赖表:`[dependencies]`、`[build-dependencies]`、 + `[dev-dependencies]` 与 `[feature-deps.]`,覆盖所有行。只在别的行或未激活的 feature + 下声明的键同样算已声明;在当前行上该转发不到达任何边,不产生效果。没有任何表声明的键会被 + 报告,在 `--strict` 下报告为错误。命令行上无论根是否声明 `[features]`,都做同样的检查。 + ### 表形式 —— 让 feature 贡献的不止是隐含 feature `[features]` 的条目除了写成数组,还可写成**表**,从而让该 feature 在隐含 feature diff --git a/docs/zh/07-workspace.md b/docs/zh/07-workspace.md index f4039e90..a19ec8d1 100644 --- a/docs/zh/07-workspace.md +++ b/docs/zh/07-workspace.md @@ -345,6 +345,11 @@ myproject/ 各成员的构建产物位于各自的 `target/` 子目录下。 +工作空间之外的项目以 member 的身份引用托管在 git 上的工作空间中的 member: +`myproject.http = { git = "...", rev = "..." }` 在根清单的 `members` 中选中 `libs/http`, +提交相同,且该 member 与在此处一样继承 `[workspace.package]`(mcpp 2026.9.16.1+;见 +[05 —— 依赖](05-dependencies.md))。 + ## 7. 与 C++ 模块的关系 工作空间与 C++23 模块机制协同工作： diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index ade2ed51..37948a80 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -1175,6 +1175,11 @@ int main() { return grpcgen::generate_all() ? 0 : 1; } 尾名 `protobuf`。全限定的 `MCPP_DEP___BIN_` 总是发布;裸名 依次绑定到 `mcpplibs.`、`compat.`、无命名空间的 ``,最后才是「剩下 的唯一候选」——存在争用时 mcpp 会说出来,而不是默默选一个。 +- **工具的名字与它的目录的名字完全相同。** `dep_dir`、`dep_linkage` 与 `dep_bin` + 使用同一个推导为提供者命名:清单的 `name`、限定名 `namespace.name`,以及已绑定的尾名。 + 因此把 `namespace = "spike"` 与 `name = "installer"` 分开写的包,既是 + `dep_bin("installer", ...)`,也是 `dep_bin("spike.installer", ...)`(mcpp 2026.9.16.1+; + 此前它的工具只以 `installer` 发布)。 #### 旧版 mcpp 读到用了这些键的 manifest diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 246a10c4..c6388c9e 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -348,6 +348,7 @@ mcpp why toolchain [--target ] [--toolchain ] --format json | `host-module-missing` | `build.mcpp` 导入了没有任何依赖以 host module 形式提供的模块 | | `tool-version-conflict` | 两处声明把同一个 xlings 包定在不能同时成立的版本上 | | `shared-library-cxx-runtime` | C++ 运行时来自图中的包,而依赖的 C++ 共享库没有声明私有副本 | +| `package-cycle` | 依赖图中存在包的环;消息列出环上的边 | | `other` | 一处还没有被命名的拒绝分支 | **只要问题被回答了就退 0,包括答案是「拒绝」。** 「它能不能构建,不能的话 From f78b8d1a5a81298de9a8420d3189eab387f2a329 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:21:09 +0800 Subject: [PATCH 09/20] xlings children are owned and bounded; the envelope reports observed network access; e2e 730-735 (#648 L3, L4) --- modules/platform/src/env.cppm | 21 +++ modules/platform/src/process.cppm | 94 +++++++++++- .../platform/src/unix/bounded_process.cppm | 54 ++++++- .../platform/src/windows/bounded_process.cppm | 27 +++- src/build/prepare.cppm | 1 + src/config.cppm | 14 ++ src/wire.cppm | 7 + src/xlings/xlings.cppm | 136 +++++++++++++++--- ...compat_dependency_is_not_a_refresh_miss.sh | 52 +++++++ ...hildren_do_not_inherit_the_callers_pipe.sh | 52 +++++++ ...732_xlings_refresh_is_bounded_and_owned.sh | 67 +++++++++ ...nvelope_reports_observed_network_access.sh | 63 ++++++++ ..._refresh_false_governs_first_index_sync.sh | 61 ++++++++ ...ffline_missing_package_has_its_own_code.sh | 68 +++++++++ 14 files changed, 682 insertions(+), 35 deletions(-) create mode 100755 tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh create mode 100755 tests/e2e/731_planning_children_do_not_inherit_the_callers_pipe.sh create mode 100755 tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh create mode 100755 tests/e2e/733_envelope_reports_observed_network_access.sh create mode 100755 tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh create mode 100755 tests/e2e/735_offline_missing_package_has_its_own_code.sh diff --git a/modules/platform/src/env.cppm b/modules/platform/src/env.cppm index 7b636bd2..02dce080 100644 --- a/modules/platform/src/env.cppm +++ b/modules/platform/src/env.cppm @@ -49,6 +49,20 @@ bool offline_mode(); // and kept next to offline_mode() so the two network knobs cannot drift apart. bool no_auto_install(); +// WHAT THIS RUN DID WITH THE NETWORK, AS OPPOSED TO WHAT IT MAY DO. +// +// docs/50 defines an envelope's `effects` as what running the command did, +// and `--protocol-version` as what a command may do. The per-run list used to +// be assembled from constants, so a plan that refreshed the index reported no +// `network` (#648 A4). A launcher of a network-bound child (an index refresh, +// an install, a git remote operation) calls `note_network_access` before it +// starts the child; an attempt that fails or times out still counts, because +// the effect is the access. Offline, those launchers return before this call. +// Kept beside offline_mode() for the reason stated above: every layer can reach +// this module, and the record has writers in several of them. +void note_network_access(); +bool network_accessed(); + // Temporarily set or unset an env var, restoring the prior value on scope exit. class ScopedEnv { public: @@ -112,6 +126,13 @@ bool no_auto_install() { return v && *v && std::string_view(v) != "0"; } +namespace { +std::atomic g_network_accessed{false}; +} + +void note_network_access() { g_network_accessed.store(true, std::memory_order_relaxed); } +bool network_accessed() { return g_network_accessed.load(std::memory_order_relaxed); } + std::optional get(std::string_view key) { std::string k(key); auto* v = std::getenv(k.c_str()); diff --git a/modules/platform/src/process.cppm b/modules/platform/src/process.cppm index 60572608..446a77bc 100644 --- a/modules/platform/src/process.cppm +++ b/modules/platform/src/process.cppm @@ -242,6 +242,31 @@ int run_silent(std::string_view command); int run_streaming(std::string_view command, std::function on_line); +// run_streaming for a child that mcpp must own and bound (#648). +// +// `run_streaming` hands its child to `popen`: the child is in mcpp's process +// group, is not killed when mcpp is, and is waited on with no limit. That was +// the xlings index refresh that held an editor for eleven minutes on a +// connection that never answered. This runs the same shell command (the same +// quoting, the same sealed stdin) through the deadline launchers instead: +// +// - the child owns a process group on POSIX, registered with the signal +// guard, and runs in a kill-on-close job object on Windows, so +// terminating mcpp terminates the command's whole tree; +// - `total` bounds the run; `idle` bounds the time without output. Either +// may be zero (no bound of that kind), not both; +// - a bound that fires kills the tree and sets *timed_out; the exit code is +// then the killed shell's. +// +// Stdout and stderr share the pipe, as they do for every captured deadline +// run; a caller that parses stdout redirects stderr in the command itself. +// Returns -1 when the launcher could not run the command at all. +int run_streaming_bounded(std::string_view command, + std::function on_line, + std::chrono::milliseconds total, + std::chrono::milliseconds idle, + bool* timed_out); + // Run `command`, passing stdout/stderr through to the terminal. // Optionally captures stdout into `output` if non-null. // Returns a platform-normalized exit code (WEXITSTATUS on POSIX). @@ -893,13 +918,20 @@ struct BoundedOutcome { // normal case) means "derive it from argv". A shell command is the one caller // that must NOT be derived that way — see windows_shell_command_line — and the // POSIX branch is unaffected either way, because it never flattens argv. +// `idle` and `ownGroup` are the streaming runner's (see run_streaming_bounded); +// every other caller passes neither and keeps its behaviour. `on_line`, when +// given, receives each complete line as it arrives instead of the output being +// accumulated. BoundedOutcome dispatch_bounded( const std::vector& argv, const std::vector>& extraEnv, std::string_view cwd, std::chrono::milliseconds deadline, bool capture, - std::string_view windowsCommandLine = {}) + std::string_view windowsCommandLine = {}, + std::chrono::milliseconds idle = std::chrono::milliseconds{0}, + bool ownGroup = false, + const std::function* on_line = nullptr) { BoundedOutcome outcome; @@ -916,21 +948,40 @@ BoundedOutcome dispatch_bounded( const char* cwdArg = cwdStore.empty() ? nullptr : cwdStore.c_str(); const auto ms = static_cast(deadline.count()); + const auto idleMs = static_cast(idle.count()); + // One sink for both, appending into the outcome's own buffer. Null when the - // caller wants the child on its own stdio. + // caller wants the child on its own stdio. With `on_line`, complete lines + // leave the buffer as they arrive, and the unterminated tail is delivered + // after the child exits, as run_streaming does. + struct StreamCtx { + std::string* buffer; + const std::function* on_line; + } streamCtx{ &outcome.output, on_line }; using Sink = void (*)(void*, const char*, unsigned long); - const Sink sink = capture + const Sink sink = !capture ? nullptr + : on_line == nullptr ? +[](void* ctx, const char* data, unsigned long len) { - static_cast(ctx)->append(data, len); + static_cast(ctx)->buffer->append(data, len); } - : nullptr; + : +[](void* ctx, const char* data, unsigned long len) { + auto* c = static_cast(ctx); + c->buffer->append(data, len); + std::size_t pos; + while ((pos = c->buffer->find('\n')) != std::string::npos) { + std::string_view line{c->buffer->data(), pos}; + while (!line.empty() && line.back() == '\r') line.remove_suffix(1); + (*c->on_line)(line); + c->buffer->erase(0, pos + 1); + } + }; if constexpr (mcpp::platform::is_windows) { const auto cmd = windowsCommandLine.empty() ? windows_command_from_argv(argv) : std::string(windowsCommandLine); auto r = mcpp::platform::winproc::capture_with_deadline( - cmd.c_str(), envArg, envCount, cwdArg, ms, sink, &outcome.output); + cmd.c_str(), envArg, envCount, cwdArg, ms, idleMs, sink, &streamCtx); outcome.supported = r.supported; outcome.exit_code = r.exit_code; outcome.timed_out = r.timed_out; @@ -941,12 +992,19 @@ BoundedOutcome dispatch_bounded( for (auto const& a : argv) argvPtrs.push_back(a.c_str()); auto r = mcpp::platform::unixproc::capture_with_deadline( argvPtrs.data(), static_cast(argvPtrs.size()), - envArg, envCount, cwdArg, ms, sink, &outcome.output); + envArg, envCount, cwdArg, ms, idleMs, ownGroup ? 1 : 0, + sink, &streamCtx); outcome.supported = r.supported; outcome.exit_code = r.exit_code; outcome.timed_out = r.timed_out; outcome.spawn_error = r.spawn_error; } + if (on_line && capture && !outcome.output.empty()) { + std::string_view tail{outcome.output}; + while (!tail.empty() && tail.back() == '\r') tail.remove_suffix(1); + if (!tail.empty()) (*on_line)(tail); + outcome.output.clear(); + } return outcome; } @@ -1005,6 +1063,28 @@ int run_shell_deadline(std::string_view command, return r.exit_code; } +int run_streaming_bounded(std::string_view command, + std::function on_line, + std::chrono::milliseconds total, + std::chrono::milliseconds idle, + bool* timed_out) +{ + if (timed_out) *timed_out = false; + if (command.empty() || (total.count() <= 0 && idle.count() <= 0)) return -1; + // The popen path's shaping, minus the outer cmd.exe pair: the Windows + // launcher's command line supplies that pair itself. + const auto sealed = seal_stdin(command); + const std::vector argv{"/bin/sh", "-c", sealed}; + std::function sink = + on_line ? std::move(on_line) : [](std::string_view) {}; + auto r = dispatch_bounded(argv, {}, {}, total, /*capture=*/true, + windows_shell_command_line(sealed), idle, + /*ownGroup=*/true, &sink); + if (!r.supported) return -1; + if (timed_out) *timed_out = r.timed_out; + return r.exit_code; +} + // The same split as dispatch_bounded, for the same reason: POSIX names a // program with an argv array, Windows with a single command line. Both // spellings are built here so neither can drift into a launcher that does not diff --git a/modules/platform/src/unix/bounded_process.cppm b/modules/platform/src/unix/bounded_process.cppm index bb017f83..ab9edf14 100644 --- a/modules/platform/src/unix/bounded_process.cppm +++ b/modules/platform/src/unix/bounded_process.cppm @@ -87,12 +87,27 @@ using OutputSink = void (*)(void* ctx, const char* data, unsigned long len); // (a) delay every line until the child exits, which is the opposite of what a // bounded `mcpp test` run is for, and (b) make the child's stdout a pipe // rather than a terminal, so gtest and friends silently drop their colors. +// +// `idleMs`, when positive, is a second bound: the child is killed once it has +// written nothing for that long. It is meaningful only with a sink, and it is +// what lets a long, progressing child (a download that prints progress) run to +// completion while a wedged one does not (#648). With `idleMs` positive a +// non-positive `deadlineMs` means "no total bound". +// +// `ownGroup` non-zero places the child in a process group of its own, kills +// the GROUP at a bound, and registers the group with the signal guard for the +// length of the call, so terminating mcpp takes the child's descendants with +// it. It is opt-in because a child in a background group that reads the +// terminal is stopped by SIGTTIN, and the uncaptured callers of this function +// hand the terminal to their child. DeadlineRun capture_with_deadline(const char* const* argvEntries, unsigned long argvCount, const char* const* envEntries, unsigned long envCount, const char* cwd, long long deadlineMs, + long long idleMs, + int ownGroup, OutputSink sink, void* ctx); @@ -188,11 +203,14 @@ DeadlineRun capture_with_deadline(const char* const* argvEntries, unsigned long envCount, const char* cwd, long long deadlineMs, + long long idleMs, + int ownGroup, OutputSink sink, void* ctx) { DeadlineRun out; - if (deadlineMs <= 0 || argvCount == 0 || !argvEntries) return out; + const bool idleBound = idleMs > 0 && sink != nullptr; + if ((deadlineMs <= 0 && !idleBound) || argvCount == 0 || !argvEntries) return out; // The child's environment: ours, minus anything overridden, plus the // overrides. Names are case-SENSITIVE here (unlike the Windows peer). @@ -244,11 +262,19 @@ DeadlineRun capture_with_deadline(const char* const* argvEntries, // else: no file actions for stdio at all — the child inherits ours, which // keeps its output live AND keeps it a terminal. + posix_spawnattr_t attr; + ::posix_spawnattr_init(&attr); + if (ownGroup) { + ::posix_spawnattr_setpgroup(&attr, 0); // 0: a new group, id == pid + ::posix_spawnattr_setflags(&attr, POSIX_SPAWN_SETPGROUP); + } pid_t pid = 0; - int sp = ::posix_spawnp(&pid, cargv[0], &fa, nullptr, cargv.data(), envp.data()); + int sp = ::posix_spawnp(&pid, cargv[0], &fa, &attr, cargv.data(), envp.data()); + ::posix_spawnattr_destroy(&attr); ::posix_spawn_file_actions_destroy(&fa); if (capture) ::close(fds[1]); if (sp != 0) { out.spawn_error = sp; if (capture) ::close(fds[0]); return out; } + if (ownGroup) guard_group_on_signal(pid); // Non-blocking reads so the deadline is still checked while the child is // quiet. A blocking read on a silent, hung child is exactly the hang this @@ -256,8 +282,11 @@ DeadlineRun capture_with_deadline(const char* const* argvEntries, if (capture) ::fcntl(fds[0], F_SETFL, ::fcntl(fds[0], F_GETFL, 0) | O_NONBLOCK); - const auto until = std::chrono::steady_clock::now() - + std::chrono::milliseconds(deadlineMs); + const auto started = std::chrono::steady_clock::now(); + const auto until = deadlineMs > 0 + ? started + std::chrono::milliseconds(deadlineMs) + : std::chrono::steady_clock::time_point::max(); + auto lastOutput = started; std::array buf{}; bool killed = false; int status = 0; @@ -270,6 +299,7 @@ DeadlineRun capture_with_deadline(const char* const* argvEntries, sink(ctx, buf.data(), static_cast(n)); any = true; } + if (any) lastOutput = std::chrono::steady_clock::now(); return any; }; @@ -283,14 +313,23 @@ DeadlineRun capture_with_deadline(const char* const* argvEntries, } if (r < 0 && errno != EINTR && errno != ECHILD) break; - if (!killed && std::chrono::steady_clock::now() >= until) { - ::kill(pid, SIGKILL); + const auto now = std::chrono::steady_clock::now(); + const bool overTotal = now >= until; + const bool overIdle = idleBound + && now - lastOutput >= std::chrono::milliseconds(idleMs); + if (!killed && (overTotal || overIdle)) { + // The group when the child owns one: a shell's grandchild (the + // program the shell ran) would otherwise keep running, and keep + // the pipe open so the drain above never saw end-of-file. + if (ownGroup) ::killpg(pid, SIGKILL); + else ::kill(pid, SIGKILL); killed = true; continue; } struct timespec ts{0, 20'000'000}; // 20ms ::nanosleep(&ts, nullptr); } + if (ownGroup) unguard_group(pid); if (capture) ::close(fds[0]); out.exit_code = normalize_status(status); @@ -472,7 +511,8 @@ void clear_group_guard() { DeadlineRun capture_with_deadline(const char* const*, unsigned long, const char* const*, unsigned long, - const char*, long long, OutputSink, void*) { + const char*, long long, long long, int, + OutputSink, void*) { // Not POSIX: mcpp.platform.windows.bounded_process owns this. return {}; } diff --git a/modules/platform/src/windows/bounded_process.cppm b/modules/platform/src/windows/bounded_process.cppm index 5d832105..0b2c403f 100644 --- a/modules/platform/src/windows/bounded_process.cppm +++ b/modules/platform/src/windows/bounded_process.cppm @@ -96,11 +96,19 @@ using OutputSink = void (*)(void* ctx, const char* data, unsigned long len); // "KEY=VALUE" strings applied on top of the current environment. `cwd` may be // null. A non-positive `deadlineMs` is rejected with supported=false — "no // bound" belongs on the caller's untimed path, which needs none of this. +// +// `idleMs`, when positive, kills the child once it has written nothing for that +// long; the POSIX peer states the reason (#648). With `idleMs` positive a +// non-positive `deadlineMs` means "no total bound". There is no `ownGroup` +// parameter: every child here runs in a job object with +// JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE already, which is what the POSIX flag +// buys there. DeadlineRun capture_with_deadline(const char* commandLine, const char* const* envEntries, unsigned long envCount, const char* cwd, long long deadlineMs, + long long idleMs, OutputSink sink, void* ctx); @@ -247,11 +255,13 @@ DeadlineRun capture_with_deadline(const char* commandLine, unsigned long envCount, const char* cwd, long long deadlineMs, + long long idleMs, OutputSink sink, void* ctx) { DeadlineRun out; - if (deadlineMs <= 0 || !commandLine || !*commandLine) return out; + const bool idleBound = idleMs > 0 && sink != nullptr; + if ((deadlineMs <= 0 && !idleBound) || !commandLine || !*commandLine) return out; const bool capture = (sink != nullptr); @@ -337,8 +347,11 @@ DeadlineRun capture_with_deadline(const char* commandLine, // EOF, even after every child has exited. writeEnd.reset(); - const auto until = std::chrono::steady_clock::now() - + std::chrono::milliseconds(deadlineMs); + const auto started = std::chrono::steady_clock::now(); + const auto until = deadlineMs > 0 + ? started + std::chrono::milliseconds(deadlineMs) + : std::chrono::steady_clock::time_point::max(); + auto lastOutput = started; std::array buf{}; bool killed = false; @@ -354,6 +367,7 @@ DeadlineRun capture_with_deadline(const char* commandLine, if (!::ReadFile(readEnd.h, buf.data(), want, &got, nullptr) || got == 0) return false; if (sink) sink(ctx, buf.data(), static_cast(got)); + lastOutput = std::chrono::steady_clock::now(); return true; }; @@ -365,7 +379,10 @@ DeadlineRun capture_with_deadline(const char* commandLine, break; } - if (!killed && std::chrono::steady_clock::now() >= until) { + const auto now = std::chrono::steady_clock::now(); + const bool overIdle = idleBound + && now - lastOutput >= std::chrono::milliseconds(idleMs); + if (!killed && (now >= until || overIdle)) { killed = true; // Closing the job takes the whole tree with it. TerminateProcess // alone would leave grandchildren holding the pipe open. @@ -573,7 +590,7 @@ int wait_background(unsigned long long process, int* exitCode) { #else DeadlineRun capture_with_deadline(const char*, const char* const*, unsigned long, - const char*, long long, OutputSink, void*) { + const char*, long long, long long, OutputSink, void*) { // Not Windows: the POSIX launcher in mcpp.platform.process owns this. return {}; } diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 94e636f5..33753281 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -1266,6 +1266,7 @@ mcpp::platform::process::RunResult run_with_network_retry( std::string_view command, const std::function& between = {}) { mcpp::platform::process::RunResult r{}; + mcpp::platform::env::note_network_access(); // the envelope's `effects` (#648 A4) for (int attempt = 1; attempt <= 3; ++attempt) { r = mcpp::platform::process::capture(command); if (r.exit_code == 0) return r; diff --git a/src/config.cppm b/src/config.cppm index 14578f0b..bd86d2bd 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -97,6 +97,12 @@ struct GlobalConfig { // being portable between a corporate LAN, a laptop and CI. bool indexAutoRefresh = true; + // From config.toml [index] refresh_timeout (seconds). The total bound on + // one index refresh; a refresh that exceeds it counts as failed and the + // build continues from the local index (#648 A3). Machine policy, like + // auto_refresh: it describes the network, not the project. + std::int64_t indexRefreshTimeoutSeconds = 120; + // From config.toml [build] std::int64_t defaultJobs = 0; @@ -345,6 +351,10 @@ default = "mcpplibs" # cannot be resolved from the local copy (never merely because time passed). # Set false to require an explicit `mcpp index update`. auto_refresh = true +# refresh_timeout: seconds one automatic or explicit index refresh may take. +# A refresh that exceeds it is stopped, counts as failed, and the build +# continues with the local index. +refresh_timeout = 120 [index.repos."mcpplibs"] url = "https://github.com/mcpplibs/mcpp-index.git" @@ -661,6 +671,10 @@ std::expected load_or_init( cfg.xlingsHomeOverride = *h; cfg.defaultIndex = doc->get_string("index.default").value_or("mcpplibs"); cfg.indexAutoRefresh = doc->get_bool("index.auto_refresh").value_or(true); + if (auto t = doc->get_int("index.refresh_timeout"); t && *t > 0) + cfg.indexRefreshTimeoutSeconds = *t; + mcpp::xlings::set_index_refresh_timeout( + std::chrono::seconds{cfg.indexRefreshTimeoutSeconds}); cfg.searchTtlSeconds = doc->get_int("cache.search_ttl_seconds").value_or(3600); cfg.defaultJobs = doc->get_int("build.default_jobs").value_or(0); cfg.defaultToolchain = doc->get_string("toolchain.default").value_or(""); diff --git a/src/wire.cppm b/src/wire.cppm index df94a93d..1dfe235b 100644 --- a/src/wire.cppm +++ b/src/wire.cppm @@ -50,6 +50,7 @@ export module mcpp.wire; import std; import mcpp.version; import mcpp.libs.json; +import mcpp.platform; // env::network_accessed (#648 A4) export namespace mcpp::wire { @@ -181,6 +182,12 @@ inline nlohmann::json to_json(const Diagnostic& d) { inline nlohmann::json to_json(const Envelope& e) { nlohmann::json effects = nlohmann::json::array(); for (auto f : e.effects) effects.push_back(std::string(effect_name(f))); + // `effects` states what running the command did (docs/50 section 2). Network + // access is observed where it happens rather than declared by each command, + // so every enveloped command reports it the same way. + if (mcpp::platform::env::network_accessed() + && std::ranges::find(e.effects, Effect::Network) == e.effects.end()) + effects.push_back(std::string(effect_name(Effect::Network))); nlohmann::json diags = nlohmann::json::array(); for (auto const& d : e.diagnostics) diags.push_back(to_json(d)); diff --git a/src/xlings/xlings.cppm b/src/xlings/xlings.cppm index 663dbd42..ab1613e0 100644 --- a/src/xlings/xlings.cppm +++ b/src/xlings/xlings.cppm @@ -521,6 +521,36 @@ std::optional find_sandbox_nasm(const Env& env); // imports this module — the reverse would be a cycle. inline constexpr std::int64_t kIndexRefreshDebounceSeconds = 120; +// ─── Bounds on the xlings children mcpp starts (#648 A3) ─────────────── +// +// Every xlings invocation on the planning path runs through +// `run_streaming_bounded`: the child owns a process group (a job object on +// Windows) that dies with mcpp, and it is bounded. Before, an `xlings update` +// on a connection that never answered held `emit build-database` for eleven +// minutes, and outlived the mcpp its caller had killed. +// +// Which bound fits depends on what the child says while it works: +// - the index refresh prints little and is small: a TOTAL bound, configured +// by `[index] refresh_timeout`. A timed-out refresh is a failed refresh, +// which the build already survives when the local index can answer; +// - an install through the NDJSON interface emits a heartbeat after five +// seconds of silence (xlings interface.cpp), so no output for +// kInterfaceIdleTimeout means the xlings process itself is wedged, while a +// slow download that makes progress is never cut short: an IDLE bound; +// - a direct `xlings install -y` has its output sent to the null device, so +// only a total bound can apply, and it is generous: a toolchain archive on a +// slow link is not a hang; +// - a local command (the one-time sandbox init) gets a total bound. +inline constexpr std::chrono::seconds kDefaultIndexRefreshTimeout{120}; +inline constexpr std::chrono::seconds kInterfaceIdleTimeout{300}; +inline constexpr std::chrono::seconds kDirectInstallTimeout{3 * 3600}; +inline constexpr std::chrono::seconds kLocalCommandTimeout{600}; + +// Set once from `[index] refresh_timeout` when the configuration is loaded; +// zero or negative restores the default. +void set_index_refresh_timeout(std::chrono::seconds timeout); +std::chrono::seconds index_refresh_timeout(); + // Check whether the default mcpplibs index data exists and is fresh // (within ttlSeconds). // Returns true if index is present and fresh, false otherwise. @@ -1415,8 +1445,13 @@ call(const Env& env, std::string_view capability, mcpp::log::verbose("xlings", std::format("interface {} exec: {}", capability, cmd)); + // An install reaches the network; the query capabilities read local state. + if (capability == "install_packages" || capability == "update_packages") + mcpp::platform::env::note_network_access(); + CallResult result; - int rc = mcpp::platform::process::run_streaming(cmd, + bool timedOut = false; + int rc = mcpp::platform::process::run_streaming_bounded(cmd, [&](std::string_view line) { if (line.empty()) return; @@ -1441,7 +1476,16 @@ call(const Env& env, std::string_view capability, if (handler) handler->on_result(e); } }, *ev); - }); + }, + std::chrono::milliseconds{0}, + std::chrono::duration_cast(kInterfaceIdleTimeout), + &timedOut); + if (timedOut) { + result.exitCode = result.exitCode != 0 ? result.exitCode : 124; + result.stderrTail.push_back(std::format( + "xlings interface {} wrote nothing for {} seconds (not even its " + "heartbeat) and was stopped", capability, kInterfaceIdleTimeout.count())); + } if (rc != 0 && result.exitCode == 0) result.exitCode = rc; if (result.exitCode != 0) { // Error-level lines only, the last 20: enough to name a rejection, and @@ -1520,10 +1564,19 @@ int install_with_progress(const Env& env, std::string_view target, // Only when interactive (not quiet, stderr/stdout is a TTY). const bool showSpinner = !quiet && mcpp::platform::terminal::is_tty(); + mcpp::platform::env::note_network_access(); std::atomic done{false}; - int directRaw = 0; + int directRc = 0; std::thread worker([&] { - directRaw = std::system(directCmd.c_str()); + bool timedOut = false; + directRc = mcpp::platform::process::run_streaming_bounded( + directCmd, [](std::string_view) {}, + std::chrono::duration_cast(kDirectInstallTimeout), + std::chrono::milliseconds{0}, &timedOut); + if (timedOut) + mcpp::log::warn("xlings", std::format( + "`xlings install {}` did not finish within {} hours and was stopped", + target, kDirectInstallTimeout.count() / 3600)); done.store(true, std::memory_order_release); }); @@ -1550,7 +1603,6 @@ int install_with_progress(const Env& env, std::string_view target, } worker.join(); - int directRc = mcpp::platform::process::extract_exit_code(directRaw); if (directRc == 0) return 0; } @@ -1615,7 +1667,17 @@ int install_with_progress(const Env& env, std::string_view target, if (!prog.files.empty()) cb(prog); }; - int closeRc = mcpp::platform::process::run_streaming(cmd, handle_line); + bool idleTimedOut = false; + int closeRc = mcpp::platform::process::run_streaming_bounded( + cmd, handle_line, std::chrono::milliseconds{0}, + std::chrono::duration_cast(kInterfaceIdleTimeout), + &idleTimedOut); + if (idleTimedOut) { + mcpp::log::warn("xlings", std::format( + "xlings interface install_packages wrote nothing for {} seconds and was stopped", + kInterfaceIdleTimeout.count())); + return resultExitCode > 0 ? resultExitCode : 124; + } return (resultExitCode != -1) ? resultExitCode : closeRc; } @@ -1627,12 +1689,19 @@ int install_direct(const Env& env, std::string_view target, bool quiet) { cmd += " "; cmd += std::string(mcpp::platform::shell::silent_redirect); } - if constexpr (mcpp::platform::is_windows) { - cmd += " (kDirectInstallTimeout), + std::chrono::milliseconds{0}, &timedOut); + if (timedOut) + mcpp::log::warn("xlings", std::format( + "`xlings install {}` did not finish within {} hours and was stopped", + target, kDirectInstallTimeout.count() / 3600)); + return rc; } // ─── Sandbox lifecycle ────────────────────────────────────────────── @@ -1719,7 +1788,11 @@ void ensure_init(const Env& env, bool quiet) { ScopedInvocationEnv scope(globalEnv); // #614 std::string cmd = build_command_prefix(globalEnv) + " self init " + std::string(mcpp::platform::shell::silent_redirect); - int rc = mcpp::platform::process::run_silent(cmd); + bool initTimedOut = false; + int rc = mcpp::platform::process::run_streaming_bounded( + cmd, [](std::string_view) {}, + std::chrono::duration_cast(kLocalCommandTimeout), + std::chrono::milliseconds{0}, &initTimedOut); if (rc != 0 && !quiet) { std::println(stderr, "warning: `xlings self init` failed for sandbox at '{}'", @@ -1877,6 +1950,20 @@ int update_index_unguarded(const Env& env, bool quiet); // back — the refresh itself was the thing that broke the machine. See // mcpp.pm.index_snapshot for why the shape is archive/judge/restore rather // than the stage-and-swap the original design assumed. +namespace { +std::atomic g_index_refresh_timeout_s{kDefaultIndexRefreshTimeout.count()}; +} + +void set_index_refresh_timeout(std::chrono::seconds timeout) { + g_index_refresh_timeout_s.store( + timeout.count() > 0 ? timeout.count() : kDefaultIndexRefreshTimeout.count(), + std::memory_order_relaxed); +} + +std::chrono::seconds index_refresh_timeout() { + return std::chrono::seconds{g_index_refresh_timeout_s.load(std::memory_order_relaxed)}; +} + int update_index(const Env& env, bool quiet) { namespace snap = mcpp::pm::index_snapshot; const auto dataRoot = paths::index_data(env); @@ -1942,6 +2029,7 @@ int update_index_unguarded(const Env& env, bool quiet) { } std::string cmd = build_command_prefix(env) + " update 2>&1"; + mcpp::platform::env::note_network_access(); // The index sync is a network git operation; a single transient blip (DNS, // TLS reset, a mirror hiccup) otherwise fails a cold `mcpp self env` / // first-run init outright (e.g. CI's index/sandbox bootstrap). Retry with @@ -1949,12 +2037,28 @@ int update_index_unguarded(const Env& env, bool quiet) { // added latency in steady state; only a genuine failure pays the backoff. constexpr int kMaxAttempts = 3; int rc = 0; + const auto refreshBound = index_refresh_timeout(); for (int attempt = 1; attempt <= kMaxAttempts; ++attempt) { - rc = mcpp::platform::process::run_streaming(cmd, + bool timedOut = false; + rc = mcpp::platform::process::run_streaming_bounded(cmd, [quiet](std::string_view line) { if (!quiet) std::println("{}", line); - }); - if (rc == 0) { mark_known_indexes_refreshed(env); return 0; } + }, + std::chrono::duration_cast(refreshBound), + std::chrono::milliseconds{0}, &timedOut); + if (rc == 0 && !timedOut) { mark_known_indexes_refreshed(env); return 0; } + // A refresh that exceeded its bound is not retried: the retries exist for + // a transient failure that ends, and a connection that never answers + // would only be waited on three times. The caller treats the refresh as + // failed and resolves from the local index, as it does for any failure. + if (timedOut) { + std::println(stderr, + "warning: the package index refresh did not finish within {} seconds and was " + "stopped; continuing with the local index " + "(the bound is [index] refresh_timeout in mcpp's config.toml)", + refreshBound.count()); + return rc != 0 ? rc : 124; + } if (attempt < kMaxAttempts) { int delay = attempt * 2; // 2s, then 4s mcpp::log::verbose("index", std::format( diff --git a/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh b/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh new file mode 100755 index 00000000..720b53e3 --- /dev/null +++ b/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# requires: elf +# 730 -- a dependency the resolver reaches through the deprecated bare-name rung +# is not a descriptor miss for the index refresh decision (#648 L1). +# +# `ftxui = "6.1.9"` omits the namespace, which means mcpplibs; the package is +# published as compat.ftxui. The resolver tries compat after the exact miss and +# resolves it from disk. The refresh decision used to stop at the exact +# coordinate and asked for a network refresh, which `-v` shows offline as a +# decision suppressed by offline mode. Criteria, planned offline with `-v`: +# A. the resolver takes the rung (its deprecation warning is printed), so the +# fixture exercises the path at all; +# B. no refresh decision for that dependency is suppressed by offline mode. +# Skipped when the local mcpplibs index has no compat descriptor to reach. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +INDEX_ROOT="${MCPP_HOME:-$HOME/.mcpp}/registry/data/mcpplibs/pkgs/c" +DESC="" +for cand in cjson argparse gtest ftxui; do + if [ -f "$INDEX_ROOT/compat.$cand.lua" ]; then DESC="$cand"; break; fi +done +if [ -z "$DESC" ]; then + echo "SKIP: no compat descriptor in the local mcpplibs index" + exit 0 +fi +VER=$(grep -o '\["[0-9][0-9.]*"\]' "$INDEX_ROOT/compat.$DESC.lua" | head -1 | tr -d '[]"') +[ -n "$VER" ] || fail "could not read a version of compat.$DESC" + +mkdir -p "$TMP/app/src" +cat > "$TMP/app/mcpp.toml" < "$TMP/app/src/main.cpp" + +cd "$TMP/app" +MCPP_OFFLINE=1 "$MCPP" emit build-database --format json -v > out.json 2> err.txt || true + +grep -q "resolved to 'compat.$DESC' through the deprecated bare-name search" err.txt \ + || fail "A: the resolver did not take the bare-name rung for $DESC" err.txt +if grep -a "index: $DESC@$VER: offline mode" err.txt; then + fail "B: the refresh decision asked for a network refresh of a dependency the resolver found" err.txt +fi +echo "PASS: 730 bare compat dependency ($DESC@$VER) is not a refresh miss" diff --git a/tests/e2e/731_planning_children_do_not_inherit_the_callers_pipe.sh b/tests/e2e/731_planning_children_do_not_inherit_the_callers_pipe.sh new file mode 100755 index 00000000..03bca566 --- /dev/null +++ b/tests/e2e/731_planning_children_do_not_inherit_the_callers_pipe.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# requires: elf +# 731 -- a child started while `emit build-database` plans does not hold the +# caller's pipe (#648 L2). +# +# `emit build-database` sends planning narration to stderr by redirecting +# descriptor 1 and saving the original. The saved copy was inheritable, so a +# build program, a hook or an xlings process started during planning held the +# caller's pipe open, and a caller reading mcpp's output waited for that child +# rather than for mcpp. Criterion: a build program lists its descriptors, and +# none of them names the pipe the reader of mcpp's standard output reads. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +mkdir -p "$TMP/app/src" +cat > "$TMP/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +echo 'int main() { return 0; }' > "$TMP/app/src/main.cpp" +cat > "$TMP/app/build.mcpp" < err.txt \ + | { readlink /proc/self/fd/0 > "$TMP/reader.txt"; cat > "$TMP/out.json"; } +[ -s "$TMP/fds.txt" ] || fail "the build program did not run" err.txt +PIPE=$(cat "$TMP/reader.txt") +case "$PIPE" in pipe:*) ;; *) fail "the reader's stdin is not a pipe: $PIPE" ;; esac +if grep -F " $PIPE" "$TMP/fds.txt"; then + fail "a build program holds the caller's pipe $PIPE" "$TMP/fds.txt" +fi +echo "PASS: 731 planning children do not inherit the caller's pipe" diff --git a/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh b/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh new file mode 100755 index 00000000..23dd3930 --- /dev/null +++ b/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 732 -- an xlings child is bounded, and it does not outlive mcpp (#648 L3). +# +# The xlings binary is a stub named through `[xlings] binary`; its `update` +# writes its pid and sleeps. Criteria: +# A. `mcpp index update` under `[index] refresh_timeout = 3` returns within the +# bound plus a margin, says which bound stopped it, and the stub is gone; +# B. with a long bound, SIGTERM to mcpp takes the stub with it: the stub's +# process group is registered with mcpp's signal guard. +set -e + +TMP=$(mktemp -d) +cleanup() { + for f in "$TMP"/stub.pid "$TMP"/stub2.pid; do + [ -s "$f" ] && kill -9 "$(cat "$f")" 2>/dev/null || true + done + rm -rf "$TMP" +} +trap cleanup EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +mkdir -p "$TMP/bin" "$TMP/home" +cat > "$TMP/bin/xlings" <<'EOF' +#!/usr/bin/env bash +case "${1:-}" in + update) echo "$$" > "${STUB_PID:?}"; exec sleep 1000 ;; +esac +exit 0 +EOF +chmod +x "$TMP/bin/xlings" + +write_config() { + cat > "$TMP/home/config.toml" < "$TMP/a.log" 2>&1 +set -e +elapsed=$(( $(date +%s) - start )) +[ -s "$TMP/stub.pid" ] || fail "A: the stub's update never ran" "$TMP/a.log" +[ "$elapsed" -lt 30 ] || fail "A: index update returned after ${elapsed}s under a 3 s bound" "$TMP/a.log" +grep -q "did not finish within 3 seconds" "$TMP/a.log" || fail "A: no line names the bound" "$TMP/a.log" +sleep 1 +if kill -0 "$(cat "$TMP/stub.pid")" 2>/dev/null; then fail "A: the stub outlived its bound"; fi + +# B. ownership +write_config 600 +STUB_PID="$TMP/stub2.pid" MCPP_HOME="$TMP/home" "$MCPP" index update > "$TMP/b.log" 2>&1 & +mcpp_pid=$! +for _ in $(seq 1 150); do [ -s "$TMP/stub2.pid" ] && break; sleep 0.1; done +[ -s "$TMP/stub2.pid" ] || { kill -9 "$mcpp_pid" 2>/dev/null; fail "B: the stub's update never ran" "$TMP/b.log"; } +kill -TERM "$mcpp_pid" +wait "$mcpp_pid" 2>/dev/null || true +for _ in $(seq 1 30); do kill -0 "$(cat "$TMP/stub2.pid")" 2>/dev/null || break; sleep 0.1; done +if kill -0 "$(cat "$TMP/stub2.pid")" 2>/dev/null; then fail "B: the stub outlived mcpp after SIGTERM"; fi + +echo "PASS: 732 xlings refresh is bounded and owned (A ${elapsed}s)" diff --git a/tests/e2e/733_envelope_reports_observed_network_access.sh b/tests/e2e/733_envelope_reports_observed_network_access.sh new file mode 100755 index 00000000..9fb4f198 --- /dev/null +++ b/tests/e2e/733_envelope_reports_observed_network_access.sh @@ -0,0 +1,63 @@ +#!/usr/bin/env bash +# requires: unix-shell python3 +# 733 -- the envelope's `effects` report the network access that happened, and +# an offline plan that needs the index has its own code (#648 L4, L6). +# +# docs/50 defines `effects` as what running the command did. A home with no +# package index plans a registry dependency: online, the plan refreshes the +# index (the xlings stub answers at once and fetches nothing); offline, it does +# not. Criteria: +# A. online, the failure envelope lists `network`; +# B. offline, it does not, and its code is MCPP_OFFLINE_DOWNLOAD_REQUIRED. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +export MCPP_HOME="$TMP/home" +source "$(dirname "$0")/_inherit_toolchain.sh" +rm -rf "$MCPP_HOME/registry/data/mcpplibs" + +mkdir -p "$TMP/bin" +cat > "$TMP/bin/xlings" <<'EOF' +#!/usr/bin/env bash +echo "$*" >> "${STUB_LOG:?}" +exit 0 +EOF +chmod +x "$TMP/bin/xlings" +STUB_HOST="$(host_path "$TMP/bin/xlings")" +{ + grep -v '^binary' "$MCPP_HOME/config.toml" 2>/dev/null | sed '/^\[xlings\]/d' + printf '\n[xlings]\nbinary = "%s"\n' "$STUB_HOST" +} > "$TMP/config.toml" +mv "$TMP/config.toml" "$MCPP_HOME/config.toml" + +mkdir -p "$TMP/app/src" +cat > "$TMP/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[dependencies.mcpplibs] +probe-not-published = "1.0.0" +EOF +echo 'int main() { return 0; }' > "$TMP/app/src/main.cpp" +cd "$TMP/app" + +effects() { python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print(" ".join(d["effects"])); print(" ".join(x["code"] for x in d["diagnostics"]))' "$1"; } + +STUB_LOG="$TMP/online.log" "$MCPP" emit build-database --format json > online.json 2> online.err || true +[ -s online.json ] || fail "A: no envelope" online.err +grep -q '^update' "$TMP/online.log" 2>/dev/null || fail "A: the plan did not refresh the index" online.err +effects online.json > online.txt +head -1 online.txt | grep -qw network || fail "A: a plan that refreshed the index reports no network effect" online.txt + +STUB_LOG="$TMP/offline.log" MCPP_OFFLINE=1 "$MCPP" emit build-database --format json > offline.json 2> offline.err || true +[ -s offline.json ] || fail "B: no envelope" offline.err +effects offline.json > offline.txt +if head -1 offline.txt | grep -qw network; then fail "B: an offline plan reports a network effect" offline.txt; fi +sed -n 2p offline.txt | grep -qw MCPP_OFFLINE_DOWNLOAD_REQUIRED \ + || fail "B: an offline plan with no index is not MCPP_OFFLINE_DOWNLOAD_REQUIRED" offline.txt +echo "PASS: 733 the envelope reports observed network access" diff --git a/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh b/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh new file mode 100755 index 00000000..44cd442e --- /dev/null +++ b/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 734 -- `[index] auto_refresh = false` governs the implicit refreshes that used +# to bypass the refresh policy (#648 L5). +# +# docs/05: auto_refresh = false means the index is never refreshed +# automatically. A project whose custom index has never been synced used to be +# synced on its first build regardless. Criteria, with an xlings stub that logs: +# A. the build refuses before syncing, and names `mcpp index update`; +# B. the stub was never asked to `update`. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +export MCPP_HOME="$TMP/home" +source "$(dirname "$0")/_inherit_toolchain.sh" +source "$(dirname "$0")/_host_path.sh" + +mkdir -p "$TMP/bin" +cat > "$TMP/bin/xlings" <<'EOF' +#!/usr/bin/env bash +echo "$*" >> "${STUB_LOG:?}" +exit 0 +EOF +chmod +x "$TMP/bin/xlings" +STUB_HOST="$(host_path "$TMP/bin/xlings")" +{ + grep -v '^binary\|^auto_refresh' "$MCPP_HOME/config.toml" 2>/dev/null \ + | sed '/^\[xlings\]/d; /^\[index\]$/d' + printf '\n[xlings]\nbinary = "%s"\n\n[index]\nauto_refresh = false\n' "$STUB_HOST" +} > "$TMP/config.toml" +mv "$TMP/config.toml" "$MCPP_HOME/config.toml" + +mkdir -p "$TMP/app/src" +cat > "$TMP/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[indices] +acme = { url = "https://example.invalid/acme-index.git" } + +[dependencies.acme] +widget = "1.0.0" +EOF +echo 'int main() { return 0; }' > "$TMP/app/src/main.cpp" +cd "$TMP/app" + +set +e +STUB_LOG="$TMP/stub.log" "$MCPP" build > build.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || fail "A: the build succeeded with an unsynced custom index" build.log +grep -q "mcpp index update" build.log || fail "A: the refusal does not name mcpp index update" build.log +grep -q "auto_refresh = false" build.log || fail "A: the refusal does not name the setting" build.log +if grep -q '^update' "$TMP/stub.log" 2>/dev/null; then + fail "B: the custom index was synced although auto_refresh = false" "$TMP/stub.log" +fi +echo "PASS: 734 auto_refresh = false governs the first custom index sync" diff --git a/tests/e2e/735_offline_missing_package_has_its_own_code.sh b/tests/e2e/735_offline_missing_package_has_its_own_code.sh new file mode 100755 index 00000000..c36d8e8a --- /dev/null +++ b/tests/e2e/735_offline_missing_package_has_its_own_code.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# requires: python3 +# 735 -- an offline plan that needs a package download reports +# MCPP_OFFLINE_DOWNLOAD_REQUIRED, and names the package (#648 L6). +# +# The package comes from a local path index, so the descriptor is present and +# the download is the only thing missing. Criteria: +# A. offline, the envelope's code is MCPP_OFFLINE_DOWNLOAD_REQUIRED, its +# message names the package and its version, and the exit status is 1; +# B. a plan that fails for another reason keeps MCPP_BUILD_DATABASE_PLAN_FAILED. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +mkdir -p "$TMP/index/pkgs/w" "$TMP/app/src" +cat > "$TMP/index/pkgs/w/widget.lua" <<'EOF' +package = { + spec = "1", + namespace = "acme", + name = "widget", + description = "a package whose archive is not installed", + licenses = {"MIT"}, + type = "package", + xpm = { + linux = { ["1.0.0"] = { url = "https://example.invalid/widget-1.0.0.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + macosx = { ["1.0.0"] = { url = "https://example.invalid/widget-1.0.0.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + windows = { ["1.0.0"] = { url = "https://example.invalid/widget-1.0.0.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + }, + mcpp = { language = "c++23", sources = { "src/*.cpp" }, targets = { ["widget"] = { kind = "lib" } } }, +} +EOF +INDEX_HOST="$(host_path "$TMP/index")" +cat > "$TMP/app/mcpp.toml" < "$TMP/app/src/main.cpp" +cd "$TMP/app" + +code_of() { python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print(d["diagnostics"][0]["code"] if d["diagnostics"] else ""); print(d["diagnostics"][0]["message"] if d["diagnostics"] else "")' "$1"; } + +set +e +MCPP_OFFLINE=1 "$MCPP" emit build-database --format json > a.json 2> a.err +rc=$? +set -e +[ "$rc" -eq 1 ] || fail "A: exit status $rc, expected 1" a.err +code_of a.json > a.txt +sed -n 1p a.txt | grep -qx MCPP_OFFLINE_DOWNLOAD_REQUIRED || fail "A: wrong code" a.txt +grep -q "widget" a.txt && grep -q "1.0.0" a.txt || fail "A: the message does not name the package and version" a.txt + +# B. another failure keeps the general code: a dependency no index declares. +sed -i.bak 's/^widget = "1.0.0"/absent-package = "1.0.0"/' mcpp.toml +set +e +MCPP_OFFLINE=1 "$MCPP" emit build-database --format json > b.json 2> b.err +set -e +code_of b.json > b.txt +sed -n 1p b.txt | grep -qx MCPP_BUILD_DATABASE_PLAN_FAILED || fail "B: a selector error took the offline code" b.txt +echo "PASS: 735 an offline plan that needs a download has its own code" From 0f2c7aa4af865b5e23e351e1e614d16384a6f517 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:22:31 +0800 Subject: [PATCH 10/20] docs: refresh bounds and auto_refresh scope, observed network effects, the offline code (#648) --- docs/05-dependencies.md | 9 ++++++++- docs/50-machine-output.md | 16 ++++++++++++++-- docs/specs/build-database.md | 21 +++++++++++++-------- docs/zh/05-dependencies.md | 7 ++++++- docs/zh/50-machine-output.md | 12 ++++++++++-- 5 files changed, 51 insertions(+), 14 deletions(-) diff --git a/docs/05-dependencies.md b/docs/05-dependencies.md index 3ca04056..e8bf29eb 100644 --- a/docs/05-dependencies.md +++ b/docs/05-dependencies.md @@ -289,11 +289,18 @@ Controls, in order of precedence: |---|---| | `--offline` (any command) | Never touch the network — no index refresh, no downloads, no toolchain auto-install, no `git ls-remote`/`clone`. Anything already installed still builds, including git deps whose commit is in `mcpp.lock` and whose clone is cached | | `MCPP_OFFLINE=1` | Same, for a whole shell session or CI job | -| `[index] auto_refresh = false` in `~/.mcpp/config.toml` | Never refresh the index automatically; downloads still work | +| `[index] auto_refresh = false` in `~/.mcpp/config.toml` | Never refresh an index implicitly: not on a dependency miss, not before installing a package the local index lacks, and not for the first sync of a project's custom index (that build stops and names `mcpp index update`). Downloads still work | `MCPP_NO_AUTO_INSTALL=1` remains accepted as the older, narrower spelling of `--offline` (it gates only toolchain auto-install). +A refresh is bounded. `[index] refresh_timeout` (seconds, default 120) is the +longest one refresh may take; a refresh that exceeds it is stopped, a warning +names the setting, and the build continues with the local index, as it does +after any failed refresh. An install through xlings is stopped when xlings +writes nothing, not even its heartbeat, for 300 seconds. Terminating mcpp +terminates the xlings process it started. + Run any command with `-v` to see the decision for each dependency and why. ## `[dev-dependencies]` — Test Dependencies diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 1f82c647..97c038ab 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -169,6 +169,12 @@ cannot separate the harmless from the thing a gate exists for: Most gates care about `exec-build-script` and `write-project`, and can ignore `init-mcpp-home` — mcpp setting itself up is not the workspace acting. +The table above is what a command **may** do. The `effects` of an envelope are +what the run **did**, and `network` is observed rather than declared: it is +listed whenever the run started an index refresh, an install or a git remote +operation, including one that failed or was stopped by its bound, and never +for a run under `--offline`. + ## 5. `--json` is not `--format json` Two commands shipped a `--json` flag before this protocol existed: @@ -395,6 +401,7 @@ a program classifying the outcome reads `reason`: | `host-module-missing` | `build.mcpp` imports a module no dependency supplies as a host module | | `tool-version-conflict` | two declarations name one xlings package at versions that cannot both hold | | `shared-library-cxx-runtime` | a dependency's C++ shared library in a graph whose C++ runtime is a package, with no private copy stated | +| `offline-download-required` | the run is offline and the plan needs a download: a toolchain, a package, a git revision or the package index | | `other` | a refusal whose branch has not been given a token yet | **Exit 0 whenever the question was answered, including "refused".** "Would @@ -430,8 +437,13 @@ no-write guarantee and the `watch` rules are [SPEC-005](specs/build-database.md). A failure omits `data` and exits 1, with the diagnostic code -`MCPP_BUILD_DATABASE_NO_PROJECT` outside a project or -`MCPP_BUILD_DATABASE_PLAN_FAILED` when planning fails. Warnings leave the +`MCPP_BUILD_DATABASE_NO_PROJECT` outside a project, +`MCPP_OFFLINE_DOWNLOAD_REQUIRED` when an offline plan (`--offline`, +`MCPP_OFFLINE`, `MCPP_NO_AUTO_INSTALL`) needs something that has to be +downloaded (a toolchain, a package, a git revision or the package index; the +message names the first one), or `MCPP_BUILD_DATABASE_PLAN_FAILED` when planning +fails for any other reason. The first of the three is not a defect of the +project: one run without `--offline` removes it. Warnings leave the document in place: | code | | diff --git a/docs/specs/build-database.md b/docs/specs/build-database.md index 53f9bd88..36648a17 100644 --- a/docs/specs/build-database.md +++ b/docs/specs/build-database.md @@ -4,12 +4,12 @@ |---|---| | 规范编号 | SPEC-005 | | 标题 | mcpp 输出的构建数据库:内容、取值规则与不写工程目录的保证 | -| 状态 | 评审中 v1.0 | -| 版本 | 1.0 | -| 最后修改 | 2026-09-15 | +| 状态 | 评审中 v1.1 | +| 版本 | 1.1 | +| 最后修改 | 2026-09-16 | | 对应实现 | mcpp >= 2026.9.15.1 | | 相关设计文档 | `.agents/docs/2026-09-14-636-build-database-and-the-latest-xlings.md` | -| 相关 issue | #636 | +| 相关 issue | #636, #648 | | 依据的外部规范 | S1「C++ Build Database: IDE Profile」profile 0.2.0 与 S2 0.2.0 §3.4,取自 https://github.com/Sunrisepeak/lsp-mcpp-private 提交 `b82859d`(schema 自提交 `28ecd6e` 起未变);JSON Compilation Database | ## 0. 适用范围 @@ -144,11 +144,15 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 (`{"name": "s1", "version": "0.2.0"}` 或 `{"name": "compile-commands"}`)、 `database`、`watch` 与 `inputs-fingerprint`。**已实现** - **R5.2** 失败时信封不含 `data`,`diagnostics` 至少含一条 `error`,退出码为 1:不在 - 工程中为 `MCPP_BUILD_DATABASE_NO_PROJECT`,规划失败为 - `MCPP_BUILD_DATABASE_PLAN_FAILED`,工作区中其消息指出成员。工作区中任一成员规划 - 失败,整次命令失败。**已实现** + 工程中为 `MCPP_BUILD_DATABASE_NO_PROJECT`;离线运行而规划需要下载时为 + `MCPP_OFFLINE_DOWNLOAD_REQUIRED`,消息指出需要下载的第一项;其他规划失败为 + `MCPP_BUILD_DATABASE_PLAN_FAILED`。工作区中消息指出成员;任一成员规划失败,整次命令 + 失败。**已实现**(离线诊断码:mcpp >= 2026.9.16.1) - **R5.3** 信封的 `effects` 为 `read-project` 与 `write-global-cache`,运行了构建程序时 - 另有 `exec-build-script`。**已实现** + 另有 `exec-build-script`,本次运行启动过网络子进程(索引刷新、安装、git 远程操作, + 失败或超时的也算)时另有 `network`。**已实现**(`network`:mcpp >= 2026.9.16.1) +- **R5.4** 规划期间启动的子进程不继承调用方读取标准输出的描述符;xlings 子进程有期限, + 并随 mcpp 一起结束。**已实现**(mcpp >= 2026.9.16.1) ## 6. `watch` 与 `inputs-fingerprint` @@ -170,3 +174,4 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 | 版本 | 日期 | 变更 | |---|---|---| | 1.0 | 2026-09-14 | 首版(#636)。 | +| 1.1 | 2026-09-16 | R5.2 增加离线诊断码 `MCPP_OFFLINE_DOWNLOAD_REQUIRED`;R5.3 的 `network` 按观测列出;新增 R5.4(子进程不继承调用方描述符,xlings 子进程有期限并随 mcpp 结束)(#648)。 | diff --git a/docs/zh/05-dependencies.md b/docs/zh/05-dependencies.md index c8363d79..53a3c1ea 100644 --- a/docs/zh/05-dependencies.md +++ b/docs/zh/05-dependencies.md @@ -247,11 +247,16 @@ mcpp index status # 看本地现状:状态、年龄、修订号 |---|---| | `--offline`(任意命令) | 完全不碰网络——不刷索引、不下载、不自动装工具链,也不发 `git ls-remote`/`clone`。已安装的东西照常构建,包括 commit 已在 `mcpp.lock`、克隆已在缓存里的 git 依赖 | | `MCPP_OFFLINE=1` | 同上,作用于整个 shell 会话或 CI job | -| `~/.mcpp/config.toml` 里 `[index] auto_refresh = false` | 永不自动刷新索引,但下载仍然可用 | +| `~/.mcpp/config.toml` 里 `[index] auto_refresh = false` | 永不隐式刷新索引:依赖未命中时不刷新,安装本地索引缺少的包之前不刷新,工程自定义索引的首次同步也不做(该次构建停止并指出 `mcpp index update`)。下载仍然可用 | `MCPP_NO_AUTO_INSTALL=1` 作为 `--offline` 的旧式窄化拼写仍然有效(它只管工具链的 自动安装)。 +刷新有期限。`[index] refresh_timeout`(秒,默认 120)是一次刷新最长可用的时间;超过 +即被终止,一条警告指出该设置,构建与任何一次刷新失败之后一样,继续使用本地索引。经由 +xlings 的安装在 xlings 连续 300 秒没有任何输出(包括心跳)时被终止。结束 mcpp 会一并 +结束它启动的 xlings 进程。 + 任意命令加 `-v` 可以看到每个依赖的判定结果与原因。 ## `[dev-dependencies]` —— 测试依赖 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 246a10c4..13ecb1e5 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -145,6 +145,10 @@ mcpp --protocol-version 多数门只在乎 `exec-build-script` 与 `write-project`,可以忽略 `init-mcpp-home` —— mcpp 给自己做初始化不是工作区在动作。 +上表是命令**可能**做的事。信封里的 `effects` 是这次运行**做了**的事,其中 `network` +按观测记录而不是按声明:本次运行启动过索引刷新、安装或 git 远程操作时即列出,包括失败 +的或被期限终止的那一次;在 `--offline` 下运行时从不列出。 + ## 5. `--json` 不等于 `--format json` 有两条命令在本协议之前就发布了 `--json`: @@ -348,6 +352,7 @@ mcpp why toolchain [--target ] [--toolchain ] --format json | `host-module-missing` | `build.mcpp` 导入了没有任何依赖以 host module 形式提供的模块 | | `tool-version-conflict` | 两处声明把同一个 xlings 包定在不能同时成立的版本上 | | `shared-library-cxx-runtime` | C++ 运行时来自图中的包,而依赖的 C++ 共享库没有声明私有副本 | +| `offline-download-required` | 本次运行离线,而规划需要下载:工具链、包、git 修订或包索引 | | `other` | 一处还没有被命名的拒绝分支 | **只要问题被回答了就退 0,包括答案是「拒绝」。** 「它能不能构建,不能的话 @@ -378,8 +383,11 @@ mcpp emit build-database [--spec s1|compile-commands] --format json 不带 `--format` 时命令只输出文档;`-o ` 把原本输出的内容写入 ``。文档的 内容、不写工程目录的保证与 `watch` 的规则见 [SPEC-005](../specs/build-database.md)。 -失败时省略 `data` 并以 1 退出:不在工程中时诊断码为 `MCPP_BUILD_DATABASE_NO_PROJECT`, -规划失败时为 `MCPP_BUILD_DATABASE_PLAN_FAILED`。警告不影响文档: +失败时省略 `data` 并以 1 退出:不在工程中时诊断码为 `MCPP_BUILD_DATABASE_NO_PROJECT`; +离线规划(`--offline`、`MCPP_OFFLINE`、`MCPP_NO_AUTO_INSTALL`)需要下载某样东西(工具链、 +包、git 修订或包索引,消息指出第一个)时为 `MCPP_OFFLINE_DOWNLOAD_REQUIRED`;其他原因的 +规划失败为 `MCPP_BUILD_DATABASE_PLAN_FAILED`。第一种不是工程的缺陷,不带 `--offline` +运行一次即可消除。警告不影响文档: | 诊断码 | | |---|---| From 3866c91ad28eb5e5394d1be090f6393cfdfdbbbd Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:24:39 +0800 Subject: [PATCH 11/20] e2e 700-705 and 307, and docs 04/20/50: one C++ runtime per process, one image per static package, the llvm row's CRT record, and the cross-image identity measurements (#646, #649 E10) --- docs/04-mcpp-toml.md | 49 +++++- docs/20-toolchains.md | 32 ++++ docs/50-machine-output.md | 2 + docs/zh/04-mcpp-toml.md | 39 ++++- docs/zh/20-toolchains.md | 24 +++ docs/zh/50-machine-output.md | 2 + tests/e2e/307_symbol_provision.sh | 52 +++--- ..._cxx_shared_library_has_one_cxx_runtime.sh | 114 +++++++++++++ ...t_what_the_build_shares_by_construction.sh | 95 +++++++++++ ...nder_a_shared_library_is_linked_into_it.sh | 152 ++++++++++++++++++ ..._on_the_msvc_abi_records_its_static_crt.sh | 70 ++++++++ .../704_cxx_identity_across_macho_images.sh | 18 +++ .../e2e/705_cxx_identity_across_pe_images.sh | 18 +++ tests/e2e/_cxx_identity_across_images_body.sh | 103 ++++++++++++ 14 files changed, 745 insertions(+), 25 deletions(-) create mode 100755 tests/e2e/700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh create mode 100755 tests/e2e/701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh create mode 100755 tests/e2e/702_a_static_package_under_a_shared_library_is_linked_into_it.sh create mode 100755 tests/e2e/703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh create mode 100755 tests/e2e/704_cxx_identity_across_macho_images.sh create mode 100755 tests/e2e/705_cxx_identity_across_pe_images.sh create mode 100644 tests/e2e/_cxx_identity_across_images_body.sh diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 162e403e..ce2b0f01 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -501,6 +501,46 @@ A per-dependency `linkage` is honoured **only in the root project's** final program is laid out; one that genuinely must be a single shared copy says so on its own target instead. +#### A static package under a shared library *(mcpp 2026.9.16.1+)* + +A shared library is linked with the static packages it reaches. Each shared +image of a build has a **static closure**: the static packages reachable from +its package without crossing another shared package. A static package in +exactly one closure, which the root project does not reach itself, is linked +into that image and not into the program. + +Before 2026.9.16.1 such a package went into the program, and the library bound +to the program's copy at run time. That worked on ELF and only for that program: +the library refused `-Wl,-z,defs`, a host that did not link the package could not +load it (`undefined symbol`), Mach-O and PE resolve every reference at link time, +and Android loads an application's shared library before anything that could +supply the package. + +A static package that **several** images reach (two shared libraries, or a shared +library and the program) has no single image to live in: + +- on Mach-O, on PE and on the Android application row the build is refused + before compiling, with reason `static-package-in-two-images` + ([50](50-machine-output.md)); +- on other ELF rows the package stays in the program as before, and the build + reports it (`build/static-placement`), which `--strict` turns into an error. + +The message names the package, the images that reach it, and the remedy: give +the package the shared form, so that every image loads one copy. + +```toml +[dependencies] +x = { path = "../x", linkage = "shared" } # on the root's edge + +# or as the package's own default, in its manifest +[targets.x] +linkage = "shared" +``` + +A package that provides a target layer (`provides = ["mcpp:..."]`, a C library or +a C++ runtime) is outside this rule: where its objects go is the runtime +contract's decision ([20](20-toolchains.md)). + #### `soname` on a library target A `soname` (§2.2) may be declared on `kind = "lib"` as well as @@ -525,7 +565,14 @@ diagnostic exists for this. The check is a measurement, not a declaration: it reads the produced image's dynamic symbol table, removes the entries that are copy relocations, and reports only those a library in the artifact's own closure **also** defines. -An arrangement with one copy in the process is silent. The verdict is recorded +An arrangement with one copy in the process is silent. Three kinds of shared +definition are counted and not reported *(2026.9.16.1+ for the last two)*: +vague linkage, which the loader unifies by design (`STB_WEAK`, and +`STB_GNU_UNIQUE`, which GCC uses for the static data of inline entities); a +definition the build links into both images from **one object**, such as the +`std` module's initialiser in every C++ image that imports `std`; and that same +initialiser against the toolchain's own C++ runtime, which exports it from +GCC 16 on. A name of the same shape defined anywhere else is still a finding. The verdict is recorded in `target///resolution.json` under `runtime.symbol_provision`, with the count and its denominator, so CI can read it without `readelf`. diff --git a/docs/20-toolchains.md b/docs/20-toolchains.md index 7a71c231..512133f0 100644 --- a/docs/20-toolchains.md +++ b/docs/20-toolchains.md @@ -1015,6 +1015,27 @@ A project-wide `cxx_runtime = "…"` (or `static_stdlib = false`) applies to sha libraries too: a human said what the whole project promises. The format-specific default applies only when nobody said anything. +**A program that loads a C++ shared library** (mcpp 2026.9.16.1+). On ELF the two +defaults above disagree inside one process: a self-contained program over a +toolchain-coupled C++ shared library holds a static C++ runtime and a shared one. +The executable exports the runtime symbols the library references, the library +binds some of them to the program's copy and keeps the rest, and the halves +disagree about shared state; measured with `llvm@22.1.8`, such a program aborted +with `std::bad_cast` the first time the library formatted a string, and with +`gcc@16.1.0` it ran with 900 libstdc++ symbols interposed. So on ELF a program or +test whose contract nobody stated takes the shared library's contract when it +loads a C++ shared library the build makes, directly or through another one. The +process already needs that runtime through the library's own `NEEDED` entry, so +no deployment gains a requirement; `resolution.json` records the resulting +contract under `runtime.cxx_runtime_by_role`. + +A stated contract is never changed. A program or test that **states** +`self-contained` while loading a C++ shared library coupled to a shared runtime +is refused before compiling (reason `program-cxx-runtime-split`, +[50](50-machine-output.md)); the ways out are to remove the statement, or to give +the shared library a private copy with `cxx_runtime = { shared = "self-contained" }`, +which keeps each runtime inside its own image. + **A dependency's shared library over a C++ runtime that is a package** (mcpp 2026.9.15.2+). When a package in the graph supplies the C++ layer (`llvm.libcxx`, [22](22-target-side.md)), its objects are linked into the program, and it compiles @@ -1086,6 +1107,17 @@ static CRT leaves no DLL to couple to — so it is reported and resolved to nothing (`--mode system`, `--mode static`) cannot deliver `toolchain-coupled` and refuses. +**Clang on the MSVC ABI** (the `llvm` row of `x86_64-windows-msvc`, mcpp +2026.9.16.1+ for the record). The table above describes `cl.exe`, the one +compiler mcpp passes a CRT model to. Clang on the MSVC ABI speaks the GNU dialect +and receives no model, and its driver links the static CRT (`-defaultlib:libcmt`): +a program built on this row imports no `vcruntime140.dll`, `msvcp140.dll` or +`api-ms-win-crt-*`, and each DLL carries its own CRT. The row is therefore +`self-contained` whatever `cxx_runtime` says, `resolution.json` records it so, and +an explicit `host-coupled` or `toolchain-coupled` prints that the row does not +deliver it. A project that needs the dynamic CRT on the MSVC ABI builds with +`msvc@system`. + **Scope.** The contract governs the C++ runtime only. Static **libc** is a separate axis (`linkage = "static"` / `--static`, e.g. a musl target), and the deployment floor is a third — `macos_deployment_target` in `[package]` for Apple targets, diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 1f82c647..f8514df4 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -395,6 +395,8 @@ a program classifying the outcome reads `reason`: | `host-module-missing` | `build.mcpp` imports a module no dependency supplies as a host module | | `tool-version-conflict` | two declarations name one xlings package at versions that cannot both hold | | `shared-library-cxx-runtime` | a dependency's C++ shared library in a graph whose C++ runtime is a package, with no private copy stated | +| `program-cxx-runtime-split` | a program or test that states a self-contained C++ runtime loads a C++ shared library of the build that couples to a shared one *(2026.9.16.1+)* | +| `static-package-in-two-images` | a static package several images of the build reach, on a target where an image cannot use another image's copy *(2026.9.16.1+)* | | `other` | a refusal whose branch has not been given a token yet | **Exit 0 whenever the question was answered, including "refused".** "Would diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index b73f52cb..56e54337 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -441,6 +441,38 @@ linkage = "shared" # 同上,只在选择器命中的行上 无权决定最终程序的布局;真正必须只有一份共享副本的包,应当在自己的 target 上 声明。 +#### 共享库之下的静态包 *(mcpp 2026.9.16.1+)* + +共享库与它到达的静态包链接在一起。一次构建中的每个共享映像都有一个**静态闭包**: +从它的包出发、不穿过另一个共享包所能到达的静态包。一个静态包若恰好只在一个闭包里, +而根工程自己并不到达它,它就链接进那个映像,而不进程序。 + +2026.9.16.1 之前,这样的包进的是程序,共享库在运行期绑定到程序里的那份。这只在 +ELF 上、且只对那个程序成立:该库通不过 `-Wl,-z,defs`,没有链接这个包的宿主加载不了它 +(`undefined symbol`),Mach-O 与 PE 在链接期就解析每一个引用,Android 则先于任何能 +提供这个包的东西加载应用的共享库。 + +被**多个**映像到达的静态包(两个共享库,或一个共享库加程序)没有唯一可放的映像: + +- 在 Mach-O、PE 与 Android 应用行上,构建在编译之前被拒绝,原因为 + `static-package-in-two-images`([50](50-machine-output.md)); +- 在其他 ELF 行上,这个包照旧留在程序里,构建会报告它(`build/static-placement`), + `--strict` 下为错误。 + +消息会写出这个包、到达它的映像,以及出路:让这个包取共享形态,使每个映像加载同一份。 + +```toml +[dependencies] +x = { path = "../x", linkage = "shared" } # 写在根工程的依赖边上 + +# 或作为这个包自己的默认值,写在它的 manifest 里 +[targets.x] +linkage = "shared" +``` + +提供目标层的包(`provides = ["mcpp:..."]`,C 库或 C++ 运行时)不在这条规则之内: +它的对象放在哪里由运行时契约决定([20](20-toolchains.md))。 + #### library 目标上的 `soname` `soname`(§2.2)在 `kind = "lib"` 上同样可以声明。它是一个库被**找到**时用的 @@ -458,7 +490,12 @@ linkage = "shared" # 同上,只在选择器命中的行上 针对其链接的构建上。链接器和加载器都不会为此报任何一句话。 这项检查是**测量**而不是声明:读产物的动态符号表,去掉 copy relocation,只报告 -产物自身闭包里**也**有定义的那些。进程里只有一份副本的安排保持静默。判定记录在 +产物自身闭包里**也**有定义的那些。进程里只有一份副本的安排保持静默。有三类共同定义 +只计数、不报告 *(后两类自 2026.9.16.1 起)*:加载器按设计统一的 vague linkage +(`STB_WEAK`,以及 GCC 用于内联实体静态数据的 `STB_GNU_UNIQUE`);构建从**同一个 +目标文件**链接进两个映像的定义,例如每个导入 `std` 的 C++ 映像里的 `std` 模块初始化 +函数;以及同一个初始化函数与工具链自身 C++ 运行时之间的重复,GCC 16 起该运行时也导出 +它。形状相同、定义在其他地方的名字仍然会被报告。判定记录在 `target///resolution.json` 的 `runtime.symbol_provision` 下,带计数 与分母,CI 不需要 `readelf` 就能读。 diff --git a/docs/zh/20-toolchains.md b/docs/zh/20-toolchains.md index 56154663..16a4c463 100644 --- a/docs/zh/20-toolchains.md +++ b/docs/zh/20-toolchains.md @@ -916,6 +916,22 @@ C++ 运行时的进程。 工程级的 `cxx_runtime = "…"`(或 `static_stdlib = false`)同样作用于共享库: 有人写下了整个工程的承诺。只有在**没人写**的时候,随格式变化的默认值才生效。 +**加载 C++ 共享库的程序**(mcpp 2026.9.16.1+)。在 ELF 上,上面两个默认值会在同一个 +进程里冲突:self-contained 的程序加载 toolchain-coupled 的 C++ 共享库,进程里就同时 +有一份静态 C++ 运行时和一份共享的。可执行文件导出该库引用的运行时符号,库把其中一部分 +绑到程序的那份、其余留给自己,两半对共享状态的认识互不一致;实测 `llvm@22.1.8` 下这样的 +程序在库第一次格式化字符串时以 `std::bad_cast` 中止,`gcc@16.1.0` 下能跑但有 900 个 +libstdc++ 符号被抢占。因此在 ELF 上,没有人声明契约的程序或测试,若直接或经由另一个 +共享库加载本次构建产出的 C++ 共享库,就取该共享库的契约。进程本来就通过库自己的 +`NEEDED` 条目需要那份运行时,所以没有任何部署因此多出依赖;`resolution.json` 在 +`runtime.cxx_runtime_by_role` 下记录最终的契约。 + +声明过的契约从不改动。**声明了** `self-contained` 的程序或测试,若加载耦合到共享运行时的 +C++ 共享库,会在编译前被拒绝(reason `program-cxx-runtime-split`,见 +[50](50-machine-output.md));出路是删掉这条声明,或用 +`cxx_runtime = { shared = "self-contained" }` 给共享库一份私有副本,让每份运行时留在 +各自的映像里。 + **C++ 运行时来自图中的包时,依赖的共享库**(mcpp 2026.9.15.2+)。当图中有包提供 C++ 层(`llvm.libcxx`,见 [22](22-target-side.md)),它的对象被链进程序,并且以隐藏 可见性编译,于是一个构建为 C++ 共享库的依赖无法解析到程序里的那份。mcpp 在编译前 @@ -975,6 +991,14 @@ Windows 组件(Win10 起),mcpp 从不分发它;而 `vcruntime140.dll` / 所以会被报出来并落到 `self-contained`。另一半由 `mcpp pack` 兜底:什么都不打包的 模式(`--mode system`、`--mode static`)兑现不了 `toolchain-coupled`,会直接拒绝。 +**MSVC ABI 上的 clang**(`x86_64-windows-msvc` 的 `llvm` 行,记录自 mcpp 2026.9.16.1 +起)。上表描述的是 `cl.exe`,mcpp 只向它传递 CRT 模型。MSVC ABI 上的 clang 使用 GNU +方言、收不到任何模型,它的驱动链接静态 CRT(`-defaultlib:libcmt`):这一行构建的程序 +不导入 `vcruntime140.dll`、`msvcp140.dll` 或 `api-ms-win-crt-*`,每个 DLL 各带一份 CRT。 +因此这一行无论 `cxx_runtime` 写什么都是 `self-contained`,`resolution.json` 如实记录, +显式写 `host-coupled` 或 `toolchain-coupled` 会打印这一行兑现不了它。需要在 MSVC ABI +上使用动态 CRT 的工程,用 `msvc@system` 构建。 + **边界。** 该契约只管 C++ 运行时。静态 **libc** 是另一根轴(`linkage = "static"` / `--static`,如 musl 目标),部署下限是第三根轴 —— Apple 目标用 `[package]` 里的 `macos_deployment_target`,Android 用 `[target.]` 下的 `min_api_level`。 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 246a10c4..9cecd9e1 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -348,6 +348,8 @@ mcpp why toolchain [--target ] [--toolchain ] --format json | `host-module-missing` | `build.mcpp` 导入了没有任何依赖以 host module 形式提供的模块 | | `tool-version-conflict` | 两处声明把同一个 xlings 包定在不能同时成立的版本上 | | `shared-library-cxx-runtime` | C++ 运行时来自图中的包,而依赖的 C++ 共享库没有声明私有副本 | +| `program-cxx-runtime-split` | 声明了自含 C++ 运行时的程序或测试,加载了本次构建中耦合到共享运行时的 C++ 共享库 *(2026.9.16.1+)* | +| `static-package-in-two-images` | 一个静态包被本次构建的多个映像到达,而在该目标上一个映像不能使用另一个映像里的副本 *(2026.9.16.1+)* | | `other` | 一处还没有被命名的拒绝分支 | **只要问题被回答了就退 0,包括答案是「拒绝」。** 「它能不能构建,不能的话 diff --git a/tests/e2e/307_symbol_provision.sh b/tests/e2e/307_symbol_provision.sh index c6267891..11ffbc20 100755 --- a/tests/e2e/307_symbol_provision.sh +++ b/tests/e2e/307_symbol_provision.sh @@ -1,22 +1,23 @@ #!/usr/bin/env bash -# requires: elf +# requires: elf python3 # 307_symbol_provision.sh — one library, one provider (issue #519). # # THE SILENT CASE IS THE IMPORTANT ONE, and it is why this test has two -# halves that differ by a single package. +# halves. # -# mcpp's own `kind = "shared"` mechanism produces the exact shape the -# diagnostic looks for: a shared dependency's link unit takes only ITS OWN -# objects, so a static package underneath it lands in the consumer's -# executable and the shared library binds back to it at run time. There is one -# copy of the code in the process and nothing is wrong. A check that reported -# "this image exports a symbol a library it loads binds to" would fire on -# every correct build of that shape, and the user could do nothing about it. +# A shared dependency over a static package used to put the static package in +# the consumer's executable, and the shared library bound back to it at run +# time; this test then asserted that the executable's export stayed silent. +# Since #646 F1 the static package is linked INTO the shared library, which is +# the only arrangement that works on every format, so the executable exports +# nothing for it and the check still measures the image (half 1). # -# The finding is the SECOND provider, not the export. So: +# The finding is a SECOND provider of a name the executable holds. So: # -# half 1 wrap.so (shared) → core (static) must stay SILENT -# half 2 ... plus alt.so, which defines the same name must REPORT both +# half 1 app -> wrap.so (shared) -> core (static) must stay SILENT, +# with core linked into wrap.so +# half 2 app -> core (static) and alt.so, which defines the same name and +# calls it must REPORT both # # Asserted against `resolution.json` rather than stdout: the message is free # to improve, the recorded verdict is the contract. @@ -105,16 +106,20 @@ status="$(verdict "$res" status)" echo "FAIL: the legitimate shared→static arrangement reported '$status'" cat "$res"; exit 1; } -# It must have been MEASURED, not skipped: the export is real and the -# denominator is real. "0 findings" and "never looked" must not read alike. +# It must have been MEASURED, not skipped: the denominator is real. "0 +# findings" and "never looked" must not read alike. exported="$(verdict "$res" exported)" total="$(verdict "$res" dynamic_symbols)" -[[ "$exported" -ge 1 ]] || { - echo "FAIL: expected the executable to export shared_answer, got $exported" - exit 1; } [[ "$total" -gt "$exported" ]] || { echo "FAIL: implausible denominator ($exported of $total)"; exit 1; } +# core lives in wrap.so now, not in the executable (#646 F1): a host that never +# linked core loads it. +wrapso="$(ls "$(dirname "$res")"/bin/libwrap.so 2>/dev/null | head -1)" +[[ -n "$wrapso" ]] || { echo "FAIL: no libwrap.so"; exit 1; } +"$PYTHON" -c 'import ctypes, os, sys; assert ctypes.CDLL(sys.argv[1], mode=os.RTLD_NOW).wrap_call() == 8' "$wrapso" \ + || { echo "FAIL: a host that did not link core cannot load libwrap.so"; exit 1; } + grep -qi "also provided by" silent.log && { echo "FAIL: a correct build printed a conflict warning"; exit 1; } @@ -144,14 +149,14 @@ c_standard = "c11" kind = "bin" main = "src/main.c" [dependencies] -wrap = { path = "../wrap" } +core = { path = "../core" } alt = { path = "../alt" } EOF cat > src/main.c <<'EOF' #include -int wrap_call(void); +int shared_answer(void); int alt_ping(void); -int main(void) { printf("%d %d\n", wrap_call(), alt_ping()); return 0; } +int main(void) { printf("%d %d\n", shared_answer(), alt_ping()); return 0; } EOF "$MCPP" build > conflict.log 2>&1 || { cat conflict.log; exit 1; } @@ -186,11 +191,12 @@ done cat conflict.log; exit 1; } # The defect is real, not theoretical: alt's 999 is unreachable because the -# executable's merged copy wins for every caller in the process. +# executable's copy wins for every caller in the process, alt's own call +# included. cbin="$conflict_dir/bin/app" [[ -x "$cbin" ]] || { echo "FAIL: no executable in $conflict_dir"; exit 1; } -"$cbin" | grep -qx "8 7" || { - echo "FAIL: expected '8 7' (the merged copy winning), got: $("$cbin")"; exit 1; } +"$cbin" | grep -qx "7 7" || { + echo "FAIL: expected '7 7' (the executable's copy winning), got: $("$cbin")"; exit 1; } # ── --strict turns the finding into a failure, in ONE place ──────────────── touch src/main.c diff --git a/tests/e2e/700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh b/tests/e2e/700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh new file mode 100755 index 00000000..be2af16d --- /dev/null +++ b/tests/e2e/700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh @@ -0,0 +1,114 @@ +#!/usr/bin/env bash +# requires: llvm elf +# 700 -- one process, one C++ runtime (#646 F3a), read with llvm@22.1.8. +# +# The ELF defaults gave a program `self-contained` and a shared library +# `toolchain-coupled`. A program that loads a C++ shared library therefore held +# a static libc++ and `libc++.so.1`: the executable exported the runtime symbols +# the library referenced, the library bound some of them there and kept the +# rest, and the program aborted the first time the library formatted a string: +# +# libc++abi: terminating due to uncaught exception of type std::bad_cast +# +# (measured on 2026.9.15.2, exit 134, the build itself green). An unstated +# program contract now takes the shared library's when the program loads a C++ +# shared library of the build. Three legs: the default shape runs, with the +# program coupled to the same runtime and the record saying so; `--strict` +# passes, so the symbol-provision check no longer reports the module +# initialiser both images link; a stated self-contained program over the +# coupled library is refused before compiling, with its reason. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME=${MCPP_HOME:-$HOME/.mcpp} + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +READELF=$(command -v readelf || true) +[ -n "$READELF" ] || READELF=$(ls "$MCPP_HOME"/registry/data/xpkgs/xim-x-llvm/22.1.8/bin/llvm-readelf 2>/dev/null | head -1) +[ -n "$READELF" ] || fail "no readelf and no llvm-readelf to read NEEDED with" + +cd "$TMP" +mkdir -p lib/src app/src +cat > lib/mcpp.toml <<'TOML' +[package] +name = "lib" +version = "0.1.0" + +[targets.lib] +kind = "shared" +TOML +cat > lib/src/lib.cppm <<'CPP' +export module lib; +import std; +export [[gnu::visibility("default")]] std::string lib_greet(int n); +CPP +cat > lib/src/lib.cpp <<'CPP' +module lib; +import std; +std::string lib_greet(int n) { return std::format("lib-{}", n); } +CPP + +write_app() { # $1 = extra [build] lines + cat > app/mcpp.toml < app/src/main.cpp <<'CPP' +import std; +import lib; +int main() { std::println("{}", lib_greet(3)); } +CPP + +# ── The default shape runs, coupled to one runtime ────────────────────────── +write_app '' +cd app +"$MCPP" build --strict > default.log 2>&1 \ + || fail "the default shape did not build under --strict" default.log +dir=$(ls -d target/*/*/ | head -1) +out=$("$dir/bin/app" 2>&1) || fail "the program aborted: $out" default.log +[ "$out" = "lib-3" ] || fail "expected lib-3, got: $out" +"$READELF" -d "$dir/bin/app" | grep -q 'Shared library: \[libc++.so.1\]' \ + || fail "the program does not load libc++.so.1, so it carries its own runtime" default.log +grep -q '"distributable": *"toolchain-coupled"' "$dir/resolution.json" \ + || fail "resolution.json does not record the program as toolchain-coupled" "$dir/resolution.json" +grep -q 'also provided by a library it loads' default.log \ + && fail "the symbol-provision check reported the shared shape" default.log +echo "ok: a program over a C++ shared library runs on one C++ runtime" +cd .. + +# ── A stated self-contained program over the coupled library is refused ───── +write_app 'cxx_runtime = { default = "self-contained", shared = "toolchain-coupled" }' +cd app +rm -rf target +if "$MCPP" build > split.log 2>&1; then + fail "a stated self-contained program over a coupled C++ shared library was not refused" split.log +fi +grep -q "state a self-contained C++ runtime and load the C++ shared library 'lib'" split.log \ + || fail "the refusal does not name the statement and the library" split.log +grep -q 'cxx_runtime = { shared = "self-contained" }' split.log \ + || fail "the refusal does not name the private-copy remedy" split.log +ninja=$(ls target/*/*/build.ninja 2>/dev/null | head -1 || true) +[ -z "$ninja" ] || ! grep -q '^build bin/liblib.so' "$ninja" \ + || fail "a build graph was written before the refusal" "$ninja" +if command -v jq >/dev/null 2>&1; then + "$MCPP" why toolchain --format json > why.json 2>/dev/null || true + reason=$(jq -r '.data.reason // empty' why.json 2>/dev/null || true) + [ "$reason" = "program-cxx-runtime-split" ] \ + || fail "the machine reason was '$reason', expected program-cxx-runtime-split" why.json +fi +echo "ok: a stated self-contained program over a coupled C++ shared library is refused" + +echo "PASS: 700 a program over a C++ shared library has one C++ runtime" diff --git a/tests/e2e/701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh b/tests/e2e/701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh new file mode 100755 index 00000000..e9c2c138 --- /dev/null +++ b/tests/e2e/701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +# requires: elf python3 +# 701 -- the symbol-provision check reports what the build did not intend +# (#646 F3), read with the default toolchain. +# +# A program over a C++ shared library, both importing `std`, reported symbols +# "provided twice" that nothing in the project had done wrong: +# +# * 882 to 900 libstdc++ symbols under the default contracts: the program +# carried a static libstdc++ beside the library's libstdc++.so.6 (#646 +# F3a, now one runtime); +# * under one runtime, the `std` module initialiser (`_ZGIW3std`), which +# every C++ image importing `std` links from one `std.o`, and which GCC 16's +# libstdc++.so.6 exports as well; +# * seven STB_GNU_UNIQUE objects, which the loader unifies by design. +# +# Every finding is a degradation, so `--strict` failed the build. This asserts +# the default shape is clean in the record, passes `--strict` and runs; the +# real duplicate the check exists for is 307's second half. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +PYTHON="$(command -v python3 || command -v python || true)" +[[ -n "$PYTHON" ]] || { echo "skip: no python for JSON assertions"; exit 0; } + +cd "$TMP" +mkdir -p lib/src app/src +cat > lib/mcpp.toml <<'TOML' +[package] +name = "lib" +version = "0.1.0" + +[targets.lib] +kind = "shared" +TOML +cat > lib/src/lib.cppm <<'CPP' +export module lib; +import std; +export [[gnu::visibility("default")]] std::string lib_greet(int n); +CPP +cat > lib/src/lib.cpp <<'CPP' +module lib; +import std; +std::string lib_greet(int n) { return std::format("lib-{}", n); } +CPP +cat > app/mcpp.toml <<'TOML' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +lib = { path = "../lib" } +TOML +cat > app/src/main.cpp <<'CPP' +import std; +import lib; +int main() { std::println("{}", lib_greet(3)); } +CPP + +cd app +"$MCPP" build --strict > strict.log 2>&1 \ + || fail "a program over a C++ shared library failed --strict" strict.log +grep -q 'also provided by a library it loads' strict.log \ + && fail "the check reported the shape the build creates by construction" strict.log +dir=$(ls -d target/*/*/ | head -1) +out=$("$dir/bin/app" 2>&1) || fail "the program exited non-zero: $out" +[ "$out" = "lib-3" ] || fail "expected lib-3, got: $out" + +verdict=$("$PYTHON" - "$dir/resolution.json" <<'PY' +import json, sys +doc = json.load(open(sys.argv[1])) +entries = doc.get("runtime", {}).get("symbol_provision") or [] +app = [e for e in entries if e["path"].endswith("bin/app")] +if not app: + print("NO-APP-ENTRY") +else: + e = app[0] + print(e.get("status"), e.get("exported", 0), e.get("dynamic_symbols", 0), + len(e.get("conflicts", []))) +PY +) +set -- $verdict +[ "$1" = "clean" ] || fail "the record says '$verdict', expected a clean verdict" "$dir/resolution.json" +# Measured, not skipped: the program does export what the library binds to, and +# the denominator is real. "clean" must not be the reading of a check that did +# not look. +[ "${2:-0}" -ge 1 ] || fail "the program exports nothing, so the check did not measure the shape" "$dir/resolution.json" +[ "${3:-0}" -gt "${2:-0}" ] || fail "implausible denominator: $verdict" "$dir/resolution.json" +echo "ok: the default shape is clean in the record and passes --strict" + +echo "PASS: 701 symbol provision does not report what the build shares by construction" diff --git a/tests/e2e/702_a_static_package_under_a_shared_library_is_linked_into_it.sh b/tests/e2e/702_a_static_package_under_a_shared_library_is_linked_into_it.sh new file mode 100755 index 00000000..bdef3597 --- /dev/null +++ b/tests/e2e/702_a_static_package_under_a_shared_library_is_linked_into_it.sh @@ -0,0 +1,152 @@ +#!/usr/bin/env bash +# requires: unix-shell python3 +# 702 -- one static package, one image (#646 F1). +# +# A dependency's shared library was linked from its own package's objects only. +# A static package beneath it went into the PROGRAM, and on ELF the library +# bound to the program's copy at run time. That worked for that program alone: +# measured on Linux, the same `libfw.so` refused `-Wl,-z,defs` and a host that +# did not link the package could not load it (`undefined symbol: x_answer`); +# Mach-O and PE resolve every reference at link time, so the library did not +# link there at all. +# +# Three legs: +# A. fw (shared) over x (static): x is linked into libfw, which then loads +# from a host that never linked x (Python's ctypes, RTLD_NOW); +# B. the program depends on x as well: no single image can own x. On ELF the +# build proceeds as it always did and says so (`--strict` fails); on +# Mach-O the build is refused before compiling, naming the remedy; +# C. the remedy the message names, `linkage = "shared"` on x, builds under +# `--strict` and runs. +set -e + +case "$(uname -s)" in + MINGW*|MSYS*|CYGWIN*) echo "skip: the PE legs of 702 are not wired on this shard"; exit 0 ;; +esac + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +PYTHON="$(command -v python3 || command -v python || true)" +[[ -n "$PYTHON" ]] || fail "no python to load the library from a foreign host" + +MACHO=0 +[ "$(uname -s)" = "Darwin" ] && MACHO=1 +SO=so +[ $MACHO = 1 ] && SO=dylib + +cd "$TMP" +mkdir -p x/src fw/src app/src +cat > x/mcpp.toml <<'TOML' +[package] +name = "x" +version = "0.1.0" + +[build] +sources = ["src/*.c"] + +[targets.x] +kind = "lib" +TOML +cat > x/src/x.c <<'C' +int x_answer(void) { return 41; } +C +cat > fw/mcpp.toml <<'TOML' +[package] +name = "fw" +version = "0.1.0" + +[build] +sources = ["src/*.c"] + +[targets.fw] +kind = "shared" + +[dependencies] +x = { path = "../x" } +TOML +cat > fw/src/fw.c <<'C' +extern int x_answer(void); +__attribute__((visibility("default"))) int fw_answer(void) { return x_answer() + 1; } +C + +write_app() { # $1 = extra [dependencies] lines, $2 = main body + cat > app/mcpp.toml < app/src/main.cpp +} + +# ── A. x belongs in libfw ──────────────────────────────────────────────────── +write_app '' 'extern "C" int fw_answer(void); +int main() { return fw_answer() == 42 ? 0 : 1; }' +cd app +"$MCPP" build > a.log 2>&1 || fail "a shared library over a static package did not build" a.log +dir=$(ls -d target/*/*/ | head -1) +"$dir/bin/app" || fail "the program over libfw exited non-zero" a.log +lib="$dir/bin/libfw.$SO" +[ -f "$lib" ] || fail "no libfw.$SO" a.log +loaded=$("$PYTHON" - "$lib" <<'PY' 2>&1 +import ctypes, os, sys +lib = ctypes.CDLL(sys.argv[1], mode=os.RTLD_NOW) +print(lib.fw_answer()) +PY +) || fail "a host that never linked x cannot load libfw: $loaded" a.log +[ "$loaded" = "42" ] || fail "expected 42 from libfw in a foreign host, got: $loaded" +echo "ok: a static package reachable only through a shared library is linked into it" +cd .. + +# ── B. the program reaches x too ──────────────────────────────────────────── +write_app 'x = { path = "../x" }' 'extern "C" int fw_answer(void); +extern "C" int x_answer(void); +int main() { return fw_answer() == 42 && x_answer() == 41 ? 0 : 1; }' +cd app +rm -rf target +if [ $MACHO = 1 ]; then + if "$MCPP" build > b.log 2>&1; then + fail "a static package in two Mach-O images was not refused" b.log + fi + grep -q "a static package is linked into more than one image" b.log \ + || fail "the refusal does not say what it refuses" b.log + grep -q "'mcpplibs.x' is reached by the program, 'fw'" b.log \ + || fail "the refusal does not name the package and its images" b.log + grep -q 'linkage = "shared"' b.log || fail "the refusal does not name the remedy" b.log + echo "ok: a static package in two Mach-O images is refused before compiling" +else + "$MCPP" build > b.log 2>&1 || fail "the two-image graph no longer builds on ELF" b.log + grep -q "a static package is reachable from more than one image" b.log \ + || fail "the ELF build does not report the two-image package" b.log + grep -q "'mcpplibs.x' is reached by the program, 'fw'" b.log \ + || fail "the report does not name the package and its images" b.log + dir=$(ls -d target/*/*/ | head -1) + "$dir/bin/app" || fail "the two-image program exited non-zero" b.log + rm -rf target + if "$MCPP" build --strict > b-strict.log 2>&1; then + fail "--strict accepted a static package in two images" b-strict.log + fi + echo "ok: a static package in two ELF images builds as before and is reported" +fi +cd .. + +# ── C. the remedy the message names ───────────────────────────────────────── +write_app 'x = { path = "../x", linkage = "shared" }' 'extern "C" int fw_answer(void); +extern "C" int x_answer(void); +int main() { return fw_answer() == 42 && x_answer() == 41 ? 0 : 1; }' +cd app +rm -rf target +"$MCPP" build --strict > c.log 2>&1 || fail "linkage = \"shared\" on x did not build under --strict" c.log +grep -q "more than one image" c.log && fail "the shared form still reports the two-image package" c.log +dir=$(ls -d target/*/*/ | head -1) +[ -f "$dir/bin/libx.$SO" ] || fail "no libx.$SO under linkage = \"shared\"" c.log +"$dir/bin/app" || fail "the program over the shared x exited non-zero" c.log +echo "ok: linkage = \"shared\" gives the package one image" + +echo "PASS: 702 a static package under a shared library is linked into it" diff --git a/tests/e2e/703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh b/tests/e2e/703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh new file mode 100755 index 00000000..a5017bc2 --- /dev/null +++ b/tests/e2e/703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +# requires: windows +# 703 -- the C++ runtime record of clang on the MSVC ABI (#649 E10). +# +# mcpp emits a CRT model (`/MT` or `/MD`) only for cl.exe. clang on the MSVC ABI +# receives none, and its driver links the static CRT (`-defaultlib:libcmt`), so +# the program imports no vcruntime or ucrt DLL. The contract table was written +# for cl.exe and recorded `host-coupled` for that artifact; an explicit +# `cxx_runtime = "host-coupled"` was recorded and not delivered, with no word. +# The record now states `self-contained`, and the explicit request prints that +# the row does not deliver it. Nothing about the artifact changes. +# +# Read only when the default toolchain on this runner is the llvm row: a runner +# whose default is msvc@system prints that and asserts nothing, because the +# cl.exe cells are unchanged and covered elsewhere. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'CPP' +import std; +int main() { std::println("crt"); } +CPP +write_app() { # $1 = [build] lines + cat > app/mcpp.toml < default.log 2>&1 || fail "the default build failed" default.log +if ! grep -q "Resolved llvm@" default.log; then + echo "READING E10: the default toolchain here is not the llvm row: $(grep -m1 'Resolved' default.log)" + echo "PASS: 703 the llvm row on the MSVC ABI records its static CRT (not the llvm row; nothing to assert)" + exit 0 +fi +contract=$(record_of) +echo "READING E10 default record: distributable=$contract" +[ "$contract" = "self-contained" ] \ + || fail "the llvm row recorded '$contract' for a program linked with the static CRT" default.log + +cd .. +write_app 'cxx_runtime = "host-coupled"' +cd app +rm -rf target +"$MCPP" build > host.log 2>&1 || fail "the host-coupled build failed" host.log +grep -q 'is not delivered for clang on the MSVC ABI' host.log \ + || fail "an explicit host-coupled request on the llvm row printed nothing" host.log +contract=$(record_of) +echo "READING E10 host-coupled record: distributable=$contract" +[ "$contract" = "self-contained" ] \ + || fail "the undelivered request was recorded as '$contract'" host.log +echo "ok: the llvm row records the static CRT it links, and says an explicit dynamic request is not delivered" + +echo "PASS: 703 the llvm row on the MSVC ABI records its static CRT" diff --git a/tests/e2e/704_cxx_identity_across_macho_images.sh b/tests/e2e/704_cxx_identity_across_macho_images.sh new file mode 100755 index 00000000..efbefe4a --- /dev/null +++ b/tests/e2e/704_cxx_identity_across_macho_images.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash +# requires: macos +# 704 -- a measurement leg (#646 F2): on Mach-O, the payload's default contract +# gives every dylib a private libc++ linked with `-load_hidden`. This prints, +# for a program over a C++ dylib under three runtime statements, whether a +# standard exception thrown in the dylib is caught by its class, whether the +# dylib's own exception class is, and whether an `std::error_code` compares +# equal across the boundary. See `_cxx_identity_across_images_body.sh`. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +IMAGE_FORMAT=macho +source "$(dirname "$0")/_cxx_identity_across_images_body.sh" + +echo "PASS: 704 C++ identity across Mach-O images was measured" diff --git a/tests/e2e/705_cxx_identity_across_pe_images.sh b/tests/e2e/705_cxx_identity_across_pe_images.sh new file mode 100755 index 00000000..5747e6f9 --- /dev/null +++ b/tests/e2e/705_cxx_identity_across_pe_images.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash +# requires: windows +# 705 -- a measurement leg (#646 F2, #649 E10): on the MSVC ABI with the default +# toolchain, every DLL links its own C and C++ runtime. This prints, for a +# program over a C++ DLL under three runtime statements, whether a standard +# exception thrown in the DLL is caught by its class, whether the DLL's own +# exception class is, and whether an `std::error_code` compares equal across the +# boundary. See `_cxx_identity_across_images_body.sh`. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +IMAGE_FORMAT=pe +source "$(dirname "$0")/_cxx_identity_across_images_body.sh" + +echo "PASS: 705 C++ identity across PE images was measured" diff --git a/tests/e2e/_cxx_identity_across_images_body.sh b/tests/e2e/_cxx_identity_across_images_body.sh new file mode 100644 index 00000000..c5d82d67 --- /dev/null +++ b/tests/e2e/_cxx_identity_across_images_body.sh @@ -0,0 +1,103 @@ +# tests/e2e/_cxx_identity_across_images_body.sh -- the measurement body shared +# by 704 (Mach-O) and 705 (PE). Sourced, not run: the caller sets `set -e`, +# `TMP`, `fail`, and `IMAGE_FORMAT` (a label for the READING lines). +# +# WHAT IS MEASURED (#646 F2, #649 E10). Each image that embeds a private copy +# of the C++ runtime holds its own type information and its own error +# categories. Whether a `std::runtime_error` thrown in a shared library is +# caught by its class in the program, and whether an `std::error_code` made in +# the library compares equal to an `std::errc` in the program, depends on how +# the platform compares those identities across images. libc++ documents +# address comparison for such types on Apple platforms; MSVC matches catch +# clauses by decorated name. This body builds one fixture under three runtime +# statements and PRINTS what each one does. It does not fail on the value it +# reads: the decision taken from the reading is recorded in the design record, +# not here. It fails only when the default leg cannot be built or run, since +# then nothing was measured. + +mkdir -p "$TMP/fw/src" "$TMP/app/src" +cat > "$TMP/fw/mcpp.toml" <<'TOML' +[package] +name = "fw" +version = "0.1.0" + +[targets.fw] +kind = "shared" +TOML +cat > "$TMP/fw/src/fw.cppm" <<'CPP' +export module fw; +import std; +export struct fw_error : std::runtime_error { + using std::runtime_error::runtime_error; +}; +export [[gnu::visibility("default")]] void fw_throw_standard(); +export [[gnu::visibility("default")]] void fw_throw_own(); +export [[gnu::visibility("default")]] std::error_code fw_code(); +CPP +cat > "$TMP/fw/src/fw.cpp" <<'CPP' +module fw; +import std; +void fw_throw_standard() { throw std::runtime_error("from fw"); } +void fw_throw_own() { throw fw_error("own"); } +std::error_code fw_code() { return std::make_error_code(std::errc::no_such_file_or_directory); } +CPP +cat > "$TMP/app/src/main.cpp" <<'CPP' +import std; +import fw; +int main() { + const char* standard = "not-matched"; + try { fw_throw_standard(); } + catch (const std::runtime_error&) { standard = "caught"; } + catch (...) {} + const char* own = "not-matched"; + try { fw_throw_own(); } + catch (const fw_error&) { own = "caught"; } + catch (...) {} + const bool equal = fw_code() == std::errc::no_such_file_or_directory; + std::println("runtime_error={} own_error={} errc={}", standard, own, + equal ? "equal" : "unequal"); +} +CPP + +measure_leg() { # $1 = leg name, $2 = [build] lines + cat > "$TMP/app/mcpp.toml" < "build-$1.log" 2>&1; then + echo "READING $IMAGE_FORMAT $1: build-failed $(grep -m1 -E 'error' "build-$1.log" | cut -c1-160)" + exit 3 + fi + local exe + exe=$(find target -type f \( -name app -o -name app.exe \) -path '*bin*' | head -1) + local out + if out=$("$exe" 2>&1); then + echo "READING $IMAGE_FORMAT $1: $out" + else + echo "READING $IMAGE_FORMAT $1: run-failed exit=$? $out" + exit 4 + fi + local contracts + contracts=$(tr -d ' \n\r' < "$(find target -name resolution.json | head -1)" 2>/dev/null \ + | grep -o '"cxx_runtime_by_role":{[^}]*}' || true) + echo "READING $IMAGE_FORMAT $1 record: ${contracts:-none}" + ) +} + +set +e +measure_leg default '' +default_rc=$? +measure_leg host-coupled 'cxx_runtime = "host-coupled"' +measure_leg shared-host-coupled 'cxx_runtime = { shared = "host-coupled" }' +set -e +[ "$default_rc" -eq 0 ] || fail "the default leg could not be measured (rc $default_rc)" "$TMP/app/build-default.log" From 3c3fe5aafa5b0dd1b0d52a55b063fa2b796d68da Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:29:55 +0800 Subject: [PATCH 12/20] Choose the link line by host and target object format (#647 E3) An Android row built on a macOS host received the Apple SDK line, which names no --target, so -fuse-ld=lld selected ld64.lld for an ELF object. link_shape takes the host and the target's object format; a target outside the host's own family takes the generic line that carries the target. Unit LinkShape.* states the choice for every host; e2e 721 links the row on macOS. --- src/build/flags.cppm | 82 ++++++++++++-- ...21_an_android_row_links_on_a_macos_host.sh | 102 ++++++++++++++++++ tests/unit/test_build_flags.cpp | 43 ++++++++ 3 files changed, 220 insertions(+), 7 deletions(-) create mode 100755 tests/e2e/721_an_android_row_links_on_a_macos_host.sh diff --git a/src/build/flags.cppm b/src/build/flags.cppm index ca7e9750..2a5f9136 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -139,6 +139,42 @@ std::string render_link_intent_flags( CompileFlags compute_flags(const BuildPlan& plan); +// ── Which link line a (host, target) pair takes (#647 E3) ───────────────── +// +// THE HOST DOES NOT DECIDE THIS ALONE. Three of the link branches describe a +// link on this machine's own platform family: `link.exe` for the MSVC dialect, +// `-fuse-ld=lld` without a payload model for a PE, and the Apple SDK line +// (`-isysroot`, the deployment floor) for a Mach-O. They used to be selected by +// `if constexpr` on the HOST, which was the same question while each host built +// only for its own family. An Android row built on a macOS host is the first +// target that separated them, and the Apple line it received had no +// `--target`, so `-fuse-ld=lld` selected `ld64.lld` for an ELF object: +// +// ld64.lld: error: unknown argument '-soname' +// +// The generic branch is the one that consumes `link_toolchain_flags`, where +// the target is named. Every target a host branch does not describe takes it. +// +// ONE EXCEPTION IS KEPT, AND IT IS STATED BY ITS INPUT. On a Windows host a +// non-PE target whose driver is NOT told its target by a flag (a canadian GCC +// cross, which names its target by its own prefix) keeps the PE-host line it +// has always had: nothing on that line is wrong for such a driver, and the +// cross build that uses it is verified in CI. A target named by `--target` +// (an SDK such as the NDK, or a retargetable clang) is exactly the case the +// host line cannot serve. +enum class LinkHost { Linux, MacOS, Windows }; +enum class LinkShape { MsvcLinkExe, WindowsLld, AppleSdk, Generic }; + +LinkShape link_shape(LinkHost host, mcpp::build::dist::Format targetFormat, + bool msvcDialect, bool targetNamedByFlag); + +// The host this binary was built for, in `link_shape`'s vocabulary. +constexpr LinkHost current_link_host() { + return mcpp::platform::is_windows ? LinkHost::Windows + : mcpp::platform::needs_explicit_libcxx ? LinkHost::MacOS + : LinkHost::Linux; +} + // The kind → role map. One line of policy, in one place: a test binary runs on // the build machine and is then thrown away; an archive embeds no runtime at // all; everything else leaves this machine. Backends ask this, never the kind. @@ -424,6 +460,23 @@ std::string render_link_intent_flags( return out; } +LinkShape link_shape(LinkHost host, mcpp::build::dist::Format targetFormat, + bool msvcDialect, bool targetNamedByFlag) { + using mcpp::build::dist::Format; + switch (host) { + case LinkHost::Windows: + if (msvcDialect) return LinkShape::MsvcLinkExe; + if (targetFormat == Format::Pe) return LinkShape::WindowsLld; + return targetNamedByFlag ? LinkShape::Generic : LinkShape::WindowsLld; + case LinkHost::MacOS: + return targetFormat == Format::MachO ? LinkShape::AppleSdk + : LinkShape::Generic; + case LinkHost::Linux: + return LinkShape::Generic; + } + return LinkShape::Generic; +} + CompileFlags compute_flags(const BuildPlan& plan) { CompileFlags f; @@ -1568,8 +1621,23 @@ CompileFlags compute_flags(const BuildPlan& plan) { // survives a replacement. Declared here rather than inside either, because // the ordering between them is the whole point. std::string platformAnchor; - if constexpr (mcpp::platform::is_windows) { - if (isMsvcDialect) { + // WHICH BRANCH, asked of the host AND the target (see `link_shape`). The + // format is the one the contract table reads, from the same function and + // the same host fallback; a MinGW toolchain is a PE whatever its triple + // spelling says. + const mcpp::build::dist::Format linkTargetFormat = + isMingwTc ? mcpp::build::dist::Format::Pe + : mcpp::build::dist::format_for(plan.toolchain.targetTriple, + mcpp::platform::needs_explicit_libcxx + ? mcpp::build::dist::Format::MachO + : mcpp::platform::is_windows + ? mcpp::build::dist::Format::Pe + : mcpp::build::dist::Format::Elf); + const LinkShape linkShape = link_shape(current_link_host(), linkTargetFormat, + isMsvcDialect, + !plan.toolchain.crossTargetFlag.empty()); + if (linkShape == LinkShape::MsvcLinkExe || linkShape == LinkShape::WindowsLld) { + if (linkShape == LinkShape::MsvcLinkExe) { // Native cl.exe: link.exe does the link (SeparateLinker). Search // paths for dependency runtime import libs via /LIBPATH; user // ldflags pass through verbatim; GNU link_extra (-flto/-s) does @@ -1634,7 +1702,7 @@ CompileFlags compute_flags(const BuildPlan& plan) { f.ld = std::format("{} -fuse-ld=lld{}{}{}", full_static, link_intent_ld, user_ldflags, link_extra); f.ldC = f.ld; // no C++ runtime token on this line - } else if constexpr (mcpp::platform::needs_explicit_libcxx) { + } else if (linkShape == LinkShape::AppleSdk) { // macOS. The C++ runtime itself is decided by the contract table above // (dist::Format::MachO) and rides unit_ldflags; what is left here is // the rest of the macOS link: @@ -1794,10 +1862,10 @@ CompileFlags compute_flags(const BuildPlan& plan) { // ── The target side comes from the graph, so the HOST's link is wrong ── // - // THE THREE BRANCHES ABOVE ARE SHAPED BY THIS MACHINE, NOT BY THE - // TARGET. `if constexpr (is_windows)` / `needs_explicit_libcxx` / else is a - // question about where mcpp itself was built, and each answer describes a - // link on that machine: an SDK path, a deployment target, a loader search + // THE BRANCHES ABOVE DESCRIBE A LINK AGAINST A PAYLOAD. `link_shape` now + // routes a target outside the host's own family to the generic branch, but + // the Apple and PE lines still describe the host platform's payload, and + // each answer describes a link there: an SDK path, a deployment target, a loader search // path, this host's `libatomic`. Every one of them is right when the target // is the host or is served by a payload, and wrong when the C library, the // C++ runtime and the platform are packages in the dependency graph. diff --git a/tests/e2e/721_an_android_row_links_on_a_macos_host.sh b/tests/e2e/721_an_android_row_links_on_a_macos_host.sh new file mode 100755 index 00000000..77dafb57 --- /dev/null +++ b/tests/e2e/721_an_android_row_links_on_a_macos_host.sh @@ -0,0 +1,102 @@ +#!/usr/bin/env bash +# requires: macos +# 721 -- an Android row links on a macOS host (#647 E3). +# +# The link line used to be chosen by the HOST: on macOS every target received +# the Apple SDK line, which carries `-isysroot ` and +# `-mmacosx-version-min` and no `--target`. The NDK's clang then linked an +# Android object as the host, `-fuse-ld=lld` selected `ld64.lld`, and the link +# failed with +# +# ld64.lld: error: unknown argument '-soname' +# +# Measured on macos-15 (mcpp-plugins run 34968851641). The line is now chosen +# by the host AND the target's object format (`link_shape`, unit +# `LinkShape.*`), so an ELF target takes the line that names its target. +# +# The NDK (`xim:android-ndk`) is installed on first use when this machine does +# not have it; on a runner without network access the leg reports that it +# could not install rather than passing. +# +# Legs: +# A. `mcpp build --target x86_64-linux-android` for a `kind = "app"` target +# over a shared dependency succeeds, and both `libapp.so` and `libdep.so` +# are ELF shared objects for x86-64. +# B. The build log names no `ld64.lld`, no `-mmacosx-version-min` and no +# `MacOSX.sdk` on the Android link. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; tail -60 "$f" 2>/dev/null; done; exit 1; } + +mkdir -p dep/src app/src +cat > dep/mcpp.toml <<'EOF' +[package] +name = "dep" +version = "0.1.0" +standard = "c++20" + +[targets.dep] +kind = "shared" + +[build] +sources = ["src/*.cpp"] +EOF +printf '[[gnu::visibility("default")]] int dep_answer() { return 42; }\n' > dep/src/dep.cpp + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +standard = "c++20" + +[dependencies] +dep = { path = "../dep" } + +[targets.app] +kind = "app" +main = "src/main.cpp" + +[target.x86_64-linux-android] +min_api_level = 24 +EOF +cat > app/src/main.cpp <<'EOF' +int dep_answer(); +[[gnu::visibility("default")]] int app_entry() { return dep_answer() + 1; } +int main() { return app_entry() == 43 ? 0 : 1; } +EOF + +cd app +# ── A ─────────────────────────────────────────────────────────────────────── +if ! "$MCPP" build --target x86_64-linux-android --verbose > a.log 2>&1; then + if grep -qi "cannot be downloaded\|download.*failed\|not installed" a.log \ + && ! grep -q "ld64.lld\|unknown argument '-soname'" a.log; then + fail "A: the NDK could not be installed on this runner" a.log + fi + fail "A: the Android row did not link on a macOS host" a.log +fi +app=$(find target -name libapp.so -path '*/bin/*' | head -1) +dep=$(find target -name libdep.so -path '*/bin/*' | head -1) +[ -n "$app" ] && [ -n "$dep" ] || fail "A: no libapp.so / libdep.so" a.log +for f in "$app" "$dep"; do + kind=$(file -b "$f") + echo "reading A: $(basename "$f"): $kind" + case "$kind" in *ELF*"shared object"*x86-64*) ;; *) fail "A: $(basename "$f") is not an x86-64 ELF shared object: $kind" a.log ;; esac +done +echo "ok: A, the Android row links on a macOS host" + +# ── B ─────────────────────────────────────────────────────────────────────── +if grep -q "ld64.lld" a.log; then fail "B: ld64.lld appears in the build" a.log; fi +graph=$(ls target/x86_64-linux-android/*/build.ninja 2>/dev/null | head -1 || true) +[ -n "$graph" ] || fail "B: no build graph for the Android row" a.log +# The link line only: `ldflags` is what the host branch assembled. +if grep -E "^[[:space:]]*ldflags[[:space:]]*=" "$graph" | grep -q "mmacosx-version-min\|MacOSX.sdk"; then + grep -nE "^[[:space:]]*ldflags[[:space:]]*=" "$graph" | head -3 + fail "B: the Android link line carries the macOS SDK or deployment floor" a.log +fi +echo "ok: B, the Android link carries no Apple SDK token" + +echo "PASS: 721" diff --git a/tests/unit/test_build_flags.cpp b/tests/unit/test_build_flags.cpp index b9d1f644..d96e2fce 100644 --- a/tests/unit/test_build_flags.cpp +++ b/tests/unit/test_build_flags.cpp @@ -2,6 +2,7 @@ import std; import mcpp.build.flags; +import mcpp.build.distribution; import mcpp.modgraph.scanner; namespace { @@ -165,3 +166,45 @@ TEST(BuildFlagsLoaderTokens, OrdinaryPathsAreNotTokens) { EXPECT_FALSE(mcpp::build::is_loader_relative_search_path(entry)) << entry; } } + +// #647 E3: which link line a (host, target) pair takes. The branch used to be +// chosen by `if constexpr` on the host, so an Android row built on a macOS host +// received the Apple SDK line with no `--target` and linked through +// `ld64.lld`. The choice is a function of the host AND the target's object +// format, stated here for every host on every CI machine. +namespace { +using mcpp::build::LinkHost; +using mcpp::build::LinkShape; +using mcpp::build::link_shape; +using Fmt = mcpp::build::dist::Format; +} // namespace + +TEST(LinkShape, LinuxHostTakesTheGenericLineForEveryTarget) { + for (auto f : {Fmt::Elf, Fmt::MachO, Fmt::Pe, Fmt::Wasm}) { + EXPECT_EQ(link_shape(LinkHost::Linux, f, false, true), LinkShape::Generic); + EXPECT_EQ(link_shape(LinkHost::Linux, f, false, false), LinkShape::Generic); + } +} + +TEST(LinkShape, MacOSHostKeepsTheAppleLineForMachOOnly) { + // macOS for macOS, and the iOS rows: the Apple SDK line, as before. + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::MachO, false, false), LinkShape::AppleSdk); + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::MachO, false, true), LinkShape::AppleSdk); + // x86_64-linux-android and wasm32-emscripten on a Mac: the line that + // carries `--target`. + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::Elf, false, true), LinkShape::Generic); + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::Wasm, false, true), LinkShape::Generic); + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::Elf, false, false), LinkShape::Generic); + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::Pe, false, true), LinkShape::Generic); +} + +TEST(LinkShape, WindowsHostSeparatesPeFromTargetsNamedByFlag) { + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Pe, true, false), LinkShape::MsvcLinkExe); + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Pe, false, true), LinkShape::WindowsLld); + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Pe, false, false), LinkShape::WindowsLld); + // An SDK or retargetable clang aimed at an ELF row: named by `--target`. + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Elf, false, true), LinkShape::Generic); + // The canadian GCC cross to x86_64-linux-musl names its target by prefix + // and keeps the line its CI job verifies. + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Elf, false, false), LinkShape::WindowsLld); +} From bf86a6d7efeb1566c01084dbbcea5c4d77ae6200 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:29:55 +0800 Subject: [PATCH 13/20] Keep [package.metadata] verbatim and report unknown [package] keys (#647 E1, #649 X4) The metadata table is kept as JSON text for the graph document and is not interpreted. [package] reports a key the parser does not read the way [build] does: a warning for the root manifest, an error under --strict. --- modules/manifest/src/toml.cppm | 96 +++++++++++++++++++++++++++++++++ modules/manifest/src/types.cppm | 12 +++++ tests/unit/test_manifest.cpp | 71 ++++++++++++++++++++++++ 3 files changed, 179 insertions(+) diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 4d310e3b..fa22a2f5 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -88,6 +88,67 @@ std::string version_req_problem(std::string_view spec) { return {}; } + +// A TOML table as JSON text, for `[package.metadata]` (#647 E1). The TOML +// reader here produces strings, integers, booleans, arrays and tables, and each +// has one JSON spelling; nothing is interpreted. +void append_json_string(std::string& out, std::string_view s) { + out += '"'; + for (unsigned char c : s) { + switch (c) { + case '"': out += "\\\""; break; + case '\\': out += "\\\\"; break; + case '\n': out += "\\n"; break; + case '\r': out += "\\r"; break; + case '\t': out += "\\t"; break; + default: + if (c < 0x20) out += std::format("\\u{:04x}", static_cast(c)); + else out += static_cast(c); + } + } + out += '"'; +} + +void append_json_value(std::string& out, const mcpp::libs::toml::Value& v) { + using Kind = mcpp::libs::toml::Value::Kind; + switch (v.kind()) { + case Kind::String: append_json_string(out, v.as_string()); break; + case Kind::Int: out += std::to_string(v.as_int()); break; + case Kind::Bool: out += v.as_bool() ? "true" : "false"; break; + case Kind::Array: { + out += '['; + bool first = true; + for (auto const& e : v.as_array()) { + if (!first) out += ','; + first = false; + append_json_value(out, e); + } + out += ']'; + break; + } + case Kind::Table: { + out += '{'; + bool first = true; + for (auto const& [k, e] : v.as_table()) { + if (!first) out += ','; + first = false; + append_json_string(out, k); + out += ':'; + append_json_value(out, e); + } + out += '}'; + break; + } + case Kind::Null: out += "null"; break; + } +} + +std::string toml_table_to_json(const mcpp::libs::toml::Table& table) { + std::string out; + append_json_value(out, mcpp::libs::toml::Value(table)); + return out; +} + } // namespace @@ -1030,6 +1091,41 @@ std::expected parse_string(std::string_view content, if (auto v = doc->get_string_array("build.std-module-flags")) m.buildConfig.stdModuleFlags = *v; + // [package.metadata] (#647 E1): kept verbatim for the graph document. + if (auto* meta = doc->get_table("package.metadata")) + m.packageMetadataJson = toml_table_to_json(*meta); + + // Surface unsupported [package] keys the way [build] does (#649 X4): a + // warning for the root manifest, an error under --strict, and nothing for + // a dependency's manifest, whose schema warnings are not surfaced (a + // package may adopt a key before its consumers upgrade). `[package]` was + // the one central section without the check, so a misspelt `licence` or + // `descripton` was accepted in silence. + // + // MUST stay in sync with the `doc->get_*("package.")` reads above. + static constexpr std::string_view kKnownPackageKeys[] = { + "accelerators", "authors", "description", "exclusive", "license", + "metadata", "name", "namespace", "platforms", "provides", "repo", + "requires", "requires_abi", "standard", "std-compat-module", + "std-module", "std-module-flags", "version", + }; + if (auto* pt = doc->get_table("package")) { + for (auto& [key, _] : *pt) { + bool known = false; + for (auto k : kKnownPackageKeys) if (key == k) { known = true; break; } + if (!known) { + std::string supported; + for (auto k : kKnownPackageKeys) { + if (!supported.empty()) supported += ", "; + supported += k; + } + m.schemaWarnings.push_back(std::format( + "[package] has unsupported key '{}' (ignored). Supported keys: {}.", + key, supported)); + } + } + } + // [capabilities] cap = "provider" — root-only provider pins. if (auto* caps = doc->get_table("capabilities"); caps && !caps->empty()) { for (auto& [cap, cval] : *caps) diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index b5959772..433098b5 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1983,6 +1983,18 @@ struct Manifest { // under [targets.]). The caller (prepare_build) prints these and, under // --strict, escalates them to errors — mirroring the feature/platform path. std::vector schemaWarnings; + + // `[package.metadata]`, as a JSON object text; empty when the manifest has + // none (#647 E1). + // + // THE ENGINE DOES NOT INTERPRET IT. The table belongs to whoever reads it + // (`[package.metadata.]`), and it travels to the root build program + // verbatim in the graph document (`mcpp::graph_file()`). A path in it is + // resolved by that reader against the package's manifest directory, which + // the document states beside it, because only the reader knows which + // values are paths. Kept as text so this module's interface names no JSON + // type. + std::string packageMetadataJson; }; struct ManifestError { diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 144ed77d..14dcbef3 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -5686,3 +5686,74 @@ kind = "shared" EXPECT_EQ(m.targets[0].kind, mcpp::manifest::Target::Library); EXPECT_EQ(m.targets[0].linkageDefault, "shared"); } + +// ── #647 E1 / #649 X4: `[package.metadata]` and the `[package]` key check ──── +// +// The metadata table is kept verbatim for the graph document, as JSON text, and +// `metadata` is a known key. Any other key the parser does not read is reported +// the way `[build]` reports one, so `licence` is no longer accepted in silence. +TEST(Manifest, PackageMetadataIsKeptVerbatimAsJson) { + constexpr auto src = R"( +[package] +name = "x" +version = "0.1.0" +[package.metadata.demo] +resources = "res" +langs = ["en", "zh"] +count = 3 +enabled = true +note = "a \"quoted\" value" +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + EXPECT_EQ(m->packageMetadataJson, + R"({"demo":{"count":3,"enabled":true,"langs":["en","zh"],"note":"a \"quoted\" value","resources":"res"}})"); + EXPECT_TRUE(m->schemaWarnings.empty()) + << (m->schemaWarnings.empty() ? "" : m->schemaWarnings[0]); +} + +TEST(Manifest, PackageWithoutMetadataHasNone) { + auto m = mcpp::manifest::parse_string("[package]\nname = \"x\"\nversion = \"0.1.0\"\n"); + ASSERT_TRUE(m.has_value()) << m.error().format(); + EXPECT_TRUE(m->packageMetadataJson.empty()); +} + +TEST(Manifest, UnknownPackageKeyIsReported) { + constexpr auto src = R"( +[package] +name = "x" +version = "0.1.0" +licence = "MIT" +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + ASSERT_EQ(m->schemaWarnings.size(), 1u); + EXPECT_NE(m->schemaWarnings[0].find("[package] has unsupported key 'licence'"), + std::string::npos) << m->schemaWarnings[0]; +} + +TEST(Manifest, EveryReadPackageKeyIsKnown) { + constexpr auto src = R"( +[package] +name = "x" +namespace = "ns" +version = "0.1.0" +standard = "c++23" +description = "d" +license = "MIT" +repo = "https://example.org" +authors = ["a"] +platforms = ["linux"] +accelerators = ["cuda"] +provides = ["cap"] +requires = ["other"] +exclusive = ["cap"] +std-module = "m" +std-compat-module = "c" +std-module-flags = ["-x"] +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + for (auto const& w : m->schemaWarnings) + EXPECT_EQ(w.find("[package] has unsupported key"), std::string::npos) << w; +} From cd4e0c2d4fd381baf43daaeffb466bfc47d647b3 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:30:02 +0800 Subject: [PATCH 14/20] The resolved graph for the root build program, pack strips what the graph built, and a machine-readable pack report (#647 E1, #649 E5, E9) - mcpp::graph_file(): the root package's program reads every package in dependency order with its manifest directory, features, targets and [package.metadata]; the document's content joins the re-run key. The same entry builder produces resolution.json's graph section. - mcpp pack strips the program on every row that strips (the Android row did not reach the strip step), every shared library the graph built and the staged copy of the toolchain's runtime; the Packing line states what the row does; mcpp::pack_strip() and mcpp::pack_debug_symbols_dir() carry the decision to a member that stages libraries of its own. - mcpp pack --message-format json prints one mcpp.pack envelope; pack takes --release and --dev; build, run, test, emit and pack resolve the profile with one rule (--profile wins over the shorthands). e2e 720, 722, 723; unit BuildProfile.*. --- docs/04-mcpp-toml.md | 14 ++ docs/10-pack-and-release.md | 35 ++- docs/30-build-mcpp.md | 55 +++++ docs/50-machine-output.md | 29 +++ docs/zh/04-mcpp-toml.md | 11 + docs/zh/10-pack-and-release.md | 28 ++- docs/zh/30-build-mcpp.md | 45 ++++ docs/zh/50-machine-output.md | 24 ++ src/build/build_program.cppm | 22 ++ src/build/hostprogram.cppm | 31 +++ src/build/prepare.cppm | 190 +++++++++++++--- src/cli.cppm | 13 ++ src/cli/cmd_build.cppm | 19 +- src/cli/cmd_publish.cppm | 117 +++++++++- src/pack/library_pipeline.cppm | 16 +- src/pack/pack.cppm | 192 +++++++++++++++- src/pack/pipeline.cppm | 70 +++++- src/wire.cppm | 8 +- ..._build_program_reads_the_resolved_graph.sh | 176 +++++++++++++++ .../722_pack_strips_what_the_graph_built.sh | 210 ++++++++++++++++++ .../e2e/723_pack_reports_what_it_produced.sh | 118 ++++++++++ tests/unit/test_build_profile.cpp | 15 ++ 22 files changed, 1376 insertions(+), 62 deletions(-) create mode 100755 tests/e2e/720_the_root_build_program_reads_the_resolved_graph.sh create mode 100755 tests/e2e/722_pack_strips_what_the_graph_built.sh create mode 100755 tests/e2e/723_pack_reports_what_it_produced.sh diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 162e403e..26211219 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -89,6 +89,20 @@ When a **dependency declares a level above the graph's**, mcpp says so before co rather than letting it fail somewhere inside that dependency's sources. See [workspace §4.2](07-workspace.md). +`[package.metadata.]` (mcpp 2026.9.16.1+) is a table the engine keeps and +does not interpret. It is the package's statement about itself for a tool that +reads it, such as a framework collecting what each library contributes, and it +reaches the root package's build program through `mcpp::graph_file()` +([30 — build.mcpp](30-build-mcpp.md)). A path in it is resolved by that reader +against the package's manifest directory. Any other key in `[package]` that +mcpp does not read is reported, as in `[build]`: a warning, and an error under +`--strict` (2026.9.16.1+). + +```toml +[package.metadata.demo] +resources = "res" +``` + #### Dialect flags and the `import std` BMI Some flags change what the standard library's headers declare, so the precompiled `import std` diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index bfe8cb88..3dc570a3 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -144,6 +144,8 @@ mcpp pack --format appimage # a format a package in the graph provid mcpp pack -o myapp.tar.gz # filename only: lands at target/dist/myapp.tar.gz mcpp pack -o /abs/path/myapp.tar.gz # includes a directory: output to the literal path mcpp pack --profile dev # build with a different profile (default: release) +mcpp pack --dev # the same, as `build` and `run` spell it; --profile wins over it +mcpp pack --message-format json # one mcpp.pack envelope on stdout (mcpp 2026.9.16.1+) mcpp pack --no-strip # ship the artifacts as built mcpp pack --debug-symbols dbg/ # write the separated *.debug files under dbg/ mcpp pack --format msi --features installer # activate root-package features for the pack @@ -156,6 +158,19 @@ one distribution is declared under `[feature-deps.]` with `tools = [...]` and only by the pack that names ``. `mcpp run --format --features ` hands the same features to the pack it performs. +`--release` and `--dev` (mcpp 2026.9.16.1+) are the shorthands `build` and `run` +take, with the same precedence: `--profile` wins over either, on all three +commands. + +`--message-format json` (mcpp 2026.9.16.1+) prints one `mcpp.pack` envelope on +stdout after the command finishes and sends every human line to stderr. Its +`data.artifacts` holds each produced file or directory with its absolute path, +its `type` (`file` or `directory`), the `--format` value and the triples of its +legs; `data.stage` holds the staged tree, its manifest and whether the closure +was walked ([50 — Machine Output](50-machine-output.md)). `--format` names the +package format on this command, so machine output is asked for the way +`mcpp test` asks for it. + ### `--format` owns one axis, and the engine owns two of its values `tar` and `dir` answer the same question `msi` and `appimage` answer — what @@ -297,9 +312,23 @@ follow. > anything; this one governs what the **package** carries. Two different > decisions, two different names. -**Bundled libraries are never stripped.** They came out of the store or off the -host, mcpp did not build them, and rewriting somebody else's shared payload for -this bundle's benefit is not the packer's business. +**What the graph built is stripped on every row that carries debug information in +the image; a library from the store or the host is not** (mcpp 2026.9.16.1+). The +rule is dh_strip's, a package strips what it built, and it covers: + +| staged file | stripped | reason | +|---|---|---| +| the program | yes, as an executable; as a shared library on the Android row, where the program is one | this build compiled it | +| a shared library a `SharedLibrary` link unit of the graph produced | yes, `--strip-unneeded` | this build compiled it from source | +| the staged copy of the toolchain's own runtime (`libc++_shared.so` from the NDK) | yes, `--strip-unneeded` | a copy in the tree is not the shared payload; the Android Gradle plugin strips the same file | +| a library from the store, the host, or a prebuilt a package deployed | no | its bytes are its publisher's, and a vendor library may be signed | + +Mach-O and MSVC PE keep their debug information beside the image, so nothing is +stripped there, and the `Packing` line says "stripped" only on a row that strips. +`--no-strip` and `--debug-symbols` reach every stripped file. A build program +that stages libraries of its own reads the same decision through +`mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()` +([30 — build.mcpp](30-build-mcpp.md)) ## Output Layout diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 798a51fd..eaa300ab 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -751,6 +751,14 @@ that follow the header state each needed name and what satisfies it ([50 — Machine Output](50-machine-output.md)), so a provider that places libraries itself reads them rather than inferring the closure from `lib/`. +**A member that stages libraries of its own follows the strip decision +(2026.9.16.1+).** `mcpp pack` strips the program, every shared library the graph +built and the staged copy of the toolchain's runtime, and `--no-strip` and +`--debug-symbols` govern all of them. A provider that places further libraries +(an Android archive's native libraries) reads `mcpp::pack_strip()` (`"1"` or +`"0"`) and `mcpp::pack_debug_symbols_dir()` in the pass that submits, so one +switch governs every file in the package. + Commands are an **argv, not a shell string** (no shell is assumed — Windows has none to rely on), and the only interpolations are a closed set: @@ -766,6 +774,50 @@ none to rely on), and the only interpolations are a closed set: The raw stdout protocol above remains the low-level substrate; `import mcpp;` is the typed layer over it. +### Reading the resolved graph: `graph_file` (2026.9.16.1+) + +`mcpp::dep_dir` answers by name for a dependency the package declares itself. +A framework that merges what every library contributes (resources, platform +sources, an Info.plist fragment) needs the libraries its application does not +name as well, in an order in which a later contribution overrides an earlier +one. `mcpp::graph_file()` names a JSON document that states the resolved graph: + +```jsonc +{ + "kind": "mcpp.graph", + "version": 1, + "packages": [ // dependencies before their requesters + { + "package": { "canonical": "spike.b@0.2.0", "namespace": "spike", + "name": "b", "version": "0.2.0", "source": "path" }, + "root": false, + "requested_by": [ { "requester": "spike.a@0.1.0", "key": "spike.b", + "table": "dependencies" } ], + "link": { "form": "static", "reason": "..." }, // a library only + "manifest_dir": "/abs/path/to/b", + "features": [], + "targets": [ { "name": "b", "kind": "lib" } ], + "metadata": { "demo": { "resources": "res" } } // [package.metadata], verbatim + } + ] +} +``` + +- **The root package's program receives it; a dependency's program reads + `""`.** The root decides the graph, and when its program runs every input of + that decision is final, which is the reason `dep_linkage` is offered to it + alone. +- **`[package.metadata.]` is the package's statement about itself.** The + engine does not interpret the table. A path in it is resolved by the reader + against the entry's `manifest_dir`, because only the reader knows which values + are paths. Older engines ignore the table, so a published package may state + it before its consumers upgrade. +- **The document's content is part of the re-run key.** Editing a dependency's + `[package.metadata]` re-runs the root program; editing that dependency's + sources does not. +- The entries are the `graph` section of `resolution.json` with four additions + (`manifest_dir`, `features`, `targets`, `metadata`), from one derivation. + ### `import mcpp;` is the surface that evolves (mcpp 2026.8.5.1+) Two ways to talk to mcpp, and they carry **different compatibility promises**: @@ -879,6 +931,8 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_TARGET_MIN_PLATFORM_VERSION` *(2026.9.12.3+)* | `mcpp::min_platform_version()` | The project's floor for this triple, in the platform's own words: on macOS, `[build] macos_deployment_target` or the engine's own default `14.0`; on iOS, `[build] ios_deployment_target` verbatim, empty when the project states none; on `*-linux-android`, `[target.] min_api_level`, or a fallback the resolved NDK payload states; empty on every other row. The value the effective triple carries, and part of the re-run key | | `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | The `--format` value of the `mcpp pack` pass this program is part of; empty for every ordinary build. The empty value is the one that carries the meaning — a member gates its submission on this, so `mcpp build` has the graph it always had | | `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | Where `mcpp pack` has already staged the closure, absolute; empty when this build is not packaging. Read it to decide the shape of the work; write `${mcpp.stage_dir}` into the action, so the path in the graph and the path the program read cannot disagree | +| `MCPP_PACK_STRIP` *(2026.9.16.1+)* | `mcpp::pack_strip()` | `1` when the `mcpp pack` pass this program is part of strips, `0` under `--no-strip`; empty for every ordinary build. A member that stages libraries of its own follows it, so one switch governs every file in the package | +| `MCPP_PACK_DEBUG_SYMBOLS_DIR` *(2026.9.16.1+)* | `mcpp::pack_debug_symbols_dir()` | Where `--debug-symbols` sends the separated `*.debug` files, absolute; empty when they are discarded or the build is not packing | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | the device-kind sources (`.cu`, `.hip`, …) the package's effective `sources` match, package-root-relative, one per line; empty when there are none. The engine compiles none of them — the rule package this program imports turns each into an `mcpp::action`. Already narrowed: a `{ glob, accel }` entry the build does not cover contributes nothing, so `--no-accel` yields an empty list | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | a writable scratch/output dir owned by mcpp | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | the package root (= CWD) | @@ -886,6 +940,7 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_FEATURES` | — | comma-separated active feature list | | `MCPP_DEP__DIR` | `mcpp::dep_dir("name")` | the resolved install dir of each declared dependency (the qualified `namespace.name`, the canonical name, **and** the namespace-stripped spelling when it is unambiguous; same `` sanitization as `MCPP_FEATURE_`). Received by dependencies' build.mcpp **and** the root project's (the root runs after dependency resolution, 0.0.100+) | | `MCPP_DEP__LINKAGE` *(2026.9.15.2+)* | `mcpp::dep_linkage("name")` | the link form each dependency takes in this build, `static` or `shared`, under the names `MCPP_DEP__DIR` uses; empty for a dependency with no library form. The value is the resolution that decides what is linked, so a generated loader entry or `dllimport` declaration follows it. Received by the **root project's** build.mcpp only: the root decides every dependency's form, and a dependency's program runs before packages discovered after it, whose programs supply facts the answer depends on, so `dep_linkage` is always empty there | +| `MCPP_GRAPH_FILE` *(2026.9.16.1+)* | `mcpp::graph_file()` | the resolved dependency graph as a JSON document, packages in dependency order, each with its manifest directory, features, targets and `[package.metadata]` (see [Reading the resolved graph](#reading-the-resolved-graph-graph_file-20269161)). Received by the **root project's** build.mcpp only; the document's content joins the re-run key | These values are folded into the re-run key **unconditionally** — changing the target, profile, or feature set re-runs the program without any diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 1f82c647..25582fbf 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -95,6 +95,13 @@ mcpp --format json streaming case and is **not** accepted — asking for it is an error, not a silent fallback. +A command whose `--format` already names its **product** asks for machine +output with `--message-format json` instead, as `mcpp test` does. `mcpp pack +--format` names the package format (`tar`, `dir`, `msi`), so its report is +`mcpp pack --message-format json` (2026.9.16.1+). The shape follows the kind: +`mcpp test` streams a record per test because tests finish over time, and +`mcpp pack` prints one envelope because a pack has one result. + ### Unsupported values and unknown options Both go to **stderr** with **exit code 2**, and write nothing to stdout: @@ -444,6 +451,28 @@ document in place: `write-global-cache` and `exec-build-script` for the command, and never `write-project`. +### `mcpp.pack` — the products of a pack *(mcpp 2026.9.16.1+)* + +``` +mcpp pack [target] [--format ] [--target ...] --message-format json +``` + +The envelope is printed once, after the command finishes; every human line goes +to stderr, including what the build programs and tools the pack starts print. +`data` is: + +| field | | +|---|---| +| `artifacts` | one record per produced artifact: `path` (absolute), `type` (`file` or `directory`), `format` (the `--format` value, `tar` when omitted) and `targets` (the canonical triple of each leg that went into it). A dispatched format reports the terminal outputs of the actions the request introduced; a several-`--target` Android pack reports one artifact whose `targets` lists every leg | +| `stage` | the tree the artifact was made from: `dir`, `manifest` (the stage manifest below) and `closure` (`walked` or `not-walked`); `null` for a library package and when no tree was staged | + +A failure omits `data`, exits with the command's exit status and carries the +diagnostic code `MCPP_PACK_FAILED`; the reason is on stderr. The per-run +`effects` are `read-project`, `write-project` and `write-global-cache`, with +`exec-build-script` when a build program ran. `--protocol-version` declares +`init-mcpp-home`, `read-project`, `write-project`, `network`, +`write-global-cache` and `exec-build-script` for `pack`. + ### `mcpp test --message-format json` — the test stream ``` diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index b73f52cb..5bbbb4e9 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -84,6 +84,17 @@ mcpp 会在编译前失败，并同时报出工具链与工程档位。 当**依赖声明的档位高于当前图**时,mcpp 会在编译前说出来,而不是让它在那个依赖的源码里 某处失败。见 [workspace §4.2](07-workspace.md)。 +`[package.metadata.]`(mcpp 2026.9.16.1+)是引擎保留但不解释的表。它是包对自身的 +陈述,供读取它的工具使用,例如收集每个库贡献内容的框架;它通过 `mcpp::graph_file()` 到达根包 +的构建程序([30 —— build.mcpp](30-build-mcpp.md))。其中的路径由读取方相对于该包的清单目录 +解析。`[package]` 中 mcpp 不读取的其他键会被报告,与 `[build]` 一致:给出警告,在 `--strict` +下报错(2026.9.16.1+)。 + +```toml +[package.metadata.demo] +resources = "res" +``` + #### 方言标志与 `import std` BMI 有些标志会改变标准库头文件**声明出什么**,因此预编译的 `import std` BMI 也必须带着它们一起 diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 2ad3918d..b7e22d8d 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -110,6 +110,8 @@ mcpp pack --format appimage # 由图里某个包提供的格式 mcpp pack -o myapp.tar.gz # 仅文件名:落到 target/dist/myapp.tar.gz mcpp pack -o /abs/path/myapp.tar.gz # 含目录:按字面路径输出 mcpp pack --profile dev # 换一个 profile 构建(默认 release) +mcpp pack --dev # 同上,拼法与 build、run 一致;--profile 优先于它 +mcpp pack --message-format json # 在 stdout 上输出一个 mcpp.pack 信封(mcpp 2026.9.16.1+) mcpp pack --no-strip # 按构建原样发货,不剥符号 mcpp pack --debug-symbols dbg/ # 把分离出的 *.debug 写到 dbg/ mcpp pack --format msi --features installer # 为这次打包启用根包 feature @@ -121,6 +123,16 @@ mcpp pack --format msi --features installer # 为这次打包启用根包 feat `[feature-deps.]` 下,只由点名 `` 的那次打包构建。 `mcpp run --format --features ` 把同样的 feature 交给它执行的那次打包。 +`--release` 与 `--dev`(mcpp 2026.9.16.1+)是 `build`、`run` 所接受的简写,优先级相同: +三条命令上都是 `--profile` 优先于它们。 + +`--message-format json`(mcpp 2026.9.16.1+)在命令结束后于 stdout 上输出一个 +`mcpp.pack` 信封,所有给人读的行都走 stderr。其 `data.artifacts` 列出产出的每个文件或 +目录:绝对路径、`type`(`file` 或 `directory`)、`--format` 取值以及各条腿的三元组; +`data.stage` 给出暂存树、它的 manifest 以及闭包是否走通 +([50 —— 机器输出](50-machine-output.md))。这条命令上的 `--format` 表示包格式,因此 +机器输出按 `mcpp test` 的方式请求。 + ### `--format` 是一个轴,引擎只拥有其中两个取值 `tar` 与 `dir` 回答的问题,和 `msi` 与 `appimage` 回答的问题是同一个 —— 输出取什么 @@ -239,8 +251,20 @@ DWARF 带着发布者源码树与构建目录的绝对路径。剥什么取决 > 既碰不到静态归档、也无法分离出任何东西;前者管的是**包里带什么**。 > 两个不同的决定,两个不同的名字。 -**被捆绑进来的库永远不 strip。** 它们来自 store 或宿主,不是 mcpp 构建的, -为了这一个 bundle 去改写别人的共享载荷不是打包器该做的事。 +**图构建出的东西在每条把调试信息放在映像内的行上都会被剥离;来自 store 或宿主的库不会** +(mcpp 2026.9.16.1+)。规则是 dh_strip 的「包剥离自己构建的东西」,覆盖范围: + +| 暂存的文件 | 是否剥离 | 原因 | +|---|---|---| +| 程序 | 是,按可执行文件;在 Android 行上按共享库,因为那里的程序就是共享库 | 本次构建编译了它 | +| 图中 `SharedLibrary` 链接单元产出的共享库 | 是,`--strip-unneeded` | 本次构建从源码编译了它 | +| 工具链自身运行时的暂存副本(NDK 的 `libc++_shared.so`) | 是,`--strip-unneeded` | 树里的副本不是共享载荷;Android Gradle 插件剥离同一个文件 | +| 来自 store、宿主的库,或包部署的预构建库 | 否 | 其字节属于其发布者,供应商的库可能带签名 | + +Mach-O 与 MSVC PE 的调试信息放在映像之外,因此那里什么都不剥离,`Packing` 行只在确实剥离 +的行上写「stripped」。`--no-strip` 与 `--debug-symbols` 管辖每一个被剥离的文件。自己暂存库的 +构建程序通过 `mcpp::pack_strip()` 与 `mcpp::pack_debug_symbols_dir()` 读到同一个决定 +([30 —— build.mcpp](30-build-mcpp.md)) ## 产物布局 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index ade2ed51..1215c164 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -640,6 +640,11 @@ mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件,永不是 被需要的名字以及满足它的东西([50 —— 机器输出](50-machine-output.md)),自行放置库的 提供方读这些行,而不是从 `lib/` 里推断闭包。 +**自己暂存库的成员跟随剥离决定(2026.9.16.1+)。** `mcpp pack` 剥离程序、图构建出的每个 +共享库以及工具链运行时的暂存副本,`--no-strip` 与 `--debug-symbols` 管辖其中每一个。 +另行放置库的提供方(Android 归档里的原生库)在提交的那次 pass 中读 `mcpp::pack_strip()` +(`"1"` 或 `"0"`)与 `mcpp::pack_debug_symbols_dir()`,于是一个开关管住包里的每个文件。 + 命令是 **argv 而不是 shell 字符串**(不假设存在 shell —— Windows 没有能依赖的那个), 插值只有封闭的一组: @@ -654,6 +659,43 @@ mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件,永不是 上面的裸 stdout 协议仍是底层基底;`import mcpp;` 是其上的类型化层。 +### 读取解析后的依赖图:`graph_file`(2026.9.16.1+) + +`mcpp::dep_dir` 按名字回答本包自己声明的依赖。一个要合并每个库贡献的东西(资源、平台源码、 +一段 Info.plist)的框架,还需要应用没有点名的那些库,而且要按「后来的贡献覆盖先前的」的 +顺序。`mcpp::graph_file()` 给出一份描述解析后依赖图的 JSON 文档: + +```jsonc +{ + "kind": "mcpp.graph", + "version": 1, + "packages": [ // 依赖排在请求它的包之前 + { + "package": { "canonical": "spike.b@0.2.0", "namespace": "spike", + "name": "b", "version": "0.2.0", "source": "path" }, + "root": false, + "requested_by": [ { "requester": "spike.a@0.1.0", "key": "spike.b", + "table": "dependencies" } ], + "link": { "form": "static", "reason": "..." }, // 仅库有 + "manifest_dir": "/abs/path/to/b", + "features": [], + "targets": [ { "name": "b", "kind": "lib" } ], + "metadata": { "demo": { "resources": "res" } } // [package.metadata],原样 + } + ] +} +``` + +- **根包的程序拿到它;依赖包的程序读到 `""`。** 根包决定这张图,而它的程序运行时这个决定的 + 每个输入都已确定 —— 这正是 `dep_linkage` 只提供给根包的原因。 +- **`[package.metadata.]` 是包对自身的陈述。** 引擎不解释这张表。其中的路径由读取方 + 相对于该条目的 `manifest_dir` 解析,因为只有读取方知道哪些值是路径。旧引擎忽略这张表, + 所以已发布的包可以在其使用方升级之前就写上它。 +- **文档内容属于重跑键。** 修改某个依赖的 `[package.metadata]` 会让根包的程序重跑;修改该 + 依赖的源码不会。 +- 这些条目就是 `resolution.json` 的 `graph` 一节再加四项(`manifest_dir`、`features`、 + `targets`、`metadata`),出自同一次推导。 + ### `import mcpp;` 才是会演进的那一面(mcpp 2026.8.5.1+) 和 mcpp 对话有两条路,它们的**兼容性承诺不同**: @@ -755,6 +797,8 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 | `MCPP_TARGET_MIN_PLATFORM_VERSION` *(2026.9.12.3+)* | `mcpp::min_platform_version()` | 项目对这个三元组的下限,用平台自己的措辞:macOS 上是 `[build] macos_deployment_target` 或引擎自带的默认值 `14.0`;iOS 上是 `[build] ios_deployment_target` 原样给出,项目没写就是空;`*-linux-android` 上是 `[target.] min_api_level`,或已解析 NDK 载荷给出的回落值;其余每一行都是空。取的是有效三元组携带的那个值,并进入重跑键 | | `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | 本程序所处的这次 `mcpp pack` 的 `--format` 取值;任何普通构建下都为空。承载含义的正是这个空值 —— 成员据此为自己的提交加闸,于是 `mcpp build` 拿到的还是它一直以来的那张图 | | `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | `mcpp pack` 已经把闭包暂存到的位置,绝对路径;本次构建不在打包时为空。读它来判断这次要干的活是什么形状,而把 `${mcpp.stage_dir}` 写进 action —— 这样图里的路径与程序读到的路径不可能不一致 | +| `MCPP_PACK_STRIP` *(2026.9.16.1+)* | `mcpp::pack_strip()` | 本程序所处的这次 `mcpp pack` 剥离时为 `1`,`--no-strip` 下为 `0`;任何普通构建下都为空。自己暂存库的成员跟随它,于是一个开关管住包里的每个文件 | +| `MCPP_PACK_DEBUG_SYMBOLS_DIR` *(2026.9.16.1+)* | `mcpp::pack_debug_symbols_dir()` | `--debug-symbols` 放置分离出的 `*.debug` 文件的位置,绝对路径;丢弃调试信息或本次构建不在打包时为空 | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | 本包有效 `sources` 匹配到的设备类源文件(`.cu`、`.hip`…),相对包根,一行一个;没有时为空串。引擎一个都不编译 —— 由本程序引入的规则包把每一个变成一条 `mcpp::action`。已经过收窄:构建未覆盖的 `{ glob, accel }` 条目贡献为空,因此 `--no-accel` 得到空列表 | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | mcpp 提供的可写输出/暂存目录 | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | 包根(= CWD) | @@ -762,6 +806,7 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 | `MCPP_FEATURES` | — | 活跃 feature 逗号列表 | | `MCPP_DEP__DIR` | `mcpp::dep_dir("name")` | 每个已声明依赖解析后的安装目录(限定名 `namespace.name`、canonical 名,以及无歧义时的去命名空间短名,都可用;`` 消毒规则同 `MCPP_FEATURE_`)。依赖包的 build.mcpp **和**根工程的 build.mcpp 都能拿到(根工程的 build.mcpp 在依赖解析之后运行,0.0.100+) | | `MCPP_DEP__LINKAGE` *(2026.9.15.2+)* | `mcpp::dep_linkage("name")` | 每个依赖在本次构建中的链接形态,`static` 或 `shared`,名字与 `MCPP_DEP__DIR` 相同;没有库形态的依赖为空。该值就是决定链接内容的那次解析,生成的加载入口或 `dllimport` 声明因此与之一致。只有**根工程**的 build.mcpp 能拿到:根工程决定每个依赖的形态,而依赖包的程序运行在发现顺序更靠后的包之前,那些包的程序提供了答案所依赖的事实,所以在那里 `dep_linkage` 总是为空 | +| `MCPP_GRAPH_FILE` *(2026.9.16.1+)* | `mcpp::graph_file()` | 以 JSON 文档表示的解析后依赖图,包按依赖顺序排列,每个包带其清单目录、feature、target 与 `[package.metadata]`(见[读取解析后的依赖图](#读取解析后的依赖图graph_file20269161))。只有**根工程**的 build.mcpp 能拿到;文档内容进入重跑键 | 这些契约值**无条件**折入重跑键——换 target、换 profile、开关 feature 都会触发重跑, 不需要任何 `rerun-if-env-changed` 声明。 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 246a10c4..d4a04a10 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -77,6 +77,11 @@ mcpp <命令> --format json 目前只支持 `json`。`ndjson` 保留给未来真正需要流式的场景,**现在不接受** —— 请求它是 错误,不是静默回落。 +`--format` 已经表示其**产物**的命令改用 `--message-format json` 请求机器输出,与 +`mcpp test` 一致。`mcpp pack --format` 表示包格式(`tar`、`dir`、`msi`),因此它的报告是 +`mcpp pack --message-format json`(2026.9.16.1+)。形状随 kind 而定:测试随时间陆续完成, +所以 `mcpp test` 每个测试一条流式记录;一次打包只有一个结果,所以 `mcpp pack` 输出一个信封。 + ### 不支持的值 / 未知选项 两者都走 **stderr**、退出码 **2**,且**不往 stdout 写任何东西**: @@ -390,6 +395,25 @@ mcpp emit build-database [--spec s1|compile-commands] --format json `--protocol-version` 为这条命令声明 `init-mcpp-home`、`read-project`、`network`、 `write-global-cache` 与 `exec-build-script`,从不声明 `write-project`。 +### `mcpp.pack` —— 一次打包的产物 *(mcpp 2026.9.16.1+)* + +``` +mcpp pack [target] [--format ] [--target ...] --message-format json +``` + +信封在命令结束后输出一次;所有给人读的行都走 stderr,包括打包启动的构建程序与工具的输出。 +`data` 为: + +| 字段 | | +|---|---| +| `artifacts` | 每个产物一条记录:`path`(绝对路径)、`type`(`file` 或 `directory`)、`format`(`--format` 取值,省略时为 `tar`)与 `targets`(进入该产物的每条腿的规范三元组)。分派格式报告本次请求引入的 action 的终端输出;多 `--target` 的 Android 打包报告一个产物,其 `targets` 列出每条腿 | +| `stage` | 产物所来自的那棵树:`dir`、`manifest`(即下文的暂存清单)与 `closure`(`walked` 或 `not-walked`);库包以及未暂存任何树时为 `null` | + +失败时省略 `data`,以命令自身的退出码退出,并带诊断码 `MCPP_PACK_FAILED`;原因在 stderr 上。 +每次运行的 `effects` 为 `read-project`、`write-project` 与 `write-global-cache`,有构建程序运行时 +再加 `exec-build-script`。`--protocol-version` 为 `pack` 声明 `init-mcpp-home`、 +`read-project`、`write-project`、`network`、`write-global-cache` 与 `exec-build-script`。 + ### `mcpp test --message-format json` —— 测试流 ``` diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index bdb75a56..a064020c 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -183,6 +183,20 @@ struct BuildProgramEnv { // actually has) and writes the placeholder into the action, so the two // never disagree. std::filesystem::path packStageDir; + // The packaging pass's RESOLVED strip decision, "1" or "0", and where the + // separated debug files go, absolute; both empty for every ordinary build + // (#649 E5). A member that stages libraries of its own follows them, so + // `--no-strip` and `--debug-symbols` reach its files as they reach the + // engine's. + std::string packStrip; + std::filesystem::path packDebugSymbolsDir; + // #647 E1: the graph document offered to the ROOT package's program, and + // the digest of its text. Empty for a dependency's program. The digest + // joins the re-run key (see `run_build_program`), so a change to what the + // document says re-runs the program and a change it does not state does + // not. + std::filesystem::path graphFile; + std::string graphDigest; // Whether this package builds C++ modules (`[language] modules`). // // Reported because a rule package that GENERATES a consumer-facing @@ -629,6 +643,9 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv e.emplace_back("MCPP_PKG_REPO", env.packageRepo); e.emplace_back("MCPP_PACK_FORMAT", env.packFormat); e.emplace_back("MCPP_PACK_STAGE_DIR", env.packStageDir.string()); + e.emplace_back("MCPP_PACK_STRIP", env.packStrip); + e.emplace_back("MCPP_PACK_DEBUG_SYMBOLS_DIR", env.packDebugSymbolsDir.string()); + e.emplace_back("MCPP_GRAPH_FILE", env.graphFile.string()); std::string csv; for (auto const& f : env.features) { if (!csv.empty()) csv += ','; @@ -970,6 +987,11 @@ std::expected run_build_program( fs::path outDir = bdir / "out"; auto childEnv = contract_env(root, outDir, env); std::string ctxHash = contract_hash(childEnv); + // THE GRAPH DOCUMENT'S CONTENT, NOT ITS PATH. The path is the same on + // every run; what the document says is what the program's answer depends + // on (#647 E1). + if (!env.graphDigest.empty()) + ctxHash = mcpp::toolchain::hash_string(ctxHash + "\ngraph=" + env.graphDigest); // ── Helper self-containment (the single decision point) ───────────────── // The compiled helper is exec'd by the host OS, outside anything mcpp diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index f5c36b2c..aecbf839 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -560,6 +560,37 @@ inline const char* pack_format() { return env_or("MCPP_PACK_FOR // then names it through the placeholder, so the path in the graph and the path // this program read cannot disagree. inline const char* pack_stage_dir() { return env_or("MCPP_PACK_STAGE_DIR"); } + +// WHETHER THIS PACKAGING PASS STRIPS: "1" or "0", and "" for every ordinary +// build (#649 E5). +// +// The engine strips the program, every shared library the graph built and the +// staged copy of the toolchain's runtime, and `--no-strip` turns all of it off. +// A member that stages libraries of its own -- an Android archive's native +// libraries, say -- asks here instead of deciding for itself, so one switch +// governs every file in the package. An engine older than this one leaves it +// empty, which a member reads as "decide as before". +inline const char* pack_strip() { return env_or("MCPP_PACK_STRIP"); } + +// Where `--debug-symbols` sends the separated `*.debug` files, absolute; "" +// when they are discarded or this build is not packing (#649 E5). +inline const char* pack_debug_symbols_dir() { return env_or("MCPP_PACK_DEBUG_SYMBOLS_DIR"); } + +// THE RESOLVED DEPENDENCY GRAPH, as a JSON document; "" for a dependency's +// program and under an older engine (#647 E1). +// +// Offered to the ROOT package's program only, for the reason `dep_linkage` is: +// the root decides the graph, and when its program runs every input of that +// decision is final. `packages` lists every package, dependencies before the +// packages that request them, each with `package` (identity), `root`, +// `requested_by`, `link` (for a library), `manifest_dir` (absolute), +// `features`, `targets` and `metadata` (its `[package.metadata]`, verbatim). +// A path inside `metadata` is the reader's to resolve, against that entry's +// `manifest_dir`. +// +// Editing a package's `[package.metadata]` re-runs this program; editing its +// sources does not. +inline const char* graph_file() { return env_or("MCPP_GRAPH_FILE"); } inline bool has_feature(const char* name) { char buf[256] = "MCPP_FEATURE_"; unsigned long o = 13; diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index d3888aac..9e0a063c 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -1019,6 +1019,22 @@ export std::string resolve_profile_name(const mcpp::manifest::Manifest& m, return fallback.empty() ? std::string("dev") : std::string(fallback); } +// THE OVERRIDE NAME THE COMMAND LINE STATES, OR "" FOR NONE (#649 E9). +// +// `--profile NAME` > `--release` > `--dev`, and one function for every verb +// that takes the spellings (`build`, `run`, `test`, `emit build-database`, +// `pack`). The rule used to be written twice and the copies disagreed: +// `mcpp build --profile dev --release` built `dev` while `mcpp run` given the +// same line built `release`. Its result is `resolve_profile_name`'s +// `override_name`. +export std::string profile_override_from_flags(std::string_view profileOption, + bool release, bool dev) { + if (!profileOption.empty()) return std::string(profileOption); + if (release) return "release"; + if (dev) return "dev"; + return {}; +} + // Command-level overrides (--target / --static). // Empty defaults preserve pre-existing behaviour exactly. export struct BuildOverrides { @@ -1140,6 +1156,13 @@ export struct BuildOverrides { // reading "this build is not packaging" for a build that plainly is would // be sent looking in the wrong place. std::string pack_stage_reason; + // #649 E5: the strip decision and the debug-symbol directory that pass + // resolved, "1" or "0" and absolute, set only beside `pack_format`. A + // member that stages libraries of its own reads them through + // `mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()`, so + // `--no-strip` reaches its files as it reaches the engine's. + std::string pack_strip; + std::filesystem::path pack_debug_symbols_dir; }; // ── git dependency helpers ────────────────────────────────────────────────── @@ -5777,9 +5800,6 @@ prepare_build(bool print_fingerprint, std::string table; // `DependencySpec::declaredIn` }; std::vector graphRequests; - // The link form each dependency took, and why (`linkage_form::Resolution`), - // by package index. - std::map> graphLinkForms; // The link form each dependency takes and the facts it was decided from, // by package index. COMPUTED ONCE, before the root build program runs, so // that program can read the answer (#642 E2); APPLIED after the scan, where @@ -9589,6 +9609,8 @@ prepare_build(bool print_fingerprint, fill_package_build_env(bpEnv, pkg.manifest); bpEnv.packFormat = overrides.pack_format; bpEnv.packStageDir = overrides.pack_stage_dir; + bpEnv.packStrip = overrides.pack_strip; + bpEnv.packDebugSymbolsDir = overrides.pack_debug_symbols_dir; bpEnv.languageModules = pkg.manifest.language.modules; bpEnv.ruleModules = pkg.manifest.buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(pkg.root.string()); dit != deviceSourcesByPackage.end()) @@ -10707,6 +10729,79 @@ prepare_build(bool print_fingerprint, } } + // ── The resolved graph, one derivation for two readers (#634 X, #647 E1) ── + // + // `resolution.json`'s `graph` section and the document the root build + // program reads (`mcpp::graph_file()`) describe the same packages, and they + // are built by this one function so they cannot disagree. Each entry holds + // the package's identity, every request that reached it (the key as + // written and the table that declared it) and, for a library, its link + // form with the reason. The build program's entries add what a program + // needs to act on a package: its manifest directory, the features it is + // built with, its targets, and its `[package.metadata]` verbatim. + // + // The link form is read from `dependencyLinkForms`, which is computed once, + // before this point, for exactly this program (#642 E2). + auto graph_package_entry = [&](std::size_t i, bool forBuildProgram) { + auto const& pm = packages[i].manifest; + const auto id = mcpp::manifest::package_id(pm.package); + nlohmann::json entry = { + {"package", { + {"canonical", id.canonical()}, + {"namespace", id.namespace_}, + {"name", id.name}, + {"version", id.version}, + {"source", id.sourceProvenance}, + }}, + {"root", i == 0}, + }; + nlohmann::json requests = nlohmann::json::array(); + for (auto const& r : graphRequests) { + if (r.dependencyPackageIndex != i) continue; + requests.push_back({ + {"requester", mcpp::manifest::package_id( + packages[r.consumerPackageIndex].manifest.package).canonical()}, + {"key", r.key}, + {"table", r.table}, + }); + } + entry["requested_by"] = std::move(requests); + if (auto form = dependencyLinkForms.find(i); + form != dependencyLinkForms.end() && form->second.recorded) + entry["link"] = { + {"form", std::string(mcpp::build::linkage_form::to_string( + form->second.answer.linkage))}, + {"reason", form->second.answer.reason}, + }; + if (!forBuildProgram) return entry; + + std::error_code ec; + auto dir = std::filesystem::absolute(packages[i].root, ec).lexically_normal(); + entry["manifest_dir"] = dir.string(); + nlohmann::json feats = nlohmann::json::array(); + if (i < activeFeaturesByPackage.size()) + for (auto const& f : activeFeaturesByPackage[i]) feats.push_back(f); + entry["features"] = std::move(feats); + nlohmann::json targets = nlohmann::json::array(); + for (auto const& t : pm.targets) { + using K = mcpp::manifest::Target::Kind; + const std::string_view kind = + t.kind == K::Library ? "lib" + : t.kind == K::Binary ? "bin" + : t.kind == K::SharedLibrary ? "shared" + : t.kind == K::TestBinary ? "test" + : "app"; + targets.push_back({{"name", t.name}, {"kind", std::string(kind)}}); + } + entry["targets"] = std::move(targets); + entry["metadata"] = pm.packageMetadataJson.empty() + ? nlohmann::json::object() + : nlohmann::json::parse(pm.packageMetadataJson, nullptr, + /*allow_exceptions=*/false); + if (entry["metadata"].is_discarded()) entry["metadata"] = nlohmann::json::object(); + return entry; + }; + // ── L3: ROOT build.mcpp (moved after dependency resolution, design §3.1 // item 4) ──────────────────────────────────────────────────────────────── // Runs HERE — after dep resolution + feature activation (so the contract @@ -10789,6 +10884,68 @@ prepare_build(bool print_fingerprint, bpEnv.hostModules = hostModulesByConsumer.count(0u) ? hostModulesByConsumer.at(0u) : decltype(bpEnv.hostModules){}; + // #649 E5: the packaging pass's strip decision, beside its format. + bpEnv.packStrip = overrides.pack_strip; + bpEnv.packDebugSymbolsDir = overrides.pack_debug_symbols_dir; + // #647 E1: THE RESOLVED GRAPH, FOR THE ROOT'S PROGRAM ONLY. + // + // Every package, dependencies before the packages that request them + // (ties in discovery order), so a program that merges what libraries + // contribute can apply them in override order without a sort of its + // own. The root decides the graph, and every input of that decision is + // final here -- the same reason `dep_linkage` is offered to this + // program alone. A file, not variables: a graph with metadata does not + // fit an environment block (`MAX_ARG_STRLEN`, the Windows limit). + // + // ITS DIGEST JOINS THE RE-RUN KEY. Editing a dependency's + // `[package.metadata]` changes what this program would answer, so it + // must run again; editing that dependency's sources does not, and the + // document does not change. + { + std::vector order; + std::vector placed(packages.size(), false); + while (order.size() < packages.size()) { + std::size_t pick = packages.size(); + for (std::size_t i = 0; i < packages.size() && pick == packages.size(); ++i) { + if (placed[i]) continue; + bool ready = true; + for (auto const& r : graphRequests) + if (r.consumerPackageIndex == i && r.dependencyPackageIndex != i + && r.dependencyPackageIndex < packages.size() + && !placed[r.dependencyPackageIndex]) { ready = false; break; } + if (ready) pick = i; + } + // A cycle leaves nothing ready; its first member in discovery + // order is taken so the document is still complete. + if (pick == packages.size()) + for (std::size_t i = 0; i < packages.size(); ++i) + if (!placed[i]) { pick = i; break; } + placed[pick] = true; + order.push_back(pick); + } + nlohmann::json doc; + doc["kind"] = "mcpp.graph"; + doc["version"] = 1; + nlohmann::json list = nlohmann::json::array(); + for (auto i : order) list.push_back(graph_package_entry(i, true)); + doc["packages"] = std::move(list); + const auto text = doc.dump(2) + "\n"; + const auto graphPath = bpEnv.artifactsDir / "graph.json"; + std::error_code gec; + std::filesystem::create_directories(graphPath.parent_path(), gec); + const auto tmp = graphPath.string() + ".tmp"; + { + std::ofstream out(tmp, std::ios::binary | std::ios::trunc); + out << text; + } + std::filesystem::rename(tmp, graphPath, gec); + if (gec) + return std::unexpected(std::format( + "cannot write the graph document '{}': {}", + graphPath.string(), gec.message())); + bpEnv.graphFile = graphPath; + bpEnv.graphDigest = mcpp::toolchain::hash_string(text); + } auto& bcRoot = m->buildConfig; const auto mark = markDirectiveTail(*m); const auto rldN = bcRoot.ldflags.size(), rsrcN = bcRoot.sources.size(), @@ -11886,9 +12043,6 @@ prepare_build(bool print_fingerprint, for (auto const& [i, form] : dependencyLinkForms) { auto const& answer = form.answer; auto const& facts = form.facts; - if (form.recorded) - graphLinkForms[i] = { std::string(lf::to_string(answer.linkage)), - answer.reason }; if (!answer.diagnostic.empty()) mcpp::diag::degraded("build/dependency-linkage", answer.diagnostic, @@ -13480,28 +13634,8 @@ prepare_build(bool print_fingerprint, // reads instead of a warning's wording. { nlohmann::json graphPackages = nlohmann::json::array(); - for (std::size_t i = 0; i < packages.size(); ++i) { - nlohmann::json entry = { - {"package", package_json( - mcpp::manifest::package_id(packages[i].manifest.package))}, - {"root", i == 0}, - }; - nlohmann::json requests = nlohmann::json::array(); - for (auto const& r : graphRequests) { - if (r.dependencyPackageIndex != i) continue; - requests.push_back({ - {"requester", mcpp::manifest::package_id( - packages[r.consumerPackageIndex].manifest.package).canonical()}, - {"key", r.key}, - {"table", r.table}, - }); - } - entry["requested_by"] = std::move(requests); - if (auto form = graphLinkForms.find(i); form != graphLinkForms.end()) - entry["link"] = { {"form", form->second.first}, - {"reason", form->second.second} }; - graphPackages.push_back(std::move(entry)); - } + for (std::size_t i = 0; i < packages.size(); ++i) + graphPackages.push_back(graph_package_entry(i, /*forBuildProgram=*/false)); j["graph"] = { {"packages", std::move(graphPackages)} }; } diff --git a/src/cli.cppm b/src/cli.cppm index 92583126..542182e0 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -620,10 +620,18 @@ int run(int argc, char** argv) { // this only replaces the "dev" fallback every other command uses. .option(cl::Option("profile").takes_value() .help("Build profile (default: [build] default-profile, else release)")) + // The shorthands `build` and `run` take, with their precedence: + // `--profile` wins over either (#649 E9). + .option(cl::Option("release").help("Shorthand for --profile release")) + .option(cl::Option("dev").help("Shorthand for --profile dev")) .option(cl::Option("toolchain").takes_value().value_name("SPEC") .help("Build with this toolchain for one invocation, e.g. llvm@22.1.8")) .option(cl::Option("features").takes_value().value_name("LIST") .help("Activate root-package features for every build pass of the pack (comma-separated)")) + // `--format` names the PACKAGE format here, so machine output is + // asked for the way `mcpp test` asks for it (docs/50 §3). + .option(cl::Option("message-format").takes_value().value_name("FMT") + .help("Output format: human (default) | json (one mcpp.pack envelope on stdout; narration on stderr)")) .option(cl::Option("no-strip") .help("Ship the artifacts as built (default: strip debug info)")) .option(cl::Option("debug-symbols").takes_value().value_name("DIR") @@ -1075,6 +1083,11 @@ int run(int argc, char** argv) { {"emit build-database", {Effect::InitMcppHome, Effect::ReadProject, Effect::Network, Effect::WriteGlobalCache, Effect::ExecBuildScript}}, + // A build, then a package under `target/dist`: the build's + // declaration plus `write-project`. + {"pack", {Effect::InitMcppHome, Effect::ReadProject, + Effect::WriteProject, Effect::Network, + Effect::WriteGlobalCache, Effect::ExecBuildScript}}, }; }; diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index d7df3682..ed63cbff 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -94,6 +94,13 @@ int run_build_with_hooks(mcpp::build::BuildContext& ctx, bool verbose, // The build selectors, read once for every command that plans a build: `mcpp // build` and `mcpp emit build-database` select the same plan from the same // flags, so the database describes the build the same flags would run. +// The profile the selectors name; see `mcpp::build::profile_override_from_flags`. +std::string profile_from_selectors(const mcpplibs::cmdline::ParsedArgs& parsed) { + return mcpp::build::profile_override_from_flags( + parsed.value("profile").value_or(""), + parsed.is_flag_set("release"), parsed.is_flag_set("dev")); +} + mcpp::build::BuildOverrides overrides_from_selectors( const mcpplibs::cmdline::ParsedArgs& parsed) { mcpp::build::BuildOverrides ov; @@ -109,9 +116,7 @@ mcpp::build::BuildOverrides overrides_from_selectors( // Profile selection precedence: --profile NAME > --release / --dev > the // project default ([build].default-profile) > "release", resolved in // prepare_build. --release/--dev are shorthands only. - if (auto pr = parsed.value("profile")) ov.profile = *pr; - else if (parsed.is_flag_set("release")) ov.profile = "release"; - else if (parsed.is_flag_set("dev")) ov.profile = "dev"; + ov.profile = profile_from_selectors(parsed); if (auto fs = parsed.value("features")) ov.features = *fs; if (auto cp = parsed.value("cap")) ov.capabilities = *cp; ov.strict = parsed.is_flag_set("strict"); @@ -472,11 +477,9 @@ export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, if (auto rn = parsed.value("runner")) runner_name = *rn; // The same two axes `build` and `test` take, read the same way. `--release` // and `--dev` are the shorthands the other verbs already accept. - std::string features, profile; + std::string features; if (auto fs = parsed.value("features")) features = *fs; - if (auto pr = parsed.value("profile")) profile = *pr; - if (parsed.is_flag_set("release")) profile = "release"; - if (parsed.is_flag_set("dev")) profile = "dev"; + const std::string profile = profile_from_selectors(parsed); // The device axis, read exactly as `build` reads it: `--no-accel` is an // explicit choice and not the absence of `--accel`, so it travels as the // same sentinel. Without this a project's CPU-only variant could be built @@ -506,7 +509,7 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, // granularity for sanitizers / contract evaluation semantics). Post-`--` // args go to each test binary. mcpp::build::BuildOverrides ov; - if (auto pr = parsed.value("profile")) ov.profile = *pr; + ov.profile = profile_from_selectors(parsed); if (auto fs = parsed.value("features")) ov.features = *fs; if (auto cp = parsed.value("cap")) ov.capabilities = *cp; ov.strict = parsed.is_flag_set("strict"); diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 0c8cc8f8..e304d57d 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -9,12 +9,16 @@ export module mcpp.cli.cmd_publish; import std; import mcpplibs.cmdline; +import mcpp.build.prepare; // profile_override_from_flags +import mcpp.libs.json; import mcpp.pack; import mcpp.pack.library_pipeline; import mcpp.pack.pipeline; import mcpp.pack.route; +import mcpp.platform.terminal; import mcpp.publish.pipeline; import mcpp.ui; +import mcpp.wire; namespace mcpp::cli { @@ -32,7 +36,99 @@ export int cmd_publish(const mcpplibs::cmdline::ParsedArgs& parsed) { parsed.is_flag_set("dry-run"), parsed.is_flag_set("allow-dirty")); } +namespace { + +int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, + mcpp::pack::PackOutcome* report, + mcpp::pack::LibraryPackReport* libraryReport, + bool* libraryRoute); + +} // namespace + +// `mcpp pack --message-format json` (#649 E9). +// +// ONE ENVELOPE ON STDOUT, AND EVERYTHING ELSE ON STDERR. `--format` names the +// PACKAGE format on this command, so machine output is asked for the way +// `mcpp test` asks for it, and the document is printed after the command has +// finished, alone: planning, building and packing narrate, and so do the +// programs they start, which is why the whole run is under `StdoutToStderr`, +// as `emit build-database`'s planning is. +// +// `data.artifacts` holds what the human `Packed` lines name, as absolute paths: +// the archive or tree of a built-in format, the terminal outputs of a +// dispatched one, or the library package. A leg is a `targets` entry of the +// artifact it went into, not an artifact of its own. export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { + std::string messageFormat = "human"; + if (auto mf = parsed.value("message-format")) messageFormat = *mf; + if (messageFormat != "human" && messageFormat != "json") { + mcpp::ui::error(std::format("unknown --message-format '{}' (human|json)", + messageFormat)); + return 2; + } + if (messageFormat == "human") + return cmd_pack_body(parsed, nullptr, nullptr, nullptr); + + mcpp::pack::PackOutcome outcome; + mcpp::pack::LibraryPackReport library; + bool libraryRoute = false; + int rc = 0; + { + mcpp::platform::terminal::StdoutToStderr narration; + rc = cmd_pack_body(parsed, &outcome, &library, &libraryRoute); + } + + using mcpp::wire::Effect; + std::vector effects{Effect::ReadProject, Effect::WriteProject, + Effect::WriteGlobalCache}; + if (outcome.ranBuildPrograms) effects.push_back(Effect::ExecBuildScript); + mcpp::wire::Envelope env{ .kind = "mcpp.pack", .effects = std::move(effects) }; + if (rc != 0) { + env.data = nullptr; + env.diagnostics.push_back({"MCPP_PACK_FAILED", mcpp::wire::Severity::Error, + "mcpp pack did not produce a package; the reason is on standard error"}); + mcpp::wire::emit(env); + return rc; + } + + auto artifact_json = [](const std::filesystem::path& p, std::string const& format, + std::vector const& targets) { + std::error_code ec; + nlohmann::json t = nlohmann::json::array(); + for (auto const& x : targets) t.push_back(x); + return nlohmann::json{ + {"path", std::filesystem::absolute(p, ec).lexically_normal().string()}, + {"type", std::filesystem::is_directory(p, ec) ? "directory" : "file"}, + {"format", format}, + {"targets", std::move(t)}, + }; + }; + nlohmann::json artifacts = nlohmann::json::array(); + nlohmann::json stage = nullptr; + if (libraryRoute) { + const auto fmt = parsed.value("format").value_or("tar"); + artifacts.push_back(artifact_json(library.artifact, fmt, library.targets)); + } else { + for (auto const& a : outcome.artifacts) + artifacts.push_back(artifact_json(a, outcome.format, outcome.targets)); + if (!outcome.stageDir.empty()) + stage = nlohmann::json{ + {"dir", outcome.stageDir.lexically_normal().string()}, + {"manifest", outcome.stageManifest.lexically_normal().string()}, + {"closure", outcome.closure}, + }; + } + env.data = nlohmann::json{{"artifacts", std::move(artifacts)}, {"stage", std::move(stage)}}; + mcpp::wire::emit(env); + return 0; +} + +namespace { + +int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, + mcpp::pack::PackOutcome* report, + mcpp::pack::LibraryPackReport* libraryReport, + bool* libraryRoute) { // ─── Resolve mode ──────────────────────────────────────────────── mcpp::pack::Options opts; bool modeFromUser = false; @@ -76,7 +172,10 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { // `mcpp run q` once became `--target=q`. `pack`'s positional is `target`, // so `--target` is read through `option()` above and stays unaffected; // `profile` and `debug-symbols` have no positional twin. - if (auto v = parsed.value("profile")) opts.profile = *v; + // `--profile` > `--release` / `--dev`, the rule `build` and `run` follow. + opts.profile = mcpp::build::profile_override_from_flags( + parsed.value("profile").value_or(""), + parsed.is_flag_set("release"), parsed.is_flag_set("dev")); if (parsed.is_flag_set("no-strip")) opts.strip = false; if (auto v = parsed.value("debug-symbols")) opts.debugSymbols = *v; if (auto v = parsed.value("features")) opts.features = *v; @@ -123,7 +222,9 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { "--mode is an application-bundle depth and does not apply to the " "library target '{}' yet; ignoring it", route->targetName)); } - return mcpp::pack::build_and_pack_library(route->targetName, triples, opts); + if (libraryRoute) *libraryRoute = true; + return mcpp::pack::build_and_pack_library(route->targetName, triples, opts, + libraryReport); } if (triples.size() > 1) { // #630 A9: THE ROUTE IS CHOSEN BY THE ARTIFACT'S FORM, NOT BY THE @@ -152,13 +253,19 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { // #622 A10: `build_and_pack` now reports the artifact(s) it packed, for // `mcpp run --format` to take as its operand -- `mcpp pack` itself only // ever needed the exit code. - return mcpp::pack::build_and_pack(std::move(opts), modeFromUser, - route->targetName, std::move(*extraLegs)).rc; + auto out = mcpp::pack::build_and_pack(std::move(opts), modeFromUser, + route->targetName, std::move(*extraLegs)); + if (report) *report = out; + return out.rc; } // #622 A10: `build_and_pack` now reports the artifact(s) it packed, for // `mcpp run --format` to take as its operand -- `mcpp pack` itself only // ever needed the exit code. - return mcpp::pack::build_and_pack(std::move(opts), modeFromUser, route->targetName).rc; + auto out = mcpp::pack::build_and_pack(std::move(opts), modeFromUser, route->targetName); + if (report) *report = out; + return out.rc; } +} // namespace + } // namespace mcpp::cli diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index b3a504e2..d3bf1599 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -110,9 +110,18 @@ std::vector extras_of(const mcpp::manifest::Manifest& m, // `triples` is the `--target` list; empty means "this host". Each entry gets // its own prepare+build, so the artifacts really are the ones this run made — // the packer never searches `target/` for something that looks right. +// +// `report`, when given, receives what `mcpp pack --message-format json` states +// (#649 E9): the package produced and the canonical triple of each leg. +export struct LibraryPackReport { + std::filesystem::path artifact; + std::vector targets; +}; + export int build_and_pack_library(const std::string& targetName, const std::vector& triples, - const mcpp::pack::Options& opts) + const mcpp::pack::Options& opts, + LibraryPackReport* report = nullptr) { std::vector legs = triples; if (legs.empty()) legs.push_back({}); // one leg, this host @@ -478,6 +487,11 @@ export int build_and_pack_library(const std::string& targetName, auto out = run_library_pack(plan); if (!out) { mcpp::ui::error(out.error().message); return 1; } mcpp::ui::status("Packed", out->string()); + if (report) { + std::error_code ec; + report->artifact = std::filesystem::absolute(*out, ec).lexically_normal(); + for (auto const& leg : plan.legs) report->targets.push_back(leg.triple); + } return 0; } diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 3035d932..c9c2c83b 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -189,7 +189,12 @@ std::filesystem::path resolve_debug_dir(const Options& opts, // its own directories (see `Options::toolchainLibraryDirs`). struct SharedLeg { std::string abi; + // The leg's canonical triple, for the pack report (#649 E9). + std::string triple; std::filesystem::path artifact; + // #649 E5: the shared libraries THIS leg's graph built. See + // `Plan::graphSharedLibraries`. + std::vector graphSharedLibraries; std::vector searchDirs; std::vector platformDirs; }; @@ -255,8 +260,21 @@ struct Plan { bool strip = true; std::filesystem::path debugDir; // absolute; empty = discard mcpp::pack::StripTools stripTools; + // #649 E5: WHAT THIS GRAPH BUILT, absolute: every `SharedLibrary` link + // unit of the plan, the dependencies' shared images included. A staged + // copy of one of these is stripped as the program is, because "a package + // strips what it built" (the dh_strip rule `strip_program` quotes) covers + // them: they were compiled from source by this build. Set after + // `make_plan`, by the caller that holds the build plan. + std::vector graphSharedLibraries; }; +// Does packing `plan` strip anything? The strip decision, AND a format whose +// debug information is inside the image. A Mach-O or MSVC PE row keeps its +// debug information beside the image, so the answer there is no whatever +// was requested; the `Packing` line states this, not the request. +bool strips_on_this_row(const Plan& plan); + struct Error { std::string message; }; // What step 4 of `run` (resolving the dependency closure) produced. @@ -453,6 +471,12 @@ std::string wrapper_dirname_from_archive(const std::filesystem::path& archive) { } // namespace detail +bool strips_on_this_row(const Plan& plan) { + // wasm32-emscripten is packed by `run_wasm`, which has no strip step: the + // launcher is JavaScript and the module is one static image. + return plan.strip && plan.stripTools.inBandDebugInfo && !plan.targetIsWasm; +} + bool resolve_strip(const Options& opts, const mcpp::manifest::PackConfig& cfg) { if (opts.strip) return *opts.strip; if (cfg.strip) return *cfg.strip; @@ -815,12 +839,12 @@ set_interpreter(const std::filesystem::path& binary, return {}; } -// Remove the program's debug information — and ONLY the program's. +// Remove the program's debug information. // -// A bundled `.so` is somebody else's file: it came out of the store or off the -// host, mcpp did not build it, and stripping it would change a shared payload's -// bytes for no gain to this bundle. dh_strip draws the same line (a package -// strips what it built). +// dh_strip's line is that a package strips what it built, and the program is +// the first thing this one built. The shared libraries the same graph compiled +// are the rest (`strip_staged_library` below, #649 E5); a library that came out +// of the store or off the host is somebody else's file and is left as shipped. // // Shared with `run_pe` deliberately: a MinGW `.exe` carries DWARF in-band just // like an ELF one, so "does the bundle ship debug info" must not depend on @@ -835,6 +859,91 @@ strip_program(const Plan& plan, const std::filesystem::path& staged) return {}; } +// IS `file` ONE OF `set`? Compared as file-system identity (after `..` and +// symbolic links are resolved), so the loader's spelling of a path and the +// plan's spelling of the same file agree. +bool path_in(const std::filesystem::path& file, + const std::vector& set) +{ + std::error_code ec; + const auto want = std::filesystem::weakly_canonical(file, ec); + for (auto const& p : set) { + std::error_code pec; + if (std::filesystem::weakly_canonical(p, pec) == want) return true; + } + return false; +} + +// DOES `file` LIE UNDER ONE OF `dirs`? +bool path_under(const std::filesystem::path& file, + const std::vector& dirs) +{ + std::error_code ec; + const auto f = std::filesystem::weakly_canonical(file, ec); + for (auto const& d : dirs) { + std::error_code dec; + const auto rel = f.lexically_relative(std::filesystem::weakly_canonical(d, dec)); + if (!rel.empty() && *rel.begin() != "..") return true; + } + return false; +} + +// SHOULD THE STAGED COPY OF `source` BE STRIPPED? (#649 E5) +// +// Two answers are yes: +// +// built by this graph a `SharedLibrary` link unit of the plan -- compiled +// from source here, so dh_strip's rule applies to it +// exactly as it applies to the program; +// the toolchain's own a runtime found in the directories the row's own +// runtime toolchain links from (`libc++_shared.so` from the +// NDK). The reason for leaving a store file alone is +// not changing a SHARED payload's bytes, and a copy in +// the staged tree is not shared. The Android Gradle +// plugin strips the same file. +// +// Everything else -- a library from the store, a prebuilt a package deployed, +// a host library -- stays as shipped: a vendor library may be signed, and its +// bytes are its publisher's. +bool strips_staged_copy(const Plan& plan, + const std::filesystem::path& source, + const std::vector& graphBuilt) +{ + if (path_in(source, graphBuilt)) return true; + if (path_under(source, plan.opts.toolchainLibraryDirs)) return true; + if (plan.opts.carryToolchainRuntime + && path_under(source, plan.opts.toolchainRuntimeDirs)) return true; + return false; +} + +// Strip the staged copy of a shared library, keeping its exports +// (`--strip-unneeded`, the `SharedLibrary` shape). `--no-strip` reaches it +// through `plan.strip`, and `--debug-symbols` through `debugDir`, as they reach +// the program. +// +// NEVER THE BUILD'S OWN FILE. A member the build already placed where it is +// staged is the same file as its source (see `stage_closure`), and editing it +// would change an output ninja believes is up to date. +std::expected +strip_staged_library(const Plan& plan, + const std::filesystem::path& staged, + const std::filesystem::path& source, + const std::filesystem::path& debugDir) +{ + if (!plan.strip) return {}; + std::error_code ec; + if (std::filesystem::equivalent(source, staged, ec)) return {}; + // A copy of a read-only store file is read-only too, and a strip tool + // rewrites the file it is given. + std::filesystem::permissions(staged, std::filesystem::perms::owner_write, + std::filesystem::perm_options::add, ec); + auto r = mcpp::pack::strip_artifact(staged, + mcpp::pack::ArtifactShape::SharedLibrary, + plan.stripTools, debugDir); + if (!r) return std::unexpected(Error{r.error()}); + return {}; +} + // Bundle all `deps` into /lib/. We dereference any // symlinks so the bundle is self-contained even if /usr/lib/foo.so → /usr/lib/foo.so.1. std::expected @@ -1132,7 +1241,8 @@ stage_closure(const Plan& plan, const ClosureRead& read, const std::filesystem::path& dir, std::vector& needs, std::vector& unresolvedLines, - std::string_view prefix) + std::string_view prefix, + std::vector>* copies = nullptr) { auto skipped = [&](const std::string& name) { const auto leaf = std::filesystem::path(name).filename().string(); @@ -1162,6 +1272,8 @@ stage_closure(const Plan& plan, const ClosureRead& read, "failed to copy {} -> {}: {}", m.source.string(), dst.string(), cec.message())}); } + // (staged, source), for the caller that strips what the graph built. + if (copies) copies->emplace_back(dst, m.source); needs.push_back({m.name, ClosureNeed::Kind::Staged, dst.lexically_relative(plan.stagingRoot).generic_string()}); } @@ -1474,6 +1586,7 @@ run_pe(const Plan& plan) } std::vector needs; + std::vector> copies; if (plan.opts.mode != Mode::None && plan.opts.mode != Mode::Static) { // `vendored` and `self-contained` collect the same set here, and that // is a property of the PLATFORM rather than a simplification. @@ -1493,7 +1606,8 @@ run_pe(const Plan& plan) // is an executable the reader could not parse, which no format can // package. std::vector unresolved; - if (auto r = stage_closure(plan, read, plan.stagingRoot, needs, unresolved, {}); !r) + if (auto r = stage_closure(plan, read, plan.stagingRoot, needs, unresolved, {}, + &copies); !r) return std::unexpected(r.error()); if (!unresolved.empty()) return std::unexpected(Error{unresolved_reason(plan.binaryName, unresolved, @@ -1501,6 +1615,13 @@ run_pe(const Plan& plan) } if (auto r = strip_program(plan, stagedExe); !r) return std::unexpected(r.error()); + // #649 E5: and the DLLs this graph built (a MinGW DLL carries its DWARF + // in-band as the `.exe` does), plus the toolchain runtime it carries. + for (auto const& [staged, source] : copies) { + if (!strips_staged_copy(plan, source, plan.graphSharedLibraries)) continue; + if (auto r = strip_staged_library(plan, staged, source, plan.debugDir); !r) + return std::unexpected(r.error()); + } if (plan.opts.format != Format::Tar) return needs; @@ -1626,6 +1747,8 @@ run_shared_program(const Plan& plan) std::filesystem::path artifact; std::vector searchDirs; std::vector platformDirs; + // #649 E5: what this leg's graph built. + std::vector graphBuilt; }; auto primarySearch = plan.searchDirs; for (auto const& d : plan.opts.toolchainLibraryDirs) primarySearch.push_back(d); @@ -1640,13 +1763,14 @@ run_shared_program(const Plan& plan) auto t = mcpp::toolchain::triple::parse(plan.triple); auto primaryAbi = t ? mcpp::toolchain::triple::android_abi(*t) : plan.triple; legs.push_back({plan.stagingRoot / "lib" / primaryAbi, primaryAbi, - plan.builtBinary, primarySearch, plan.opts.platformLibraryDirs}); + plan.builtBinary, primarySearch, plan.opts.platformLibraryDirs, + plan.graphSharedLibraries}); for (auto const& leg : plan.extraSharedLegs) legs.push_back({plan.stagingRoot / "lib" / leg.abi, leg.abi, leg.artifact, - leg.searchDirs, leg.platformDirs}); + leg.searchDirs, leg.platformDirs, leg.graphSharedLibraries}); } else { legs.push_back({plan.stagingRoot / "lib", {}, plan.builtBinary, primarySearch, - plan.opts.platformLibraryDirs}); + plan.opts.platformLibraryDirs, plan.graphSharedLibraries}); } std::vector needs; @@ -1679,8 +1803,38 @@ run_shared_program(const Plan& plan) in.searchDirs = leg.searchDirs; in.platformDirs = leg.platformDirs; const auto read = read_closure(in); - if (auto r = stage_closure(plan, read, leg.dir, needs, unresolved, prefix); !r) + std::vector> copies; + if (auto r = stage_closure(plan, read, leg.dir, needs, unresolved, prefix, + &copies); !r) return std::unexpected(r.error()); + + // #649 E5: THIS ROW USED TO STRIP NOTHING. `run` dispatches a + // shared-object program here before the ELF path that calls + // `strip_program`, so the program, the libraries its graph built and + // the NDK's `libc++_shared.so` all shipped their symbol tables while + // the `Packing` line said "stripped", and `--debug-symbols` was + // ignored. The program is a shared object, so it takes the + // `SharedLibrary` shape, which keeps the `.dynsym` its host loads it + // through. One tool serves every leg: the NDK's `llvm-strip` reads + // every Android ABI. A several-ABI tree separates its debug files by + // ABI, since two legs stage files of one name. + const auto legDebugDir = (plan.debugDir.empty() || leg.abi.empty()) + ? plan.debugDir : plan.debugDir / leg.abi; + if (plan.strip) { + const auto stagedProgram = leg.dir / plan.binaryName; + std::error_code pec; + std::filesystem::permissions(stagedProgram, std::filesystem::perms::owner_write, + std::filesystem::perm_options::add, pec); + auto r = mcpp::pack::strip_artifact(stagedProgram, + mcpp::pack::ArtifactShape::SharedLibrary, + plan.stripTools, legDebugDir); + if (!r) return std::unexpected(Error{r.error()}); + } + for (auto const& [staged, source] : copies) { + if (!strips_staged_copy(plan, source, leg.graphBuilt)) continue; + if (auto r = strip_staged_library(plan, staged, source, legDebugDir); !r) + return std::unexpected(r.error()); + } } // THE RUNTIME FILES TRAVEL AS ON EVERY OTHER ROW. `deploy` placed them @@ -1847,6 +2001,10 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // text when it is incomplete. Both stay empty under `--mode static`. std::vector needs; std::string unresolvedText; + // #649 E5: (staged, source) of every bundled library, for the strip step + // at the end, which must see the final images (after the search-path edits). + std::vector> bundledCopies; + std::string bundledLoader; if (plan.opts.mode != Mode::Static) { // THE BUILT BINARY, NOT THE STAGED COPY, and the difference is // `$ORIGIN`. @@ -1954,6 +2112,9 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) if (auto r = bundle_libs(toBundle, plan.stagingRoot); !r) return std::unexpected(Error{r.error()}); + for (auto const& d : toBundle) + bundledCopies.emplace_back(plan.stagingRoot / "lib" / d.soname, d.path); + bundledLoader = find_loader_soname(toBundle); // Search path: point at bundled libs, or REMOVE THE TAG if there are none. // @@ -2077,6 +2238,15 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // must see the stripped one. Same ordering rule the library packer states // at its leg loop. if (auto r = strip_program(plan, bundledBinary); !r) return std::unexpected(r.error()); + // #649 E5: and the shared libraries this graph built. The dynamic loader + // is never one of them and is never edited, for the reason the + // search-path loop above gives. + for (auto const& [staged, source] : bundledCopies) { + if (!bundledLoader.empty() && staged.filename() == bundledLoader) continue; + if (!strips_staged_copy(plan, source, plan.graphSharedLibraries)) continue; + if (auto r = strip_staged_library(plan, staged, source, plan.debugDir); !r) + return std::unexpected(r.error()); + } // A dispatched format reaches this point with an incomplete closure; an // archive was refused before any file was edited. diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 9f7a8221..140519b1 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -43,6 +43,21 @@ namespace mcpp::pack { export struct PackOutcome { int rc = 0; std::vector artifacts; + // #649 E9: what `mcpp pack --message-format json` reports beside the + // artifacts. All of it was already answered by the pass that produced + // them; nothing here is derived for the report. + // + // `format` is the `--format` value as requested (`tar` when omitted); + // `targets` the canonical triple of every leg, the primary first. The + // stage fields are empty when no tree was staged; `closure` is the stage + // manifest's own word, `walked` or `not-walked`. + std::string format; + std::vector targets; + std::filesystem::path stageDir; + std::filesystem::path stageManifest; + std::string closure; + // Whether a build program ran in this pack, for the envelope's `effects`. + bool ranBuildPrograms = false; }; // #634 A3: the two directory lists an Android row's closure is read against, @@ -168,7 +183,13 @@ build_extra_android_legs(const std::string& targetName, // this leg's own plan and driver. SharedLeg leg; leg.abi = std::move(abi); + leg.triple = canonical; leg.artifact = ctx->outputDir / lu->output; + // #649 E5: what THIS leg's graph built, which pack strips as it strips + // the program; see `Plan::graphSharedLibraries`. + for (auto const& u : ctx->plan.linkUnits) + if (u.kind == mcpp::build::LinkUnit::SharedLibrary) + leg.graphSharedLibraries.push_back(ctx->outputDir / u.output); leg.searchDirs.push_back(leg.artifact.parent_path()); for (auto const& d : ctx->plan.runtimeLibraryDirs) leg.searchDirs.push_back(d); for (auto const& d : ctx->plan.linkIntent.runtimeSearchDirs) @@ -473,6 +494,12 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, // parameter comment on `extraLegs` above. A no-op (default-constructed, // empty) for every caller before this item. plan->extraSharedLegs = std::move(extraLegs); + // #649 E5: the shared libraries this graph built, primary leg. From the + // plan's link units, never from a directory listing: a vendor library + // deployed beside the program is not one of them and stays as shipped. + for (auto const& u : ctx->plan.linkUnits) + if (u.kind == mcpp::build::LinkUnit::SharedLibrary) + plan->graphSharedLibraries.push_back(ctx->outputDir / u.output); // The RESOLVED debug-information decision. On the plan, not in Options: // Options is the request, this is what it came out as once the manifest @@ -496,10 +523,14 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, }()), }; + // "stripped" STATES WHAT THIS ROW DOES, NOT WHAT WAS REQUESTED (#649 E5). + // It used to print the decision alone, so a Mach-O row, whose debug + // information is not in the image and is never stripped, and the Android + // row, which did not reach the strip step at all, both said "stripped". mcpp::ui::info("Packing", std::format("{} v{} ({}{})", plan->packageName, plan->packageVersion, mcpp::pack::mode_cli_name(plan->opts.mode), - plan->strip ? ", stripped" : "")); + mcpp::pack::strips_on_this_row(*plan) ? ", stripped" : "")); // STAGING IS A SERVICE TO THE PROVIDER, NOT A PRECONDITION FOR DISPATCH. // @@ -565,6 +596,34 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, // no tree exists, so no manifest describes a tree that is not there. if (stageFailure.empty()) mcpp::pack::write_stage_manifest(plan->stagingRoot, closure); + // #649 E9: the outcome a machine reader receives, from values this pass + // has already answered. Both returns below go through it. + auto outcome_with = [&](std::vector artifacts) { + PackOutcome o; + o.artifacts = std::move(artifacts); + o.format = opts.format == mcpp::pack::Format::Tar ? std::string("tar") + : opts.format == mcpp::pack::Format::Dir ? std::string("dir") + : opts.formatName; + const auto canonical = [](std::string const& t) { + if (t.empty()) return mcpp::toolchain::triple::host_triple().str(); + auto parsed = mcpp::toolchain::triple::parse(t); + return parsed ? parsed->str() : t; + }; + o.targets.push_back(canonical(plan->triple)); + for (auto const& leg : plan->extraSharedLegs) o.targets.push_back(leg.triple); + if (stageFailure.empty()) { + o.stageDir = plan->stagingRoot; + o.stageManifest = mcpp::pack::stage_manifest_path(plan->stagingRoot); + o.closure = closure.walked ? "walked" : "not-walked"; + } + std::error_code bec; + o.ranBuildPrograms = std::filesystem::exists(ctx->projectRoot / "build.mcpp", bec) + || !ctx->manifest.buildConfig.ruleModules.empty(); + for (auto const& sp : ctx->sourcePackages) + if (std::filesystem::exists(sp.root / "build.mcpp", bec)) o.ranBuildPrograms = true; + return o; + }; + auto pathCtx = mcpp::fetcher::make_path_ctx(&*cfg, ctx->projectRoot); // ─── The dispatch pass ─────────────────────────────────────────── @@ -595,6 +654,11 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, preexistingArtifacts.emplace(a.packageName, a.id); ov.pack_format = opts.formatName; + // #649 E5: the RESOLVED strip decision and debug directory, for a + // member that stages libraries of its own and must follow the same + // switch `--no-strip` and `--debug-symbols` set for this tree. + ov.pack_strip = plan->strip ? "1" : "0"; + ov.pack_debug_symbols_dir = plan->debugDir; // Empty when staging was refused, which is what makes // `${mcpp.stage_dir}` refuse with the reason attached rather than // expand to a directory that does not exist. @@ -710,13 +774,13 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, // Every output consumed by another: a cycle a provider should not // write, reported as all outputs rather than as nothing. if (reported.empty()) reported = std::move(intermediate); - return PackOutcome{0, std::move(reported)}; + return outcome_with(std::move(reported)); } auto outPath = (opts.format == mcpp::pack::Format::Tar) ? plan->archivePath : plan->stagingRoot; mcpp::ui::status("Packed", mcpp::ui::shorten_path(outPath, pathCtx)); - return PackOutcome{0, {outPath}}; + return outcome_with({outPath}); } } // namespace mcpp::pack diff --git a/src/wire.cppm b/src/wire.cppm index df94a93d..bb9f7381 100644 --- a/src/wire.cppm +++ b/src/wire.cppm @@ -67,7 +67,7 @@ inline constexpr int kEnvelopeVersion = 1; // from what comes back. struct KindVersion { std::string_view kind; int version; }; -inline constexpr std::array kKinds{{ +inline constexpr std::array kKinds{{ {"mcpp.env", 1}, {"mcpp.xpkg", 1}, {"mcpp.cache", 1}, @@ -84,6 +84,12 @@ inline constexpr std::array kKinds{{ // (`data.watch`) and their digest (`data.inputs-fingerprint`). Written // into nothing; see docs/specs/build-database.md. {"mcpp.build-database", 1}, + // `mcpp pack --message-format json`: what the pack produced -- each + // artifact with its absolute path, whether it is a file or a directory, + // the format and the triples of its legs -- and the staged tree it was + // made from (`data.stage`). `--format` names the package format on that + // command, hence the `test`-style spelling (docs/50 §3). + {"mcpp.pack", 1}, }}; // What running a command does, beyond writing to stdout. diff --git a/tests/e2e/720_the_root_build_program_reads_the_resolved_graph.sh b/tests/e2e/720_the_root_build_program_reads_the_resolved_graph.sh new file mode 100755 index 00000000..8561ae34 --- /dev/null +++ b/tests/e2e/720_the_root_build_program_reads_the_resolved_graph.sh @@ -0,0 +1,176 @@ +#!/usr/bin/env bash +# requires: python3 +# 720 -- the root package's build program reads the resolved dependency graph +# and each package's `[package.metadata]` through `mcpp::graph_file()` (#647 E1). +# +# A framework merges what its libraries contribute (resources, platform +# sources) at build time, in dependency order, including libraries the +# application does not name. `dep_dir` answers by name for a direct dependency +# only, so a transitive library had no name to be asked for. +# +# Legs: +# A. app -> spike.a -> spike.b: the document lists b before a before app, the +# root last; b's entry carries its absolute manifest directory, its target +# kind and `metadata.demo.resources = "res"` verbatim. +# B. A second build with nothing changed does not re-run the root program. +# C. Editing b's `[package.metadata]` re-runs the root program, and the +# document states the new value. +# D. Editing b's source does not re-run the root program. +# E. A dependency's program reads "" from graph_file(). +# F. `[package]` reports an unknown key the way `[build]` does: a warning, +# an error under --strict; `metadata` is a known key. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p "$TMP/b/src" "$TMP/a/src" "$TMP/app/src" + +write_b() { # write_b + cat > "$TMP/b/mcpp.toml" < "$TMP/b/src/b.cpp" + +cat > "$TMP/a/mcpp.toml" <<'EOF' +[package] +name = "a" +namespace = "spike" +version = "0.1.0" + +[dependencies] +spike.b = { path = "../b" } + +[targets.a] +kind = "lib" +EOF +printf 'int b_answer();\nint a_answer() { return b_answer() + 1; }\n' > "$TMP/a/src/a.cpp" +# The dependency's program: graph_file() is not offered to it. +cat > "$TMP/a/build.mcpp" <<'EOF' +#include +import mcpp; +int main() { + const char* g = mcpp::graph_file(); + if (g != nullptr && g[0] != '\0') { + std::fprintf(stderr, "a: a dependency's program was offered graph_file()=%s\n", g); + return 1; + } + return 0; +} +EOF + +cat > "$TMP/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +spike.a = { path = "../a" } +EOF +printf 'int a_answer();\nint main() { return a_answer() == 41 ? 0 : 1; }\n' > "$TMP/app/src/main.cpp" +# The root's program copies the document it was given, one copy per run. +cat > "$TMP/app/build.mcpp" <<'EOF' +#include +#include +#include +#include +import mcpp; +int main() { + const char* g = mcpp::graph_file(); + if (g == nullptr || g[0] == '\0') { + std::fprintf(stderr, "app: graph_file() is empty\n"); + return 1; + } + std::ifstream in(g, std::ios::binary); + std::stringstream ss; + ss << in.rdbuf(); + const std::string dir = mcpp::manifest_dir(); + int n = 0; + for (;; ++n) { + std::ifstream probe(dir + "/graph-" + std::to_string(n) + ".json"); + if (!probe) break; + } + std::ofstream(dir + "/graph-" + std::to_string(n) + ".json", std::ios::binary) << ss.str(); + return 0; +} +EOF + +cd "$TMP/app" +runs() { ls graph-*.json 2>/dev/null | wc -l | tr -d ' '; } + +# ── A, E ──────────────────────────────────────────────────────────────────── +"$MCPP" build > a.log 2>&1 || fail "A: build failed" a.log +[ "$(runs)" -eq 1 ] || fail "A: the root program ran $(runs) times" a.log +python3 - "$TMP" graph-0.json res <<'EOF' || fail "A: the graph document" graph-0.json +import json, os, sys +tmp, path, want = sys.argv[1], sys.argv[2], sys.argv[3] +d = json.load(open(path)) +names = [p["package"]["canonical"] for p in d["packages"]] +assert names == ["spike.b@0.2.0", "spike.a@0.1.0", "mcpplibs.app@0.1.0"], names +b, a, app = d["packages"] +assert app["root"] is True and b["root"] is False, (app["root"], b["root"]) +assert os.path.isabs(b["manifest_dir"]), b["manifest_dir"] +assert os.path.realpath(b["manifest_dir"]) == os.path.realpath(os.path.join(tmp, "b")), b["manifest_dir"] +assert b["targets"] == [{"name": "b", "kind": "lib"}], b["targets"] +assert b["metadata"] == {"demo": {"resources": want, "languages": ["en", "zh"]}}, b["metadata"] +assert a["metadata"] == {}, a["metadata"] +assert [r["requester"] for r in b["requested_by"]] == ["spike.a@0.1.0"], b["requested_by"] +assert isinstance(b["features"], list) +EOF +echo "ok: A, the document lists the graph in dependency order with metadata" +echo "ok: E, a dependency's program is not offered the document" + +# ── B ─────────────────────────────────────────────────────────────────────── +"$MCPP" build > b.log 2>&1 || fail "B: build failed" b.log +[ "$(runs)" -eq 1 ] || fail "B: an unchanged build re-ran the root program" b.log +echo "ok: B, an unchanged build does not re-run the root program" + +# ── C ─────────────────────────────────────────────────────────────────────── +write_b assets +"$MCPP" build > c.log 2>&1 || fail "C: build failed" c.log +[ "$(runs)" -eq 2 ] || fail "C: editing a dependency's metadata did not re-run the root program" c.log +python3 - "$TMP" graph-1.json assets <<'EOF' || fail "C: the document after the edit" graph-1.json +import json, sys +d = json.load(open(sys.argv[2])) +b = d["packages"][0] +assert b["metadata"]["demo"]["resources"] == sys.argv[3], b["metadata"] +EOF +echo "ok: C, a metadata edit re-runs the root program with the new value" + +# ── D ─────────────────────────────────────────────────────────────────────── +printf 'int b_answer() { return 39 + 1; }\n' > "$TMP/b/src/b.cpp" +"$MCPP" build > d.log 2>&1 || fail "D: build failed" d.log +[ "$(runs)" -eq 2 ] || fail "D: editing a dependency's source re-ran the root program" d.log +echo "ok: D, a source edit does not re-run the root program" + +# ── F ─────────────────────────────────────────────────────────────────────── +cat >> "$TMP/app/mcpp.toml" <<'EOF' +EOF +python3 - "$TMP/app/mcpp.toml" <<'EOF' +import sys +p = sys.argv[1] +s = open(p).read().replace('version = "0.1.0"\n', 'version = "0.1.0"\nlicence = "MIT"\n', 1) +open(p, "w").write(s) +EOF +"$MCPP" build > f.log 2>&1 || fail "F: build failed on a warning" f.log +grep -q "\[package\] has unsupported key 'licence'" f.log || fail "F: no warning for an unknown [package] key" f.log +if "$MCPP" build --strict > f2.log 2>&1; then fail "F: --strict accepted an unknown [package] key" f2.log; fi +grep -q "\[package\] has unsupported key 'licence'" f2.log || fail "F: --strict failed for another reason" f2.log +grep -q "unsupported key 'metadata'" a.log c.log f.log && fail "F: metadata reported as unknown" a.log +echo "ok: F, [package] reports an unknown key and knows metadata" + +echo "PASS: 720" diff --git a/tests/e2e/722_pack_strips_what_the_graph_built.sh b/tests/e2e/722_pack_strips_what_the_graph_built.sh new file mode 100755 index 00000000..97b14633 --- /dev/null +++ b/tests/e2e/722_pack_strips_what_the_graph_built.sh @@ -0,0 +1,210 @@ +#!/usr/bin/env bash +# requires: pack python3 +# 722 -- `mcpp pack` strips what the graph built on every row that strips, and +# says so only when it does (#649 E5). +# +# `strip_program` used to strip the staged PROGRAM and nothing else, on the ELF +# and PE paths only. A shared library the same graph compiled from source +# shipped its `.symtab` and `.debug_*`, and the Android row, which `run` +# dispatches before either path, stripped nothing at all while the `Packing` +# line said "stripped". A member that stages libraries of its own had no way to +# learn the decision, so `--no-strip` did not reach its files. +# +# Legs: +# A. Linux desktop, `bin` over a shared dependency: `bin/hostapp` and +# `lib/libdep.so` carry no `.symtab` and no `.debug_*`; `libdep.so` still +# exports `dep_answer` in `.dynsym`; the unpacked program runs. +# B. `--no-strip`: `lib/libdep.so` keeps its `.symtab`, and the `Packing` +# line does not say "stripped". +# C. `--debug-symbols DIR`: `DIR/libdep.so.debug` exists and the packed +# library names it through `.gnu_debuglink`. +# D. A dispatched format's build program reads `pack_strip()` = "1" and +# `pack_debug_symbols_dir()` = DIR, and "0" under `--no-strip`. +# E. When the NDK is installed: `--target x86_64-linux-android` gives +# `lib/libapp.so`, `lib/libdep.so` and `lib/libc++_shared.so` with no +# `.symtab`, and `.dynsym` still names `app_entry` and `dep_answer`. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +# Sections and dynamic symbols, read from the ELF itself. +elf_facts() { # elf_facts -> "symtab=N debug=N dynsym=" +python3 - "$1" <<'PY' +import struct, sys +d = open(sys.argv[1], 'rb').read() +assert d[:4] == b'\x7fELF' and d[4] == 2, "not ELF64" +shoff, = struct.unpack_from(' dep/mcpp.toml <<'EOF' +[package] +name = "dep" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[targets.dep] +kind = "shared" + +[build] +sources = ["src/*.cpp"] +EOF +cat > dep/src/dep.cpp <<'EOF' +#include +[[gnu::visibility("default")]] int dep_answer() { return static_cast(std::to_string(42).size()) + 40; } +EOF + +cat > hostapp/mcpp.toml <<'EOF' +[package] +name = "hostapp" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.dep = { path = "../dep" } + +[targets.hostapp] +kind = "bin" +main = "src/main.cpp" +EOF +printf 'int dep_answer();\nint main() { return dep_answer() == 42 ? 0 : 1; }\n' > hostapp/src/main.cpp +cd hostapp + +# ── A ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --format tar > a.log 2>&1 || fail "A: pack failed" a.log +grep -q "Packing hostapp v0.1.0 (vendored, stripped)" a.log || fail "A: the Packing line" a.log +rm -rf xa && mkdir xa && tar -xzf target/dist/hostapp-0.1.0-x86_64-linux-gnu.tar.gz -C xa +lib=$(ls xa/*/lib/libdep.so); prog=$(ls xa/*/bin/hostapp) +fa=$(elf_facts "$lib"); fp=$(elf_facts "$prog") +echo "reading A: libdep.so $fa" +echo "reading A: hostapp $fp" +case "$fa" in "symtab=0 debug=0 "*) ;; *) fail "A: lib/libdep.so is not stripped: $fa" ;; esac +case "$fa" in *dep_answer*) ;; *) fail "A: lib/libdep.so lost its export: $fa" ;; esac +case "$fp" in "symtab=0 debug=0 "*) ;; *) fail "A: bin/hostapp is not stripped: $fp" ;; esac +"$(ls -d xa/*)/hostapp" || fail "A: the unpacked program does not run" a.log +echo "ok: A, the program and the graph-built library are stripped and run" + +# ── B ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --format tar --no-strip > b.log 2>&1 || fail "B: pack failed" b.log +grep -q "Packing hostapp v0.1.0 (vendored)" b.log || fail "B: the Packing line" b.log +rm -rf xb && mkdir xb && tar -xzf target/dist/hostapp-0.1.0-x86_64-linux-gnu.tar.gz -C xb +fb=$(elf_facts "$(ls xb/*/lib/libdep.so)") +echo "reading B: libdep.so $fb" +case "$fb" in "symtab=1 "*) ;; *) fail "B: --no-strip did not reach lib/libdep.so: $fb" ;; esac +echo "ok: B, --no-strip reaches the graph-built library" + +# ── C ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --format tar --debug-symbols "$TMP/dbg" > c.log 2>&1 || fail "C: pack failed" c.log +[ -s "$TMP/dbg/libdep.so.debug" ] || fail "C: no libdep.so.debug" c.log +rm -rf xc && mkdir xc && tar -xzf target/dist/hostapp-0.1.0-x86_64-linux-gnu.tar.gz -C xc +fc=$(elf_facts "$(ls xc/*/lib/libdep.so)") +case "$fc" in *"debuglink=1"*) ;; *) fail "C: lib/libdep.so names no debug file: $fc" ;; esac +echo "ok: C, --debug-symbols separates the graph-built library's debug information" + +# ── D ─────────────────────────────────────────────────────────────────────── +# Written only now, so the legs above measure the strip itself on any engine. +# D's provider: records what a dispatched format's program is told. +cat > build.mcpp <<'EOF' +import mcpp; +#include +#include +#include +int main() { + mcpp::provides_pack_format("probe"); + if (std::string_view(mcpp::pack_format()) != "probe") return 0; + const std::string out = std::string(mcpp::out_dir()) + "/probe.txt"; + const std::string text = std::string("strip=") + mcpp::pack_strip() + + " debug=" + mcpp::pack_debug_symbols_dir(); + mcpp::action a; + a.id = "probe"; + a.role = "artifact"; + a.description = "probe"; + a.arg("/bin/sh").arg("-c").arg(("printf '%s' '" + text + "' > " + out).c_str()) + .input("${mcpp.target_file:hostapp}") + .output(out.c_str()) + .submit(); + return 0; +} +EOF + +"$MCPP" pack --format probe --debug-symbols "$TMP/dbg2" > d.log 2>&1 || fail "D: pack failed" d.log +probe=$(find target -name probe.txt | head -1) +[ "$(cat "$probe")" = "strip=1 debug=$TMP/dbg2" ] || fail "D: the program read '$(cat "$probe" 2>/dev/null)'" d.log +"$MCPP" pack --format probe --no-strip > d2.log 2>&1 || fail "D: pack --no-strip failed" d2.log +[ "$(cat "$probe")" = "strip=0 debug=" ] || fail "D: under --no-strip the program read '$(cat "$probe")'" d2.log +echo "ok: D, a dispatched format's program reads the strip decision" + +# ── E ─────────────────────────────────────────────────────────────────────── +ndk=$(ls -d "${MCPP_HOME:-$HOME/.mcpp}"/registry/data/xpkgs/xim-x-android-ndk/*/ 2>/dev/null | head -1 || true) +if [ -z "$ndk" ]; then + echo "skip: E, xim:android-ndk is not installed on this machine" +else + cd "$TMP" + cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.dep = { path = "../dep" } + +[targets.app] +kind = "app" +main = "src/main.cpp" + +[target.x86_64-linux-android] +min_api_level = 23 +EOF + cat > app/src/main.cpp <<'EOF' +#include +int dep_answer(); +[[gnu::visibility("default")]] int app_entry() { return dep_answer() + static_cast(std::to_string(1).size()); } +int main() { return app_entry() == 43 ? 0 : 1; } +EOF + cd app + "$MCPP" pack --target x86_64-linux-android --format tar > e.log 2>&1 || fail "E: pack failed" e.log + grep -q "Packing app v0.1.0 (vendored, stripped)" e.log || fail "E: the Packing line" e.log + rm -rf xe && mkdir xe && tar -xzf target/dist/app-0.1.0-x86_64-linux-android.tar.gz -C xe + for f in libapp.so libdep.so libc++_shared.so; do + fe=$(elf_facts "$(ls xe/*/lib/$f)") + echo "reading E: $f $(echo "$fe" | cut -d' ' -f1-3)" + case "$fe" in "symtab=0 debug=0 "*) ;; *) fail "E: lib/$f is not stripped: $fe" e.log ;; esac + done + case "$(elf_facts "$(ls xe/*/lib/libapp.so)")" in *app_entry*) ;; *) fail "E: libapp.so lost app_entry" ;; esac + case "$(elf_facts "$(ls xe/*/lib/libdep.so)")" in *dep_answer*) ;; *) fail "E: libdep.so lost dep_answer" ;; esac + echo "ok: E, the Android row strips the program, the graph's library and the NDK runtime" +fi + +echo "PASS: 722" diff --git a/tests/e2e/723_pack_reports_what_it_produced.sh b/tests/e2e/723_pack_reports_what_it_produced.sh new file mode 100755 index 00000000..dd3bba3a --- /dev/null +++ b/tests/e2e/723_pack_reports_what_it_produced.sh @@ -0,0 +1,118 @@ +#!/usr/bin/env bash +# requires: pack python3 +# 723 -- `mcpp pack --message-format json` reports what it produced, and `pack` +# takes the profile shorthands `build` and `run` take, with one precedence +# (#649 E9). +# +# A driver that publishes a pack's artifacts used to parse the human `Packed` +# lines, whose paths are shortened (`@mcpp/`, `~/`, project-relative), and a +# driver mapping one profile switch onto three verbs found `--release` refused +# on the third. `mcpp run --profile dev --release` built `release` while +# `mcpp build` given the same line built `dev`. +# +# Legs: +# A. `pack --format tar --message-format json`: stdout is one `mcpp.pack` +# envelope and nothing else; the artifact path is absolute and exists, its +# type is `file`, its format `tar`, its target the host triple; the staged +# tree and its manifest exist, and the closure is `walked`. +# B. `pack --format dir --message-format json`: the artifact is a directory. +# C. `pack --release` and `pack --dev` are accepted; `--dev` builds `dev`. +# D. `run --profile dev --release` builds `dev`, as `build` does. +# E. `pack --message-format yaml` is refused with exit 2. +# F. `--protocol-version` lists the `mcpp.pack` kind and the `pack` command. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p app/src +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.3.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +printf '#include \nint main() { std::puts("app ran"); return 0; }\n' > app/src/main.cpp +cd app + +# ── A ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --format tar --message-format json > a.json 2> a.err || fail "A: pack failed" a.err a.json +python3 - a.json <<'EOF' || fail "A: the envelope" a.json a.err +import json, os, sys +text = open(sys.argv[1]).read() +e = json.loads(text) # the whole of stdout is one document +assert e["kind"] == "mcpp.pack" and e["kindVersion"] == 1, (e["kind"], e.get("kindVersion")) +assert "write-project" in e["effects"], e["effects"] +arts = e["data"]["artifacts"] +assert len(arts) == 1, arts +a = arts[0] +assert os.path.isabs(a["path"]) and os.path.isfile(a["path"]), a +assert a["path"].endswith(".tar.gz") and a["type"] == "file" and a["format"] == "tar", a +assert len(a["targets"]) == 1 and a["targets"][0].endswith("linux-gnu"), a["targets"] +st = e["data"]["stage"] +assert os.path.isdir(st["dir"]) and os.path.isfile(st["manifest"]), st +assert st["closure"] == "walked", st +assert e["diagnostics"] == [], e["diagnostics"] +EOF +grep -q "Packed" a.err || fail "A: the human lines did not go to stderr" a.err +echo "ok: A, one envelope on stdout names the archive, its legs and the staged tree" + +# ── B ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --format dir --message-format json > b.json 2> b.err || fail "B: pack failed" b.err +python3 - b.json <<'EOF' || fail "B: the envelope" b.json +import json, os, sys +a = json.load(open(sys.argv[1]))["data"]["artifacts"][0] +assert a["type"] == "directory" and os.path.isdir(a["path"]) and a["format"] == "dir", a +EOF +echo "ok: B, a directory artifact is reported as one" + +# ── C ─────────────────────────────────────────────────────────────────────── +# `pack` narrates no `Finished` line, so the profile is read from the graph the +# pack built: `dev` compiles at `-O0`, `release` does not. Each sub-leg starts +# from an empty `target/`, so the one graph there is the one this pack wrote. +built_dev() { grep -q -- " -O0" "$(ls target/*-linux-gnu/*/build.ninja | head -1)"; } +rm -rf target +"$MCPP" pack --release --format tar > c1.log 2>&1 || fail "C: pack --release was refused" c1.log +built_dev && fail "C: pack --release built dev" c1.log +rm -rf target +"$MCPP" pack --dev --format tar > c2.log 2>&1 || fail "C: pack --dev was refused" c2.log +built_dev || fail "C: pack --dev did not build dev" c2.log +rm -rf target +"$MCPP" pack --profile release --dev --format tar > c3.log 2>&1 || fail "C: pack --profile --dev failed" c3.log +built_dev && fail "C: --dev won over --profile release on pack" c3.log +echo "ok: C, pack takes --release and --dev, and --profile wins" + +# ── D ─────────────────────────────────────────────────────────────────────── +"$MCPP" run --profile dev --release > d.log 2>&1 || fail "D: run failed" d.log +grep -q "Finished dev" d.log || fail "D: run --profile dev --release did not build dev" d.log +"$MCPP" build --profile dev --release > d2.log 2>&1 || fail "D: build failed" d2.log +grep -q "Finished dev" d2.log || fail "D: build --profile dev --release did not build dev" d2.log +echo "ok: D, run and build resolve --profile over the shorthands alike" + +# ── E ─────────────────────────────────────────────────────────────────────── +set +e +"$MCPP" pack --format tar --message-format yaml > e.out 2> e.err +rc=$? +set -e +[ "$rc" -eq 2 ] || fail "E: --message-format yaml exited $rc" e.err +grep -q "unknown --message-format 'yaml'" e.err || fail "E: the refusal does not name the value" e.err +echo "ok: E, an unknown message format is refused" + +# ── F ─────────────────────────────────────────────────────────────────────── +"$MCPP" --protocol-version > f.json +python3 - f.json <<'EOF' || fail "F: the protocol document" f.json +import json, sys +p = json.load(open(sys.argv[1])) +assert p["kinds"]["mcpp.pack"] == 1, p["kinds"] +fx = p["commands"]["pack"]["effects"] +assert "write-project" in fx and "exec-build-script" in fx, fx +EOF +echo "ok: F, the protocol document declares the kind and the command" + +echo "PASS: 723" diff --git a/tests/unit/test_build_profile.cpp b/tests/unit/test_build_profile.cpp index df1cc3b8..18c306e2 100644 --- a/tests/unit/test_build_profile.cpp +++ b/tests/unit/test_build_profile.cpp @@ -170,3 +170,18 @@ TEST(BuildProfile, UnknownManifestCacheModeFallsThroughToDefault) { EXPECT_EQ(mcpp::build::resolve_cache_mode(m, "bogus"), mcpp::build::CacheMode::Local); } + +// ── #649 E9: the override the command line states, one rule for every verb ─── +// +// `build` preferred `--profile` over the shorthands and `run` preferred the +// shorthands, so one command line built two profiles. Every verb now reads this. +TEST(BuildProfile, ProfileOptionBeatsTheShorthands) { + EXPECT_EQ(mcpp::build::profile_override_from_flags("dev", true, false), "dev"); + EXPECT_EQ(mcpp::build::profile_override_from_flags("release", false, true), "release"); +} + +TEST(BuildProfile, ShorthandsNameTheirProfile) { + EXPECT_EQ(mcpp::build::profile_override_from_flags("", true, false), "release"); + EXPECT_EQ(mcpp::build::profile_override_from_flags("", false, true), "dev"); + EXPECT_EQ(mcpp::build::profile_override_from_flags("", false, false), ""); +} From f7adbcfd11ba0a603e2dbceba5dc1ec7c626ddaa Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:35:18 +0800 Subject: [PATCH 15/20] The default mcpplibs artifact is a GLOBAL/CN region object, and existing homes gain the CN half (#648 L7) mcpplibs/mcpp-index#432 made the GitCode copy of the index artifact the same bytes as the GitHub copy. With mirror = CN, mcpp index update now reaches only raw.gitcode.com, gitcode.com and file-cdn.gitcode.com (traced). e2e 151 covers the fresh seed, the flat-to-region upgrade and its idempotence, and a user base that keeps its own value. --- src/config.cppm | 49 ++++++++++++++++--- src/fallback/config_migration.cppm | 21 ++++++-- src/xlings/xlings.cppm | 9 +++- tests/e2e/151_index_url_artifact_migration.sh | 39 +++++++++++++-- 4 files changed, 102 insertions(+), 16 deletions(-) diff --git a/src/config.cppm b/src/config.cppm index bd86d2bd..7d370d75 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -40,6 +40,10 @@ struct IndexRepo { std::string name; std::string url; std::string artifact; // optional artifact source base (xlings >= 0.4.68, #269) + // The CN base of the same artifact. Non-empty makes `artifact` the GLOBAL + // half of a region object, which xlings resolves against its mirror + // setting (xlings #377): `{"GLOBAL": artifact, "CN": artifactCn}`. + std::string artifactCn; std::string source; // optional "auto" | "artifact" | "git" ("" = xlings default auto) // Read from a `[index.repos.]` table rather than added by mcpp as a // default. The provisioning line names an index that such a table @@ -58,6 +62,14 @@ inline constexpr std::string_view kMcpplibsIndexUrlLegacy = "https://github.com/mcpp-community/mcpp-index.git"; inline constexpr std::string_view kMcpplibsIndexArtifact = "https://github.com/xlings-res/mcpp-index"; +// The GitCode mirror of the same artifact, published by mcpp-index's +// `tools/publish_mcpp_index.sh` with the same bytes as the GitHub copy +// (mcpplibs/mcpp-index#432 made that a property the script checks). A CN +// machine used to fetch the index from GitHub even with `mirror = CN`, and a +// connection that never answered held a background plan for eleven minutes +// (#648 A6). +inline constexpr std::string_view kMcpplibsIndexArtifactCn = + "https://gitcode.com/xlings-res/mcpp-index"; struct GlobalConfig { // Resolved paths @@ -358,7 +370,7 @@ refresh_timeout = 120 [index.repos."mcpplibs"] url = "https://github.com/mcpplibs/mcpp-index.git" -artifact = "https://github.com/xlings-res/mcpp-index" +artifact = { GLOBAL = "https://github.com/xlings-res/mcpp-index", CN = "https://gitcode.com/xlings-res/mcpp-index" } # source = "auto" # default: artifact first, git fallback; set "git" to force git # xlings auto-adds xim / awesome / scode / d2x as defaults. @@ -382,7 +394,7 @@ bool write_default_xlings_json(const std::filesystem::path& path, std::vector pairs; pairs.reserve(repos.size()); for (auto& r : repos) - pairs.push_back({ r.name, r.url, r.artifact, r.source }); + pairs.push_back({ r.name, r.url, r.artifact, r.source, r.artifactCn }); // seed_xlings_json writes to env.home / ".xlings.json", so we // construct a temporary Env with home = path.parent_path(). mcpp::xlings::Env env; @@ -446,7 +458,10 @@ bool write_json_file(const std::filesystem::path& path, const nlohmann::json& do nlohmann::json index_repo_entry(const IndexRepo& r) { nlohmann::json e = { {"name", r.name}, {"url", r.url} }; - if (!r.artifact.empty()) e["artifact"] = r.artifact; + if (!r.artifact.empty() && !r.artifactCn.empty()) + e["artifact"] = { {"GLOBAL", r.artifact}, {"CN", r.artifactCn} }; + else if (!r.artifact.empty()) + e["artifact"] = r.artifact; if (!r.source.empty()) e["source"] = r.source; return e; } @@ -606,6 +621,12 @@ void canonicalize_legacy_index_names(GlobalConfig& cfg) { // explicit opt-out; a user-set artifact base always wins. if (r.url == kMcpplibsIndexUrl && r.artifact.empty() && r.source != "git") r.artifact = std::string(kMcpplibsIndexArtifact); + // The CN half of the default (#648 A6), for a configuration written + // before it existed. Only when the GLOBAL base is the default one: a + // user's own artifact base has no mirror mcpp knows of. + if (r.url == kMcpplibsIndexUrl && r.artifact == kMcpplibsIndexArtifact + && r.artifactCn.empty()) + r.artifactCn = std::string(kMcpplibsIndexArtifactCn); bool duplicate = std::any_of(normalized.begin(), normalized.end(), [&](const IndexRepo& existing) { return existing.name == r.name && existing.url == r.url; @@ -705,8 +726,17 @@ std::expected load_or_init( if (it == tt.end() || !it->second.is_string()) continue; IndexRepo r{ name, it->second.as_string() }; r.fromConfig = true; - if (auto a = tt.find("artifact"); a != tt.end() && a->second.is_string()) + if (auto a = tt.find("artifact"); a != tt.end() && a->second.is_string()) { r.artifact = a->second.as_string(); + } else if (a != tt.end() && a->second.is_table()) { + // The region form xlings reads: GLOBAL is the base every + // mirror falls back to, CN the base a CN mirror asks first. + auto& region = a->second.as_table(); + if (auto g = region.find("GLOBAL"); g != region.end() && g->second.is_string()) + r.artifact = g->second.as_string(); + if (auto c = region.find("CN"); c != region.end() && c->second.is_string()) + r.artifactCn = c->second.as_string(); + } if (auto s = tt.find("source"); s != tt.end() && s->second.is_string()) r.source = s->second.as_string(); cfg.indexRepos.push_back(std::move(r)); @@ -742,12 +772,15 @@ std::expected load_or_init( // dependency resolution (e.g. linux-headers existing in both // scode and xim). See docs/21 §VII. auto add_default = [&](std::string_view name, std::string_view url, - std::string_view artifact = {}) { + std::string_view artifact = {}, + std::string_view artifactCn = {}) { for (auto& r : cfg.indexRepos) if (r.name == name) return; - cfg.indexRepos.push_back({ std::string(name), std::string(url), - std::string(artifact), std::string() }); + IndexRepo r{ std::string(name), std::string(url), std::string(artifact) }; + r.artifactCn = std::string(artifactCn); + cfg.indexRepos.push_back(std::move(r)); }; - add_default("mcpplibs", kMcpplibsIndexUrl, kMcpplibsIndexArtifact); + add_default("mcpplibs", kMcpplibsIndexUrl, kMcpplibsIndexArtifact, + kMcpplibsIndexArtifactCn); canonicalize_legacy_index_names(cfg); // 5. Seed registry/.xlings.json if missing; migrate legacy cached diff --git a/src/fallback/config_migration.cppm b/src/fallback/config_migration.cppm index 56e75872..3732fe43 100644 --- a/src/fallback/config_migration.cppm +++ b/src/fallback/config_migration.cppm @@ -94,13 +94,28 @@ bool migrate_xlings_json_index_names(const std::filesystem::path& path) { // means a previous run (or a fresh seed) already declared it — a plain // replace_all would re-inject on every run. Both spacing variants // because the file has two writers (mcpp pretty / xlings compact). + constexpr std::string_view region = + "{ \"GLOBAL\": \"https://github.com/xlings-res/mcpp-index\", " + "\"CN\": \"https://gitcode.com/xlings-res/mcpp-index\" }"; if (updated.find("xlings-res/mcpp-index") == std::string::npos) { - constexpr std::string_view art = - ", \"artifact\": \"https://github.com/xlings-res/mcpp-index\""; + const std::string art = std::string(", \"artifact\": ") + std::string(region); for (std::string_view urlkv : { "\"url\": \"https://github.com/mcpplibs/mcpp-index.git\"", "\"url\":\"https://github.com/mcpplibs/mcpp-index.git\"" }) - replace_all(updated, urlkv, std::string(urlkv) + std::string(art)); + replace_all(updated, urlkv, std::string(urlkv) + art); + } + + // The CN half of the default artifact (#648 A6). An existing home keeps the + // GitHub-only base it was seeded with, and its default entry is not + // reconciled from configuration, so this is the channel that gives it the + // GitCode mirror. Only the exact default value is rewritten: a base the user + // wrote is theirs. Idempotent, because the rewritten value no longer + // contains the flat spelling. Both spacings, for the two writers. + if (updated.find("gitcode.com/xlings-res/mcpp-index") == std::string::npos) { + for (std::string_view flat : { + "\"artifact\": \"https://github.com/xlings-res/mcpp-index\"", + "\"artifact\":\"https://github.com/xlings-res/mcpp-index\"" }) + replace_all(updated, flat, std::string("\"artifact\": ") + std::string(region)); } return write_text_if_changed(path, original, updated); diff --git a/src/xlings/xlings.cppm b/src/xlings/xlings.cppm index ab1613e0..8089bb87 100644 --- a/src/xlings/xlings.cppm +++ b/src/xlings/xlings.cppm @@ -470,6 +470,9 @@ struct SeedRepo { std::string url; std::string artifact; // artifact source base, e.g. https://github.com/xlings-res/mcpp-index std::string source; // "auto" | "artifact" | "git" + // Non-empty: `artifact` is written as the region object + // {"GLOBAL": artifact, "CN": artifactCn} (xlings #377). + std::string artifactCn; }; void seed_xlings_json(const Env& env, @@ -1718,7 +1721,11 @@ void seed_xlings_json(const Env& env, json += std::format(" {{ \"name\": \"{}\", \"url\": \"{}\"", json_escape(repos[i].name), json_escape(repos[i].url)); - if (!repos[i].artifact.empty()) + if (!repos[i].artifact.empty() && !repos[i].artifactCn.empty()) + json += std::format(", \"artifact\": {{ \"GLOBAL\": \"{}\", \"CN\": \"{}\" }}", + json_escape(repos[i].artifact), + json_escape(repos[i].artifactCn)); + else if (!repos[i].artifact.empty()) json += std::format(", \"artifact\": \"{}\"", json_escape(repos[i].artifact)); if (!repos[i].source.empty()) diff --git a/tests/e2e/151_index_url_artifact_migration.sh b/tests/e2e/151_index_url_artifact_migration.sh index 1ad8c745..21c9728a 100755 --- a/tests/e2e/151_index_url_artifact_migration.sh +++ b/tests/e2e/151_index_url_artifact_migration.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# requires: python3 # #267/#269: fresh init seeds the mcpplibs-org index URL + artifact source; # legacy config.toml / .xlings.json (old org URL, no artifact) are healed in # place, idempotently, in both pretty (mcpp writer) and compact (xlings @@ -13,6 +14,10 @@ export MCPP_HOME="$TMP/mcpp-home" NEW_URL='https://github.com/mcpplibs/mcpp-index.git' OLD_URL='https://github.com/mcpp-community/mcpp-index.git' ART='https://github.com/xlings-res/mcpp-index' +ART_CN='https://gitcode.com/xlings-res/mcpp-index' +# The default artifact is a region object since 2026.9.16.1 (#648 A6): the +# GitHub base for every mirror and the GitCode base a CN mirror asks first. +REGION="{ \"GLOBAL\": \"$ART\", \"CN\": \"$ART_CN\" }" CFG="$MCPP_HOME/config.toml" XJ="$MCPP_HOME/registry/.xlings.json" @@ -21,11 +26,16 @@ XJ="$MCPP_HOME/registry/.xlings.json" # assert on the key/value pairs, not on mcpp's seed line layout.) "$MCPP" self env > /dev/null grep -q "url = \"$NEW_URL\"" "$CFG" || { echo "config.toml missing new url"; exit 1; } -grep -q "artifact = \"$ART\"" "$CFG" || { echo "config.toml missing artifact"; exit 1; } +grep -qF "artifact = { GLOBAL = \"$ART\", CN = \"$ART_CN\" }" "$CFG" \ + || { echo "config.toml missing the region artifact"; cat "$CFG"; exit 1; } grep -q "\"url\": \"$NEW_URL\"" "$XJ" \ || { echo "seeded .xlings.json missing new url"; cat "$XJ"; exit 1; } -grep -q "\"artifact\": \"$ART\"" "$XJ" \ - || { echo "seeded .xlings.json missing artifact"; cat "$XJ"; exit 1; } +python3 - "$XJ" "$ART" "$ART_CN" <<'PYEOF' || { echo "seeded .xlings.json missing the region artifact"; cat "$XJ"; exit 1; } +import json, sys +doc = json.load(open(sys.argv[1])) +entry = next(r for r in doc["index_repos"] if r["name"] == "mcpplibs") +assert entry["artifact"] == {"GLOBAL": sys.argv[2], "CN": sys.argv[3]}, entry +PYEOF if grep -q 'mcpp-community/mcpp-index' "$CFG" "$XJ"; then echo "old org URL leaked into fresh seed"; exit 1 fi @@ -54,7 +64,7 @@ cat > "$XJ" < /dev/null grep -q "url = \"$NEW_URL\"" "$CFG" || { echo "config.toml url not healed"; cat "$CFG"; exit 1; } -grep -q "\"url\": \"$NEW_URL\", \"artifact\": \"$ART\"" "$XJ" \ +grep -qF "\"url\": \"$NEW_URL\", \"artifact\": $REGION" "$XJ" \ || { echo ".xlings.json not healed"; cat "$XJ"; exit 1; } grep -q '"subos": "default"' "$XJ" || { echo "unrelated .xlings.json key lost"; exit 1; } if grep -q 'mcpp-community/mcpp-index' "$CFG" "$XJ"; then @@ -72,6 +82,27 @@ printf '{"index_repos":[{"name":"mcpp-index","url":"%s"}],"mirror":"auto"}' "$OL "$MCPP" self env > /dev/null grep -q '"name":"mcpplibs"' "$XJ" || { echo "compact name not healed"; cat "$XJ"; exit 1; } grep -q 'xlings-res/mcpp-index' "$XJ" || { echo "compact artifact not injected"; cat "$XJ"; exit 1; } +grep -q 'gitcode.com/xlings-res/mcpp-index' "$XJ" || { echo "compact artifact has no CN half"; cat "$XJ"; exit 1; } + +# 4. A home seeded before the region object (flat GitHub base, both spacings) +# gains the CN half, once; a base the user wrote is left alone. +printf '{\n "index_repos": [\n { "name": "mcpplibs", "url": "%s", "artifact": "%s" }\n ],\n "mirror": "CN"\n}\n' "$NEW_URL" "$ART" > "$XJ" +"$MCPP" self env > /dev/null +grep -qF "\"artifact\": $REGION" "$XJ" || { echo "flat default artifact not upgraded"; cat "$XJ"; exit 1; } +cp "$XJ" "$TMP/xj4"; "$MCPP" self env > /dev/null +cmp -s "$XJ" "$TMP/xj4" || { echo "region upgrade not idempotent"; diff "$TMP/xj4" "$XJ"; exit 1; } +# A base the user states in config.toml owns the entry, and has no CN half. +cat > "$CFG" < /dev/null +grep -q 'https://mirror.example/idx' "$XJ" || { echo "a user artifact base was rewritten"; cat "$XJ"; exit 1; } +if grep -q 'gitcode.com/xlings-res/mcpp-index' "$XJ"; then echo "a user artifact base gained a CN half"; cat "$XJ"; exit 1; fi if grep -q 'mcpp-community' "$XJ"; then echo "compact old org URL survived"; exit 1 fi From 66b29cd69b64d12f78eaca084b79e3c436c6b8ef Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:37:08 +0800 Subject: [PATCH 16/20] Name the PE host link shape by its format WindowsLld spelled the vocabulary the runtime-contract source scan reserves for provider selection (RuntimeContract.SourceOwnsNoProviderSpecificSelectionOrProbeBranch). --- src/build/flags.cppm | 8 ++++---- tests/unit/test_build_flags.cpp | 6 +++--- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 2a5f9136..353fe7d5 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -163,7 +163,7 @@ CompileFlags compute_flags(const BuildPlan& plan); // (an SDK such as the NDK, or a retargetable clang) is exactly the case the // host line cannot serve. enum class LinkHost { Linux, MacOS, Windows }; -enum class LinkShape { MsvcLinkExe, WindowsLld, AppleSdk, Generic }; +enum class LinkShape { MsvcLinkExe, PeLld, AppleSdk, Generic }; LinkShape link_shape(LinkHost host, mcpp::build::dist::Format targetFormat, bool msvcDialect, bool targetNamedByFlag); @@ -466,8 +466,8 @@ LinkShape link_shape(LinkHost host, mcpp::build::dist::Format targetFormat, switch (host) { case LinkHost::Windows: if (msvcDialect) return LinkShape::MsvcLinkExe; - if (targetFormat == Format::Pe) return LinkShape::WindowsLld; - return targetNamedByFlag ? LinkShape::Generic : LinkShape::WindowsLld; + if (targetFormat == Format::Pe) return LinkShape::PeLld; + return targetNamedByFlag ? LinkShape::Generic : LinkShape::PeLld; case LinkHost::MacOS: return targetFormat == Format::MachO ? LinkShape::AppleSdk : LinkShape::Generic; @@ -1636,7 +1636,7 @@ CompileFlags compute_flags(const BuildPlan& plan) { const LinkShape linkShape = link_shape(current_link_host(), linkTargetFormat, isMsvcDialect, !plan.toolchain.crossTargetFlag.empty()); - if (linkShape == LinkShape::MsvcLinkExe || linkShape == LinkShape::WindowsLld) { + if (linkShape == LinkShape::MsvcLinkExe || linkShape == LinkShape::PeLld) { if (linkShape == LinkShape::MsvcLinkExe) { // Native cl.exe: link.exe does the link (SeparateLinker). Search // paths for dependency runtime import libs via /LIBPATH; user diff --git a/tests/unit/test_build_flags.cpp b/tests/unit/test_build_flags.cpp index d96e2fce..dbc088b4 100644 --- a/tests/unit/test_build_flags.cpp +++ b/tests/unit/test_build_flags.cpp @@ -200,11 +200,11 @@ TEST(LinkShape, MacOSHostKeepsTheAppleLineForMachOOnly) { TEST(LinkShape, WindowsHostSeparatesPeFromTargetsNamedByFlag) { EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Pe, true, false), LinkShape::MsvcLinkExe); - EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Pe, false, true), LinkShape::WindowsLld); - EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Pe, false, false), LinkShape::WindowsLld); + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Pe, false, true), LinkShape::PeLld); + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Pe, false, false), LinkShape::PeLld); // An SDK or retargetable clang aimed at an ELF row: named by `--target`. EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Elf, false, true), LinkShape::Generic); // The canadian GCC cross to x86_64-linux-musl names its target by prefix // and keeps the line its CI job verifies. - EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Elf, false, false), LinkShape::WindowsLld); + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Elf, false, false), LinkShape::PeLld); } From 759ec6856265615fa76446ba750497733bd14024 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:41:18 +0800 Subject: [PATCH 17/20] CI: e2e 700 on the hermetic llvm job, measurement readings in the macOS and Windows job summaries, 721 as its own macOS step; migration unit tests for the region artifact --- .github/workflows/ci-linux-e2e.yml | 8 ++++++- .github/workflows/ci-macos-e2e.yml | 35 ++++++++++++++++++++++++++-- .github/workflows/ci-windows-e2e.yml | 16 ++++++++++++- tests/unit/test_config.cpp | 33 ++++++++++++++++++++++---- 4 files changed, 84 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 62dfb5fe..870b3be1 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -422,7 +422,7 @@ jobs: # skips. They run here, where llvm is installed, and each is held to the # line it prints only when it ran to the end, and to the line of the step # that runs a program (#641). - - name: "graph C++ runtime e2e (663, 690, 696)" + - name: "C++ runtime e2e that needs llvm (663, 690, 696, 700)" run: | set -o pipefail export PATH="$HOME/.xlings/subos/current/bin:$PATH" @@ -443,3 +443,9 @@ jobs: "PASS: 690 a shared library over a graph C++ runtime is refused or carries a stated private copy" run_and_assert tests/e2e/696_a_cxx_layer_provider_keeps_its_own_standard.sh \ "PASS: a C++-layer provider compiles its implementation units at its own standard" + # #646 F3a: the default llvm shape of a program over a C++ shared library + # aborted with std::bad_cast before its programs took the library's contract. + run_and_assert tests/e2e/700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh \ + "ok: a program over a C++ shared library runs on one C++ runtime" \ + "ok: a stated self-contained program over a coupled C++ shared library is refused" \ + "PASS: 700 a program over a C++ shared library has one C++ runtime" diff --git a/.github/workflows/ci-macos-e2e.yml b/.github/workflows/ci-macos-e2e.yml index 7a47e850..80a0e0a4 100644 --- a/.github/workflows/ci-macos-e2e.yml +++ b/.github/workflows/ci-macos-e2e.yml @@ -24,7 +24,7 @@ jobs: e2e: name: e2e suite (macOS ARM64, self-host) runs-on: macos-15 - timeout-minutes: 30 + timeout-minutes: 60 # NOTE: no MCPP_VERBOSE — the e2e suite asserts mcpp's default quiet # output (tests 48/53). steps: @@ -64,4 +64,35 @@ jobs: "$MCPP" self config # macOS default toolchain is LLVM "$MCPP" toolchain default "llvm@${MCPP_LLVM_VER}" - bash tests/e2e/run_all.sh + echo "MCPP=$MCPP" >> "$GITHUB_ENV" + set -o pipefail + bash tests/e2e/run_all.sh 2>&1 | tee "$RUNNER_TEMP/e2e-suite.log" + + # Measurement legs print READING lines and pass whatever they read; the + # readings are what a decision is taken from (#646 F2: whether a C++ + # exception thrown in a dylib is caught by its class under the payload's + # default runtime), so they are collected where a reader finds them. + - name: Measurement readings + if: always() + shell: bash + run: | + { + echo "### Measurement readings (macOS)" + echo '```' + grep -h '^READING' "$RUNNER_TEMP/e2e-suite.log" 2>/dev/null || echo "(none)" + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + + # #647 E3: an Android row links on a macOS host. Its own step, because the + # NDK is a large download the suite's per-test bound does not allow for, + # and the shard does not carry it (the script declares android-ndk). + - name: "Android row on a macOS host (721)" + timeout-minutes: 30 + shell: bash + run: | + set -o pipefail + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL + "$MCPP" toolchain install android-ndk@30.0.16248370 + bash tests/e2e/721_*.sh 2>&1 | tee "$RUNNER_TEMP/721.log" + grep -q '^PASS: 721' "$RUNNER_TEMP/721.log" diff --git a/.github/workflows/ci-windows-e2e.yml b/.github/workflows/ci-windows-e2e.yml index cdd48e33..da100042 100644 --- a/.github/workflows/ci-windows-e2e.yml +++ b/.github/workflows/ci-windows-e2e.yml @@ -107,4 +107,18 @@ jobs: export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL "$MCPP_SELF" self config --mirror GLOBAL "$MCPP_SELF" toolchain default llvm@20.1.7 - bash tests/e2e/run_all.sh + set -o pipefail + bash tests/e2e/run_all.sh 2>&1 | tee "$RUNNER_TEMP/e2e-suite.log" + + # Measurement legs print READING lines (#646 F2 across PE images, #649 E10 + # the llvm row's recorded CRT); collected where a reader finds them. + - name: Measurement readings + if: always() + shell: bash + run: | + { + echo "### Measurement readings (Windows, shard ${{ matrix.shard }})" + echo '```' + grep -h '^READING' "$RUNNER_TEMP/e2e-suite.log" 2>/dev/null || echo "(none)" + echo '```' + } >> "$GITHUB_STEP_SUMMARY" diff --git a/tests/unit/test_config.cpp b/tests/unit/test_config.cpp index f3d1f2d1..e036fbf7 100644 --- a/tests/unit/test_config.cpp +++ b/tests/unit/test_config.cpp @@ -265,7 +265,8 @@ TEST(ConfigIndexMigration, XlingsJsonHealsPrettyLegacyEntry) { auto text = read_all(p); EXPECT_NE(text.find( "\"url\": \"https://github.com/mcpplibs/mcpp-index.git\", " - "\"artifact\": \"https://github.com/xlings-res/mcpp-index\""), + "\"artifact\": { \"GLOBAL\": \"https://github.com/xlings-res/mcpp-index\", " + "\"CN\": \"https://gitcode.com/xlings-res/mcpp-index\" }"), std::string::npos) << text; EXPECT_NE(text.find("\"subos\": \"default\""), std::string::npos); EXPECT_EQ(text.find("mcpp-community/mcpp-index"), std::string::npos); @@ -288,20 +289,44 @@ TEST(ConfigIndexMigration, XlingsJsonHealsCompactLegacyNameAndUrl) { EXPECT_NE(text.find("\"name\":\"mcpplibs\""), std::string::npos); EXPECT_NE(text.find( "\"url\":\"https://github.com/mcpplibs/mcpp-index.git\", " - "\"artifact\": \"https://github.com/xlings-res/mcpp-index\""), + "\"artifact\": { \"GLOBAL\": \"https://github.com/xlings-res/mcpp-index\", " + "\"CN\": \"https://gitcode.com/xlings-res/mcpp-index\" }"), std::string::npos) << text; EXPECT_FALSE(mcpp::fallback::migrate_xlings_json_index_names(p)); std::filesystem::remove_all(dir); } -TEST(ConfigIndexMigration, XlingsJsonSkipsInjectionWhenArtifactPresent) { +// #648 A6. A home seeded with the GitHub-only default artifact gains the CN half +// of the region object once; the migration is idempotent afterwards. +TEST(ConfigIndexMigration, XlingsJsonUpgradesTheFlatDefaultArtifactToTheRegionObject) { auto dir = make_tempdir("mcpp-migrate-xja"); auto p = dir / ".xlings.json"; + { + std::ofstream os(p); + os << "{\n \"index_repos\": [\n" + " { \"name\": \"mcpplibs\", \"url\": " + "\"https://github.com/mcpplibs/mcpp-index.git\", " + "\"artifact\": \"https://github.com/xlings-res/mcpp-index\" }\n" + " ]\n}\n"; + } + EXPECT_TRUE(mcpp::fallback::migrate_xlings_json_index_names(p)); + auto text = read_all(p); + EXPECT_NE(text.find("\"artifact\": { \"GLOBAL\": \"https://github.com/xlings-res/mcpp-index\", " + "\"CN\": \"https://gitcode.com/xlings-res/mcpp-index\" }"), std::string::npos) << text; + EXPECT_FALSE(mcpp::fallback::migrate_xlings_json_index_names(p)); + EXPECT_EQ(read_all(p), text); + std::filesystem::remove_all(dir); +} + +// An artifact base the user wrote is theirs: no CN half is added to it. +TEST(ConfigIndexMigration, XlingsJsonLeavesAUserArtifactBaseAlone) { + auto dir = make_tempdir("mcpp-migrate-xju"); + auto p = dir / ".xlings.json"; std::string body = "{\n \"index_repos\": [\n" " { \"name\": \"mcpplibs\", \"url\": " "\"https://github.com/mcpplibs/mcpp-index.git\", " - "\"artifact\": \"https://github.com/xlings-res/mcpp-index\" }\n" + "\"artifact\": \"https://mirror.example/xlings-res/mcpp-index\" }\n" " ]\n}\n"; { std::ofstream os(p); From 818152338a6c5ac272ff4db35d2652c5b5f0fa7c Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 06:47:41 +0800 Subject: [PATCH 18/20] One derivation of the target object format for the contract table and the link shape; version 2026.9.16.1, CHANGELOG, records updated with the implementation's readings --- .../2026-09-16-646-649-four-issues-by-home.md | 3 +- .../2026-09-16-646-649-implementation-plan.md | 126 +++++++++++++----- CHANGELOG.md | 88 ++++++++++++ docs/50-machine-output.md | 4 +- docs/zh/50-machine-output.md | 4 +- mcpp.toml | 2 +- modules/versioning/src/version.cppm | 2 +- src/build/flags.cppm | 35 +++-- 8 files changed, 205 insertions(+), 59 deletions(-) diff --git a/.agents/docs/2026-09-16-646-649-four-issues-by-home.md b/.agents/docs/2026-09-16-646-649-four-issues-by-home.md index d0a3e512..6c8795f8 100644 --- a/.agents/docs/2026-09-16-646-649-four-issues-by-home.md +++ b/.agents/docs/2026-09-16-646-649-four-issues-by-home.md @@ -5,7 +5,8 @@ status: active # Four issues from a framework and an editor: #646, #647, #648 and #649, read against the engine and routed by home -**Status:** revision 1, for review. Nothing is implemented. +**Status:** revision 1, adopted on 2026-09-16. The implementation, its +refinements and its readings are `2026-09-16-646-649-implementation-plan.md`. Engine code was read at `2fc7b5b0` (origin/main, after mcpp 2026.9.15.2). mcpp-plugins, openxlings/xlings (`3cd8061`) and mcpplibs/mcpp-index were read diff --git a/.agents/docs/2026-09-16-646-649-implementation-plan.md b/.agents/docs/2026-09-16-646-649-implementation-plan.md index 64844061..d41dc00d 100644 --- a/.agents/docs/2026-09-16-646-649-implementation-plan.md +++ b/.agents/docs/2026-09-16-646-649-implementation-plan.md @@ -29,51 +29,51 @@ the parallel work trees of §8. Triage sections are cited as `T§`. | id | task (triage §) | owner | depends on | status | |---|---|---|---|---| -| L1 | the refresh decision walks the resolver's deprecated bare-name rung before calling a miss (T§7.0) | lead | - | todo | -| L2 | the saved standard output is close-on-exec, and not inheritable on Windows (T§7.2) | lead | - | todo | -| L3 | xlings runs under the owned launcher; a total deadline for the index refresh, an inactivity deadline for installs (T§7.3) | lead | L2 | todo | -| L4 | a per-run record of observed effects; `network` when a network child is launched (T§7.4) | lead | L3 | todo | -| L5 | the three refreshes that bypass `mcpp.pm.index_refresh` go through `decide_for_miss` (T§7.5) | lead | L1 | todo | -| L6 | the refusal code `offline-download-required` at every offline refusal site; `MCPP_OFFLINE_DOWNLOAD_REQUIRED` in the envelope (T§7.1) | lead | - | todo | -| L7 | the default `artifact` of the mcpplibs index is the region object; the existing `.xlings.json` migration carries it (T§7.6 step 2) | lead | X1 | todo | -| R1 | ELF: programs and tests over a plan-built C++ shared library take the shared-library runtime contract; an explicit `self-contained` program there is refused (T§4.1, D1) | W1 | - | todo | -| R2 | symbol provision: `STB_GNU_UNIQUE` is vague linkage; a duplicate whose definitions come from one plan object is not reported (T§4.2) | W1 | - | todo | -| R3 | a static package reachable from one shared image only is linked into that image; one reachable from two images is refused where the link or load cannot succeed and diagnosed elsewhere (T§4.3, D2, §1.5) | W1 | - | todo | -| R4 | clang on the MSVC ABI records the runtime it delivers; an undeliverable `cxx_runtime` is diagnosed; docs/20 states the row's model (T§4.5 step 1) | W1 | - | todo | -| R5 | measurement legs: exception identity across a Mach-O dylib under the payload default (macos-15), and across an llvm-row DLL (windows-2022), each printing its reading (T§4.4, §9.2) | W1 | - | todo | -| G1 | the forward validator accepts a key declared in any dependency table on any row (T§6.1, X8) | W2 | - | todo | -| G2 | a `[feature-deps]` restatement whose source differs is refused; docs/05 says to restate the source (T§6.2, D8) | W2 | - | todo | -| G3 | one helper names a provider for a consumer; `dep_bin` gains the qualified spelling (T§6.3) | W2 | - | todo | -| G4 | a package with no library target contributes nothing to a consumer's target graph; package cycles are checked at resolution; a repeated tool is refused at its first repetition (T§6.4, D3, X2, X3) | W2 | - | todo | -| G5 | a git dependency selects a repository member by identity; a second declaration merges additively; the git banner names the commit (T§6.5, D9, X1, X6) | W2 | G4 | todo | -| G6 | `--features dep/feature` is a root forward; `why deps --features` (T§6.6, X7) | W2 | G1 | todo | -| S1 | `mcpp::graph_file()` for the root build program, with `[package.metadata]` and a digest in the re-run key (T§5.1, D4) | W3 | - | todo | -| S2 | the link branch is chosen by host and target object format, as a pure function with a host-by-row unit test (T§5.3) | W3 | - | todo | -| S3 | pack strips the program on every stripping row, every graph-built shared library and staged runtime copies; the status line reports what was done; `mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()` (T§5.4, D5) | W3 | - | todo | -| S4 | `mcpp pack --message-format json` prints one `mcpp.pack` envelope; `pack --release/--dev`; `run` takes `build`'s profile precedence (T§5.5, D6, X5) | W3 | - | todo | -| S5 | `[package]` warns about an unknown key, `metadata` included in its known set (X4) | W3 | S1 | todo | -| C1 | CI: new llvm-dependent e2e scripts run on the hermetic job with their PASS lines asserted; the macOS and Windows measurement legs print their readings to the job summary | lead | R1-R5, S2 | todo | -| C2 | user documentation with its Chinese mirror, docs/50 codes and kinds, CHANGELOG, version 2026.9.16.1 | lead, W1-W3 | all | todo | +| L1 | the refresh decision walks the resolver's deprecated bare-name rung before calling a miss (T§7.0) | lead | - | branch: unit `PmIndexRefresh.BareNameResolvedThroughTheLegacyRungIsNotAMiss`; e2e 730 (fails on 2026.9.15.2: the decision asked for a refresh of `cjson@1.7.19`) | +| L2 | the saved standard output is close-on-exec, and not inheritable on Windows (T§7.2) | lead | - | branch: e2e 731 (fails on 2026.9.15.2: a build program holds the caller's pipe as descriptor 3) | +| L3 | xlings runs under the owned launcher; a total deadline for the index refresh, an inactivity deadline for installs (T§7.3) | lead | L2 | branch: e2e 732 (fails on 2026.9.15.2: `index update` returned after 60 s under a 3 s bound); SIGTERM to mcpp took the stub's group (local) | +| L4 | a per-run record of observed effects; `network` when a network child is launched (T§7.4) | lead | L3 | branch: e2e 733 (fails on 2026.9.15.2: a plan that refreshed the index reports no `network`) | +| L5 | the three refreshes that bypass `mcpp.pm.index_refresh` go through `decide_for_miss` (T§7.5) | lead | L1 | branch: e2e 734 (fails on 2026.9.15.2: the unsynced custom index was synced under `auto_refresh = false`) | +| L6 | the refusal code `offline-download-required` at every offline refusal site; `MCPP_OFFLINE_DOWNLOAD_REQUIRED` in the envelope (T§7.1) | lead | - | branch: e2e 733 leg B and 735 (fail on 2026.9.15.2: `MCPP_BUILD_DATABASE_PLAN_FAILED`) | +| L7 | the default `artifact` of the mcpplibs index is the region object; the existing `.xlings.json` migration carries it (T§7.6 step 2) | lead | X1 | branch: e2e 151 (fails on 2026.9.15.2: no region artifact); traced `mcpp index update` with `mirror = CN` connects only to raw.gitcode.com, gitcode.com and file-cdn.gitcode.com | +| R1 | ELF: programs and tests over a plan-built C++ shared library take the shared-library runtime contract; an explicit `self-contained` program there is refused (T§4.1, D1) | W1 | - | branch: unit `Distribution.*`; e2e 700 (fails on 2026.9.15.2: the default llvm shape aborts with `std::bad_cast`) | +| R2 | symbol provision: `STB_GNU_UNIQUE` is vague linkage; a duplicate whose definitions come from one plan object is not reported (T§4.2) | W1 | - | branch: unit `SymbolProvision.*`; e2e 701 (fails on 2026.9.15.2: `--strict` exits 1) | +| R3 | a static package reachable from one shared image only is linked into that image; one reachable from two images is refused where the link or load cannot succeed and diagnosed elsewhere (T§4.3, D2, §1.5) | W1 | - | branch: unit `StaticPlacement.*` (9); e2e 702 and the rewritten 307 (fail on 2026.9.15.2: `undefined symbol: x_answer` through a foreign `dlopen`) | +| R4 | clang on the MSVC ABI records the runtime it delivers; an undeliverable `cxx_runtime` is diagnosed; docs/20 states the row's model (T§4.5 step 1) | W1 | - | branch: e2e 703 (Windows CI) | +| R5 | measurement legs: exception identity across a Mach-O dylib under the payload default (macos-15), and across an llvm-row DLL (windows-2022), each printing its reading (T§4.4, §9.2) | W1 | - | branch: e2e 704 (macOS CI), 705 (Windows CI); readings to the job summaries | +| G1 | the forward validator accepts a key declared in any dependency table on any row (T§6.1, X8) | W2 | - | branch: e2e 710 (fails on 2026.9.15.2: the two-level build-dependency forward refused under `--strict`) | +| G2 | a `[feature-deps]` restatement whose source differs is refused; docs/05 says to restate the source (T§6.2, D8) | W2 | - | branch: e2e 711 (fails on 2026.9.15.2: the differing restatement is not refused) | +| G3 | one helper names a provider for a consumer; `dep_bin` gains the qualified spelling (T§6.3) | W2 | - | branch: e2e 711 and a new 187 leg (fail on 2026.9.15.2: `dep_bin("spike.installer")` reads nothing) | +| G4 | a package with no library target contributes nothing to a consumer's target graph; package cycles are checked at resolution; a repeated tool is refused at its first repetition (T§6.4, D3, X2, X3) | W2 | - | branch: e2e 712 (fails on 2026.9.15.2: the feature tool depending on its declaring package does not build); scan of 902 manifests found no consumer relying on the old edge | +| G5 | a git dependency selects a repository member by identity; a second declaration merges additively; the git banner names the commit (T§6.5, D9, X1, X6) | W2 | G4 | branch: e2e 713 (fails on 2026.9.15.2: a git dependency naming a repository member is refused) | +| G6 | `--features dep/feature` is a root forward; `why deps --features` (T§6.6, X7) | W2 | G1 | branch: e2e 714 (fails on 2026.9.15.2: `--features spike.fw/installer` does not open the feature) | +| S1 | `mcpp::graph_file()` for the root build program, with `[package.metadata]` and a digest in the re-run key (T§5.1, D4) | W3 | - | branch: e2e 720 (fails on 2026.9.15.2: `graph_file` is not a member of `mcpp`) | +| S2 | the link branch is chosen by host and target object format, as a pure function with a host-by-row unit test (T§5.3) | W3 | - | branch: unit `LinkShape.*`; e2e 721 (macOS CI step) | +| S3 | pack strips the program on every stripping row, every graph-built shared library and staged runtime copies; the status line reports what was done; `mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()` (T§5.4, D5) | W3 | - | branch: e2e 722 (fails on 2026.9.15.2: `lib/libdep.so is not stripped: symtab=1 debug=7`; the Android leg ran locally) | +| S4 | `mcpp pack --message-format json` prints one `mcpp.pack` envelope; `pack --release/--dev`; `run` takes `build`'s profile precedence (T§5.5, D6, X5) | W3 | - | branch: unit `BuildProfile.*`; e2e 723 (fails on 2026.9.15.2: `unknown option: --message-format`) | +| S5 | `[package]` warns about an unknown key, `metadata` included in its known set (X4) | W3 | S1 | branch: unit `Manifest.PackageMetadata*`, `UnknownPackageKeyIsReported` | +| C1 | CI: new llvm-dependent e2e scripts run on the hermetic job with their PASS lines asserted; the macOS and Windows measurement legs print their readings to the job summary | lead | R1-R5, S2 | branch: 700 on the hermetic job; READING lines to the macOS and Windows summaries; 721 as its own macOS step | +| C2 | user documentation with its Chinese mirror, docs/50 codes and kinds, CHANGELOG, version 2026.9.16.1 | lead, W1-W3 | all | branch: docs 04, 05, 06, 07, 10, 20, 30, 50 with mirrors; SPEC-005 v1.1; CHANGELOG; 2026.9.16.1 | | C3 | the two records closed with their readings | lead | all | todo | ### 0.2 Plugins: mcpp-community/mcpp-plugins, one pull request (`feat/646-649`, 0.12.0) | id | task | owner | depends on | status | |---|---|---|---|---| -| P1 | `dist-apple` `options::omit_keys` over the defaulted keys (T§8) | lead | - | todo | -| P2 | `dist-web` `options::page`, default `index.html` (T§8) | lead | - | todo | -| P3 | `dist-apk` follows `mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()` when the engine provides them, and keeps its own strip otherwise | lead | S3 released | todo | -| P4 | `rules-swift`: one package's Swift sources compile through an `object` action, a generated header through a `source` action, and the runtime link flags through `link_flag`; a macos-15 consumer runs (T§5.2) | lead | - | todo | +| P1 | `dist-apple` `options::omit_keys` over the defaulted keys (T§8) | lead | - | branch: `b5f759e`, plan checks on Linux; the bundle on macos-15 CI | +| P2 | `dist-web` `options::page`, default `index.html` (T§8) | lead | - | branch: `c804312`, `check-web-plan.sh` with a named-page leg | +| P3 | `dist-apk` follows `mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()` when the engine provides them, and keeps its own strip otherwise | lead | S3 released | branch: `ad7a3dd`; unset and simulated variables verified; legs (m) and (n) run once the CI pin is 2026.9.16.1 | +| P4 | `rules-swift`: one package's Swift sources compile through an `object` action, a generated header through a `source` action, and the runtime link flags through `link_flag`; a macos-15 consumer runs (T§5.2) | lead | - | branch: `38ac2ba`, `all-rules-compile` on Linux; build, run and the header on macos-15 CI | | P5 | `dist-apk` and `dist-apple` collect library contributions from `[package.metadata.dist-apk]` and `[package.metadata.dist-apple]` through `mcpp::graph_file()` when present | lead | S1 released | todo | ### 0.3 Index, payloads and projects | id | task | owner | depends on | status | |---|---|---|---|---| -| X1 | mcpplibs/mcpp-index: the artifact is byte-reproducible, and a version whose GitCode asset differs is republished under a new name (T§7.6 step 1) | lead | - | todo | +| X1 | mcpplibs/mcpp-index: the artifact is byte-reproducible, and a version whose GitCode asset differs is republished under a new name (T§7.6 step 1) | lead | - | done: mcpplibs/mcpp-index#432 (`0cbac960`); the publish of `0cbac96` verified both forges itself, and the probe reads pointer, GitHub and GitCode all `02a437110017` (547679 bytes) | | X2 | openxlings/xim-pkgindex: the release bot's `mcpp` bump merged | lead | release | todo | | X3 | mcpplibs/mcpp-index: the plugins descriptor 0.12.0; the index CI's `MCPP_VERSION` moves to 2026.9.16.1 | lead | X2, P1-P5 | todo | -| X4 | openxlings/xlings: `compat.ftxui` and `compat.gtest` spelled with their namespace (T§7.0) | lead | - | todo | +| X4 | openxlings/xlings: `compat.ftxui` and `compat.gtest` spelled with their namespace (T§7.0) | lead | - | done: openxlings/xlings#597 (`4ea4eac9`), CI 9 of 9 | ### 0.4 Verification @@ -203,7 +203,69 @@ means the default. ### 1.9 Refinements found while implementing -(appended during implementation) +1. **L5 adds a module.** `mcpp.pm.index_refresh` reads descriptors through + `mcpp.pm.index_route`, which imports the fetcher, so the fetcher could not + ask it. The policy half (reasons, policy, `decide_for_miss`, `apply`, the + one-sync guard) moved to `mcpp.pm.refresh_policy`, which + `mcpp.pm.index_refresh` re-exports. §1.1's "no task adds a module" does not + hold for this task, and no other derivation of the policy remains. +2. **L3 keeps the shell command.** The xlings invocations keep their command + strings (quoting, environment prefix, stderr redirection, all of which carry + earlier Windows lessons) and run them through `run_streaming_bounded`, which + is the deadline launcher with a streaming sink, an inactivity bound and, on + POSIX, an opt-in process group registered with the signal guard. The group + is opt-in because the launcher's uncaptured callers hand the terminal to + their child, and a background group reading the terminal is stopped by + SIGTTIN. A direct `xlings install -y` sends its output to the null device, + so it has a total bound of three hours rather than an inactivity bound. +3. **X1 reuses rather than renames.** A version already published is reused, + GitCode's copy first because GitCode cannot replace an asset, so a republish + converges on bytes that are already fixed; a fresh pack is reproducible. + This is the rule xim-pkgindex adopted on 2026-09-05, and it needs no new + asset name. +4. **GitCode's raw endpoint answered 403 for a while** for the mcpp-index + pointer through `main`, and 200 by commit id, while xim-index's pointer + answered 200; the responses carry Huawei WAF cookies, and the same request + answered 200 later. A client whose CN pointer fetch fails falls back to the + git source rather than to the GLOBAL artifact base; that is xlings' + behaviour, recorded here and reported as openxlings/xlings#598, not changed in + this batch. It is no regression: before the region object a CN client fetched + the pointer from GitHub, and its fallback was the same git source. +5. **R2 also excuses the toolchain's own std module initialisers.** GCC 16's + `libstdc++.so.6` exports `_ZGIW3std` and `_ZGIW3stdW6compat` itself, so the + one-plan-object rule alone left one finding on the gcc default; names that + `std.o` defines are also excused when the other provider lives inside the + compiler's installation (W1). +6. **R3's placement skips layer providers and distribution packages** + (`provides = ["mcpp:..."]`), which would otherwise report a graph C++ runtime + as a two-image conflict (e2e 690 passes), and the root's reach includes its + dev-dependencies. The criterion is a foreign `dlopen` with `RTLD_NOW` rather + than a replayed `-z defs` link (W1). +7. **G4 needs `targetsInferred`.** A package with only `src/main.cpp` and no + `[targets]` infers a program target; the rule applies to declared targets + only, so the manifest records which of the two it produced (W2). +8. **G4's first-repetition refusal also covers an unconditional + `[build-dependencies]` edge** on a tool that depends on its declarer, which is + a genuine cycle (W2). +9. **G5 selects members without the adoption warning**: the root and a member + pinned by the same revision are two identities over one clone, and the + member's source string carries a `#member=` suffix (W2). +10. **S4's profile rule lives in `mcpp.build.prepare`.** Importing + `mcpp.cli.cmd_build` from `mcpp.cli.cmd_publish` made GCC 16.1 crash at + `import mcpp.cli;` on every build (W3). +11. **S2's shape value is `PeLld`, not `WindowsLld`**: the substring `wsl` is + reserved by the runtime-contract source check (W3). A Windows host building + a non-PE target whose driver is not named by `--target` keeps today's lld + line, which keeps the Windows-to-Linux cross job unchanged. +12. **S3 uses one strip tool for every Android leg**: the NDK's `llvm-strip` + reads every Android ABI (W3). +13. **P3 reads the environment rather than new accessors**, so `dist-apk` keeps + working on an engine that does not publish the variables, and writes debug + files per ABI, because one APK carries one library name per ABI (plugins). +14. **P4 serves the iOS device row as well**, as `rules-metal` does; the + consumer declares the `@_cdecl` function itself rather than including the + generated header, whose C++ visibility depends on the Swift version + (plugins). ## 2. Engine tasks diff --git a/CHANGELOG.md b/CHANGELOG.md index cae242da..8a6ca480 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,94 @@ ## [Unreleased] +### 一个进程一个 C++ 运行时、图与打包的事实、离线与有期限的规划:#646 至 #649(2026.9.16.1) + +一个 UI 框架与一个编辑器报告的 23 项,外加实测发现的 12 项。按归属分诊后,引擎承担其中的通用 +缺陷与通用能力;插件、索引与 xlings 清单各自的一项在各自仓库。设计、测量与计划: +`.agents/docs/2026-09-16-646-649-*.md`。 + +**图像与运行时(#646、#649 E10)** + +- **程序与共享库共用一个 C++ 运行时(F3a)。** ELF 上程序默认 `self-contained`、共享库默认 + `toolchain-coupled`,一个加载本次构建所出 C++ 共享库的程序于是在一个进程里带两份运行时: + llvm 默认构建运行即以 `std::bad_cast` 中止(退出 134),gcc 带 900 个被抢占的 libstdc++ + 符号。现在这样的程序与测试取共享库的契约;显式写明自含运行时的程序被编译前拒绝,reason 为 + `program-cxx-runtime-split`。(单测 `Distribution.*`,e2e 700) +- **符号重复检查不再报告按构造共享的定义(F3b)。** `STB_GNU_UNIQUE` 按弱定义处理;两个映像 + 取自同一个构建对象(如 `std.o`)的定义,以及工具链自身运行时提供的 std 模块初始化器,不算 + 冲突;真实的重复定义仍然报告。(单测 `SymbolProvision.*`,e2e 701) +- **静态包放进到达它的那个共享库(F1)。** 只被一个依赖共享库到达的静态包链进该库,不再链进 + 程序(此前库里留着未定义符号,只在 ELF 上靠程序的副本侥幸运行);被多个映像到达时,在 + Mach-O、PE 与 Android `app` 行上编译前拒绝(`static-package-in-two-images`),其他 ELF 行 + 照旧构建并给出 `build/static-placement` 警告,`--strict` 下失败,出路是 + `linkage = "shared"`。(单测 `StaticPlacement.*`,e2e 702、307) +- **llvm 行在 MSVC ABI 上如实记录静态 CRT(E10 第一步)。** 解析记录此前写 `host-coupled` + 而产物静态链接 `libcmt`;显式要求本行不能交付的动态运行时会得到说明。默认值是否改为 `/MD` + 另行测量与记录。(e2e 703;Mach-O 与 PE 上跨映像的异常身份由 e2e 704、705 测量) + +**图、构建程序与打包(#647 E1–E3、#649 E5、E9)** + +- **根构建程序读取解析后的依赖图(E1)。** `mcpp::graph_file()` / `MCPP_GRAPH_FILE` 指向按 + 依赖在前排序的 JSON 文档,每个包含 `manifest_dir`、`features`、`targets`、链接形态与原样 + 保留的 `[package.metadata.]`;文档摘要进入构建程序的重跑键,编辑依赖的 metadata 会 + 重跑根构建程序,编辑其源码不会。`[package]` 的未知键像 `[build]` 一样报告。(e2e 720) +- **macOS 主机上 Android 行可以链接(E3)。** 链接行按主机与目标对象格式选择 + (`link_shape`),Apple SDK 分支只用于 Mach-O 目标;Linux 上的链接行逐字节不变。 + (单测 `LinkShape.*`,e2e 721 在 macOS CI 上运行) +- **打包剥离构建出来的一切(E5)。** Android 行的程序此前根本没有经过剥离步骤;现在每条剥离的 + 行上剥离程序、本图构建的共享库与暂存的工具链运行时副本(`--strip-unneeded`,保留导出), + 状态行只在真正剥离时写 "stripped";`--no-strip` 与 `--debug-symbols` 作用于每个文件,构建 + 程序通过 `MCPP_PACK_STRIP` / `MCPP_PACK_DEBUG_SYMBOLS_DIR` 读到同一决定。(e2e 722) +- **`mcpp pack --message-format json`(E9)。** 输出一个 `mcpp.pack` 信封,逐项列出产物的 + 绝对路径、类型、格式与目标行;人类可读的行改走 stderr。`pack` 接受 `--release` / `--dev`; + `build`、`run`、`test`、`pack` 共用一个 profile 判定,`--profile` 优先于简写(此前 `run` + 相反)。(单测 `BuildProfile.*`,e2e 723) + +**feature、工具与 git 依赖(#647 E4、#649 E6–E8)** + +- **转发校验覆盖每一张依赖表与每一行(E4.1)。** 经 `[build-dependencies]` 的转发、只在另一行 + 声明的依赖不再被报告为未声明;无处声明的键仍然报告。(e2e 710) +- **`[feature-deps]` 的重述(E4.2)。** 文档改为写明重述来源;来源与生效声明不同的重述被拒绝, + 消息给出两个来源。(e2e 711) +- **`dep_bin` 发布限定名(E4.3)。** 一个函数给出提供者对消费者的所有名字,`dep_dir`、 + `dep_linkage` 与 `dep_bin` 共用;`namespace = "ns"` 加 `name = "x"` 的包也有 + `MCPP_DEP_NS_X_BIN_*`。(e2e 187、711) +- **只提供程序的包不进入消费者的图(E6)。** 这样的包不被扫描、不链接进消费者,其程序只由工具 + 子构建产出;feature 工具于是可以依赖声明它的包。包之间的环在解析时拒绝(`package-cycle`), + 各种缓存模式一致;工具请求自身时在第一次重复即拒绝。(e2e 712) +- **git 依赖可以选择仓库里的成员包(E7)。** 键的身份不是根包时,在根的 `[workspace] members` + 中按身份查找;同一消费者对同一依赖的第二次声明合并其 `tools`、`features`、`host-module` 与 + `reexport`;git 依赖的编译行写提交而不是空版本。(e2e 713) +- **`--features dep/feature`(E8)。** 作为根的转发应用;不指向任何依赖时警告,`--strict` 下 + 失败,且不再变成宏;普通名字保持文档所述的纯宏用法。`mcpp why deps` 接受 `--features`。 + (e2e 714) + +**编辑器在后台规划(#648)** + +- **刷新判定与解析器走同一条阶梯(新发现)。** 省略命名空间的 `ftxui = "6.1.9"` 由解析器经 + 已弃用的裸名回退找到 `compat.ftxui`,刷新判定却只查精确坐标并判为缺失;防抖只有 120 秒, + 这样的工程每次联网规划都执行一次 `xlings update`。现在判定也走该回退。(单测 + `PmIndexRefresh.BareNameResolvedThroughTheLegacyRungIsNotAMiss`,e2e 730) +- **规划期间的子进程不继承调用方的管道(A2)。** 保存的标准输出是 close-on-exec(Windows 上不可 + 继承);此前构建程序以描述符 3 持有调用方读取的管道。(e2e 731) +- **xlings 子进程有期限并随 mcpp 结束(A3)。** 刷新受 `[index] refresh_timeout`(秒,默认 + 120)约束,超时视为刷新失败并继续用本地索引;经接口的安装在 300 秒无任何输出(含心跳)时终止; + 子进程在自己的进程组(Windows 上为作业对象)中运行,结束 mcpp 即一并结束。(e2e 732) +- **信封的 `effects` 按观测报告 `network`(A4)。** 本次运行启动过刷新、安装或 git 远程操作时 + 列出,离线运行从不列出。(e2e 733) +- **`auto_refresh = false` 约束所有隐式刷新(A5)。** 安装前的刷新、重试前的刷新与自定义索引的 + 首次同步改走同一个刷新策略(`mcpp.pm.refresh_policy`)。(e2e 734) +- **离线缺下载有自己的诊断码(A1)。** `MCPP_OFFLINE_DOWNLOAD_REQUIRED` 与 refusal + `offline-download-required`,消息指出第一个需要下载的工具链、包、git 修订或索引。 + (e2e 733、735) +- **默认索引制品按镜像分区(A6)。** mcpplibs 索引的 `artifact` 默认是 + `{ GLOBAL = github, CN = gitcode }`,已有 home 的 `.xlings.json` 就地升级;`mirror = CN` 时 + `mcpp index update` 只访问 GitCode。前提是 mcpplibs/mcpp-index#432 让两端制品逐字节一致。 + (e2e 151) + +**CI**:hermetic llvm job 运行 700 并断言结束行;macOS 与 Windows e2e 把测量行写入 job +summary;macOS 上单独一步运行 721。 + ### 链接形态、标准档位与路径长度:#641 与 #642(2026.9.15.2) 一个 UI 框架迁到 macOS 12 下限与 Android 独立共享库时报告的七项,全部在引擎内处理。 diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index c19822fc..295ef7ac 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -408,8 +408,8 @@ a program classifying the outcome reads `reason`: | `host-module-missing` | `build.mcpp` imports a module no dependency supplies as a host module | | `tool-version-conflict` | two declarations name one xlings package at versions that cannot both hold | | `shared-library-cxx-runtime` | a dependency's C++ shared library in a graph whose C++ runtime is a package, with no private copy stated | -| `offline-download-required` | the run is offline and the plan needs a download: a toolchain, a package, a git revision or the package index | -| `package-cycle` | the dependency graph contains a cycle of packages; the message names its edges | +| `offline-download-required` | the run is offline and the plan needs a download: a toolchain, a package, a git revision or the package index *(2026.9.16.1+)* | +| `package-cycle` | the dependency graph contains a cycle of packages; the message names its edges *(2026.9.16.1+)* | | `program-cxx-runtime-split` | a program or test that states a self-contained C++ runtime loads a C++ shared library of the build that couples to a shared one *(2026.9.16.1+)* | | `static-package-in-two-images` | a static package several images of the build reach, on a target where an image cannot use another image's copy *(2026.9.16.1+)* | | `other` | a refusal whose branch has not been given a token yet | diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 34d8734c..ea2ef3d6 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -357,8 +357,8 @@ mcpp why toolchain [--target ] [--toolchain ] --format json | `host-module-missing` | `build.mcpp` 导入了没有任何依赖以 host module 形式提供的模块 | | `tool-version-conflict` | 两处声明把同一个 xlings 包定在不能同时成立的版本上 | | `shared-library-cxx-runtime` | C++ 运行时来自图中的包,而依赖的 C++ 共享库没有声明私有副本 | -| `offline-download-required` | 本次运行离线,而规划需要下载:工具链、包、git 修订或包索引 | -| `package-cycle` | 依赖图中存在包的环;消息列出环上的边 | +| `offline-download-required` | 本次运行离线,而规划需要下载:工具链、包、git 修订或包索引 *(2026.9.16.1+)* | +| `package-cycle` | 依赖图中存在包的环;消息列出环上的边 *(2026.9.16.1+)* | | `program-cxx-runtime-split` | 声明了自含 C++ 运行时的程序或测试,加载了本次构建中耦合到共享运行时的 C++ 共享库 *(2026.9.16.1+)* | | `static-package-in-two-images` | 一个静态包被本次构建的多个映像到达,而在该目标上一个映像不能使用另一个映像里的副本 *(2026.9.16.1+)* | | `other` | 一处还没有被命名的拒绝分支 | diff --git a/mcpp.toml b/mcpp.toml index d5c896a9..9a00dcb8 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.15.2" +version = "2026.9.16.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index a9dc4199..512d0715 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.15.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.16.1"; } // namespace mcpp diff --git a/src/build/flags.cppm b/src/build/flags.cppm index fb6ff5d1..7e656f68 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -813,6 +813,18 @@ CompileFlags compute_flags(const BuildPlan& plan) { // is only assemblable by its own x86_64-w64-mingw32-as. bool isMuslTc = mcpp::toolchain::is_musl_target(plan.toolchain); bool isMingwTc = mcpp::toolchain::is_mingw_target(plan.toolchain); + // The object format the TARGET produces, derived once: the runtime contract + // table and the link-line shape both read it (#647 E3). Target-keyed, with + // the host's format only as the fallback for a triple that names none; a + // MinGW toolchain is a PE whatever its triple spelling says. + const mcpp::build::dist::Format targetObjectFormat = + isMingwTc ? mcpp::build::dist::Format::Pe + : mcpp::build::dist::format_for(plan.toolchain.targetTriple, + mcpp::platform::needs_explicit_libcxx + ? mcpp::build::dist::Format::MachO + : mcpp::platform::is_windows + ? mcpp::build::dist::Format::Pe + : mcpp::build::dist::Format::Elf); const auto linkIntentFlavor = [&] { if (isMingwTc) return LinkIntentFlavor::PeGnu; if (isMsvcDialect) return LinkIntentFlavor::PeMsvc; @@ -1097,14 +1109,7 @@ CompileFlags compute_flags(const BuildPlan& plan) { // table because it recognises a mingw toolchain by more than its // triple; everything after it is `dist::format_for`, which is where // the question is answered and where it is tested. - const dist::Format format = - isMingwTc ? dist::Format::Pe - : dist::format_for(plan.toolchain.targetTriple, - mcpp::platform::needs_explicit_libcxx - ? dist::Format::MachO - : mcpp::platform::is_windows - ? dist::Format::Pe - : dist::Format::Elf); + const dist::Format format = targetObjectFormat; // `static_stdlib` is a faithful alias of the two ends of the contract: // its documented meaning has always been exactly self-contained vs the @@ -1625,18 +1630,8 @@ CompileFlags compute_flags(const BuildPlan& plan) { // the ordering between them is the whole point. std::string platformAnchor; // WHICH BRANCH, asked of the host AND the target (see `link_shape`). The - // format is the one the contract table reads, from the same function and - // the same host fallback; a MinGW toolchain is a PE whatever its triple - // spelling says. - const mcpp::build::dist::Format linkTargetFormat = - isMingwTc ? mcpp::build::dist::Format::Pe - : mcpp::build::dist::format_for(plan.toolchain.targetTriple, - mcpp::platform::needs_explicit_libcxx - ? mcpp::build::dist::Format::MachO - : mcpp::platform::is_windows - ? mcpp::build::dist::Format::Pe - : mcpp::build::dist::Format::Elf); - const LinkShape linkShape = link_shape(current_link_host(), linkTargetFormat, + // format is `targetObjectFormat`, the one the contract table reads. + const LinkShape linkShape = link_shape(current_link_host(), targetObjectFormat, isMsvcDialect, !plan.toolchain.crossTargetFlag.empty()); if (linkShape == LinkShape::MsvcLinkExe || linkShape == LinkShape::PeLld) { From 25e8e1cf5faea618e60e3718e36b3b7df8672f9c Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 07:13:14 +0800 Subject: [PATCH 19/20] Mach-O: the measured runtime identity split is reported; the stream-init shim belongs to C++ units; e2e 732 bounds itself portably The macos-15 run of e2e 704 reads runtime_error=not-matched, errc=unequal under the payload default and caught/equal under host-coupled, so #646 F2 holds. The default is unchanged, and such a build is told once. The same run showed the #336 shim prepended to a C-only shared library, whose link carries no libc++, and that macOS runners have no `timeout`. --- .../2026-09-16-646-649-implementation-plan.md | 25 +- .agents/docs/2026-09-16-646-649-verify.sh | 322 ++++++++++++++++++ CHANGELOG.md | 5 + docs/20-toolchains.md | 13 + docs/zh/20-toolchains.md | 9 + src/build/ninja_backend.cppm | 11 +- src/build/prepare.cppm | 34 ++ ...732_xlings_refresh_is_bounded_and_owned.sh | 12 +- 8 files changed, 427 insertions(+), 4 deletions(-) create mode 100644 .agents/docs/2026-09-16-646-649-verify.sh diff --git a/.agents/docs/2026-09-16-646-649-implementation-plan.md b/.agents/docs/2026-09-16-646-649-implementation-plan.md index d41dc00d..2cd00db9 100644 --- a/.agents/docs/2026-09-16-646-649-implementation-plan.md +++ b/.agents/docs/2026-09-16-646-649-implementation-plan.md @@ -40,7 +40,7 @@ the parallel work trees of §8. Triage sections are cited as `T§`. | R2 | symbol provision: `STB_GNU_UNIQUE` is vague linkage; a duplicate whose definitions come from one plan object is not reported (T§4.2) | W1 | - | branch: unit `SymbolProvision.*`; e2e 701 (fails on 2026.9.15.2: `--strict` exits 1) | | R3 | a static package reachable from one shared image only is linked into that image; one reachable from two images is refused where the link or load cannot succeed and diagnosed elsewhere (T§4.3, D2, §1.5) | W1 | - | branch: unit `StaticPlacement.*` (9); e2e 702 and the rewritten 307 (fail on 2026.9.15.2: `undefined symbol: x_answer` through a foreign `dlopen`) | | R4 | clang on the MSVC ABI records the runtime it delivers; an undeliverable `cxx_runtime` is diagnosed; docs/20 states the row's model (T§4.5 step 1) | W1 | - | branch: e2e 703 (Windows CI) | -| R5 | measurement legs: exception identity across a Mach-O dylib under the payload default (macos-15), and across an llvm-row DLL (windows-2022), each printing its reading (T§4.4, §9.2) | W1 | - | branch: e2e 704 (macOS CI), 705 (Windows CI); readings to the job summaries | +| R5 | measurement legs: exception identity across a Mach-O dylib under the payload default (macos-15), and across an llvm-row DLL (windows-2022), each printing its reading (T§4.4, §9.2) | W1 | - | done: macos-15 run 35032727668 reads `macho default: runtime_error=not-matched own_error=caught errc=unequal`, `macho host-coupled: runtime_error=caught errc=equal`, `macho shared-host-coupled: not-matched`. F2 is confirmed; this release warns (`build/cxx-runtime-identity`) and leaves the default to its own record | | G1 | the forward validator accepts a key declared in any dependency table on any row (T§6.1, X8) | W2 | - | branch: e2e 710 (fails on 2026.9.15.2: the two-level build-dependency forward refused under `--strict`) | | G2 | a `[feature-deps]` restatement whose source differs is refused; docs/05 says to restate the source (T§6.2, D8) | W2 | - | branch: e2e 711 (fails on 2026.9.15.2: the differing restatement is not refused) | | G3 | one helper names a provider for a consumer; `dep_bin` gains the qualified spelling (T§6.3) | W2 | - | branch: e2e 711 and a new 187 leg (fail on 2026.9.15.2: `dep_bin("spike.installer")` reads nothing) | @@ -267,6 +267,27 @@ means the default. generated header, whose C++ visibility depends on the Swift version (plugins). +15. **F2 is confirmed, and answered with a diagnostic (macos-15, run + 35032727668).** Under the payload's Mach-O default a `std::runtime_error` + thrown in a dylib is not caught by its class in the program and two + `std::error_code` categories compare unequal; a host-coupled graph catches + it and compares equal; the mixed leg (a self-contained program over a + host-coupled library) splits as well. The default is unchanged in this + release, and a build whose program loads a C++ dylib of its own is told + once through `build/cxx-runtime-identity`, with `cxx_runtime = + "host-coupled"` as the remedy. Changing the Mach-O default belongs to its + own record, as T§4.4 states. +16. **The macOS stream-init shim was prepended to every link unit.** The object + calls libc++'s `ios_base::Init` constructor, and a C-only shared library + links without the C++ runtime, so on macos-15 the F1 fixture failed with + `ld64.lld: error: undefined symbol: std::__1::ios_base::Init::Init()`. The + shim now follows the predicate the link line itself uses + (`unit_needs_cxx_runtime`). The defect predates this batch; no fixture had + a C-only shared library beside a C++ program on macOS before e2e 702. +17. **`timeout` is not on a macOS runner.** e2e 732 bounds its own commands + with `timeout`, `gtimeout` or neither, since the bound under test is the + engine's. + ## 2. Engine tasks ### 2.1 Lead: #648 (L1 to L7) @@ -469,4 +490,4 @@ its `target/` after its branch is merged. ## 9. Closure -(written when the rows of §0 are closed) +(written when the rows of §0 are closed) \ No newline at end of file diff --git a/.agents/docs/2026-09-16-646-649-verify.sh b/.agents/docs/2026-09-16-646-649-verify.sh new file mode 100644 index 00000000..57ee9e78 --- /dev/null +++ b/.agents/docs/2026-09-16-646-649-verify.sh @@ -0,0 +1,322 @@ +#!/usr/bin/env bash +# Ecosystem verification for #646 to #649 against the PUBLISHED mcpp, plugins and +# index, run inside a SubOS sandbox with CN mirrors for xlings and mcpp. +# +# B64=$(base64 -w0 .agents/docs/2026-09-16-646-649-verify.sh) +# xlings subos new v646 +# xlings subos use v646 --sandbox --cmd \ +# "echo $B64 | base64 -d > /tmp/v.sh && MCPP_VERIFY_VERSION=2026.9.16.1 bash /tmp/v.sh" +# +# The script is run once against the previous release first +# (MCPP_VERIFY_VERSION=2026.9.15.2): the sections marked CHANGE must fail there +# and pass on the new release; the sections marked GUARD must pass on both. A +# script that passes on both releases in a CHANGE section measured nothing. +# +# The sandbox shares the xlings data directory, so a published mcpp is addressed +# by its store path and exact version. Its $HOME content persists between runs +# of one SubOS, so every section clears its own probe directory first. A section +# that cannot run says so and is listed again in the summary. +set -u + +VER="${MCPP_VERIFY_VERSION:?set MCPP_VERIFY_VERSION}" +STORE="${MCPP_VERIFY_BIN:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/$VER/bin/mcpp}" +PLUGINS="${MCPP_VERIFY_PLUGINS:-0.12.0}" + +fails=0 +skipped="" +fail() { printf 'ASSERT-FAIL: %s\n' "$1"; fails=$((fails + 1)); } +ok() { printf 'ok: %s\n' "$1"; } +section() { printf '\n== %s ==\n' "$1"; } +skip() { printf 'NOT RUN: %s\n' "$1"; skipped="$skipped + - $1"; } +has_text() { grep -qF -- "$2" "$1"; } +reading() { printf 'READING %s\n' "$*"; } +unset XLINGS_ACTIVE_SUBOS + +root="$HOME/verify-646" +rm -rf "$root"; mkdir -p "$root" + +section "A. GUARD: the published mcpp answers for itself, with CN mirrors for xlings and mcpp" +if [ ! -x "$STORE" ]; then + fail "no mcpp at $STORE"; printf '\nfails=%d (nothing else can run)\n' "$fails"; exit 1 +fi +got=$("$STORE" --version 2>&1 | head -1) +case "$got" in *"$VER"*) ok "mcpp --version says $got" ;; *) fail "mcpp --version says '$got', expected $VER" ;; esac +if command -v xlings >/dev/null 2>&1; then + xlings config --mirror CN >/dev/null 2>&1 && ok "xlings config --mirror CN" || fail "xlings config --mirror CN" +fi +"$STORE" self config --mirror CN >/dev/null 2>&1 && ok "mcpp self config --mirror CN" || fail "mcpp self config --mirror CN" +grep -q '"mirror": *"CN"' "$HOME/.mcpp/registry/.xlings.json" 2>/dev/null \ + && ok "mcpp's xlings reads mirror CN" || fail "mcpp's xlings does not read mirror CN" + +section "B. CHANGE (#648 A6): the mcpplibs index artifact has a CN base, and an index update succeeds" +if grep -q 'gitcode.com/xlings-res/mcpp-index' "$HOME/.mcpp/registry/.xlings.json" 2>/dev/null; then + ok "the mcpplibs artifact names the GitCode mirror" +else + fail "the mcpplibs artifact has no CN base"; grep -n artifact "$HOME/.mcpp/registry/.xlings.json" | head -3 +fi +if timeout 600 "$STORE" index update > "$root/index-update.log" 2>&1; then + ok "mcpp index update" +else + fail "mcpp index update failed"; tail -5 "$root/index-update.log" +fi +reading "A6 mcpplibs index version: $(cat "$HOME/.mcpp/registry/data/mcpplibs/.xlings-index-version" 2>/dev/null || echo none)" + +section "C. CHANGE (#646 F3a): an llvm program over a C++ shared library runs" +d=$root/c; rm -rf "$d"; mkdir -p "$d/app/src" "$d/lib/src" +printf '[package]\nname = "lib"\nversion = "0.1.0"\n[targets.lib]\nkind = "shared"\n' > "$d/lib/mcpp.toml" +printf 'export module lib;\nimport std;\nexport [[gnu::visibility("default")]] std::string lib_greet(int n);\n' > "$d/lib/src/lib.cppm" +printf 'module lib;\nimport std;\nstd::string lib_greet(int n) { return std::format("lib-{}", n); }\n' > "$d/lib/src/lib.cpp" +printf '[package]\nname = "app"\nversion = "0.1.0"\n[toolchain]\ndefault = "llvm@22.1.8"\n[dependencies.lib]\npath = "../lib"\n' > "$d/app/mcpp.toml" +printf 'import std;\nimport lib;\nint main() { std::println("{}", lib_greet(3)); }\n' > "$d/app/src/main.cpp" +( cd "$d/app" && timeout 1800 "$STORE" run > run.log 2>&1 ); rc=$? +reading "F3a run exit=$rc: $(grep -m1 -E 'lib-3|bad_cast' "$d/app/run.log")" +if [ "$rc" -eq 0 ] && has_text "$d/app/run.log" "lib-3"; then + ok "the program runs on one C++ runtime" +else + fail "the llvm program over a C++ shared library did not run"; tail -5 "$d/app/run.log" +fi + +section "D. CHANGE (#646 F1): a static package under a shared package is linked into it" +d=$root/d; rm -rf "$d"; mkdir -p "$d/x/src" "$d/fw/src" "$d/app/src" +printf '[package]\nname = "x"\nversion = "0.1.0"\n[targets.x]\nkind = "lib"\n' > "$d/x/mcpp.toml" +printf 'int x_answer(void) { return 42; }\n' > "$d/x/src/x.c" +printf '[package]\nname = "fw"\nversion = "0.1.0"\n[targets.fw]\nkind = "shared"\n[dependencies.x]\npath = "../x"\n' > "$d/fw/mcpp.toml" +printf 'int x_answer(void);\n__attribute__((visibility("default"))) int fw_answer(void) { return x_answer(); }\n' > "$d/fw/src/fw.c" +printf '[package]\nname = "app"\nversion = "0.1.0"\n[dependencies.fw]\npath = "../fw"\n' > "$d/app/mcpp.toml" +printf 'extern "C" int fw_answer(void);\nint main() { return fw_answer() == 42 ? 0 : 1; }\n' > "$d/app/src/main.cpp" +( cd "$d/app" && timeout 1200 "$STORE" build > build.log 2>&1 ) || true +lib=$(ls "$d"/app/target/*/*/bin/libfw.so 2>/dev/null | head -1) +if [ -z "$lib" ]; then + fail "libfw.so was not built"; tail -5 "$d/app/build.log" +elif command -v python3 >/dev/null 2>&1; then + if python3 -c 'import ctypes,sys; ctypes.CDLL(sys.argv[1], mode=ctypes.RTLD_GLOBAL|2)' "$lib" 2> "$d/dlopen.err"; then + ok "libfw.so loads on its own (RTLD_NOW): x is inside it" + else + fail "libfw.so does not load on its own: $(cat "$d/dlopen.err" | tail -1)" + fi +else + skip "D: no python3 in the sandbox to dlopen libfw.so" +fi + +section "E. CHANGE (#649 E6, #647 E4.3): a feature tool that depends on its declaring package" +d=$root/e; rm -rf "$d"; mkdir -p "$d/fw/src" "$d/fw/tool/src" "$d/app/src" +cat > "$d/fw/mcpp.toml" <<'EOF' +[package] +name = "fw" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[targets.fw] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[features] +installer = [] + +[target.'cfg(os = "linux")'.feature-deps.installer] +spike.fw-installer = { path = "tool", tools = ["fw-installer"], reexport = true } +EOF +cat > "$d/fw/tool/mcpp.toml" <<'EOF' +[package] +name = "fw-installer" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[targets.fw-installer] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw = { path = ".." } +EOF +cat > "$d/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.fw = { path = "../fw" } + +[features] +windows-installer = ["spike.fw/installer"] + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +cat > "$d/app/build.mcpp" <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("fw-installer", "fw-installer"); + const char* b = mcpp::dep_bin("spike.fw-installer", "fw-installer"); + std::println("SPIKE short=[{}] qualified=[{}]", a ? a : "", b ? b : ""); + return 0; +} +EOF +printf 'int fw_answer() { return 42; }\n' > "$d/fw/src/fw.cpp" +printf 'int fw_answer();\n#include \nint main() { std::printf("fw-installer ran %%d\\n", fw_answer()); return 0; }\n' > "$d/fw/tool/src/main.cpp" +printf 'int fw_answer();\nint main() { return fw_answer() == 42 ? 0 : 1; }\n' > "$d/app/src/main.cpp" +( cd "$d/app" && timeout 1200 "$STORE" build --features windows-installer > build.log 2>&1 ); rc=$? +reading "E6 build exit=$rc: $(grep -m1 -E 'SPIKE|cycle' "$d/app/build.log")" +if [ "$rc" -eq 0 ] && grep -q 'qualified=\[/' "$d/app/build.log" && grep -q 'short=\[/' "$d/app/build.log"; then + ok "the tool builds, and dep_bin answers under both spellings" +else + fail "the feature tool over its declaring package did not build with both dep_bin spellings"; tail -5 "$d/app/build.log" +fi + +section "F. CHANGE (#649 E8): --features / opens a dependency's feature" +sed -i 's/^windows-installer = \["spike.fw\/installer"\]$/windows-installer = []/' "$d/app/mcpp.toml" +rm -rf "$d/app/target" +( cd "$d/app" && timeout 1200 "$STORE" build --strict --features spike.fw/installer > f.log 2>&1 ); rc=$? +reading "E8 exit=$rc: $(grep -m1 -E 'SPIKE|features' "$d/app/f.log")" +if [ "$rc" -eq 0 ] && grep -q 'short=\[/' "$d/app/f.log"; then + ok "--features spike.fw/installer built the dependency's tool" +else + fail "--features spike.fw/installer did not open the feature"; tail -5 "$d/app/f.log" +fi + +section "G. CHANGE (#647 E1): the root build program reads the graph and a dependency's metadata" +d=$root/g; rm -rf "$d"; mkdir -p "$d/app/src" "$d/a/src" "$d/b/src" +printf '[package]\nname = "b"\nnamespace = "spike"\nversion = "0.2.0"\n\n[package.metadata.demo]\nresources = "res"\n\n[targets.b]\nkind = "lib"\n' > "$d/b/mcpp.toml" +printf 'int b_answer() { return 40; }\n' > "$d/b/src/b.cpp" +printf '[package]\nname = "a"\nnamespace = "spike"\nversion = "0.1.0"\n\n[targets.a]\nkind = "lib"\n\n[dependencies]\nspike.b = { path = "../b" }\n' > "$d/a/mcpp.toml" +printf 'int b_answer();\nint a_answer() { return b_answer() + 2; }\n' > "$d/a/src/a.cpp" +printf '[package]\nname = "app"\nversion = "0.1.0"\n\n[dependencies]\nspike.a = { path = "../a" }\n' > "$d/app/mcpp.toml" +printf 'int a_answer();\nint main() { return a_answer() == 42 ? 0 : 1; }\n' > "$d/app/src/main.cpp" +cat > "$d/app/build.mcpp" <<'EOF' +import std; +import mcpp; +int main() { + std::ifstream in(mcpp::graph_file()); + std::stringstream text; + text << in.rdbuf(); + const auto doc = text.str(); + const auto posA = doc.find("\"spike.a@"); + const auto posB = doc.find("\"spike.b@"); + std::println("GRAPH b-before-a={} metadata={}", posB < posA, + doc.find("\"resources\"") != std::string::npos); + return 0; +} +EOF +( cd "$d/app" && timeout 1200 "$STORE" build > build.log 2>&1 ); rc=$? +reading "E1 exit=$rc: $(grep -m1 -E 'GRAPH|graph_file' "$d/app/build.log")" +if grep -q 'GRAPH b-before-a=true metadata=true' "$d/app/build.log"; then + ok "the graph lists b before a, with b's [package.metadata]" +else + fail "the root build program did not read the graph"; tail -5 "$d/app/build.log" +fi + +section "H. CHANGE (#649 E5, E9): pack strips a graph-built library, and reports what it produced" +d=$root/h; rm -rf "$d"; mkdir -p "$d/dep/src" "$d/app/src" +printf '[package]\nname = "dep"\nnamespace = "spike"\nversion = "0.1.0"\nstandard = "c++20"\n\n[targets.dep]\nkind = "shared"\n\n[build]\nsources = ["src/*.cpp"]\n' > "$d/dep/mcpp.toml" +printf '#include \n[[gnu::visibility("default")]] int dep_answer() { return static_cast(std::to_string(42).size()) + 40; }\n' > "$d/dep/src/dep.cpp" +printf '[package]\nname = "hostapp"\nversion = "0.1.0"\nstandard = "c++20"\n\n[dependencies]\nspike.dep = { path = "../dep" }\n\n[targets.hostapp]\nkind = "bin"\nmain = "src/main.cpp"\n' > "$d/app/mcpp.toml" +printf 'int dep_answer();\nint main() { return dep_answer() == 42 ? 0 : 1; }\n' > "$d/app/src/main.cpp" +( cd "$d/app" && timeout 1800 "$STORE" pack --release --format tar --message-format json > pack.json 2> pack.err ); rc=$? +reading "E9 exit=$rc stdout-bytes=$(wc -c < "$d/app/pack.json")" +art=$(python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print(d["data"]["artifacts"][0]["path"])' "$d/app/pack.json" 2>/dev/null) +if [ -n "$art" ] && [ -f "$art" ]; then + ok "pack --release --message-format json names an artifact that exists" + mkdir -p "$d/x" && tar -xzf "$art" -C "$d/x" + so=$(find "$d/x" -name libdep.so | head -1) + if [ -n "$so" ] && command -v readelf >/dev/null 2>&1; then + symtab=$(readelf -S -W "$so" | grep -c ' .symtab') + reading "E5 libdep.so symtab=$symtab" + [ "$symtab" -eq 0 ] && ok "the graph-built libdep.so is stripped" || fail "the graph-built libdep.so keeps .symtab" + else + skip "H: no libdep.so in the archive or no readelf" + fi +else + fail "pack produced no machine-readable report"; tail -3 "$d/app/pack.err" +fi + +section "I. CHANGE (#648 A1, A2): offline code, and planning children do not hold the caller's pipe" +d=$root/i; rm -rf "$d"; mkdir -p "$d/index/pkgs/w" "$d/app/src" +cat > "$d/index/pkgs/w/widget.lua" <<'EOF' +package = { + spec = "1", namespace = "acme", name = "widget", description = "not installed", + licenses = {"MIT"}, type = "package", + xpm = { linux = { ["1.0.0"] = { url = "https://example.invalid/w.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } } }, + mcpp = { language = "c++23", sources = { "src/*.cpp" }, targets = { ["widget"] = { kind = "lib" } } }, +} +EOF +printf '[package]\nname = "app"\nversion = "0.1.0"\n\n[indices]\nacme = { path = "%s" }\n\n[dependencies.acme]\nwidget = "1.0.0"\n' "$d/index" > "$d/app/mcpp.toml" +printf 'int main() { return 0; }\n' > "$d/app/src/main.cpp" +( cd "$d/app" && MCPP_OFFLINE=1 timeout 300 "$STORE" emit build-database --format json > a1.json 2> a1.err ) || true +code=$(python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print(d["diagnostics"][0]["code"])' "$d/app/a1.json" 2>/dev/null) +reading "A1 code=$code" +[ "$code" = "MCPP_OFFLINE_DOWNLOAD_REQUIRED" ] && ok "an offline plan that needs a download has its own code" \ + || fail "the offline code is '$code'" +d2=$root/i2; rm -rf "$d2"; mkdir -p "$d2/src" +printf '[package]\nname = "fds"\nversion = "0.1.0"\n\n[targets.fds]\nkind = "bin"\nmain = "src/main.cpp"\n' > "$d2/mcpp.toml" +printf 'int main() { return 0; }\n' > "$d2/src/main.cpp" +cat > "$d2/build.mcpp" </dev/null \ + | { readlink /proc/self/fd/0 > "$d2/reader.txt"; cat > /dev/null; } ) +pipe=$(cat "$d2/reader.txt" 2>/dev/null) +if [ ! -s "$d2/fds.txt" ]; then + fail "A2: the build program did not run" +elif grep -qxF "$pipe" "$d2/fds.txt"; then + fail "A2: a build program holds the caller's pipe $pipe" +else + ok "A2: no planning child holds the caller's pipe" +fi + +section "J. CHANGE (#648 T): a bare compat.* dependency is not a refresh miss" +idx="$HOME/.mcpp/registry/data/mcpplibs/pkgs/c" +cand=""; for c in cjson argparse gtest; do [ -f "$idx/compat.$c.lua" ] && { cand=$c; break; }; done +if [ -z "$cand" ]; then + skip "J: no compat descriptor in the local index" +else + v=$(grep -o '\["[0-9][0-9.]*"\]' "$idx/compat.$cand.lua" | head -1 | tr -d '[]"') + d=$root/j; rm -rf "$d"; mkdir -p "$d/src" + printf '[package]\nname = "j"\nversion = "0.1.0"\n\n[dependencies]\n%s = "%s"\n' "$cand" "$v" > "$d/mcpp.toml" + printf 'int main() { return 0; }\n' > "$d/src/main.cpp" + ( cd "$d" && MCPP_OFFLINE=1 timeout 300 "$STORE" emit build-database --format json -v > /dev/null 2> j.err ) || true + if grep -q "index: $cand@$v: offline mode" "$d/j.err"; then + fail "T: the refresh decision still wants a network refresh for $cand@$v" + elif grep -q "deprecated bare-name search" "$d/j.err"; then + ok "T: $cand@$v resolves through the rung and asks for no refresh" + else + skip "T: the fixture did not reach the bare-name rung" + fi +fi + +section "K. GUARD then CHANGE: mcpp:plugins $PLUGINS resolves from the index and builds a consumer" +d=$root/k; rm -rf "$d"; mkdir -p "$d/src" +cat > "$d/mcpp.toml" < "$d/src/main.cpp" +( cd "$d" && timeout 1800 "$STORE" build > build.log 2>&1 ); rc=$? +reading "plugins exit=$rc: $(grep -m1 -E 'plugins|error' "$d/build.log")" +[ "$rc" -eq 0 ] && ok "mcpp:plugins $PLUGINS resolves and a consumer builds" \ + || { fail "mcpp:plugins $PLUGINS did not build a consumer"; tail -5 "$d/build.log"; } + +printf '\n== summary ==\nversion=%s fails=%d\n' "$VER" "$fails" +[ -n "$skipped" ] && printf 'not run:%s\n' "$skipped" +exit $(( fails > 0 ? 1 : 0 )) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8a6ca480..15d731d2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,11 @@ Mach-O、PE 与 Android `app` 行上编译前拒绝(`static-package-in-two-images`),其他 ELF 行 照旧构建并给出 `build/static-placement` 警告,`--strict` 下失败,出路是 `linkage = "shared"`。(单测 `StaticPlacement.*`,e2e 702、307) +- **Mach-O 上跨映像的 C++ 身份(F2,macos-15 实测)。** 默认每个映像内嵌一份隐藏的 + `libc++.a`:dylib 里抛出的 `std::runtime_error` 在程序中不按该类捕获,两个 + `std::error_code` 的 category 比较不相等;全角色 `cxx_runtime = "host-coupled"` 时 + 两者成立。默认值本次不改(它是今天每个 macOS 构建的形态,改动另行记录),但这样的构建 + 会被告知一次:`build/cxx-runtime-identity`。(e2e 704 的读数) - **llvm 行在 MSVC ABI 上如实记录静态 CRT(E10 第一步)。** 解析记录此前写 `host-coupled` 而产物静态链接 `libcmt`;显式要求本行不能交付的动态运行时会得到说明。默认值是否改为 `/MD` 另行测量与记录。(e2e 703;Mach-O 与 PE 上跨映像的异常身份由 e2e 704、705 测量) diff --git a/docs/20-toolchains.md b/docs/20-toolchains.md index 512133f0..f66e43ed 100644 --- a/docs/20-toolchains.md +++ b/docs/20-toolchains.md @@ -1036,6 +1036,19 @@ is refused before compiling (reason `program-cxx-runtime-split`, the shared library a private copy with `cxx_runtime = { shared = "self-contained" }`, which keeps each runtime inside its own image. +**On Mach-O every image carries its own runtime** (mcpp 2026.9.16.1+). The +default there is self-contained for every role, and each image embeds the +payload's `libc++.a` with hidden visibility, so the type information of a +standard library class exists once per image and libc++ compares it by address. +Measured on macos-15: with the default, a `std::runtime_error` thrown in a dylib +is not caught by `catch (const std::runtime_error&)` in the program and two +`std::error_code` categories compare unequal; with `cxx_runtime = "host-coupled"` +for every role, both hold. The default is unchanged, because it is what every +macOS build ships, and a build whose program loads a C++ dylib of its own is told +once (`build/cxx-runtime-identity`): state one runtime for the process when +objects cross the boundary as exceptions, or as libc++ values compared by +identity. + **A dependency's shared library over a C++ runtime that is a package** (mcpp 2026.9.15.2+). When a package in the graph supplies the C++ layer (`llvm.libcxx`, [22](22-target-side.md)), its objects are linked into the program, and it compiles diff --git a/docs/zh/20-toolchains.md b/docs/zh/20-toolchains.md index 16a4c463..ff6e73e8 100644 --- a/docs/zh/20-toolchains.md +++ b/docs/zh/20-toolchains.md @@ -932,6 +932,15 @@ C++ 共享库,会在编译前被拒绝(reason `program-cxx-runtime-split`,见 `cxx_runtime = { shared = "self-contained" }` 给共享库一份私有副本,让每份运行时留在 各自的映像里。 +**Mach-O 上每个映像各带一份运行时**(mcpp 2026.9.16.1+)。那里每个角色的默认值都是 +self-contained,每个映像以隐藏可见性内嵌载荷的 `libc++.a`,于是标准库类的类型信息每个 +映像各有一份,而 libc++ 按地址比较它。macos-15 实测:按默认值,dylib 里抛出的 +`std::runtime_error` 在程序中不被 `catch (const std::runtime_error&)` 捕获,两个 +`std::error_code` 的 category 比较不相等;所有角色都写 `cxx_runtime = "host-coupled"` +时两者都成立。默认值不变——它是今天每个 macOS 构建的形态——但程序加载了本次构建的 C++ +dylib 时会被告知一次(`build/cxx-runtime-identity`):当对象以异常、或以按身份比较的 +libc++ 值跨过边界时,为整个进程声明一份运行时。 + **C++ 运行时来自图中的包时,依赖的共享库**(mcpp 2026.9.15.2+)。当图中有包提供 C++ 层(`llvm.libcxx`,见 [22](22-target-side.md)),它的对象被链进程序,并且以隐藏 可见性编译,于是一个构建为 C++ 共享库的依赖无法解析到程序里的那份。mcpp 在编译前 diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index fb148574..9445d493 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -2184,7 +2184,16 @@ std::string emit_ninja_string(const BuildPlan& plan) { // LINK order and has no priority-ordered init section, so "runs // before the user's global constructors" is spelled "is the first // input" — nothing else about this edge achieves it (#336). - if (need_ios_init_shim && lu.kind != LinkUnit::StaticLibrary) + // ... and only into a unit that HOLDS a C++ runtime. The shim calls + // libc++'s `ios_base::Init` constructor, so a unit linked without the + // C++ runtime (a C-only shared library, whose line carries neither the + // C++ driver nor libc++.a) has nothing to resolve it against: measured + // on macos-15, `ld64.lld: error: undefined symbol: + // std::__1::ios_base::Init::Init()` from this object, in a build whose + // program is C++ and whose dependency's shared library is C (#646 F1's + // fixture). The predicate is the one the link line itself uses. + if (need_ios_init_shim && lu.kind != LinkUnit::StaticLibrary + && unit_needs_cxx_runtime(lu)) ins += " " + escape_ninja_path(ios_init_obj); for (auto& o : lu.objects) { ins += " " + escape_ninja_path(o); diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 88a09789..1311da41 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -12680,6 +12680,40 @@ prepare_build(bool print_fingerprint, tests ? "tests" : "programs", tests ? "tests" : "default")); } + // MACH-O: EVERY IMAGE CARRIES ITS OWN HIDDEN libc++ (#646 F2). + // + // The Mach-O default is self-contained for every role, and each image + // embeds the payload's `libc++.a` through `-load_hidden`, so the type + // information of a standard library class exists once per image and libc++ + // compares it by address. Measured on macos-15 for this release: with the + // default, a `std::runtime_error` thrown in a dylib is NOT caught by its + // class in the program and two `std::error_code` categories compare + // unequal; with `cxx_runtime = "host-coupled"` for every role, both hold. + // The default is not changed here, because it is what every macOS build + // ships today and changing it is its own record; a build that would meet + // the split is told, once, what it is and how to avoid it. + if (format == dist::Format::MachO && (load.program || load.tests) + && contracts.shared == dist::Contract::SelfContained) { + std::string libraries; + for (auto const& lu : ctx.plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::SharedLibrary) continue; + if (!mcpp::build::link_unit_holds_cxx(ctx.plan, lu)) continue; + libraries += (libraries.empty() ? "'" : ", '") + lu.targetName + "'"; + } + mcpp::diag::degraded("build/cxx-runtime-identity", + std::format("this build's program and the C++ shared library {} each " + "carry a private copy of the C++ runtime", + libraries.empty() ? std::string("'(unnamed)'") : libraries), + "on Mach-O every image embeds the payload's libc++ with hidden " + "visibility, so the type information of a standard library class exists " + "once per image: measured on macOS, an exception of such a class thrown " + "in the library is not caught by that class in the program, and two " + "error categories compare unequal", + "state one runtime for the process, for example [build] cxx_runtime = " + "\"host-coupled\", when objects cross the boundary as exceptions or as " + "libc++ values compared by identity"); + } + // F1. A static package that several images reach. Refused where the // build cannot work (Mach-O and PE resolve every reference at link // time; Android's Java host loads an application's shared library diff --git a/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh b/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh index 23dd3930..b65625aa 100755 --- a/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh +++ b/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh @@ -20,6 +20,16 @@ cleanup() { trap cleanup EXIT fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } +# `timeout` is GNU coreutils and is absent on macOS; `gtimeout` is there when +# coreutils is installed. Without either, the bound under test is the engine's +# own, and the suite's per-test bound is the backstop. +bounded() { # + local secs="$1"; shift + if command -v timeout >/dev/null 2>&1; then timeout "$secs" "$@"; return $?; fi + if command -v gtimeout >/dev/null 2>&1; then gtimeout "$secs" "$@"; return $?; fi + "$@" +} + mkdir -p "$TMP/bin" "$TMP/home" cat > "$TMP/bin/xlings" <<'EOF' #!/usr/bin/env bash @@ -44,7 +54,7 @@ EOF write_config 3 start=$(date +%s) set +e -STUB_PID="$TMP/stub.pid" MCPP_HOME="$TMP/home" timeout 60 "$MCPP" index update > "$TMP/a.log" 2>&1 +STUB_PID="$TMP/stub.pid" MCPP_HOME="$TMP/home" bounded 60 "$MCPP" index update > "$TMP/a.log" 2>&1 set -e elapsed=$(( $(date +%s) - start )) [ -s "$TMP/stub.pid" ] || fail "A: the stub's update never ran" "$TMP/a.log" From bae5062ec0a9b08242b4d7f442ce4e16013b5867 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 16 Sep 2026 07:23:37 +0800 Subject: [PATCH 20/20] docs(record): the local verification readings of the integrated branch --- .agents/docs/2026-09-16-646-649-implementation-plan.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.agents/docs/2026-09-16-646-649-implementation-plan.md b/.agents/docs/2026-09-16-646-649-implementation-plan.md index 2cd00db9..92394aef 100644 --- a/.agents/docs/2026-09-16-646-649-implementation-plan.md +++ b/.agents/docs/2026-09-16-646-649-implementation-plan.md @@ -284,6 +284,11 @@ means the default. shim now follows the predicate the link line itself uses (`unit_needs_cxx_runtime`). The defect predates this batch; no fixture had a C-only shared library beside a C++ program on macOS before e2e 702. +18. **The local verification of the integrated branch** (Linux x86_64, this + host): the full unit suite is 121 of 121, and `tests/e2e/run_all.sh` reads + `E2E Summary: 398 passed, 0 failed, 47 skipped`, the skips being the + capabilities this host lacks (msvc, mingw, qemu, a device). The macOS and + Windows legs are the pull request's own CI. 17. **`timeout` is not on a macOS runner.** e2e 732 bounds its own commands with `timeout`, `gtimeout` or neither, since the bound under test is the engine's.