diff --git a/.agents/docs/2026-09-16-646-649-four-issues-by-home.md b/.agents/docs/2026-09-16-646-649-four-issues-by-home.md new file mode 100644 index 000000000..6c8795f88 --- /dev/null +++ b/.agents/docs/2026-09-16-646-649-four-issues-by-home.md @@ -0,0 +1,1347 @@ +--- +subject: triage +status: active +--- + +# Four issues from a framework and an editor: #646, #647, #648 and #649, read against the engine and routed by home + +**Status:** revision 1, adopted on 2026-09-16. The implementation, its +refinements and its readings are `2026-09-16-646-649-implementation-plan.md`. + +Engine code was read at `2fc7b5b0` (origin/main, after mcpp 2026.9.15.2). +mcpp-plugins, openxlings/xlings (`3cd8061`) and mcpplibs/mcpp-index were read +at their current heads. A statement marked *measured* was run on a Linux +x86_64 host with the released mcpp 2026.9.15.2, `gcc@16.1.0`, `llvm@22.1.8` +and `xim:android-ndk@30.0.16248370`; the probes are +`2026-09-16-646-649-probes.sh` (groups a to d), and §11 quotes their +`READING` lines. A statement marked *read* names a file and line and was not +executed. Nothing in this revision ran on macOS or Windows; §9 lists the +premises that need a temporary pull request before their decisions are taken. + +The four issues hold 23 items. Measuring them found twelve defects that no +issue reports, two of which are more severe than anything the issues list: the +default llvm build of a program over a C++ shared library aborts on Linux +(F3a), and every plan of a project with a bare `compat.*` dependency starts a +network index refresh once the two-minute debounce has passed (T). + +§0 is the ledger. §1 lists where the measured state differs from the issues, +§2 the defects beyond them. §3 states the rules the decisions share. §4 to §8 +take the items by subject. §9 collects the decisions for the reviewer and the +measurements still owed, §10 the order of work, §11 the readings, §12 what +each project can do today, §13 the self-review. + +## 0. The ledger + +"Home" follows the routing rule of the #634 record: **usage** (a spelling or +an existing mechanism), **project**, **official plugin** (mcpp-plugins), +**ecosystem data** (xim-pkgindex, xlings, mcpp-index), **engine**. A defect +stays in the engine whatever its size; the rule filters features, not +defects. "Beyond" marks a finding no issue reports. + +| # | real | general | home | decision (§) | evidence | +|---|---|---|---|---|---| +| F3a | defect, beyond | yes | engine | on ELF, every C++ image a plan loads into one process shares one C++ runtime; a program over a plan-built C++ shared library takes the shared-library contract (§4.1) | measured: llvm default aborts with `std::bad_cast`, exit 134; gcc default runs with 900 interposed libstdc++ symbols | +| F3b | defect (false report) | yes | engine | `STB_GNU_UNIQUE` is vague linkage; a duplicate whose two definitions come from one plan object is not a conflict (§4.2) | measured: 2 (llvm) and 8 (gcc) findings under a uniform contract; `--strict` exits 1 | +| F1 | defect | yes | engine | a static package reachable from exactly one shared image is linked into that image; one reachable from two images is refused with the `linkage = "shared"` remedy (§4.3) | measured: `-z defs` link fails, a foreign `dlopen` fails, ELF interposition hides both | +| F2 | probable defect | yes | engine, measurement first | measure on macos-15; the F3a rule per format decides the change (§4.4) | read: libc++ `typeinfo`; not measured on the payload origin | +| E10 | defect in the record, and a default to decide | yes | engine | step 1: the record and docs state the static CRT the row delivers, and a `cxx_runtime` the row cannot deliver is diagnosed; step 2 (own record, after windows-2022): whether the row adopts `/MD` (§4.5) | measured: the driver's `-defaultlib:libcmt`; read: no CRT flag outside the `msvc` dialect | +| E1 | gap | yes | engine | the root build program reads a graph file (`mcpp::graph_file()`): packages in dependency order with manifest directory, kind, features, link form and verbatim `[package.metadata]`; its digest joins the re-run key (§5.1) | measured: the program sees direct dependencies only; a metadata edit replays the cached run | +| E2 | feature | Apple rows | official plugin (`rules-swift`) | one package's Swift compiles through `action` objects, `link_flag` and a generated header; cross-package Swift modules wait for an engine interface channel (§5.2) | read; no `swiftc` on the host | +| E3 | defect | yes (macOS host, every non-Apple hosted row; latent on Windows) | engine | choose the link branch by the target's object format, in a pure function with a host-by-row unit test that runs on Linux (§5.3) | read line by line; the issue's CI log | +| E5 | defect | yes | engine; plugin workaround retires | strip the program on every row and every shared library the graph built; strip staged toolchain runtime copies; report what was done; expose the decision to build programs (§5.4) | measured: desktop `libdep.so`, Android program, `libdep.so` and `libc++_shared.so` unstripped under "stripped" | +| E9 | gap, and an inconsistency | yes | engine | `mcpp pack --message-format json` prints one `mcpp.pack` envelope; `pack` accepts `--release` and `--dev` with `build`'s precedence (§5.5) | measured: three `unknown option` refusals | +| E4.1 | defect | yes | engine | the forward validator accepts a key declared in any dependency table of the manifest, on any row (§6.1) | measured: warning and `--strict` exit 2 while the forward applies | +| E4.2 | documentation defect, and a silent drop beyond | yes | docs; engine (one check) | the docs say to restate the source; a restatement whose source differs is refused (§6.2) | measured: the restated form works; a different path is ignored under `--strict` | +| E4.3 | defect | yes | engine | one helper names a provider for a consumer; `dep_dir`, `dep_linkage` and `dep_bin` use it (§6.3) | measured: `qualified=[]` | +| E6 | defect | yes | engine | a package with no library target contributes nothing to a consumer's target graph; its programs come from the tool sub-build alone; package cycles are checked at resolution (§6.4) | measured: builds with `--cache=local`; the tool's own dependency is linked into the application | +| E7 | gap | yes | engine | a git dependency whose identity differs from the root manifest's is looked up among the repository root's `[workspace] members`; no `subdir` key (§6.5) | measured: member-only is refused; both keys drop the `tools` request | +| E8 | defect (narrower than stated), and a gap | yes | engine | a `dep/feature` token is a root forward and never a macro; a plain undeclared name without `[features]` keeps its documented meaning (§6.6) | measured: `-DMCPP_FEATURE_SPIKE_FW_INSTALLER` | +| T | defect, beyond | yes | engine; the xlings manifest | the refresh decision walks the resolver's deprecated bare-name rung before it calls a miss (§7.0) | measured on openxlings/xlings | +| A1 | gap | yes | engine | refusal code `offline-download-required` at every offline refusal site; envelope code `MCPP_OFFLINE_DOWNLOAD_REQUIRED` naming the first missing item (§7.1) | read: five prose-only sites | +| A2 | defect | yes | engine | the saved stdout descriptor is close-on-exec (Windows: not inheritable) (§7.2) | measured: a build program holds the caller's pipe as fd 3 | +| A3 | defect | yes | engine; xlings for its own connection timeout | xlings runs under the owned launcher; refresh has a total deadline, install an inactivity deadline (§7.3) | read | +| A4 | defect against docs/50 | yes | engine | a per-run record of observed effects; a launched network child records `network` (§7.4) | read | +| A5 | defect (docs and code disagree) | yes | engine | the three refreshes that bypass `mcpp.pm.index_refresh` go through `decide_for_miss` (§7.5) | read | +| A6 | ecosystem-data defect, beyond; then a default | yes | mcpp-index, then engine | the index artifact becomes byte-reproducible and is republished; then the default `artifact` becomes a region object (§7.6) | measured: GitCode's asset digest differs from the pointer | +| P1 | feature | yes | official plugin | `dist-apple` `options::omit_keys`, restricted to the defaulted keys (§8) | read | +| P2 | feature | yes | official plugin | `dist-web` `options::page`, default `index.html` (§8) | read | + +The mcppls-side measures of #648 (offline by default, its own process group, +a cached last document) belong to the caller and are correct as stated (§7.7). + +## 1. Where the measured state differs from the issues + +1. **F3's 882 symbols are a split runtime, not the module initialiser.** The + default ELF contracts give the program `self-contained` and the shared + library `toolchain-coupled` (`resolution.json`, reading a). Under gcc the + program's static libstdc++ is exported and interposes the library's + `libstdc++.so.6` (900 conflicts, the run succeeds). Under llvm the same + shape aborts when run (exit 134), even when the program itself does not + import `std`. Only under a uniform contract do the initialiser findings the + issue describes remain (§4.1, §4.2). +2. **F1 is described as intended by the code.** `symbol_provision.cppm:36-43` + and the #519 design call the arrangement benign and state that PE and + Mach-O "structurally do not have this problem". That holds only on ELF, and + only for a consumer that linked the static package itself: `-z defs` + refuses the library, and a program that did not link `x` cannot `dlopen` + it (reading a, F1c and F1d). +3. **E10's record is false, and no value of `cxx_runtime` reaches the row.** + The PE contract table treats clang on the MSVC ABI like `cl.exe` and + records `host-coupled` (`distribution.cppm:615-686`), while the driver + links `libcmt` (reading a, E10) because no CRT flag is emitted outside the + `msvc` dialect (`flags.cppm:888-895`). +4. **E1: dependency build programs run dependents first**, in discovery order + (`prepare.cppm:9551-9640`); `[package.metadata.*]` is accepted silently + today, even under `--strict`; and the engine cannot resolve paths inside + metadata it does not interpret. +5. **E3 is wider than Android.** The macOS host branch serves every non-Apple + hosted row whose C library comes from a payload (wasm32-emscripten + included, read), the Windows non-MSVC branch has the same shape + (`flags.cppm:1568-1570`, latent: the NDK publishes no Windows archive), and + no Linux test can see it because the branch is chosen by `if constexpr` + and the Android e2e scripts are gated `# requires: gcc`. +6. **E5: the Android row strips nothing because it never reaches the strip + step.** `run()` dispatches a shared-object program to + `run_shared_program` (`pack.cppm:1733`), which does not call + `strip_program` and ignores `--debug-symbols`. The word "stripped" is the + decision, printed before the outcome (`pipeline.cppm:499-502`). +7. **E4.2: the restated form already works**, and docs/30 states it + correctly (`docs/30:1387-1392`). The defect is docs/05 and its translation, + plus a restatement with a different source that is ignored without a word. +8. **E6: the cycle is in the consumer's own target graph, not in a key.** The + tool package's `[dependencies]` are walked into the consumer graph + (`prepare.cppm:7680-7686`), so the same manifest builds with + `--cache=local` (the only walk that detects cycles runs for the global + cache, `:12790`), and a tool's own library dependency is linked into the + application (reading c, E6c). Keying builds on features does not remove + that edge. +9. **E7: the both-keys spelling drops the member's `tools` request silently** + (`recordDependencyEdge` returns early on a second declaration, + `prepare.cppm:6304-6313`). +10. **E8: accepting any plain name without a `[features]` table is + documented** (docs/06:40-42, pure macro usage). The defect is that a + `dep/feature` token, which can only be a forward, becomes the macro + `MCPP_FEATURE_SPIKE_FW_INSTALLER`. +11. **#648: the refresh was started by a disagreement, not by a stale index** + (§7.0). The pipe leak also reaches children started through + `posix_spawn`, not only `popen` (§7.2). The CN mirror the issue asks for + is deployed, and its artifact does not match its pointer (§7.6). +12. **A1 cannot name every missing item.** Planning stops at the first + refusal; the diagnostic names the first (§7.1). +13. **A3: mcpp already owns its children.** The process group, the signal + guard and the Windows job object were built for ninja; the xlings calls + are the remaining exception (§7.3). + +## 2. Defects beyond the issues + +| # | defect | severity | § | +|---|---|---|---| +| F3a | the default ELF contracts put two C++ runtimes in one process; llvm aborts | a working-looking build that does not run | §4.1 | +| T | the refresh decision disagrees with the resolver on bare `compat.*` names; a network refresh every two minutes | every build of such a project, online | §7.0 | +| A6 | GitCode's index artifact digest differs from the pointer | the CN route falls back to GitHub | §7.6 | +| X1 | a second declaration of one dependency by one consumer loses its `tools`, `features`, `host-module` and `reexport` (`prepare.cppm:6304-6313`) | silent, under `--strict` | §6.5 | +| X2 | package-edge cycles are detected only by the global-cache key walk | the same manifest passes with `--cache=local` | §6.4 | +| X3 | a tool cycle is refused after four nested sub-builds, with a repeated prefix and no edge named (`tool_store.cppm:62`) | diagnostic | §6.4 | +| X4 | `[package]` accepts unknown keys silently, under `--strict` too; `[build]`, `[targets]` and `[test]` do not | a misspelt key is ignored | §5.1 | +| X5 | `run` gives `--release` precedence over `--profile`; `build` does the reverse (`cmd_build.cppm:477-479`, `:109-114`) | one command line, two profiles | §5.5 | +| X6 | a git dependency's compile banner prints `v` with no version (`execute.cppm:886-891`) | cosmetic | §6.5 | +| X7 | `mcpp why deps --features` is `unknown option` | a feature build's graph is visible only in `resolution.json` | §6.6 | +| X8 | a forward to a dependency declared for another row fails `--strict` | a portable manifest cannot be strict | §6.1 | +| X9 | the only qualified-`dep_bin` test uses the legacy dotted name (`tests/e2e/187:170`) | E4.3 was untested | §6.3 | + +## 3. The rules the decisions share + +The items are many; the rules they violate are few. Each decision below is an +application of one of these, and a reviewer who rejects a rule rejects its +decisions together. + +- **R1. One process, one C++ runtime; one static package, one image.** A + type's identity, a locale facet's id and a static package's state exist + once per process only if one image defines them. F3a, F2 and E10 are the + runtime instance; F1 is the general instance. #641's refusal + `shared-library-cxx-runtime` is the precedent: it enforced the rule for a + graph runtime and left the payload's defaults and ordinary static packages + outside it. +- **R2. One question, one derivation.** Every "second copy" in this batch + answers a question another function already answers: the refresh decision + and the resolver (T), the refresh policy and three bypasses (A5), the + forward validator and the injector (E4.1), the tool variable and + `fillDepDirs` (E4.3), the host branch and the target format (E3), the CLI + feature parser and the manifest forward (E8), `run` and `build` profile + precedence (X5). The fix in each case deletes a derivation rather than + correcting it, because a corrected copy drifts again. +- **R3. A child mcpp starts is owned, bounded, and inherits only what it + needs.** The launcher that owns ninja and build programs already exists. + A2 and A3 move the remaining children onto it and close the one descriptor + that escapes it. +- **R4. What mcpp reports is what happened.** The envelope's `effects` (A4), + its error codes (A1), pack's status line (E5), pack's artifacts (E9), the + runtime record (E10) and the symbol-provision check (F3b) each state + something the build did not do, or omit something it did. docs/50 already + states the rule for `effects`; this batch applies it to the rest. +- **R5. The engine states facts; plugins build products.** A graph with + metadata (E1), a strip decision (E5) and the existing `action` and + `link_flag` channels (E2) are facts and mechanisms; merging resources, + packaging an APK, compiling Swift and writing an Info.plist are products + (E1's consumers, E2, P1, P2). +- **R6. Published data keeps loading on older clients.** A manifest shape + that older clients refuse cannot be introduced to express something the + current grammar already expresses (E4.2), and a new key that older clients + ignore must not change what is built when ignored (E7's rejected `subdir`). + E1's metadata table is admissible under this rule precisely because older + clients already ignore it. + +## 4. #646 and #649 E10: images, runtimes and placement + +### 4.1 F3a: the default ELF contracts put two C++ runtimes in one process + +**Measured (group a).** A program over a `kind = "shared"` C++ library, both +importing `std`, with `std::format` in the library: + +``` +F3 gcc contracts in resolution.json: {"distributable": "self-contained", ..., "shared-library": "toolchain-coupled", ...} +F3 gcc NEEDED program: [liblib.so] [libm.so.6] [libgcc_s.so.1] [libc.so.6] NEEDED liblib.so: [libstdc++.so.6] ... +F3 gcc run: lib-3 exit=0 bin/app conflicts=900 +F3 llvm run: libc++abi: terminating due to uncaught exception of type std::bad_cast: std::bad_cast exit=134 +F3 llvm-no-println run: ... std::bad_cast exit=134 +F3 llvm-tc (cxx_runtime = "toolchain-coupled") run: lib-3 exit=0 +F3 llvm-static-lib (kind = "lib") run: lib-3 exit=0 +``` + +The lead reproduced the llvm abort independently, through `mcpp run` as well. + +**Mechanism (read).** `default_contract` returns `toolchain-coupled` for an +ELF shared library since #414, which stopped shared libraries from embedding +libstdc++ and hijacking the program's runtime, and `self-contained` for an ELF +program. Neither default is wrong alone; together they place a static C++ +runtime in the executable and a shared one in the library. The executable +exports the runtime symbols the library references, so the library binds +part of its runtime to the program's copy and part to its own. The `_Unwind_*` +symbols among the 46 llvm conflicts are the partial interposition already +recorded for the unwinder in 2026-08; the `bad_cast` is probably a locale +facet looked up by an id that exists twice, which was not isolated. + +- **Kind:** defect, general to every ELF project whose program depends on a + C++ shared library it builds. It is the ELF instance of F2. +- **Home:** engine (`distribution.cppm`). +- **Decision (R1).** When a plan's root image links a C++ shared library that + the plan builds, the contracts of its programs and tests are derived from + the shared-library contract (`toolchain-coupled`) instead of the + per-role default. An explicit `cxx_runtime = "self-contained"` on the + program in such a graph is refused before compiling, with a new refusal + code beside `shared-library-cxx-runtime`, naming the library and the split. + Unlike #641 M3, no refusal is needed for the default: a mechanism that + delivers one runtime exists. +- **Cost.** A gcc program over a C++ shared library gains `NEEDED + libstdc++.so.6`. The process already requires that file, through the + library's own `NEEDED`, so no deployment gains a requirement; the change is + where the program binds. +- **Scope.** The first criterion covers shared libraries the plan builds. A + prebuilt C++ shared library from the store is the same class and is + detectable from its `DT_NEEDED`; it is named here and not decided, because + its detection runs after linking rather than at planning. +- **Criterion.** The llvm default shape runs (today exit 134); the gcc default + shape reports no libstdc++ conflicts; `cxx_runtime = "self-contained"` on + the program is refused with the code. + +### 4.2 F3b: the symbol-provision check reports what the loader unifies by design + +**Measured (group a).** Under a uniform contract the llvm shape reports two +conflicts (`_ZGIW3std`, `_ZGIW3stdW6compat`) and the gcc shape eight (the +initialiser and seven `STB_GNU_UNIQUE` objects such as +`__from_chars_alnum_to_val_table::value`). Each finding is +`diag::degraded` (`ninja_backend.cppm:3283-3297`), so `--strict` exits 1 on +every leg. The initialiser in `std.o` is one global function whose body is a +return (gcc) or a frame push, pop and return (clang): no guard, no state. + +- **Kind:** defect (a false report, R4), general. +- **Home:** engine (`symbol_provision.cppm`, `src/runtime/elf.cppm:742`). +- **Decision.** (1) `STB_GNU_UNIQUE` counts as vague linkage, as `STB_WEAK` + already does. (2) A duplicate is not reported when both definitions come + from one object of this plan linked into both images; the decision is by + provenance, not by a `_ZGIW` name pattern, so a real duplicate with an + initialiser-shaped name is still reported. Today that object set is + `std.o` and `std.compat.o`. +- **Rejected: one owner of `std.o`.** An image without the definition cannot + link on Mach-O or PE and cannot be loaded by a host that did not link the + owner (F1c and F1d prove the same for any symbol); sibling shared libraries + have no common owner; and the object holds nothing worth owning. +- **Criterion.** Under a uniform contract, no `build/symbol-provision` finding + and `--strict` exits 0; a static zlib in the program over a shared `libz` + still reports. + +### 4.3 F1: a static dependency of a shared package goes into the program + +**Read.** A dependency-owned shared unit receives its own package's objects +and its direct shared dependencies (`plan.cppm:1914-1929`); the root's units +receive every compile unit of every non-shared package +(`plan.cppm:1994-2000`, `2100-2115`). + +**Measured (group a).** + +``` +F1a libfw.so inputs: obj/mcpplibs_fw/src/fw.o undefined x_answer=1 program exports x_answer=1 run exit=0 +F1c libfw.so relinked with -z defs: exit=1 undefined reference to `x_answer' +F1d foreign host dlopen(libfw.so, RTLD_NOW): exit=1 ... undefined symbol: x_answer +F1b root also depends on x: x.o linked into program=1 libfw.so=0; run counter=2 +F1f libfw.so relinked with x.o as well: libfw defines x_counter=1; run counter=2 +F1e root-owned shared image over x: libfw.so defines x_answer=1 undefined=0 +``` + +F1c is the Linux form of the Mach-O `-undefined error` and PE link claims; +F1d is the Linux form of Android's `System.loadLibrary("fw")` before the +application, and of any consumer outside mcpp. F1f shows that ELF also hides +the state split a duplicate would cause elsewhere: the interposed copy wins +and the counter reads as shared. + +- **Kind:** defect, general (every format but ELF-with-this-consumer). +- **Home:** engine (`make_plan`). Placement is plan logic; no cheaper home + exists. +- **Decision (R1).** For each shared image, its static closure is the set of + static packages reachable from its package without crossing another shared + package; `directPackageDeps` (`plan.cppm:1800-1827`) already holds the + edges. A static package in exactly one closure is linked into that image and + removed from the root's object set. A static package in two or more closures + (two images, or an image and the program) is the "one library, one provider, + one form" conflict and is refused before compiling, with a refusal code + naming the package, the images, and the remedy that `linkage_form` already + implements: `linkage = "shared"` on that package, per dependency or as its + default. Silent duplication is rejected because on Mach-O and PE each image + would keep its own state, which F1f shows ELF would hide. +- **Consequence.** The comment at `symbol_provision.cppm:36-43` is rewritten; + the arrangement it calls benign is removed. +- **Criterion.** On Linux, M3b's `libfw.so` links under `-Wl,-z,defs` and + loads from a foreign host; F1b is refused naming `x` and builds after + `linkage = "shared"` on `x`; on macos-15, M3b's dylib links (temporary + pull request, §9.2). + +### 4.4 F2: the payload's Mach-O shared default + +**Read.** On macOS the deployment floor is never empty (`macos.cppm:91, +105-111`), so programs and dylibs alike are `self-contained` and embed +`-Wl,-load_hidden,libc++.a` and `libc++abi.a` +(`distribution.cppm:214-251, 572-593`); `toolchain-coupled` is refused on +Mach-O (#202). The payload's `include/c++/v1/typeinfo:146-187` states that on +arm64 Apple types are "considered different types" across linked image +boundaries under the non-unique RTTI bit, and x86_64 Apple compares unique +type information by address. The inference is that a `std::runtime_error` +thrown in a dylib is not caught by its class in the program, and that any +libc++ object compared by address, such as an `std::error_code` category, +compares unequal across images. + +- **Kind:** probable defect, general; working builds depend on the default. +- **Home:** engine, after a measurement. +- **Decision.** No change before the measurement. The measurement (§9.2) runs + the #641 M3 app/fw pair without `llvm.libcxx` on macos-15 against the + released tarball, in three legs: the default; `cxx_runtime = "host-coupled"` + project-wide (both images on `/usr/lib/libc++.1.dylib`); and the default + program over a `{ shared = "host-coupled" }` library. The reading names + whether `std::runtime_error`, the library's own exception type and + `std::errc` comparisons survive the boundary. If the default leg splits, + R1 applies per format: on Mach-O the only runtime shared across images is + the system's, so a graph with a C++ dylib couples every image to it, subject + to the floor, and an explicit `self-contained` program receives a + diagnostic. Because working builds change, the first release warns, as the + rule-E precedent did. +- **PE.** The `msvc` dialect defaults to `/MD` and shares one CRT across DLLs + (`distribution.cppm:615-620`). The llvm row gives every image a static CRT + (§4.5); MSVC exception handling matches catch types by decorated name + (read), so the type split probably does not occur there, while per-image + CRT state (`FILE*`, `errno`, locale, `atexit`) does. The windows-2022 leg of + the same fixture answers it. + +### 4.5 E10: the llvm row's runtime on `x86_64-windows-msvc` + +**Measured (group a, the driver's decision is host-independent).** + +``` +E10 compile, no flag: dependent-lib= defines= +E10 link, no flag: -defaultlib:libcmt -defaultlib:oldnames +E10 -fms-runtime-lib=dll: compile --dependent-lib=msvcrt --dependent-lib=oldnames link -defaultlib:libcmt -defaultlib:oldnames +``` + +**Read.** The CRT flag is emitted only for `isMsvcDialect` +(`flags.cppm:888-895`); the clang std module path never takes +`msvc_crt_flag` (`stdmod.cppm:330-346`), whose comment says non-MSVC dialects +yield `""` while `msvc_crt_flag(gnu, true)` returns `"-static"` +(`dialect.cppm:67, 134-135, 211`). The PE contract table records +`host-coupled` for the row (`distribution.cppm:615-686`), and runtime staging +is guarded on `CompilerId::MSVC` (`flags.cppm:1370`). + +The issue's three questions, answered: + +1. The static runtime is not a decision. It is clang's link-time default + (`-defaultlib:libcmt`), and without `_DLL` the MSVC STL headers select the + static C++ library, which matches the imports HuxerUI read. The record says + otherwise. +2. No value selects the dynamic runtime. `host-coupled` and + `toolchain-coupled` are recorded, not delivered, with no diagnostic. +3. Every llvm-row DLL carries its own CRT and STL, so the row is in F2's class + for CRT state; whether exception identity also splits is the windows-2022 + measurement. + +- **Kind:** defect (R4: the record and the artifact disagree), and a default + that deserves a decision. General. +- **Home:** engine. +- **Decision, in two steps.** Step 1 changes no artifact: for clang on the + MSVC ABI the record states `self-contained` unless a runtime flag is + emitted; an explicit `host-coupled` or `toolchain-coupled` receives a + degraded diagnostic saying the row does not deliver it; the `msvc_crt_flag` + comment is corrected; docs/20 states the row's model. Step 2, in its own + record after the windows-2022 measurement, decides whether the llvm row + adopts the `cl.exe` model (`/MD` by default through `-fms-runtime-lib`, the + same `msvc_wants_static_crt` derivation, the std module included, the + redistributable staging extended to the row). Step 2 is CMake parity and + changes every existing llvm-row artifact, which is why it is not taken here. +- **Criterion (step 1).** `resolution.json` for the default llvm row on + windows-2022 records `self-contained`, and `cxx_runtime = "host-coupled"` + prints the diagnostic. + +## 5. #647 E1 to E3 and #649 E5, E9: what build programs see, and what pack does + +### 5.1 E1: the resolved graph and package metadata + +**Read.** The root build program runs at step L3 +(`prepare.cppm:10710-10808`), after resolution, feature activation, every +dependency's build program and link-form resolution, so the whole graph is +known when it runs. `fillDepDirs` publishes only the consumer's visible +provisions: direct dependencies and reexports, under the bare and the +qualified name (`:6016-6055`). `resolution.json` receives a `graph` section +(`:13479-13505`, the #634 X decision) only at the end of `prepare`, and a +build program never sees it. A build program re-runs when its program, its +compiler, its contract environment (`contract_hash`, +`build_program.cppm:721-727, 971-972`) or a declared input changes; nothing +about a transitive package is in that key. + +**Measured (group b).** + +``` +E1-root-env: SPIKE deps: MCPP_DEP_A_DIR MCPP_DEP_SPIKE_A_DIR MCPP_DEP_A_LINKAGE MCPP_DEP_SPIKE_A_LINKAGE +E1-program-order: SPIKE order a SPIKE order b +E1-metadata-edit: build.mcpp up to date (cached)x3 replayed=yes +E1-strict-metadata-table: exit=0 warnings=0 +E1-unknown-package-key-strict: exit=0 mentions=0 +``` + +- **Kind:** gap, general: the capability CMake target walks and + `cargo metadata` provide. +- **Home:** engine. A plugin cannot enumerate transitive packages: registry + and git roots are invisible to it, and `dep_dir` is scoped to provisions on + purpose. #634 A8 deferred this "until a need transitive deploy cannot + answer"; merging every library's resources into one package at build time + is such a need, because deploy stages files and does not declare a build + action over them. +- **Decision (R5).** + 1. The root build program receives `mcpp::graph_file()` + (`MCPP_GRAPH_FILE`), an absolute path to a JSON document written before + L3. It holds the per-package objects of `resolution.json`'s `graph`, + extended with `manifest_dir`, active `features`, target kinds and the + resolved link form, in topological order (dependencies first, ties in + discovery order). A file rather than variables, because a graph with + metadata exceeds what `MAX_ARG_STRLEN` and the Windows environment + block allow; the precedent for a path a program reads is + `pack_stage_dir()`. Root only, as `dep_linkage` is: the root holds the + final facts. The JSON reader `mcpp-plugins/dist/apk.cppm:1027-1135` + moves into a shared plugin module so rules read the file with one + parser. + 2. `[package.metadata.]` is copied verbatim into that package's + entry as `metadata`. The engine does not interpret it; a consumer + resolves paths against `manifest_dir`. Older clients already ignore the + table (X4), so publishing it breaks none of them (R6). + 3. The document's digest joins the contract environment, so a change of any + graph package's identity, features or metadata re-runs the root program, + and an edit to a package's sources does not. + 4. The open question (a dependency's program emitting values for its + dependents) is deferred: manifest metadata serves both stated needs, and + such a channel first requires dependency programs to run dependencies + first, which they do not (E1-program-order). +- **X4.** When `[package]` gains a known-key check, `metadata` is in its list; + the check itself is a separate, compatibility-sensitive change and warns + before it refuses. +- **Criterion.** The issue's fixture: the root program prints `spike.b` after + `spike.a` with `b`'s resolved metadata; editing `b`'s metadata changes the + run, editing `b/src/b.cpp` does not. + +### 5.2 E2: Swift sources + +**Read.** The inferred glob has no `.swift`, and nothing in mcpp or +mcpp-plugins mentions it. `rules-metal` is the precedent for a language that +lives in a rule package: its feature declares +`device_extensions = [".metal"]` (`mcpp-plugins/mcpp.toml:91-95`), the rule +receives its sources through `mcpp::device_sources()` and locates tools with +`xcrun --sdk --find`. The engine already offers what one package needs: +an `action` whose role `object` joins the image link, a `source` action with +companion outputs for a generated header, `link_flag` (LinkGlobal, +`directives.cppm:320-340`), and `include_dir` (PackagePrivate by design, +`:28-34`). + +- **Kind:** feature, Apple rows. +- **Home:** official plugin, `rules-swift`. The issue's CI check (one + package, `@_cdecl` exports, Swift calling C through a bridging header) needs + no engine change: one whole-module `swiftc -emit-object -target ` + action, a generated `-Swift.h` through a `source` action and `include_dir`, + and `link_flag` for the Swift library search paths and + `-Wl,-rpath,/usr/lib/swift`. +- **What a plugin cannot do.** A Swift `import` of another package's module, + and a C++ consumer in another package reading a generated header, both need + an interface directory that dependents can see. Compile-interface directives + are PackagePrivate on purpose, and dependency programs run dependents first + (§5.1). That engine capability is named here and not designed; it is E1's + deferred question in another form. +- **Unmeasured (macos-15).** Whether `ld64.lld` under `-fuse-ld=lld` honours + the `LC_LINKER_OPTION` autolink entries of swiftc objects + (`swiftCompatibility*`, `swiftCore`), and the `aarch64-ios-sim` link. + +### 5.3 E3: an Android row does not link on a macOS host + +**Read, line by line.** `crossTarget` is `--target=` +(`flags.cppm:553-555`), and an own-sysroot row carries it only in +`link_toolchain_flags` (`:729-754`). The host branch is chosen by +`if constexpr`: Windows at `:1561`, macOS at `:1637`. The macOS branch +assembles `f.ld` from `full_static`, `b_flag`, `apple_cross_ld`, `macos_sdk`, +`version_min`, `-fuse-ld=lld` and the user and extra flags (`:1753-1756`), and +never appends `link_toolchain_flags`; `apple_cross_ld` is set only for an +Apple SDK row (`:1748-1752`); `macos_sdk` falls back to the host SDK +(`:1707-1711`); `version_min` is added whenever the row is not an Apple +cross (`:1683-1685`). An Android link therefore has no `--target`, which +selects `ld64.lld`, and carries the macOS `-isysroot` and +`-mmacosx-version-min`. The issue's cause is exact. Freestanding rows and +graph-supplied C-library rows replace the line after the host branch +(`:1841`, `:1958`) and are not affected. + +- **Kind:** defect (R2: the host answers a question that belongs to the + target), general. +- **Home:** engine. +- **Decision.** The link branch is chosen by host and target object format, + not by host alone: the Apple SDK branch applies to Mach-O targets, the + Windows `lld-link` branch to PE targets (MSVC keeps `link.exe`), and every + other target takes the generic branch that consumes + `link_toolchain_flags`. The choice is a pure function, for example + `link_shape(host_os, triple, dialect)`, with a unit test over host by row + that runs on Linux (macOS host and `x86_64-linux-android` gives the target + driver; macOS host and `aarch64-ios-sim` gives the Apple SDK; Windows host + and `x86_64-linux-android` gives the target driver). The generic branch is + already compiled on every host, since a discarded `if constexpr` branch in + non-template code is still checked. +- **Criterion.** The unit test above; on macos-15, + `mcpp build --target x86_64-linux-android` produces a file `file` reports + as ELF, and mcpp-plugins restores its macOS `dist-apk` steps. + +### 5.4 E5: pack strips the program of some rows and nothing the graph built + +**Measured (group b).** + +``` +E5-desktop-status: Packing hostapp v0.1.0 (vendored, stripped) +E5-desktop: bin/hostapp bytes=14280 symtab=0 debug=0 +E5-desktop: lib/libdep.so bytes=16592 symtab=1 debug=7 +E5-android-status: Packing app v0.1.0 (vendored, stripped) +E5-android: lib/libapp.so bytes=5904 symtab=1 debug=1 +E5-android: lib/libc++_shared.so bytes=9091400 symtab=1 debug=6 +E5-android: lib/libdep.so bytes=5496 symtab=1 debug=1 +``` + +**Read.** `strip_program` strips the staged program with the `Executable` +shape (`pack.cppm:818-837`) and is called from the PE and ELF paths +(`:1503`, `:2079`); `bundle_libs` copies closure libraries, graph-built ones +included, unchanged; the Android row's dispatch skips the strip step entirely +(§1 item 6). The Mach-O row does not strip on purpose (`:1774`). The NDK's +`llvm-strip` is already resolved by `binutils_tool` +(`registry.cppm:1446-1450`). + +- **Kind:** defect (a dropped row, and a status line that reports a decision + as an outcome, R4), general. +- **Home:** engine; `dist-apk` 0.11.1's own strip becomes unnecessary. +- **Decision.** + 1. The program is stripped on every row that strips, inside + `run_shared_program` too, per leg with that leg's tools, using the + `SharedLibrary` shape when the program is a shared object. + 2. Every shared library this graph built is stripped in its staged copy + with `--strip-unneeded`, which keeps `.dynsym`. `pack::Plan` receives the + absolute outputs of `plan.linkUnits` whose kind is `SharedLibrary`. The + comment's rule ("a bundled `.so` is somebody else's file") stays true for + store and host files; the dh_strip rule it quotes, "a package strips + what it built", covers these. + 3. A staged copy of the toolchain's runtime (`libc++_shared.so`) is stripped + with `--strip-unneeded`. The comment's reason, not changing a shared + payload's bytes, does not apply to a copy; Gradle strips it. `--no-strip` + covers every case. + 4. The status line reports what was done: a row that stripped nothing does + not print "stripped". + 5. Build programs read the decision through `mcpp::pack_strip()` and + `mcpp::pack_debug_symbols_dir()`, in the contract environment beside + `pack_format()` and `pack_stage_dir()` (`hostprogram.cppm:544-562`), for + the AAR and Maven libraries `dist-apk` stages itself. +- **Criterion.** The issue's three checks; they fail today per the readings. + +### 5.5 E9: a machine-readable pack report, and the profile switches + +**Measured (group b).** + +``` +E9-pack --release: exit=2 [error: unknown option: --release] +E9-pack --dev: exit=2 [error: unknown option: --dev] +E9-pack --message-format json: exit=2 [error: unknown option: --message-format] +E9-human-lines-on-stdout: bytes=273 +E9-run-precedence(--profile dev --release): Finished release +E9-build-precedence(--profile dev --release): Finished dev +``` + +**Read.** `PackOutcome` already holds absolute artifact paths and moves the +outputs another action consumed into `intermediate` +(`pipeline.cppm:34-46, 693-719`); only `shorten_path`'s form is printed. +docs/50 §3 reserves `--format json` for machine output, while `pack`'s +`--format` names the package format; `test --message-format json` is an NDJSON +event stream; `emit build-database` sends its narration to stderr. + +- **Kind:** gap and inconsistency, general. +- **Home:** engine; only the engine holds `PackOutcome`. +- **Decision.** `mcpp pack --message-format json` prints one enveloped + `mcpp.pack` document on stdout and sends every human line to stderr, as + `emit build-database` does. `data.artifacts[]` holds + `{path, type: "file" | "directory", format, targets[]}` with absolute paths; + legs appear as `targets` of the artifact they produced, and intermediate + outputs are omitted; `data.stage` holds the stage directory, manifest and + closure. `--protocol-version` advertises the kind. docs/50 §3 gains one + sentence: a command whose `--format` names its product asks for machine + output with `--message-format json`. The shape follows the kind: a stream + for `test`, whose events arrive over time, and one document for `pack`, + which has one result. `pack` accepts `--release` and `--dev`; the pack + default stays `release`. +- **X5.** `run` adopts `build`'s precedence (`--profile` over the + shorthands), one derivation for the three commands (R2). +- **Criterion.** The issue's two checks, and `mcpp run --profile dev + --release` finishing `dev`. + +## 6. #647 E4 and #649 E6 to E8: features, tools and git sources + +### 6.1 E4.1: a forward along a build-dependency edge is applied and reported as undeclared + +**Read.** The validator looks the key up in `dependencies` and +`devDependencies` (`prepare.cppm:6677-6678`), while `injectForwards` applies +forwards to `buildDependencies` for the root (`:6718-6728`) and for +transitive packages (`:7690-7698`). Active `[feature-deps]` are merged into +`dependencies` before validation (`:6693`, `:7611`), and conditional tables +are folded into the same maps (`:497-506`). + +**Measured (group c).** + +``` +E4.1-build: 2 forward warnings; tool kt built: 1; rc=0 +E4.1-strict: error: feature 'kotlin' of 'fw' forwards to dependency 'spike.rules' ... rc=2 +E4.1-control-without-feature: tool kt built: 0; rc=0 +E4.1b-forward-to-other-row-dep: error: feature 'kotlin' of 'fw' forwards to dependency 'spike.win' ... rc=2 +``` + +- **Kind:** defect (R2), general; X8 is its conditional-row twin. +- **Home:** engine. +- **Decision.** The validator asks the question the forward language + defines: is the key declared in any dependency table of this manifest, + on any row and under any feature? `conditionalConfigs` survives the merge + (`types.cppm:1774`), so the check can read every row. A declared key that + is not in the graph on this row does nothing and says nothing; a key + declared nowhere keeps the #243 warning and its `--strict` error. +- **Criterion.** E4.1 and E4.1b build under `--strict` without a warning; a + forward to a key declared nowhere still exits 2. + +### 6.2 E4.2: `[feature-deps]` and a source-less `tools` entry + +**Read.** docs/05 (`docs/05-dependencies.md:344-346`, `zh/05:304`) says a +feature-deps entry "may add `tools`" to a dependency declared elsewhere. The +parser tells a dependency table from a namespace table by the presence of a +source key (`toml.cppm:1383-1389`) and refuses an options-only table with a +message that prescribes restating the source (`:1400-1440`). The merge +already implements the intent of #359: `tools` and `features` are unioned, +`host-module` and `reexport` are or-ed, and identity fields are dropped +(`prepare.cppm:6603-6631`). docs/30 states the restated form. + +**Measured (group c).** + +``` +E4.2-sourceless: error: ... must be a string, inline dep table, or nested table ... rc=2 +E4.2-restated: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/installer@.../bin/installer] qualified=[]; rc=0 +E4.2-restated-without-feature: SPIKE short=[] qualified=[]; rc=0 +E4.2-restated-different-path: (--strict) SPIKE short=[.../installer...]; rc=0 +``` + +- **Kind:** documentation defect; plus a silent drop (a restatement whose + source differs from the declaration in effect is ignored). +- **Home:** docs; engine for the check. +- **Decision (R6).** The documentation changes, not the parser. A + source-less table would change the parser's discriminator, and a published + package that used it would fail to load on every older client, which is the + hazard the parser's own comment describes (`toml.cppm:1370-1382`). + Restating the source is already the grammar for conditional dependencies + (#634 A1) and is what the refusal prescribes. `mergeActiveFeatureDeps` + refuses a restatement whose path, git source or version differs from the + declaration in effect on that row, after the conditional fold. +- **Criterion.** E4.2-restated-different-path exits 2 naming both sources; + docs/05 and its translation say to restate the source. +- **Read, not measured.** Feature-deps merge into `dependencies` + (`prepare.cppm:6609`), so a feature cannot scope a build-only dependency; + restating a `[build-dependencies]` key under a feature probably moves it to + the target axis. This is recorded for the implementation, not decided. + +### 6.3 E4.3: the qualified `dep_bin` spelling is missing + +**Read.** Tool variables are published at `prepare.cppm:9226-9233`, from +`manifest.package.name`, whose short form is taken after the last dot. For a +manifest with `namespace = "spike"` and `name = "installer"`, the name has no +dot, the qualified and short spellings coincide, and only +`MCPP_DEP_INSTALLER_BIN_*` is emitted (`:9272-9289`). #642 fixed the same +shape for `dep_dir` and `dep_linkage` in `fillDepDirs` (`:6036-6044`, through +`qualified_package_name`, `plan.cppm:435`); the tool site is a second +publication point that fix did not reach. The only test uses the legacy +dotted name (X9). + +- **Kind:** defect (R2), general. +- **Home:** engine. +- **Decision.** One helper returns the names under which a provider is + published to a consumer (qualified, canonical, and bare when bound), and + both `fillDepDirs` and the tool record use it. `bareBindingsFor` + (`:5838-5848`) reads `package.name` as well; two packages with one name in + different namespaces probably collide on the bare variable (read), which + the helper's test covers. +- **Criterion.** e2e 187 gains a `namespace =` plus `name =` package, and + `dep_bin("myns.tp", ...)` answers. + +### 6.4 E6: a feature-gated tool that depends on the package declaring it + +**Read.** The error is raised by the build-cache key walk +(`prepare.cppm:12861-12930`), which runs only for the global cache +(`:12790`). The cycle is in the consumer's graph: `fw-installer` is merged +into `fw.dependencies` (`:6609`), the worklist walks the tool package's own +`[dependencies]` into the consumer's graph (`:7680-7686`), and `spike.fw` +meets the resolved `fw`, recording `fw-installer -> fw` +(`:7266-7270`). The tool itself is built by a sub-build "with its own +resolution" (`:8876-8881`) and succeeds. + +**Measured (group c).** + +``` +E6-issue-fixture: SPIKE short=[.../fw-installer] qualified=[] | error: dependency cycle through package 'fw' while computing its build-cache key; rc=2 +E6-tool-runs: fw-installer ran 42 +E6-cache-local: rc=0; app runs: exit=0 +E6c-tool-dependency-in-consumer: app link line: obj/main.o obj/spike_z/src/z.o obj/spike_fw/src/fw.o; app defines z symbol: 1; rc=0 +E6b-tool-activates-declaring-feature: ... tool provisioning nested more than 4 levels deep — this is almost certainly a cycle.; rc=2 +``` + +E6c is the wider defect: when the tool depends on a library `z`, `z.o` is +linked into the application, and the tool's `main.cpp` is compiled in the +consumer's build. + +- **Kind:** defect, general. +- **Home:** engine. +- **Decision.** A package that declares no library target has nothing to link + and contributes nothing to a consumer's target graph: its `[dependencies]` + and `[build-dependencies]` are not walked into that graph and its sources + are not compiled there. Its programs come from the tool sub-build, which + resolves them on its own; that is the model the sub-build's comment already + states, and the shape of Cargo's artifact dependencies and Bazel's exec + configuration. A package with no `[targets]` (whose inferred library + compiles into the consumer) keeps today's behaviour. This removes the cycle + edge, the link leak and the tool's participation in the consumer's version + resolution, with no change to any key. +- **Rejected: keying builds on (package, features, toolchain).** The tool + store's key already carries features, host triple and compiler identity + (`tool_store.cppm`), and splitting target-graph nodes by feature set would + contradict one identity per graph with features unioned (the + one-package-one-version rule). +- **X2 and X3.** Package-edge cycles are checked at resolution, in one place, + with a message naming the edges, so `--cache=local` and the global cache + agree. A repeated (package source, tool) in the tool chain is refused at + its first repetition, naming the feature edge, instead of after four + nested sub-builds. +- **Criterion.** The issue's fixture exits 0 with the default cache, `dep_bin` + answers, and the tool prints `fw-installer ran 42`; E6c's application link + line has no `spike_z`; E6b is refused once, naming the edge. +- **Blast radius to measure.** Before implementation, the manifests of the + e2e corpus, the examples, mcpp-index and mcpp-plugins are scanned for a + dependency on a package whose targets are all programs and which is not + requested for `tools`; any such edge is a consumer that relied on the leak. +- **A modelling note, not a decision.** `tools` builds for the host. + HuxerUI's installer interface is a program for the Windows target, so the + channel serves it only when host and target coincide, which holds for a + Windows host packaging a Windows setup. + +### 6.5 E7: a git dependency resolves only the repository's root package + +**Read.** A git dependency's root is the clone root +(`prepare.cppm:7376-7455`); its identity is url plus ref (`sourceRefOf`, +`:6736-6744`), so a second key over the same source takes the first key's +identity (#634 A2, `:6800-6829`). Declaring the member alone fails the name +check (`:7536-7549`), and membership is tested against the consumer's +workspace, not the repository's (`:7477-7483`). + +**Measured (group c).** + +``` +E7-both-keys: 1 adoption warning(s) | SPIKE short=[] | Compiling spike.fw v | rc=0 +E7-member-only: error: dependency 'spike.fw-installer' resolved to package 'fw' (mismatch with declared name 'fw-installer'); rc=2 +E7-member-by-path: SPIKE short=[.../fw-installer...] | rc=0 +EDUP-dependencies-and-build-dependencies: tool built: 0 | SPIKE short=[] | rc=0 +``` + +- **Kind:** gap, general to multi-package repositories; X1 and X6 beside it. +- **Home:** engine. +- **Decision (R6).** A git dependency whose key names an identity other than + the root manifest's is looked up among the repository root's + `[workspace] members`, at the same commit. The source identity becomes + (url, commit, member path); the member inherits the root's + `[workspace.package]`; a member's `path` edge that stays inside the clone + resolves as the same git source at that commit, so `spike.fw = { path = + ".." }` inside the member and `spike.fw = { git = ... }` in the application + name one package. The #634 A2 adoption applies only when no member declares + the key's identity. Cargo resolves a git dependency by package name within + the repository in the same way. +- **Rejected: a `subdir` key.** An older client ignores an unknown key and + would build the root package, which is a silent wrong build, not a + degradation; and a path is a second statement of identity that can disagree + with the key. +- **X1.** A second declaration of one dependency by one consumer merges into + the existing edge additively, as `mergeActiveFeatureDeps` already does, + instead of returning early. +- **X6.** The banner prints the commit for a git source, or the manifest's + version. +- **Criterion.** E7-member-only and the both-keys form build the member's tool + with no identity warning; EDUP builds the tool. + +### 6.6 E8: `--features` and a dependency's feature + +**Read.** `parse_feature_request` splits on commas and spaces +(`prepare.cppm:650-661`). The unknown-name check applies only when the root +declares `[features]` (`:8531-8548`), and docs/06:40-42 documents that a +package without the table "accepts any request (pure macro usage)"; each name +becomes `-DMCPP_FEATURE_` (`:8348`). + +**Measured (group c).** + +``` +E8-dep-feature-no-table: SPIKE short=[] qualified=[] | tool built: 0; rc=0 +E8-dep-feature-no-table-macro: -DMCPP_FEATURE_SPIKE_FW_INSTALLER +E8-unknown-no-table: rc=0; macros: -DMCPP_FEATURE_NOTHING_HERE +E8-dep-feature-with-table-strict: error: --features requests 'spike.fw/installer' which [features] does not declare; rc=2 +E8-manifest-forward-control: SPIKE short=[.../fw-installer...] | tool built: 1; rc=0 +``` + +- **Kind:** a defect narrower than the issue states (a token that can only be + a forward becomes a macro), and a gap (Cargo accepts `--features + dep/feature`). +- **Home:** engine. Today's usage-side answer is the root forward line of + the control reading. +- **Decision (R2).** A token containing `/` is split with + `split_feature_forward_token` and applied through `injectForwards` and + `validateForwards` as a forward of the root, over the table set of §6.1. + One that names no dependency is warned about, and refused under `--strict`, + whether or not the root declares `[features]`; it never becomes a macro. + Plain names keep their documented meaning. `pack`, `run` and `test` inherit + the behaviour through the shared `overrides.features`. +- **X7.** `mcpp why deps` accepts `--features`, so the graph of a feature + build is visible where the graph of a default build is. +- **Criterion.** E8-dep-feature-no-table builds the tool and `dep_bin` + answers, with no `MCPP_FEATURE_SPIKE_FW_INSTALLER`; `--strict --features + nope/x` exits 2. +- **Open.** Whether a forward, in a manifest or on the command line, may name + a dependency by the qualified identity after A2 adoption as well as by its + key. + +## 7. #648: an editor that plans in the background + +The issue's six asks are three defects of one kind (children that are not +owned or bounded), two contract gaps of the machine interface, and one +ecosystem-data item. The trigger of the field hang is a seventh finding that +the issue does not name, and it is the only one that sends a project to the +network on every build. + +### 7.0 T: what started the refresh on openxlings/xlings + +The issue establishes that an offline plan of the same project is complete +and byte-identical, so the refresh was not needed. It does not establish which +decision asked for it. Planning openxlings/xlings with `MCPP_OFFLINE=1 -v` +prints the refresh decision per dependency, and the decision for `ftxui` is +the suppressed kind, which `decide_for_dependency` assigns only after it has +decided to refresh (`index_refresh.cppm:237-280`). The same run then resolves +the dependency: + +``` +T decision ftxui@6.1.9: offline mode +T resolver warning: dependency 'ftxui' resolved to 'compat.ftxui' through the deprecated bare-name search; namespace omission means `mcpplibs` only. +``` + +The two functions answer one question with two ladders. The refresh decision +looks up the canonical coordinate `(mcpplibs, ftxui)`, which is conclusive +because `mcpplibs` is authoritative, and calls the miss a `DescriptorMiss` +(`index_refresh.cppm:221-249`; the reason is inferred, since the offline +opt-out overwrites it). The resolver, after the same miss, tries +`mcpp::pm::legacy_bare_candidates` for a version selector whose namespace was +omitted and reaches `compat.ftxui` (`prepare.cppm:5113-5160`); its comment +states the reach of that rung: every manifest written before it spells such +dependencies bare. The debounce is 120 seconds (`xlings.cppm:522`), so online +every plan or build of such a project that starts more than two minutes after +the last successful refresh runs a network `xlings update`, editor or not. An +editor that plans on every save meets it several times an hour; it made the +unbounded wait visible. + +- **Kind:** defect (R2), general to every manifest with a bare `compat.*` + dependency; also a usage issue in the xlings manifest, whose warning names + the spelling. +- **Home:** engine; openxlings/xlings. +- **Decision.** The refresh decision consults `legacy_bare_candidates` under + the resolver's condition (`spec.isVersion() && spec.namespaceOmitted`) + before it calls a miss, so both walk one ladder for as long as the + deprecated rung exists. The rung's warning announces its removal in 2026.9; + the decision must not depend on that schedule. openxlings/xlings writes + `compat.ftxui = "6.1.9"`. +- **Criterion.** A fixture whose dependency resolves only through the + deprecated rung, planned offline with `-v`, prints no suppressed decision + for it; planned online against a stub xlings that records its argv, it + starts no `update`. + +### 7.1 A1: an offline plan that needs a download has no code + +**Read.** Every offline refusal is prose: the toolchain payload +(`prepare.cppm:3452-3475`), `[xlings]` provisioning (`:1541`), a dependency +download (`:5468-5474`), the fetcher's install (`package_fetcher.cppm:1114-1119`) +and a git dependency (`prepare.cppm:7308-7321`). `emit build-database` maps +every planning failure to `MCPP_BUILD_DATABASE_PLAN_FAILED` +(`cmd_build.cppm:375`). An absent index under `--offline` is not refused at +all: `update_index_unguarded` reports success (`xlings.cppm:1923-1926`) and the +resolver later fails on a missing descriptor. + +- **Kind:** gap in the machine interface (R4), general. +- **Home:** engine; no client can separate the two failures without matching + the message, which docs/50 forbids. +- **Decision.** One refusal code, `offline-download-required`, recorded in + the per-run refusal sink (`src/build/refusal.cppm`, set immediately before + the `return`) at each site above and at the absent-index case, with the + subject's kind, name, version and index. `emit build-database` then uses + `MCPP_OFFLINE_DOWNLOAD_REQUIRED` and one diagnostic naming the subject and + the command (`mcpp build` without `--offline`, or `mcpp index update`); + `mcpp why toolchain` reports the token. Planning stops at the first + refusal, so the diagnostic names the first missing item. The issue asks for + one per missing item; that would require resolution to continue past a + refusal, which no other refusal does, and the client's remedy (one online + build) is the same. +- **Criterion.** The issue's e2e case, and a home without an index planned + offline yields the code. + +### 7.2 A2: children inherit the caller's pipe + +**Measured (group d).** + +``` +A2 reader stdin=pipe:[429071926] +A2 child fd 3 -> pipe:[429071926] +``` + +A build program started by `emit build-database` holds, as descriptor 3, the +pipe the caller reads. The leak is not specific to `popen`: the owned +launcher's `posix_spawnp` (`process.cppm:648-655`) passes it on too, because +the saved descriptor is created by `dup(1)` without close-on-exec +(`terminal.cppm:76-85`). A build program, hook or xlings process that outlives +mcpp keeps the caller from reading end-of-file. + +- **Kind:** defect (R3), general to every caller that reads mcpp through a + pipe. +- **Home:** engine (`mcpp.platform.terminal`). +- **Decision.** Save with `fcntl(1, F_DUPFD_CLOEXEC, 3)`; on Windows clear + `HANDLE_FLAG_INHERIT` on the saved handle. The issue's second measure, + closing every descriptor above 2 in each child, is not available through + `posix_spawn` on musl, which the Linux release links, so it cannot be the + rule; the rule is that a descriptor mcpp keeps open across a spawn is + close-on-exec. The survey found one such descriptor: `FileLock` already + opens with `O_CLOEXEC` (`fs.cppm:314`), and every other open or stream is + scoped to a function that starts no child. +- **Criterion.** Reading A2 shows no descriptor naming the reader's pipe; with + a build program that leaves a sleeping grandchild, a reader of the envelope + sees end-of-file within one second of mcpp's exit. + +### 7.3 A3: xlings runs unbounded and outside mcpp's ownership + +**Read.** `run_exec` and the deadline runners give a child its own process +group, register the group with a signal guard that kills it when mcpp is +terminated, and use a job object on Windows (`process.cppm:631-667`, +`unix/bounded_process.cppm:414-470`); this was built after an orphaned ninja +outlived its sandbox. The xlings calls are the exception: `call`, +`update_index_unguarded`, `install_direct` and the bootstrap go through +`popen` or `std::system` (`xlings.cppm:1421, 1620, 1692-1724, 1955`), with no +deadline and no group. The refresh retries three times on a non-zero exit +(`:1950-1967`), which does nothing for a stall. + +- **Kind:** defect (R3), general: a terminal build on a black-holed network + hangs the same way. +- **Home:** engine for the bound; xlings for the missing connection timeout in + `xlings update`, which the reporter files there. +- **Decision.** + 1. xlings invocations move to the owned launcher, argv-based, with the + streaming sink the deadline runner already accepts. Ownership and the + move happen together: a child in its own group without the guard would + no longer receive a terminal's Ctrl-C, which would be a regression. + 2. The index refresh has a total deadline, `[index] refresh_timeout` + (seconds). A timed-out refresh is a failed refresh, is not retried, and + the existing rule then holds: warn, resolve from local data, fail only if + the local data cannot answer (`prepare.cppm:4465-4472`). + 3. An install has an inactivity deadline. The interface protocol emits a + heartbeat after five seconds without output + (`xlings/src/interface.cpp:249-263`), so no line for the deadline means + the xlings process is wedged, while a slow download that progresses is + never killed. A stalled connection inside a live xlings is bounded by + xlings' downloader. + The default values are measured before they are fixed (§9.3). +- **Criterion.** The issue's two e2e cases with the stub named through + `[xlings] binary`, and a Ctrl-C in a pseudo-terminal during a stubbed + install that leaves no stub process. + +### 7.4 A4: the envelope omits the network access that happened + +docs/50 §2 defines the envelope's `effects` as "what running the command did", +and §4 defines the `--protocol-version` table as what a command may do. `emit +build-database` builds its list from constants and one observation +(`cmd_build.cppm:427-428`), so a run that refreshed the index reports no +`network`. + +- **Kind:** defect against docs/50 (R4), general to enveloped commands. +- **Home:** engine. +- **Decision.** A per-run record of observed effects, written where a network + child is launched (the index refresh, an install, a git remote operation, + the bootstrap) and read by every enveloped command. An attempt that failed + or timed out counts: the effect is the access. Under `--offline` those sites + return before launching, so the record cannot contain `network`. +- **Criterion.** The issue's. + +### 7.5 A5: `auto_refresh = false` does not govern three refreshes + +**Read.** `mcpp.pm.index_refresh` exists so one function decides whether a +run touches the network for an index; its header records that five +independent derivations had drifted (#315). Three callers bypass it: the +fetcher's pre-install refresh (`package_fetcher.cppm:1127`, through +`ensure_official_package_index_fresh`, which checks neither the policy nor the +module's debounce), the fetcher's refresh before a retry (`:1205`), and the +first sync of a project's custom index (`prepare.cppm:4644-4662`). docs/05 +states that `auto_refresh = false` means "never refresh the index +automatically". + +- **Kind:** defect (R2), general. +- **Home:** engine. +- **Decision.** The three callers go through `decide_for_miss`, which already + applies `--offline`, `auto_refresh` and the debounce. The issue's + alternative, documenting that only `--offline` is complete, would keep the + second derivation. A first sync is not exempted: the decision module already + lets `auto_refresh` suppress an absent index, and a failure that names + `mcpp index update` is the documented behaviour. +- **Criterion.** The issue's, and the custom-index case. + +### 7.6 A6: the CN mirror exists, and its artifact does not match the pointer + +**Measured (group d).** `publish_mcpp_index.sh` already publishes to +`xlings-res/mcpp-index` on GitHub and GitCode, and the GitCode pointer names +the same version as GitHub's. The asset behind it differs: + +``` +A6 pointer version=7649883 sha=78535f362b9e... +A6 github sha=78535f362b9e... bytes=553573 first-entry=... 2026-09-16 04:41 ./ +A6 gitcode sha=d89135a348c9... bytes=553557 first-entry=... 2026-09-16 03:00 ./ +``` + +The trees are identical; the entries carry different modification times. The +script ran twice for one version, GitHub's upload replaced its asset with +`--clobber`, GitCode cannot replace an asset, and both pointers were +rewritten with the second digest. The tar invocation fixes names and owners +but not times (`publish_mcpp_index.sh:35`). A CN-routed client today would +reject GitCode's bytes and fall through to GitHub, the connection that hung. + +- **Kind:** ecosystem-data defect first; then a small engine default. +- **Home:** mcpp-index (publish script), then engine (`config.cppm`). +- **Decision, in order.** + 1. The artifact becomes byte-reproducible (`--mtime` from the source + commit, `gzip -n`), and a version whose GitCode asset exists with a + different digest is published under a new name instead of keeping bytes + that no pointer describes. + 2. The engine's default `artifact` becomes the region object + `{"GLOBAL": ..., "CN": ...}` that xlings parses since #377, in the + configuration template, the in-memory default and the existing + `.xlings.json` text migration (#269). +- **Criterion.** Two runs of the script on one commit produce one digest, and + group d's three digests agree; with `mirror = CN`, `mcpp index update` + fetches no `github.com` URL in xlings' verbose download log and succeeds. + +### 7.7 What belongs to the caller + +mcppls running offline by default, owning its children through a process +group, bounding its read after mcpp exits and caching the last good document +are the caller's decisions and are correct independently of mcpp. The engine +items make them unnecessary as defences, not wrong. + +## 8. Plugin items + +- **P1, `dist-apple`.** `defaulted_plist_keys()` holds `UIDeviceFamily`, + `LSRequiresIPhoneOS` and `NSHighResolutionCapable` + (`mcpp-plugins/dist/apple.cppm:631-634`), and a project entry can only + replace a default (`:700`, `:568-586`); a property list has no null to + express removal. `options::omit_keys` lists defaulted keys to leave out; + any other key is refused by name, as derived keys already are, and naming a + key in both `info_plist` and `omit_keys` is refused. +- **P2, `dist-web`.** The page name is fixed in the `mcpp.dist.web.index` + action (`dist/web.cppm:363-368`) and in `tests/web-consumer/check-web-plan.sh:42`. + `options::page` names it, default `index.html`, a bare `*.html` name with no + separator; the plan test gains a named-page leg. +- **`dist-apk`'s strip** (0.11.1) is retired once E5 is released and the + plugin reads `mcpp::pack_strip()`. +- **`rules-swift`** is §5.2. + +## 9. Decisions for the reviewer, and what is still owed + +### 9.1 Decisions + +| # | decision | alternative not taken | why | +|---|---|---|---| +| D1 | F3a: on ELF, programs over a plan-built C++ shared library take the shared-library contract | refuse the default shape | a mechanism that delivers one runtime exists; the process already requires the shared runtime | +| D2 | F1: a static package in two shared-image closures is refused with the `linkage = "shared"` remedy | link a copy into each image | each image would keep its own state on Mach-O and PE, and ELF would hide it (F1f) | +| D3 | E6: a package with no library target contributes nothing to a consumer's target graph | key builds by (package, features, toolchain) | the edge itself is wrong (E6c); feature-split nodes contradict one identity per graph | +| D4 | E1: a root-only graph file with verbatim metadata | a function family; dependency programs emitting values | one document serves both needs; dependency programs run dependents first | +| D5 | E5: staged toolchain runtime copies are stripped by default | leave them as the payload ships them | the reason for leaving store files alone does not apply to a copy; parity with Gradle | +| D6 | E9: `--message-format json` on `pack` prints one envelope | an NDJSON stream as on `test` | pack has one result; the flag names machine output, the kind names its shape | +| D7 | A3: a total deadline for the refresh and an inactivity deadline for installs, defaults measured | one total deadline for both | a total bound kills slow but progressing toolchain downloads | +| D8 | E4.2: the documentation changes | the parser accepts a source-less table | older clients could not load a package that used it | +| D9 | E7: members selected by identity | a `subdir` key | an older client ignoring `subdir` builds the wrong package silently | +| D10 | E10 and F2: step 1 corrects the record now; any change of default waits for its measurement and its own record | change the defaults here | working builds depend on both defaults | + +### 9.2 Measurements owed, by platform + +One temporary pull request carries a measurement workflow only, as #635 did. + +| platform | fixture | reading | decides | +|---|---|---|---| +| macos-15 | #641 M3 app/fw without `llvm.libcxx`, three legs | exception and `errc` identity across the dylib boundary; `otool -L` | F2, D10 | +| macos-15 | #646 M3b | the dylib link with and without F1 placement | F1 criterion | +| macos-15 | an Android `kind = "app"` | `file` on the linked object | E3 criterion | +| macos-15 | a `@_cdecl` Swift function, C callback | the autolink entries under `ld64.lld`; `aarch64-ios-sim` | E2 | +| windows-2022 | the llvm row default and `-fms-runtime-lib=dll` spelled through flags | `llvm-objdump -p` imports; lld warnings | E10 step 2 | +| windows-2022 | the F2 fixture across an llvm-row DLL | exception identity, CRT state | F2 PE half | + +### 9.3 Values owed + +The A3 defaults: a cold `xlings update` of the default indexes from a +GitHub-hosted runner and from a CN host, on both mirrors, with the 95th +percentile of ten runs; the refresh deadline is a small multiple of it. The +install inactivity deadline is a multiple of the heartbeat interval. + +## 10. Order of work + +The engine items group into three batches by what they risk. Within a batch +the items are independent unless stated. + +**Batch 1: defects whose fix does not change what a working build does when +it runs** (one release; Linux-testable except the E3 criterion). E5 changes +the bytes of packed libraries and nothing they do. + +- R3: A2, A3. +- R2: T, A5, E4.1 with X8, E4.3, E8 with X7, X5. +- R4: A1, A4, F3b, E10 step 1, E5 (including the status line), E9. +- E3 (the pure function and its unit test; the macOS criterion rides the + temporary pull request). +- X1, X3, X6; E4.2's refusal of a differing restatement; E4.2's + documentation. + +**Batch 2: decisions that change what some builds produce** (one release, +each with its own e2e and a CHANGELOG entry naming the change). + +- F3a (D1), then F1 (D2), which share the image-closure computation. +- E6 (D3) with X2, after the blast-radius scan of §6.4. +- E7 (D9). +- E1 (D4). + +**Batch 3: measured first, in their own records.** F2 and E10 step 2 (D10). + +**Outside the engine.** + +- mcpp-index: A6 step 1, before the engine's A6 step 2 (which joins batch 1 + once the digests agree). +- openxlings/xlings: `compat.ftxui`, now; it removes T's trigger for that + project before any release. +- mcpp-plugins: P1, P2 now; `rules-swift` after its measurement; `dist-apk`'s + strip retired after E5 is released. + +The release chain follows the established order: mcpp release, mirror, index, +then the plugin pins that need the new engine surface (`pack_strip`, +`graph_file`). + +## 11. Readings + +Each block is quoted from the group's `READING` lines, with the home +directory written as `~` and scratch paths shortened. + +**Group a (#646, E10).** + +``` +F1a M3b exit=0 +F1a libfw.so inputs: obj/mcpplibs_fw/src/fw.o +F1a libfw.so undefined x_answer=1 program defines x_answer=1 program exports x_answer=1 +F1a run exit=0 +F1c libfw.so relinked with -z defs: exit=1 undefined reference to `x_answer' +F1d foreign host dlopen(libfw.so, RTLD_NOW): exit=1 output: dlopen failed: .../bin/libfw.so: undefined symbol: x_answer +F1b root also depends on x: exit=0 +F1b x.o linked into: program=1 libfw.so=0 +F1b run exit=0 counter=2 +F1f libfw.so relinked with x.o: exit=0 libfw defines x_counter=1 run: counter=2 (1 shared counter => 2; split => 1) +F1e root-owned shared image over x: exit=0 libfw.so defines x_answer=1 undefined=0 +F3 gcc run: lib-3 exit=0 +F3 gcc NEEDED program: [liblib.so] [libm.so.6] [libgcc_s.so.1] [libc.so.6] [ld-linux-x86-64.so.2] +F3 gcc NEEDED liblib.so: [libstdc++.so.6] [libm.so.6] [libgcc_s.so.1] [libc.so.6] +F3 gcc contracts in resolution.json: {"distributable": "self-contained", "intermediate": "self-contained", "shared-library": "toolchain-coupled", "test": "self-contained"} +F3 gcc bin/app status=conflict exported=4571 conflicts=900 defined-in-std.o=1 first=['_ZGIW3std', '_ZGTtNKSt11logic_error4whatEv', ...] +F3 gcc --strict after touch exit=1 mentions=1 +F3 llvm run: libc++abi: terminating due to uncaught exception of type std::bad_cast: std::bad_cast exit=134 +F3 llvm NEEDED liblib.so: [libc++.so.1] [libc++abi.so.1] [libunwind.so.1] [libm.so.6] [libc.so.6] +F3 llvm bin/app status=conflict exported=145 conflicts=46 defined-in-std.o=1 first=['_Unwind_DeleteException', '_Unwind_ForcedUnwind', ...] +F3 llvm-static-lib run: lib-3 exit=0 +F3 llvm-no-println run: libc++abi: terminating due to uncaught exception of type std::bad_cast: std::bad_cast exit=134 +F3 llvm-tc run: lib-3 exit=0 +F3 llvm-tc bin/app status=conflict exported=84 conflicts=2 defined-in-std.o=1 first=['_ZGIW3std', '_ZGIW3stdW6compat'] +F3 llvm-tc --strict after touch exit=1 mentions=1 +F3 gcc-tc run: lib-3 exit=0 +F3 gcc-tc bin/app status=conflict exported=656 conflicts=8 defined-in-std.o=1 first=['_ZGIW3std', '_ZNSt8__detail31__from_chars_alnum_to_val_tableILb0EE5valueE', ...] +F3 gcc-tc --strict after touch exit=1 mentions=1 +F3 std.o global definitions=1 strong=1 sample=['_ZGIW3std'] (gcc and llvm) +E10 compile, no flag: dependent-lib= defines= +E10 link, no flag: -defaultlib:libcmt -defaultlib:oldnames +E10 -fms-runtime-lib=dll: compile --dependent-lib=msvcrt --dependent-lib=oldnames link -defaultlib:libcmt -defaultlib:oldnames +E10 -fms-runtime-lib=static: compile --dependent-lib=libcmt --dependent-lib=oldnames link -defaultlib:libcmt -defaultlib:oldnames +``` + +The lead's independent run of the F3 llvm default shape: + +``` +F3A build exit=0 +libc++abi: terminating due to uncaught exception of type std::bad_cast: std::bad_cast +F3A run exit=134 +F3A mcpp-run exit=134 +warning: app: 46 symbols in this image are also provided by a library it loads. +``` + +**Group b (E1, E5, E9).** + +``` +E1-strict-metadata-table: exit=0 warnings=0 +E1-root-env: SPIKE deps: MCPP_DEP_A_DIR MCPP_DEP_SPIKE_A_DIR MCPP_DEP_A_LINKAGE MCPP_DEP_SPIKE_A_LINKAGE +E1-program-order: SPIKE order a SPIKE order b +E1-graph-record: [('mcpplibs.app@0.1.0', ['package','requested_by','root']), ('spike.a@0.1.0', ['link','package','requested_by','root']), ('spike.b@0.2.0', [...])] +E1-metadata-edit: build.mcpp up to date (cached)x3 replayed=yes +E1-unknown-package-key-strict: exit=0 mentions=0 +E5-desktop-status: Packing hostapp v0.1.0 (vendored, stripped) +E5-desktop: bin/hostapp bytes=14280 symtab=0 debug=0 +E5-desktop: lib/libdep.so bytes=16592 symtab=1 debug=7 +E5-android-status: Packing app v0.1.0 (vendored, stripped) +E5-android: lib/libapp.so bytes=5904 symtab=1 debug=1 +E5-android: lib/libc++_shared.so bytes=9091400 symtab=1 debug=6 +E5-android: lib/libdep.so bytes=5496 symtab=1 debug=1 +E9-pack --release: exit=2 [error: unknown option: --release] +E9-pack --dev: exit=2 [error: unknown option: --dev] +E9-pack --message-format json: exit=2 [error: unknown option: --message-format] +E9-human-lines-on-stdout: bytes=273 +E9-run-precedence(--profile dev --release): Finished release +E9-build-precedence(--profile dev --release): Finished dev +E2-host: no swiftc on this host; E2 needs macos-15 +``` + +**Group c (E4, E6, E7, E8).** + +``` +E4.1-build: 2 forward warnings; tool kt built: 1; rc=0 +E4.1-strict: error: feature 'kotlin' of 'fw' forwards to dependency 'spike.rules' (as 'spike.rules/kotlin') which is not declared in ; rc=2 +E4.1-control-without-feature: tool kt built: 0; rc=0 +E4.1b-forward-to-other-row-dep: error: feature 'kotlin' of 'fw' forwards to dependency 'spike.win' (as 'spike.win/x') which is not declared in [dependencies] or [feature-de; rc=2 +E4.2-sourceless: error: .../e42/app/mcpp.toml: error: [f; rc=2 +E4.2-restated: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/installer@0.1.0+path.c728166a178c2a19/38e9d63bb5f6464d/bin/installer] qualified=[]; rc=0 +E4.2-restated-without-feature: SPIKE short=[] qualified=[]; rc=0 +E4.2-restated-different-path: warnings=1 tool-from: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/installer@0.1.0+path.c728166a178c2a19/38e9d63bb5f6464d/bin/installer] qualified=[]; rc=0 +E6-issue-fixture: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/fw-installer@0.1.0+path.2c270cde13410c30/46f3d4215af669cd/bin/fw-installer] qualified=[] | error: dependency cycle through package 'fw' while computing its build-cache key; rc=2 +E6-tool-runs: fw-installer ran 42 +E6-without-feature: ; rc=0 +E6-cache-local: ; rc=0; app runs: exit=0 +E6-cache-local-link: obj/main.o obj/spike_fw/src/fw.o +E6b-tool-activates-declaring-feature: 4 host-tool lines | error: building host tool 'fw-installer:fw-installer' failed: (x4) ... tool provisioning nested more than 4 levels deep — this is almost certainly a cycle.; rc=2 +E6d-build-dependency-tool-edge: | error: building host tool 'fw-installer:fw-installer' failed: (x4) ... nested more than 4 levels deep ...; rc=2 +E7-both-keys: 1 adoption warning(s) | SPIKE short=[] | Compiling spike.fw v | ; rc=0 +E7-member-only: error: dependency 'spike.fw-installer' resolved to package 'fw' (mismatch with declared name 'fw-installer'); rc=2 +E7-member-by-path: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/fw-installer@0.1.0+path.2db8369d57c7c650/47cd46c89a4debd7/bin/fw-installe | ; rc=0 +EDUP-dependencies-and-build-dependencies: tool built: 0 | SPIKE short=[] | ; rc=0 +E8-dep-feature-no-table: SPIKE short=[] qualified=[] | tool built: 0; rc=0 +E8-dep-feature-no-table-macro: -DMCPP_FEATURE_SPIKE_FW_INSTALLER +E8-unknown-no-table: ; rc=0; macros: -DMCPP_FEATURE_NOTHING_HERE +E8-dep-feature-with-table: warning: --features requests 'spike.fw/installer' which [features] does not declare | tool built: 0; rc=0 +E8-dep-feature-with-table-strict: error: --features requests 'spike.fw/installer' which [features] does not declare; rc=2 +E8-manifest-forward-control: SPIKE short=[~/.mcpp/build-cache/v1/tool/spike/fw-installer@0.1.0+path.5020fbc16166960c/21a7db9275c93a63/bin/f | tool built: 1; rc=0 +E6c-tool-dependency-in-consumer: app link line: obj/main.o obj/spike_z/src/z.o obj/spike_fw/src/fw.o; app defines z symbol: 1; rc=0 +``` + +**Group d (#648).** + +``` +A2 reader stdin=pipe:[429071926] +A2 child fd 0 -> socket:[430147394] +A2 child fd 1 -> pipe:[430149989] +A2 child fd 2 -> pipe:[430149989] +A2 child fd 3 -> pipe:[429071926] +A2 child fd 4 -> .../a2/fds.txt +T exit=0 +T decision ftxui@6.1.9: offline mode +T resolver warning: dependency 'ftxui' resolved to 'compat.ftxui' through the deprecated bare-name search; namespace omission means `mcpplibs` only. Write the exact package: +T resolver warning: dependency 'gtest' resolved to 'compat.gtest' through the deprecated bare-name search; namespace omission means `mcpplibs` only. Write the exact package: +A6 pointer version=7649883 sha=78535f362b9e4803e2036a5fd64730751074240001c262f204e07460dda6bc42 +A6 github sha=78535f362b9e4803e2036a5fd64730751074240001c262f204e07460dda6bc42 bytes=553573 +A6 gitcode sha=d89135a348c99c20baeba211f4bbaf2892b210fce0114ebee954880835c32a60 bytes=553557 +A6 github first-entry=drwxr-xr-x 0/0 0 2026-09-16 04:41 ./ +A6 gitcode first-entry=drwxr-xr-x 0/0 0 2026-09-16 03:00 ./ +``` + +The T reading planned openxlings/xlings at `59068d6`, the local checkout; its +`[dependencies]` spell `ftxui = "6.1.9"` as the current head `3cd8061` does +(`mcpp.toml:92`). `gtest` resolves through the same rung and prints no +decision, because the refresh loop walks `[dependencies]` only +(`prepare.cppm:4455`) and `gtest` is a dev-dependency. The A6 +digests are of the day's artifacts and change with the next publish. + +## 12. What each project can do today + +- **HuxerUI.** E4.2: restate the source in `[feature-deps]`. E6: keep the + per-application installer package until D3 is released. E7: pin the tool by + path within a checkout, or by a published version. E8: keep the root + forward line. E1: keep hand-written library paths. E5: `dist-apk` 0.11.1 + strips. E3: build Android on Linux. E10: the specification's allowed + difference stands. +- **mcppls.** Run offline by default, own the process group and bound the + read after mcpp exits, as planned. Until A2 is released, a child that mcpp + started in its own group can hold the pipe after the caller's group is + killed, so the bounded read is the part that cannot be dropped. +- **openxlings/xlings.** Write `compat.ftxui = "6.1.9"`; the refresh every two + minutes stops at once. + +## 13. Self-review + +1. **Unmeasured premises that carry decisions.** F2's split on the payload + origin; E10's step 2; the Mach-O and PE halves of F1; E3's criterion; E2's + autolink. Each is in §9.2, and no decision that changes an artifact is + taken on them. +2. **T's reason is inferred.** The offline opt-out overwrites the reason the + decision computed. The conclusion that the decision wanted a refresh does + not depend on it (the suppressed reason is assigned only after + `shouldRefresh` is set); which of `DescriptorMiss` or `IndexAbsent` fired + does, and the bare-name warning of the same run makes the first the only + consistent one. The criterion of §7.0 measures it directly. +3. **D3's blast radius is not yet known.** A consumer that depended on a + program-only package's dependencies leaking into its link would break; + §6.4 names the scan that precedes implementation. +4. **D1's scope stops at plan-built libraries.** A prebuilt C++ shared library + from the store is the same class and is left undecided. +5. **A2's second measure was dismissed on musl's `posix_spawn`.** macOS offers + `POSIX_SPAWN_CLOEXEC_DEFAULT`; the rule chosen is the one that holds on + every host, and the platform-specific closing can be added where it + exists without changing the rule. +6. **The readings are from one host.** Group b's Android leg used an NDK + already installed on this host; groups a to d ran against a home shared by + other processes, and no reading depends on an index refresh. +7. **Two forks' conclusions were checked against the code by the lead** before + entering this record: E3's branch assembly, E5's dispatch, E1's program + order, E4.3's publication site, E8's documentation and the X1 early return; + and F3a was reproduced independently. diff --git a/.agents/docs/2026-09-16-646-649-implementation-plan.md b/.agents/docs/2026-09-16-646-649-implementation-plan.md new file mode 100644 index 000000000..92394aeff --- /dev/null +++ b/.agents/docs/2026-09-16-646-649-implementation-plan.md @@ -0,0 +1,498 @@ +--- +subject: triage +status: active +--- + +# #646 to #649 implemented: the plan, its review from eight angles, and the ledger + +**Status:** active. This record turns the decisions of +`2026-09-16-646-649-four-issues-by-home.md` ("the triage record") into work: +one pull request per repository, the order in which they merge and release, +the criterion each task is held to, and a ledger whose rows close only with a +reading. The triage record's decisions D1 to D10 are adopted; §1 states the +refinements this review makes before code is written, and a refinement found +while implementing is appended to §1.9. + +Base commits: mcpp `2fc7b5b0` (2026.9.15.2), mcpp-plugins `d6bee6a` (0.11.1), +mcpplibs/mcpp-index, openxlings/xlings `3cd8061`, openxlings/xim-pkgindex at +their heads on 2026-09-16. The engine release is 2026.9.16.1; the plugins +release is 0.12.0. + +## 0. The ledger + +Status is `todo`, `doing`, `branch` (implemented on the pull request's branch, +with its reading), `done` (merged or published, with its reading) or `dropped` +(with the reason). Owner `lead` is the integrating session; `W1` to `W3` are +the parallel work trees of §8. Triage sections are cited as `T§`. + +### 0.1 Engine: mcpp-community/mcpp, one pull request (`feat/646-649`) + +| id | task (triage §) | owner | depends on | status | +|---|---|---|---|---| +| L1 | the refresh decision walks the resolver's deprecated bare-name rung before calling a miss (T§7.0) | lead | - | branch: unit `PmIndexRefresh.BareNameResolvedThroughTheLegacyRungIsNotAMiss`; e2e 730 (fails on 2026.9.15.2: the decision asked for a refresh of `cjson@1.7.19`) | +| L2 | the saved standard output is close-on-exec, and not inheritable on Windows (T§7.2) | lead | - | branch: e2e 731 (fails on 2026.9.15.2: a build program holds the caller's pipe as descriptor 3) | +| L3 | xlings runs under the owned launcher; a total deadline for the index refresh, an inactivity deadline for installs (T§7.3) | lead | L2 | branch: e2e 732 (fails on 2026.9.15.2: `index update` returned after 60 s under a 3 s bound); SIGTERM to mcpp took the stub's group (local) | +| L4 | a per-run record of observed effects; `network` when a network child is launched (T§7.4) | lead | L3 | branch: e2e 733 (fails on 2026.9.15.2: a plan that refreshed the index reports no `network`) | +| L5 | the three refreshes that bypass `mcpp.pm.index_refresh` go through `decide_for_miss` (T§7.5) | lead | L1 | branch: e2e 734 (fails on 2026.9.15.2: the unsynced custom index was synced under `auto_refresh = false`) | +| L6 | the refusal code `offline-download-required` at every offline refusal site; `MCPP_OFFLINE_DOWNLOAD_REQUIRED` in the envelope (T§7.1) | lead | - | branch: e2e 733 leg B and 735 (fail on 2026.9.15.2: `MCPP_BUILD_DATABASE_PLAN_FAILED`) | +| L7 | the default `artifact` of the mcpplibs index is the region object; the existing `.xlings.json` migration carries it (T§7.6 step 2) | lead | X1 | branch: e2e 151 (fails on 2026.9.15.2: no region artifact); traced `mcpp index update` with `mirror = CN` connects only to raw.gitcode.com, gitcode.com and file-cdn.gitcode.com | +| R1 | ELF: programs and tests over a plan-built C++ shared library take the shared-library runtime contract; an explicit `self-contained` program there is refused (T§4.1, D1) | W1 | - | branch: unit `Distribution.*`; e2e 700 (fails on 2026.9.15.2: the default llvm shape aborts with `std::bad_cast`) | +| R2 | symbol provision: `STB_GNU_UNIQUE` is vague linkage; a duplicate whose definitions come from one plan object is not reported (T§4.2) | W1 | - | branch: unit `SymbolProvision.*`; e2e 701 (fails on 2026.9.15.2: `--strict` exits 1) | +| R3 | a static package reachable from one shared image only is linked into that image; one reachable from two images is refused where the link or load cannot succeed and diagnosed elsewhere (T§4.3, D2, §1.5) | W1 | - | branch: unit `StaticPlacement.*` (9); e2e 702 and the rewritten 307 (fail on 2026.9.15.2: `undefined symbol: x_answer` through a foreign `dlopen`) | +| R4 | clang on the MSVC ABI records the runtime it delivers; an undeliverable `cxx_runtime` is diagnosed; docs/20 states the row's model (T§4.5 step 1) | W1 | - | branch: e2e 703 (Windows CI) | +| R5 | measurement legs: exception identity across a Mach-O dylib under the payload default (macos-15), and across an llvm-row DLL (windows-2022), each printing its reading (T§4.4, §9.2) | W1 | - | done: macos-15 run 35032727668 reads `macho default: runtime_error=not-matched own_error=caught errc=unequal`, `macho host-coupled: runtime_error=caught errc=equal`, `macho shared-host-coupled: not-matched`. F2 is confirmed; this release warns (`build/cxx-runtime-identity`) and leaves the default to its own record | +| G1 | the forward validator accepts a key declared in any dependency table on any row (T§6.1, X8) | W2 | - | branch: e2e 710 (fails on 2026.9.15.2: the two-level build-dependency forward refused under `--strict`) | +| G2 | a `[feature-deps]` restatement whose source differs is refused; docs/05 says to restate the source (T§6.2, D8) | W2 | - | branch: e2e 711 (fails on 2026.9.15.2: the differing restatement is not refused) | +| G3 | one helper names a provider for a consumer; `dep_bin` gains the qualified spelling (T§6.3) | W2 | - | branch: e2e 711 and a new 187 leg (fail on 2026.9.15.2: `dep_bin("spike.installer")` reads nothing) | +| G4 | a package with no library target contributes nothing to a consumer's target graph; package cycles are checked at resolution; a repeated tool is refused at its first repetition (T§6.4, D3, X2, X3) | W2 | - | branch: e2e 712 (fails on 2026.9.15.2: the feature tool depending on its declaring package does not build); scan of 902 manifests found no consumer relying on the old edge | +| G5 | a git dependency selects a repository member by identity; a second declaration merges additively; the git banner names the commit (T§6.5, D9, X1, X6) | W2 | G4 | branch: e2e 713 (fails on 2026.9.15.2: a git dependency naming a repository member is refused) | +| G6 | `--features dep/feature` is a root forward; `why deps --features` (T§6.6, X7) | W2 | G1 | branch: e2e 714 (fails on 2026.9.15.2: `--features spike.fw/installer` does not open the feature) | +| S1 | `mcpp::graph_file()` for the root build program, with `[package.metadata]` and a digest in the re-run key (T§5.1, D4) | W3 | - | branch: e2e 720 (fails on 2026.9.15.2: `graph_file` is not a member of `mcpp`) | +| S2 | the link branch is chosen by host and target object format, as a pure function with a host-by-row unit test (T§5.3) | W3 | - | branch: unit `LinkShape.*`; e2e 721 (macOS CI step) | +| S3 | pack strips the program on every stripping row, every graph-built shared library and staged runtime copies; the status line reports what was done; `mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()` (T§5.4, D5) | W3 | - | branch: e2e 722 (fails on 2026.9.15.2: `lib/libdep.so is not stripped: symtab=1 debug=7`; the Android leg ran locally) | +| S4 | `mcpp pack --message-format json` prints one `mcpp.pack` envelope; `pack --release/--dev`; `run` takes `build`'s profile precedence (T§5.5, D6, X5) | W3 | - | branch: unit `BuildProfile.*`; e2e 723 (fails on 2026.9.15.2: `unknown option: --message-format`) | +| S5 | `[package]` warns about an unknown key, `metadata` included in its known set (X4) | W3 | S1 | branch: unit `Manifest.PackageMetadata*`, `UnknownPackageKeyIsReported` | +| C1 | CI: new llvm-dependent e2e scripts run on the hermetic job with their PASS lines asserted; the macOS and Windows measurement legs print their readings to the job summary | lead | R1-R5, S2 | branch: 700 on the hermetic job; READING lines to the macOS and Windows summaries; 721 as its own macOS step | +| C2 | user documentation with its Chinese mirror, docs/50 codes and kinds, CHANGELOG, version 2026.9.16.1 | lead, W1-W3 | all | branch: docs 04, 05, 06, 07, 10, 20, 30, 50 with mirrors; SPEC-005 v1.1; CHANGELOG; 2026.9.16.1 | +| C3 | the two records closed with their readings | lead | all | todo | + +### 0.2 Plugins: mcpp-community/mcpp-plugins, one pull request (`feat/646-649`, 0.12.0) + +| id | task | owner | depends on | status | +|---|---|---|---|---| +| P1 | `dist-apple` `options::omit_keys` over the defaulted keys (T§8) | lead | - | branch: `b5f759e`, plan checks on Linux; the bundle on macos-15 CI | +| P2 | `dist-web` `options::page`, default `index.html` (T§8) | lead | - | branch: `c804312`, `check-web-plan.sh` with a named-page leg | +| P3 | `dist-apk` follows `mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()` when the engine provides them, and keeps its own strip otherwise | lead | S3 released | branch: `ad7a3dd`; unset and simulated variables verified; legs (m) and (n) run once the CI pin is 2026.9.16.1 | +| P4 | `rules-swift`: one package's Swift sources compile through an `object` action, a generated header through a `source` action, and the runtime link flags through `link_flag`; a macos-15 consumer runs (T§5.2) | lead | - | branch: `38ac2ba`, `all-rules-compile` on Linux; build, run and the header on macos-15 CI | +| P5 | `dist-apk` and `dist-apple` collect library contributions from `[package.metadata.dist-apk]` and `[package.metadata.dist-apple]` through `mcpp::graph_file()` when present | lead | S1 released | todo | + +### 0.3 Index, payloads and projects + +| id | task | owner | depends on | status | +|---|---|---|---|---| +| X1 | mcpplibs/mcpp-index: the artifact is byte-reproducible, and a version whose GitCode asset differs is republished under a new name (T§7.6 step 1) | lead | - | done: mcpplibs/mcpp-index#432 (`0cbac960`); the publish of `0cbac96` verified both forges itself, and the probe reads pointer, GitHub and GitCode all `02a437110017` (547679 bytes) | +| X2 | openxlings/xim-pkgindex: the release bot's `mcpp` bump merged | lead | release | todo | +| X3 | mcpplibs/mcpp-index: the plugins descriptor 0.12.0; the index CI's `MCPP_VERSION` moves to 2026.9.16.1 | lead | X2, P1-P5 | todo | +| X4 | openxlings/xlings: `compat.ftxui` and `compat.gtest` spelled with their namespace (T§7.0) | lead | - | done: openxlings/xlings#597 (`4ea4eac9`), CI 9 of 9 | + +### 0.4 Verification + +| id | task | owner | depends on | status | +|---|---|---|---|---| +| V1 | a fresh SubOS sandbox with CN mirrors for xlings and mcpp runs §7's scenarios against the published engine, plugins and index | lead | X2, X3 | todo | +| V2 | replies on #646, #647, #648 and #649 naming the releases and what each project writes | lead | V1 | todo | + +## 1. The review + +Each decision was held to eight questions. The answers either confirm the +triage record's decision or refine it; a refinement is stated where the +decision is. + +### 1.1 Architecture + +- Every engine change lands in the module that already owns the question: + runtime contracts in `distribution.cppm`, placement in `make_plan`, the + refresh policy in `mcpp.pm.index_refresh`, child ownership in + `mcpp.platform.process`, forwards in the feature pass of `prepare`, + build-program facts in `hostprogram.cppm`, pack policy in `src/pack`. No + task adds a module. +- The two new facts a build program reads (`graph_file`, `pack_strip`) extend + the accessor family that exists (`dep_dir`, `pack_stage_dir`), reach the + program through the contract environment, and therefore take part in the + re-run key by construction. +- The refusal codes added here (`offline-download-required`, + `program-cxx-runtime-split`, `static-package-in-two-images`, + `package-cycle`) join the existing sink; `mcpp why toolchain` and the + envelope read them without a second classification. + +### 1.2 Stability + +- L3 changes how every xlings child is started. The streaming sink, the exit + code and the NDJSON parsing stay as they are; only the launcher changes. The + deadline paths are exercised by a stub xlings named through + `[xlings] binary`, on Linux in CI and on Windows through the e2e shard. +- R3 moves objects between images. Its unit test enumerates the closures of + six graph shapes (a chain, a diamond through one image, a diamond through two + images, a root-owned image, an image over an image, a header-only package) + before any link line changes. +- G4 removes edges from the consumer graph. Before it is implemented, the + manifests of the e2e corpus, the examples, mcpp-index and mcpp-plugins are + scanned for a dependency on a program-only package that is not requested for + `tools` (§1.9 records the result). + +### 1.3 Simplicity + +- A2 is one call. A5 deletes a derivation. L1 adds one condition to a lookup + that exists. G3 deletes a duplicated derivation. E3's fix is a pure function + replacing an `if constexpr` choice. +- The deadline design uses the launcher's existing total deadline for the + refresh; only the install path needs an inactivity bound, which is one more + parameter of the same launcher, not a second launcher. +- D6 adds a kind to the envelope table rather than a new output mechanism. + +### 1.4 User experience + +- The status line of `pack` states what was done (S3); the envelope states + what happened (L4, L6); a refusal names its remedy (R1, R3, G2, G4); a + deadline names itself and the configuration key that sets it (L3). +- **Refinement U1 (L3).** A timed-out refresh prints one warning naming the + deadline and `[index] refresh_timeout`, and the build continues from local + data; it is never silent and never fatal on its own. +- **Refinement U2 (G6).** `--features dep/feature` is accepted by `build`, + `run`, `test`, `pack` and `emit build-database` alike, because they share + one overrides structure; the help text of each says so. + +### 1.5 Compatibility and seamless upgrade + +Each change that alters what a working build produces is listed with why an +upgrade is not a cliff. + +| task | what changes for a working build | why it is not a cliff | +|---|---|---| +| R1 | a gcc program over a C++ shared library it builds gains `NEEDED libstdc++.so.6` | the process already loads that file through the library; `pack` already bundles it | +| R3 | on ELF, a static package reachable only through a dependency's shared image moves from the program into the image | the program never referred to it; its symbols resolved to the same single copy | +| R3 | a static package reachable from two images | **Refinement C1.** It is refused only where the build cannot work today (Mach-O and PE, whose links fail, and the Android `app` row, whose loader binds the image before the program). On other ELF rows the build proceeds as it does today, with a degraded diagnostic naming `linkage = "shared"`; `--strict` fails it. Refusing a build that runs today on Linux would be an upgrade cliff for no gain on that row. | +| R4 | the recorded contract of an llvm-row PE artifact changes from `host-coupled` to `self-contained` | the artifact is byte-identical; the record now matches it | +| G4 | a program-only package's dependencies leave the consumer's link | §1.2's scan decides whether any consumer relied on it | +| S3 | packed shared libraries lose their symbol tables | `--no-strip` restores them; debug sections go to `--debug-symbols` as the program's do | +| S4 | `mcpp run --profile dev --release` builds `dev` | the same line already builds `dev` under `mcpp build` | +| L5 | `auto_refresh = false` blocks the pre-install refresh | the documentation already states that behaviour | +| L7 | a CN machine fetches the mcpplibs index from GitCode | only after X1 makes the GitCode artifact match its pointer; a mismatch falls back to GitHub, which is today's route | + +No manifest key is removed and no manifest that loads today stops loading. +The new manifest surface is `[package.metadata.*]` (already ignored by every +client) and one configuration key, `[index] refresh_timeout`, whose absence +means the default. + +### 1.6 Cross-platform + +| task | Linux | macOS | Windows | +|---|---|---|---| +| L2 | `F_DUPFD_CLOEXEC` | `F_DUPFD_CLOEXEC` | `HANDLE_FLAG_INHERIT` cleared | +| L3 | process group, signal guard | process group, signal guard | job object with kill-on-close | +| R1 | the rule | not applicable (Mach-O is R5's measurement) | not applicable | +| R3 | ELF: diagnosed; Android app row: refused | refused | refused | +| R4 | unit test of the record | - | e2e on the llvm row | +| R5 | - | e2e reading | e2e reading | +| S2 | unit test over host by row | e2e: Android row links | unit test | +| S3 | ELF, Android | Mach-O unchanged (not stripped by design) | PE, MinGW DWARF | + +### 1.7 Consistency + +- Profile precedence, feature tokens, dependency names and runtime contracts + each get one derivation (R2 of the triage record). The review confirmed that + no second copy of these remains after the tasks: `run`, `build`, `test` and + `pack` read one profile resolver; manifest forwards and CLI forwards share + the validator; `fillDepDirs` and the tool publication share the name helper. +- **Refinement K1 (S4).** docs/50 §3 gains the rule for a command whose + `--format` names its product, and `test`'s existing `--message-format json` + is cited as the precedent, so the two spellings are one convention. + +### 1.8 Test coverage + +- Every task has a criterion that fails on 2026.9.15.2: an e2e script whose + first line of output names the release it was measured against, or a unit + test of a pure function. +- e2e scripts that need llvm run on the hermetic job and their PASS lines are + asserted (the #641 lesson: `# requires: llvm` alone never runs on a shard). +- Measurement legs (R5) print `READING` lines into the job summary and do not + fail on the value they read; they fail only if the probe could not run. +- The sandbox verification (§7) runs its script once against 2026.9.15.2 first: + the change-detecting scenarios must fail there and pass on 2026.9.16.1, and + the regression guards must pass on both. + +### 1.9 Refinements found while implementing + +1. **L5 adds a module.** `mcpp.pm.index_refresh` reads descriptors through + `mcpp.pm.index_route`, which imports the fetcher, so the fetcher could not + ask it. The policy half (reasons, policy, `decide_for_miss`, `apply`, the + one-sync guard) moved to `mcpp.pm.refresh_policy`, which + `mcpp.pm.index_refresh` re-exports. §1.1's "no task adds a module" does not + hold for this task, and no other derivation of the policy remains. +2. **L3 keeps the shell command.** The xlings invocations keep their command + strings (quoting, environment prefix, stderr redirection, all of which carry + earlier Windows lessons) and run them through `run_streaming_bounded`, which + is the deadline launcher with a streaming sink, an inactivity bound and, on + POSIX, an opt-in process group registered with the signal guard. The group + is opt-in because the launcher's uncaptured callers hand the terminal to + their child, and a background group reading the terminal is stopped by + SIGTTIN. A direct `xlings install -y` sends its output to the null device, + so it has a total bound of three hours rather than an inactivity bound. +3. **X1 reuses rather than renames.** A version already published is reused, + GitCode's copy first because GitCode cannot replace an asset, so a republish + converges on bytes that are already fixed; a fresh pack is reproducible. + This is the rule xim-pkgindex adopted on 2026-09-05, and it needs no new + asset name. +4. **GitCode's raw endpoint answered 403 for a while** for the mcpp-index + pointer through `main`, and 200 by commit id, while xim-index's pointer + answered 200; the responses carry Huawei WAF cookies, and the same request + answered 200 later. A client whose CN pointer fetch fails falls back to the + git source rather than to the GLOBAL artifact base; that is xlings' + behaviour, recorded here and reported as openxlings/xlings#598, not changed in + this batch. It is no regression: before the region object a CN client fetched + the pointer from GitHub, and its fallback was the same git source. +5. **R2 also excuses the toolchain's own std module initialisers.** GCC 16's + `libstdc++.so.6` exports `_ZGIW3std` and `_ZGIW3stdW6compat` itself, so the + one-plan-object rule alone left one finding on the gcc default; names that + `std.o` defines are also excused when the other provider lives inside the + compiler's installation (W1). +6. **R3's placement skips layer providers and distribution packages** + (`provides = ["mcpp:..."]`), which would otherwise report a graph C++ runtime + as a two-image conflict (e2e 690 passes), and the root's reach includes its + dev-dependencies. The criterion is a foreign `dlopen` with `RTLD_NOW` rather + than a replayed `-z defs` link (W1). +7. **G4 needs `targetsInferred`.** A package with only `src/main.cpp` and no + `[targets]` infers a program target; the rule applies to declared targets + only, so the manifest records which of the two it produced (W2). +8. **G4's first-repetition refusal also covers an unconditional + `[build-dependencies]` edge** on a tool that depends on its declarer, which is + a genuine cycle (W2). +9. **G5 selects members without the adoption warning**: the root and a member + pinned by the same revision are two identities over one clone, and the + member's source string carries a `#member=` suffix (W2). +10. **S4's profile rule lives in `mcpp.build.prepare`.** Importing + `mcpp.cli.cmd_build` from `mcpp.cli.cmd_publish` made GCC 16.1 crash at + `import mcpp.cli;` on every build (W3). +11. **S2's shape value is `PeLld`, not `WindowsLld`**: the substring `wsl` is + reserved by the runtime-contract source check (W3). A Windows host building + a non-PE target whose driver is not named by `--target` keeps today's lld + line, which keeps the Windows-to-Linux cross job unchanged. +12. **S3 uses one strip tool for every Android leg**: the NDK's `llvm-strip` + reads every Android ABI (W3). +13. **P3 reads the environment rather than new accessors**, so `dist-apk` keeps + working on an engine that does not publish the variables, and writes debug + files per ABI, because one APK carries one library name per ABI (plugins). +14. **P4 serves the iOS device row as well**, as `rules-metal` does; the + consumer declares the `@_cdecl` function itself rather than including the + generated header, whose C++ visibility depends on the Swift version + (plugins). + +15. **F2 is confirmed, and answered with a diagnostic (macos-15, run + 35032727668).** Under the payload's Mach-O default a `std::runtime_error` + thrown in a dylib is not caught by its class in the program and two + `std::error_code` categories compare unequal; a host-coupled graph catches + it and compares equal; the mixed leg (a self-contained program over a + host-coupled library) splits as well. The default is unchanged in this + release, and a build whose program loads a C++ dylib of its own is told + once through `build/cxx-runtime-identity`, with `cxx_runtime = + "host-coupled"` as the remedy. Changing the Mach-O default belongs to its + own record, as T§4.4 states. +16. **The macOS stream-init shim was prepended to every link unit.** The object + calls libc++'s `ios_base::Init` constructor, and a C-only shared library + links without the C++ runtime, so on macos-15 the F1 fixture failed with + `ld64.lld: error: undefined symbol: std::__1::ios_base::Init::Init()`. The + shim now follows the predicate the link line itself uses + (`unit_needs_cxx_runtime`). The defect predates this batch; no fixture had + a C-only shared library beside a C++ program on macOS before e2e 702. +18. **The local verification of the integrated branch** (Linux x86_64, this + host): the full unit suite is 121 of 121, and `tests/e2e/run_all.sh` reads + `E2E Summary: 398 passed, 0 failed, 47 skipped`, the skips being the + capabilities this host lacks (msvc, mingw, qemu, a device). The macOS and + Windows legs are the pull request's own CI. +17. **`timeout` is not on a macOS runner.** e2e 732 bounds its own commands + with `timeout`, `gtimeout` or neither, since the bound under test is the + engine's. + +## 2. Engine tasks + +### 2.1 Lead: #648 (L1 to L7) + +- **L1.** `decide_for_dependency` calls `legacy_bare_candidates` under + `spec.isVersion() && spec.namespaceOmitted` after a conclusive miss, and + treats a hit as `None`. Unit: `IndexRefresh.LegacyBareRung`. e2e 730: a + fixture whose bare dependency resolves through the rung, planned offline with + `-v`, prints no suppressed decision. +- **L2.** `StdoutToStderr` saves with `F_DUPFD_CLOEXEC` (Windows: the saved + CRT descriptor's handle is made non-inheritable). e2e 731: reading A2's + build program lists no descriptor naming the reader's pipe. +- **L3.** `call`, `update_index_unguarded`, `install_direct`, the bootstrap + and `config` invocations run through the owned launcher with argv, the + environment the `ScopedInvocationEnv` states, and the working directory as + a parameter. The refresh is bounded by `[index] refresh_timeout` (default + 120 seconds); an install by an inactivity bound of 300 seconds without an + NDJSON line. A timeout is not retried. e2e 732: a stub xlings that sleeps; + `emit build-database` with local data returns within the deadline plus a + margin and succeeds; after SIGTERM to mcpp no stub process remains. +- **L4.** `mcpp::wire` keeps a per-run set of observed effects; the launcher + of a network child adds `network`. Every enveloped command merges it into + its `effects`. e2e 733 (with 732's stub): a refresh run's envelope lists + `network`; an offline run's does not. +- **L5.** The fetcher's pre-install refresh, its refresh before a retry, and + the first custom-index sync call `decide_for_miss` and apply its decision. + e2e 734: `auto_refresh = false` with a missing package runs no `update` in + the stub's argv log and names `mcpp index update`. +- **L6.** `refusal::Code::OfflineDownloadRequired` at the five offline sites and + the absent-index case; `emit build-database` maps it to + `MCPP_OFFLINE_DOWNLOAD_REQUIRED`. e2e 735. +- **L7.** The configuration template, the in-memory default and the + `.xlings.json` migration write `artifact` as + `{"GLOBAL": "https://github.com/xlings-res/mcpp-index", "CN": "https://gitcode.com/xlings-res/mcpp-index"}`. + Unit: the migration is idempotent and preserves unrelated state. + +### 2.2 W1: images and runtimes (R1 to R5) + +- **R1.** In the contract resolution, when the root image of a plan links a + `SharedLibrary` unit of the plan whose objects are C++, the `Distributable` + and `Test` roles on an ELF target take the `SharedLibrary` role's contract. + An explicit `cxx_runtime = "self-contained"` (or `{ program = ... }`) in + that graph is refused with `program-cxx-runtime-split`. e2e 700 (llvm and + gcc legs): the default shape runs; the explicit statement is refused. +- **R2.** `STB_GNU_UNIQUE` joins `STB_WEAK` as vague linkage; the check + receives, per image, the plan objects linked into it and suppresses a + duplicate defined by one object in both. Unit tests on both rules. e2e 701: + a uniform-contract program over a C++ shared library reports no + `build/symbol-provision` finding and `--strict` exits 0; a program with a + static zlib over a shared libz still reports. +- **R3.** `make_plan` computes each shared image's static closure over + `directPackageDeps`; a static package in exactly one closure is linked into + that image and removed from the root's objects; one in several closures is + refused (`static-package-in-two-images`) on Mach-O, PE and the Android app + row, and diagnosed (degraded) on other ELF rows. Unit: the six shapes of + §1.2. e2e 702: M3b's library links under `-Wl,-z,defs` and loads through a + foreign `dlopen`; the two-closure shape builds with the diagnostic on Linux + and without it after `linkage = "shared"`. +- **R4.** The PE contract table records `self-contained` for clang on the MSVC + ABI unless a runtime flag is emitted; an explicit `host-coupled` or + `toolchain-coupled` on that row is diagnosed; the `msvc_crt_flag` comment is + corrected; docs/20 states the row's model. Unit test of the table. e2e 703 + (Windows shard): `resolution.json` records `self-contained`. +- **R5.** e2e 704 (macOS shard): the #641 M3 app/fw pair without + `llvm.libcxx`, three legs, prints whether `std::runtime_error`, the + library's own exception and an `std::errc` comparison cross the dylib + boundary. e2e 705 (Windows shard): the same fixture over an llvm-row DLL. + Both print `READING` lines; the decision of T§4.4 is taken from them in + §1.9. + +### 2.3 W2: features, tools and git sources (G1 to G6) + +- **G1.** The forward validator reads the dependency tables of every row and + feature (`conditionalConfigs`) and the build-dependency table. e2e 710. +- **G2.** `mergeActiveFeatureDeps` refuses a restatement whose path, git + source or version differs from the declaration in effect on the row. docs/05 + and its mirror say to restate the source. e2e 711. +- **G3.** `published_names(provider, consumer)` is used by `fillDepDirs` and + the tool publication. e2e 187 gains a `namespace =` plus `name =` leg. +- **G4.** Step 0 is the scan of §1.2. A package whose targets are all + programs is not walked into a consumer's target graph and compiles nothing + there; its tools come from the sub-build. Package-edge cycles are detected + in the resolution pass with a `package-cycle` refusal naming the edges; the + tool chain refuses a repeated (source, tool) at its first repetition. e2e 712 + (the issue's fixture with the default cache; E6c's link line; E6b refused + once). +- **G5.** A git dependency whose identity is not the root manifest's is + looked up among the root's `[workspace] members` at the same commit; a + member's in-clone path edges resolve as the same git source; a second + declaration of one dependency by one consumer merges `tools`, `features`, + `host-module` and `reexport` into the edge; the banner prints the short + commit. e2e 713. +- **G6.** A `/` token in `--features` is split and applied as a root forward; + one naming no dependency warns (error under `--strict`) whether or not the + root declares `[features]`, and never becomes a macro. `mcpp why deps` + accepts `--features`. e2e 714. + +### 2.4 W3: build programs, the link line and pack (S1 to S5) + +- **S1.** Before the root build program, `prepare` writes + `/.mcpp/graph.json` (the `resolution.json` `graph` objects plus + `manifest_dir`, `features`, `targets` kinds, `link`, `metadata`) in + dependency order, and publishes its path as `MCPP_GRAPH_FILE` + (`mcpp::graph_file()`); its digest joins the contract environment. The + manifest parser keeps `[package.metadata]` as a raw table. e2e 720. +- **S2.** `link_shape(host, target_format, dialect)` selects the branch; + `tests/unit/test_build_flags.cpp` gains host-by-row cases for macOS and + Windows hosts. e2e 721 (macOS shard): `mcpp build --target + x86_64-linux-android` links an ELF object. +- **S3.** `pack::Plan` carries the graph's `SharedLibrary` outputs; + `run_shared_program` strips per leg; `bundle_libs`, `stage_closure` and the + multi-leg path strip graph-built libraries and toolchain runtime copies with + `--strip-unneeded`; the status line prints what was done; + `MCPP_PACK_STRIP` and `MCPP_PACK_DEBUG_SYMBOLS_DIR`. e2e 722 (Linux desktop + and, where the NDK is present, Android). +- **S4.** `pack` accepts `--release`, `--dev` and `--message-format json`; + the `mcpp.pack` kind is in `kKinds` and `--protocol-version`; `run` resolves + the profile with `build`'s function. e2e 723. +- **S5.** `[package]` reports an unknown key the way `[build]` does, with + `metadata` known. Unit test in the manifest suite. + +## 3. Plugin tasks + +One pull request on mcpp-plugins, version 0.12.0, after the engine release +(P3 and P5 need the released accessors; P1, P2 and P4 do not, and are written +first). Its CI runs the consumers against the pinned engine and, before the +release, against `mcpp_ref=feat/646-649`. + +## 4. Index and project tasks + +- **X1** is independent of the engine and merges first; the next index publish + republishes the artifact, and the group d probe of the triage record then + reads three equal digests. +- **X2** is the release bot's pull request. +- **X3** follows X2 and the plugins tag. +- **X4** is a two-line manifest change in openxlings/xlings. + +## 5. Order + +1. X1 and X4 open at once (independent of everything). +2. W1, W2, W3 and the lead's L1-L6 in parallel; the lead merges each tree into + `feat/646-649` as it completes, builds, and runs the unit tests and the new + e2e scripts. +3. L7, C1 and C2 on the integrated branch; one pull request; CI green, + including the measurement legs; §1.9 records R5's readings and the decision + they imply. +4. Self-review of the diff and of the ecosystem; merge. +5. Release 2026.9.16.1; GitCode assets uploaded locally as each archive + appears; X2. +6. P1-P5 on the plugins branch; its CI against the released engine; tag 0.12.0, + mirrored to GitCode; X3. +7. V1; the bootstrap pin; V2; C3. + +## 6. Release chain + +`gh workflow run release.yml --ref main` from a clean `main` whose HEAD is the +merge commit, with nothing pushed to `main` until the tag exists. Each archive +and its `.sha256` go to `xlings-res/mcpp` on GitCode with the local tool as soon +as the build job puts it on the GitHub release; an asset counts as done when a +GET returns 200 with the upstream size and the bytes compare equal. +`publish-ecosystem` then opens the index pull request, whose four hashes are +compared with the downloaded archives by script. The release is complete when +the published xim index artifact (not git main) names 2026.9.16.1 as `latest`. + +## 7. Sandbox verification (V1) + +A new SubOS `v646`, entered with `xlings subos use v646 --sandbox --cmd`, +after `xlings config --mirror CN` and `mcpp self config --mirror CN` inside it. +The engine is addressed by its store path and exact version. The script runs +first against 2026.9.15.2 (change detectors fail, guards pass), then against +2026.9.16.1 (all pass). Scenarios, each with a `READING` line: + +1. F3a: an llvm program over a C++ shared library runs. +2. F1: a shared package over a static package links under `-z defs`. +3. E6: a feature-gated tool that depends on its declaring package builds and + runs; `dep_bin` answers under both spellings. +4. E8: `--features /` builds the dependency's tool. +5. E1: the root build program reads the graph file and a dependency's + metadata. +6. E5: `mcpp pack --format tar` ships a stripped graph-built `libdep.so`. +7. E9: `mcpp pack --release --message-format json` prints a record whose path + exists. +8. #648: offline `emit build-database` with a removed package yields + `MCPP_OFFLINE_DOWNLOAD_REQUIRED`; a build program lists no inherited pipe; + a bare `compat.*` dependency produces no refresh decision. +9. A6: `mcpp index update` with `mirror = CN` succeeds from GitCode. +10. plugins 0.12.0 from the index: `dist-web` with `page` names the page. + +## 8. Parallel work trees + +| tree | branch | owns | may touch minimally | +|---|---|---|---| +| lead `mcpp-646` | `feat/646-649` | `modules/platform/src/{terminal,process}.cppm`, `modules/platform/src/{unix,windows}/bounded_process.cppm`, `src/xlings/xlings.cppm`, `src/pm/index_refresh.cppm`, `src/pm/package_fetcher.cppm`, `src/config.cppm`, the refresh, custom-index and offline-refusal blocks of `prepare.cppm`, the `emit build-database` function of `cmd_build.cppm`, the effects half of `src/wire.cppm`, workflows, CHANGELOG, version files | `src/build/refusal.cppm` (one code) | +| W1 `mcpp-646-runtime` | `feat/646-649-runtime` | `src/build/distribution.cppm`, the link-unit assembly of `src/build/plan.cppm`, `src/build/symbol_provision.cppm`, `src/runtime/elf.cppm`, `src/build/runtime_validation.cppm`, `src/toolchain/dialect.cppm`, the CRT block of `flags.cppm`, docs/20, docs/22 | `src/build/refusal.cppm` (two codes), the contract call site in `prepare.cppm` | +| W2 `mcpp-646-graph` | `feat/646-649-graph` | the feature, forward, worklist, edge, git-source, tool-publication and cache-key-walk blocks of `prepare.cppm`, `src/build/execute.cppm` (banner), `src/build/tool_store.cppm`, the `why deps` option table in `src/cli.cppm`, docs/05 (features and git), docs/06, docs/07, the `dep_bin` section of docs/30 | `modules/manifest` (feature-deps check), `src/build/refusal.cppm` (one code) | +| W3 `mcpp-646-surface` | `feat/646-649-surface` | the root build program block and `resolution.json` writer of `prepare.cppm`, `src/build/hostprogram.cppm`, `src/build/build_program.cppm`, the link-branch block of `flags.cppm`, `src/pack/*`, the pack option table of `src/cli.cppm`, the `run` profile resolution of `cmd_build.cppm`, the kinds table of `src/wire.cppm`, `tests/unit/test_build_flags.cpp`, docs/10, the accessor sections of docs/30, the `mcpp.pack` section of docs/50 | `modules/manifest` (`[package]` keys) | + +e2e numbers: W1 700-709, W2 710-719, W3 720-729, lead 730-739. A tree removes +its `target/` after its branch is merged. + +## 9. Closure + +(written when the rows of §0 are closed) \ No newline at end of file diff --git a/.agents/docs/2026-09-16-646-649-probes.sh b/.agents/docs/2026-09-16-646-649-probes.sh new file mode 100755 index 000000000..c00fe64f2 --- /dev/null +++ b/.agents/docs/2026-09-16-646-649-probes.sh @@ -0,0 +1,778 @@ +#!/usr/bin/env bash +# Probes for .agents/docs/2026-09-16-646-649-four-issues-by-home.md. +# +# Four independent groups. Each is written out as a standalone script and run +# with its own options and variables. Every result is a line starting +# with READING, and the record quotes those lines. +# +# a #646 F1, F3 and the driver half of #649 E10 (needs gcc@16.1.0, llvm@22.1.8) +# b #647 E1 and #649 E5, E9 (the Android leg needs xim:android-ndk installed) +# c #647 E4 and #649 E6, E7, E8 (host only) +# d #648: the inherited pipe (A2), the refresh trigger (T), the mirror digests (A6) +# +# Usage: 2026-09-16-646-649-probes.sh [a|b|c|d|all] [work dir] +# Engine: the released mcpp 2026.9.15.2 (override with M=/path/to/mcpp); home ~/.mcpp. +# Group d's T reading plans a checkout of openxlings/xlings (XL=). +set -u +WHICH=${1:-all} +WORK=${2:-$(mktemp -d)} +mkdir -p "$WORK" && WORK=$(cd "$WORK" && pwd) +export M=${M:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/2026.9.15.2/bin/mcpp} +export MCPP="$M" + +group_a() { mkdir -p "$WORK/a"; cat > "$WORK/a.sh" <<'__PROBES_646_649_GROUP_A__' +# Group a: probes for #646 F1/F3 (Linux x86_64) and the driver half of #649 E10. +# Engine: the released mcpp 2026.9.15.2 (xlings package; home ~/.mcpp). +# Every result is a line starting with READING. +# Usage: probes.sh [work dir] (default: ./work next to this script) +set -u +HERE=$(cd "$(dirname "$0")" && pwd) +WORK=${1:-$HERE/work} +mkdir -p "$WORK" && WORK=$(cd "$WORK" && pwd) +M=${MCPP:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/2026.9.15.2/bin/mcpp} +NINJA=$(find "$HOME/.mcpp/registry/data/xpkgs/xim-x-ninja" -name ninja -type f | head -1) +LLVM=$HOME/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8 +GCCBIN=$HOME/.mcpp/registry/data/xpkgs/xim-x-gcc/16.1.0/bin +unset XLINGS_ACTIVE_SUBOS +reading() { echo "READING $*"; } +# Replay the engine's own link of bin/libfw.so from the build directory, with +# extra inputs ($1) and extra flags ($2). Ninja deletes response files after a +# link, so the file is rewritten from the edge's inputs, as probe M5 did. +relink_fw() { + local cmd ins + cmd=$("$NINJA" -t commands bin/libfw.so | tail -1) + ins=$(grep '^build bin/libfw.so' build.ninja | sed 's/^build bin\/libfw.so : c[a-z_]* //; s/ || .*//; s/ | .*//') + case "$cmd" in + *libfw.so.rsp*) printf '%s %s\n' "$ins" "$1" > bin/libfw.so.rsp; eval "$cmd $2" ;; + *) eval "${cmd/ -o / $1 -o } $2" ;; + esac +} +"$M" --version | sed 's/^/READING engine: /' + +# ---------------------------------------------------------------- F1 -------- +# F1a: M3b verbatim (x static C, fw shared C over x, root C++ over fw). +f1_fixture() { # $1 dir, $2 extra root deps line (may be empty), $3 main body + mkdir -p "$1/app/src" "$1/fw/src" "$1/x/src" + printf '[package]\nname = "x"\nversion = "0.1.0"\n[build]\nsources = ["src/*.c"]\n[targets.x]\nkind = "lib"\n' > "$1/x/mcpp.toml" + printf 'int x_counter = 0;\nint x_answer(void) { return 41; }\nint x_bump(void) { return ++x_counter; }\n' > "$1/x/src/x.c" + printf '[package]\nname = "fw"\nversion = "0.1.0"\n[build]\nsources = ["src/*.c"]\n[targets.fw]\nkind = "shared"\n[dependencies.x]\npath = "../x"\n' > "$1/fw/mcpp.toml" + printf 'extern int x_answer(void);\nextern int x_bump(void);\nint fw_answer(void) { return x_answer() + 1; }\nint fw_bump(void) { return x_bump(); }\n' > "$1/fw/src/fw.c" + printf '[package]\nname = "app"\nversion = "0.1.0"\n[toolchain]\ndefault = "gcc@16.1.0"\n[dependencies.fw]\npath = "../fw"\n%s' "$2" > "$1/app/mcpp.toml" + printf '%s\n' "$3" > "$1/app/src/main.cpp" +} +f1_fixture "$WORK/f1a" "" 'extern "C" int fw_answer(void); +int main() { return fw_answer() == 42 ? 0 : 1; }' +cd "$WORK/f1a/app" && timeout 900 "$M" build > build.log 2>&1 +reading "F1a M3b exit=$?" +d=$(ls -d target/*/*/ | head -1) +so=$d/bin/libfw.so; bin=$d/bin/app +reading "F1a libfw.so inputs: $(grep '^build bin/libfw.so' "$d/build.ninja" | sed 's/^build bin\/libfw.so : c[a-z_]* //; s/ |.*//')" +reading "F1a libfw.so undefined x_answer=$(nm -D --undefined-only "$so" | grep -c ' x_answer$') program defines x_answer=$(nm "$bin" | grep -c ' T x_answer$') program exports x_answer=$(nm -D --defined-only "$bin" | grep -c ' x_answer$')" +"$bin"; reading "F1a run exit=$?" + +# F1c: the same library linked the way Mach-O and PE link (every reference +# resolved at link time): ELF's -z defs. +cd "$d" +relink_fw "" "-Wl,-z,defs" > f1c.log 2>&1 +reading "F1c libfw.so relinked with -z defs: exit=$? $(grep -m1 -o 'undefined reference to .x_answer.\|undefined symbol: x_answer' f1c.log)" +relink_fw "" "" > /dev/null 2>&1 # restore + +# F1d: a host that did not link x (a foreign program, or Android's +# System.loadLibrary("fw") before "app") loads the library with RTLD_NOW. +cat > "$WORK/f1a/host.c" <<'C' +#include +#include +int main(int argc, char** argv) { + void* h = dlopen(argv[1], RTLD_NOW | RTLD_LOCAL); + if (!h) { printf("dlopen failed: %s\n", dlerror()); return 1; } + printf("dlopen ok\n"); return 0; +} +C +"$GCCBIN/gcc" -o "$WORK/f1a/host" "$WORK/f1a/host.c" -ldl > /dev/null 2>&1 +out=$("$WORK/f1a/host" "$PWD/bin/libfw.so" 2>&1); rc=$? +reading "F1d foreign host dlopen(libfw.so, RTLD_NOW): exit=$rc output: $out" + +# F1b: x reachable from both the program and the shared image. +f1_fixture "$WORK/f1b" '[dependencies.x] +path = "../x" +' '#include +extern "C" int fw_answer(void); extern "C" int fw_bump(void); +extern "C" int x_bump(void); extern "C" int x_counter; +int main() { x_bump(); fw_bump(); std::printf("counter=%d\n", x_counter); return fw_answer() == 42 ? 0 : 1; }' +cd "$WORK/f1b/app" && timeout 900 "$M" build > build.log 2>&1 +reading "F1b root also depends on x: exit=$?" +d=$(ls -d target/*/*/ | head -1) +reading "F1b x.o linked into: program=$(grep '^build bin/app ' "$d/build.ninja" | grep -c 'x\.c\.o\|/x\.o') libfw.so=$(grep '^build bin/libfw.so' "$d/build.ninja" | grep -c 'x\.c\.o\|/x\.o')" +out=$("$d/bin/app"); reading "F1b run exit=$? $out" + +# F1f: what "the static package goes into the image" produces on ELF when the +# program links it as well: relink libfw.so with x's objects added. +cd "$d" +xo=$(grep -o '[^ ]*x\.c\.o\|[^ ]*/x\.o' build.ninja | head -1) +relink_fw "$xo" "" > f1f.log 2>&1 +reading "F1f libfw.so relinked with x.o: exit=$? libfw defines x_counter=$(nm -D --defined-only bin/libfw.so | grep -c ' x_counter$') run: $(./bin/app) (1 shared counter => 2; split => 1)" + +# F1e: the root itself is the shared image over x (root-owned unit). +mkdir -p "$WORK/f1e" && cp -r "$WORK/f1a/x" "$WORK/f1e/x" && mkdir -p "$WORK/f1e/fw/src" +printf '[package]\nname = "fw"\nversion = "0.1.0"\n[toolchain]\ndefault = "gcc@16.1.0"\n[build]\nsources = ["src/*.c"]\n[targets.fw]\nkind = "shared"\n[dependencies.x]\npath = "../x"\n' > "$WORK/f1e/fw/mcpp.toml" +cp "$WORK/f1a/fw/src/fw.c" "$WORK/f1e/fw/src/fw.c" +cd "$WORK/f1e/fw" && timeout 900 "$M" build > build.log 2>&1 +so=$(ls target/*/*/bin/libfw.so | head -1) +reading "F1e root-owned shared image over x: exit=$? libfw.so defines x_answer=$(nm -D --defined-only "$so" | grep -c ' x_answer$') undefined=$(nm -D --undefined-only "$so" | grep -c ' x_answer$')" + +# ---------------------------------------------------------------- F3 -------- +F3_DEFAULT_MAIN='import std; +import lib; +int main() { std::println("{}", lib_greet(3)); }' +f3() { # $1 id, $2 toolchain, [$3 root manifest tail], [$4 lib kind], [$5 main body] + local w="$WORK/f3-$1" + mkdir -p "$w/app/src" "$w/lib/src" + printf '[package]\nname = "lib"\nversion = "0.1.0"\n[targets.lib]\nkind = "%s"\n' "${4:-shared}" > "$w/lib/mcpp.toml" + printf 'export module lib;\nimport std;\nexport [[gnu::visibility("default")]] std::string lib_greet(int n);\n' > "$w/lib/src/lib.cppm" + printf 'module lib;\nimport std;\nstd::string lib_greet(int n) { return std::format("lib-{}", n); }\n' > "$w/lib/src/lib.cpp" + printf '[package]\nname = "app"\nversion = "0.1.0"\n[toolchain]\ndefault = "%s"\n[dependencies.lib]\npath = "../lib"\n%s' "$2" "${3:-}" > "$w/app/mcpp.toml" + local body=${5:-$F3_DEFAULT_MAIN} + printf '%s\n' "$body" > "$w/app/src/main.cpp" + cd "$w/app" + timeout 1800 "$M" build > build.log 2>&1; local rc=$? + reading "F3 $1 build exit=$rc symbol-provision mentions=$(grep -c 'symbol-provision\|also provided by a library it loads' build.log)" + local d; d=$(ls -d target/*/*/ 2>/dev/null | head -1) + [ -n "$d" ] || return + reading "F3 $1 run: $("$d/bin/app" 2>&1) exit=$?" + reading "F3 $1 NEEDED program: $(readelf -d "$d/bin/app" | grep -o 'Shared library: \[[^]]*\]' | tr '\n' ' ')" + [ -f "$d/bin/liblib.so" ] || { reading "F3 $1 no liblib.so (static form)"; return; } + reading "F3 $1 NEEDED liblib.so: $(readelf -d "$d/bin/liblib.so" | grep -o 'Shared library: \[[^]]*\]' | tr '\n' ' ')" + reading "F3 $1 contracts in resolution.json: $(python3 -c 'import json,sys,glob; d=json.load(open(sys.argv[1])); print(json.dumps(d["runtime"]["cxx_runtime_by_role"]))' "$d/resolution.json" 2>&1)" + reading "F3 $1 liblib.so link inputs: $(grep '^build bin/liblib.so' "$d/build.ninja" | sed 's/^build bin\/liblib.so : c[a-z_]* //; s/ |.*//')" + python3 - "$d/.mcpp-runtime-verdicts.json" "$d/obj/std.o" "$1" <<'PY' +import json, subprocess, sys +doc = json.load(open(sys.argv[1])) +def walk(o): + if isinstance(o, dict): + if 'conflicts' in o and 'path' in o: yield o + for v in o.values(): yield from walk(v) + elif isinstance(o, list): + for v in o: yield from walk(v) +stdo = set() +try: + out = subprocess.run(['nm', '--defined-only', '-g', sys.argv[2]], capture_output=True, text=True).stdout + stdo = {l.split()[-1] for l in out.splitlines() if l.strip()} + strong = sum(1 for l in out.splitlines() if l.split()[-2] in ('T','D','B','R')) + print(f"READING F3 {sys.argv[3]} std.o global definitions={len(stdo)} strong={strong} sample={sorted(stdo)[:3]}") +except Exception as e: + print(f"READING F3 {sys.argv[3]} std.o unreadable: {e}") +for e in walk(doc): + c = e.get('conflicts') or [] + names = [x.get('symbol') for x in c] + fromstd = sum(1 for n in names if n in stdo) + by = sorted({p for x in c for p in x.get('also_provided_by', [])}) + print(f"READING F3 {sys.argv[3]} {e['path']} status={e.get('status')} exported={e.get('exported')} conflicts={len(c)} defined-in-std.o={fromstd} first={names[:4]} providers={by}") +PY + timeout 600 "$M" build --strict > strict.log 2>&1 + reading "F3 $1 --strict exit=$? (fast path may skip the check: $(grep -c 'symbol-provision' strict.log) mentions)" + touch src/main.cpp; timeout 600 "$M" build --strict > strict2.log 2>&1 + reading "F3 $1 --strict after touch exit=$? mentions=$(grep -c 'symbol-provision\|also provided by' strict2.log)" +} +f3 gcc "gcc@16.1.0" +f3 llvm "llvm@22.1.8" +# Controls: which part of the default shape the findings and the abort belong to. +f3 llvm-static-lib "llvm@22.1.8" "" "lib" +f3 llvm-no-println "llvm@22.1.8" "" "shared" 'import lib; +#include +int main() { std::puts(lib_greet(3).c_str()); }' +f3 llvm-tc "llvm@22.1.8" '[build] +cxx_runtime = "toolchain-coupled" +' +f3 gcc-tc "gcc@16.1.0" '[build] +cxx_runtime = "toolchain-coupled" +' + +# ---------------------------------------------------------------- E10 ------- +# The GNU-style clang driver on the MSVC triple, with no runtime flag: which +# CRT does it select at compile and at link? (Driver logic is host-independent.) +cd "$WORK" && printf 'int main(){return 0;}\n' > e10.cpp +c=$("$LLVM/bin/clang++" --target=x86_64-pc-windows-msvc -### -c e10.cpp 2>&1 | tail -1) +reading "E10 compile, no flag: dependent-lib=$(echo "$c" | grep -o -- '--dependent-lib=[a-z]*' | tr '\n' ' ') defines=$(echo "$c" | grep -o -- '"-D" "_[A-Z]*"' | tr '\n' ' ')" +l=$("$LLVM/bin/clang++" --target=x86_64-pc-windows-msvc -fuse-ld=lld -### e10.o -o e10.exe 2>&1 | tail -1) +reading "E10 link, no flag: $(echo "$l" | grep -o -- '-defaultlib:[a-z]*' | tr '\n' ' ')" +for v in dll static; do + c=$("$LLVM/bin/clang++" --target=x86_64-pc-windows-msvc -fms-runtime-lib=$v -### -c e10.cpp 2>&1 | tail -1) + l=$("$LLVM/bin/clang++" --target=x86_64-pc-windows-msvc -fms-runtime-lib=$v -fuse-ld=lld -### e10.o -o e10.exe 2>&1 | tail -1) + reading "E10 -fms-runtime-lib=$v: compile $(echo "$c" | grep -o -- '--dependent-lib=[a-z]*\|"-D" "_[A-Z]*"' | tr '\n' ' ') link $(echo "$l" | grep -o -- '-defaultlib:[a-z]*' | tr '\n' ' ')" +done +__PROBES_646_649_GROUP_A__ + bash "$WORK/a.sh" "$WORK/a" "$WORK/b.sh" <<'__PROBES_646_649_GROUP_B__' +# Group b: probes for #647 E1 and #649 E5/E9, against the released mcpp 2026.9.15.2 +# on a Linux x86_64 host. Each reading is printed as `READING : ...`. +# Usage: bash probes.sh [workdir] (the Android leg needs xim:android-ndk installed) +set -uo pipefail +M=${M:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/2026.9.15.2/bin/mcpp} +W=${1:-$(mktemp -d)} +mkdir -p "$W"; cd "$W" +reading() { echo "READING $*"; } +reading "version: $($M --version)" + +# ── E1: app -> a -> b (path deps); b declares [package.metadata.demo] ────── +mkdir -p e1/app/src e1/a/src e1/b/src +cat > e1/b/mcpp.toml <<'EOF' +[package] +name = "b" +namespace = "spike" +version = "0.2.0" +standard = "c++20" + +[package.metadata.demo] +resources = "res" + +[targets.b] +kind = "lib" + +[build] +sources = ["src/*.cpp"] +EOF +echo 'int b_answer() { return 40; }' > e1/b/src/b.cpp +cat > e1/a/mcpp.toml <<'EOF' +[package] +name = "a" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.b = { path = "../b" } + +[targets.a] +kind = "lib" + +[build] +sources = ["src/*.cpp"] +EOF +echo 'int b_answer(); int a_answer() { return b_answer() + 2; }' > e1/a/src/a.cpp +cat > e1/app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.a = { path = "../a" } + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +printf 'int a_answer();\nint main() { return a_answer() == 42 ? 0 : 1; }\n' > e1/app/src/main.cpp +cat > e1/app/build.mcpp <<'EOF' +import std; +import mcpp; +extern char** environ; +int main() { + std::string s = "SPIKE deps:"; + for (char** e = environ; *e; ++e) { + std::string_view v(*e); + if (v.starts_with("MCPP_DEP_")) { s += " "; s += v.substr(0, v.find('=')); } + } + mcpp::warning(s.c_str()); + std::string c = "SPIKE run " + std::to_string(std::chrono::system_clock::now().time_since_epoch().count()); + mcpp::warning(c.c_str()); + return 0; +} +EOF +for p in a b; do cat > e1/$p/build.mcpp <&1); rc=$? +reading "E1-strict-metadata-table: exit=$rc warnings=$(grep -c '^warning: \(schema\|\[package\]\)' <<<"$out")" +reading "E1-root-env: $(grep -o 'SPIKE deps:.*' <<<"$out")" +reading "E1-program-order: $(grep -o 'SPIKE order [ab]' <<<"$out" | tr '\n' ' ')" +f=$(find target -name resolution.json | head -1) +reading "E1-graph-record: $(python3 -c "import json;j=json.load(open('$f'));print([ (p['package']['canonical'], sorted(p.keys())) for p in j['graph']['packages']])")" +first=$(grep -o 'SPIKE run [0-9]*' <<<"$out") +sleep 1; sed -i 's/resources = "res"/resources = "res2"/' ../b/mcpp.toml +out=$($M build 2>&1) +reading "E1-metadata-edit: $(grep -o 'build.mcpp [a-z ()]*' <<<"$out" | tr '\n' ';') replayed=$([ "$(grep -o 'SPIKE run [0-9]*' <<<"$out")" = "$first" ] && echo yes || echo no)" +sed -i 's/^standard = "c++20"/standard = "c++20"\nbogus_key = 1/' mcpp.toml +out=$($M build --strict 2>&1); rc=$? +reading "E1-unknown-package-key-strict: exit=$rc mentions=$(grep -c bogus_key <<<"$out")" +cd "$W" + +# ── E5 / E9: shared dependency, Android app row and Linux desktop row ──────── +mkdir -p e5/dep/src e5/app/src e5/hostapp/src +cat > e5/dep/mcpp.toml <<'EOF' +[package] +name = "dep" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[targets.dep] +kind = "shared" + +[build] +sources = ["src/*.cpp"] +EOF +printf '#include \n[[gnu::visibility("default")]] int dep_answer() { return static_cast(std::to_string(42).size()) + 40; }\n' > e5/dep/src/dep.cpp +cat > e5/app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.dep = { path = "../dep" } + +[targets.app] +kind = "app" +main = "src/main.cpp" + +[target.x86_64-linux-android] +min_api_level = 23 +EOF +printf '#include \nint dep_answer();\n[[gnu::visibility("default")]] int app_entry() { return dep_answer() + static_cast(std::to_string(1).size()); }\nint main() { return app_entry() == 43 ? 0 : 1; }\n' > e5/app/src/main.cpp +cat > e5/hostapp/mcpp.toml <<'EOF' +[package] +name = "hostapp" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.dep = { path = "../dep" } + +[targets.hostapp] +kind = "bin" +main = "src/main.cpp" +EOF +printf 'int dep_answer();\nint main() { return dep_answer() == 42 ? 0 : 1; }\n' > e5/hostapp/src/main.cpp + +sections() { echo "bytes=$(stat -c %s "$1") symtab=$(readelf -S -W "$1" | grep -c ' .symtab') debug=$(readelf -S -W "$1" | grep -c ' .debug_')"; } + +cd e5/hostapp +line=$($M pack --format tar 2>&1 | grep -E 'Packing') +reading "E5-desktop-status: $line" +rm -rf x && mkdir x && tar -xzf target/dist/hostapp-0.1.0-x86_64-linux-gnu.tar.gz -C x +for f in x/*/bin/hostapp x/*/lib/libdep.so; do reading "E5-desktop: ${f#x/*/} $(sections "$f")"; done +for a in "--release" "--dev" "--message-format json"; do + err=$($M pack $a --format tar 2>&1 >/dev/null | tail -1); $M pack $a --format tar >/dev/null 2>&1; rc=$? + reading "E9-pack $a: exit=$rc [$err]" +done +reading "E9-human-lines-on-stdout: bytes=$($M pack --format tar 2>/dev/null | wc -c)" +reading "E9-run-precedence(--profile dev --release): $($M run --profile dev --release 2>&1 | grep -o 'Finished [a-z]*')" +reading "E9-build-precedence(--profile dev --release): $($M build --profile dev --release 2>&1 | grep -o 'Finished [a-z]*')" +cd "$W" + +if ls -d "$HOME"/.mcpp/registry/data/xpkgs/xim-x-android-ndk/* >/dev/null 2>&1; then + cd e5/app + line=$(MCPP_OFFLINE=1 $M pack --target x86_64-linux-android --format tar 2>&1 | grep -E 'Packing') + reading "E5-android-status: $line" + rm -rf x && mkdir x && tar -xzf target/dist/app-0.1.0-x86_64-linux-android.tar.gz -C x + for f in x/*/lib/*.so; do reading "E5-android: ${f#x/*/} $(sections "$f")"; done + cd "$W" +else + reading "E5-android: skipped (xim:android-ndk not installed)" +fi +command -v swiftc >/dev/null || reading "E2-host: no swiftc on this host; E2 needs macos-15" +__PROBES_646_649_GROUP_B__ + bash "$WORK/b.sh" "$WORK/b" "$WORK/c.sh" <<'__PROBES_646_649_GROUP_C__' +# Group c: probes for #647 E4 and #649 E6/E7/E8, against the released mcpp 2026.9.15.2. +# Usage: probes.sh [e41|e42|e43|e6|e6b|e6c|e7|e8|all] +set +e +M=${M:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/2026.9.15.2/bin/mcpp} +ROOT=${ROOT:-$(cd "$(dirname "$0")" && pwd)/work} +reading() { echo "READING $1: $2"; } +pkg() { # dir ns name kind [extra toml] + mkdir -p "$1/src" + { echo '[package]'; echo "name = \"$3\""; [ -n "$2" ] && echo "namespace = \"$2\""; + echo 'version = "0.1.0"'; echo 'standard = "c++20"'; echo; echo "[targets.$3]"; echo "kind = \"$4\""; + [ "$4" = bin ] && echo 'main = "src/main.cpp"'; [ "$4" = lib ] && { echo; echo '[build]'; echo 'sources = ["src/*.cpp"]'; } + echo; printf '%s\n' "${5:-}"; } > "$1/mcpp.toml" +} +run() { # label dir args... + local label=$1 dir=$2; shift 2 + out=$(cd "$dir" && timeout 900 "$M" "$@" 2>&1); rc=$? + printf '%s\n' "$out" > "$ROOT/$label.log" + echo "rc=$rc" >> "$ROOT/$label.log" +} +mkdir -p "$ROOT" +$M --version + +e41() { # a forward along [build-dependencies] edges, two levels, host only + local W=$ROOT/e41; rm -rf "$W"; mkdir -p "$W" + pkg "$W/kt" spike kt bin; echo 'int main(){return 0;}' > "$W/kt/src/main.cpp" + pkg "$W/leaf" spike leaf lib '[features] +kt = [] + +[feature-deps.kt] +spike.kt = { path = "../kt", tools = ["kt"] }' + echo 'int leaf_answer(){return 1;}' > "$W/leaf/src/leaf.cpp" + pkg "$W/rules" spike rules lib '[build-dependencies] +spike.leaf = { path = "../leaf" } + +[features] +kotlin = ["spike.leaf/kt"]' + echo 'int rules_answer(){return 1;}' > "$W/rules/src/rules.cpp" + pkg "$W/fw" spike fw lib '[build-dependencies] +spike.rules = { path = "../rules" } + +[features] +kotlin = ["spike.rules/kotlin"]' + echo 'int fw_answer(){return 42;}' > "$W/fw/src/fw.cpp" + pkg "$W/app" "" app bin '[dependencies] +spike.fw = { path = "../fw", features = ["kotlin"] }' + printf 'int fw_answer();\nint main(){return fw_answer()==42?0:1;}\n' > "$W/app/src/main.cpp" + run e41-build "$W/app" build + reading E4.1-build "$(grep -c 'forwards to dependency' $ROOT/e41-build.log) forward warnings; tool kt built: $(grep -c 'Building host tool kt' $ROOT/e41-build.log); $(tail -1 $ROOT/e41-build.log)" + grep 'forwards to dependency' $ROOT/e41-build.log | sed 's/^/ /' + run e41-strict "$W/app" build --strict + reading E4.1-strict "$(grep -m1 -o 'error: .*' $ROOT/e41-strict.log | cut -c1-120); $(tail -1 $ROOT/e41-strict.log)" + sed -i 's/features = \["kotlin"\]/features = []/' "$W/app/mcpp.toml"; rm -rf "$W/app/target" + run e41-off "$W/app" build + reading E4.1-control-without-feature "tool kt built: $(grep -c 'Building host tool kt' $ROOT/e41-off.log); $(tail -1 $ROOT/e41-off.log)" + rm -rf "$W"/*/target +} + +e42() { # [feature-deps.] adding tools to an unconditionally declared dependency + local W=$ROOT/e42; rm -rf "$W"; mkdir -p "$W" + pkg "$W/installer" spike installer bin; echo '#include +int main(){std::puts("installer ran");return 0;}' > "$W/installer/src/main.cpp" + # the installer package also needs a library so that the unconditional edge means something + pkg "$W/app" "" app bin '[dependencies] +spike.installer = { path = "../installer" } + +[features] +installer = [] + +[feature-deps.installer] +spike.installer = { tools = ["installer"] }' + echo 'int main(){return 0;}' > "$W/app/src/main.cpp" + run e42-sourceless "$W/app" build --features installer + reading E4.2-sourceless "$(grep -m1 -o 'error: .*' $ROOT/e42-sourceless.log | cut -c1-160); $(tail -1 $ROOT/e42-sourceless.log)" + # restated with the same source: the documented intent, spelled as the parser requires + sed -i 's|spike.installer = { tools = \["installer"\] }|spike.installer = { path = "../installer", tools = ["installer"] }|' "$W/app/mcpp.toml" + cat > "$W/app/build.mcpp" <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("installer", "installer"); + const char* b = mcpp::dep_bin("spike.installer", "installer"); + mcpp::warning((std::string("SPIKE short=[") + (a ? a : "") + "] qualified=[" + (b ? b : "") + "]").c_str()); + return 0; +} +EOF + run e42-restated "$W/app" build --features installer + reading E4.2-restated "$(grep -o 'SPIKE.*' $ROOT/e42-restated.log | sed "s|$HOME|~|g" | cut -c1-200); $(tail -1 $ROOT/e42-restated.log)" + rm -rf "$W/app/target" + run e42-restated-off "$W/app" build + reading E4.2-restated-without-feature "$(grep -o 'SPIKE.*' $ROOT/e42-restated-off.log); $(tail -1 $ROOT/e42-restated-off.log)" + # a restatement naming a DIFFERENT source than the unconditional declaration + mkdir -p "$W/other"; cp -r "$W/installer/." "$W/other/" + sed -i 's|spike.installer = { path = "../installer", tools|spike.installer = { path = "../other", tools|' "$W/app/mcpp.toml" + rm -rf "$W/app/target" + run e42-mismatch "$W/app" build --features installer --strict + reading E4.2-restated-different-path "warnings=$(grep -c '^warning' $ROOT/e42-mismatch.log) tool-from: $(grep -o 'Building host tool.*' $ROOT/e42-mismatch.log | head -1) $(grep -o 'SPIKE.*' $ROOT/e42-mismatch.log | sed "s|$HOME|~|g" | cut -c1-200); $(tail -1 $ROOT/e42-mismatch.log)" + rm -rf "$W"/*/target +} + + +e41b() { # a forward to a dependency declared for another row, and to one behind an inactive feature + local W=$ROOT/e41b; rm -rf "$W"; mkdir -p "$W" + pkg "$W/win" spike win lib '[features] +x = []' + echo 'int win_answer(){return 1;}' > "$W/win/src/win.cpp" + pkg "$W/fw" spike fw lib '[target.'"'"'cfg(os = "windows")'"'"'.dependencies] +spike.win = { path = "../win" } + +[features] +kotlin = ["spike.win/x"]' + echo 'int fw_answer(){return 42;}' > "$W/fw/src/fw.cpp" + pkg "$W/app" "" app bin '[dependencies] +spike.fw = { path = "../fw", features = ["kotlin"] }' + printf 'int fw_answer();\nint main(){return fw_answer()==42?0:1;}\n' > "$W/app/src/main.cpp" + run e41b-other-row "$W/app" build --strict + reading E4.1b-forward-to-other-row-dep "$(grep -m1 -o 'error: .*' $ROOT/e41b-other-row.log | cut -c1-140); $(tail -1 $ROOT/e41b-other-row.log)" + rm -rf "$W"/*/target +} + + +e6_fixture() { # W installer-edge-toml tool-dep-toml + local W=$1; rm -rf "$W"; mkdir -p "$W" + pkg "$W/fw" spike fw lib "[features] +installer = [] + +$2" + echo 'int fw_answer() { return 42; }' > "$W/fw/src/fw.cpp" + pkg "$W/fw/tool" spike fw-installer bin "$3" + printf 'int fw_answer();\n#include \nint main() { std::printf("fw-installer ran %%d\\n", fw_answer()); return 0; }\n' > "$W/fw/tool/src/main.cpp" + pkg "$W/app" "" app bin '[dependencies] +spike.fw = { path = "../fw" } + +[features] +windows-installer = ["spike.fw/installer"]' + printf 'int fw_answer();\nint main() { return fw_answer() == 42 ? 0 : 1; }\n' > "$W/app/src/main.cpp" + cat > "$W/app/build.mcpp" <<'EOF2' +import std; +import mcpp; + +int main() { + const char* a = mcpp::dep_bin("fw-installer", "fw-installer"); + const char* b = mcpp::dep_bin("spike.fw-installer", "fw-installer"); + const std::string message = std::string("SPIKE short=[") + (a ? a : "") + "] qualified=[" + (b ? b : "") + "]"; + mcpp::warning(message.c_str()); + return 0; +} +EOF2 +} +e6() { # the issue's fixture + local W=$ROOT/e6 + e6_fixture "$W" "[target.'cfg(os = \"linux\")'.feature-deps.installer] +spike.fw-installer = { path = \"tool\", tools = [\"fw-installer\"], reexport = true }" '[dependencies] +spike.fw = { path = ".." }' + run e6-build "$W/app" build --features windows-installer + reading E6-issue-fixture "$(grep -o 'SPIKE.*' $ROOT/e6-build.log | sed "s|$HOME|~|g" | cut -c1-160) | $(grep -m1 -o 'error: .*' $ROOT/e6-build.log); $(tail -1 $ROOT/e6-build.log)" + local tool=$(grep -o 'short=\[[^]]*' $ROOT/e6-build.log | sed 's/short=\[//') + [ -n "$tool" ] && reading E6-tool-runs "$($tool)" + run e6-nofeature "$W/app" build + reading E6-without-feature "$(grep -m1 -o 'error: .*' $ROOT/e6-nofeature.log); $(tail -1 $ROOT/e6-nofeature.log)" + # the resolved graph: which edges exist + run e6-local "$W/app" build --features windows-installer --cache=local + local exe=$(find "$W/app/target" -type f -perm -u+x -name app | head -1) + reading E6-cache-local "$(grep -m1 -o 'error: .*' $ROOT/e6-local.log); $(tail -1 $ROOT/e6-local.log); app runs: $([ -n "$exe" ] && { "$exe"; echo exit=$?; })" + reading E6-cache-local-link "$(grep -h '^build bin/app' $(find "$W/app/target" -name build.ninja) | sed 's|.*cxx_link||')" + rm -rf "$W"/*/target "$W"/fw/tool/target +} +e6b() { # the tool graph REALLY activates the declaring feature + local W=$ROOT/e6b + e6_fixture "$W" "[target.'cfg(os = \"linux\")'.feature-deps.installer] +spike.fw-installer = { path = \"tool\", tools = [\"fw-installer\"], reexport = true }" '[dependencies] +spike.fw = { path = "..", features = ["installer"] }' + run e6b-build "$W/app" build --features windows-installer + reading E6b-tool-activates-declaring-feature "$(grep -c 'Building host tool' $ROOT/e6b-build.log) host-tool lines | $(grep -m1 -o 'error: .*' $ROOT/e6b-build.log | cut -c1-200); $(tail -1 $ROOT/e6b-build.log)" + grep -A3 'error:' $ROOT/e6b-build.log | head -6 | sed "s|$HOME|~|g; s/^/ /" + rm -rf "$W"/*/target "$W"/fw/tool/target +} +e6c() { # does a tool-only dependency's own library reach the consumer's link? + local W=$ROOT/e6c + e6_fixture "$W" "[feature-deps.installer] +spike.fw-installer = { path = \"tool\", tools = [\"fw-installer\"], reexport = true }" '[dependencies] +spike.z = { path = "../../z" }' + pkg "$W/z" spike z lib '' + echo 'int z_only_in_the_tool() { return 7; }' > "$W/z/src/z.cpp" + printf 'int z_only_in_the_tool();\n#include \nint main() { std::printf("tool %%d\\n", z_only_in_the_tool()); return 0; }\n' > "$W/fw/tool/src/main.cpp" + run e6c-build "$W/app" build --features windows-installer --verbose + local exe=$(find "$W/app/target" -type f -name app -perm -u+x | head -1) + reading E6c-tool-dependency-in-consumer "app link line:$(grep -h '^build bin/app' $(find "$W/app/target" -name build.ninja) | sed 's|.*cxx_link||'); app defines z symbol: $(nm -C "$exe" 2>/dev/null | grep -c z_only_in_the_tool); $(tail -1 $ROOT/e6c-build.log)" + rm -rf "$W"/*/target "$W"/fw/tool/target +} +e6d() { # the same cycle without a feature: an unconditional [build-dependencies] tool edge + local W=$ROOT/e6d + e6_fixture "$W" '[build-dependencies] +spike.fw-installer = { path = "tool", tools = ["fw-installer"], reexport = true }' '[dependencies] +spike.fw = { path = ".." }' + run e6d-build "$W/app" build + reading E6d-build-dependency-tool-edge "$(grep -o 'SPIKE.*' $ROOT/e6d-build.log | sed "s|$HOME|~|g" | cut -c1-120) | $(grep -m1 -o 'error: .*' $ROOT/e6d-build.log); $(tail -1 $ROOT/e6d-build.log)" + rm -rf "$W"/*/target "$W"/fw/tool/target +} + + +e7() { # a git dependency naming a workspace member of the repository + local W=$ROOT/e7; rm -rf "$W"; mkdir -p "$W" + pkg "$W/repo" spike fw lib '[workspace] +members = ["tool"]' + echo 'int fw_answer() { return 42; }' > "$W/repo/src/fw.cpp" + pkg "$W/repo/tool" spike fw-installer bin '[dependencies] +spike.fw = { path = ".." }' + printf 'int fw_answer();\n#include \nint main() { std::printf("fw-installer ran %%d\\n", fw_answer()); return 0; }\n' > "$W/repo/tool/src/main.cpp" + (cd "$W/repo" && git init -q && git add -A && git -c user.email=p@p -c user.name=p commit -qm init) + local rev=$(git -C "$W/repo" rev-parse HEAD) + local url="file://$W/repo" + pkg "$W/app" "" app bin "[dependencies] +spike.fw = { git = \"$url\", rev = \"$rev\" } + +[features] +windows-installer = [] + +[target.'cfg(os = \"linux\")'.feature-deps.windows-installer] +spike.fw-installer = { git = \"$url\", rev = \"$rev\", tools = [\"fw-installer\"] }" + printf 'int fw_answer();\nint main() { return fw_answer() == 42 ? 0 : 1; }\n' > "$W/app/src/main.cpp" + cat > "$W/app/build.mcpp" <<'EOF2' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("fw-installer", "fw-installer"); + mcpp::warning((std::string("SPIKE short=[") + (a ? a : "") + "]").c_str()); + return 0; +} +EOF2 + run e7-both "$W/app" build --features windows-installer + reading E7-both-keys "$(grep -c 'that identity is used' $ROOT/e7-both.log) adoption warning(s) | $(grep -o 'SPIKE.*' $ROOT/e7-both.log) | $(grep -o 'Compiling spike.fw.*' $ROOT/e7-both.log) | $(grep -m1 -o 'error: .*' $ROOT/e7-both.log | cut -c1-160); $(tail -1 $ROOT/e7-both.log)" + # the member alone, with no key naming the root package + rm -rf "$W/app/target" + sed -i '/^spike.fw = { git/d' "$W/app/mcpp.toml" + sed -i 's/^int fw_answer();$//; s/return fw_answer() == 42 ? 0 : 1;/return 0;/' "$W/app/src/main.cpp" + run e7-member-only "$W/app" build --features windows-installer + reading E7-member-only "$(grep -m1 -o 'error: .*' $ROOT/e7-member-only.log | sed "s|$HOME|~|g" | cut -c1-200); $(tail -1 $ROOT/e7-member-only.log)" + # control: the same member by path from a checkout works + rm -rf "$W/app/target" + sed -i "s|spike.fw-installer = { git = \"$url\", rev = \"$rev\",|spike.fw-installer = { path = \"../repo/tool\",|" "$W/app/mcpp.toml" + run e7-member-by-path "$W/app" build --features windows-installer + reading E7-member-by-path "$(grep -o 'SPIKE.*' $ROOT/e7-member-by-path.log | sed "s|$HOME|~|g" | cut -c1-120) | $(grep -m1 -o 'error: .*' $ROOT/e7-member-by-path.log); $(tail -1 $ROOT/e7-member-by-path.log)" + rm -rf "$W"/app/target "$W"/repo/target "$W"/repo/tool/target +} + +edup() { # one consumer naming one package in [dependencies] and [build-dependencies] with different requests + local W=$ROOT/edup; rm -rf "$W"; mkdir -p "$W" + mkdir -p "$W/installer/src" + cat > "$W/installer/mcpp.toml" <<'EOF2' +[package] +name = "installer" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[targets.installer-lib] +kind = "lib" + +[targets.installer] +kind = "bin" +main = "src/main.cpp" + +[build] +sources = ["src/lib.cpp"] +EOF2 + echo 'int installer_answer() { return 42; }' > "$W/installer/src/lib.cpp" + echo 'int main() { return 0; }' > "$W/installer/src/main.cpp" + pkg "$W/app" "" app bin '[dependencies] +spike.installer = { path = "../installer" } + +[build-dependencies] +spike.installer = { path = "../installer", tools = ["installer"] }' + printf 'int installer_answer();\nint main() { return installer_answer() == 42 ? 0 : 1; }\n' > "$W/app/src/main.cpp" + cat > "$W/app/build.mcpp" <<'EOF2' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("installer", "installer"); + mcpp::warning((std::string("SPIKE short=[") + (a ? a : "") + "]").c_str()); + return 0; +} +EOF2 + run edup-two-tables "$W/app" build --strict + reading EDUP-dependencies-and-build-dependencies "tool built: $(grep -c 'Building host tool' $ROOT/edup-two-tables.log) | $(grep -o 'SPIKE.*' $ROOT/edup-two-tables.log | sed "s|$HOME|~|g" | cut -c1-100) | $(grep -m1 -o 'error: .*' $ROOT/edup-two-tables.log); $(tail -1 $ROOT/edup-two-tables.log)" + rm -rf "$W"/*/target +} + +e8() { # --features with a dependency's feature, with and without a root [features] table + local W=$ROOT/e8 + e6_fixture "$W" "[target.'cfg(os = \"linux\")'.feature-deps.installer] +spike.fw-installer = { path = \"tool\", tools = [\"fw-installer\"], reexport = true }" '' + printf '#include \nint main() { std::puts("fw-installer ran"); return 0; }\n' > "$W/fw/tool/src/main.cpp" + sed -i '/^\[features\]$/,$d' "$W/app/mcpp.toml" # no [features] table in the application + run e8-dep-feature-no-table "$W/app" build --strict --features spike.fw/installer + reading E8-dep-feature-no-table "$(grep -o 'SPIKE.*' $ROOT/e8-dep-feature-no-table.log) | tool built: $(grep -c 'Building host tool' $ROOT/e8-dep-feature-no-table.log); $(tail -1 $ROOT/e8-dep-feature-no-table.log)" + reading E8-dep-feature-no-table-macro "$(grep -o -- '-DMCPP_FEATURE_[A-Z_]*' $W/app/compile_commands.json | sort -u | tr '\n' ' ')" + run e8-unknown-no-table "$W/app" build --strict --features nothing-here + reading E8-unknown-no-table "$(grep -m1 -o 'error: .*' $ROOT/e8-unknown-no-table.log); $(tail -1 $ROOT/e8-unknown-no-table.log); macros: $(grep -o -- '-DMCPP_FEATURE_[A-Z_]*' $W/app/compile_commands.json | sort -u | tr '\n' ' ')" + printf '\n[features]\nwindows-installer = []\n' >> "$W/app/mcpp.toml" + run e8-dep-feature-table "$W/app" build --features spike.fw/installer + reading E8-dep-feature-with-table "$(grep -m1 -o 'warning: --features.*' $ROOT/e8-dep-feature-table.log) | tool built: $(grep -c 'Building host tool' $ROOT/e8-dep-feature-table.log); $(tail -1 $ROOT/e8-dep-feature-table.log)" + run e8-dep-feature-table-strict "$W/app" build --strict --features spike.fw/installer + reading E8-dep-feature-with-table-strict "$(grep -m1 -o 'error: .*' $ROOT/e8-dep-feature-table-strict.log); $(tail -1 $ROOT/e8-dep-feature-table-strict.log)" + # control: the manifest forward the CLI form would replace + sed -i 's/^windows-installer = \[\]$/windows-installer = ["spike.fw\/installer"]/' "$W/app/mcpp.toml" + run e8-manifest-forward "$W/app" build --strict --features windows-installer + reading E8-manifest-forward-control "$(grep -o 'SPIKE.*' $ROOT/e8-manifest-forward.log | sed "s|$HOME|~|g" | cut -c1-110) | tool built: $(grep -c 'Building host tool' $ROOT/e8-manifest-forward.log); $(tail -1 $ROOT/e8-manifest-forward.log)" + rm -rf "$W"/*/target "$W"/fw/tool/target +} +case "${1:-all}" in + all) e41; e41b; e42; e6; e6b; e6c; e6d; e7; edup; e8 ;; + *) for f in "$@"; do "$f"; done ;; +esac +__PROBES_646_649_GROUP_C__ + ROOT="$WORK/c" bash "$WORK/c.sh" all "$WORK/d.sh" <<'__PROBES_646_649_GROUP_D__' +# Group d: probes for #648. Each prints READING lines. Linux x86_64, mcpp 2026.9.15.2. +set +e +M=${M:-~/.xlings/data/xpkgs/xim-x-mcpp/2026.9.15.2/bin/mcpp} +W=${W:-$(mktemp -d)} + +# ── A2: the descriptors a child started during planning inherits ────────── +# A build program lists /proc/self/fd; the reader prints the inode of the pipe +# it reads mcpp's stdout from. A child fd naming the same inode is the leak. +mkdir -p "$W/a2/src" && cd "$W/a2" || exit 1 +cat > mcpp.toml <<'T' +[package] +name = "a2probe" +version = "0.1.0" +standard = "c++23" + +[targets.a2probe] +kind = "bin" +main = "src/main.cpp" +T +echo 'int main() { return 0; }' > src/main.cpp +cat > build.mcpp < " + << std::filesystem::read_symlink(e.path(), ec).string() << "\n"; + } + return 0; +} +T +MCPP_OFFLINE=1 "$M" emit build-database --format json 2>/dev/null \ + | { echo "READING A2 reader stdin=$(readlink /proc/self/fd/0)"; cat >/dev/null; } +sed 's/^/READING A2 child fd /' "$W/a2/fds.txt" + +# ── T: what triggered the refresh on openxlings/xlings ──────────────────── +# Offline, the refresh decision is still computed; a dependency reported as +# "offline mode" is one the decision wanted to refresh. The same run resolves it. +XL=${XL:-$HOME/workspace/github/openxlings/xlings} +if [ -f "$XL/mcpp.toml" ]; then + (cd "$XL" && MCPP_OFFLINE=1 timeout 180 "$M" emit build-database --format json -v \ + 2>"$W/xl.err" >"$W/xl.json"; echo "READING T exit=$?") + grep -a "index: .*offline mode" "$W/xl.err" | sed 's/\x1b\[[0-9;]*m//g; s/^.*index: /READING T decision /' + grep -a "resolved to .* through the deprecated bare-name search" "$W/xl.err" | sed 's/^/READING T resolver /' +fi + +# ── A6: the GitCode mirror of the mcpp-index artifact against its pointer ── +ptr=$(curl -sSL https://raw.githubusercontent.com/xlings-res/mcpp-index/main/mcpp-index-pointers.json) +ver=$(printf '%s' "$ptr" | python3 -c 'import json,sys; print(json.load(sys.stdin)["indexes"]["mcpp"]["index_version"])') +sha=$(printf '%s' "$ptr" | python3 -c 'import json,sys; print(json.load(sys.stdin)["indexes"]["mcpp"]["artifact"]["sha256"])') +echo "READING A6 pointer version=$ver sha=$sha" +for host in github gitcode; do + url="https://$host.com/xlings-res/mcpp-index/releases/download/v$ver/mcpp-index-$ver.tar.gz" + curl -sSL -o "$W/$host.tgz" "$url" + echo "READING A6 $host sha=$(sha256sum "$W/$host.tgz" | cut -d' ' -f1) bytes=$(wc -c < "$W/$host.tgz")" + echo "READING A6 $host first-entry=$(tar -tvzf "$W/$host.tgz" | head -1)" +done +__PROBES_646_649_GROUP_D__ + W="$WORK/d" bash "$WORK/d.sh" &2; exit 2 ;; +esac diff --git a/.agents/docs/2026-09-16-646-649-verify.sh b/.agents/docs/2026-09-16-646-649-verify.sh new file mode 100644 index 000000000..57ee9e78f --- /dev/null +++ b/.agents/docs/2026-09-16-646-649-verify.sh @@ -0,0 +1,322 @@ +#!/usr/bin/env bash +# Ecosystem verification for #646 to #649 against the PUBLISHED mcpp, plugins and +# index, run inside a SubOS sandbox with CN mirrors for xlings and mcpp. +# +# B64=$(base64 -w0 .agents/docs/2026-09-16-646-649-verify.sh) +# xlings subos new v646 +# xlings subos use v646 --sandbox --cmd \ +# "echo $B64 | base64 -d > /tmp/v.sh && MCPP_VERIFY_VERSION=2026.9.16.1 bash /tmp/v.sh" +# +# The script is run once against the previous release first +# (MCPP_VERIFY_VERSION=2026.9.15.2): the sections marked CHANGE must fail there +# and pass on the new release; the sections marked GUARD must pass on both. A +# script that passes on both releases in a CHANGE section measured nothing. +# +# The sandbox shares the xlings data directory, so a published mcpp is addressed +# by its store path and exact version. Its $HOME content persists between runs +# of one SubOS, so every section clears its own probe directory first. A section +# that cannot run says so and is listed again in the summary. +set -u + +VER="${MCPP_VERIFY_VERSION:?set MCPP_VERIFY_VERSION}" +STORE="${MCPP_VERIFY_BIN:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/$VER/bin/mcpp}" +PLUGINS="${MCPP_VERIFY_PLUGINS:-0.12.0}" + +fails=0 +skipped="" +fail() { printf 'ASSERT-FAIL: %s\n' "$1"; fails=$((fails + 1)); } +ok() { printf 'ok: %s\n' "$1"; } +section() { printf '\n== %s ==\n' "$1"; } +skip() { printf 'NOT RUN: %s\n' "$1"; skipped="$skipped + - $1"; } +has_text() { grep -qF -- "$2" "$1"; } +reading() { printf 'READING %s\n' "$*"; } +unset XLINGS_ACTIVE_SUBOS + +root="$HOME/verify-646" +rm -rf "$root"; mkdir -p "$root" + +section "A. GUARD: the published mcpp answers for itself, with CN mirrors for xlings and mcpp" +if [ ! -x "$STORE" ]; then + fail "no mcpp at $STORE"; printf '\nfails=%d (nothing else can run)\n' "$fails"; exit 1 +fi +got=$("$STORE" --version 2>&1 | head -1) +case "$got" in *"$VER"*) ok "mcpp --version says $got" ;; *) fail "mcpp --version says '$got', expected $VER" ;; esac +if command -v xlings >/dev/null 2>&1; then + xlings config --mirror CN >/dev/null 2>&1 && ok "xlings config --mirror CN" || fail "xlings config --mirror CN" +fi +"$STORE" self config --mirror CN >/dev/null 2>&1 && ok "mcpp self config --mirror CN" || fail "mcpp self config --mirror CN" +grep -q '"mirror": *"CN"' "$HOME/.mcpp/registry/.xlings.json" 2>/dev/null \ + && ok "mcpp's xlings reads mirror CN" || fail "mcpp's xlings does not read mirror CN" + +section "B. CHANGE (#648 A6): the mcpplibs index artifact has a CN base, and an index update succeeds" +if grep -q 'gitcode.com/xlings-res/mcpp-index' "$HOME/.mcpp/registry/.xlings.json" 2>/dev/null; then + ok "the mcpplibs artifact names the GitCode mirror" +else + fail "the mcpplibs artifact has no CN base"; grep -n artifact "$HOME/.mcpp/registry/.xlings.json" | head -3 +fi +if timeout 600 "$STORE" index update > "$root/index-update.log" 2>&1; then + ok "mcpp index update" +else + fail "mcpp index update failed"; tail -5 "$root/index-update.log" +fi +reading "A6 mcpplibs index version: $(cat "$HOME/.mcpp/registry/data/mcpplibs/.xlings-index-version" 2>/dev/null || echo none)" + +section "C. CHANGE (#646 F3a): an llvm program over a C++ shared library runs" +d=$root/c; rm -rf "$d"; mkdir -p "$d/app/src" "$d/lib/src" +printf '[package]\nname = "lib"\nversion = "0.1.0"\n[targets.lib]\nkind = "shared"\n' > "$d/lib/mcpp.toml" +printf 'export module lib;\nimport std;\nexport [[gnu::visibility("default")]] std::string lib_greet(int n);\n' > "$d/lib/src/lib.cppm" +printf 'module lib;\nimport std;\nstd::string lib_greet(int n) { return std::format("lib-{}", n); }\n' > "$d/lib/src/lib.cpp" +printf '[package]\nname = "app"\nversion = "0.1.0"\n[toolchain]\ndefault = "llvm@22.1.8"\n[dependencies.lib]\npath = "../lib"\n' > "$d/app/mcpp.toml" +printf 'import std;\nimport lib;\nint main() { std::println("{}", lib_greet(3)); }\n' > "$d/app/src/main.cpp" +( cd "$d/app" && timeout 1800 "$STORE" run > run.log 2>&1 ); rc=$? +reading "F3a run exit=$rc: $(grep -m1 -E 'lib-3|bad_cast' "$d/app/run.log")" +if [ "$rc" -eq 0 ] && has_text "$d/app/run.log" "lib-3"; then + ok "the program runs on one C++ runtime" +else + fail "the llvm program over a C++ shared library did not run"; tail -5 "$d/app/run.log" +fi + +section "D. CHANGE (#646 F1): a static package under a shared package is linked into it" +d=$root/d; rm -rf "$d"; mkdir -p "$d/x/src" "$d/fw/src" "$d/app/src" +printf '[package]\nname = "x"\nversion = "0.1.0"\n[targets.x]\nkind = "lib"\n' > "$d/x/mcpp.toml" +printf 'int x_answer(void) { return 42; }\n' > "$d/x/src/x.c" +printf '[package]\nname = "fw"\nversion = "0.1.0"\n[targets.fw]\nkind = "shared"\n[dependencies.x]\npath = "../x"\n' > "$d/fw/mcpp.toml" +printf 'int x_answer(void);\n__attribute__((visibility("default"))) int fw_answer(void) { return x_answer(); }\n' > "$d/fw/src/fw.c" +printf '[package]\nname = "app"\nversion = "0.1.0"\n[dependencies.fw]\npath = "../fw"\n' > "$d/app/mcpp.toml" +printf 'extern "C" int fw_answer(void);\nint main() { return fw_answer() == 42 ? 0 : 1; }\n' > "$d/app/src/main.cpp" +( cd "$d/app" && timeout 1200 "$STORE" build > build.log 2>&1 ) || true +lib=$(ls "$d"/app/target/*/*/bin/libfw.so 2>/dev/null | head -1) +if [ -z "$lib" ]; then + fail "libfw.so was not built"; tail -5 "$d/app/build.log" +elif command -v python3 >/dev/null 2>&1; then + if python3 -c 'import ctypes,sys; ctypes.CDLL(sys.argv[1], mode=ctypes.RTLD_GLOBAL|2)' "$lib" 2> "$d/dlopen.err"; then + ok "libfw.so loads on its own (RTLD_NOW): x is inside it" + else + fail "libfw.so does not load on its own: $(cat "$d/dlopen.err" | tail -1)" + fi +else + skip "D: no python3 in the sandbox to dlopen libfw.so" +fi + +section "E. CHANGE (#649 E6, #647 E4.3): a feature tool that depends on its declaring package" +d=$root/e; rm -rf "$d"; mkdir -p "$d/fw/src" "$d/fw/tool/src" "$d/app/src" +cat > "$d/fw/mcpp.toml" <<'EOF' +[package] +name = "fw" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[targets.fw] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[features] +installer = [] + +[target.'cfg(os = "linux")'.feature-deps.installer] +spike.fw-installer = { path = "tool", tools = ["fw-installer"], reexport = true } +EOF +cat > "$d/fw/tool/mcpp.toml" <<'EOF' +[package] +name = "fw-installer" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[targets.fw-installer] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw = { path = ".." } +EOF +cat > "$d/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.fw = { path = "../fw" } + +[features] +windows-installer = ["spike.fw/installer"] + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +cat > "$d/app/build.mcpp" <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("fw-installer", "fw-installer"); + const char* b = mcpp::dep_bin("spike.fw-installer", "fw-installer"); + std::println("SPIKE short=[{}] qualified=[{}]", a ? a : "", b ? b : ""); + return 0; +} +EOF +printf 'int fw_answer() { return 42; }\n' > "$d/fw/src/fw.cpp" +printf 'int fw_answer();\n#include \nint main() { std::printf("fw-installer ran %%d\\n", fw_answer()); return 0; }\n' > "$d/fw/tool/src/main.cpp" +printf 'int fw_answer();\nint main() { return fw_answer() == 42 ? 0 : 1; }\n' > "$d/app/src/main.cpp" +( cd "$d/app" && timeout 1200 "$STORE" build --features windows-installer > build.log 2>&1 ); rc=$? +reading "E6 build exit=$rc: $(grep -m1 -E 'SPIKE|cycle' "$d/app/build.log")" +if [ "$rc" -eq 0 ] && grep -q 'qualified=\[/' "$d/app/build.log" && grep -q 'short=\[/' "$d/app/build.log"; then + ok "the tool builds, and dep_bin answers under both spellings" +else + fail "the feature tool over its declaring package did not build with both dep_bin spellings"; tail -5 "$d/app/build.log" +fi + +section "F. CHANGE (#649 E8): --features / opens a dependency's feature" +sed -i 's/^windows-installer = \["spike.fw\/installer"\]$/windows-installer = []/' "$d/app/mcpp.toml" +rm -rf "$d/app/target" +( cd "$d/app" && timeout 1200 "$STORE" build --strict --features spike.fw/installer > f.log 2>&1 ); rc=$? +reading "E8 exit=$rc: $(grep -m1 -E 'SPIKE|features' "$d/app/f.log")" +if [ "$rc" -eq 0 ] && grep -q 'short=\[/' "$d/app/f.log"; then + ok "--features spike.fw/installer built the dependency's tool" +else + fail "--features spike.fw/installer did not open the feature"; tail -5 "$d/app/f.log" +fi + +section "G. CHANGE (#647 E1): the root build program reads the graph and a dependency's metadata" +d=$root/g; rm -rf "$d"; mkdir -p "$d/app/src" "$d/a/src" "$d/b/src" +printf '[package]\nname = "b"\nnamespace = "spike"\nversion = "0.2.0"\n\n[package.metadata.demo]\nresources = "res"\n\n[targets.b]\nkind = "lib"\n' > "$d/b/mcpp.toml" +printf 'int b_answer() { return 40; }\n' > "$d/b/src/b.cpp" +printf '[package]\nname = "a"\nnamespace = "spike"\nversion = "0.1.0"\n\n[targets.a]\nkind = "lib"\n\n[dependencies]\nspike.b = { path = "../b" }\n' > "$d/a/mcpp.toml" +printf 'int b_answer();\nint a_answer() { return b_answer() + 2; }\n' > "$d/a/src/a.cpp" +printf '[package]\nname = "app"\nversion = "0.1.0"\n\n[dependencies]\nspike.a = { path = "../a" }\n' > "$d/app/mcpp.toml" +printf 'int a_answer();\nint main() { return a_answer() == 42 ? 0 : 1; }\n' > "$d/app/src/main.cpp" +cat > "$d/app/build.mcpp" <<'EOF' +import std; +import mcpp; +int main() { + std::ifstream in(mcpp::graph_file()); + std::stringstream text; + text << in.rdbuf(); + const auto doc = text.str(); + const auto posA = doc.find("\"spike.a@"); + const auto posB = doc.find("\"spike.b@"); + std::println("GRAPH b-before-a={} metadata={}", posB < posA, + doc.find("\"resources\"") != std::string::npos); + return 0; +} +EOF +( cd "$d/app" && timeout 1200 "$STORE" build > build.log 2>&1 ); rc=$? +reading "E1 exit=$rc: $(grep -m1 -E 'GRAPH|graph_file' "$d/app/build.log")" +if grep -q 'GRAPH b-before-a=true metadata=true' "$d/app/build.log"; then + ok "the graph lists b before a, with b's [package.metadata]" +else + fail "the root build program did not read the graph"; tail -5 "$d/app/build.log" +fi + +section "H. CHANGE (#649 E5, E9): pack strips a graph-built library, and reports what it produced" +d=$root/h; rm -rf "$d"; mkdir -p "$d/dep/src" "$d/app/src" +printf '[package]\nname = "dep"\nnamespace = "spike"\nversion = "0.1.0"\nstandard = "c++20"\n\n[targets.dep]\nkind = "shared"\n\n[build]\nsources = ["src/*.cpp"]\n' > "$d/dep/mcpp.toml" +printf '#include \n[[gnu::visibility("default")]] int dep_answer() { return static_cast(std::to_string(42).size()) + 40; }\n' > "$d/dep/src/dep.cpp" +printf '[package]\nname = "hostapp"\nversion = "0.1.0"\nstandard = "c++20"\n\n[dependencies]\nspike.dep = { path = "../dep" }\n\n[targets.hostapp]\nkind = "bin"\nmain = "src/main.cpp"\n' > "$d/app/mcpp.toml" +printf 'int dep_answer();\nint main() { return dep_answer() == 42 ? 0 : 1; }\n' > "$d/app/src/main.cpp" +( cd "$d/app" && timeout 1800 "$STORE" pack --release --format tar --message-format json > pack.json 2> pack.err ); rc=$? +reading "E9 exit=$rc stdout-bytes=$(wc -c < "$d/app/pack.json")" +art=$(python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print(d["data"]["artifacts"][0]["path"])' "$d/app/pack.json" 2>/dev/null) +if [ -n "$art" ] && [ -f "$art" ]; then + ok "pack --release --message-format json names an artifact that exists" + mkdir -p "$d/x" && tar -xzf "$art" -C "$d/x" + so=$(find "$d/x" -name libdep.so | head -1) + if [ -n "$so" ] && command -v readelf >/dev/null 2>&1; then + symtab=$(readelf -S -W "$so" | grep -c ' .symtab') + reading "E5 libdep.so symtab=$symtab" + [ "$symtab" -eq 0 ] && ok "the graph-built libdep.so is stripped" || fail "the graph-built libdep.so keeps .symtab" + else + skip "H: no libdep.so in the archive or no readelf" + fi +else + fail "pack produced no machine-readable report"; tail -3 "$d/app/pack.err" +fi + +section "I. CHANGE (#648 A1, A2): offline code, and planning children do not hold the caller's pipe" +d=$root/i; rm -rf "$d"; mkdir -p "$d/index/pkgs/w" "$d/app/src" +cat > "$d/index/pkgs/w/widget.lua" <<'EOF' +package = { + spec = "1", namespace = "acme", name = "widget", description = "not installed", + licenses = {"MIT"}, type = "package", + xpm = { linux = { ["1.0.0"] = { url = "https://example.invalid/w.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } } }, + mcpp = { language = "c++23", sources = { "src/*.cpp" }, targets = { ["widget"] = { kind = "lib" } } }, +} +EOF +printf '[package]\nname = "app"\nversion = "0.1.0"\n\n[indices]\nacme = { path = "%s" }\n\n[dependencies.acme]\nwidget = "1.0.0"\n' "$d/index" > "$d/app/mcpp.toml" +printf 'int main() { return 0; }\n' > "$d/app/src/main.cpp" +( cd "$d/app" && MCPP_OFFLINE=1 timeout 300 "$STORE" emit build-database --format json > a1.json 2> a1.err ) || true +code=$(python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print(d["diagnostics"][0]["code"])' "$d/app/a1.json" 2>/dev/null) +reading "A1 code=$code" +[ "$code" = "MCPP_OFFLINE_DOWNLOAD_REQUIRED" ] && ok "an offline plan that needs a download has its own code" \ + || fail "the offline code is '$code'" +d2=$root/i2; rm -rf "$d2"; mkdir -p "$d2/src" +printf '[package]\nname = "fds"\nversion = "0.1.0"\n\n[targets.fds]\nkind = "bin"\nmain = "src/main.cpp"\n' > "$d2/mcpp.toml" +printf 'int main() { return 0; }\n' > "$d2/src/main.cpp" +cat > "$d2/build.mcpp" </dev/null \ + | { readlink /proc/self/fd/0 > "$d2/reader.txt"; cat > /dev/null; } ) +pipe=$(cat "$d2/reader.txt" 2>/dev/null) +if [ ! -s "$d2/fds.txt" ]; then + fail "A2: the build program did not run" +elif grep -qxF "$pipe" "$d2/fds.txt"; then + fail "A2: a build program holds the caller's pipe $pipe" +else + ok "A2: no planning child holds the caller's pipe" +fi + +section "J. CHANGE (#648 T): a bare compat.* dependency is not a refresh miss" +idx="$HOME/.mcpp/registry/data/mcpplibs/pkgs/c" +cand=""; for c in cjson argparse gtest; do [ -f "$idx/compat.$c.lua" ] && { cand=$c; break; }; done +if [ -z "$cand" ]; then + skip "J: no compat descriptor in the local index" +else + v=$(grep -o '\["[0-9][0-9.]*"\]' "$idx/compat.$cand.lua" | head -1 | tr -d '[]"') + d=$root/j; rm -rf "$d"; mkdir -p "$d/src" + printf '[package]\nname = "j"\nversion = "0.1.0"\n\n[dependencies]\n%s = "%s"\n' "$cand" "$v" > "$d/mcpp.toml" + printf 'int main() { return 0; }\n' > "$d/src/main.cpp" + ( cd "$d" && MCPP_OFFLINE=1 timeout 300 "$STORE" emit build-database --format json -v > /dev/null 2> j.err ) || true + if grep -q "index: $cand@$v: offline mode" "$d/j.err"; then + fail "T: the refresh decision still wants a network refresh for $cand@$v" + elif grep -q "deprecated bare-name search" "$d/j.err"; then + ok "T: $cand@$v resolves through the rung and asks for no refresh" + else + skip "T: the fixture did not reach the bare-name rung" + fi +fi + +section "K. GUARD then CHANGE: mcpp:plugins $PLUGINS resolves from the index and builds a consumer" +d=$root/k; rm -rf "$d"; mkdir -p "$d/src" +cat > "$d/mcpp.toml" < "$d/src/main.cpp" +( cd "$d" && timeout 1800 "$STORE" build > build.log 2>&1 ); rc=$? +reading "plugins exit=$rc: $(grep -m1 -E 'plugins|error' "$d/build.log")" +[ "$rc" -eq 0 ] && ok "mcpp:plugins $PLUGINS resolves and a consumer builds" \ + || { fail "mcpp:plugins $PLUGINS did not build a consumer"; tail -5 "$d/build.log"; } + +printf '\n== summary ==\nversion=%s fails=%d\n' "$VER" "$fails" +[ -n "$skipped" ] && printf 'not run:%s\n' "$skipped" +exit $(( fails > 0 ? 1 : 0 )) diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 056f8a57f..c04f94f6a 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -289 records. +291 records. ## By subject @@ -62,6 +62,8 @@ Records that declare one. Everything else is listed by date below. ### triage +- [#646 to #649 implemented: the plan, its review from eight angles, and the ledger](2026-09-16-646-649-implementation-plan.md) — active +- [Four issues from a framework and an editor: #646, #647, #648 and #649, read against the engine and routed by home](2026-09-16-646-649-four-issues-by-home.md) — active - [Link forms, standard levels and a path limit: the asks of #641 and #642, read against the code](2026-09-15-641-642-link-forms-standards-and-paths.md) — landed - [#641 and #642 implemented: the plan, its review from eight angles, and the ledger](2026-09-15-641-642-implementation-plan.md) — landed - [#634 implemented across five repositories: the plan, its review, and the ledger that tracks it](2026-09-14-634-implementation-plan.md) — landed @@ -74,6 +76,8 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 +- [#646 to #649 implemented: the plan, its review from eight angles, and the ledger](2026-09-16-646-649-implementation-plan.md) — active +- [Four issues from a framework and an editor: #646, #647, #648 and #649, read against the engine and routed by home](2026-09-16-646-649-four-issues-by-home.md) — active - [Link forms, standard levels and a path limit: the asks of #641 and #642, read against the code](2026-09-15-641-642-link-forms-standards-and-paths.md) — landed - [#641 and #642 implemented: the plan, its review from eight angles, and the ledger](2026-09-15-641-642-implementation-plan.md) — landed - [The build database of #636, and two defects on the way to the latest xlings](2026-09-14-636-build-database-and-the-latest-xlings.md) — active diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 62dfb5fe9..870b3be1f 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -422,7 +422,7 @@ jobs: # skips. They run here, where llvm is installed, and each is held to the # line it prints only when it ran to the end, and to the line of the step # that runs a program (#641). - - name: "graph C++ runtime e2e (663, 690, 696)" + - name: "C++ runtime e2e that needs llvm (663, 690, 696, 700)" run: | set -o pipefail export PATH="$HOME/.xlings/subos/current/bin:$PATH" @@ -443,3 +443,9 @@ jobs: "PASS: 690 a shared library over a graph C++ runtime is refused or carries a stated private copy" run_and_assert tests/e2e/696_a_cxx_layer_provider_keeps_its_own_standard.sh \ "PASS: a C++-layer provider compiles its implementation units at its own standard" + # #646 F3a: the default llvm shape of a program over a C++ shared library + # aborted with std::bad_cast before its programs took the library's contract. + run_and_assert tests/e2e/700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh \ + "ok: a program over a C++ shared library runs on one C++ runtime" \ + "ok: a stated self-contained program over a coupled C++ shared library is refused" \ + "PASS: 700 a program over a C++ shared library has one C++ runtime" diff --git a/.github/workflows/ci-macos-e2e.yml b/.github/workflows/ci-macos-e2e.yml index 7a47e8508..80a0e0a4d 100644 --- a/.github/workflows/ci-macos-e2e.yml +++ b/.github/workflows/ci-macos-e2e.yml @@ -24,7 +24,7 @@ jobs: e2e: name: e2e suite (macOS ARM64, self-host) runs-on: macos-15 - timeout-minutes: 30 + timeout-minutes: 60 # NOTE: no MCPP_VERBOSE — the e2e suite asserts mcpp's default quiet # output (tests 48/53). steps: @@ -64,4 +64,35 @@ jobs: "$MCPP" self config # macOS default toolchain is LLVM "$MCPP" toolchain default "llvm@${MCPP_LLVM_VER}" - bash tests/e2e/run_all.sh + echo "MCPP=$MCPP" >> "$GITHUB_ENV" + set -o pipefail + bash tests/e2e/run_all.sh 2>&1 | tee "$RUNNER_TEMP/e2e-suite.log" + + # Measurement legs print READING lines and pass whatever they read; the + # readings are what a decision is taken from (#646 F2: whether a C++ + # exception thrown in a dylib is caught by its class under the payload's + # default runtime), so they are collected where a reader finds them. + - name: Measurement readings + if: always() + shell: bash + run: | + { + echo "### Measurement readings (macOS)" + echo '```' + grep -h '^READING' "$RUNNER_TEMP/e2e-suite.log" 2>/dev/null || echo "(none)" + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + + # #647 E3: an Android row links on a macOS host. Its own step, because the + # NDK is a large download the suite's per-test bound does not allow for, + # and the shard does not carry it (the script declares android-ndk). + - name: "Android row on a macOS host (721)" + timeout-minutes: 30 + shell: bash + run: | + set -o pipefail + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL + "$MCPP" toolchain install android-ndk@30.0.16248370 + bash tests/e2e/721_*.sh 2>&1 | tee "$RUNNER_TEMP/721.log" + grep -q '^PASS: 721' "$RUNNER_TEMP/721.log" diff --git a/.github/workflows/ci-windows-e2e.yml b/.github/workflows/ci-windows-e2e.yml index cdd48e338..da100042e 100644 --- a/.github/workflows/ci-windows-e2e.yml +++ b/.github/workflows/ci-windows-e2e.yml @@ -107,4 +107,18 @@ jobs: export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL "$MCPP_SELF" self config --mirror GLOBAL "$MCPP_SELF" toolchain default llvm@20.1.7 - bash tests/e2e/run_all.sh + set -o pipefail + bash tests/e2e/run_all.sh 2>&1 | tee "$RUNNER_TEMP/e2e-suite.log" + + # Measurement legs print READING lines (#646 F2 across PE images, #649 E10 + # the llvm row's recorded CRT); collected where a reader finds them. + - name: Measurement readings + if: always() + shell: bash + run: | + { + echo "### Measurement readings (Windows, shard ${{ matrix.shard }})" + echo '```' + grep -h '^READING' "$RUNNER_TEMP/e2e-suite.log" 2>/dev/null || echo "(none)" + echo '```' + } >> "$GITHUB_STEP_SUMMARY" diff --git a/CHANGELOG.md b/CHANGELOG.md index cae242da1..15d731d24 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,99 @@ ## [Unreleased] +### 一个进程一个 C++ 运行时、图与打包的事实、离线与有期限的规划:#646 至 #649(2026.9.16.1) + +一个 UI 框架与一个编辑器报告的 23 项,外加实测发现的 12 项。按归属分诊后,引擎承担其中的通用 +缺陷与通用能力;插件、索引与 xlings 清单各自的一项在各自仓库。设计、测量与计划: +`.agents/docs/2026-09-16-646-649-*.md`。 + +**图像与运行时(#646、#649 E10)** + +- **程序与共享库共用一个 C++ 运行时(F3a)。** ELF 上程序默认 `self-contained`、共享库默认 + `toolchain-coupled`,一个加载本次构建所出 C++ 共享库的程序于是在一个进程里带两份运行时: + llvm 默认构建运行即以 `std::bad_cast` 中止(退出 134),gcc 带 900 个被抢占的 libstdc++ + 符号。现在这样的程序与测试取共享库的契约;显式写明自含运行时的程序被编译前拒绝,reason 为 + `program-cxx-runtime-split`。(单测 `Distribution.*`,e2e 700) +- **符号重复检查不再报告按构造共享的定义(F3b)。** `STB_GNU_UNIQUE` 按弱定义处理;两个映像 + 取自同一个构建对象(如 `std.o`)的定义,以及工具链自身运行时提供的 std 模块初始化器,不算 + 冲突;真实的重复定义仍然报告。(单测 `SymbolProvision.*`,e2e 701) +- **静态包放进到达它的那个共享库(F1)。** 只被一个依赖共享库到达的静态包链进该库,不再链进 + 程序(此前库里留着未定义符号,只在 ELF 上靠程序的副本侥幸运行);被多个映像到达时,在 + Mach-O、PE 与 Android `app` 行上编译前拒绝(`static-package-in-two-images`),其他 ELF 行 + 照旧构建并给出 `build/static-placement` 警告,`--strict` 下失败,出路是 + `linkage = "shared"`。(单测 `StaticPlacement.*`,e2e 702、307) +- **Mach-O 上跨映像的 C++ 身份(F2,macos-15 实测)。** 默认每个映像内嵌一份隐藏的 + `libc++.a`:dylib 里抛出的 `std::runtime_error` 在程序中不按该类捕获,两个 + `std::error_code` 的 category 比较不相等;全角色 `cxx_runtime = "host-coupled"` 时 + 两者成立。默认值本次不改(它是今天每个 macOS 构建的形态,改动另行记录),但这样的构建 + 会被告知一次:`build/cxx-runtime-identity`。(e2e 704 的读数) +- **llvm 行在 MSVC ABI 上如实记录静态 CRT(E10 第一步)。** 解析记录此前写 `host-coupled` + 而产物静态链接 `libcmt`;显式要求本行不能交付的动态运行时会得到说明。默认值是否改为 `/MD` + 另行测量与记录。(e2e 703;Mach-O 与 PE 上跨映像的异常身份由 e2e 704、705 测量) + +**图、构建程序与打包(#647 E1–E3、#649 E5、E9)** + +- **根构建程序读取解析后的依赖图(E1)。** `mcpp::graph_file()` / `MCPP_GRAPH_FILE` 指向按 + 依赖在前排序的 JSON 文档,每个包含 `manifest_dir`、`features`、`targets`、链接形态与原样 + 保留的 `[package.metadata.]`;文档摘要进入构建程序的重跑键,编辑依赖的 metadata 会 + 重跑根构建程序,编辑其源码不会。`[package]` 的未知键像 `[build]` 一样报告。(e2e 720) +- **macOS 主机上 Android 行可以链接(E3)。** 链接行按主机与目标对象格式选择 + (`link_shape`),Apple SDK 分支只用于 Mach-O 目标;Linux 上的链接行逐字节不变。 + (单测 `LinkShape.*`,e2e 721 在 macOS CI 上运行) +- **打包剥离构建出来的一切(E5)。** Android 行的程序此前根本没有经过剥离步骤;现在每条剥离的 + 行上剥离程序、本图构建的共享库与暂存的工具链运行时副本(`--strip-unneeded`,保留导出), + 状态行只在真正剥离时写 "stripped";`--no-strip` 与 `--debug-symbols` 作用于每个文件,构建 + 程序通过 `MCPP_PACK_STRIP` / `MCPP_PACK_DEBUG_SYMBOLS_DIR` 读到同一决定。(e2e 722) +- **`mcpp pack --message-format json`(E9)。** 输出一个 `mcpp.pack` 信封,逐项列出产物的 + 绝对路径、类型、格式与目标行;人类可读的行改走 stderr。`pack` 接受 `--release` / `--dev`; + `build`、`run`、`test`、`pack` 共用一个 profile 判定,`--profile` 优先于简写(此前 `run` + 相反)。(单测 `BuildProfile.*`,e2e 723) + +**feature、工具与 git 依赖(#647 E4、#649 E6–E8)** + +- **转发校验覆盖每一张依赖表与每一行(E4.1)。** 经 `[build-dependencies]` 的转发、只在另一行 + 声明的依赖不再被报告为未声明;无处声明的键仍然报告。(e2e 710) +- **`[feature-deps]` 的重述(E4.2)。** 文档改为写明重述来源;来源与生效声明不同的重述被拒绝, + 消息给出两个来源。(e2e 711) +- **`dep_bin` 发布限定名(E4.3)。** 一个函数给出提供者对消费者的所有名字,`dep_dir`、 + `dep_linkage` 与 `dep_bin` 共用;`namespace = "ns"` 加 `name = "x"` 的包也有 + `MCPP_DEP_NS_X_BIN_*`。(e2e 187、711) +- **只提供程序的包不进入消费者的图(E6)。** 这样的包不被扫描、不链接进消费者,其程序只由工具 + 子构建产出;feature 工具于是可以依赖声明它的包。包之间的环在解析时拒绝(`package-cycle`), + 各种缓存模式一致;工具请求自身时在第一次重复即拒绝。(e2e 712) +- **git 依赖可以选择仓库里的成员包(E7)。** 键的身份不是根包时,在根的 `[workspace] members` + 中按身份查找;同一消费者对同一依赖的第二次声明合并其 `tools`、`features`、`host-module` 与 + `reexport`;git 依赖的编译行写提交而不是空版本。(e2e 713) +- **`--features dep/feature`(E8)。** 作为根的转发应用;不指向任何依赖时警告,`--strict` 下 + 失败,且不再变成宏;普通名字保持文档所述的纯宏用法。`mcpp why deps` 接受 `--features`。 + (e2e 714) + +**编辑器在后台规划(#648)** + +- **刷新判定与解析器走同一条阶梯(新发现)。** 省略命名空间的 `ftxui = "6.1.9"` 由解析器经 + 已弃用的裸名回退找到 `compat.ftxui`,刷新判定却只查精确坐标并判为缺失;防抖只有 120 秒, + 这样的工程每次联网规划都执行一次 `xlings update`。现在判定也走该回退。(单测 + `PmIndexRefresh.BareNameResolvedThroughTheLegacyRungIsNotAMiss`,e2e 730) +- **规划期间的子进程不继承调用方的管道(A2)。** 保存的标准输出是 close-on-exec(Windows 上不可 + 继承);此前构建程序以描述符 3 持有调用方读取的管道。(e2e 731) +- **xlings 子进程有期限并随 mcpp 结束(A3)。** 刷新受 `[index] refresh_timeout`(秒,默认 + 120)约束,超时视为刷新失败并继续用本地索引;经接口的安装在 300 秒无任何输出(含心跳)时终止; + 子进程在自己的进程组(Windows 上为作业对象)中运行,结束 mcpp 即一并结束。(e2e 732) +- **信封的 `effects` 按观测报告 `network`(A4)。** 本次运行启动过刷新、安装或 git 远程操作时 + 列出,离线运行从不列出。(e2e 733) +- **`auto_refresh = false` 约束所有隐式刷新(A5)。** 安装前的刷新、重试前的刷新与自定义索引的 + 首次同步改走同一个刷新策略(`mcpp.pm.refresh_policy`)。(e2e 734) +- **离线缺下载有自己的诊断码(A1)。** `MCPP_OFFLINE_DOWNLOAD_REQUIRED` 与 refusal + `offline-download-required`,消息指出第一个需要下载的工具链、包、git 修订或索引。 + (e2e 733、735) +- **默认索引制品按镜像分区(A6)。** mcpplibs 索引的 `artifact` 默认是 + `{ GLOBAL = github, CN = gitcode }`,已有 home 的 `.xlings.json` 就地升级;`mirror = CN` 时 + `mcpp index update` 只访问 GitCode。前提是 mcpplibs/mcpp-index#432 让两端制品逐字节一致。 + (e2e 151) + +**CI**:hermetic llvm job 运行 700 并断言结束行;macOS 与 Windows e2e 把测量行写入 job +summary;macOS 上单独一步运行 721。 + ### 链接形态、标准档位与路径长度:#641 与 #642(2026.9.15.2) 一个 UI 框架迁到 macOS 12 下限与 Android 独立共享库时报告的七项,全部在引擎内处理。 diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 162e403e5..ef2f95b39 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -89,6 +89,20 @@ When a **dependency declares a level above the graph's**, mcpp says so before co rather than letting it fail somewhere inside that dependency's sources. See [workspace §4.2](07-workspace.md). +`[package.metadata.]` (mcpp 2026.9.16.1+) is a table the engine keeps and +does not interpret. It is the package's statement about itself for a tool that +reads it, such as a framework collecting what each library contributes, and it +reaches the root package's build program through `mcpp::graph_file()` +([30 — build.mcpp](30-build-mcpp.md)). A path in it is resolved by that reader +against the package's manifest directory. Any other key in `[package]` that +mcpp does not read is reported, as in `[build]`: a warning, and an error under +`--strict` (2026.9.16.1+). + +```toml +[package.metadata.demo] +resources = "res" +``` + #### Dialect flags and the `import std` BMI Some flags change what the standard library's headers declare, so the precompiled `import std` @@ -501,6 +515,46 @@ A per-dependency `linkage` is honoured **only in the root project's** final program is laid out; one that genuinely must be a single shared copy says so on its own target instead. +#### A static package under a shared library *(mcpp 2026.9.16.1+)* + +A shared library is linked with the static packages it reaches. Each shared +image of a build has a **static closure**: the static packages reachable from +its package without crossing another shared package. A static package in +exactly one closure, which the root project does not reach itself, is linked +into that image and not into the program. + +Before 2026.9.16.1 such a package went into the program, and the library bound +to the program's copy at run time. That worked on ELF and only for that program: +the library refused `-Wl,-z,defs`, a host that did not link the package could not +load it (`undefined symbol`), Mach-O and PE resolve every reference at link time, +and Android loads an application's shared library before anything that could +supply the package. + +A static package that **several** images reach (two shared libraries, or a shared +library and the program) has no single image to live in: + +- on Mach-O, on PE and on the Android application row the build is refused + before compiling, with reason `static-package-in-two-images` + ([50](50-machine-output.md)); +- on other ELF rows the package stays in the program as before, and the build + reports it (`build/static-placement`), which `--strict` turns into an error. + +The message names the package, the images that reach it, and the remedy: give +the package the shared form, so that every image loads one copy. + +```toml +[dependencies] +x = { path = "../x", linkage = "shared" } # on the root's edge + +# or as the package's own default, in its manifest +[targets.x] +linkage = "shared" +``` + +A package that provides a target layer (`provides = ["mcpp:..."]`, a C library or +a C++ runtime) is outside this rule: where its objects go is the runtime +contract's decision ([20](20-toolchains.md)). + #### `soname` on a library target A `soname` (§2.2) may be declared on `kind = "lib"` as well as @@ -525,7 +579,14 @@ diagnostic exists for this. The check is a measurement, not a declaration: it reads the produced image's dynamic symbol table, removes the entries that are copy relocations, and reports only those a library in the artifact's own closure **also** defines. -An arrangement with one copy in the process is silent. The verdict is recorded +An arrangement with one copy in the process is silent. Three kinds of shared +definition are counted and not reported *(2026.9.16.1+ for the last two)*: +vague linkage, which the loader unifies by design (`STB_WEAK`, and +`STB_GNU_UNIQUE`, which GCC uses for the static data of inline entities); a +definition the build links into both images from **one object**, such as the +`std` module's initialiser in every C++ image that imports `std`; and that same +initialiser against the toolchain's own C++ runtime, which exports it from +GCC 16 on. A name of the same shape defined anywhere else is still a finding. The verdict is recorded in `target///resolution.json` under `runtime.symbol_provision`, with the count and its denominator, so CI can read it without `readelf`. diff --git a/docs/05-dependencies.md b/docs/05-dependencies.md index 3ca040567..4283257bf 100644 --- a/docs/05-dependencies.md +++ b/docs/05-dependencies.md @@ -167,6 +167,34 @@ source: the build is refused before scanning, naming both, and the fix is a `namespace` in that manifest or one key in both places. A `version` dependency is unaffected; its identity is the key. +### A package of a git repository (mcpp 2026.9.16.1+) + +A `git` dependency names a repository, and its key names which package of the +repository is meant. The root manifest's package is one; each entry of that +manifest's `[workspace] members` is another. A key whose identity is not the +root package's selects the member whose manifest declares it, at the same +commit: + +```toml +# repo/mcpp.toml declares spike.fw and [workspace] members = ["tool"]; +# repo/tool/mcpp.toml declares spike.fw-installer +[dependencies] +spike.fw = { git = "https://example.org/fw.git", rev = "cc3c74c5" } +spike.fw-installer = { git = "https://example.org/fw.git", rev = "cc3c74c5", tools = ["fw-installer"] } +``` + +- The member inherits the repository's `[workspace.package]`, as it does when + the repository is built from its own checkout. +- A member's `path` edge that stays inside the clone, such as + `spike.fw = { path = ".." }`, names the same git source at the same commit, + so it is the package the root's key resolved rather than a second, + path-sourced declaration of it. +- A key that names neither the root package nor a member keeps the rule of the + section above: the root manifest's identity is used, with the warning. + +A key selects by identity, and no `subdir` key exists: an older client would +ignore such a key and build the root package without a word. + ### Namespace resolution rules Every package has a two-part identity: a **namespace** and a **name**. Every @@ -289,11 +317,18 @@ Controls, in order of precedence: |---|---| | `--offline` (any command) | Never touch the network — no index refresh, no downloads, no toolchain auto-install, no `git ls-remote`/`clone`. Anything already installed still builds, including git deps whose commit is in `mcpp.lock` and whose clone is cached | | `MCPP_OFFLINE=1` | Same, for a whole shell session or CI job | -| `[index] auto_refresh = false` in `~/.mcpp/config.toml` | Never refresh the index automatically; downloads still work | +| `[index] auto_refresh = false` in `~/.mcpp/config.toml` | Never refresh an index implicitly: not on a dependency miss, not before installing a package the local index lacks, and not for the first sync of a project's custom index (that build stops and names `mcpp index update`). Downloads still work | `MCPP_NO_AUTO_INSTALL=1` remains accepted as the older, narrower spelling of `--offline` (it gates only toolchain auto-install). +A refresh is bounded. `[index] refresh_timeout` (seconds, default 120) is the +longest one refresh may take; a refresh that exceeds it is stopped, a warning +names the setting, and the build continues with the local index, as it does +after any failed refresh. An install through xlings is stopped when xlings +writes nothing, not even its heartbeat, for 300 seconds. Terminating mcpp +terminates the xlings process it started. + Run any command with `-v` to see the decision for each dependency and why. ## `[dev-dependencies]` — Test Dependencies @@ -335,15 +370,49 @@ protobuf = { version = "35.1", tools = ["protoc"] } express — a package whose library must not reach the target while its tool or its rule is still wanted. Naming one package in both tables is not an error: the ordinary declaration wins, because a `[build-dependencies]` line must not -quietly drop a library the target needs. +quietly drop a library the target needs, and what each declaration requests +(`tools`, `features`, `host-module`, `reexport`) is requested of the one edge +(mcpp 2026.9.16.1+; before that release the second declaration's requests were +dropped). + +**A package of programs contributes only its programs (mcpp 2026.9.16.1+).** A +dependency whose declared `[targets]` are all programs (`bin`, `app`, `test`) +has nothing to link. Its tools are built by the tool sub-build, which resolves +the package as its own root, and in the consumer's graph it provides its tools +and its directory and nothing else: its own dependencies are not resolved +there, its sources are not compiled there, and its `ldflags` do not reach the +consumer's link. A program may therefore depend on the package that requests +it, which is how an SDK provides a program built against itself. A package +that declares no `[targets]` table is unaffected, even when a `src/main.cpp` +infers a program for it. + +A cycle among packages is refused where the graph is resolved, naming its +edges, under every cache mode; a tool whose own sub-build requests it again is +refused at that first repetition, naming the chain. Unlike `[dev-dependencies]`, these **are** walked transitively: a build dependency's own dependencies are what make it work, and they inherit its build-only nature. -A feature scopes build-time requests without a second declaration site — -`[feature-deps.]` may add `tools` to a dependency already declared -unconditionally, so "only when needed" needs no separate table. +A feature scopes build-time requests without a second declaration site. A +`[feature-deps.]` entry may restate a dependency already declared +unconditionally, with the same source, and add `tools` to it, so "only when +needed" needs no separate table: + +```toml +[dependencies] +spike.installer = { path = "../installer" } + +[feature-deps.installer] +spike.installer = { path = "../installer", tools = ["installer"] } +``` + +The restatement names its source because every dependency table does: an entry +without `path`, `git`, `version` or `workspace` is read as a namespace table +and refused, and the refusal says to restate the source. `tools`, `features`, +`host-module` and `reexport` of the restatement are added to the declaration in +effect on the row. A restatement that names another source is refused, naming +both sources (mcpp 2026.9.16.1+); before that release it was ignored. > The section has been parsed since early versions and, until 2026.8.29.1, read > by nothing that made a decision: writing it produced a manifest that loaded, diff --git a/docs/06-features-and-capabilities.md b/docs/06-features-and-capabilities.md index ba15baa5a..6871b01c1 100644 --- a/docs/06-features-and-capabilities.md +++ b/docs/06-features-and-capabilities.md @@ -41,6 +41,33 @@ extra = [] requesting an undeclared feature produces a warning; an error under `--strict`. A package that does not declare `[features]` accepts any request (pure macro usage). +### A feature of a dependency + +A token `/` opens a feature of a dependency. The dependency +is named by the key its consumer's manifest writes (`spike.fw`, `compat.opencv`), +and the token is additive: it opens more of the dependency, never pulls the +dependency in. + +```toml +[dependencies] +spike.fw = { path = "../fw" } + +[features] +windows-installer = ["spike.fw/installer"] # the same as `forward = [...]` in the table form +``` + +- **On the command line** (mcpp 2026.9.16.1+), `mcpp build --features spike.fw/installer` + opens the same feature for one command, as a forward of the root does; + `run`, `test`, `pack`, `emit build-database` and `why deps` accept the token + alike. It is never a feature of the root and never becomes a macro. +- **Validation** reads every dependency table of the manifest that writes the + forward: `[dependencies]`, `[build-dependencies]`, `[dev-dependencies]` and + `[feature-deps.]`, on every row. A key declared only for another row or + under an inactive feature is declared; on this row the forward reaches no edge + and does nothing. A key no table declares is reported, and the report is an + error under `--strict`. On the command line the same check runs whether or not + the root declares `[features]`. + ### Table form — a feature that contributes more than implied features A `[features]` entry may be written as a **table** instead of an array, letting the diff --git a/docs/07-workspace.md b/docs/07-workspace.md index f1020e0bc..c8dcf2201 100644 --- a/docs/07-workspace.md +++ b/docs/07-workspace.md @@ -372,6 +372,12 @@ myproject/ Each member's build artifacts live under its own `target/` subdirectory. +A project outside the workspace reaches a member of a git-hosted workspace by +the member's identity: `myproject.http = { git = "...", rev = "..." }` selects +`libs/http` among the root manifest's `members`, at the same commit, and the +member inherits `[workspace.package]` as it does here (mcpp 2026.9.16.1+; see +[05 — Dependencies](05-dependencies.md)). + ## 7. Relationship to C++ Modules Workspaces work in concert with the C++23 module mechanism: diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index bfe8cb88d..3dc570a36 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -144,6 +144,8 @@ mcpp pack --format appimage # a format a package in the graph provid mcpp pack -o myapp.tar.gz # filename only: lands at target/dist/myapp.tar.gz mcpp pack -o /abs/path/myapp.tar.gz # includes a directory: output to the literal path mcpp pack --profile dev # build with a different profile (default: release) +mcpp pack --dev # the same, as `build` and `run` spell it; --profile wins over it +mcpp pack --message-format json # one mcpp.pack envelope on stdout (mcpp 2026.9.16.1+) mcpp pack --no-strip # ship the artifacts as built mcpp pack --debug-symbols dbg/ # write the separated *.debug files under dbg/ mcpp pack --format msi --features installer # activate root-package features for the pack @@ -156,6 +158,19 @@ one distribution is declared under `[feature-deps.]` with `tools = [...]` and only by the pack that names ``. `mcpp run --format --features ` hands the same features to the pack it performs. +`--release` and `--dev` (mcpp 2026.9.16.1+) are the shorthands `build` and `run` +take, with the same precedence: `--profile` wins over either, on all three +commands. + +`--message-format json` (mcpp 2026.9.16.1+) prints one `mcpp.pack` envelope on +stdout after the command finishes and sends every human line to stderr. Its +`data.artifacts` holds each produced file or directory with its absolute path, +its `type` (`file` or `directory`), the `--format` value and the triples of its +legs; `data.stage` holds the staged tree, its manifest and whether the closure +was walked ([50 — Machine Output](50-machine-output.md)). `--format` names the +package format on this command, so machine output is asked for the way +`mcpp test` asks for it. + ### `--format` owns one axis, and the engine owns two of its values `tar` and `dir` answer the same question `msi` and `appimage` answer — what @@ -297,9 +312,23 @@ follow. > anything; this one governs what the **package** carries. Two different > decisions, two different names. -**Bundled libraries are never stripped.** They came out of the store or off the -host, mcpp did not build them, and rewriting somebody else's shared payload for -this bundle's benefit is not the packer's business. +**What the graph built is stripped on every row that carries debug information in +the image; a library from the store or the host is not** (mcpp 2026.9.16.1+). The +rule is dh_strip's, a package strips what it built, and it covers: + +| staged file | stripped | reason | +|---|---|---| +| the program | yes, as an executable; as a shared library on the Android row, where the program is one | this build compiled it | +| a shared library a `SharedLibrary` link unit of the graph produced | yes, `--strip-unneeded` | this build compiled it from source | +| the staged copy of the toolchain's own runtime (`libc++_shared.so` from the NDK) | yes, `--strip-unneeded` | a copy in the tree is not the shared payload; the Android Gradle plugin strips the same file | +| a library from the store, the host, or a prebuilt a package deployed | no | its bytes are its publisher's, and a vendor library may be signed | + +Mach-O and MSVC PE keep their debug information beside the image, so nothing is +stripped there, and the `Packing` line says "stripped" only on a row that strips. +`--no-strip` and `--debug-symbols` reach every stripped file. A build program +that stages libraries of its own reads the same decision through +`mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()` +([30 — build.mcpp](30-build-mcpp.md)) ## Output Layout diff --git a/docs/20-toolchains.md b/docs/20-toolchains.md index 7a71c2317..f66e43eda 100644 --- a/docs/20-toolchains.md +++ b/docs/20-toolchains.md @@ -1015,6 +1015,40 @@ A project-wide `cxx_runtime = "…"` (or `static_stdlib = false`) applies to sha libraries too: a human said what the whole project promises. The format-specific default applies only when nobody said anything. +**A program that loads a C++ shared library** (mcpp 2026.9.16.1+). On ELF the two +defaults above disagree inside one process: a self-contained program over a +toolchain-coupled C++ shared library holds a static C++ runtime and a shared one. +The executable exports the runtime symbols the library references, the library +binds some of them to the program's copy and keeps the rest, and the halves +disagree about shared state; measured with `llvm@22.1.8`, such a program aborted +with `std::bad_cast` the first time the library formatted a string, and with +`gcc@16.1.0` it ran with 900 libstdc++ symbols interposed. So on ELF a program or +test whose contract nobody stated takes the shared library's contract when it +loads a C++ shared library the build makes, directly or through another one. The +process already needs that runtime through the library's own `NEEDED` entry, so +no deployment gains a requirement; `resolution.json` records the resulting +contract under `runtime.cxx_runtime_by_role`. + +A stated contract is never changed. A program or test that **states** +`self-contained` while loading a C++ shared library coupled to a shared runtime +is refused before compiling (reason `program-cxx-runtime-split`, +[50](50-machine-output.md)); the ways out are to remove the statement, or to give +the shared library a private copy with `cxx_runtime = { shared = "self-contained" }`, +which keeps each runtime inside its own image. + +**On Mach-O every image carries its own runtime** (mcpp 2026.9.16.1+). The +default there is self-contained for every role, and each image embeds the +payload's `libc++.a` with hidden visibility, so the type information of a +standard library class exists once per image and libc++ compares it by address. +Measured on macos-15: with the default, a `std::runtime_error` thrown in a dylib +is not caught by `catch (const std::runtime_error&)` in the program and two +`std::error_code` categories compare unequal; with `cxx_runtime = "host-coupled"` +for every role, both hold. The default is unchanged, because it is what every +macOS build ships, and a build whose program loads a C++ dylib of its own is told +once (`build/cxx-runtime-identity`): state one runtime for the process when +objects cross the boundary as exceptions, or as libc++ values compared by +identity. + **A dependency's shared library over a C++ runtime that is a package** (mcpp 2026.9.15.2+). When a package in the graph supplies the C++ layer (`llvm.libcxx`, [22](22-target-side.md)), its objects are linked into the program, and it compiles @@ -1086,6 +1120,17 @@ static CRT leaves no DLL to couple to — so it is reported and resolved to nothing (`--mode system`, `--mode static`) cannot deliver `toolchain-coupled` and refuses. +**Clang on the MSVC ABI** (the `llvm` row of `x86_64-windows-msvc`, mcpp +2026.9.16.1+ for the record). The table above describes `cl.exe`, the one +compiler mcpp passes a CRT model to. Clang on the MSVC ABI speaks the GNU dialect +and receives no model, and its driver links the static CRT (`-defaultlib:libcmt`): +a program built on this row imports no `vcruntime140.dll`, `msvcp140.dll` or +`api-ms-win-crt-*`, and each DLL carries its own CRT. The row is therefore +`self-contained` whatever `cxx_runtime` says, `resolution.json` records it so, and +an explicit `host-coupled` or `toolchain-coupled` prints that the row does not +deliver it. A project that needs the dynamic CRT on the MSVC ABI builds with +`msvc@system`. + **Scope.** The contract governs the C++ runtime only. Static **libc** is a separate axis (`linkage = "static"` / `--static`, e.g. a musl target), and the deployment floor is a third — `macos_deployment_target` in `[package]` for Apple targets, diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 798a51fdf..d06cce58a 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -751,6 +751,14 @@ that follow the header state each needed name and what satisfies it ([50 — Machine Output](50-machine-output.md)), so a provider that places libraries itself reads them rather than inferring the closure from `lib/`. +**A member that stages libraries of its own follows the strip decision +(2026.9.16.1+).** `mcpp pack` strips the program, every shared library the graph +built and the staged copy of the toolchain's runtime, and `--no-strip` and +`--debug-symbols` govern all of them. A provider that places further libraries +(an Android archive's native libraries) reads `mcpp::pack_strip()` (`"1"` or +`"0"`) and `mcpp::pack_debug_symbols_dir()` in the pass that submits, so one +switch governs every file in the package. + Commands are an **argv, not a shell string** (no shell is assumed — Windows has none to rely on), and the only interpolations are a closed set: @@ -766,6 +774,50 @@ none to rely on), and the only interpolations are a closed set: The raw stdout protocol above remains the low-level substrate; `import mcpp;` is the typed layer over it. +### Reading the resolved graph: `graph_file` (2026.9.16.1+) + +`mcpp::dep_dir` answers by name for a dependency the package declares itself. +A framework that merges what every library contributes (resources, platform +sources, an Info.plist fragment) needs the libraries its application does not +name as well, in an order in which a later contribution overrides an earlier +one. `mcpp::graph_file()` names a JSON document that states the resolved graph: + +```jsonc +{ + "kind": "mcpp.graph", + "version": 1, + "packages": [ // dependencies before their requesters + { + "package": { "canonical": "spike.b@0.2.0", "namespace": "spike", + "name": "b", "version": "0.2.0", "source": "path" }, + "root": false, + "requested_by": [ { "requester": "spike.a@0.1.0", "key": "spike.b", + "table": "dependencies" } ], + "link": { "form": "static", "reason": "..." }, // a library only + "manifest_dir": "/abs/path/to/b", + "features": [], + "targets": [ { "name": "b", "kind": "lib" } ], + "metadata": { "demo": { "resources": "res" } } // [package.metadata], verbatim + } + ] +} +``` + +- **The root package's program receives it; a dependency's program reads + `""`.** The root decides the graph, and when its program runs every input of + that decision is final, which is the reason `dep_linkage` is offered to it + alone. +- **`[package.metadata.]` is the package's statement about itself.** The + engine does not interpret the table. A path in it is resolved by the reader + against the entry's `manifest_dir`, because only the reader knows which values + are paths. Older engines ignore the table, so a published package may state + it before its consumers upgrade. +- **The document's content is part of the re-run key.** Editing a dependency's + `[package.metadata]` re-runs the root program; editing that dependency's + sources does not. +- The entries are the `graph` section of `resolution.json` with four additions + (`manifest_dir`, `features`, `targets`, `metadata`), from one derivation. + ### `import mcpp;` is the surface that evolves (mcpp 2026.8.5.1+) Two ways to talk to mcpp, and they carry **different compatibility promises**: @@ -879,6 +931,8 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_TARGET_MIN_PLATFORM_VERSION` *(2026.9.12.3+)* | `mcpp::min_platform_version()` | The project's floor for this triple, in the platform's own words: on macOS, `[build] macos_deployment_target` or the engine's own default `14.0`; on iOS, `[build] ios_deployment_target` verbatim, empty when the project states none; on `*-linux-android`, `[target.] min_api_level`, or a fallback the resolved NDK payload states; empty on every other row. The value the effective triple carries, and part of the re-run key | | `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | The `--format` value of the `mcpp pack` pass this program is part of; empty for every ordinary build. The empty value is the one that carries the meaning — a member gates its submission on this, so `mcpp build` has the graph it always had | | `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | Where `mcpp pack` has already staged the closure, absolute; empty when this build is not packaging. Read it to decide the shape of the work; write `${mcpp.stage_dir}` into the action, so the path in the graph and the path the program read cannot disagree | +| `MCPP_PACK_STRIP` *(2026.9.16.1+)* | `mcpp::pack_strip()` | `1` when the `mcpp pack` pass this program is part of strips, `0` under `--no-strip`; empty for every ordinary build. A member that stages libraries of its own follows it, so one switch governs every file in the package | +| `MCPP_PACK_DEBUG_SYMBOLS_DIR` *(2026.9.16.1+)* | `mcpp::pack_debug_symbols_dir()` | Where `--debug-symbols` sends the separated `*.debug` files, absolute; empty when they are discarded or the build is not packing | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | the device-kind sources (`.cu`, `.hip`, …) the package's effective `sources` match, package-root-relative, one per line; empty when there are none. The engine compiles none of them — the rule package this program imports turns each into an `mcpp::action`. Already narrowed: a `{ glob, accel }` entry the build does not cover contributes nothing, so `--no-accel` yields an empty list | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | a writable scratch/output dir owned by mcpp | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | the package root (= CWD) | @@ -886,6 +940,7 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_FEATURES` | — | comma-separated active feature list | | `MCPP_DEP__DIR` | `mcpp::dep_dir("name")` | the resolved install dir of each declared dependency (the qualified `namespace.name`, the canonical name, **and** the namespace-stripped spelling when it is unambiguous; same `` sanitization as `MCPP_FEATURE_`). Received by dependencies' build.mcpp **and** the root project's (the root runs after dependency resolution, 0.0.100+) | | `MCPP_DEP__LINKAGE` *(2026.9.15.2+)* | `mcpp::dep_linkage("name")` | the link form each dependency takes in this build, `static` or `shared`, under the names `MCPP_DEP__DIR` uses; empty for a dependency with no library form. The value is the resolution that decides what is linked, so a generated loader entry or `dllimport` declaration follows it. Received by the **root project's** build.mcpp only: the root decides every dependency's form, and a dependency's program runs before packages discovered after it, whose programs supply facts the answer depends on, so `dep_linkage` is always empty there | +| `MCPP_GRAPH_FILE` *(2026.9.16.1+)* | `mcpp::graph_file()` | the resolved dependency graph as a JSON document, packages in dependency order, each with its manifest directory, features, targets and `[package.metadata]` (see [Reading the resolved graph](#reading-the-resolved-graph-graph_file-20269161)). Received by the **root project's** build.mcpp only; the document's content joins the re-run key | These values are folded into the re-run key **unconditionally** — changing the target, profile, or feature set re-runs the program without any @@ -1399,6 +1454,13 @@ int main() { return grpcgen::generate_all() ? 0 : 1; } bare spelling is bound to `mcpplibs.`, else `compat.`, else an unnamespaced ``, else the single remaining candidate — and when it is contested mcpp says so instead of picking silently. +- **A tool is addressed by exactly the names its directory is.** One derivation + names a provider for `dep_dir`, `dep_linkage` and `dep_bin`: the manifest's + `name`, the qualified `namespace.name`, and the bound tail. A package that + writes `namespace = "spike"` and `name = "installer"` apart is therefore + `dep_bin("spike.installer", ...)` as well as `dep_bin("installer", ...)` + (mcpp 2026.9.16.1+; before that release its tool was published under + `installer` alone). #### Older mcpp reading a manifest that uses this diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 1f82c6476..295ef7acd 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -95,6 +95,13 @@ mcpp --format json streaming case and is **not** accepted — asking for it is an error, not a silent fallback. +A command whose `--format` already names its **product** asks for machine +output with `--message-format json` instead, as `mcpp test` does. `mcpp pack +--format` names the package format (`tar`, `dir`, `msi`), so its report is +`mcpp pack --message-format json` (2026.9.16.1+). The shape follows the kind: +`mcpp test` streams a record per test because tests finish over time, and +`mcpp pack` prints one envelope because a pack has one result. + ### Unsupported values and unknown options Both go to **stderr** with **exit code 2**, and write nothing to stdout: @@ -169,6 +176,12 @@ cannot separate the harmless from the thing a gate exists for: Most gates care about `exec-build-script` and `write-project`, and can ignore `init-mcpp-home` — mcpp setting itself up is not the workspace acting. +The table above is what a command **may** do. The `effects` of an envelope are +what the run **did**, and `network` is observed rather than declared: it is +listed whenever the run started an index refresh, an install or a git remote +operation, including one that failed or was stopped by its bound, and never +for a run under `--offline`. + ## 5. `--json` is not `--format json` Two commands shipped a `--json` flag before this protocol existed: @@ -395,6 +408,10 @@ a program classifying the outcome reads `reason`: | `host-module-missing` | `build.mcpp` imports a module no dependency supplies as a host module | | `tool-version-conflict` | two declarations name one xlings package at versions that cannot both hold | | `shared-library-cxx-runtime` | a dependency's C++ shared library in a graph whose C++ runtime is a package, with no private copy stated | +| `offline-download-required` | the run is offline and the plan needs a download: a toolchain, a package, a git revision or the package index *(2026.9.16.1+)* | +| `package-cycle` | the dependency graph contains a cycle of packages; the message names its edges *(2026.9.16.1+)* | +| `program-cxx-runtime-split` | a program or test that states a self-contained C++ runtime loads a C++ shared library of the build that couples to a shared one *(2026.9.16.1+)* | +| `static-package-in-two-images` | a static package several images of the build reach, on a target where an image cannot use another image's copy *(2026.9.16.1+)* | | `other` | a refusal whose branch has not been given a token yet | **Exit 0 whenever the question was answered, including "refused".** "Would @@ -430,8 +447,13 @@ no-write guarantee and the `watch` rules are [SPEC-005](specs/build-database.md). A failure omits `data` and exits 1, with the diagnostic code -`MCPP_BUILD_DATABASE_NO_PROJECT` outside a project or -`MCPP_BUILD_DATABASE_PLAN_FAILED` when planning fails. Warnings leave the +`MCPP_BUILD_DATABASE_NO_PROJECT` outside a project, +`MCPP_OFFLINE_DOWNLOAD_REQUIRED` when an offline plan (`--offline`, +`MCPP_OFFLINE`, `MCPP_NO_AUTO_INSTALL`) needs something that has to be +downloaded (a toolchain, a package, a git revision or the package index; the +message names the first one), or `MCPP_BUILD_DATABASE_PLAN_FAILED` when planning +fails for any other reason. The first of the three is not a defect of the +project: one run without `--offline` removes it. Warnings leave the document in place: | code | | @@ -444,6 +466,28 @@ document in place: `write-global-cache` and `exec-build-script` for the command, and never `write-project`. +### `mcpp.pack` — the products of a pack *(mcpp 2026.9.16.1+)* + +``` +mcpp pack [target] [--format ] [--target ...] --message-format json +``` + +The envelope is printed once, after the command finishes; every human line goes +to stderr, including what the build programs and tools the pack starts print. +`data` is: + +| field | | +|---|---| +| `artifacts` | one record per produced artifact: `path` (absolute), `type` (`file` or `directory`), `format` (the `--format` value, `tar` when omitted) and `targets` (the canonical triple of each leg that went into it). A dispatched format reports the terminal outputs of the actions the request introduced; a several-`--target` Android pack reports one artifact whose `targets` lists every leg | +| `stage` | the tree the artifact was made from: `dir`, `manifest` (the stage manifest below) and `closure` (`walked` or `not-walked`); `null` for a library package and when no tree was staged | + +A failure omits `data`, exits with the command's exit status and carries the +diagnostic code `MCPP_PACK_FAILED`; the reason is on stderr. The per-run +`effects` are `read-project`, `write-project` and `write-global-cache`, with +`exec-build-script` when a build program ran. `--protocol-version` declares +`init-mcpp-home`, `read-project`, `write-project`, `network`, +`write-global-cache` and `exec-build-script` for `pack`. + ### `mcpp test --message-format json` — the test stream ``` diff --git a/docs/specs/build-database.md b/docs/specs/build-database.md index 53f9bd88c..36648a178 100644 --- a/docs/specs/build-database.md +++ b/docs/specs/build-database.md @@ -4,12 +4,12 @@ |---|---| | 规范编号 | SPEC-005 | | 标题 | mcpp 输出的构建数据库:内容、取值规则与不写工程目录的保证 | -| 状态 | 评审中 v1.0 | -| 版本 | 1.0 | -| 最后修改 | 2026-09-15 | +| 状态 | 评审中 v1.1 | +| 版本 | 1.1 | +| 最后修改 | 2026-09-16 | | 对应实现 | mcpp >= 2026.9.15.1 | | 相关设计文档 | `.agents/docs/2026-09-14-636-build-database-and-the-latest-xlings.md` | -| 相关 issue | #636 | +| 相关 issue | #636, #648 | | 依据的外部规范 | S1「C++ Build Database: IDE Profile」profile 0.2.0 与 S2 0.2.0 §3.4,取自 https://github.com/Sunrisepeak/lsp-mcpp-private 提交 `b82859d`(schema 自提交 `28ecd6e` 起未变);JSON Compilation Database | ## 0. 适用范围 @@ -144,11 +144,15 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 (`{"name": "s1", "version": "0.2.0"}` 或 `{"name": "compile-commands"}`)、 `database`、`watch` 与 `inputs-fingerprint`。**已实现** - **R5.2** 失败时信封不含 `data`,`diagnostics` 至少含一条 `error`,退出码为 1:不在 - 工程中为 `MCPP_BUILD_DATABASE_NO_PROJECT`,规划失败为 - `MCPP_BUILD_DATABASE_PLAN_FAILED`,工作区中其消息指出成员。工作区中任一成员规划 - 失败,整次命令失败。**已实现** + 工程中为 `MCPP_BUILD_DATABASE_NO_PROJECT`;离线运行而规划需要下载时为 + `MCPP_OFFLINE_DOWNLOAD_REQUIRED`,消息指出需要下载的第一项;其他规划失败为 + `MCPP_BUILD_DATABASE_PLAN_FAILED`。工作区中消息指出成员;任一成员规划失败,整次命令 + 失败。**已实现**(离线诊断码:mcpp >= 2026.9.16.1) - **R5.3** 信封的 `effects` 为 `read-project` 与 `write-global-cache`,运行了构建程序时 - 另有 `exec-build-script`。**已实现** + 另有 `exec-build-script`,本次运行启动过网络子进程(索引刷新、安装、git 远程操作, + 失败或超时的也算)时另有 `network`。**已实现**(`network`:mcpp >= 2026.9.16.1) +- **R5.4** 规划期间启动的子进程不继承调用方读取标准输出的描述符;xlings 子进程有期限, + 并随 mcpp 一起结束。**已实现**(mcpp >= 2026.9.16.1) ## 6. `watch` 与 `inputs-fingerprint` @@ -170,3 +174,4 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 | 版本 | 日期 | 变更 | |---|---|---| | 1.0 | 2026-09-14 | 首版(#636)。 | +| 1.1 | 2026-09-16 | R5.2 增加离线诊断码 `MCPP_OFFLINE_DOWNLOAD_REQUIRED`;R5.3 的 `network` 按观测列出;新增 R5.4(子进程不继承调用方描述符,xlings 子进程有期限并随 mcpp 结束)(#648)。 | diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index b73f52cb9..07b98fbc1 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -84,6 +84,17 @@ mcpp 会在编译前失败，并同时报出工具链与工程档位。 当**依赖声明的档位高于当前图**时,mcpp 会在编译前说出来,而不是让它在那个依赖的源码里 某处失败。见 [workspace §4.2](07-workspace.md)。 +`[package.metadata.]`(mcpp 2026.9.16.1+)是引擎保留但不解释的表。它是包对自身的 +陈述,供读取它的工具使用,例如收集每个库贡献内容的框架;它通过 `mcpp::graph_file()` 到达根包 +的构建程序([30 —— build.mcpp](30-build-mcpp.md))。其中的路径由读取方相对于该包的清单目录 +解析。`[package]` 中 mcpp 不读取的其他键会被报告,与 `[build]` 一致:给出警告,在 `--strict` +下报错(2026.9.16.1+)。 + +```toml +[package.metadata.demo] +resources = "res" +``` + #### 方言标志与 `import std` BMI 有些标志会改变标准库头文件**声明出什么**,因此预编译的 `import std` BMI 也必须带着它们一起 @@ -441,6 +452,38 @@ linkage = "shared" # 同上,只在选择器命中的行上 无权决定最终程序的布局;真正必须只有一份共享副本的包,应当在自己的 target 上 声明。 +#### 共享库之下的静态包 *(mcpp 2026.9.16.1+)* + +共享库与它到达的静态包链接在一起。一次构建中的每个共享映像都有一个**静态闭包**: +从它的包出发、不穿过另一个共享包所能到达的静态包。一个静态包若恰好只在一个闭包里, +而根工程自己并不到达它,它就链接进那个映像,而不进程序。 + +2026.9.16.1 之前,这样的包进的是程序,共享库在运行期绑定到程序里的那份。这只在 +ELF 上、且只对那个程序成立:该库通不过 `-Wl,-z,defs`,没有链接这个包的宿主加载不了它 +(`undefined symbol`),Mach-O 与 PE 在链接期就解析每一个引用,Android 则先于任何能 +提供这个包的东西加载应用的共享库。 + +被**多个**映像到达的静态包(两个共享库,或一个共享库加程序)没有唯一可放的映像: + +- 在 Mach-O、PE 与 Android 应用行上,构建在编译之前被拒绝,原因为 + `static-package-in-two-images`([50](50-machine-output.md)); +- 在其他 ELF 行上,这个包照旧留在程序里,构建会报告它(`build/static-placement`), + `--strict` 下为错误。 + +消息会写出这个包、到达它的映像,以及出路:让这个包取共享形态,使每个映像加载同一份。 + +```toml +[dependencies] +x = { path = "../x", linkage = "shared" } # 写在根工程的依赖边上 + +# 或作为这个包自己的默认值,写在它的 manifest 里 +[targets.x] +linkage = "shared" +``` + +提供目标层的包(`provides = ["mcpp:..."]`,C 库或 C++ 运行时)不在这条规则之内: +它的对象放在哪里由运行时契约决定([20](20-toolchains.md))。 + #### library 目标上的 `soname` `soname`(§2.2)在 `kind = "lib"` 上同样可以声明。它是一个库被**找到**时用的 @@ -458,7 +501,12 @@ linkage = "shared" # 同上,只在选择器命中的行上 针对其链接的构建上。链接器和加载器都不会为此报任何一句话。 这项检查是**测量**而不是声明:读产物的动态符号表,去掉 copy relocation,只报告 -产物自身闭包里**也**有定义的那些。进程里只有一份副本的安排保持静默。判定记录在 +产物自身闭包里**也**有定义的那些。进程里只有一份副本的安排保持静默。有三类共同定义 +只计数、不报告 *(后两类自 2026.9.16.1 起)*:加载器按设计统一的 vague linkage +(`STB_WEAK`,以及 GCC 用于内联实体静态数据的 `STB_GNU_UNIQUE`);构建从**同一个 +目标文件**链接进两个映像的定义,例如每个导入 `std` 的 C++ 映像里的 `std` 模块初始化 +函数;以及同一个初始化函数与工具链自身 C++ 运行时之间的重复,GCC 16 起该运行时也导出 +它。形状相同、定义在其他地方的名字仍然会被报告。判定记录在 `target///resolution.json` 的 `runtime.symbol_provision` 下,带计数 与分母,CI 不需要 `readelf` 就能读。 diff --git a/docs/zh/05-dependencies.md b/docs/zh/05-dependencies.md index c8363d795..8090f86ef 100644 --- a/docs/zh/05-dependencies.md +++ b/docs/zh/05-dependencies.md @@ -150,6 +150,27 @@ warning: 'huxdemo.comp@path' declares the dependency 'fw', which names mcpplibs. 点名二者;修正方式是在该清单中声明 `namespace`,或两处写同一个键。`version` 依赖 不受影响,它的身份就是键。 +### git 仓库中的一个包(mcpp 2026.9.16.1+) + +`git` 依赖指向一个仓库,键说明指的是仓库里的哪一个包。根清单的包是其一;根清单 +`[workspace] members` 的每一项是另一个。键的身份不是根包时,选中清单声明该身份的那个 +member,提交相同: + +```toml +# repo/mcpp.toml 声明 spike.fw 与 [workspace] members = ["tool"]; +# repo/tool/mcpp.toml 声明 spike.fw-installer +[dependencies] +spike.fw = { git = "https://example.org/fw.git", rev = "cc3c74c5" } +spike.fw-installer = { git = "https://example.org/fw.git", rev = "cc3c74c5", tools = ["fw-installer"] } +``` + +- member 继承仓库的 `[workspace.package]`,与从仓库自己的检出构建时相同。 +- member 中留在克隆目录之内的 `path` 边(例如 `spike.fw = { path = ".." }`)指向同一 + git 源的同一提交,因此它就是根的键解析到的那个包,而不是第二条以 path 为源的声明。 +- 既不指根包也不指任何 member 的键沿用上一节的规则:使用根清单的身份,并给出警告。 + +键按身份选择,不存在 `subdir` 键:较旧的客户端会忽略这样的键,并不声不响地构建根包。 + ### 命名空间解析规则 每个包的身份是**命名空间 + 名字**二元组。每个 selector 都只规范化成一个身份: @@ -247,11 +268,16 @@ mcpp index status # 看本地现状:状态、年龄、修订号 |---|---| | `--offline`(任意命令) | 完全不碰网络——不刷索引、不下载、不自动装工具链,也不发 `git ls-remote`/`clone`。已安装的东西照常构建,包括 commit 已在 `mcpp.lock`、克隆已在缓存里的 git 依赖 | | `MCPP_OFFLINE=1` | 同上,作用于整个 shell 会话或 CI job | -| `~/.mcpp/config.toml` 里 `[index] auto_refresh = false` | 永不自动刷新索引,但下载仍然可用 | +| `~/.mcpp/config.toml` 里 `[index] auto_refresh = false` | 永不隐式刷新索引:依赖未命中时不刷新,安装本地索引缺少的包之前不刷新,工程自定义索引的首次同步也不做(该次构建停止并指出 `mcpp index update`)。下载仍然可用 | `MCPP_NO_AUTO_INSTALL=1` 作为 `--offline` 的旧式窄化拼写仍然有效(它只管工具链的 自动安装)。 +刷新有期限。`[index] refresh_timeout`(秒,默认 120)是一次刷新最长可用的时间;超过 +即被终止,一条警告指出该设置,构建与任何一次刷新失败之后一样,继续使用本地索引。经由 +xlings 的安装在 xlings 连续 300 秒没有任何输出(包括心跳)时被终止。结束 mcpp 会一并 +结束它启动的 xlings 进程。 + 任意命令加 `-v` 可以看到每个依赖的判定结果与原因。 ## `[dev-dependencies]` —— 测试依赖 @@ -296,13 +322,39 @@ protobuf = { version = "35.1", tools = ["protoc"] } `[build-dependencies]` 用于第一个轴无法表达的那种组合 —— 某个包的库不得进入目标, 而它的工具或规则仍然需要。同一个包同时出现在两张表里不是错误:普通声明胜出,因为 -一行 `[build-dependencies]` 不应该悄悄拿掉目标真正需要的库。 +一行 `[build-dependencies]` 不应该悄悄拿掉目标真正需要的库;两条声明各自的请求 +(`tools`、`features`、`host-module`、`reexport`)都作用于这一条边(mcpp 2026.9.16.1+; +此前第二条声明的请求被丢弃)。 + +**只含程序的包只贡献它的程序(mcpp 2026.9.16.1+)。** 声明的 `[targets]` 全部是程序 +(`bin`、`app`、`test`)的依赖没有可链接的东西。它的工具由工具子构建构建,子构建把这个包 +当作自己的根来解析;在消费方的图里它只提供工具与目录:它自己的依赖不在那里解析,它的源码 +不在那里编译,它的 `ldflags` 不进入消费方的链接。因此一个程序可以依赖请求它的那个包, +SDK 正是这样提供一个针对自身构建的程序。没有 `[targets]` 表的包不受影响,即使 +`src/main.cpp` 为它推断出一个程序。 + +包之间的环在解析依赖图的地方被拒绝,并列出环上的边,与缓存模式无关;工具子构建再次请求 +正在构建的同一个工具时,在第一次重复处被拒绝,并给出请求链。 与 `[dev-dependencies]` 不同,这些依赖**会**被传递遍历:一个构建期依赖自己的依赖正是 让它能工作的东西,并且继承它「只服务构建」的性质。 -feature 可以为构建期请求划定范围而无需第二个声明处 —— `[feature-deps.]` 可以给 -一条已经无条件声明的依赖追加 `tools`,所以「按需才要」不需要另开一张表。 +feature 可以为构建期请求划定范围而无需第二个声明处。`[feature-deps.]` 的条目可以 +以相同的源重述一条已经无条件声明的依赖,并为它追加 `tools`,所以「按需才要」不需要另开 +一张表: + +```toml +[dependencies] +spike.installer = { path = "../installer" } + +[feature-deps.installer] +spike.installer = { path = "../installer", tools = ["installer"] } +``` + +重述要写出源,因为每张依赖表都如此:没有 `path`、`git`、`version` 或 `workspace` 的条目 +被当作命名空间表读取并被拒绝,拒绝消息会说明要重述源。重述中的 `tools`、`features`、 +`host-module` 与 `reexport` 加到该行生效的声明上。重述写了另一个源时被拒绝,并列出两个源 +(mcpp 2026.9.16.1+);此前它被忽略。 > 这个段很早就能被解析,而直到 2026.8.29.1 之前没有任何做决定的代码读它:写下它得到的是 > 一份能加载的清单、零诊断、零效果。 diff --git a/docs/zh/06-features-and-capabilities.md b/docs/zh/06-features-and-capabilities.md index 21360dc29..862a87d32 100644 --- a/docs/zh/06-features-and-capabilities.md +++ b/docs/zh/06-features-and-capabilities.md @@ -35,6 +35,27 @@ extra = [] - **严格校验**:目标包声明了 `[features]` 表时,请求一个未声明的 feature 产生 警告,在 `--strict` 下是错误。不声明 `[features]` 的包接受任意请求(纯宏用法)。 +### 依赖的 feature + +`<依赖>/` 形式的记号打开某个依赖的 feature。依赖以消费方清单所写的键命名 +(`spike.fw`、`compat.opencv`),记号是可加的:它只打开依赖更多的部分,从不把依赖拉进来。 + +```toml +[dependencies] +spike.fw = { path = "../fw" } + +[features] +windows-installer = ["spike.fw/installer"] # 与表形式中的 `forward = [...]` 相同 +``` + +- **命令行上**(mcpp 2026.9.16.1+),`mcpp build --features spike.fw/installer` 为一条命令 + 打开同一个 feature,与根的转发相同;`run`、`test`、`pack`、`emit build-database` 与 + `why deps` 同样接受该记号。它从不是根的 feature,也从不变成宏。 +- **校验**读取写下该转发的清单的每一张依赖表:`[dependencies]`、`[build-dependencies]`、 + `[dev-dependencies]` 与 `[feature-deps.]`,覆盖所有行。只在别的行或未激活的 feature + 下声明的键同样算已声明;在当前行上该转发不到达任何边,不产生效果。没有任何表声明的键会被 + 报告,在 `--strict` 下报告为错误。命令行上无论根是否声明 `[features]`,都做同样的检查。 + ### 表形式 —— 让 feature 贡献的不止是隐含 feature `[features]` 的条目除了写成数组,还可写成**表**,从而让该 feature 在隐含 feature diff --git a/docs/zh/07-workspace.md b/docs/zh/07-workspace.md index f4039e902..a19ec8d15 100644 --- a/docs/zh/07-workspace.md +++ b/docs/zh/07-workspace.md @@ -345,6 +345,11 @@ myproject/ 各成员的构建产物位于各自的 `target/` 子目录下。 +工作空间之外的项目以 member 的身份引用托管在 git 上的工作空间中的 member: +`myproject.http = { git = "...", rev = "..." }` 在根清单的 `members` 中选中 `libs/http`, +提交相同,且该 member 与在此处一样继承 `[workspace.package]`(mcpp 2026.9.16.1+;见 +[05 —— 依赖](05-dependencies.md))。 + ## 7. 与 C++ 模块的关系 工作空间与 C++23 模块机制协同工作： diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 2ad3918d9..b7e22d8d6 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -110,6 +110,8 @@ mcpp pack --format appimage # 由图里某个包提供的格式 mcpp pack -o myapp.tar.gz # 仅文件名:落到 target/dist/myapp.tar.gz mcpp pack -o /abs/path/myapp.tar.gz # 含目录:按字面路径输出 mcpp pack --profile dev # 换一个 profile 构建(默认 release) +mcpp pack --dev # 同上,拼法与 build、run 一致;--profile 优先于它 +mcpp pack --message-format json # 在 stdout 上输出一个 mcpp.pack 信封(mcpp 2026.9.16.1+) mcpp pack --no-strip # 按构建原样发货,不剥符号 mcpp pack --debug-symbols dbg/ # 把分离出的 *.debug 写到 dbg/ mcpp pack --format msi --features installer # 为这次打包启用根包 feature @@ -121,6 +123,16 @@ mcpp pack --format msi --features installer # 为这次打包启用根包 feat `[feature-deps.]` 下,只由点名 `` 的那次打包构建。 `mcpp run --format --features ` 把同样的 feature 交给它执行的那次打包。 +`--release` 与 `--dev`(mcpp 2026.9.16.1+)是 `build`、`run` 所接受的简写,优先级相同: +三条命令上都是 `--profile` 优先于它们。 + +`--message-format json`(mcpp 2026.9.16.1+)在命令结束后于 stdout 上输出一个 +`mcpp.pack` 信封,所有给人读的行都走 stderr。其 `data.artifacts` 列出产出的每个文件或 +目录:绝对路径、`type`(`file` 或 `directory`)、`--format` 取值以及各条腿的三元组; +`data.stage` 给出暂存树、它的 manifest 以及闭包是否走通 +([50 —— 机器输出](50-machine-output.md))。这条命令上的 `--format` 表示包格式,因此 +机器输出按 `mcpp test` 的方式请求。 + ### `--format` 是一个轴,引擎只拥有其中两个取值 `tar` 与 `dir` 回答的问题,和 `msi` 与 `appimage` 回答的问题是同一个 —— 输出取什么 @@ -239,8 +251,20 @@ DWARF 带着发布者源码树与构建目录的绝对路径。剥什么取决 > 既碰不到静态归档、也无法分离出任何东西;前者管的是**包里带什么**。 > 两个不同的决定,两个不同的名字。 -**被捆绑进来的库永远不 strip。** 它们来自 store 或宿主,不是 mcpp 构建的, -为了这一个 bundle 去改写别人的共享载荷不是打包器该做的事。 +**图构建出的东西在每条把调试信息放在映像内的行上都会被剥离;来自 store 或宿主的库不会** +(mcpp 2026.9.16.1+)。规则是 dh_strip 的「包剥离自己构建的东西」,覆盖范围: + +| 暂存的文件 | 是否剥离 | 原因 | +|---|---|---| +| 程序 | 是,按可执行文件;在 Android 行上按共享库,因为那里的程序就是共享库 | 本次构建编译了它 | +| 图中 `SharedLibrary` 链接单元产出的共享库 | 是,`--strip-unneeded` | 本次构建从源码编译了它 | +| 工具链自身运行时的暂存副本(NDK 的 `libc++_shared.so`) | 是,`--strip-unneeded` | 树里的副本不是共享载荷;Android Gradle 插件剥离同一个文件 | +| 来自 store、宿主的库,或包部署的预构建库 | 否 | 其字节属于其发布者,供应商的库可能带签名 | + +Mach-O 与 MSVC PE 的调试信息放在映像之外,因此那里什么都不剥离,`Packing` 行只在确实剥离 +的行上写「stripped」。`--no-strip` 与 `--debug-symbols` 管辖每一个被剥离的文件。自己暂存库的 +构建程序通过 `mcpp::pack_strip()` 与 `mcpp::pack_debug_symbols_dir()` 读到同一个决定 +([30 —— build.mcpp](30-build-mcpp.md)) ## 产物布局 diff --git a/docs/zh/20-toolchains.md b/docs/zh/20-toolchains.md index 56154663c..ff6e73e82 100644 --- a/docs/zh/20-toolchains.md +++ b/docs/zh/20-toolchains.md @@ -916,6 +916,31 @@ C++ 运行时的进程。 工程级的 `cxx_runtime = "…"`(或 `static_stdlib = false`)同样作用于共享库: 有人写下了整个工程的承诺。只有在**没人写**的时候,随格式变化的默认值才生效。 +**加载 C++ 共享库的程序**(mcpp 2026.9.16.1+)。在 ELF 上,上面两个默认值会在同一个 +进程里冲突:self-contained 的程序加载 toolchain-coupled 的 C++ 共享库,进程里就同时 +有一份静态 C++ 运行时和一份共享的。可执行文件导出该库引用的运行时符号,库把其中一部分 +绑到程序的那份、其余留给自己,两半对共享状态的认识互不一致;实测 `llvm@22.1.8` 下这样的 +程序在库第一次格式化字符串时以 `std::bad_cast` 中止,`gcc@16.1.0` 下能跑但有 900 个 +libstdc++ 符号被抢占。因此在 ELF 上,没有人声明契约的程序或测试,若直接或经由另一个 +共享库加载本次构建产出的 C++ 共享库,就取该共享库的契约。进程本来就通过库自己的 +`NEEDED` 条目需要那份运行时,所以没有任何部署因此多出依赖;`resolution.json` 在 +`runtime.cxx_runtime_by_role` 下记录最终的契约。 + +声明过的契约从不改动。**声明了** `self-contained` 的程序或测试,若加载耦合到共享运行时的 +C++ 共享库,会在编译前被拒绝(reason `program-cxx-runtime-split`,见 +[50](50-machine-output.md));出路是删掉这条声明,或用 +`cxx_runtime = { shared = "self-contained" }` 给共享库一份私有副本,让每份运行时留在 +各自的映像里。 + +**Mach-O 上每个映像各带一份运行时**(mcpp 2026.9.16.1+)。那里每个角色的默认值都是 +self-contained,每个映像以隐藏可见性内嵌载荷的 `libc++.a`,于是标准库类的类型信息每个 +映像各有一份,而 libc++ 按地址比较它。macos-15 实测:按默认值,dylib 里抛出的 +`std::runtime_error` 在程序中不被 `catch (const std::runtime_error&)` 捕获,两个 +`std::error_code` 的 category 比较不相等;所有角色都写 `cxx_runtime = "host-coupled"` +时两者都成立。默认值不变——它是今天每个 macOS 构建的形态——但程序加载了本次构建的 C++ +dylib 时会被告知一次(`build/cxx-runtime-identity`):当对象以异常、或以按身份比较的 +libc++ 值跨过边界时,为整个进程声明一份运行时。 + **C++ 运行时来自图中的包时,依赖的共享库**(mcpp 2026.9.15.2+)。当图中有包提供 C++ 层(`llvm.libcxx`,见 [22](22-target-side.md)),它的对象被链进程序,并且以隐藏 可见性编译,于是一个构建为 C++ 共享库的依赖无法解析到程序里的那份。mcpp 在编译前 @@ -975,6 +1000,14 @@ Windows 组件(Win10 起),mcpp 从不分发它;而 `vcruntime140.dll` / 所以会被报出来并落到 `self-contained`。另一半由 `mcpp pack` 兜底:什么都不打包的 模式(`--mode system`、`--mode static`)兑现不了 `toolchain-coupled`,会直接拒绝。 +**MSVC ABI 上的 clang**(`x86_64-windows-msvc` 的 `llvm` 行,记录自 mcpp 2026.9.16.1 +起)。上表描述的是 `cl.exe`,mcpp 只向它传递 CRT 模型。MSVC ABI 上的 clang 使用 GNU +方言、收不到任何模型,它的驱动链接静态 CRT(`-defaultlib:libcmt`):这一行构建的程序 +不导入 `vcruntime140.dll`、`msvcp140.dll` 或 `api-ms-win-crt-*`,每个 DLL 各带一份 CRT。 +因此这一行无论 `cxx_runtime` 写什么都是 `self-contained`,`resolution.json` 如实记录, +显式写 `host-coupled` 或 `toolchain-coupled` 会打印这一行兑现不了它。需要在 MSVC ABI +上使用动态 CRT 的工程,用 `msvc@system` 构建。 + **边界。** 该契约只管 C++ 运行时。静态 **libc** 是另一根轴(`linkage = "static"` / `--static`,如 musl 目标),部署下限是第三根轴 —— Apple 目标用 `[package]` 里的 `macos_deployment_target`,Android 用 `[target.]` 下的 `min_api_level`。 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index ade2ed513..c042d4d9e 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -640,6 +640,11 @@ mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件,永不是 被需要的名字以及满足它的东西([50 —— 机器输出](50-machine-output.md)),自行放置库的 提供方读这些行,而不是从 `lib/` 里推断闭包。 +**自己暂存库的成员跟随剥离决定(2026.9.16.1+)。** `mcpp pack` 剥离程序、图构建出的每个 +共享库以及工具链运行时的暂存副本,`--no-strip` 与 `--debug-symbols` 管辖其中每一个。 +另行放置库的提供方(Android 归档里的原生库)在提交的那次 pass 中读 `mcpp::pack_strip()` +(`"1"` 或 `"0"`)与 `mcpp::pack_debug_symbols_dir()`,于是一个开关管住包里的每个文件。 + 命令是 **argv 而不是 shell 字符串**(不假设存在 shell —— Windows 没有能依赖的那个), 插值只有封闭的一组: @@ -654,6 +659,43 @@ mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件,永不是 上面的裸 stdout 协议仍是底层基底;`import mcpp;` 是其上的类型化层。 +### 读取解析后的依赖图:`graph_file`(2026.9.16.1+) + +`mcpp::dep_dir` 按名字回答本包自己声明的依赖。一个要合并每个库贡献的东西(资源、平台源码、 +一段 Info.plist)的框架,还需要应用没有点名的那些库,而且要按「后来的贡献覆盖先前的」的 +顺序。`mcpp::graph_file()` 给出一份描述解析后依赖图的 JSON 文档: + +```jsonc +{ + "kind": "mcpp.graph", + "version": 1, + "packages": [ // 依赖排在请求它的包之前 + { + "package": { "canonical": "spike.b@0.2.0", "namespace": "spike", + "name": "b", "version": "0.2.0", "source": "path" }, + "root": false, + "requested_by": [ { "requester": "spike.a@0.1.0", "key": "spike.b", + "table": "dependencies" } ], + "link": { "form": "static", "reason": "..." }, // 仅库有 + "manifest_dir": "/abs/path/to/b", + "features": [], + "targets": [ { "name": "b", "kind": "lib" } ], + "metadata": { "demo": { "resources": "res" } } // [package.metadata],原样 + } + ] +} +``` + +- **根包的程序拿到它;依赖包的程序读到 `""`。** 根包决定这张图,而它的程序运行时这个决定的 + 每个输入都已确定 —— 这正是 `dep_linkage` 只提供给根包的原因。 +- **`[package.metadata.]` 是包对自身的陈述。** 引擎不解释这张表。其中的路径由读取方 + 相对于该条目的 `manifest_dir` 解析,因为只有读取方知道哪些值是路径。旧引擎忽略这张表, + 所以已发布的包可以在其使用方升级之前就写上它。 +- **文档内容属于重跑键。** 修改某个依赖的 `[package.metadata]` 会让根包的程序重跑;修改该 + 依赖的源码不会。 +- 这些条目就是 `resolution.json` 的 `graph` 一节再加四项(`manifest_dir`、`features`、 + `targets`、`metadata`),出自同一次推导。 + ### `import mcpp;` 才是会演进的那一面(mcpp 2026.8.5.1+) 和 mcpp 对话有两条路,它们的**兼容性承诺不同**: @@ -755,6 +797,8 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 | `MCPP_TARGET_MIN_PLATFORM_VERSION` *(2026.9.12.3+)* | `mcpp::min_platform_version()` | 项目对这个三元组的下限,用平台自己的措辞:macOS 上是 `[build] macos_deployment_target` 或引擎自带的默认值 `14.0`;iOS 上是 `[build] ios_deployment_target` 原样给出,项目没写就是空;`*-linux-android` 上是 `[target.] min_api_level`,或已解析 NDK 载荷给出的回落值;其余每一行都是空。取的是有效三元组携带的那个值,并进入重跑键 | | `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | 本程序所处的这次 `mcpp pack` 的 `--format` 取值;任何普通构建下都为空。承载含义的正是这个空值 —— 成员据此为自己的提交加闸,于是 `mcpp build` 拿到的还是它一直以来的那张图 | | `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | `mcpp pack` 已经把闭包暂存到的位置,绝对路径;本次构建不在打包时为空。读它来判断这次要干的活是什么形状,而把 `${mcpp.stage_dir}` 写进 action —— 这样图里的路径与程序读到的路径不可能不一致 | +| `MCPP_PACK_STRIP` *(2026.9.16.1+)* | `mcpp::pack_strip()` | 本程序所处的这次 `mcpp pack` 剥离时为 `1`,`--no-strip` 下为 `0`;任何普通构建下都为空。自己暂存库的成员跟随它,于是一个开关管住包里的每个文件 | +| `MCPP_PACK_DEBUG_SYMBOLS_DIR` *(2026.9.16.1+)* | `mcpp::pack_debug_symbols_dir()` | `--debug-symbols` 放置分离出的 `*.debug` 文件的位置,绝对路径;丢弃调试信息或本次构建不在打包时为空 | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | 本包有效 `sources` 匹配到的设备类源文件(`.cu`、`.hip`…),相对包根,一行一个;没有时为空串。引擎一个都不编译 —— 由本程序引入的规则包把每一个变成一条 `mcpp::action`。已经过收窄:构建未覆盖的 `{ glob, accel }` 条目贡献为空,因此 `--no-accel` 得到空列表 | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | mcpp 提供的可写输出/暂存目录 | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | 包根(= CWD) | @@ -762,6 +806,7 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 | `MCPP_FEATURES` | — | 活跃 feature 逗号列表 | | `MCPP_DEP__DIR` | `mcpp::dep_dir("name")` | 每个已声明依赖解析后的安装目录(限定名 `namespace.name`、canonical 名,以及无歧义时的去命名空间短名,都可用;`` 消毒规则同 `MCPP_FEATURE_`)。依赖包的 build.mcpp **和**根工程的 build.mcpp 都能拿到(根工程的 build.mcpp 在依赖解析之后运行,0.0.100+) | | `MCPP_DEP__LINKAGE` *(2026.9.15.2+)* | `mcpp::dep_linkage("name")` | 每个依赖在本次构建中的链接形态,`static` 或 `shared`,名字与 `MCPP_DEP__DIR` 相同;没有库形态的依赖为空。该值就是决定链接内容的那次解析,生成的加载入口或 `dllimport` 声明因此与之一致。只有**根工程**的 build.mcpp 能拿到:根工程决定每个依赖的形态,而依赖包的程序运行在发现顺序更靠后的包之前,那些包的程序提供了答案所依赖的事实,所以在那里 `dep_linkage` 总是为空 | +| `MCPP_GRAPH_FILE` *(2026.9.16.1+)* | `mcpp::graph_file()` | 以 JSON 文档表示的解析后依赖图,包按依赖顺序排列,每个包带其清单目录、feature、target 与 `[package.metadata]`(见[读取解析后的依赖图](#读取解析后的依赖图graph_file20269161))。只有**根工程**的 build.mcpp 能拿到;文档内容进入重跑键 | 这些契约值**无条件**折入重跑键——换 target、换 profile、开关 feature 都会触发重跑, 不需要任何 `rerun-if-env-changed` 声明。 @@ -1175,6 +1220,11 @@ int main() { return grpcgen::generate_all() ? 0 : 1; } 尾名 `protobuf`。全限定的 `MCPP_DEP___BIN_` 总是发布;裸名 依次绑定到 `mcpplibs.`、`compat.`、无命名空间的 ``,最后才是「剩下 的唯一候选」——存在争用时 mcpp 会说出来,而不是默默选一个。 +- **工具的名字与它的目录的名字完全相同。** `dep_dir`、`dep_linkage` 与 `dep_bin` + 使用同一个推导为提供者命名:清单的 `name`、限定名 `namespace.name`,以及已绑定的尾名。 + 因此把 `namespace = "spike"` 与 `name = "installer"` 分开写的包,既是 + `dep_bin("installer", ...)`,也是 `dep_bin("spike.installer", ...)`(mcpp 2026.9.16.1+; + 此前它的工具只以 `installer` 发布)。 #### 旧版 mcpp 读到用了这些键的 manifest diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 246a10c4a..ea2ef3d62 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -77,6 +77,11 @@ mcpp <命令> --format json 目前只支持 `json`。`ndjson` 保留给未来真正需要流式的场景,**现在不接受** —— 请求它是 错误,不是静默回落。 +`--format` 已经表示其**产物**的命令改用 `--message-format json` 请求机器输出,与 +`mcpp test` 一致。`mcpp pack --format` 表示包格式(`tar`、`dir`、`msi`),因此它的报告是 +`mcpp pack --message-format json`(2026.9.16.1+)。形状随 kind 而定:测试随时间陆续完成, +所以 `mcpp test` 每个测试一条流式记录;一次打包只有一个结果,所以 `mcpp pack` 输出一个信封。 + ### 不支持的值 / 未知选项 两者都走 **stderr**、退出码 **2**,且**不往 stdout 写任何东西**: @@ -145,6 +150,10 @@ mcpp --protocol-version 多数门只在乎 `exec-build-script` 与 `write-project`,可以忽略 `init-mcpp-home` —— mcpp 给自己做初始化不是工作区在动作。 +上表是命令**可能**做的事。信封里的 `effects` 是这次运行**做了**的事,其中 `network` +按观测记录而不是按声明:本次运行启动过索引刷新、安装或 git 远程操作时即列出,包括失败 +的或被期限终止的那一次;在 `--offline` 下运行时从不列出。 + ## 5. `--json` 不等于 `--format json` 有两条命令在本协议之前就发布了 `--json`: @@ -348,6 +357,10 @@ mcpp why toolchain [--target ] [--toolchain ] --format json | `host-module-missing` | `build.mcpp` 导入了没有任何依赖以 host module 形式提供的模块 | | `tool-version-conflict` | 两处声明把同一个 xlings 包定在不能同时成立的版本上 | | `shared-library-cxx-runtime` | C++ 运行时来自图中的包,而依赖的 C++ 共享库没有声明私有副本 | +| `offline-download-required` | 本次运行离线,而规划需要下载:工具链、包、git 修订或包索引 *(2026.9.16.1+)* | +| `package-cycle` | 依赖图中存在包的环;消息列出环上的边 *(2026.9.16.1+)* | +| `program-cxx-runtime-split` | 声明了自含 C++ 运行时的程序或测试,加载了本次构建中耦合到共享运行时的 C++ 共享库 *(2026.9.16.1+)* | +| `static-package-in-two-images` | 一个静态包被本次构建的多个映像到达,而在该目标上一个映像不能使用另一个映像里的副本 *(2026.9.16.1+)* | | `other` | 一处还没有被命名的拒绝分支 | **只要问题被回答了就退 0,包括答案是「拒绝」。** 「它能不能构建,不能的话 @@ -378,8 +391,11 @@ mcpp emit build-database [--spec s1|compile-commands] --format json 不带 `--format` 时命令只输出文档;`-o ` 把原本输出的内容写入 ``。文档的 内容、不写工程目录的保证与 `watch` 的规则见 [SPEC-005](../specs/build-database.md)。 -失败时省略 `data` 并以 1 退出:不在工程中时诊断码为 `MCPP_BUILD_DATABASE_NO_PROJECT`, -规划失败时为 `MCPP_BUILD_DATABASE_PLAN_FAILED`。警告不影响文档: +失败时省略 `data` 并以 1 退出:不在工程中时诊断码为 `MCPP_BUILD_DATABASE_NO_PROJECT`; +离线规划(`--offline`、`MCPP_OFFLINE`、`MCPP_NO_AUTO_INSTALL`)需要下载某样东西(工具链、 +包、git 修订或包索引,消息指出第一个)时为 `MCPP_OFFLINE_DOWNLOAD_REQUIRED`;其他原因的 +规划失败为 `MCPP_BUILD_DATABASE_PLAN_FAILED`。第一种不是工程的缺陷,不带 `--offline` +运行一次即可消除。警告不影响文档: | 诊断码 | | |---|---| @@ -390,6 +406,25 @@ mcpp emit build-database [--spec s1|compile-commands] --format json `--protocol-version` 为这条命令声明 `init-mcpp-home`、`read-project`、`network`、 `write-global-cache` 与 `exec-build-script`,从不声明 `write-project`。 +### `mcpp.pack` —— 一次打包的产物 *(mcpp 2026.9.16.1+)* + +``` +mcpp pack [target] [--format ] [--target ...] --message-format json +``` + +信封在命令结束后输出一次;所有给人读的行都走 stderr,包括打包启动的构建程序与工具的输出。 +`data` 为: + +| 字段 | | +|---|---| +| `artifacts` | 每个产物一条记录:`path`(绝对路径)、`type`(`file` 或 `directory`)、`format`(`--format` 取值,省略时为 `tar`)与 `targets`(进入该产物的每条腿的规范三元组)。分派格式报告本次请求引入的 action 的终端输出;多 `--target` 的 Android 打包报告一个产物,其 `targets` 列出每条腿 | +| `stage` | 产物所来自的那棵树:`dir`、`manifest`(即下文的暂存清单)与 `closure`(`walked` 或 `not-walked`);库包以及未暂存任何树时为 `null` | + +失败时省略 `data`,以命令自身的退出码退出,并带诊断码 `MCPP_PACK_FAILED`;原因在 stderr 上。 +每次运行的 `effects` 为 `read-project`、`write-project` 与 `write-global-cache`,有构建程序运行时 +再加 `exec-build-script`。`--protocol-version` 为 `pack` 声明 `init-mcpp-home`、 +`read-project`、`write-project`、`network`、`write-global-cache` 与 `exec-build-script`。 + ### `mcpp test --message-format json` —— 测试流 ``` diff --git a/mcpp.toml b/mcpp.toml index d5c896a93..9a00dcb8d 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.15.2" +version = "2026.9.16.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 4d310e3bc..49dab0b5a 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -88,6 +88,67 @@ std::string version_req_problem(std::string_view spec) { return {}; } + +// A TOML table as JSON text, for `[package.metadata]` (#647 E1). The TOML +// reader here produces strings, integers, booleans, arrays and tables, and each +// has one JSON spelling; nothing is interpreted. +void append_json_string(std::string& out, std::string_view s) { + out += '"'; + for (unsigned char c : s) { + switch (c) { + case '"': out += "\\\""; break; + case '\\': out += "\\\\"; break; + case '\n': out += "\\n"; break; + case '\r': out += "\\r"; break; + case '\t': out += "\\t"; break; + default: + if (c < 0x20) out += std::format("\\u{:04x}", static_cast(c)); + else out += static_cast(c); + } + } + out += '"'; +} + +void append_json_value(std::string& out, const mcpp::libs::toml::Value& v) { + using Kind = mcpp::libs::toml::Value::Kind; + switch (v.kind()) { + case Kind::String: append_json_string(out, v.as_string()); break; + case Kind::Int: out += std::to_string(v.as_int()); break; + case Kind::Bool: out += v.as_bool() ? "true" : "false"; break; + case Kind::Array: { + out += '['; + bool first = true; + for (auto const& e : v.as_array()) { + if (!first) out += ','; + first = false; + append_json_value(out, e); + } + out += ']'; + break; + } + case Kind::Table: { + out += '{'; + bool first = true; + for (auto const& [k, e] : v.as_table()) { + if (!first) out += ','; + first = false; + append_json_string(out, k); + out += ':'; + append_json_value(out, e); + } + out += '}'; + break; + } + case Kind::Null: out += "null"; break; + } +} + +std::string toml_table_to_json(const mcpp::libs::toml::Table& table) { + std::string out; + append_json_value(out, mcpp::libs::toml::Value(table)); + return out; +} + } // namespace @@ -1030,6 +1091,41 @@ std::expected parse_string(std::string_view content, if (auto v = doc->get_string_array("build.std-module-flags")) m.buildConfig.stdModuleFlags = *v; + // [package.metadata] (#647 E1): kept verbatim for the graph document. + if (auto* meta = doc->get_table("package.metadata")) + m.packageMetadataJson = toml_table_to_json(*meta); + + // Surface unsupported [package] keys the way [build] does (#649 X4): a + // warning for the root manifest, an error under --strict, and nothing for + // a dependency's manifest, whose schema warnings are not surfaced (a + // package may adopt a key before its consumers upgrade). `[package]` was + // the one central section without the check, so a misspelt `licence` or + // `descripton` was accepted in silence. + // + // MUST stay in sync with the `doc->get_*("package.")` reads above. + static constexpr std::string_view kKnownPackageKeys[] = { + "accelerators", "authors", "description", "exclusive", "license", + "metadata", "name", "namespace", "platforms", "provides", "repo", + "requires", "requires_abi", "standard", "std-compat-module", + "std-module", "std-module-flags", "version", + }; + if (auto* pt = doc->get_table("package")) { + for (auto& [key, _] : *pt) { + bool known = false; + for (auto k : kKnownPackageKeys) if (key == k) { known = true; break; } + if (!known) { + std::string supported; + for (auto k : kKnownPackageKeys) { + if (!supported.empty()) supported += ", "; + supported += k; + } + m.schemaWarnings.push_back(std::format( + "[package] has unsupported key '{}' (ignored). Supported keys: {}.", + key, supported)); + } + } + } + // [capabilities] cap = "provider" — root-only provider pins. if (auto* caps = doc->get_table("capabilities"); caps && !caps->empty()) { for (auto& [cap, cval] : *caps) @@ -3799,6 +3895,7 @@ void apply_defaults_and_infer(Manifest& m, const std::filesystem::path& root) { t.kind = Target::Binary; t.main = "src/main.cpp"; m.targets.push_back(std::move(t)); + m.targetsInferred = true; m.inferredNotes.push_back( std::format("target {} (bin from src/main.cpp)", m.package.name)); } else if (hasModuleInterface) { @@ -3806,6 +3903,7 @@ void apply_defaults_and_infer(Manifest& m, const std::filesystem::path& root) { t.name = m.package.name; t.kind = Target::Library; m.targets.push_back(std::move(t)); + m.targetsInferred = true; m.inferredNotes.push_back( std::format("target {} (lib from {} in src/)", m.package.name, moduleInterfaceExt)); } diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index b59597722..68bec4f5a 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1978,11 +1978,29 @@ struct Manifest { bool usesModules = true; // refined by scanner bool usesImportStd = true; // refined by scanner std::vector inferredNotes; // for `Inferred ...` banner + // `targets` was inferred from the source tree (no `[targets]` table), not + // declared. A consumer treats a package whose DECLARED targets are all + // programs as a tool provider that contributes nothing to its own graph + // (#649 E6); a package that declared nothing keeps the older reading, so an + // inferred `bin` from `src/main.cpp` is not taken as that statement. + bool targetsInferred = false; // Non-fatal schema warnings collected during parse (e.g. unsupported keys // under [targets.]). The caller (prepare_build) prints these and, under // --strict, escalates them to errors — mirroring the feature/platform path. std::vector schemaWarnings; + + // `[package.metadata]`, as a JSON object text; empty when the manifest has + // none (#647 E1). + // + // THE ENGINE DOES NOT INTERPRET IT. The table belongs to whoever reads it + // (`[package.metadata.]`), and it travels to the root build program + // verbatim in the graph document (`mcpp::graph_file()`). A path in it is + // resolved by that reader against the package's manifest directory, which + // the document states beside it, because only the reader knows which + // values are paths. Kept as text so this module's interface names no JSON + // type. + std::string packageMetadataJson; }; struct ManifestError { diff --git a/modules/platform/src/env.cppm b/modules/platform/src/env.cppm index 7b636bd29..02dce080d 100644 --- a/modules/platform/src/env.cppm +++ b/modules/platform/src/env.cppm @@ -49,6 +49,20 @@ bool offline_mode(); // and kept next to offline_mode() so the two network knobs cannot drift apart. bool no_auto_install(); +// WHAT THIS RUN DID WITH THE NETWORK, AS OPPOSED TO WHAT IT MAY DO. +// +// docs/50 defines an envelope's `effects` as what running the command did, +// and `--protocol-version` as what a command may do. The per-run list used to +// be assembled from constants, so a plan that refreshed the index reported no +// `network` (#648 A4). A launcher of a network-bound child (an index refresh, +// an install, a git remote operation) calls `note_network_access` before it +// starts the child; an attempt that fails or times out still counts, because +// the effect is the access. Offline, those launchers return before this call. +// Kept beside offline_mode() for the reason stated above: every layer can reach +// this module, and the record has writers in several of them. +void note_network_access(); +bool network_accessed(); + // Temporarily set or unset an env var, restoring the prior value on scope exit. class ScopedEnv { public: @@ -112,6 +126,13 @@ bool no_auto_install() { return v && *v && std::string_view(v) != "0"; } +namespace { +std::atomic g_network_accessed{false}; +} + +void note_network_access() { g_network_accessed.store(true, std::memory_order_relaxed); } +bool network_accessed() { return g_network_accessed.load(std::memory_order_relaxed); } + std::optional get(std::string_view key) { std::string k(key); auto* v = std::getenv(k.c_str()); diff --git a/modules/platform/src/process.cppm b/modules/platform/src/process.cppm index 605726081..446a77bcb 100644 --- a/modules/platform/src/process.cppm +++ b/modules/platform/src/process.cppm @@ -242,6 +242,31 @@ int run_silent(std::string_view command); int run_streaming(std::string_view command, std::function on_line); +// run_streaming for a child that mcpp must own and bound (#648). +// +// `run_streaming` hands its child to `popen`: the child is in mcpp's process +// group, is not killed when mcpp is, and is waited on with no limit. That was +// the xlings index refresh that held an editor for eleven minutes on a +// connection that never answered. This runs the same shell command (the same +// quoting, the same sealed stdin) through the deadline launchers instead: +// +// - the child owns a process group on POSIX, registered with the signal +// guard, and runs in a kill-on-close job object on Windows, so +// terminating mcpp terminates the command's whole tree; +// - `total` bounds the run; `idle` bounds the time without output. Either +// may be zero (no bound of that kind), not both; +// - a bound that fires kills the tree and sets *timed_out; the exit code is +// then the killed shell's. +// +// Stdout and stderr share the pipe, as they do for every captured deadline +// run; a caller that parses stdout redirects stderr in the command itself. +// Returns -1 when the launcher could not run the command at all. +int run_streaming_bounded(std::string_view command, + std::function on_line, + std::chrono::milliseconds total, + std::chrono::milliseconds idle, + bool* timed_out); + // Run `command`, passing stdout/stderr through to the terminal. // Optionally captures stdout into `output` if non-null. // Returns a platform-normalized exit code (WEXITSTATUS on POSIX). @@ -893,13 +918,20 @@ struct BoundedOutcome { // normal case) means "derive it from argv". A shell command is the one caller // that must NOT be derived that way — see windows_shell_command_line — and the // POSIX branch is unaffected either way, because it never flattens argv. +// `idle` and `ownGroup` are the streaming runner's (see run_streaming_bounded); +// every other caller passes neither and keeps its behaviour. `on_line`, when +// given, receives each complete line as it arrives instead of the output being +// accumulated. BoundedOutcome dispatch_bounded( const std::vector& argv, const std::vector>& extraEnv, std::string_view cwd, std::chrono::milliseconds deadline, bool capture, - std::string_view windowsCommandLine = {}) + std::string_view windowsCommandLine = {}, + std::chrono::milliseconds idle = std::chrono::milliseconds{0}, + bool ownGroup = false, + const std::function* on_line = nullptr) { BoundedOutcome outcome; @@ -916,21 +948,40 @@ BoundedOutcome dispatch_bounded( const char* cwdArg = cwdStore.empty() ? nullptr : cwdStore.c_str(); const auto ms = static_cast(deadline.count()); + const auto idleMs = static_cast(idle.count()); + // One sink for both, appending into the outcome's own buffer. Null when the - // caller wants the child on its own stdio. + // caller wants the child on its own stdio. With `on_line`, complete lines + // leave the buffer as they arrive, and the unterminated tail is delivered + // after the child exits, as run_streaming does. + struct StreamCtx { + std::string* buffer; + const std::function* on_line; + } streamCtx{ &outcome.output, on_line }; using Sink = void (*)(void*, const char*, unsigned long); - const Sink sink = capture + const Sink sink = !capture ? nullptr + : on_line == nullptr ? +[](void* ctx, const char* data, unsigned long len) { - static_cast(ctx)->append(data, len); + static_cast(ctx)->buffer->append(data, len); } - : nullptr; + : +[](void* ctx, const char* data, unsigned long len) { + auto* c = static_cast(ctx); + c->buffer->append(data, len); + std::size_t pos; + while ((pos = c->buffer->find('\n')) != std::string::npos) { + std::string_view line{c->buffer->data(), pos}; + while (!line.empty() && line.back() == '\r') line.remove_suffix(1); + (*c->on_line)(line); + c->buffer->erase(0, pos + 1); + } + }; if constexpr (mcpp::platform::is_windows) { const auto cmd = windowsCommandLine.empty() ? windows_command_from_argv(argv) : std::string(windowsCommandLine); auto r = mcpp::platform::winproc::capture_with_deadline( - cmd.c_str(), envArg, envCount, cwdArg, ms, sink, &outcome.output); + cmd.c_str(), envArg, envCount, cwdArg, ms, idleMs, sink, &streamCtx); outcome.supported = r.supported; outcome.exit_code = r.exit_code; outcome.timed_out = r.timed_out; @@ -941,12 +992,19 @@ BoundedOutcome dispatch_bounded( for (auto const& a : argv) argvPtrs.push_back(a.c_str()); auto r = mcpp::platform::unixproc::capture_with_deadline( argvPtrs.data(), static_cast(argvPtrs.size()), - envArg, envCount, cwdArg, ms, sink, &outcome.output); + envArg, envCount, cwdArg, ms, idleMs, ownGroup ? 1 : 0, + sink, &streamCtx); outcome.supported = r.supported; outcome.exit_code = r.exit_code; outcome.timed_out = r.timed_out; outcome.spawn_error = r.spawn_error; } + if (on_line && capture && !outcome.output.empty()) { + std::string_view tail{outcome.output}; + while (!tail.empty() && tail.back() == '\r') tail.remove_suffix(1); + if (!tail.empty()) (*on_line)(tail); + outcome.output.clear(); + } return outcome; } @@ -1005,6 +1063,28 @@ int run_shell_deadline(std::string_view command, return r.exit_code; } +int run_streaming_bounded(std::string_view command, + std::function on_line, + std::chrono::milliseconds total, + std::chrono::milliseconds idle, + bool* timed_out) +{ + if (timed_out) *timed_out = false; + if (command.empty() || (total.count() <= 0 && idle.count() <= 0)) return -1; + // The popen path's shaping, minus the outer cmd.exe pair: the Windows + // launcher's command line supplies that pair itself. + const auto sealed = seal_stdin(command); + const std::vector argv{"/bin/sh", "-c", sealed}; + std::function sink = + on_line ? std::move(on_line) : [](std::string_view) {}; + auto r = dispatch_bounded(argv, {}, {}, total, /*capture=*/true, + windows_shell_command_line(sealed), idle, + /*ownGroup=*/true, &sink); + if (!r.supported) return -1; + if (timed_out) *timed_out = r.timed_out; + return r.exit_code; +} + // The same split as dispatch_bounded, for the same reason: POSIX names a // program with an argv array, Windows with a single command line. Both // spellings are built here so neither can drift into a launcher that does not diff --git a/modules/platform/src/terminal.cppm b/modules/platform/src/terminal.cppm index 8d5f62033..b8a7fe249 100644 --- a/modules/platform/src/terminal.cppm +++ b/modules/platform/src/terminal.cppm @@ -12,9 +12,17 @@ module; #include #endif #if defined(_WIN32) -#include // _dup, _dup2, _close +#include // _dup, _dup2, _close, _get_osfhandle +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include // SetHandleInformation, HANDLE_FLAG_INHERIT #else -#include // dup, dup2, close +#include // dup2, close +#include // fcntl, F_DUPFD_CLOEXEC #endif export module mcpp.platform.terminal; @@ -39,6 +47,15 @@ std::size_t cols(); // a document (`mcpp emit build-database`) plans under one of these and prints // the document after it is gone: people still see the progress, on stderr, and // the document arrives alone. +// +// THE SAVED DESCRIPTOR IS NOT INHERITED. It is the caller's pipe, and a child +// started during the redirection (a build program, an xlings refresh) that +// inherited it would keep the caller from reading end-of-file for as long as +// the child lives, however long after mcpp itself exited. Measured +// (mcpp-community/mcpp#648): a build program started by +// `emit build-database` held the caller's pipe as its descriptor 3. The copy +// is therefore close-on-exec on POSIX and not inheritable on Windows, so a +// child receives only descriptors 0 to 2, and 1 is standard error here. class StdoutToStderr { public: StdoutToStderr(); @@ -77,9 +94,18 @@ StdoutToStderr::StdoutToStderr() { std::fflush(stdout); #if defined(_WIN32) saved_ = ::_dup(1); - if (saved_ >= 0) ::_dup2(2, 1); + if (saved_ >= 0) { + // `_dup` duplicates the handle as inheritable, and CreateProcess with + // handle inheritance (every `_popen` and `system`) passes it on. + const auto h = reinterpret_cast(::_get_osfhandle(saved_)); + if (h != INVALID_HANDLE_VALUE) + ::SetHandleInformation(h, HANDLE_FLAG_INHERIT, 0); + ::_dup2(2, 1); + } #else - saved_ = ::dup(1); + // Not `dup`: its copy survives exec. Descriptor 3 or above, as `dup` + // would have chosen, so nothing else about the redirection moves. + saved_ = ::fcntl(1, F_DUPFD_CLOEXEC, 3); if (saved_ >= 0) ::dup2(2, 1); #endif } diff --git a/modules/platform/src/unix/bounded_process.cppm b/modules/platform/src/unix/bounded_process.cppm index bb017f839..ab9edf14b 100644 --- a/modules/platform/src/unix/bounded_process.cppm +++ b/modules/platform/src/unix/bounded_process.cppm @@ -87,12 +87,27 @@ using OutputSink = void (*)(void* ctx, const char* data, unsigned long len); // (a) delay every line until the child exits, which is the opposite of what a // bounded `mcpp test` run is for, and (b) make the child's stdout a pipe // rather than a terminal, so gtest and friends silently drop their colors. +// +// `idleMs`, when positive, is a second bound: the child is killed once it has +// written nothing for that long. It is meaningful only with a sink, and it is +// what lets a long, progressing child (a download that prints progress) run to +// completion while a wedged one does not (#648). With `idleMs` positive a +// non-positive `deadlineMs` means "no total bound". +// +// `ownGroup` non-zero places the child in a process group of its own, kills +// the GROUP at a bound, and registers the group with the signal guard for the +// length of the call, so terminating mcpp takes the child's descendants with +// it. It is opt-in because a child in a background group that reads the +// terminal is stopped by SIGTTIN, and the uncaptured callers of this function +// hand the terminal to their child. DeadlineRun capture_with_deadline(const char* const* argvEntries, unsigned long argvCount, const char* const* envEntries, unsigned long envCount, const char* cwd, long long deadlineMs, + long long idleMs, + int ownGroup, OutputSink sink, void* ctx); @@ -188,11 +203,14 @@ DeadlineRun capture_with_deadline(const char* const* argvEntries, unsigned long envCount, const char* cwd, long long deadlineMs, + long long idleMs, + int ownGroup, OutputSink sink, void* ctx) { DeadlineRun out; - if (deadlineMs <= 0 || argvCount == 0 || !argvEntries) return out; + const bool idleBound = idleMs > 0 && sink != nullptr; + if ((deadlineMs <= 0 && !idleBound) || argvCount == 0 || !argvEntries) return out; // The child's environment: ours, minus anything overridden, plus the // overrides. Names are case-SENSITIVE here (unlike the Windows peer). @@ -244,11 +262,19 @@ DeadlineRun capture_with_deadline(const char* const* argvEntries, // else: no file actions for stdio at all — the child inherits ours, which // keeps its output live AND keeps it a terminal. + posix_spawnattr_t attr; + ::posix_spawnattr_init(&attr); + if (ownGroup) { + ::posix_spawnattr_setpgroup(&attr, 0); // 0: a new group, id == pid + ::posix_spawnattr_setflags(&attr, POSIX_SPAWN_SETPGROUP); + } pid_t pid = 0; - int sp = ::posix_spawnp(&pid, cargv[0], &fa, nullptr, cargv.data(), envp.data()); + int sp = ::posix_spawnp(&pid, cargv[0], &fa, &attr, cargv.data(), envp.data()); + ::posix_spawnattr_destroy(&attr); ::posix_spawn_file_actions_destroy(&fa); if (capture) ::close(fds[1]); if (sp != 0) { out.spawn_error = sp; if (capture) ::close(fds[0]); return out; } + if (ownGroup) guard_group_on_signal(pid); // Non-blocking reads so the deadline is still checked while the child is // quiet. A blocking read on a silent, hung child is exactly the hang this @@ -256,8 +282,11 @@ DeadlineRun capture_with_deadline(const char* const* argvEntries, if (capture) ::fcntl(fds[0], F_SETFL, ::fcntl(fds[0], F_GETFL, 0) | O_NONBLOCK); - const auto until = std::chrono::steady_clock::now() - + std::chrono::milliseconds(deadlineMs); + const auto started = std::chrono::steady_clock::now(); + const auto until = deadlineMs > 0 + ? started + std::chrono::milliseconds(deadlineMs) + : std::chrono::steady_clock::time_point::max(); + auto lastOutput = started; std::array buf{}; bool killed = false; int status = 0; @@ -270,6 +299,7 @@ DeadlineRun capture_with_deadline(const char* const* argvEntries, sink(ctx, buf.data(), static_cast(n)); any = true; } + if (any) lastOutput = std::chrono::steady_clock::now(); return any; }; @@ -283,14 +313,23 @@ DeadlineRun capture_with_deadline(const char* const* argvEntries, } if (r < 0 && errno != EINTR && errno != ECHILD) break; - if (!killed && std::chrono::steady_clock::now() >= until) { - ::kill(pid, SIGKILL); + const auto now = std::chrono::steady_clock::now(); + const bool overTotal = now >= until; + const bool overIdle = idleBound + && now - lastOutput >= std::chrono::milliseconds(idleMs); + if (!killed && (overTotal || overIdle)) { + // The group when the child owns one: a shell's grandchild (the + // program the shell ran) would otherwise keep running, and keep + // the pipe open so the drain above never saw end-of-file. + if (ownGroup) ::killpg(pid, SIGKILL); + else ::kill(pid, SIGKILL); killed = true; continue; } struct timespec ts{0, 20'000'000}; // 20ms ::nanosleep(&ts, nullptr); } + if (ownGroup) unguard_group(pid); if (capture) ::close(fds[0]); out.exit_code = normalize_status(status); @@ -472,7 +511,8 @@ void clear_group_guard() { DeadlineRun capture_with_deadline(const char* const*, unsigned long, const char* const*, unsigned long, - const char*, long long, OutputSink, void*) { + const char*, long long, long long, int, + OutputSink, void*) { // Not POSIX: mcpp.platform.windows.bounded_process owns this. return {}; } diff --git a/modules/platform/src/windows/bounded_process.cppm b/modules/platform/src/windows/bounded_process.cppm index 5d832105e..0b2c403f2 100644 --- a/modules/platform/src/windows/bounded_process.cppm +++ b/modules/platform/src/windows/bounded_process.cppm @@ -96,11 +96,19 @@ using OutputSink = void (*)(void* ctx, const char* data, unsigned long len); // "KEY=VALUE" strings applied on top of the current environment. `cwd` may be // null. A non-positive `deadlineMs` is rejected with supported=false — "no // bound" belongs on the caller's untimed path, which needs none of this. +// +// `idleMs`, when positive, kills the child once it has written nothing for that +// long; the POSIX peer states the reason (#648). With `idleMs` positive a +// non-positive `deadlineMs` means "no total bound". There is no `ownGroup` +// parameter: every child here runs in a job object with +// JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE already, which is what the POSIX flag +// buys there. DeadlineRun capture_with_deadline(const char* commandLine, const char* const* envEntries, unsigned long envCount, const char* cwd, long long deadlineMs, + long long idleMs, OutputSink sink, void* ctx); @@ -247,11 +255,13 @@ DeadlineRun capture_with_deadline(const char* commandLine, unsigned long envCount, const char* cwd, long long deadlineMs, + long long idleMs, OutputSink sink, void* ctx) { DeadlineRun out; - if (deadlineMs <= 0 || !commandLine || !*commandLine) return out; + const bool idleBound = idleMs > 0 && sink != nullptr; + if ((deadlineMs <= 0 && !idleBound) || !commandLine || !*commandLine) return out; const bool capture = (sink != nullptr); @@ -337,8 +347,11 @@ DeadlineRun capture_with_deadline(const char* commandLine, // EOF, even after every child has exited. writeEnd.reset(); - const auto until = std::chrono::steady_clock::now() - + std::chrono::milliseconds(deadlineMs); + const auto started = std::chrono::steady_clock::now(); + const auto until = deadlineMs > 0 + ? started + std::chrono::milliseconds(deadlineMs) + : std::chrono::steady_clock::time_point::max(); + auto lastOutput = started; std::array buf{}; bool killed = false; @@ -354,6 +367,7 @@ DeadlineRun capture_with_deadline(const char* commandLine, if (!::ReadFile(readEnd.h, buf.data(), want, &got, nullptr) || got == 0) return false; if (sink) sink(ctx, buf.data(), static_cast(got)); + lastOutput = std::chrono::steady_clock::now(); return true; }; @@ -365,7 +379,10 @@ DeadlineRun capture_with_deadline(const char* commandLine, break; } - if (!killed && std::chrono::steady_clock::now() >= until) { + const auto now = std::chrono::steady_clock::now(); + const bool overIdle = idleBound + && now - lastOutput >= std::chrono::milliseconds(idleMs); + if (!killed && (now >= until || overIdle)) { killed = true; // Closing the job takes the whole tree with it. TerminateProcess // alone would leave grandchildren holding the pipe open. @@ -573,7 +590,7 @@ int wait_background(unsigned long long process, int* exitCode) { #else DeadlineRun capture_with_deadline(const char*, const char* const*, unsigned long, - const char*, long long, OutputSink, void*) { + const char*, long long, long long, OutputSink, void*) { // Not Windows: the POSIX launcher in mcpp.platform.process owns this. return {}; } diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index a9dc41997..512d07159 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.15.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.16.1"; } // namespace mcpp diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index bdb75a562..a064020cb 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -183,6 +183,20 @@ struct BuildProgramEnv { // actually has) and writes the placeholder into the action, so the two // never disagree. std::filesystem::path packStageDir; + // The packaging pass's RESOLVED strip decision, "1" or "0", and where the + // separated debug files go, absolute; both empty for every ordinary build + // (#649 E5). A member that stages libraries of its own follows them, so + // `--no-strip` and `--debug-symbols` reach its files as they reach the + // engine's. + std::string packStrip; + std::filesystem::path packDebugSymbolsDir; + // #647 E1: the graph document offered to the ROOT package's program, and + // the digest of its text. Empty for a dependency's program. The digest + // joins the re-run key (see `run_build_program`), so a change to what the + // document says re-runs the program and a change it does not state does + // not. + std::filesystem::path graphFile; + std::string graphDigest; // Whether this package builds C++ modules (`[language] modules`). // // Reported because a rule package that GENERATES a consumer-facing @@ -629,6 +643,9 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv e.emplace_back("MCPP_PKG_REPO", env.packageRepo); e.emplace_back("MCPP_PACK_FORMAT", env.packFormat); e.emplace_back("MCPP_PACK_STAGE_DIR", env.packStageDir.string()); + e.emplace_back("MCPP_PACK_STRIP", env.packStrip); + e.emplace_back("MCPP_PACK_DEBUG_SYMBOLS_DIR", env.packDebugSymbolsDir.string()); + e.emplace_back("MCPP_GRAPH_FILE", env.graphFile.string()); std::string csv; for (auto const& f : env.features) { if (!csv.empty()) csv += ','; @@ -970,6 +987,11 @@ std::expected run_build_program( fs::path outDir = bdir / "out"; auto childEnv = contract_env(root, outDir, env); std::string ctxHash = contract_hash(childEnv); + // THE GRAPH DOCUMENT'S CONTENT, NOT ITS PATH. The path is the same on + // every run; what the document says is what the program's answer depends + // on (#647 E1). + if (!env.graphDigest.empty()) + ctxHash = mcpp::toolchain::hash_string(ctxHash + "\ngraph=" + env.graphDigest); // ── Helper self-containment (the single decision point) ───────────────── // The compiled helper is exec'd by the host OS, outside anything mcpp diff --git a/src/build/distribution.cppm b/src/build/distribution.cppm index fc094c7b2..6819ded9d 100644 --- a/src/build/distribution.cppm +++ b/src/build/distribution.cppm @@ -250,6 +250,44 @@ Contract default_contract(Role r, Format f) { return Contract::SelfContained; } +// What a manifest states about the C++ runtime, read once for every role. +struct ContractStatement { + std::string_view cxxRuntime; // `cxx_runtime = "..."` or its `default` + std::string_view cxxRuntimeTests; // `cxx_runtime = { tests = "..." }` + std::string_view cxxRuntimeShared; // `cxx_runtime = { shared = "..." }` + bool staticStdlib = true; +}; + +// Which images of a build load a C++ shared library the build itself makes: +// its programs, and its test programs. +struct CxxSharedLoad { + bool program = false; + bool tests = false; +}; + +// The contract each role holds, and whether a human stated it. +// +// A stated contract is what the manifest said; an unstated one is a default, +// and a default is the only thing `role_contracts` is allowed to move. +struct RoleContracts { + Contract program = Contract::SelfContained; + Contract tests = Contract::SelfContained; + Contract intermediate = Contract::SelfContained; + Contract shared = Contract::SelfContained; + bool programStated = false; + bool testsStated = false; + bool sharedStated = false; +}; + +// THE ONE DERIVATION of every role's contract, for the flag assembly and for +// the refusal that reads the same answer before anything compiles. +RoleContracts role_contracts(const ContractStatement& s, Format f, CxxSharedLoad load); + +// The role whose STATED contract splits the process's C++ runtime: on ELF, a +// self-contained program or test that loads a C++ shared library coupled to a +// shared runtime. Empty when no role does. +std::optional runtime_split(const RoleContracts& c, Format f, CxxSharedLoad load); + // The contract a manifest STATES for shared libraries, or nothing when it // states none and the role's default applies. // @@ -269,6 +307,57 @@ std::optional stated_shared_library_contract(std::string_view cxxRunti staticStdlib ? default_contract(Role::Distributable, f) : Contract::HostCoupled); } +RoleContracts role_contracts(const ContractStatement& s, Format f, CxxSharedLoad load) { + RoleContracts c; + c.programStated = !s.cxxRuntime.empty() || !s.staticStdlib; + c.program = parse_contract(s.cxxRuntime).value_or( + s.staticStdlib ? default_contract(Role::Distributable, f) : Contract::HostCoupled); + c.intermediate = c.program; + c.testsStated = c.programStated || !s.cxxRuntimeTests.empty(); + c.tests = parse_contract(s.cxxRuntimeTests).value_or(c.program); + c.sharedStated = c.programStated || !s.cxxRuntimeShared.empty(); + c.shared = stated_shared_library_contract(s.cxxRuntime, s.cxxRuntimeShared, + s.staticStdlib, f) + .value_or(default_contract(Role::SharedLibrary, f)); + + // ONE PROCESS, ONE C++ RUNTIME (#646 F3a). + // + // The ELF defaults are right one at a time and wrong together. A program + // is self-contained and a shared library couples to the toolchain's + // runtime, so a program that LOADS such a library holds a static runtime + // and a shared one. The executable exports the runtime symbols the library + // references, the library binds some of them to the program's copy and + // keeps the rest, and the two halves disagree about shared state. + // Measured on Linux x86_64 with llvm@22.1.8: the program aborted with + // `std::bad_cast` (exit 134) as soon as the library formatted a string; + // with gcc@16.1.0 it ran with 900 libstdc++ symbols interposed. + // + // So a role nobody stated takes the shared library's contract when its + // image loads a C++ shared library this build makes. The process already + // needs that runtime through the library's own NEEDED entry, so no + // deployment gains a requirement; what changes is that the program binds + // to the same copy. A stated contract is never changed here; one that + // splits the runtime is refused by the caller (`runtime_split`). + if (f == Format::Elf) { + if (load.program && !c.programStated) c.program = c.shared; + if (load.tests && !c.testsStated) c.tests = c.shared; + } + return c; +} + +std::optional runtime_split(const RoleContracts& c, Format f, CxxSharedLoad load) { + if (f != Format::Elf) return std::nullopt; + // A shared library that embeds a hidden private copy keeps its runtime to + // itself (`hide_static_cxx_runtime`); that is the documented private-copy + // arrangement, not a split. A coupled library shares the process's. + if (c.shared == Contract::SelfContained) return std::nullopt; + if (load.program && c.programStated && c.program == Contract::SelfContained) + return Role::Distributable; + if (load.tests && c.testsStated && c.tests == Contract::SelfContained) + return Role::Test; + return std::nullopt; +} + // ---------------------------------------------------------------- Layer 3 struct MechanismInput { @@ -294,6 +383,13 @@ struct MechanismInput { // decoration. Derived by `msvc_wants_static_crt`, which is also what // emits the flag. bool msvcStaticCrt = false; + // MSVC STL only: does mcpp pass a CRT model (`/MT` or `/MD`) to this + // compiler? True for cl.exe. FALSE FOR CLANG ON THE MSVC ABI: that driver + // speaks the GNU dialect, mcpp emits no runtime flag for it, and clang + // then links the static CRT (`-defaultlib:libcmt`, measured with the + // 22.1.8 driver). The table must report the model the compiler was given, + // not the model `cl.exe` would have been given (#649 E10). + bool msvcCrtModelEmitted = true; // Toolchain capability id: "libstdc++", "libc++", or an MSVC STL spelling. std::string_view stdlibId; Format format = Format::Elf; @@ -625,6 +721,26 @@ Mechanism resolve(const MechanismInput& in) { // // No unit flags: the model is a COMPILE flag on every TU, not // something added to the link line. + // + // CLANG ON THE MSVC ABI IS GIVEN NO MODEL, so the table records + // the one its driver chooses. The rows below were written for + // cl.exe, and for this row they recorded `host-coupled` beside an + // artifact that imports no vcruntime DLL at all (#649 E10). The + // artifact is left as it is; the record, and an explicit request + // the row does not deliver, now say what it is. + if (!in.msvcCrtModelEmitted) { + m.effective = Contract::SelfContained; + if (in.requested != Contract::SelfContained && in.explicitRequest) { + m.degraded = true; + m.diagnostic = std::format( + "cxx_runtime = \"{}\" is not delivered for clang on the " + "MSVC ABI: mcpp passes this driver no CRT model, and clang " + "links the static CRT (libcmt) by default. Use msvc@system " + "for the dynamic CRT; using self-contained", + to_string(in.requested)); + } + return m; + } m.effective = in.msvcStaticCrt ? Contract::SelfContained : Contract::HostCoupled; if (in.requested == Contract::SelfContained && !in.msvcStaticCrt) { diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 27e6c4f40..fb3ddb901 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -886,8 +886,20 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, // result over in ctx.resolvedVersions; fall back to the spec only for // deps that never went through resolution (git, or an exact pin). auto rit = ctx.resolvedVersions.find(name); + // A git dependency has no version to announce: `spec.version` is empty + // for it, and the banner used to read "Compiling spike.fw v" (#649 E7). + // It names the reference the manifest wrote instead, shortening a + // commit to the length `git` itself abbreviates to. + auto gitReference = [&] { + std::string ref = spec.gitRev; + if (spec.gitRefKind == "rev" && ref.size() > 12) ref.resize(12); + return std::format("(git {} {})", + spec.gitRefKind.empty() ? "rev" : spec.gitRefKind, ref); + }; std::string ver = spec.isPath() ? "(path)" + : spec.isGit() + ? gitReference() : std::string("v") + (rit != ctx.resolvedVersions.end() ? rit->second : spec.version); auto it = cachedUnits.find(name); diff --git a/src/build/flags.cppm b/src/build/flags.cppm index ca7e97502..7e656f686 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -139,6 +139,42 @@ std::string render_link_intent_flags( CompileFlags compute_flags(const BuildPlan& plan); +// ── Which link line a (host, target) pair takes (#647 E3) ───────────────── +// +// THE HOST DOES NOT DECIDE THIS ALONE. Three of the link branches describe a +// link on this machine's own platform family: `link.exe` for the MSVC dialect, +// `-fuse-ld=lld` without a payload model for a PE, and the Apple SDK line +// (`-isysroot`, the deployment floor) for a Mach-O. They used to be selected by +// `if constexpr` on the HOST, which was the same question while each host built +// only for its own family. An Android row built on a macOS host is the first +// target that separated them, and the Apple line it received had no +// `--target`, so `-fuse-ld=lld` selected `ld64.lld` for an ELF object: +// +// ld64.lld: error: unknown argument '-soname' +// +// The generic branch is the one that consumes `link_toolchain_flags`, where +// the target is named. Every target a host branch does not describe takes it. +// +// ONE EXCEPTION IS KEPT, AND IT IS STATED BY ITS INPUT. On a Windows host a +// non-PE target whose driver is NOT told its target by a flag (a canadian GCC +// cross, which names its target by its own prefix) keeps the PE-host line it +// has always had: nothing on that line is wrong for such a driver, and the +// cross build that uses it is verified in CI. A target named by `--target` +// (an SDK such as the NDK, or a retargetable clang) is exactly the case the +// host line cannot serve. +enum class LinkHost { Linux, MacOS, Windows }; +enum class LinkShape { MsvcLinkExe, PeLld, AppleSdk, Generic }; + +LinkShape link_shape(LinkHost host, mcpp::build::dist::Format targetFormat, + bool msvcDialect, bool targetNamedByFlag); + +// The host this binary was built for, in `link_shape`'s vocabulary. +constexpr LinkHost current_link_host() { + return mcpp::platform::is_windows ? LinkHost::Windows + : mcpp::platform::needs_explicit_libcxx ? LinkHost::MacOS + : LinkHost::Linux; +} + // The kind → role map. One line of policy, in one place: a test binary runs on // the build machine and is then thrown away; an archive embeds no runtime at // all; everything else leaves this machine. Backends ask this, never the kind. @@ -424,6 +460,23 @@ std::string render_link_intent_flags( return out; } +LinkShape link_shape(LinkHost host, mcpp::build::dist::Format targetFormat, + bool msvcDialect, bool targetNamedByFlag) { + using mcpp::build::dist::Format; + switch (host) { + case LinkHost::Windows: + if (msvcDialect) return LinkShape::MsvcLinkExe; + if (targetFormat == Format::Pe) return LinkShape::PeLld; + return targetNamedByFlag ? LinkShape::Generic : LinkShape::PeLld; + case LinkHost::MacOS: + return targetFormat == Format::MachO ? LinkShape::AppleSdk + : LinkShape::Generic; + case LinkHost::Linux: + return LinkShape::Generic; + } + return LinkShape::Generic; +} + CompileFlags compute_flags(const BuildPlan& plan) { CompileFlags f; @@ -760,6 +813,18 @@ CompileFlags compute_flags(const BuildPlan& plan) { // is only assemblable by its own x86_64-w64-mingw32-as. bool isMuslTc = mcpp::toolchain::is_musl_target(plan.toolchain); bool isMingwTc = mcpp::toolchain::is_mingw_target(plan.toolchain); + // The object format the TARGET produces, derived once: the runtime contract + // table and the link-line shape both read it (#647 E3). Target-keyed, with + // the host's format only as the fallback for a triple that names none; a + // MinGW toolchain is a PE whatever its triple spelling says. + const mcpp::build::dist::Format targetObjectFormat = + isMingwTc ? mcpp::build::dist::Format::Pe + : mcpp::build::dist::format_for(plan.toolchain.targetTriple, + mcpp::platform::needs_explicit_libcxx + ? mcpp::build::dist::Format::MachO + : mcpp::platform::is_windows + ? mcpp::build::dist::Format::Pe + : mcpp::build::dist::Format::Elf); const auto linkIntentFlavor = [&] { if (isMingwTc) return LinkIntentFlavor::PeGnu; if (isMsvcDialect) return LinkIntentFlavor::PeMsvc; @@ -1044,41 +1109,33 @@ CompileFlags compute_flags(const BuildPlan& plan) { // table because it recognises a mingw toolchain by more than its // triple; everything after it is `dist::format_for`, which is where // the question is answered and where it is tested. - const dist::Format format = - isMingwTc ? dist::Format::Pe - : dist::format_for(plan.toolchain.targetTriple, - mcpp::platform::needs_explicit_libcxx - ? dist::Format::MachO - : mcpp::platform::is_windows - ? dist::Format::Pe - : dist::Format::Elf); + const dist::Format format = targetObjectFormat; // `static_stdlib` is a faithful alias of the two ends of the contract: // its documented meaning has always been exactly self-contained vs the // dynamic system runtime. An explicit `cxx_runtime` wins. // - // The role defaults come from `dist::default_contract` rather than - // being spelled again here. They were spelled again here, and that - // second derivation is why `default_contract` sat with no caller while - // this file quietly disagreed with it about shared libraries. - const dist::Contract base = - dist::parse_contract(bc.cxxRuntime).value_or( - bc.staticStdlib - ? dist::default_contract(dist::Role::Distributable, format) - : dist::Contract::HostCoupled); - const dist::Contract testsContract = - dist::parse_contract(bc.cxxRuntimeTests).value_or(base); - // A project-wide statement (`cxx_runtime = "…"` or `static_stdlib = - // false`) applies to shared libraries too — a human said what the - // whole project promises. Only when nobody said anything does the - // role's own default apply, which is the case that changes on ELF. - // The statement is read by `stated_shared_library_contract`, which the - // refusal of a C++ shared library over a graph runtime reads too (#641). - const bool projectWideExplicit = !bc.cxxRuntime.empty() || !bc.staticStdlib; - const dist::Contract sharedContract = - dist::stated_shared_library_contract(bc.cxxRuntime, bc.cxxRuntimeShared, - bc.staticStdlib, format) - .value_or(dist::default_contract(dist::Role::SharedLibrary, format)); + // Every role's contract comes from `dist::role_contracts`, the one + // derivation the refusal of a split runtime reads as well. The role + // defaults were spelled here once, and that second derivation is why + // `default_contract` sat with no caller while this file quietly + // disagreed with it about shared libraries. A project-wide statement + // (`cxx_runtime = "..."` or `static_stdlib = false`) applies to shared + // libraries too; only when nobody said anything does a role's own + // default apply, and on ELF a program or test that loads a C++ shared + // library of this build takes that library's contract (#646 F3a). + const dist::CxxSharedLoad cxxSharedLoad{ + .program = mcpp::build::image_loads_cxx_shared_library(plan, LinkUnit::Binary), + .tests = mcpp::build::image_loads_cxx_shared_library(plan, LinkUnit::TestBinary), + }; + const dist::RoleContracts contracts = dist::role_contracts( + dist::ContractStatement{ + .cxxRuntime = bc.cxxRuntime, + .cxxRuntimeTests = bc.cxxRuntimeTests, + .cxxRuntimeShared = bc.cxxRuntimeShared, + .staticStdlib = bc.staticStdlib, + }, + format, cxxSharedLoad); // Archive lookup, directory half. LLVM lays these out either directly // under lib/ (the macOS packages) or under lib// (the @@ -1134,6 +1191,9 @@ CompileFlags compute_flags(const BuildPlan& plan) { // ignoring a role override. mi.msvcStaticCrt = mcpp::toolchain::msvc_wants_static_crt( bc.linkage, bc.cxxRuntime); + // The model is emitted only for the `msvc` dialect (above); clang on + // the MSVC ABI receives none and links the static CRT (#649 E10). + mi.msvcCrtModelEmitted = isMsvcDialect; mi.mingw = isMingwTc; mi.macosFloor = !macosDeploymentTarget.empty(); // READ from the one value prepare resolved. The SDK being located for @@ -1233,9 +1293,10 @@ CompileFlags compute_flags(const BuildPlan& plan) { }; const bool wantsArchives = - (base == dist::Contract::SelfContained - || testsContract == dist::Contract::SelfContained - || sharedContract == dist::Contract::SelfContained) + (contracts.program == dist::Contract::SelfContained + || contracts.tests == dist::Contract::SelfContained + || contracts.intermediate == dist::Contract::SelfContained + || contracts.shared == dist::Contract::SelfContained) && caps.stdlib_id == "libc++"; // THE ARCHIVES A LINKER SCRIPT OPENS, by file name. // @@ -1317,9 +1378,9 @@ CompileFlags compute_flags(const BuildPlan& plan) { // "Explicit" = a human wrote it down. `static_stdlib = false` counts: // nobody sets a flag to its default to get non-default behavior. - const bool explicitBase = projectWideExplicit; - const bool explicitTests = explicitBase || !bc.cxxRuntimeTests.empty(); - const bool explicitShared = explicitBase || !bc.cxxRuntimeShared.empty(); + const bool explicitBase = contracts.programStated; + const bool explicitTests = contracts.testsStated; + const bool explicitShared = contracts.sharedStated; // Report a role's degradation only if this build HAS that role. // @@ -1337,10 +1398,10 @@ CompileFlags compute_flags(const BuildPlan& plan) { bool wantsToolchainRuntime = false; for (auto [role, requested, wasAsked] : { - std::tuple{dist::Role::Distributable, base, explicitBase}, - std::tuple{dist::Role::Test, testsContract, explicitTests}, - std::tuple{dist::Role::Intermediate, base, explicitBase}, - std::tuple{dist::Role::SharedLibrary, sharedContract, explicitShared}}) { + std::tuple{dist::Role::Distributable, contracts.program, explicitBase}, + std::tuple{dist::Role::Test, contracts.tests, explicitTests}, + std::tuple{dist::Role::Intermediate, contracts.intermediate, explicitBase}, + std::tuple{dist::Role::SharedLibrary, contracts.shared, explicitShared}}) { mi.role = role; mi.requested = requested; mi.explicitRequest = wasAsked; @@ -1568,8 +1629,13 @@ CompileFlags compute_flags(const BuildPlan& plan) { // survives a replacement. Declared here rather than inside either, because // the ordering between them is the whole point. std::string platformAnchor; - if constexpr (mcpp::platform::is_windows) { - if (isMsvcDialect) { + // WHICH BRANCH, asked of the host AND the target (see `link_shape`). The + // format is `targetObjectFormat`, the one the contract table reads. + const LinkShape linkShape = link_shape(current_link_host(), targetObjectFormat, + isMsvcDialect, + !plan.toolchain.crossTargetFlag.empty()); + if (linkShape == LinkShape::MsvcLinkExe || linkShape == LinkShape::PeLld) { + if (linkShape == LinkShape::MsvcLinkExe) { // Native cl.exe: link.exe does the link (SeparateLinker). Search // paths for dependency runtime import libs via /LIBPATH; user // ldflags pass through verbatim; GNU link_extra (-flto/-s) does @@ -1634,7 +1700,7 @@ CompileFlags compute_flags(const BuildPlan& plan) { f.ld = std::format("{} -fuse-ld=lld{}{}{}", full_static, link_intent_ld, user_ldflags, link_extra); f.ldC = f.ld; // no C++ runtime token on this line - } else if constexpr (mcpp::platform::needs_explicit_libcxx) { + } else if (linkShape == LinkShape::AppleSdk) { // macOS. The C++ runtime itself is decided by the contract table above // (dist::Format::MachO) and rides unit_ldflags; what is left here is // the rest of the macOS link: @@ -1794,10 +1860,10 @@ CompileFlags compute_flags(const BuildPlan& plan) { // ── The target side comes from the graph, so the HOST's link is wrong ── // - // THE THREE BRANCHES ABOVE ARE SHAPED BY THIS MACHINE, NOT BY THE - // TARGET. `if constexpr (is_windows)` / `needs_explicit_libcxx` / else is a - // question about where mcpp itself was built, and each answer describes a - // link on that machine: an SDK path, a deployment target, a loader search + // THE BRANCHES ABOVE DESCRIBE A LINK AGAINST A PAYLOAD. `link_shape` now + // routes a target outside the host's own family to the generic branch, but + // the Apple and PE lines still describe the host platform's payload, and + // each answer describes a link there: an SDK path, a deployment target, a loader search // path, this host's `libatomic`. Every one of them is right when the target // is the host or is served by a payload, and wrong when the C library, the // C++ runtime and the platform are packages in the dependency graph. diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index f5c36b2c4..aecbf8398 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -560,6 +560,37 @@ inline const char* pack_format() { return env_or("MCPP_PACK_FOR // then names it through the placeholder, so the path in the graph and the path // this program read cannot disagree. inline const char* pack_stage_dir() { return env_or("MCPP_PACK_STAGE_DIR"); } + +// WHETHER THIS PACKAGING PASS STRIPS: "1" or "0", and "" for every ordinary +// build (#649 E5). +// +// The engine strips the program, every shared library the graph built and the +// staged copy of the toolchain's runtime, and `--no-strip` turns all of it off. +// A member that stages libraries of its own -- an Android archive's native +// libraries, say -- asks here instead of deciding for itself, so one switch +// governs every file in the package. An engine older than this one leaves it +// empty, which a member reads as "decide as before". +inline const char* pack_strip() { return env_or("MCPP_PACK_STRIP"); } + +// Where `--debug-symbols` sends the separated `*.debug` files, absolute; "" +// when they are discarded or this build is not packing (#649 E5). +inline const char* pack_debug_symbols_dir() { return env_or("MCPP_PACK_DEBUG_SYMBOLS_DIR"); } + +// THE RESOLVED DEPENDENCY GRAPH, as a JSON document; "" for a dependency's +// program and under an older engine (#647 E1). +// +// Offered to the ROOT package's program only, for the reason `dep_linkage` is: +// the root decides the graph, and when its program runs every input of that +// decision is final. `packages` lists every package, dependencies before the +// packages that request them, each with `package` (identity), `root`, +// `requested_by`, `link` (for a library), `manifest_dir` (absolute), +// `features`, `targets` and `metadata` (its `[package.metadata]`, verbatim). +// A path inside `metadata` is the reader's to resolve, against that entry's +// `manifest_dir`. +// +// Editing a package's `[package.metadata]` re-runs this program; editing its +// sources does not. +inline const char* graph_file() { return env_or("MCPP_GRAPH_FILE"); } inline bool has_feature(const char* name) { char buf[256] = "MCPP_FEATURE_"; unsigned long o = 13; diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index fb1485749..9445d4932 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -2184,7 +2184,16 @@ std::string emit_ninja_string(const BuildPlan& plan) { // LINK order and has no priority-ordered init section, so "runs // before the user's global constructors" is spelled "is the first // input" — nothing else about this edge achieves it (#336). - if (need_ios_init_shim && lu.kind != LinkUnit::StaticLibrary) + // ... and only into a unit that HOLDS a C++ runtime. The shim calls + // libc++'s `ios_base::Init` constructor, so a unit linked without the + // C++ runtime (a C-only shared library, whose line carries neither the + // C++ driver nor libc++.a) has nothing to resolve it against: measured + // on macos-15, `ld64.lld: error: undefined symbol: + // std::__1::ios_base::Init::Init()` from this object, in a build whose + // program is C++ and whose dependency's shared library is C (#646 F1's + // fixture). The predicate is the one the link line itself uses. + if (need_ios_init_shim && lu.kind != LinkUnit::StaticLibrary + && unit_needs_cxx_runtime(lu)) ins += " " + escape_ninja_path(ios_init_obj); for (auto& o : lu.objects) { ins += " " + escape_ninja_path(o); diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 94d9844d6..432630726 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -178,6 +178,46 @@ struct ResolvedRuntimeContract { ResolvedRuntimeContract resolve_runtime_contract( const std::vector& packages); +// A static package this build could not link into exactly one image (#646 F1). +// +// `images` names the shared libraries of the plan whose static closure +// reaches the package (by target name); `program` says whether the root's own +// images reach it as well. `make_plan` leaves such a package where it always +// was, in the root's images, and records it here: whether that is refused or +// reported depends on the target format, which prepare decides. +struct StaticPlacementConflict { + std::string package; // qualified package name + std::vector images; // shared library target names + bool program = false; +}; + +// Where each static package of a graph is linked. +// +// A shared image's STATIC CLOSURE is the set of static packages reachable from +// its package without crossing another shared package. A static package in +// exactly one closure, which the root does not reach itself, belongs in that +// image; one in several closures, or in one and the root's, has no single +// image to live in (`conflicts`). Packages in no closure stay with the root. +// +// Indices are package indices; 0 is the root. `sharedImages` maps each shared +// package to the number of images it produces; `boundaries` are packages that +// are neither placed nor traversed (a distribution package's library is +// already built). +struct StaticPlacement { + std::map intoImage; // static package -> shared package + struct Conflict { + std::size_t package = 0; + std::vector images; // shared packages reaching it + bool root = false; + }; + std::vector conflicts; +}; + +StaticPlacement place_static_packages( + const std::map>& edges, + const std::map& sharedImages, + const std::set& boundaries); + struct BuildPlan { mcpp::manifest::Manifest manifest; mcpp::toolchain::Toolchain toolchain; @@ -215,6 +255,10 @@ struct BuildPlan { // the failure is `relocation R_X86_64_32S ... can not be used when making // a shared object` on a file nobody edited. bool needsPic = false; + // Static packages that no single image could take (#646 F1); see + // `StaticPlacementConflict`. Empty for every graph without a shared + // library over a static package that something else reaches too. + std::vector staticPlacementConflicts; std::string scheduleTag = "none"; // Whether `--accel` / `--no-accel` selected this graph's device variant // over `[build] accel`. The variant is in the fingerprint, so the two @@ -428,6 +472,14 @@ package_link_objects(const BuildPlan& plan, std::string_view packageName); // library without a C++ runtime reads the same answer (#641). bool link_unit_holds_cxx(const BuildPlan& plan, const LinkUnit& lu); +// Does an image of `kind` load a shared library of this plan that holds C++, +// directly or through another shared library of the plan? Answered from the +// link edges the plan declares (a consumer lists each shared library it links +// among its implicit inputs), so it is the process the build will produce and +// not a guess about it. The C++ runtime contract of programs and tests reads +// this (#646 F3a). +bool image_loads_cxx_shared_library(const BuildPlan& plan, LinkUnit::Kind kind); + } // namespace mcpp::build namespace mcpp::build { @@ -453,6 +505,69 @@ bool link_unit_holds_cxx(const BuildPlan& plan, const LinkUnit& lu) { }); } +StaticPlacement place_static_packages( + const std::map>& edges, + const std::map& sharedImages, + const std::set& boundaries) { + auto closure = [&](std::size_t start) { + std::set out; + std::vector stack{start}; + while (!stack.empty()) { + const auto at = stack.back(); + stack.pop_back(); + auto it = edges.find(at); + if (it == edges.end()) continue; + for (auto next : it->second) { + if (next == 0 || next == start) continue; + if (sharedImages.contains(next) || boundaries.contains(next)) continue; + if (out.insert(next).second) stack.push_back(next); + } + } + return out; + }; + const auto rootClosure = closure(0); + std::map> owners; + for (auto const& [shared, count] : sharedImages) { + (void)count; + for (auto s : closure(shared)) owners[s].push_back(shared); + } + StaticPlacement out; + for (auto const& [s, images] : owners) { + std::size_t imageCount = 0; + for (auto p : images) imageCount += sharedImages.at(p); + const bool root = rootClosure.contains(s); + if (!root && images.size() == 1 && imageCount == 1) + out.intoImage.emplace(s, images.front()); + else + out.conflicts.push_back({ .package = s, .images = images, .root = root }); + } + return out; +} + +bool image_loads_cxx_shared_library(const BuildPlan& plan, LinkUnit::Kind kind) { + std::map sharedByOutput; + for (auto const& lu : plan.linkUnits) + if (lu.kind == LinkUnit::SharedLibrary) + sharedByOutput.emplace(lu.output.lexically_normal().generic_string(), &lu); + if (sharedByOutput.empty()) return false; + for (auto const& lu : plan.linkUnits) { + if (lu.kind != kind) continue; + std::vector stack{&lu}; + std::set seen; + while (!stack.empty()) { + const LinkUnit* at = stack.back(); + stack.pop_back(); + for (auto const& input : at->implicitInputs) { + auto it = sharedByOutput.find(input.lexically_normal().generic_string()); + if (it == sharedByOutput.end() || !seen.insert(it->second).second) continue; + if (link_unit_holds_cxx(plan, *it->second)) return true; + stack.push_back(it->second); + } + } + } + return false; +} + namespace { std::string sanitize_for_path(std::string_view module_name) { @@ -1714,6 +1829,10 @@ make_plan(const mcpp::manifest::Manifest& manifest, std::set sharedDepPackages; std::map> sharedTargetsByPackage; std::map> packageIndexByName; + // #646 F1: static packages linked into a dependency's shared image instead + // of the root's, by name and by the shared package that takes them. + std::set placedInImage; + std::map> staticsByImagePackage; for (std::size_t i = 0; i < packages.size(); ++i) { auto const& p = packages[i]; packageIndexByName[qualified_package_name(p.manifest)] = i; @@ -1827,6 +1946,69 @@ make_plan(const mcpp::manifest::Manifest& manifest, } } + // WHERE A STATIC PACKAGE UNDER A SHARED LIBRARY IS LINKED (#646 F1). + // + // A dependency's shared library used to be linked from its own package's + // objects only, and every static package went into the root's images. A + // static package reachable only through the library then lived in the + // program, and the library bound to the program's copy at run time. ELF + // allows that, and only for a consumer that linked the package itself: + // measured on Linux, the same library refused `-Wl,-z,defs`, and a + // program that did not link the package could not `dlopen` it + // (`undefined symbol: x_answer`). Mach-O and PE resolve at link time, and + // Android's Java host loads the library before the application. + // + // So a static package in exactly one shared image's closure, which the + // root does not reach itself, is linked into that image. One that several + // images reach has no single home and is recorded; prepare refuses it + // where the link or load cannot succeed and reports it on ELF, where the + // build proceeds as it always did. The root's edges include its + // dev-dependencies: a test program that links a package directly keeps it. + { + auto placementEdges = directPackageDeps; + for (auto const& [depName, spec] : manifest.devDependencies) { + for (auto const& candidate : dependency_name_candidates(depName, spec)) { + auto it = packageIndexByName.find(candidate); + if (it == packageIndexByName.end() || it->second == 0) continue; + auto& deps = placementEdges[0]; + if (std::ranges::find(deps, it->second) == deps.end()) + deps.push_back(it->second); + break; + } + } + std::map sharedImages; + for (auto const& [i, targets] : sharedTargetsByPackage) + sharedImages[i] = targets.size(); + // Two kinds of package are neither placed nor walked through. A + // distribution package's library is already built. A package that + // provides a target layer (a C library, a compiler runtime, a C++ + // runtime: `provides = ["mcpp:..."]`) is beneath every image, and + // where its objects go is the runtime contract's decision, which + // #641 made (`cxx_runtime = { shared = "self-contained" }`). + std::set boundaries; + for (std::size_t i = 1; i < packages.size(); ++i) { + auto const& pm = packages[i].manifest; + const bool layer = std::ranges::any_of(pm.provides, [](const std::string& p) { + return p.starts_with("mcpp:") || p == "hosted-standard-library"; + }); + if (layer || mcpp::pack::is_distribution_package(pm)) boundaries.insert(i); + } + auto placement = place_static_packages(placementEdges, sharedImages, boundaries); + for (auto const& [staticIndex, sharedIndex] : placement.intoImage) { + placedInImage.insert(qualified_package_name(packages[staticIndex].manifest)); + staticsByImagePackage[sharedIndex].push_back(staticIndex); + } + for (auto const& conflict : placement.conflicts) { + StaticPlacementConflict c; + c.package = qualified_package_name(packages[conflict.package].manifest); + c.program = conflict.root; + for (auto sharedIndex : conflict.images) + for (auto targetIndex : sharedTargetsByPackage[sharedIndex]) + c.images.push_back(sharedDepTargets[targetIndex].target.name); + plan.staticPlacementConflicts.push_back(std::move(c)); + } + } + auto append_direct_shared_deps = [&](LinkUnit& lu, std::size_t packageIndex) { auto depsIt = directPackageDeps.find(packageIndex); if (depsIt == directPackageDeps.end()) return; @@ -1886,6 +2068,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, linkedPackages.insert(0); for (auto& cu : plan.compileUnits) { if (sharedDepPackages.contains(cu.packageName)) continue; + if (placedInImage.contains(cu.packageName)) continue; auto it = packageIndexByName.find(cu.packageName); if (it == packageIndexByName.end()) continue; linkedPackages.insert(it->second); @@ -1927,6 +2110,14 @@ make_plan(const mcpp::manifest::Manifest& manifest, lu.loaderTagFlag = loader_tag_flag(lu.kind); append_package_objects(lu, dep.packageName); append_direct_shared_deps(lu, dep.packageIndex); + if (auto it = staticsByImagePackage.find(dep.packageIndex); + it != staticsByImagePackage.end()) { + for (auto staticIndex : it->second) { + append_package_objects( + lu, qualified_package_name(packages[staticIndex].manifest)); + append_direct_shared_deps(lu, staticIndex); + } + } plan.linkUnits.push_back(std::move(lu)); } @@ -1993,6 +2184,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, // For binary target, also include main.cpp's object if main is present. for (auto& cu : plan.compileUnits) { if (sharedDepPackages.contains(cu.packageName)) continue; + if (placedInImage.contains(cu.packageName)) continue; if (mcpp::links_unconditionally(cu.kind)) { lu.objects.push_back(cu.object); } @@ -2107,6 +2299,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, // is exclusive to that binary). for (auto& cu : plan.compileUnits) { if (sharedDepPackages.contains(cu.packageName)) continue; + if (placedInImage.contains(cu.packageName)) continue; if (!is_implementation_source(cu.kind)) continue; if (lu.entryMain && cu.source == *lu.entryMain) continue; // own entry: already added above if (entryFilesAcrossTargets.contains(cu.source)) continue; // foreign entry: skip diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index d3888aacc..1311da41a 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -647,7 +647,7 @@ std::vector feature_closure(const mcpp::manifest::Manifest& pm, } // --features value → tokens (comma/space separated). -std::vector parse_feature_request(std::string_view s) { +std::vector feature_request_tokens(std::string_view s) { std::vector out; for (std::size_t p = 0; p < s.size();) { auto c = s.find_first_of(", ", p); @@ -659,6 +659,33 @@ std::vector parse_feature_request(std::string_view s) { return out; } +// The root's own features among the --features tokens. +// +// A TOKEN CONTAINING `/` IS NOT A FEATURE OF THE ROOT (#649 E8). It can only +// mean "open this feature of that dependency", which is what the same token +// means inside `[features]`, so it is taken out here and applied as a forward +// of the root (`feature_forward_request`). It used to stay in this list, where +// a root without `[features]` turned it into `-DMCPP_FEATURE_SPIKE_FW_INSTALLER` +// and a root with the table reported it as an undeclared feature; neither +// opened the dependency's feature. +std::vector parse_feature_request(std::string_view s) { + std::vector out; + for (auto& tok : feature_request_tokens(s)) + if (tok.find('/') == std::string::npos) out.push_back(std::move(tok)); + return out; +} + +// The `/` tokens of --features, in the keyspace of a +// `[features]` forward. A token with an empty half is kept whole and named by +// the caller, rather than being dropped as the manifest parser drops it: on a +// command line the user typed it just now. +std::vector feature_forward_request_tokens(std::string_view s) { + std::vector out; + for (auto& tok : feature_request_tokens(s)) + if (tok.find('/') != std::string::npos) out.push_back(std::move(tok)); + return out; +} + bool is_std_module(std::string_view name) { return name == "std" || name == "std.compat"; } @@ -1019,6 +1046,22 @@ export std::string resolve_profile_name(const mcpp::manifest::Manifest& m, return fallback.empty() ? std::string("dev") : std::string(fallback); } +// THE OVERRIDE NAME THE COMMAND LINE STATES, OR "" FOR NONE (#649 E9). +// +// `--profile NAME` > `--release` > `--dev`, and one function for every verb +// that takes the spellings (`build`, `run`, `test`, `emit build-database`, +// `pack`). The rule used to be written twice and the copies disagreed: +// `mcpp build --profile dev --release` built `dev` while `mcpp run` given the +// same line built `release`. Its result is `resolve_profile_name`'s +// `override_name`. +export std::string profile_override_from_flags(std::string_view profileOption, + bool release, bool dev) { + if (!profileOption.empty()) return std::string(profileOption); + if (release) return "release"; + if (dev) return "dev"; + return {}; +} + // Command-level overrides (--target / --static). // Empty defaults preserve pre-existing behaviour exactly. export struct BuildOverrides { @@ -1059,6 +1102,10 @@ export struct BuildOverrides { int tool_depth = 0; // The request chain, for that diagnostic. "root → grpc:grpc_cpp_plugin → …" std::string tool_chain; + // The (package source, tool) pairs being built by the enclosing sub-builds, + // outermost first. A request for one of them is the tool's own build asking + // for itself, refused at its first repetition (#649 E6). + std::vector tool_chain_sources; // Use THIS manifest instead of reading `/mcpp.toml`. // // Required for a `compat`-style registry package (Form B), which ships no @@ -1140,6 +1187,13 @@ export struct BuildOverrides { // reading "this build is not packaging" for a build that plainly is would // be sent looking in the wrong place. std::string pack_stage_reason; + // #649 E5: the strip decision and the debug-symbol directory that pass + // resolved, "1" or "0" and absolute, set only beside `pack_format`. A + // member that stages libraries of its own reads them through + // `mcpp::pack_strip()` and `mcpp::pack_debug_symbols_dir()`, so + // `--no-strip` reaches its files as it reaches the engine's. + std::string pack_strip; + std::filesystem::path pack_debug_symbols_dir; }; // ── git dependency helpers ────────────────────────────────────────────────── @@ -1266,6 +1320,7 @@ mcpp::platform::process::RunResult run_with_network_retry( std::string_view command, const std::function& between = {}) { mcpp::platform::process::RunResult r{}; + mcpp::platform::env::note_network_access(); // the envelope's `effects` (#648 A4) for (int attempt = 1; attempt <= 3; ++attempt) { r = mcpp::platform::process::capture(command); if (r.exit_code == 0) return r; @@ -1560,6 +1615,7 @@ provision_xlings_addresses(const mcpp::config::GlobalConfig& cfg, mcpp::platform::env::offline_mode() ? "drop --offline / unset MCPP_OFFLINE" : "unset MCPP_NO_AUTO_INSTALL"; + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "{} are declared but not provisioned, " "and auto-install is off.\n" @@ -3472,6 +3528,7 @@ prepare_build(bool print_fingerprint, // will actually work there instead. if (mcpp::platform::is_windows && !msvc_usable_either_origin()) { + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "no toolchain configured (and no Visual Studio found).\n" " run one of:\n" @@ -3482,6 +3539,7 @@ prepare_build(bool print_fingerprint, pins::kFirstRunWinGnu, pins::kFirstRunWinGnuTarget, release)); } if constexpr (mcpp::platform::is_macos || mcpp::platform::is_windows) { + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "no toolchain configured.\n" " run one of:\n" @@ -3490,6 +3548,7 @@ prepare_build(bool print_fingerprint, " {}", pins::kSuggestLlvm, pins::kFirstRunMac, release)); } else { + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "no toolchain configured.\n" " run one of:\n" @@ -4650,7 +4709,19 @@ prepare_build(bool print_fingerprint, needsRemoteUpdate = true; break; } - if (needsRemoteUpdate) { + // A first sync is a refresh of an index that has no local + // copy yet, and `[index] auto_refresh = false` means that no + // refresh happens implicitly (docs/05). The policy's answer is + // taken rather than restated (#648 A5); offline, the sync is a + // no-op as before and resolution reports what is missing. + const auto refreshPolicy = mcpp::pm::policy_for(**cfg2); + if (needsRemoteUpdate && !refreshPolicy.offline && !refreshPolicy.autoRefresh) { + return std::unexpected(std::string( + "the project's custom index repositories have never been synced, " + "and [index] auto_refresh = false forbids syncing them implicitly\n" + " run `mcpp index update` once, then build again")); + } + if (needsRemoteUpdate && !refreshPolicy.offline) { mcpp::ui::status("Fetching", "custom index repos (first use)"); auto projEnv = mcpp::config::make_project_xlings_env(**cfg2, *root); int rc = mcpp::xlings::update_index(projEnv, /*quiet=*/true); @@ -4850,6 +4921,48 @@ prepare_build(bool print_fingerprint, std::map identityBySource; std::map declaringManifest; std::set> adoptionsReported; + // A GIT DEPENDENCY NAMES A REPOSITORY, AND THE KEY NAMES WHICH PACKAGE OF + // IT (#649 E7). The root manifest's package is one; each `[workspace] + // members` entry of that manifest is another. Before this a git source + // always yielded the root package, so a repository holding a framework and + // its tools could be pinned by revision for the framework only. The clone + // of every git source resolved so far is kept with the reference it was + // resolved from, so a later key over the same source, and a member's + // `path` edge that stays inside the clone, find it. + struct GitClone { + std::filesystem::path root; + std::string url, refKind, ref; + }; + std::map gitCloneBySource; + // The member of the repository at `cloneRoot` whose manifest declares + // `want`, when the root manifest's package is not `want` itself. + auto gitMemberDeclaring = [&](const std::filesystem::path& cloneRoot, + const ResolvedKey& want) + -> std::optional { + auto declares = [&](const mcpp::manifest::Manifest& mm) { + auto rn = mcpp::pm::compat::resolve_package_name( + mm.package.name, mm.package.namespace_); + // A manifest that names no namespace takes the key's, as a path + // dependency's does (#634 A2), so only the short name is compared. + const bool nsDeclared = !mm.package.namespace_.empty() || rn.usedLegacySplit; + return rn.shortName == want.shortName + && (!nsDeclared || rn.namespace_ == want.ns); + }; + std::error_code ec; + if (!std::filesystem::exists(cloneRoot / "mcpp.toml", ec)) return std::nullopt; + auto rootManifest = mcpp::manifest::load(cloneRoot / "mcpp.toml"); + if (!rootManifest || declares(*rootManifest) + || !rootManifest->workspace.present) + return std::nullopt; + for (auto const& member : rootManifest->workspace.members) { + const auto path = cloneRoot / member / "mcpp.toml"; + if (!std::filesystem::exists(path, ec)) continue; + auto mm = mcpp::manifest::load(path, {.insideWorkspace = true}); + if (mm && declares(*mm)) + return std::filesystem::path(member).lexically_normal().generic_string(); + } + return std::nullopt; + }; auto qualifiedKey = [](const ResolvedKey& k) { return k.ns.empty() ? k.shortName : std::format("{}.{}", k.ns, k.shortName); }; @@ -5238,6 +5351,18 @@ prepare_build(bool print_fingerprint, mcpp::pm::staleness_note( mcpp::config::make_xlings_env(**cfgA))); } + // Offline with no local copy of the index at all, the miss says + // nothing about the package: nothing has been downloaded to look + // in. That is a download the run needs, not a wrong selector. + if (mcpp::platform::env::offline_mode()) { + if (auto cfgO = get_cfg(); + cfgO && !mcpp::xlings::default_index_status( + mcpp::config::make_xlings_env(**cfgO), 0).present) { + hint += "\n offline: the package index has never been fetched; " + "run `mcpp index update` without --offline"; + refusal::record(refusal::Code::OfflineDownloadRequired); + } + } return std::unexpected(with_index_cause(std::format( "dependency '{}': no package found for exact selector" "\n tried: {}{}", @@ -5466,6 +5591,7 @@ prepare_build(bool print_fingerprint, // down. (The toolchain payload path has its own gate; this is the // dependency path, which does not go through resolve_xpkg_path.) if (mcpp::platform::env::offline_mode()) { + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "offline mode: dependency '{}' v{} is not installed and " "cannot be downloaded\n" @@ -5777,9 +5903,6 @@ prepare_build(bool print_fingerprint, std::string table; // `DependencySpec::declaredIn` }; std::vector graphRequests; - // The link form each dependency took, and why (`linkage_form::Resolution`), - // by package index. - std::map> graphLinkForms; // The link form each dependency takes and the facts it was decided from, // by package index. COMPUTED ONCE, before the root build program runs, so // that program can read the answer (#642 E2); APPLIED after the scan, where @@ -5847,7 +5970,42 @@ prepare_build(bool print_fingerprint, } return prov::bind_bare_names(fqns); }; + // THE NAMES UNDER WHICH ONE PROVIDER IS PUBLISHED TO ONE CONSUMER, derived + // once for every channel (#647 E4.3). The manifest's `name`, the qualified + // `namespace.name` when the manifest writes the two apart, and the bare + // tail where the namespace ladder binds it to this provider for this + // consumer. `dep_dir`/`dep_linkage` and `dep_bin` used to derive this list + // separately; #642 added the qualified spelling to the first and the second + // kept publishing `MCPP_DEP_INSTALLER_BIN_*` alone for a package written + // `namespace = "spike"`, `name = "installer"`, so + // `dep_bin("spike.installer", ...)` read nothing. + auto publishedNamesFor = + [&](std::size_t provider, + const std::map& bind) { + std::vector out; + auto const& manifest = packages[provider].manifest; + auto const& canon = manifest.package.name; + out.push_back(canon); + if (auto qualified = mcpp::build::qualified_package_name(manifest); + qualified != canon) + out.push_back(std::move(qualified)); + if (auto tail = prov::tail_of(canon); tail != canon) { + auto it = bind.find(tail); + if (it != bind.end() && it->second.owner == canon) + out.push_back(std::move(tail)); + } + return out; + }; + // A package whose DECLARED targets are all programs (#649 E6). See the + // worklist, where such a package is not walked into a consumer's graph. + auto isProgramOnlyPackage = [](const mcpp::manifest::Manifest& pm) { + if (pm.targetsInferred || pm.targets.empty()) return false; + return std::ranges::none_of(pm.targets, [](const mcpp::manifest::Target& t) { + return t.kind == mcpp::manifest::Target::Library + || t.kind == mcpp::manifest::Target::SharedLibrary; + }); + }; auto parseVisibility = [](std::string_view visibility) { if (visibility == "private") return mcpp::modgraph::DependencyVisibility::Private; @@ -6030,25 +6188,13 @@ prepare_build(bool print_fingerprint, if (linkForms) if (auto f = linkForms->find(pr.provider); f != linkForms->end()) form = &f->second; - e.depDirs.emplace_back(canon, depPkg.root); - if (form) e.depLinkages.emplace_back(canon, *form); - // AND THE SPELLING THE CONSUMER'S MANIFEST USES. A package that - // writes `namespace = "ns"` and `name = "fw"` is a dependency named - // `ns.fw`, and `dep_dir("ns.fw")` read nothing while the reference - // above promised the canonical spelling (#642: the framework's rule - // asks `dep_linkage("huxerui.huxerui")`). The qualified name is - // unique in the graph, so it needs no binding check. - if (auto qualified = mcpp::build::qualified_package_name(depPkg.manifest); - qualified != canon) { - e.depDirs.emplace_back(qualified, depPkg.root); - if (form) e.depLinkages.emplace_back(qualified, *form); - } - auto tail = prov::tail_of(canon); - if (tail == canon) continue; - auto it = bind.find(tail); - if (it != bind.end() && it->second.owner == canon) { - e.depDirs.emplace_back(tail, depPkg.root); - if (form) e.depLinkages.emplace_back(tail, *form); + // Every spelling of `publishedNamesFor`: the manifest's name, the + // qualified name a manifest writing `namespace = "ns"` and + // `name = "fw"` is addressed by (#642: the framework's rule asks + // `dep_linkage("huxerui.huxerui")`), and the bound tail. + for (auto const& n : publishedNamesFor(pr.provider, bind)) { + e.depDirs.emplace_back(n, depPkg.root); + if (form) e.depLinkages.emplace_back(n, *form); } } }; @@ -6309,6 +6455,30 @@ prepare_build(bool print_fingerprint, // way round would let a `[build-dependencies]` line quietly drop a // library the target needs. if (!buildOnly) it->buildOnly = false; + // AND THE SECOND DECLARATION'S REQUESTS ARE KEPT (#649 E7). Both + // declarations name one edge, so what each asks of the dependency + // is asked of that edge: this used to return here and lose the + // second one's `tools`, `features`, `host-module` and `reexport` + // without a word, under `--strict` too. The rule is the one + // `mergeActiveFeatureDeps` already applies to a feature's + // restatement: additive fields union, `default-features` stays on + // unless every declaration opts out. + for (auto const& t : spec.tools) + if (std::ranges::find(it->requestedTools, t) == it->requestedTools.end()) + it->requestedTools.push_back(t); + for (auto const& f : spec.features) + if (std::ranges::find(it->requestedFeatures, f) + == it->requestedFeatures.end()) + it->requestedFeatures.push_back(f); + it->defaultFeatures = it->defaultFeatures || spec.defaultFeatures; + if (spec.hostModule) it->hostModule = true; + else if (dependencyPackageIndex < packages.size()) + for (auto const& f : spec.features) + if (packages[dependencyPackageIndex].manifest.featureRuleModule.contains(f)) { + it->hostModule = true; + break; + } + it->reexport = it->reexport || spec.reexport; return; } // A REQUESTED FEATURE THAT IS A BUILD RULE IMPLIES `host-module`. @@ -6349,6 +6519,10 @@ prepare_build(bool print_fingerprint, } auto& consumer = packages[edge.consumerPackageIndex]; auto const& dependency = packages[edge.dependencyPackageIndex]; + // A package of programs publishes no usage requirements to its + // consumers (#649 E6): nothing of it is compiled or linked here. + if (edge.dependencyPackageIndex > 0 + && isProgramOnlyPackage(dependency.manifest)) continue; if (edge.visibility == mcpp::modgraph::DependencyVisibility::Private || edge.visibility == mcpp::modgraph::DependencyVisibility::Public) { @@ -6599,16 +6773,52 @@ prepare_build(bool print_fingerprint, // when that feature is active. `seedDefault` carries consumer-side // `default-features = false` (#242): when a consumer opts out of this dep's // default set, feature-deps behind the default pseudo-feature stay dormant. + // + // A RESTATEMENT NAMES THE SAME SOURCE OR IS REFUSED (#647 E4.2). The grammar + // asks a `[feature-deps]` entry to restate its dependency's source, and the + // merge below takes only the additive fields from it, so a restatement + // that names another path, repository or version was dropped without a + // word, under `--strict` too: the tool came from the declaration in effect + // while the manifest said it came from somewhere else. The comparison runs + // against `dependencies` after the conditional fold, so a row's replacement + // (#634 A1) is the declaration a restatement is held to. + auto dependencySourceOf = [](const mcpp::manifest::DependencySpec& s) { + if (s.inheritWorkspace) return std::string("workspace = true"); + if (s.isPath()) { + auto norm = std::filesystem::path(s.path).lexically_normal().generic_string(); + while (norm.size() > 1 && norm.back() == '/') norm.pop_back(); + return std::format("path = \"{}\"", norm); + } + if (s.isGit()) + return std::format("git = \"{}\", {} = \"{}\"", s.git, + s.gitRefKind.empty() ? "rev" : s.gitRefKind, s.gitRev); + return std::format("version = \"{}\"", s.version); + }; auto mergeActiveFeatureDeps = [&](mcpp::manifest::Manifest& pm, const std::vector& requested, - bool seedDefault = true) { - if (pm.featureDeps.empty()) return; + bool seedDefault = true) + -> std::expected { + if (pm.featureDeps.empty()) return {}; for (auto& f : activateFeatures(pm, requested, seedDefault)) { auto it = pm.featureDeps.find(f); if (it == pm.featureDeps.end()) continue; for (auto& [k, spec] : it->second) { auto [pos, fresh] = pm.dependencies.try_emplace(k, spec); if (fresh) continue; + if (!pos->second.inheritWorkspace && !spec.inheritWorkspace) { + const auto inEffect = dependencySourceOf(pos->second); + const auto restated = dependencySourceOf(spec); + if (inEffect != restated) + return std::unexpected(std::format( + "[feature-deps.{}] of '{}' restates the dependency '{}' " + "with {}, while the declaration in effect on this row " + "names {}.\n" + " One dependency has one source, so the " + "restatement would be ignored.\n" + " fix: restate the same source ({}), or declare " + "'{}' only under the feature.", + f, pm.package.name, k, restated, inEffect, inEffect, k)); + } // #359: the key already exists unconditionally, and dropping // the feature's spec here loses REQUESTS the feature exists to // make. gRPC is the shape: it depends on compat.protobuf @@ -6635,6 +6845,7 @@ prepare_build(bool print_fingerprint, dst.reexport = dst.reexport || spec.reexport; } } + return {}; }; // #243: dep/feat forwarding. When a resolved package's feature F is active, @@ -6662,10 +6873,36 @@ prepare_build(bool print_fingerprint, } }; // #243: a forward whose active feature targets a dependency that is not - // declared (in [dependencies], [dev-dependencies], or an active - // [feature-deps] already folded into `dependencies`) is a manifest bug — - // name it instead of silently dropping. Only active features' forwards are - // checked (lazy, like the unknown-requested-feature gate at ~2875). + // declared is a manifest bug — name it instead of silently dropping. Only + // active features' forwards are checked (lazy, like the + // unknown-requested-feature gate at ~2875). + // + // THE VALIDATOR ASKS WHAT THE FORWARD LANGUAGE DEFINES: IS THE KEY DECLARED + // IN ANY DEPENDENCY TABLE OF THIS MANIFEST, ON ANY ROW, UNDER ANY FEATURE + // (#647 E4.1). It used to look in `dependencies` and `devDependencies` + // only, while `injectForwards` applies a forward to the build-dependency + // edge as well, so a forward along `[build-dependencies]` was applied and + // reported as undeclared in the same run, and `--strict` refused a build + // whose forward had worked. A key declared only for another row, or only + // under an inactive feature, is declared: on this row the forward reaches + // no edge and does nothing, which is what a portable manifest means by it. + auto declaresDependencyKey = [](const mcpp::manifest::Manifest& pm, + const std::string& key) { + auto inFeatureDeps = [&](const auto& byFeature) { + for (auto const& [f, deps] : byFeature) + if (deps.contains(key)) return true; + return false; + }; + if (pm.dependencies.contains(key) || pm.devDependencies.contains(key) + || pm.buildDependencies.contains(key) || inFeatureDeps(pm.featureDeps)) + return true; + for (auto const& cc : pm.conditionalConfigs) + if (cc.dependencies.contains(key) || cc.devDependencies.contains(key) + || cc.buildDependencies.contains(key) + || inFeatureDeps(cc.featureDeps)) + return true; + return false; + }; auto validateForwards = [&](const mcpp::manifest::Manifest& parent, const std::vector& parentActive, std::string_view parentName) @@ -6674,12 +6911,12 @@ prepare_build(bool print_fingerprint, auto it = parent.featureForwards.find(f); if (it == parent.featureForwards.end()) continue; for (auto const& [depKey, depFeat] : it->second) { - if (parent.dependencies.contains(depKey) - || parent.devDependencies.contains(depKey)) continue; + if (declaresDependencyKey(parent, depKey)) continue; auto msg = std::format( "feature '{}' of '{}' forwards to dependency '{}' (as " - "'{}/{}') which is not declared in [dependencies] or " - "[feature-deps]", f, parentName, depKey, depKey, depFeat); + "'{}/{}') which no dependency table declares ([dependencies], " + "[build-dependencies], [dev-dependencies] or [feature-deps], " + "on any row)", f, parentName, depKey, depKey, depFeat); if (overrides.strict) return std::unexpected(msg); mcpp::diag::warning("features/forwarding", msg); } @@ -6690,25 +6927,60 @@ prepare_build(bool print_fingerprint, // Pull the root package's active feature-deps into its dependency set before // seeding, so `mcpp build --features X` resolves X's optional deps. std::vector rootReq = parse_feature_request(overrides.features); - mergeActiveFeatureDeps(*m, rootReq); + if (auto fm = mergeActiveFeatureDeps(*m, rootReq); !fm) + return std::unexpected(fm.error()); // #243: the root's active features may forward features to its direct deps. std::vector rootActive = feature_closure(*m, rootReq, true); if (auto fe = validateForwards(*m, rootActive, m->package.name); !fe) return std::unexpected(fe.error()); activeFeaturesByPackage.assign(1, rootActive); + // `--features /` (#649 E8): a forward of the root, + // applied to the edges exactly as a `[features]` forward is and checked + // against the same tables. Named whether or not the root declares + // `[features]`: the token cannot be a macro of the root, so there is no + // "pure macro usage" to preserve for it. + std::vector> cliForwards; + for (auto const& tok : feature_forward_request_tokens(overrides.features)) { + auto fwd = mcpp::pm::split_feature_forward_token(tok); + std::string msg; + if (!fwd) + msg = std::format("--features requests '{}', which names neither a " + "feature nor `/`", tok); + else if (!declaresDependencyKey(*m, fwd->first)) + msg = std::format("--features requests '{}', and no dependency table " + "of '{}' declares '{}'", tok, m->package.name, + fwd->first); + if (!msg.empty()) { + if (overrides.strict) return std::unexpected(msg); + mcpp::diag::warning("features/request", msg); + continue; + } + cliForwards.push_back(std::move(*fwd)); + } + auto injectCliForwards = [&](const std::string& childKey, + mcpp::manifest::DependencySpec& childSpec) { + for (auto const& [depKey, depFeat] : cliForwards) + if (depKey == childKey + && std::ranges::find(childSpec.features, depFeat) + == childSpec.features.end()) + childSpec.features.push_back(depFeat); + }; + // Seed the worklist from the main manifest. Dev-deps only when the // caller wants them; they're never propagated transitively. const std::string mainPkgLabel = m->package.name; for (auto& [n, s] : m->dependencies) { auto req = s; injectForwards(*m, rootActive, n, req); + injectCliForwards(n, req); worklist.push_back({n, req, mainPkgLabel, req.version, kMainConsumer, {}}); } if (includeDevDeps) { for (auto& [n, s] : m->devDependencies) { auto req = s; injectForwards(*m, rootActive, n, req); + injectCliForwards(n, req); worklist.push_back({n, req, mainPkgLabel + " (dev-dep)", req.version, kMainConsumer, {}, /*devOnly=*/true}); } @@ -6722,6 +6994,7 @@ prepare_build(bool print_fingerprint, for (auto& [n, s] : m->buildDependencies) { auto req = s; injectForwards(*m, rootActive, n, req); + injectCliForwards(n, req); worklist.push_back({n, req, mainPkgLabel + " (build-dep)", req.version, kMainConsumer, {}, /*devOnly=*/false, /*buildOnly=*/true}); @@ -6779,6 +7052,41 @@ prepare_build(bool print_fingerprint, } } + // A `path` edge that stays inside a git clone this graph already + // resolved names the same git source at the same commit (#649 E7): + // a member's `spike.fw = { path = ".." }` reaches the repository the + // application pinned by revision, and is that package rather than a + // second, path-sourced declaration of it. Only the clone root itself + // and its `[workspace] members` are mapped; any other directory keeps + // being an ordinary path. + if (spec.isPath() && !gitCloneBySource.empty()) { + std::filesystem::path p = spec.path; + auto base = item.resolveRoot.empty() ? *root : item.resolveRoot; + if (p.is_relative()) p = base / p; + std::error_code ec; + auto canon = std::filesystem::weakly_canonical(p, ec); + if (ec) canon = p.lexically_normal(); + for (auto const& [src, clone] : gitCloneBySource) { + auto rel = canon.lexically_relative(clone.root).generic_string(); + if (rel.empty() || rel.starts_with("..")) continue; + bool mapped = rel == "."; + if (!mapped) { + if (auto rm = mcpp::manifest::load(clone.root / "mcpp.toml"); + rm && rm->workspace.present) + for (auto const& member : rm->workspace.members) + if (std::filesystem::path(member).lexically_normal() + .generic_string() == rel) + mapped = true; + } + if (!mapped) continue; + spec.path.clear(); + spec.git = clone.url; + spec.gitRefKind = clone.refKind; + spec.gitRev = clone.ref; + break; + } + } + // Pin SemVer constraint before dedup/fetch. if (auto r = resolveSemver(spec, name); !r) { return std::unexpected(r.error()); @@ -6795,14 +7103,26 @@ prepare_build(bool print_fingerprint, // The commit a `git` dependency resolved to, carried out of the clone // branch below for the cache identity. std::string sourceCommit; + // The repository member a `git` dependency selected; empty for the + // repository's root package (#649 E7). + std::string gitMember; + std::filesystem::path gitMemberCloneRoot; // A second key over a source that is already resolved takes the // identity resolved there; its manifest is not loaded again. if (sourceKind != "version") { const auto source = sourceRefOf(sourceKind, spec, item.resolveRoot, item.originalConstraint); + // A key naming another package of the same repository is that + // member, not a second key over the root's identity (#649 E7). + bool namesMember = false; + if (sourceKind == "git") + if (auto clone = gitCloneBySource.find(source); + clone != gitCloneBySource.end()) + namesMember = gitMemberDeclaring(clone->second.root, key).has_value(); if (auto bySource = identityBySource.find(source); - bySource != identityBySource.end() && !(bySource->second == key)) { + !namesMember && bySource != identityBySource.end() + && !(bySource->second == key)) { const auto& existing = resolved.at(bySource->second); const auto& declaring = declaringManifest.at(bySource->second); if (!declaring.namespaceDeclared) { @@ -7311,6 +7631,7 @@ prepare_build(bool print_fingerprint, auto refuse_offline = [&](std::string_view need, std::string_view why, std::string_view verb) { + refusal::record(refusal::Code::OfflineDownloadRequired); return std::unexpected(std::format( "offline mode: git dependency '{}' needs {} of '{}'\n" " {}\n" @@ -7449,6 +7770,14 @@ prepare_build(bool print_fingerprint, } sourceCommit = resolvedGitRev; dep_root = gitRoot; + gitCloneBySource.try_emplace( + sourceRefOf("git", spec, item.resolveRoot, item.originalConstraint), + GitClone{ gitRoot, spec.git, spec.gitRefKind, spec.gitRev }); + if (auto member = gitMemberDeclaring(gitRoot, key)) { + gitMember = *member; + gitMemberCloneRoot = gitRoot; + dep_root = gitRoot / *member; + } } // (version-source: dep_root + manifest are loaded together via // loadVersionDep below since the index entry drives both.) @@ -7484,7 +7813,7 @@ prepare_build(bool print_fingerprint, *wsManifest, runtimeWorkspaceRoot, dep_root); auto dm = mcpp::manifest::load( dep_root / "mcpp.toml", - {.insideWorkspace = depIsMember}); + {.insideWorkspace = depIsMember || !gitMember.empty()}); if (!dm) { return std::unexpected(std::format( "dependency '{}' (at '{}'): {}", @@ -7501,6 +7830,16 @@ prepare_build(bool print_fingerprint, if (auto bad = mcpp::project::workspace_inheritance_error( *dep_manifest, dep_root)) return std::unexpected(*bad); + } else if (!gitMember.empty()) { + // A repository member inherits its repository's + // `[workspace.package]`, as it does when the repository is + // built from its own checkout. + if (auto rm = mcpp::manifest::load(gitMemberCloneRoot / "mcpp.toml")) { + mcpp::project::inherit_workspace_package(*dep_manifest, *rm); + if (auto bad = mcpp::project::workspace_inheritance_error( + *dep_manifest, dep_root)) + return std::unexpected(*bad); + } } // #229: path/git-dep half of the L1 cfg funnel — mirrors the // loadVersionDep call site above (loadFrom's L1 cfg merge, ~1740 @@ -7608,9 +7947,22 @@ prepare_build(bool print_fingerprint, // dependency set before its children are pushed, so a dep's feature can // transitively pull a provider. `spec.features` = features the consumer // requested for this dep. - mergeActiveFeatureDeps(*dep_manifest, spec.features, spec.defaultFeatures); - - auto linkFlagsAdded = propagateLinkFlags(dep_root, *dep_manifest); + if (auto fm = mergeActiveFeatureDeps(*dep_manifest, spec.features, + spec.defaultFeatures); !fm) + return std::unexpected(fm.error()); + + // A PACKAGE OF PROGRAMS HAS NOTHING TO LINK (#649 E6). Its tools are + // built by the tool sub-build, which resolves the package as its own + // root; in this graph it is a provider of tools and of its directory, + // and nothing more. Walking its dependencies here put a tool's own + // library into the application's link (a tool depending on `z` gave the + // application `z.o`), compiled its sources in the consumer's build, and + // made a tool that depends on the package declaring it a cycle of the + // consumer's graph although the two builds never meet. + const bool depProgramOnly = isProgramOnlyPackage(*dep_manifest); + auto linkFlagsAdded = depProgramOnly + ? std::vector{} + : propagateLinkFlags(dep_root, *dep_manifest); // Move the manifest into stable storage so we can later look it up // by depIndex (the SemVer merger needs to overwrite the slot). @@ -7636,7 +7988,8 @@ prepare_build(bool print_fingerprint, // (ns, name) hit the fast path (skip / merge / conflict). if (sourceKind != "version") { identityBySource.emplace( - sourceRefOf(sourceKind, spec, item.resolveRoot, item.originalConstraint), + sourceRefOf(sourceKind, spec, item.resolveRoot, item.originalConstraint) + + (gitMember.empty() ? std::string{} : "#member=" + gitMember), key); declaringManifest[key] = DeclaringManifest{ manifestPath, namespaceDeclared }; } @@ -7656,6 +8009,8 @@ prepare_build(bool print_fingerprint, // Recurse: the dep's own [dependencies] become new worklist items. // dev-dependencies are intentionally NOT walked — those are // private to the dep's test runs, not part of its public ABI. + // A package of programs is not walked at all; see `depProgramOnly`. + if (depProgramOnly) continue; const std::string thisDepLabel = std::format( "{}{}{}@{}", key.ns, @@ -7702,6 +8057,52 @@ prepare_build(bool print_fingerprint, } } + // ONE PLACE DETECTS A CYCLE OF PACKAGES, AND IT IS HERE, WHERE THE GRAPH + // IS RESOLVED (#649 E6). The build-cache key walk was the only reader that + // noticed, and it runs for the global cache only, so the same manifest was + // refused by default and built under `--cache=local`. Every edge counts, + // build-only ones included, as the key walk counts them. + { + std::vector state(packages.size(), 0); // 0 new / 1 on stack / 2 done + std::vector stack, cycle; + auto visit = [&](auto&& self, std::size_t u) -> bool { + state[u] = 1; + stack.push_back(u); + for (auto const& e : dependencyEdges) { + if (e.consumerPackageIndex != u) continue; + const auto v = e.dependencyPackageIndex; + if (v >= packages.size()) continue; + if (state[v] == 1) { + cycle.assign(std::ranges::find(stack, v), stack.end()); + cycle.push_back(v); + return true; + } + if (state[v] == 0 && self(self, v)) return true; + } + stack.pop_back(); + state[u] = 2; + return false; + }; + for (std::size_t i = 0; i < packages.size() && cycle.empty(); ++i) + if (state[i] == 0) (void)visit(visit, i); + if (!cycle.empty()) { + std::string path; + for (auto p : cycle) { + if (!path.empty()) path += " -> "; + path += std::format("'{}'", + mcpp::build::qualified_package_name(packages[p].manifest)); + } + refusal::record(refusal::Code::PackageCycle); + return std::unexpected(std::format( + "dependency cycle: {}.\n" + " A package cannot reach itself through its own dependencies.\n" + " fix: remove one of these edges. A program that depends on the " + "package requesting it builds without a cycle when its package " + "declares only `kind = \"bin\"` targets: it is then built by its " + "own tool sub-build.", path)); + } + } + computeUsageRequirements(); // ─── The toolchain, resolved now that the graph exists ────────────────── @@ -9269,26 +9670,25 @@ prepare_build(bool print_fingerprint, binList.empty() ? std::string("none") : binList)); } - auto var = mcpp::build::tool_store::env_var_name(depName, toolName); - auto varShort = - mcpp::build::tool_store::env_var_name(depShort, toolName); - // #359: every consumer that can SEE this tool gets it, not // just the one whose edge asked for it. The bare spelling // is emitted only where the namespace ladder binds the tail // to this package — otherwise two libraries re-exporting a // same-tailed tool would decide the winner by append order. + // The spellings are `publishedNamesFor`'s, so a tool is + // addressed by exactly the names its directory is. const prov::Provision want{ prov::Kind::Tool, depIdx, toolName }; auto record = [&](const std::filesystem::path& p) { for (std::size_t c = 0; c < provisionGraph.visible.size(); ++c) { if (!provisionGraph.visible[c].contains(want)) continue; auto& v = toolEnvByConsumer[c]; - v.emplace_back(var, p.string()); - if (varShort == var) continue; - auto bind = bareBindingsFor(c); - auto it = bind.find(depShort); - if (it != bind.end() && it->second.owner == depName) - v.emplace_back(varShort, p.string()); + std::vector vars; + for (auto const& n : publishedNamesFor(depIdx, bareBindingsFor(c))) { + auto var = mcpp::build::tool_store::env_var_name(n, toolName); + if (std::ranges::find(vars, var) != vars.end()) continue; + vars.push_back(var); + v.emplace_back(std::move(var), p.string()); + } } }; @@ -9309,6 +9709,42 @@ prepare_build(bool print_fingerprint, continue; } + // A TOOL WHOSE OWN BUILD REQUESTS IT AGAIN IS REFUSED AT + // THE FIRST REPETITION (#649 E6). The depth bound below + // caught it only after four nested sub-builds, with the + // same prefix repeated four times and no word about which + // edge asked. The edge is the one whose request reached + // this package in THIS graph. + const std::string toolSource = std::format( + "{}|{}", depPkg.root.lexically_normal().generic_string(), toolName); + if (std::ranges::find(overrides.tool_chain_sources, toolSource) + != overrides.tool_chain_sources.end()) { + std::string askedBy; + for (auto const& edge : dependencyEdges) { + if (edge.dependencyPackageIndex != depIdx) continue; + if (std::ranges::find(edge.requestedTools, toolName) + == edge.requestedTools.end()) continue; + if (edge.consumerPackageIndex < packages.size()) { + askedBy = mcpp::build::qualified_package_name( + packages[edge.consumerPackageIndex].manifest); + break; + } + } + return std::unexpected(std::format( + "the host tool '{}:{}' is requested by its own build: " + "{} -> {}:{}.\n" + " The request comes from '{}', which the tool's " + "sub-build resolves with the feature or dependency that " + "asks for the tool.\n" + " fix: the tool's own graph must not activate " + "that request (a feature it does not enable, or a " + "`[target..feature-deps]` row it does not match).", + depName, toolName, + overrides.tool_chain.empty() ? "root" : overrides.tool_chain, + depName, toolName, + askedBy.empty() ? std::string("a package of its graph") : askedBy)); + } + // Build it. The feature set is the tool package's own // defaults PLUS the target's required_features — in a tool // sub-build the target is what was ASKED FOR, so its @@ -9411,6 +9847,8 @@ prepare_build(bool print_fingerprint, sub.profile = "release"; sub.cache_mode = overrides.cache_mode; sub.tool_depth = overrides.tool_depth + 1; + sub.tool_chain_sources = overrides.tool_chain_sources; + sub.tool_chain_sources.push_back(toolSource); // The PRISTINE manifest the resolver produced for this // package — `packages[depIdx].manifest` is a copy that // feature activation has already mutated, and re-activating @@ -9558,6 +9996,9 @@ prepare_build(bool print_fingerprint, // written to. for (std::size_t i = 1; i < packages.size(); ++i) { auto& pkg = packages[i]; + // A package of programs runs its build program in its own tool + // sub-build, where its sources are compiled (#649 E6). + if (isProgramOnlyPackage(pkg.manifest)) continue; std::error_code bpEc; if (!std::filesystem::exists(pkg.root / "build.mcpp", bpEc) && pkg.manifest.buildConfig.ruleModules.empty()) continue; @@ -9589,6 +10030,8 @@ prepare_build(bool print_fingerprint, fill_package_build_env(bpEnv, pkg.manifest); bpEnv.packFormat = overrides.pack_format; bpEnv.packStageDir = overrides.pack_stage_dir; + bpEnv.packStrip = overrides.pack_strip; + bpEnv.packDebugSymbolsDir = overrides.pack_debug_symbols_dir; bpEnv.languageModules = pkg.manifest.language.modules; bpEnv.ruleModules = pkg.manifest.buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(pkg.root.string()); dit != deviceSourcesByPackage.end()) @@ -10707,6 +11150,79 @@ prepare_build(bool print_fingerprint, } } + // ── The resolved graph, one derivation for two readers (#634 X, #647 E1) ── + // + // `resolution.json`'s `graph` section and the document the root build + // program reads (`mcpp::graph_file()`) describe the same packages, and they + // are built by this one function so they cannot disagree. Each entry holds + // the package's identity, every request that reached it (the key as + // written and the table that declared it) and, for a library, its link + // form with the reason. The build program's entries add what a program + // needs to act on a package: its manifest directory, the features it is + // built with, its targets, and its `[package.metadata]` verbatim. + // + // The link form is read from `dependencyLinkForms`, which is computed once, + // before this point, for exactly this program (#642 E2). + auto graph_package_entry = [&](std::size_t i, bool forBuildProgram) { + auto const& pm = packages[i].manifest; + const auto id = mcpp::manifest::package_id(pm.package); + nlohmann::json entry = { + {"package", { + {"canonical", id.canonical()}, + {"namespace", id.namespace_}, + {"name", id.name}, + {"version", id.version}, + {"source", id.sourceProvenance}, + }}, + {"root", i == 0}, + }; + nlohmann::json requests = nlohmann::json::array(); + for (auto const& r : graphRequests) { + if (r.dependencyPackageIndex != i) continue; + requests.push_back({ + {"requester", mcpp::manifest::package_id( + packages[r.consumerPackageIndex].manifest.package).canonical()}, + {"key", r.key}, + {"table", r.table}, + }); + } + entry["requested_by"] = std::move(requests); + if (auto form = dependencyLinkForms.find(i); + form != dependencyLinkForms.end() && form->second.recorded) + entry["link"] = { + {"form", std::string(mcpp::build::linkage_form::to_string( + form->second.answer.linkage))}, + {"reason", form->second.answer.reason}, + }; + if (!forBuildProgram) return entry; + + std::error_code ec; + auto dir = std::filesystem::absolute(packages[i].root, ec).lexically_normal(); + entry["manifest_dir"] = dir.string(); + nlohmann::json feats = nlohmann::json::array(); + if (i < activeFeaturesByPackage.size()) + for (auto const& f : activeFeaturesByPackage[i]) feats.push_back(f); + entry["features"] = std::move(feats); + nlohmann::json targets = nlohmann::json::array(); + for (auto const& t : pm.targets) { + using K = mcpp::manifest::Target::Kind; + const std::string_view kind = + t.kind == K::Library ? "lib" + : t.kind == K::Binary ? "bin" + : t.kind == K::SharedLibrary ? "shared" + : t.kind == K::TestBinary ? "test" + : "app"; + targets.push_back({{"name", t.name}, {"kind", std::string(kind)}}); + } + entry["targets"] = std::move(targets); + entry["metadata"] = pm.packageMetadataJson.empty() + ? nlohmann::json::object() + : nlohmann::json::parse(pm.packageMetadataJson, nullptr, + /*allow_exceptions=*/false); + if (entry["metadata"].is_discarded()) entry["metadata"] = nlohmann::json::object(); + return entry; + }; + // ── L3: ROOT build.mcpp (moved after dependency resolution, design §3.1 // item 4) ──────────────────────────────────────────────────────────────── // Runs HERE — after dep resolution + feature activation (so the contract @@ -10789,6 +11305,68 @@ prepare_build(bool print_fingerprint, bpEnv.hostModules = hostModulesByConsumer.count(0u) ? hostModulesByConsumer.at(0u) : decltype(bpEnv.hostModules){}; + // #649 E5: the packaging pass's strip decision, beside its format. + bpEnv.packStrip = overrides.pack_strip; + bpEnv.packDebugSymbolsDir = overrides.pack_debug_symbols_dir; + // #647 E1: THE RESOLVED GRAPH, FOR THE ROOT'S PROGRAM ONLY. + // + // Every package, dependencies before the packages that request them + // (ties in discovery order), so a program that merges what libraries + // contribute can apply them in override order without a sort of its + // own. The root decides the graph, and every input of that decision is + // final here -- the same reason `dep_linkage` is offered to this + // program alone. A file, not variables: a graph with metadata does not + // fit an environment block (`MAX_ARG_STRLEN`, the Windows limit). + // + // ITS DIGEST JOINS THE RE-RUN KEY. Editing a dependency's + // `[package.metadata]` changes what this program would answer, so it + // must run again; editing that dependency's sources does not, and the + // document does not change. + { + std::vector order; + std::vector placed(packages.size(), false); + while (order.size() < packages.size()) { + std::size_t pick = packages.size(); + for (std::size_t i = 0; i < packages.size() && pick == packages.size(); ++i) { + if (placed[i]) continue; + bool ready = true; + for (auto const& r : graphRequests) + if (r.consumerPackageIndex == i && r.dependencyPackageIndex != i + && r.dependencyPackageIndex < packages.size() + && !placed[r.dependencyPackageIndex]) { ready = false; break; } + if (ready) pick = i; + } + // A cycle leaves nothing ready; its first member in discovery + // order is taken so the document is still complete. + if (pick == packages.size()) + for (std::size_t i = 0; i < packages.size(); ++i) + if (!placed[i]) { pick = i; break; } + placed[pick] = true; + order.push_back(pick); + } + nlohmann::json doc; + doc["kind"] = "mcpp.graph"; + doc["version"] = 1; + nlohmann::json list = nlohmann::json::array(); + for (auto i : order) list.push_back(graph_package_entry(i, true)); + doc["packages"] = std::move(list); + const auto text = doc.dump(2) + "\n"; + const auto graphPath = bpEnv.artifactsDir / "graph.json"; + std::error_code gec; + std::filesystem::create_directories(graphPath.parent_path(), gec); + const auto tmp = graphPath.string() + ".tmp"; + { + std::ofstream out(tmp, std::ios::binary | std::ios::trunc); + out << text; + } + std::filesystem::rename(tmp, graphPath, gec); + if (gec) + return std::unexpected(std::format( + "cannot write the graph document '{}': {}", + graphPath.string(), gec.message())); + bpEnv.graphFile = graphPath; + bpEnv.graphDigest = mcpp::toolchain::hash_string(text); + } auto& bcRoot = m->buildConfig; const auto mark = markDirectiveTail(*m); const auto rldN = bcRoot.ldflags.size(), rsrcN = bcRoot.sources.size(), @@ -11032,16 +11610,25 @@ prepare_build(bool print_fingerprint, // Modgraph: regex scanner by default; opt-in to compiler-driven P1689 // scanner via env var MCPP_SCANNER=p1689 (see docs/27). + // + // A dependency whose declared targets are all programs compiles nothing in + // this build (#649 E6): its programs come from the tool sub-build, which + // scans it as its own root. The root itself is always scanned. + std::vector scannedPackages; + scannedPackages.reserve(packages.size()); + for (std::size_t i = 0; i < packages.size(); ++i) + if (i == 0 || !isProgramOnlyPackage(packages[i].manifest)) + scannedPackages.push_back(packages[i]); auto scan = [&] { const char* sel = std::getenv("MCPP_SCANNER"); if (sel && std::string_view(sel) == "p1689") { auto tmp = std::filesystem::temp_directory_path() / std::format("mcpp_p1689_{}", std::random_device{}()); std::filesystem::create_directories(tmp); - return mcpp::modgraph::scan_packages_p1689(packages, *tc, tmp, + return mcpp::modgraph::scan_packages_p1689(scannedPackages, *tc, tmp, stdFlagAndDialect); } - return mcpp::modgraph::scan_packages(packages); + return mcpp::modgraph::scan_packages(scannedPackages); }(); if (!scan.errors.empty()) { std::string msg = "scanner errors:\n"; @@ -11589,8 +12176,10 @@ prepare_build(bool print_fingerprint, // construction the fingerprint promised (stdFlagAndDialect above). // #422: the CRT model reaches the std module too. Derived from the // SAME expression the project's TUs use (flags.cppm), through the one - // helper, so the two cannot drift. Non-MSVC dialects yield "" and the - // command is unchanged. + // helper, so the two cannot drift. A GNU dialect yields "-static" or "" + // here, and the gcc and clang std module builders do not read it, so + // their commands are unchanged; clang on the MSVC ABI is given no CRT + // model at all (see `MechanismInput::msvcCrtModelEmitted`). const auto& stdDialect = mcpp::toolchain::dialect_for(*tc); const auto stdCrt = mcpp::toolchain::msvc_crt_flag( stdDialect, mcpp::toolchain::msvc_wants_static_crt( @@ -11886,9 +12475,6 @@ prepare_build(bool print_fingerprint, for (auto const& [i, form] : dependencyLinkForms) { auto const& answer = form.answer; auto const& facts = form.facts; - if (form.recorded) - graphLinkForms[i] = { std::string(lf::to_string(answer.linkage)), - answer.reason }; if (!answer.diagnostic.empty()) mcpp::diag::degraded("build/dependency-linkage", answer.diagnostic, @@ -12030,6 +12616,157 @@ prepare_build(bool print_fingerprint, } } + // ONE PROCESS, ONE C++ RUNTIME; ONE STATIC PACKAGE, ONE IMAGE (#646). + // + // Both are decided by `make_plan` and the contract table; this is where a + // decision that cannot be delivered stops the build before it compiles. + { + namespace dist = mcpp::build::dist; + auto const& bc = ctx.plan.manifest.buildConfig; + const auto format = dist::format_for( + tc->targetTriple, + mcpp::platform::is_windows ? dist::Format::Pe + : mcpp::platform::is_macos ? dist::Format::MachO + : dist::Format::Elf); + const dist::CxxSharedLoad load{ + .program = mcpp::build::image_loads_cxx_shared_library( + ctx.plan, mcpp::build::LinkUnit::Binary), + .tests = mcpp::build::image_loads_cxx_shared_library( + ctx.plan, mcpp::build::LinkUnit::TestBinary), + }; + const auto contracts = dist::role_contracts( + dist::ContractStatement{ + .cxxRuntime = bc.cxxRuntime, + .cxxRuntimeTests = bc.cxxRuntimeTests, + .cxxRuntimeShared = bc.cxxRuntimeShared, + .staticStdlib = bc.staticStdlib, + }, + format, load); + // F3a. A stated self-contained program over a coupled C++ shared + // library of this build: the program would carry a static C++ runtime + // and the library would load a shared one. The unstated case needs no + // refusal, because `role_contracts` then gives the program the + // library's contract. + if (auto role = dist::runtime_split(contracts, format, load)) { + std::string libraries; + for (auto const& lu : ctx.plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::SharedLibrary) continue; + if (!mcpp::build::link_unit_holds_cxx(ctx.plan, lu)) continue; + libraries += (libraries.empty() ? "'" : ", '") + lu.targetName + "'"; + } + const bool tests = *role == dist::Role::Test; + refusal::record(refusal::Code::ProgramCxxRuntimeSplit); + return std::unexpected(std::format( + "this build's {} state a self-contained C++ runtime and load the C++ " + "shared library {}, which is linked against the {} C++ runtime.\n" + " The process would hold two C++ runtimes: the program exports the " + "runtime symbols the\n" + " library references, the library binds some of them there and keeps " + "the rest, and the two\n" + " halves disagree about shared state (measured: a string formatted in " + "the library aborts\n" + " with std::bad_cast).\n" + " Remove the self-contained statement for {} (the `{}` value of " + "[build] cxx_runtime), and\n" + " they take the shared library's contract, or give the shared library " + "a private copy of the\n" + " runtime:\n" + "\n" + " [build]\n" + " cxx_runtime = {{ shared = \"self-contained\" }}", + tests ? "tests" : "programs", + libraries.empty() ? std::string("'(unnamed)'") : libraries, + dist::to_string(contracts.shared), + tests ? "tests" : "programs", tests ? "tests" : "default")); + } + + // MACH-O: EVERY IMAGE CARRIES ITS OWN HIDDEN libc++ (#646 F2). + // + // The Mach-O default is self-contained for every role, and each image + // embeds the payload's `libc++.a` through `-load_hidden`, so the type + // information of a standard library class exists once per image and libc++ + // compares it by address. Measured on macos-15 for this release: with the + // default, a `std::runtime_error` thrown in a dylib is NOT caught by its + // class in the program and two `std::error_code` categories compare + // unequal; with `cxx_runtime = "host-coupled"` for every role, both hold. + // The default is not changed here, because it is what every macOS build + // ships today and changing it is its own record; a build that would meet + // the split is told, once, what it is and how to avoid it. + if (format == dist::Format::MachO && (load.program || load.tests) + && contracts.shared == dist::Contract::SelfContained) { + std::string libraries; + for (auto const& lu : ctx.plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::SharedLibrary) continue; + if (!mcpp::build::link_unit_holds_cxx(ctx.plan, lu)) continue; + libraries += (libraries.empty() ? "'" : ", '") + lu.targetName + "'"; + } + mcpp::diag::degraded("build/cxx-runtime-identity", + std::format("this build's program and the C++ shared library {} each " + "carry a private copy of the C++ runtime", + libraries.empty() ? std::string("'(unnamed)'") : libraries), + "on Mach-O every image embeds the payload's libc++ with hidden " + "visibility, so the type information of a standard library class exists " + "once per image: measured on macOS, an exception of such a class thrown " + "in the library is not caught by that class in the program, and two " + "error categories compare unequal", + "state one runtime for the process, for example [build] cxx_runtime = " + "\"host-coupled\", when objects cross the boundary as exceptions or as " + "libc++ values compared by identity"); + } + + // F1. A static package that several images reach. Refused where the + // build cannot work (Mach-O and PE resolve every reference at link + // time; Android's Java host loads an application's shared library + // before anything that could supply the package), reported on other + // ELF rows, where the library binds to the program's copy at run time + // as it always has. + if (!ctx.plan.staticPlacementConflicts.empty()) { + const bool applicationRow = std::ranges::any_of(ctx.plan.linkUnits, + [](auto const& lu) { + return lu.kind == mcpp::build::LinkUnit::SharedLibrary + && !lu.dependencyOwned && lu.entryMain.has_value(); + }); + const bool refuse = format == dist::Format::MachO + || format == dist::Format::Pe || applicationRow; + std::string listing; + for (auto const& c : ctx.plan.staticPlacementConflicts) { + std::string reachers; + if (c.program) reachers = "the program"; + for (auto const& image : c.images) + reachers += (reachers.empty() ? "'" : ", '") + image + "'"; + listing += std::format(" '{}' is reached by {}\n", c.package, reachers); + } + const std::string first = ctx.plan.staticPlacementConflicts.front().package; + const std::string remedy = std::format( + " Link the package shared, so that every image loads one copy: on its " + "edge in [dependencies],\n" + "\n" + " {} = {{ ..., linkage = \"shared\" }}\n" + "\n" + " or as the package's own default, in its manifest:\n" + "\n" + " [targets.]\n" + " linkage = \"shared\"", first); + if (refuse) { + refusal::record(refusal::Code::StaticPackageInTwoImages); + return std::unexpected(std::format( + "a static package is linked into more than one image of this build, " + "and on this target\n" + " an image cannot use another image's copy:\n{}{}", + listing, remedy)); + } + mcpp::diag::degraded("build/static-placement", + std::format("a static package is reachable from more than one image of " + "this build and is linked into the program only:\n{}", + listing), + "the shared libraries bind to the program's copy at run time, which only " + "an ELF process whose program links the package can do; the same graph " + "is refused on Mach-O, PE and the Android application row", + std::format("give the package the shared form, e.g. {} = {{ ..., linkage " + "= \"shared\" }}", first)); + } + } + // The module graph outlives the plan for one consumer: `mcpp pack`, which // has to know which units are INTERFACE (published as source) and which // are implementation (published only as an object). The plan flattens that @@ -13480,28 +14217,8 @@ prepare_build(bool print_fingerprint, // reads instead of a warning's wording. { nlohmann::json graphPackages = nlohmann::json::array(); - for (std::size_t i = 0; i < packages.size(); ++i) { - nlohmann::json entry = { - {"package", package_json( - mcpp::manifest::package_id(packages[i].manifest.package))}, - {"root", i == 0}, - }; - nlohmann::json requests = nlohmann::json::array(); - for (auto const& r : graphRequests) { - if (r.dependencyPackageIndex != i) continue; - requests.push_back({ - {"requester", mcpp::manifest::package_id( - packages[r.consumerPackageIndex].manifest.package).canonical()}, - {"key", r.key}, - {"table", r.table}, - }); - } - entry["requested_by"] = std::move(requests); - if (auto form = graphLinkForms.find(i); form != graphLinkForms.end()) - entry["link"] = { {"form", form->second.first}, - {"reason", form->second.second} }; - graphPackages.push_back(std::move(entry)); - } + for (std::size_t i = 0; i < packages.size(); ++i) + graphPackages.push_back(graph_package_entry(i, /*forBuildProgram=*/false)); j["graph"] = { {"packages", std::move(graphPackages)} }; } diff --git a/src/build/refusal.cppm b/src/build/refusal.cppm index e5d2f8d01..46a13c9a3 100644 --- a/src/build/refusal.cppm +++ b/src/build/refusal.cppm @@ -105,6 +105,30 @@ enum class Code { // Distinct from LayerOrdering, which is about the layers not stacking: here // they stack, and one image would link with no C++ runtime at all (#641). SharedLibraryCxxRuntime, + // The plan needs something that has to be downloaded -- a toolchain, a + // package, a git revision, the package index -- and the run is offline + // (`--offline`, `MCPP_OFFLINE`, or the older `MCPP_NO_AUTO_INSTALL`). + // Distinct from every other code: nothing is wrong with the project, and + // one online run fixes it. An editor that plans offline by default tells + // the two apart by this code (#648). + OfflineDownloadRequired, + // The dependency graph contains a cycle of packages. Detected where the + // graph is resolved, so every cache mode refuses it alike (#649 E6); the + // build-cache key walk used to be the only place that noticed, and a + // `--cache=local` build of the same graph went ahead. + PackageCycle, + // A program or test whose manifest states a self-contained C++ runtime + // loads a C++ shared library of the same build that couples to a shared + // runtime, so the process would hold two C++ runtimes (#646 F3a). Distinct + // from SharedLibraryCxxRuntime, where the library has none: here both + // images have one and they are different copies. + ProgramCxxRuntimeSplit, + // A static package is reachable from two images of one build (two shared + // libraries, or a shared library and the program) on a format whose link + // or load cannot share one copy between them (#646 F1). Distinct from + // ExclusiveCapability, which is about two implementations of an interface: + // here it is one package that has no single image to live in. + StaticPackageInTwoImages, Other, // a refusal that has not been given a code yet }; @@ -137,6 +161,13 @@ constexpr std::string_view name(Code c) { case Code::ToolVersionConflict: return "tool-version-conflict"; case Code::SharedLibraryCxxRuntime: return "shared-library-cxx-runtime"; + case Code::OfflineDownloadRequired: + return "offline-download-required"; + case Code::PackageCycle: return "package-cycle"; + case Code::ProgramCxxRuntimeSplit: + return "program-cxx-runtime-split"; + case Code::StaticPackageInTwoImages: + return "static-package-in-two-images"; case Code::Other: return "other"; } return "other"; diff --git a/src/build/runtime_validation.cppm b/src/build/runtime_validation.cppm index e7871a436..d96d20153 100644 --- a/src/build/runtime_validation.cppm +++ b/src/build/runtime_validation.cppm @@ -955,6 +955,98 @@ check_symbol_provision(const mcpp::build::BuildPlan& plan, return closureCache.emplace(object, std::move(names)).first->second; }; + // The objects a link unit links, as absolute paths. The staged `std` + // module objects are added to every unit: the emitter appends them to + // each C++ image that imports `std` and records them nowhere else, and an + // image that did not link them cannot define the names they define, so + // listing them for every unit attributes nothing that is not there. + auto objects_of = [&](const mcpp::build::LinkUnit& lu) { + std::set out; + for (auto const& o : lu.objects) + out.insert((o.is_absolute() ? o : plan.outputDir / o).lexically_normal()); + for (auto name : {"std.o", "std.compat.o"}) + out.insert((plan.outputDir / "obj" / name).lexically_normal()); + return out; + }; + std::map> objectSymbolCache; + auto object_defines = [&](const std::filesystem::path& object) + -> const std::vector& { + auto it = objectSymbolCache.find(object); + if (it != objectSymbolCache.end()) return it->second; + std::vector names; + std::error_code ec; + if (std::filesystem::exists(object, ec)) + if (auto symbols = mcpp::platform::elf::defined_object_symbols(object)) + names = std::move(*symbols); + return objectSymbolCache.emplace(object, std::move(names)).first->second; + }; + // The installation the toolchain's compiler belongs to, canonical, for + // the second rule below: `/bin/`, whose C++ runtime + // library and module source both live under ``. + std::filesystem::path toolchainRoot; + { + std::error_code ec; + auto compiler = std::filesystem::weakly_canonical(plan.toolchain.binaryPath, ec); + if (!ec && compiler.has_parent_path()) + toolchainRoot = compiler.parent_path().parent_path(); + } + auto in_toolchain = [&](const std::filesystem::path& file) { + if (toolchainRoot.empty()) return false; + std::error_code ec; + auto canonical = std::filesystem::weakly_canonical(file, ec); + if (ec) return false; + auto rel = canonical.lexically_relative(toolchainRoot); + return !rel.empty() && *rel.begin() != ".."; + }; + // For each provider in `resolution` that is a shared library of this + // plan: the names defined by objects linked into both it and `image`. + // + // AND ONE PROVIDER THAT IS NOT A LINK UNIT: the toolchain's own C++ + // runtime. The staged `std` module object is compiled from the module + // source that runtime ships, and GCC 16's `libstdc++.so.6` exports the + // same initialisers (`_ZGIW3std@@GLIBCXX_3.4.35`, measured), so a program + // coupled to that runtime defines them twice by construction. The names + // excused are the ones the std objects define, and only against a library + // that belongs to the toolchain's own installation. + auto shared_plan_definitions = + [&](const mcpp::build::LinkUnit& image, + const mcpp::platform::elf::RuntimeResolution& resolution) { + std::map> out; + const auto imageObjects = objects_of(image); + for (auto const& object : resolution.objects) { + if (in_toolchain(object.artifact)) { + auto& names = out[object.artifact.string()]; + for (auto name : {"std.o", "std.compat.o"}) + for (auto const& defined : + object_defines((plan.outputDir / "obj" / name).lexically_normal())) + names.insert(defined); + continue; + } + const mcpp::build::LinkUnit* provider = nullptr; + for (auto const& lu : plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::SharedLibrary) continue; + std::vector names{plan.outputDir / lu.output}; + for (auto const& alias : lu.runtimeAliases) + names.push_back(plan.outputDir / alias); + for (auto const& candidate : names) { + std::error_code ec; + if (std::filesystem::equivalent(candidate, object.artifact, ec)) { + provider = &lu; + break; + } + } + if (provider) break; + } + if (!provider) continue; + auto& names = out[object.artifact.string()]; + for (auto const& o : objects_of(*provider)) { + if (!imageObjects.contains(o)) continue; + for (auto const& name : object_defines(o)) names.insert(name); + } + } + return out; + }; + for (auto const& [artifact, oldStamp] : before) { auto now = stamp(artifact); if (!now.exists) continue; @@ -1079,6 +1171,18 @@ check_symbol_provision(const mcpp::build::BuildPlan& plan, for (auto const& conflict : all) if (conflict.isWeak) ++report.sharedWeak; std::erase_if(all, [](auto const& c) { return c.isWeak; }); + // WHAT THIS BUILD PUT INTO BOTH IMAGES ON PURPOSE (#646 F3). + // + // A provider that is itself a link unit of this plan may define a + // name from an object this image links as well. The `std` module's + // object is linked into every C++ image that imports `std`, so its + // empty initialiser is defined twice in a program over a C++ shared + // library by construction; reporting it made `--strict` unusable for + // that shape. Attributed by the objects both units link, never by + // the shape of the name. + if (unit && !all.empty()) + report.sharedPlanDefinitions = sp::drop_shared_plan_definitions( + all, shared_plan_definitions(*unit, resolution)); report.conflicts = std::move(all); report.status = report.conflicts.empty() ? sp::Status::Clean : sp::Status::Conflict; diff --git a/src/build/symbol_provision.cppm b/src/build/symbol_provision.cppm index c5c6bdf3c..4cf48da48 100644 --- a/src/build/symbol_provision.cppm +++ b/src/build/symbol_provision.cppm @@ -33,14 +33,22 @@ // mcpp binary (measured: 0 of 217 dynamic symbols). Stage two runs only when // stage one is not, and it is the stage that makes the report TRUE. // -// STAGE TWO IS NOT OPTIONAL, and the reason is mcpp's own doing. A -// `kind = "shared"` dependency's link unit receives only ITS OWN objects -// (mcpp.build.plan), so a static package underneath it lands in the CONSUMER's -// executable instead, and the shared library binds back to it at run time. -// That is the shape above — arranged by mcpp, with exactly one copy of the -// code in the process, and completely benign. Reporting stage one alone would -// warn about a correct build that the user cannot do anything about, which is -// precisely the noise `mcpp.build.distribution` refuses to emit. +// STAGE TWO IS NOT OPTIONAL. An executable's dynamic symbol table also holds +// what the linker exported because a shared object references it, and one of +// those shapes is mcpp's own: a static package that several images reach is +// linked into the program, and the shared libraries bind back to it at run +// time. Reporting stage one alone would name that as a finding. +// +// THE ARRANGEMENT IS NOT BENIGN, AND IT IS NO LONGER THE DEFAULT (#646 F1). +// This comment used to call it completely benign, and that held only on ELF +// and only for a program that links the package: the same library refuses +// `-Wl,-z,defs`, a host that did not link the package cannot `dlopen` it, +// Mach-O and PE resolve at link time, and Android loads the library first. A +// static package reachable from ONE shared image is now linked into that image +// (mcpp.build.plan, `place_static_packages`). One reachable from several is +// refused where it cannot work and reported by prepare on ELF, where the +// build still binds to the program's copy; this check then reports what the +// linker exported, which is a true statement about that build. // // Design: .agents/docs/2026-08-28-issue519-dependency-linkage-form.md §2. @@ -106,6 +114,11 @@ struct Report { // to be told which ones this check decided about, or "clean" reads as // "did not look". std::size_t sharedWeak = 0; + // Definitions this image and a library it loads both take from ONE object + // of this build, counted and not listed. The module initialiser in + // `std.o` is the case that exists: every C++ image that imports `std` + // links it, and the function is empty. See `drop_shared_plan_definitions`. + std::size_t sharedPlanDefinitions = 0; // Why, for the two non-answers. Empty for Clean and Conflict. std::string reason; @@ -134,6 +147,22 @@ exported_definitions(const mcpp::platform::elf::DynamicSymbols& symbols); std::vector conflicting_exports(std::span exports, std::span closure); +// Remove what this build put into both images on purpose. +// +// `sharedDefinitions` maps a provider's label to the names defined by the +// objects this build links into BOTH the image being checked and that +// provider. A provider is removed from a conflict whose name it defines only +// through such an object, a conflict left with no provider is dropped, and the +// number of removed (symbol, provider) pairs is returned. +// +// PROVENANCE, NOT A NAME PATTERN. The case this exists for is the `std` +// module's initialiser (`_ZGIW3std`), which every C++ image importing `std` +// links from the same `std.o`; matching `_ZGIW` instead would also excuse a +// real duplicate whose name happens to have that shape (#646 F3). +std::size_t drop_shared_plan_definitions( + std::vector& conflicts, + const std::map>& sharedDefinitions); + // The report an image with no dynamic symbol table gets, and the one an image // whose author asked for exports gets. Named constructors rather than raw // struct literals so every non-answer carries its reason. @@ -206,6 +235,23 @@ std::vector conflicting_exports(std::span exports, return out; } +std::size_t drop_shared_plan_definitions( + std::vector& conflicts, + const std::map>& sharedDefinitions) { + std::size_t removed = 0; + for (auto& conflict : conflicts) { + std::erase_if(conflict.alsoProvidedBy, [&](const std::string& label) { + auto it = sharedDefinitions.find(label); + const bool shared = it != sharedDefinitions.end() + && it->second.contains(conflict.name); + if (shared) ++removed; + return shared; + }); + } + std::erase_if(conflicts, [](const Conflict& c) { return c.alsoProvidedBy.empty(); }); + return removed; +} + Report not_applicable(std::string reason) { return Report{ .status = Status::NotApplicable, .reason = std::move(reason) }; } @@ -246,6 +292,12 @@ std::string Report::explain(std::string_view artifact) const { " finding: the C++ ABI emits one per image and the loader keeps one.)\n", sharedWeak, sharedWeak == 1 ? "" : "s", sharedWeak == 1 ? "is" : "are"); + if (sharedPlanDefinitions > 0) + body += std::format( + " ({} definition{} this build links into both images from one object\n" + " {} NOT part of this finding either.)\n", + sharedPlanDefinitions, sharedPlanDefinitions == 1 ? "" : "s", + sharedPlanDefinitions == 1 ? "is" : "are"); // WHY it matters, then what to do — IN THE ORDER THAT ACTUALLY WORKS. // diff --git a/src/cli.cppm b/src/cli.cppm index 925831267..fd3456c8f 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -381,7 +381,7 @@ int run(int argc, char** argv) { .option(cl::Option("release").help("Shorthand for --profile release")) .option(cl::Option("dev").help("Shorthand for --profile dev (-O0 -g)")) .option(cl::Option("features").takes_value().value_name("LIST") - .help("Activate root-package features (comma-separated)")) + .help("Activate root-package features, and a dependency's as / (comma-separated)")) .option(cl::Option("cap").takes_value().value_name("LIST") .help("Pin capability providers (e.g. blas=openblas,lapack=mkl)")) .option(cl::Option("strict") @@ -511,7 +511,7 @@ int run(int argc, char** argv) { .option(cl::Option("profile").takes_value().value_name("NAME") .help("Build profile for the test build: dev (default) | release | dist | <[profile.*] name>")) .option(cl::Option("features").takes_value().value_name("LIST") - .help("Activate root-package features for the test build (comma-separated)")) + .help("Activate root-package features, and a dependency's as /, for the test build (comma-separated)")) .option(cl::Option("cap").takes_value().value_name("LIST") .help("Pin capability providers (e.g. blas=openblas,lapack=mkl)")) .option(cl::Option("strict") @@ -548,6 +548,8 @@ int run(int argc, char** argv) { .help("Ask about this target instead of the project's default")) .option(cl::Option("toolchain").takes_value() .help("Ask about this toolchain, e.g. llvm@22.1.8")) + .option(cl::Option("features").takes_value().value_name("LIST") + .help("Resolve with these features, as `build --features` does (root features and /)")) .option(cl::Option("format").takes_value().value_name("json") .help("Machine-readable output (enveloped; see docs/50-machine-output.md)")) .action(wrap_rc(cmd_why))) @@ -620,10 +622,18 @@ int run(int argc, char** argv) { // this only replaces the "dev" fallback every other command uses. .option(cl::Option("profile").takes_value() .help("Build profile (default: [build] default-profile, else release)")) + // The shorthands `build` and `run` take, with their precedence: + // `--profile` wins over either (#649 E9). + .option(cl::Option("release").help("Shorthand for --profile release")) + .option(cl::Option("dev").help("Shorthand for --profile dev")) .option(cl::Option("toolchain").takes_value().value_name("SPEC") .help("Build with this toolchain for one invocation, e.g. llvm@22.1.8")) .option(cl::Option("features").takes_value().value_name("LIST") - .help("Activate root-package features for every build pass of the pack (comma-separated)")) + .help("Activate root-package features, and a dependency's as /, for every build pass of the pack (comma-separated)")) + // `--format` names the PACKAGE format here, so machine output is + // asked for the way `mcpp test` asks for it (docs/50 §3). + .option(cl::Option("message-format").takes_value().value_name("FMT") + .help("Output format: human (default) | json (one mcpp.pack envelope on stdout; narration on stderr)")) .option(cl::Option("no-strip") .help("Ship the artifacts as built (default: strip debug info)")) .option(cl::Option("debug-symbols").takes_value().value_name("DIR") @@ -678,7 +688,7 @@ int run(int argc, char** argv) { .option(cl::Option("release").help("Shorthand for --profile release")) .option(cl::Option("dev").help("Shorthand for --profile dev")) .option(cl::Option("features").takes_value().value_name("LIST") - .help("Activate root-package features (comma-separated)")) + .help("Activate root-package features, and a dependency's as / (comma-separated)")) .option(cl::Option("cap").takes_value().value_name("LIST") .help("Pin capability providers (e.g. blas=openblas,lapack=mkl)")) .option(cl::Option("strict") @@ -1075,6 +1085,11 @@ int run(int argc, char** argv) { {"emit build-database", {Effect::InitMcppHome, Effect::ReadProject, Effect::Network, Effect::WriteGlobalCache, Effect::ExecBuildScript}}, + // A build, then a package under `target/dist`: the build's + // declaration plus `write-project`. + {"pack", {Effect::InitMcppHome, Effect::ReadProject, + Effect::WriteProject, Effect::Network, + Effect::WriteGlobalCache, Effect::ExecBuildScript}}, }; }; diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index d7df3682f..ca7a9b2ea 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -21,6 +21,7 @@ import mcpp.build.schedule.detach_codegen; import mcpp.build.test_targets; import mcpp.build.build_database; import mcpp.build.build_program; +import mcpp.build.refusal; // offline-download-required (#648 A1) import mcpp.dyndep; import mcpp.home; import mcpp.hooks; @@ -94,6 +95,13 @@ int run_build_with_hooks(mcpp::build::BuildContext& ctx, bool verbose, // The build selectors, read once for every command that plans a build: `mcpp // build` and `mcpp emit build-database` select the same plan from the same // flags, so the database describes the build the same flags would run. +// The profile the selectors name; see `mcpp::build::profile_override_from_flags`. +std::string profile_from_selectors(const mcpplibs::cmdline::ParsedArgs& parsed) { + return mcpp::build::profile_override_from_flags( + parsed.value("profile").value_or(""), + parsed.is_flag_set("release"), parsed.is_flag_set("dev")); +} + mcpp::build::BuildOverrides overrides_from_selectors( const mcpplibs::cmdline::ParsedArgs& parsed) { mcpp::build::BuildOverrides ov; @@ -109,9 +117,7 @@ mcpp::build::BuildOverrides overrides_from_selectors( // Profile selection precedence: --profile NAME > --release / --dev > the // project default ([build].default-profile) > "release", resolved in // prepare_build. --release/--dev are shorthands only. - if (auto pr = parsed.value("profile")) ov.profile = *pr; - else if (parsed.is_flag_set("release")) ov.profile = "release"; - else if (parsed.is_flag_set("dev")) ov.profile = "dev"; + ov.profile = profile_from_selectors(parsed); if (auto fs = parsed.value("features")) ov.features = *fs; if (auto cp = parsed.value("cap")) ov.capabilities = *cp; ov.strict = parsed.is_flag_set("strict"); @@ -372,7 +378,20 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) testDiscovery.push_back(std::move(discovery)); } } - if (planError) return failed("MCPP_BUILD_DATABASE_PLAN_FAILED", *planError); + // An offline plan that needs a download is not a defect of the project, and + // a client that plans offline by default (an editor) has to tell the two + // apart without reading the message (#648 A1). The code is taken only while + // the run is offline, so a refusal recorded on a path that recovered cannot + // relabel an unrelated failure. + if (planError) { + const bool offline = mcpp::platform::env::offline_mode() + || mcpp::platform::env::no_auto_install(); + if (mcpp::build::refusal::take() + == mcpp::build::refusal::Code::OfflineDownloadRequired + && offline) + return failed("MCPP_OFFLINE_DOWNLOAD_REQUIRED", *planError); + return failed("MCPP_BUILD_DATABASE_PLAN_FAILED", *planError); + } // The lock this planning produced, against the project's. The project's is // never written; a difference is reported. @@ -472,11 +491,9 @@ export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, if (auto rn = parsed.value("runner")) runner_name = *rn; // The same two axes `build` and `test` take, read the same way. `--release` // and `--dev` are the shorthands the other verbs already accept. - std::string features, profile; + std::string features; if (auto fs = parsed.value("features")) features = *fs; - if (auto pr = parsed.value("profile")) profile = *pr; - if (parsed.is_flag_set("release")) profile = "release"; - if (parsed.is_flag_set("dev")) profile = "dev"; + const std::string profile = profile_from_selectors(parsed); // The device axis, read exactly as `build` reads it: `--no-accel` is an // explicit choice and not the absence of `--accel`, so it travels as the // same sentinel. Without this a project's CPU-only variant could be built @@ -506,7 +523,7 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, // granularity for sanitizers / contract evaluation semantics). Post-`--` // args go to each test binary. mcpp::build::BuildOverrides ov; - if (auto pr = parsed.value("profile")) ov.profile = *pr; + ov.profile = profile_from_selectors(parsed); if (auto fs = parsed.value("features")) ov.features = *fs; if (auto cp = parsed.value("cap")) ov.capabilities = *cp; ov.strict = parsed.is_flag_set("strict"); diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 0c8cc8f8e..e304d57d4 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -9,12 +9,16 @@ export module mcpp.cli.cmd_publish; import std; import mcpplibs.cmdline; +import mcpp.build.prepare; // profile_override_from_flags +import mcpp.libs.json; import mcpp.pack; import mcpp.pack.library_pipeline; import mcpp.pack.pipeline; import mcpp.pack.route; +import mcpp.platform.terminal; import mcpp.publish.pipeline; import mcpp.ui; +import mcpp.wire; namespace mcpp::cli { @@ -32,7 +36,99 @@ export int cmd_publish(const mcpplibs::cmdline::ParsedArgs& parsed) { parsed.is_flag_set("dry-run"), parsed.is_flag_set("allow-dirty")); } +namespace { + +int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, + mcpp::pack::PackOutcome* report, + mcpp::pack::LibraryPackReport* libraryReport, + bool* libraryRoute); + +} // namespace + +// `mcpp pack --message-format json` (#649 E9). +// +// ONE ENVELOPE ON STDOUT, AND EVERYTHING ELSE ON STDERR. `--format` names the +// PACKAGE format on this command, so machine output is asked for the way +// `mcpp test` asks for it, and the document is printed after the command has +// finished, alone: planning, building and packing narrate, and so do the +// programs they start, which is why the whole run is under `StdoutToStderr`, +// as `emit build-database`'s planning is. +// +// `data.artifacts` holds what the human `Packed` lines name, as absolute paths: +// the archive or tree of a built-in format, the terminal outputs of a +// dispatched one, or the library package. A leg is a `targets` entry of the +// artifact it went into, not an artifact of its own. export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { + std::string messageFormat = "human"; + if (auto mf = parsed.value("message-format")) messageFormat = *mf; + if (messageFormat != "human" && messageFormat != "json") { + mcpp::ui::error(std::format("unknown --message-format '{}' (human|json)", + messageFormat)); + return 2; + } + if (messageFormat == "human") + return cmd_pack_body(parsed, nullptr, nullptr, nullptr); + + mcpp::pack::PackOutcome outcome; + mcpp::pack::LibraryPackReport library; + bool libraryRoute = false; + int rc = 0; + { + mcpp::platform::terminal::StdoutToStderr narration; + rc = cmd_pack_body(parsed, &outcome, &library, &libraryRoute); + } + + using mcpp::wire::Effect; + std::vector effects{Effect::ReadProject, Effect::WriteProject, + Effect::WriteGlobalCache}; + if (outcome.ranBuildPrograms) effects.push_back(Effect::ExecBuildScript); + mcpp::wire::Envelope env{ .kind = "mcpp.pack", .effects = std::move(effects) }; + if (rc != 0) { + env.data = nullptr; + env.diagnostics.push_back({"MCPP_PACK_FAILED", mcpp::wire::Severity::Error, + "mcpp pack did not produce a package; the reason is on standard error"}); + mcpp::wire::emit(env); + return rc; + } + + auto artifact_json = [](const std::filesystem::path& p, std::string const& format, + std::vector const& targets) { + std::error_code ec; + nlohmann::json t = nlohmann::json::array(); + for (auto const& x : targets) t.push_back(x); + return nlohmann::json{ + {"path", std::filesystem::absolute(p, ec).lexically_normal().string()}, + {"type", std::filesystem::is_directory(p, ec) ? "directory" : "file"}, + {"format", format}, + {"targets", std::move(t)}, + }; + }; + nlohmann::json artifacts = nlohmann::json::array(); + nlohmann::json stage = nullptr; + if (libraryRoute) { + const auto fmt = parsed.value("format").value_or("tar"); + artifacts.push_back(artifact_json(library.artifact, fmt, library.targets)); + } else { + for (auto const& a : outcome.artifacts) + artifacts.push_back(artifact_json(a, outcome.format, outcome.targets)); + if (!outcome.stageDir.empty()) + stage = nlohmann::json{ + {"dir", outcome.stageDir.lexically_normal().string()}, + {"manifest", outcome.stageManifest.lexically_normal().string()}, + {"closure", outcome.closure}, + }; + } + env.data = nlohmann::json{{"artifacts", std::move(artifacts)}, {"stage", std::move(stage)}}; + mcpp::wire::emit(env); + return 0; +} + +namespace { + +int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, + mcpp::pack::PackOutcome* report, + mcpp::pack::LibraryPackReport* libraryReport, + bool* libraryRoute) { // ─── Resolve mode ──────────────────────────────────────────────── mcpp::pack::Options opts; bool modeFromUser = false; @@ -76,7 +172,10 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { // `mcpp run q` once became `--target=q`. `pack`'s positional is `target`, // so `--target` is read through `option()` above and stays unaffected; // `profile` and `debug-symbols` have no positional twin. - if (auto v = parsed.value("profile")) opts.profile = *v; + // `--profile` > `--release` / `--dev`, the rule `build` and `run` follow. + opts.profile = mcpp::build::profile_override_from_flags( + parsed.value("profile").value_or(""), + parsed.is_flag_set("release"), parsed.is_flag_set("dev")); if (parsed.is_flag_set("no-strip")) opts.strip = false; if (auto v = parsed.value("debug-symbols")) opts.debugSymbols = *v; if (auto v = parsed.value("features")) opts.features = *v; @@ -123,7 +222,9 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { "--mode is an application-bundle depth and does not apply to the " "library target '{}' yet; ignoring it", route->targetName)); } - return mcpp::pack::build_and_pack_library(route->targetName, triples, opts); + if (libraryRoute) *libraryRoute = true; + return mcpp::pack::build_and_pack_library(route->targetName, triples, opts, + libraryReport); } if (triples.size() > 1) { // #630 A9: THE ROUTE IS CHOSEN BY THE ARTIFACT'S FORM, NOT BY THE @@ -152,13 +253,19 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { // #622 A10: `build_and_pack` now reports the artifact(s) it packed, for // `mcpp run --format` to take as its operand -- `mcpp pack` itself only // ever needed the exit code. - return mcpp::pack::build_and_pack(std::move(opts), modeFromUser, - route->targetName, std::move(*extraLegs)).rc; + auto out = mcpp::pack::build_and_pack(std::move(opts), modeFromUser, + route->targetName, std::move(*extraLegs)); + if (report) *report = out; + return out.rc; } // #622 A10: `build_and_pack` now reports the artifact(s) it packed, for // `mcpp run --format` to take as its operand -- `mcpp pack` itself only // ever needed the exit code. - return mcpp::pack::build_and_pack(std::move(opts), modeFromUser, route->targetName).rc; + auto out = mcpp::pack::build_and_pack(std::move(opts), modeFromUser, route->targetName); + if (report) *report = out; + return out.rc; } +} // namespace + } // namespace mcpp::cli diff --git a/src/cli/cmd_self.cppm b/src/cli/cmd_self.cppm index f118c3707..9ff92c8ef 100644 --- a/src/cli/cmd_self.cppm +++ b/src/cli/cmd_self.cppm @@ -107,7 +107,8 @@ export int cmd_why(const mcpplibs::cmdline::ParsedArgs& parsed) { parsed.option_or_empty("target").value(), parsed.option_or_empty("toolchain").value()); } - return mcpp::doctor::why_report(parsed.positional(0)); + return mcpp::doctor::why_report(parsed.positional(0), + parsed.option_or_empty("features").value()); } // Also called directly by the dispatcher for the legacy `--explain CODE` form. diff --git a/src/config.cppm b/src/config.cppm index 14578f0b6..7d370d757 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -40,6 +40,10 @@ struct IndexRepo { std::string name; std::string url; std::string artifact; // optional artifact source base (xlings >= 0.4.68, #269) + // The CN base of the same artifact. Non-empty makes `artifact` the GLOBAL + // half of a region object, which xlings resolves against its mirror + // setting (xlings #377): `{"GLOBAL": artifact, "CN": artifactCn}`. + std::string artifactCn; std::string source; // optional "auto" | "artifact" | "git" ("" = xlings default auto) // Read from a `[index.repos.]` table rather than added by mcpp as a // default. The provisioning line names an index that such a table @@ -58,6 +62,14 @@ inline constexpr std::string_view kMcpplibsIndexUrlLegacy = "https://github.com/mcpp-community/mcpp-index.git"; inline constexpr std::string_view kMcpplibsIndexArtifact = "https://github.com/xlings-res/mcpp-index"; +// The GitCode mirror of the same artifact, published by mcpp-index's +// `tools/publish_mcpp_index.sh` with the same bytes as the GitHub copy +// (mcpplibs/mcpp-index#432 made that a property the script checks). A CN +// machine used to fetch the index from GitHub even with `mirror = CN`, and a +// connection that never answered held a background plan for eleven minutes +// (#648 A6). +inline constexpr std::string_view kMcpplibsIndexArtifactCn = + "https://gitcode.com/xlings-res/mcpp-index"; struct GlobalConfig { // Resolved paths @@ -97,6 +109,12 @@ struct GlobalConfig { // being portable between a corporate LAN, a laptop and CI. bool indexAutoRefresh = true; + // From config.toml [index] refresh_timeout (seconds). The total bound on + // one index refresh; a refresh that exceeds it counts as failed and the + // build continues from the local index (#648 A3). Machine policy, like + // auto_refresh: it describes the network, not the project. + std::int64_t indexRefreshTimeoutSeconds = 120; + // From config.toml [build] std::int64_t defaultJobs = 0; @@ -345,10 +363,14 @@ default = "mcpplibs" # cannot be resolved from the local copy (never merely because time passed). # Set false to require an explicit `mcpp index update`. auto_refresh = true +# refresh_timeout: seconds one automatic or explicit index refresh may take. +# A refresh that exceeds it is stopped, counts as failed, and the build +# continues with the local index. +refresh_timeout = 120 [index.repos."mcpplibs"] url = "https://github.com/mcpplibs/mcpp-index.git" -artifact = "https://github.com/xlings-res/mcpp-index" +artifact = { GLOBAL = "https://github.com/xlings-res/mcpp-index", CN = "https://gitcode.com/xlings-res/mcpp-index" } # source = "auto" # default: artifact first, git fallback; set "git" to force git # xlings auto-adds xim / awesome / scode / d2x as defaults. @@ -372,7 +394,7 @@ bool write_default_xlings_json(const std::filesystem::path& path, std::vector pairs; pairs.reserve(repos.size()); for (auto& r : repos) - pairs.push_back({ r.name, r.url, r.artifact, r.source }); + pairs.push_back({ r.name, r.url, r.artifact, r.source, r.artifactCn }); // seed_xlings_json writes to env.home / ".xlings.json", so we // construct a temporary Env with home = path.parent_path(). mcpp::xlings::Env env; @@ -436,7 +458,10 @@ bool write_json_file(const std::filesystem::path& path, const nlohmann::json& do nlohmann::json index_repo_entry(const IndexRepo& r) { nlohmann::json e = { {"name", r.name}, {"url", r.url} }; - if (!r.artifact.empty()) e["artifact"] = r.artifact; + if (!r.artifact.empty() && !r.artifactCn.empty()) + e["artifact"] = { {"GLOBAL", r.artifact}, {"CN", r.artifactCn} }; + else if (!r.artifact.empty()) + e["artifact"] = r.artifact; if (!r.source.empty()) e["source"] = r.source; return e; } @@ -596,6 +621,12 @@ void canonicalize_legacy_index_names(GlobalConfig& cfg) { // explicit opt-out; a user-set artifact base always wins. if (r.url == kMcpplibsIndexUrl && r.artifact.empty() && r.source != "git") r.artifact = std::string(kMcpplibsIndexArtifact); + // The CN half of the default (#648 A6), for a configuration written + // before it existed. Only when the GLOBAL base is the default one: a + // user's own artifact base has no mirror mcpp knows of. + if (r.url == kMcpplibsIndexUrl && r.artifact == kMcpplibsIndexArtifact + && r.artifactCn.empty()) + r.artifactCn = std::string(kMcpplibsIndexArtifactCn); bool duplicate = std::any_of(normalized.begin(), normalized.end(), [&](const IndexRepo& existing) { return existing.name == r.name && existing.url == r.url; @@ -661,6 +692,10 @@ std::expected load_or_init( cfg.xlingsHomeOverride = *h; cfg.defaultIndex = doc->get_string("index.default").value_or("mcpplibs"); cfg.indexAutoRefresh = doc->get_bool("index.auto_refresh").value_or(true); + if (auto t = doc->get_int("index.refresh_timeout"); t && *t > 0) + cfg.indexRefreshTimeoutSeconds = *t; + mcpp::xlings::set_index_refresh_timeout( + std::chrono::seconds{cfg.indexRefreshTimeoutSeconds}); cfg.searchTtlSeconds = doc->get_int("cache.search_ttl_seconds").value_or(3600); cfg.defaultJobs = doc->get_int("build.default_jobs").value_or(0); cfg.defaultToolchain = doc->get_string("toolchain.default").value_or(""); @@ -691,8 +726,17 @@ std::expected load_or_init( if (it == tt.end() || !it->second.is_string()) continue; IndexRepo r{ name, it->second.as_string() }; r.fromConfig = true; - if (auto a = tt.find("artifact"); a != tt.end() && a->second.is_string()) + if (auto a = tt.find("artifact"); a != tt.end() && a->second.is_string()) { r.artifact = a->second.as_string(); + } else if (a != tt.end() && a->second.is_table()) { + // The region form xlings reads: GLOBAL is the base every + // mirror falls back to, CN the base a CN mirror asks first. + auto& region = a->second.as_table(); + if (auto g = region.find("GLOBAL"); g != region.end() && g->second.is_string()) + r.artifact = g->second.as_string(); + if (auto c = region.find("CN"); c != region.end() && c->second.is_string()) + r.artifactCn = c->second.as_string(); + } if (auto s = tt.find("source"); s != tt.end() && s->second.is_string()) r.source = s->second.as_string(); cfg.indexRepos.push_back(std::move(r)); @@ -728,12 +772,15 @@ std::expected load_or_init( // dependency resolution (e.g. linux-headers existing in both // scode and xim). See docs/21 §VII. auto add_default = [&](std::string_view name, std::string_view url, - std::string_view artifact = {}) { + std::string_view artifact = {}, + std::string_view artifactCn = {}) { for (auto& r : cfg.indexRepos) if (r.name == name) return; - cfg.indexRepos.push_back({ std::string(name), std::string(url), - std::string(artifact), std::string() }); + IndexRepo r{ std::string(name), std::string(url), std::string(artifact) }; + r.artifactCn = std::string(artifactCn); + cfg.indexRepos.push_back(std::move(r)); }; - add_default("mcpplibs", kMcpplibsIndexUrl, kMcpplibsIndexArtifact); + add_default("mcpplibs", kMcpplibsIndexUrl, kMcpplibsIndexArtifact, + kMcpplibsIndexArtifactCn); canonicalize_legacy_index_names(cfg); // 5. Seed registry/.xlings.json if missing; migrate legacy cached diff --git a/src/doctor.cppm b/src/doctor.cppm index 7cd87e482..7bd2848a0 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -1012,7 +1012,7 @@ export int why_toolchain_json(std::string_view target, std::string_view tcSpec) return 0; } -export int why_report(const std::string& topic) { +export int why_report(const std::string& topic, const std::string& features) { const bool all = topic.empty() || topic == "all"; // The dedicated runtime view is a pure interpreter of the build's stored @@ -1020,7 +1020,13 @@ export int why_report(const std::string& topic) { // artifact re-parse is allowed on this path. if (topic == "runtime") return print_stored_runtime_resolution(); - auto ctx = mcpp::build::prepare_build(/*print_fingerprint=*/false); + // `--features` reaches the resolution this report reads (#649 E8): the + // graph of a feature build is a different graph, and before this it was + // visible only in the `resolution.json` of that build. + mcpp::build::BuildOverrides ov; + ov.features = features; + auto ctx = mcpp::build::prepare_build(/*print_fingerprint=*/false, + /*includeDevDeps=*/false, {}, ov); if (!ctx) { std::println(stderr, "error: {}", ctx.error()); return 2; } auto& tc = ctx->tc; diff --git a/src/fallback/config_migration.cppm b/src/fallback/config_migration.cppm index 56e758723..3732fe43a 100644 --- a/src/fallback/config_migration.cppm +++ b/src/fallback/config_migration.cppm @@ -94,13 +94,28 @@ bool migrate_xlings_json_index_names(const std::filesystem::path& path) { // means a previous run (or a fresh seed) already declared it — a plain // replace_all would re-inject on every run. Both spacing variants // because the file has two writers (mcpp pretty / xlings compact). + constexpr std::string_view region = + "{ \"GLOBAL\": \"https://github.com/xlings-res/mcpp-index\", " + "\"CN\": \"https://gitcode.com/xlings-res/mcpp-index\" }"; if (updated.find("xlings-res/mcpp-index") == std::string::npos) { - constexpr std::string_view art = - ", \"artifact\": \"https://github.com/xlings-res/mcpp-index\""; + const std::string art = std::string(", \"artifact\": ") + std::string(region); for (std::string_view urlkv : { "\"url\": \"https://github.com/mcpplibs/mcpp-index.git\"", "\"url\":\"https://github.com/mcpplibs/mcpp-index.git\"" }) - replace_all(updated, urlkv, std::string(urlkv) + std::string(art)); + replace_all(updated, urlkv, std::string(urlkv) + art); + } + + // The CN half of the default artifact (#648 A6). An existing home keeps the + // GitHub-only base it was seeded with, and its default entry is not + // reconciled from configuration, so this is the channel that gives it the + // GitCode mirror. Only the exact default value is rewritten: a base the user + // wrote is theirs. Idempotent, because the rewritten value no longer + // contains the flat spelling. Both spacings, for the two writers. + if (updated.find("gitcode.com/xlings-res/mcpp-index") == std::string::npos) { + for (std::string_view flat : { + "\"artifact\": \"https://github.com/xlings-res/mcpp-index\"", + "\"artifact\":\"https://github.com/xlings-res/mcpp-index\"" }) + replace_all(updated, flat, std::string("\"artifact\": ") + std::string(region)); } return write_text_if_changed(path, original, updated); diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index b3a504e2b..d3bf15997 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -110,9 +110,18 @@ std::vector extras_of(const mcpp::manifest::Manifest& m, // `triples` is the `--target` list; empty means "this host". Each entry gets // its own prepare+build, so the artifacts really are the ones this run made — // the packer never searches `target/` for something that looks right. +// +// `report`, when given, receives what `mcpp pack --message-format json` states +// (#649 E9): the package produced and the canonical triple of each leg. +export struct LibraryPackReport { + std::filesystem::path artifact; + std::vector targets; +}; + export int build_and_pack_library(const std::string& targetName, const std::vector& triples, - const mcpp::pack::Options& opts) + const mcpp::pack::Options& opts, + LibraryPackReport* report = nullptr) { std::vector legs = triples; if (legs.empty()) legs.push_back({}); // one leg, this host @@ -478,6 +487,11 @@ export int build_and_pack_library(const std::string& targetName, auto out = run_library_pack(plan); if (!out) { mcpp::ui::error(out.error().message); return 1; } mcpp::ui::status("Packed", out->string()); + if (report) { + std::error_code ec; + report->artifact = std::filesystem::absolute(*out, ec).lexically_normal(); + for (auto const& leg : plan.legs) report->targets.push_back(leg.triple); + } return 0; } diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 3035d9325..c9c2c83bb 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -189,7 +189,12 @@ std::filesystem::path resolve_debug_dir(const Options& opts, // its own directories (see `Options::toolchainLibraryDirs`). struct SharedLeg { std::string abi; + // The leg's canonical triple, for the pack report (#649 E9). + std::string triple; std::filesystem::path artifact; + // #649 E5: the shared libraries THIS leg's graph built. See + // `Plan::graphSharedLibraries`. + std::vector graphSharedLibraries; std::vector searchDirs; std::vector platformDirs; }; @@ -255,8 +260,21 @@ struct Plan { bool strip = true; std::filesystem::path debugDir; // absolute; empty = discard mcpp::pack::StripTools stripTools; + // #649 E5: WHAT THIS GRAPH BUILT, absolute: every `SharedLibrary` link + // unit of the plan, the dependencies' shared images included. A staged + // copy of one of these is stripped as the program is, because "a package + // strips what it built" (the dh_strip rule `strip_program` quotes) covers + // them: they were compiled from source by this build. Set after + // `make_plan`, by the caller that holds the build plan. + std::vector graphSharedLibraries; }; +// Does packing `plan` strip anything? The strip decision, AND a format whose +// debug information is inside the image. A Mach-O or MSVC PE row keeps its +// debug information beside the image, so the answer there is no whatever +// was requested; the `Packing` line states this, not the request. +bool strips_on_this_row(const Plan& plan); + struct Error { std::string message; }; // What step 4 of `run` (resolving the dependency closure) produced. @@ -453,6 +471,12 @@ std::string wrapper_dirname_from_archive(const std::filesystem::path& archive) { } // namespace detail +bool strips_on_this_row(const Plan& plan) { + // wasm32-emscripten is packed by `run_wasm`, which has no strip step: the + // launcher is JavaScript and the module is one static image. + return plan.strip && plan.stripTools.inBandDebugInfo && !plan.targetIsWasm; +} + bool resolve_strip(const Options& opts, const mcpp::manifest::PackConfig& cfg) { if (opts.strip) return *opts.strip; if (cfg.strip) return *cfg.strip; @@ -815,12 +839,12 @@ set_interpreter(const std::filesystem::path& binary, return {}; } -// Remove the program's debug information — and ONLY the program's. +// Remove the program's debug information. // -// A bundled `.so` is somebody else's file: it came out of the store or off the -// host, mcpp did not build it, and stripping it would change a shared payload's -// bytes for no gain to this bundle. dh_strip draws the same line (a package -// strips what it built). +// dh_strip's line is that a package strips what it built, and the program is +// the first thing this one built. The shared libraries the same graph compiled +// are the rest (`strip_staged_library` below, #649 E5); a library that came out +// of the store or off the host is somebody else's file and is left as shipped. // // Shared with `run_pe` deliberately: a MinGW `.exe` carries DWARF in-band just // like an ELF one, so "does the bundle ship debug info" must not depend on @@ -835,6 +859,91 @@ strip_program(const Plan& plan, const std::filesystem::path& staged) return {}; } +// IS `file` ONE OF `set`? Compared as file-system identity (after `..` and +// symbolic links are resolved), so the loader's spelling of a path and the +// plan's spelling of the same file agree. +bool path_in(const std::filesystem::path& file, + const std::vector& set) +{ + std::error_code ec; + const auto want = std::filesystem::weakly_canonical(file, ec); + for (auto const& p : set) { + std::error_code pec; + if (std::filesystem::weakly_canonical(p, pec) == want) return true; + } + return false; +} + +// DOES `file` LIE UNDER ONE OF `dirs`? +bool path_under(const std::filesystem::path& file, + const std::vector& dirs) +{ + std::error_code ec; + const auto f = std::filesystem::weakly_canonical(file, ec); + for (auto const& d : dirs) { + std::error_code dec; + const auto rel = f.lexically_relative(std::filesystem::weakly_canonical(d, dec)); + if (!rel.empty() && *rel.begin() != "..") return true; + } + return false; +} + +// SHOULD THE STAGED COPY OF `source` BE STRIPPED? (#649 E5) +// +// Two answers are yes: +// +// built by this graph a `SharedLibrary` link unit of the plan -- compiled +// from source here, so dh_strip's rule applies to it +// exactly as it applies to the program; +// the toolchain's own a runtime found in the directories the row's own +// runtime toolchain links from (`libc++_shared.so` from the +// NDK). The reason for leaving a store file alone is +// not changing a SHARED payload's bytes, and a copy in +// the staged tree is not shared. The Android Gradle +// plugin strips the same file. +// +// Everything else -- a library from the store, a prebuilt a package deployed, +// a host library -- stays as shipped: a vendor library may be signed, and its +// bytes are its publisher's. +bool strips_staged_copy(const Plan& plan, + const std::filesystem::path& source, + const std::vector& graphBuilt) +{ + if (path_in(source, graphBuilt)) return true; + if (path_under(source, plan.opts.toolchainLibraryDirs)) return true; + if (plan.opts.carryToolchainRuntime + && path_under(source, plan.opts.toolchainRuntimeDirs)) return true; + return false; +} + +// Strip the staged copy of a shared library, keeping its exports +// (`--strip-unneeded`, the `SharedLibrary` shape). `--no-strip` reaches it +// through `plan.strip`, and `--debug-symbols` through `debugDir`, as they reach +// the program. +// +// NEVER THE BUILD'S OWN FILE. A member the build already placed where it is +// staged is the same file as its source (see `stage_closure`), and editing it +// would change an output ninja believes is up to date. +std::expected +strip_staged_library(const Plan& plan, + const std::filesystem::path& staged, + const std::filesystem::path& source, + const std::filesystem::path& debugDir) +{ + if (!plan.strip) return {}; + std::error_code ec; + if (std::filesystem::equivalent(source, staged, ec)) return {}; + // A copy of a read-only store file is read-only too, and a strip tool + // rewrites the file it is given. + std::filesystem::permissions(staged, std::filesystem::perms::owner_write, + std::filesystem::perm_options::add, ec); + auto r = mcpp::pack::strip_artifact(staged, + mcpp::pack::ArtifactShape::SharedLibrary, + plan.stripTools, debugDir); + if (!r) return std::unexpected(Error{r.error()}); + return {}; +} + // Bundle all `deps` into /lib/. We dereference any // symlinks so the bundle is self-contained even if /usr/lib/foo.so → /usr/lib/foo.so.1. std::expected @@ -1132,7 +1241,8 @@ stage_closure(const Plan& plan, const ClosureRead& read, const std::filesystem::path& dir, std::vector& needs, std::vector& unresolvedLines, - std::string_view prefix) + std::string_view prefix, + std::vector>* copies = nullptr) { auto skipped = [&](const std::string& name) { const auto leaf = std::filesystem::path(name).filename().string(); @@ -1162,6 +1272,8 @@ stage_closure(const Plan& plan, const ClosureRead& read, "failed to copy {} -> {}: {}", m.source.string(), dst.string(), cec.message())}); } + // (staged, source), for the caller that strips what the graph built. + if (copies) copies->emplace_back(dst, m.source); needs.push_back({m.name, ClosureNeed::Kind::Staged, dst.lexically_relative(plan.stagingRoot).generic_string()}); } @@ -1474,6 +1586,7 @@ run_pe(const Plan& plan) } std::vector needs; + std::vector> copies; if (plan.opts.mode != Mode::None && plan.opts.mode != Mode::Static) { // `vendored` and `self-contained` collect the same set here, and that // is a property of the PLATFORM rather than a simplification. @@ -1493,7 +1606,8 @@ run_pe(const Plan& plan) // is an executable the reader could not parse, which no format can // package. std::vector unresolved; - if (auto r = stage_closure(plan, read, plan.stagingRoot, needs, unresolved, {}); !r) + if (auto r = stage_closure(plan, read, plan.stagingRoot, needs, unresolved, {}, + &copies); !r) return std::unexpected(r.error()); if (!unresolved.empty()) return std::unexpected(Error{unresolved_reason(plan.binaryName, unresolved, @@ -1501,6 +1615,13 @@ run_pe(const Plan& plan) } if (auto r = strip_program(plan, stagedExe); !r) return std::unexpected(r.error()); + // #649 E5: and the DLLs this graph built (a MinGW DLL carries its DWARF + // in-band as the `.exe` does), plus the toolchain runtime it carries. + for (auto const& [staged, source] : copies) { + if (!strips_staged_copy(plan, source, plan.graphSharedLibraries)) continue; + if (auto r = strip_staged_library(plan, staged, source, plan.debugDir); !r) + return std::unexpected(r.error()); + } if (plan.opts.format != Format::Tar) return needs; @@ -1626,6 +1747,8 @@ run_shared_program(const Plan& plan) std::filesystem::path artifact; std::vector searchDirs; std::vector platformDirs; + // #649 E5: what this leg's graph built. + std::vector graphBuilt; }; auto primarySearch = plan.searchDirs; for (auto const& d : plan.opts.toolchainLibraryDirs) primarySearch.push_back(d); @@ -1640,13 +1763,14 @@ run_shared_program(const Plan& plan) auto t = mcpp::toolchain::triple::parse(plan.triple); auto primaryAbi = t ? mcpp::toolchain::triple::android_abi(*t) : plan.triple; legs.push_back({plan.stagingRoot / "lib" / primaryAbi, primaryAbi, - plan.builtBinary, primarySearch, plan.opts.platformLibraryDirs}); + plan.builtBinary, primarySearch, plan.opts.platformLibraryDirs, + plan.graphSharedLibraries}); for (auto const& leg : plan.extraSharedLegs) legs.push_back({plan.stagingRoot / "lib" / leg.abi, leg.abi, leg.artifact, - leg.searchDirs, leg.platformDirs}); + leg.searchDirs, leg.platformDirs, leg.graphSharedLibraries}); } else { legs.push_back({plan.stagingRoot / "lib", {}, plan.builtBinary, primarySearch, - plan.opts.platformLibraryDirs}); + plan.opts.platformLibraryDirs, plan.graphSharedLibraries}); } std::vector needs; @@ -1679,8 +1803,38 @@ run_shared_program(const Plan& plan) in.searchDirs = leg.searchDirs; in.platformDirs = leg.platformDirs; const auto read = read_closure(in); - if (auto r = stage_closure(plan, read, leg.dir, needs, unresolved, prefix); !r) + std::vector> copies; + if (auto r = stage_closure(plan, read, leg.dir, needs, unresolved, prefix, + &copies); !r) return std::unexpected(r.error()); + + // #649 E5: THIS ROW USED TO STRIP NOTHING. `run` dispatches a + // shared-object program here before the ELF path that calls + // `strip_program`, so the program, the libraries its graph built and + // the NDK's `libc++_shared.so` all shipped their symbol tables while + // the `Packing` line said "stripped", and `--debug-symbols` was + // ignored. The program is a shared object, so it takes the + // `SharedLibrary` shape, which keeps the `.dynsym` its host loads it + // through. One tool serves every leg: the NDK's `llvm-strip` reads + // every Android ABI. A several-ABI tree separates its debug files by + // ABI, since two legs stage files of one name. + const auto legDebugDir = (plan.debugDir.empty() || leg.abi.empty()) + ? plan.debugDir : plan.debugDir / leg.abi; + if (plan.strip) { + const auto stagedProgram = leg.dir / plan.binaryName; + std::error_code pec; + std::filesystem::permissions(stagedProgram, std::filesystem::perms::owner_write, + std::filesystem::perm_options::add, pec); + auto r = mcpp::pack::strip_artifact(stagedProgram, + mcpp::pack::ArtifactShape::SharedLibrary, + plan.stripTools, legDebugDir); + if (!r) return std::unexpected(Error{r.error()}); + } + for (auto const& [staged, source] : copies) { + if (!strips_staged_copy(plan, source, leg.graphBuilt)) continue; + if (auto r = strip_staged_library(plan, staged, source, legDebugDir); !r) + return std::unexpected(r.error()); + } } // THE RUNTIME FILES TRAVEL AS ON EVERY OTHER ROW. `deploy` placed them @@ -1847,6 +2001,10 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // text when it is incomplete. Both stay empty under `--mode static`. std::vector needs; std::string unresolvedText; + // #649 E5: (staged, source) of every bundled library, for the strip step + // at the end, which must see the final images (after the search-path edits). + std::vector> bundledCopies; + std::string bundledLoader; if (plan.opts.mode != Mode::Static) { // THE BUILT BINARY, NOT THE STAGED COPY, and the difference is // `$ORIGIN`. @@ -1954,6 +2112,9 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) if (auto r = bundle_libs(toBundle, plan.stagingRoot); !r) return std::unexpected(Error{r.error()}); + for (auto const& d : toBundle) + bundledCopies.emplace_back(plan.stagingRoot / "lib" / d.soname, d.path); + bundledLoader = find_loader_soname(toBundle); // Search path: point at bundled libs, or REMOVE THE TAG if there are none. // @@ -2077,6 +2238,15 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // must see the stripped one. Same ordering rule the library packer states // at its leg loop. if (auto r = strip_program(plan, bundledBinary); !r) return std::unexpected(r.error()); + // #649 E5: and the shared libraries this graph built. The dynamic loader + // is never one of them and is never edited, for the reason the + // search-path loop above gives. + for (auto const& [staged, source] : bundledCopies) { + if (!bundledLoader.empty() && staged.filename() == bundledLoader) continue; + if (!strips_staged_copy(plan, source, plan.graphSharedLibraries)) continue; + if (auto r = strip_staged_library(plan, staged, source, plan.debugDir); !r) + return std::unexpected(r.error()); + } // A dispatched format reaches this point with an incomplete closure; an // archive was refused before any file was edited. diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 9f7a8221b..140519b12 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -43,6 +43,21 @@ namespace mcpp::pack { export struct PackOutcome { int rc = 0; std::vector artifacts; + // #649 E9: what `mcpp pack --message-format json` reports beside the + // artifacts. All of it was already answered by the pass that produced + // them; nothing here is derived for the report. + // + // `format` is the `--format` value as requested (`tar` when omitted); + // `targets` the canonical triple of every leg, the primary first. The + // stage fields are empty when no tree was staged; `closure` is the stage + // manifest's own word, `walked` or `not-walked`. + std::string format; + std::vector targets; + std::filesystem::path stageDir; + std::filesystem::path stageManifest; + std::string closure; + // Whether a build program ran in this pack, for the envelope's `effects`. + bool ranBuildPrograms = false; }; // #634 A3: the two directory lists an Android row's closure is read against, @@ -168,7 +183,13 @@ build_extra_android_legs(const std::string& targetName, // this leg's own plan and driver. SharedLeg leg; leg.abi = std::move(abi); + leg.triple = canonical; leg.artifact = ctx->outputDir / lu->output; + // #649 E5: what THIS leg's graph built, which pack strips as it strips + // the program; see `Plan::graphSharedLibraries`. + for (auto const& u : ctx->plan.linkUnits) + if (u.kind == mcpp::build::LinkUnit::SharedLibrary) + leg.graphSharedLibraries.push_back(ctx->outputDir / u.output); leg.searchDirs.push_back(leg.artifact.parent_path()); for (auto const& d : ctx->plan.runtimeLibraryDirs) leg.searchDirs.push_back(d); for (auto const& d : ctx->plan.linkIntent.runtimeSearchDirs) @@ -473,6 +494,12 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, // parameter comment on `extraLegs` above. A no-op (default-constructed, // empty) for every caller before this item. plan->extraSharedLegs = std::move(extraLegs); + // #649 E5: the shared libraries this graph built, primary leg. From the + // plan's link units, never from a directory listing: a vendor library + // deployed beside the program is not one of them and stays as shipped. + for (auto const& u : ctx->plan.linkUnits) + if (u.kind == mcpp::build::LinkUnit::SharedLibrary) + plan->graphSharedLibraries.push_back(ctx->outputDir / u.output); // The RESOLVED debug-information decision. On the plan, not in Options: // Options is the request, this is what it came out as once the manifest @@ -496,10 +523,14 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, }()), }; + // "stripped" STATES WHAT THIS ROW DOES, NOT WHAT WAS REQUESTED (#649 E5). + // It used to print the decision alone, so a Mach-O row, whose debug + // information is not in the image and is never stripped, and the Android + // row, which did not reach the strip step at all, both said "stripped". mcpp::ui::info("Packing", std::format("{} v{} ({}{})", plan->packageName, plan->packageVersion, mcpp::pack::mode_cli_name(plan->opts.mode), - plan->strip ? ", stripped" : "")); + mcpp::pack::strips_on_this_row(*plan) ? ", stripped" : "")); // STAGING IS A SERVICE TO THE PROVIDER, NOT A PRECONDITION FOR DISPATCH. // @@ -565,6 +596,34 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, // no tree exists, so no manifest describes a tree that is not there. if (stageFailure.empty()) mcpp::pack::write_stage_manifest(plan->stagingRoot, closure); + // #649 E9: the outcome a machine reader receives, from values this pass + // has already answered. Both returns below go through it. + auto outcome_with = [&](std::vector artifacts) { + PackOutcome o; + o.artifacts = std::move(artifacts); + o.format = opts.format == mcpp::pack::Format::Tar ? std::string("tar") + : opts.format == mcpp::pack::Format::Dir ? std::string("dir") + : opts.formatName; + const auto canonical = [](std::string const& t) { + if (t.empty()) return mcpp::toolchain::triple::host_triple().str(); + auto parsed = mcpp::toolchain::triple::parse(t); + return parsed ? parsed->str() : t; + }; + o.targets.push_back(canonical(plan->triple)); + for (auto const& leg : plan->extraSharedLegs) o.targets.push_back(leg.triple); + if (stageFailure.empty()) { + o.stageDir = plan->stagingRoot; + o.stageManifest = mcpp::pack::stage_manifest_path(plan->stagingRoot); + o.closure = closure.walked ? "walked" : "not-walked"; + } + std::error_code bec; + o.ranBuildPrograms = std::filesystem::exists(ctx->projectRoot / "build.mcpp", bec) + || !ctx->manifest.buildConfig.ruleModules.empty(); + for (auto const& sp : ctx->sourcePackages) + if (std::filesystem::exists(sp.root / "build.mcpp", bec)) o.ranBuildPrograms = true; + return o; + }; + auto pathCtx = mcpp::fetcher::make_path_ctx(&*cfg, ctx->projectRoot); // ─── The dispatch pass ─────────────────────────────────────────── @@ -595,6 +654,11 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, preexistingArtifacts.emplace(a.packageName, a.id); ov.pack_format = opts.formatName; + // #649 E5: the RESOLVED strip decision and debug directory, for a + // member that stages libraries of its own and must follow the same + // switch `--no-strip` and `--debug-symbols` set for this tree. + ov.pack_strip = plan->strip ? "1" : "0"; + ov.pack_debug_symbols_dir = plan->debugDir; // Empty when staging was refused, which is what makes // `${mcpp.stage_dir}` refuse with the reason attached rather than // expand to a directory that does not exist. @@ -710,13 +774,13 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, // Every output consumed by another: a cycle a provider should not // write, reported as all outputs rather than as nothing. if (reported.empty()) reported = std::move(intermediate); - return PackOutcome{0, std::move(reported)}; + return outcome_with(std::move(reported)); } auto outPath = (opts.format == mcpp::pack::Format::Tar) ? plan->archivePath : plan->stagingRoot; mcpp::ui::status("Packed", mcpp::ui::shorten_path(outPath, pathCtx)); - return PackOutcome{0, {outPath}}; + return outcome_with({outPath}); } } // namespace mcpp::pack diff --git a/src/pm/index_refresh.cppm b/src/pm/index_refresh.cppm index 591b5d819..e3c2efc81 100644 --- a/src/pm/index_refresh.cppm +++ b/src/pm/index_refresh.cppm @@ -41,50 +41,16 @@ import mcpp.log; import mcpp.platform; import mcpp.platform.axis; import mcpp.pm.dep_spec; +import mcpp.pm.dependency_selector; // legacy_bare_candidates import mcpp.pm.index_contract; import mcpp.pm.index_route; +export import mcpp.pm.refresh_policy; // the policy half, shared with the fetcher import mcpp.pm.resolver; import mcpp.ui; import mcpp.xlings; export namespace mcpp::pm { -// Why a refresh was, or was not, triggered. Every value is user-visible under -// `-v`, and the Suppressed* ones are the interesting half when diagnosing -// "why did/didn't mcpp go to the network". -enum class RefreshReason { - None, // resolvable locally — the steady state, zero network - IndexAbsent, // no local index at all (cold start) - DescriptorMiss, // package unknown locally, and the index can say so - VersionMiss, // package known, constraint unsatisfiable locally - SuppressedOffline, // --offline / MCPP_OFFLINE - SuppressedDisabled, // [index] auto_refresh = false - SuppressedDebounce, // refreshed moments ago; upstream simply lacks it - SuppressedInconclusive, // a miss here proves nothing (see header) - SuppressedIndexUnusable,// the index that would answer is too new to read - SuppressedMalformedDescriptor, // local bytes exist but violate identity -}; - -struct RefreshPolicy { - bool offline = false; - bool autoRefresh = true; - // Shared with the xim install gate — one constant, one rationale, in the - // leaf module both layers can see (mcpp::xlings). - std::int64_t debounceSeconds = mcpp::xlings::kIndexRefreshDebounceSeconds; -}; - -struct RefreshDecision { - bool shouldRefresh = false; - RefreshReason reason = RefreshReason::None; - std::string subject; // "mcpplibs:fmt@^1.3" — for logs and errors -}; - -// Human tail for the status line: "package index — ". -std::string_view reason_text(RefreshReason r); - -// flag > env > config. `offlineFlag` is the parsed `--offline`. -RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg); - // Pure: no network, no filesystem writes, no side effects. The judgement table // is the contract — `tests/unit/test_pm_index_refresh.cpp` locks it row by row. RefreshDecision decide_for_dependency(const IndexRoute& route, @@ -99,39 +65,12 @@ RefreshDecision decide_for_dependency(const IndexRoute& route, // nothing for the others, so callers use this to skip the sync entirely. bool routes_to_builtin(const IndexRoute& route, const DependencySpec& spec); -// For a caller that has ALREADY established a conclusive miss through -// `lookup_descriptor` (that is `mcpp add`): only the policy half of the -// judgement is left to make. Keeps the opt-outs in one place rather than -// re-tested at each call site. -RefreshDecision decide_for_miss(const RefreshPolicy& policy, - const mcpp::xlings::Env& env, - std::string_view subject); - -// Run the sync a decision asked for. At most ONE per process: a build whose -// deps miss for the same reason should pay one sync, not one per dep. -// Returns an error only when the sync itself failed; callers decide whether -// that is fatal (it is not, if the build can still resolve locally). -std::expected apply(const RefreshDecision& d, - const mcpp::xlings::Env& env); - -// Explicit user intent (`mcpp update`): ignores debounce and the once-per- -// process guard. Still refuses when offline, and says so. -std::expected force_refresh(const mcpp::xlings::Env& env); - -// One line of advisory context for a resolution failure: what the index is and -// how old it is. Never a gate — only ever appended to an error the user is -// already seeing. -std::string staleness_note(const mcpp::xlings::Env& env); - } // namespace mcpp::pm namespace mcpp::pm { namespace { -// One sync per process, however many dependencies ask for one. -bool g_refreshed_this_process = false; - // Echo back what the USER wrote (the `[dependencies]` key), not the resolved // coordinate. `xim.nasm` is parsed into the candidate `(mcpplibs.xim, nasm)`, // and reporting that spelling in a diagnostic sends the reader looking for a @@ -169,41 +108,8 @@ std::vector coords_of(std::string_view depKey, } }; } -std::string age_phrase(std::int64_t s) { - if (s < 0) return "never refreshed"; - if (s < 90) return std::format("refreshed {}s ago", s); - if (s < 5400) return std::format("refreshed {}m ago", s / 60); - if (s < 172800) return std::format("refreshed {}h ago", s / 3600); - return std::format("refreshed {}d ago", s / 86400); -} - } // namespace -std::string_view reason_text(RefreshReason r) { - switch (r) { - case RefreshReason::None: return "resolvable locally"; - case RefreshReason::IndexAbsent: return "no local package index"; - case RefreshReason::DescriptorMiss: return "not found locally"; - case RefreshReason::VersionMiss: return "not satisfiable locally"; - case RefreshReason::SuppressedOffline: return "offline mode"; - case RefreshReason::SuppressedDisabled: return "[index] auto_refresh = false"; - case RefreshReason::SuppressedDebounce: return "index was just refreshed"; - case RefreshReason::SuppressedInconclusive: return "no index can refute this"; - case RefreshReason::SuppressedIndexUnusable: - return "an index requires a newer mcpp — refreshing cannot help"; - case RefreshReason::SuppressedMalformedDescriptor: - return "descriptor identity is malformed — refreshing cannot help"; - } - return ""; -} - -RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg) { - RefreshPolicy p; - p.offline = mcpp::platform::env::offline_mode(); - p.autoRefresh = cfg.indexAutoRefresh; - return p; -} - RefreshDecision decide_for_dependency(const IndexRoute& route, std::string_view depKey, const DependencySpec& spec, @@ -223,6 +129,23 @@ RefreshDecision decide_for_dependency(const IndexRoute& route, // 3. INV-3: a miss only counts when the index could have refuted it. auto found = lookup_descriptor(route, coords); + + // 3b. THE RESOLVER'S LADDER, NOT A SHORTER ONE. After the exact coordinate + // misses, the resolver tries the deprecated bare-name rung for a + // version selector whose namespace was omitted (`ftxui = "6.1.9"` + // reaches `compat.ftxui`, prepare.cppm). A decision that stopped at the + // exact coordinate called that dependency missing while the resolver + // found it on disk, so every build of such a manifest started a network + // refresh once the debounce had passed (mcpp-community/mcpp#648: an + // editor planning in the background waited on one for 11 minutes). The + // same condition as the resolver's, for as long as the rung exists. + if (!found.hit && found.error.empty() && found.conclusive + && spec.isVersion() && spec.namespaceOmitted && !coords.empty()) { + auto legacy = lookup_descriptor( + route, mcpp::pm::legacy_bare_candidates(coords.front())); + if (legacy.hit) found = std::move(legacy); + } + if (!found.error.empty()) { d.reason = RefreshReason::SuppressedMalformedDescriptor; return d; @@ -299,84 +222,4 @@ bool routes_to_builtin(const IndexRoute& route, const DependencySpec& spec) { return coords_route_to_builtin(route, coords_of(spec.shortName, spec)); } -RefreshDecision decide_for_miss(const RefreshPolicy& policy, - const mcpp::xlings::Env& env, - std::string_view subject) { - RefreshDecision d; - d.subject = std::string(subject); - // See decide(): a refresh cannot fix an index this binary cannot read. - if (mcpp::pm::any_index_unusable()) { - d.reason = RefreshReason::SuppressedIndexUnusable; - return d; - } - if (policy.offline) { d.reason = RefreshReason::SuppressedOffline; return d; } - if (!policy.autoRefresh) { d.reason = RefreshReason::SuppressedDisabled; return d; } - // Same debounce as the build path: two `mcpp add` typos in a row should not - // buy two multi-repo syncs, for the same reason a build with two missing - // packages does not. - auto status = mcpp::xlings::default_index_status(env, policy.debounceSeconds); - if (status.present && status.ageSeconds >= 0 - && status.ageSeconds < policy.debounceSeconds) { - d.reason = RefreshReason::SuppressedDebounce; - return d; - } - d.reason = RefreshReason::DescriptorMiss; - d.shouldRefresh = true; - return d; -} - -namespace { - -std::expected run_sync(const mcpp::xlings::Env& env, - std::string_view banner) { - auto before = mcpp::xlings::default_index_status(env, 0).rev; - mcpp::ui::status("Refreshing", banner); - int rc = mcpp::xlings::update_index(env, /*quiet=*/true); - if (rc != 0) - return std::unexpected(std::format("package index refresh failed (rc {})", rc)); - auto after = mcpp::xlings::default_index_status(env, 0).rev; - if (before && after && *before != *after) - mcpp::ui::status("Updated", std::format("package index {} → {}", *before, *after)); - else if (after) - mcpp::log::verbose("index", std::format("package index still at {}", *after)); - return {}; -} - -} // namespace - -std::expected apply(const RefreshDecision& d, - const mcpp::xlings::Env& env) -{ - if (!d.shouldRefresh) { - mcpp::log::verbose("index", std::format( - "skip refresh for {}: {}", d.subject, reason_text(d.reason))); - return {}; - } - if (g_refreshed_this_process) { - mcpp::log::verbose("index", std::format( - "skip refresh for {}: already refreshed in this run", d.subject)); - return {}; - } - g_refreshed_this_process = true; - return run_sync(env, std::format("package index — `{}` {} (one-time)", - d.subject, reason_text(d.reason))); -} - -std::expected force_refresh(const mcpp::xlings::Env& env) { - if (mcpp::platform::env::offline_mode()) - return std::unexpected( - "offline mode is on — cannot refresh the package index " - "(unset MCPP_OFFLINE or drop --offline)"); - g_refreshed_this_process = true; - return run_sync(env, "package index (requested)"); -} - -std::string staleness_note(const mcpp::xlings::Env& env) { - auto st = mcpp::xlings::default_index_status(env, 0); - if (!st.present) return "no local package index"; - return st.rev - ? std::format("local index {} ({})", *st.rev, age_phrase(st.ageSeconds)) - : std::format("local index {}", age_phrase(st.ageSeconds)); -} - } // namespace mcpp::pm diff --git a/src/pm/package_fetcher.cppm b/src/pm/package_fetcher.cppm index 7cee61e21..e6286e624 100644 --- a/src/pm/package_fetcher.cppm +++ b/src/pm/package_fetcher.cppm @@ -20,6 +20,8 @@ import mcpp.manifest; // xpkg_lua_identity_matches — descriptor identit import mcpp.pm.compat; import mcpp.pm.dep_spec; import mcpp.pm.index_contract; // index.toml floor check (E0006) +import mcpp.pm.refresh_policy; // decide_for_miss / apply (#648 A5) +import mcpp.build.refusal; // offline-download-required (#648 A1) import mcpp.ui; import mcpp.pm.index_spec; import mcpp.xlings; @@ -1112,6 +1114,8 @@ Fetcher::resolve_xpkg_path(std::string_view target, // download attempt is refused, and it names the package so the user // knows what to fetch rather than seeing a generic network error. if (mcpp::platform::env::offline_mode()) { + mcpp::build::refusal::record( + mcpp::build::refusal::Code::OfflineDownloadRequired); return std::unexpected(CallError{ std::format( "offline mode: `{}@{}` is not installed and cannot be downloaded\n" " run without --offline (or unset MCPP_OFFLINE) to fetch it", @@ -1119,13 +1123,18 @@ Fetcher::resolve_xpkg_path(std::string_view target, } if (parsed.indexName == "xim") { mcpp::xlings::Env xlEnv{ cfg_.xlingsBinary, cfg_.xlingsHome() }; - // quiet=false: this only ever prints when a dependency is missing - // from the local index and we refresh once to fetch it — a rare, - // intentional event worth surfacing so a one-time network pause - // doesn't look like a silent hang. Steady-state builds (deps - // present) return early without a word. - mcpp::xlings::ensure_official_package_index_fresh( - xlEnv, parsed.packageName, cfg_.searchTtlSeconds, /*quiet=*/false); + // A descriptor missing from the local xim index may be one the + // remote index has. Whether that miss refreshes is the refresh + // policy's decision, so `[index] auto_refresh = false` and the + // debounce hold here as on every other path (#648 A5). The status + // line `apply` prints makes the one-time network pause visible. + if (!mcpp::xlings::official_package_present(xlEnv, parsed.packageName)) { + auto decision = mcpp::pm::decide_for_miss( + mcpp::pm::policy_for(cfg_), xlEnv, + std::format("xim:{}", parsed.packageName)); + if (auto r = mcpp::pm::apply(decision, xlEnv); !r) + mcpp::ui::warning(r.error()); + } } std::vector targets { @@ -1200,9 +1209,12 @@ Fetcher::resolve_xpkg_path(std::string_view target, if (parsed.indexName == "xim") { mcpp::xlings::Env refreshEnv{ cfg_.xlingsBinary, cfg_.xlingsHome() }; mcpp::log::verbose("fetcher", - std::format("install failed for {}; refreshing index before retry", + std::format("install failed for {}; asking the refresh policy before retry", targets[0])); - mcpp::xlings::update_index(refreshEnv, /*quiet=*/true); + auto decision = mcpp::pm::decide_for_miss( + mcpp::pm::policy_for(cfg_), refreshEnv, targets[0]); + if (auto r = mcpp::pm::apply(decision, refreshEnv); !r) + mcpp::log::verbose("fetcher", r.error()); } mcpp::log::verbose("fetcher", diff --git a/src/pm/refresh_policy.cppm b/src/pm/refresh_policy.cppm new file mode 100644 index 000000000..00b06c9fe --- /dev/null +++ b/src/pm/refresh_policy.cppm @@ -0,0 +1,212 @@ +// mcpp.pm.refresh_policy -- the policy half of "may this run refresh a package +// index now?", in a module the package fetcher can import. +// +// WHY THIS IS NOT PART OF mcpp.pm.index_refresh. That module answers the whole +// question for a dependency, and to do so it reads descriptors through +// mcpp.pm.index_route, which imports the fetcher. The fetcher could therefore +// not ask it, and it asked a second derivation instead: +// `ensure_official_package_index_fresh` before an install, and a bare +// `update_index` before a retry. Neither consulted `[index] auto_refresh`, +// which docs/05 documents as "never refresh the index automatically" +// (mcpp-community/mcpp#648 A5). The opt-outs, the debounce, the one-sync-per- +// process guard and the sync itself live here, and every caller that refreshes +// an index goes through `decide_for_miss` or `decide_for_dependency` and then +// `apply`. + +export module mcpp.pm.refresh_policy; + +import std; +import mcpp.config; +import mcpp.log; +import mcpp.platform; +import mcpp.pm.index_contract; +import mcpp.ui; +import mcpp.xlings; + +export namespace mcpp::pm { + +// Why a refresh was, or was not, triggered. Every value is user-visible under +// `-v`, and the Suppressed* ones are the interesting half when diagnosing +// "why did/didn't mcpp go to the network". +enum class RefreshReason { + None, // resolvable locally — the steady state, zero network + IndexAbsent, // no local index at all (cold start) + DescriptorMiss, // package unknown locally, and the index can say so + VersionMiss, // package known, constraint unsatisfiable locally + SuppressedOffline, // --offline / MCPP_OFFLINE + SuppressedDisabled, // [index] auto_refresh = false + SuppressedDebounce, // refreshed moments ago; upstream simply lacks it + SuppressedInconclusive, // a miss here proves nothing (see header) + SuppressedIndexUnusable,// the index that would answer is too new to read + SuppressedMalformedDescriptor, // local bytes exist but violate identity +}; + +struct RefreshPolicy { + bool offline = false; + bool autoRefresh = true; + // Shared with the xim install gate — one constant, one rationale, in the + // leaf module both layers can see (mcpp::xlings). + std::int64_t debounceSeconds = mcpp::xlings::kIndexRefreshDebounceSeconds; +}; + +struct RefreshDecision { + bool shouldRefresh = false; + RefreshReason reason = RefreshReason::None; + std::string subject; // "mcpplibs:fmt@^1.3" — for logs and errors +}; + +// Human tail for the status line: "package index — ". +std::string_view reason_text(RefreshReason r); + +// flag > env > config. `offlineFlag` is the parsed `--offline`. +RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg); + +// For a caller that has ALREADY established a conclusive miss through +// `lookup_descriptor` (that is `mcpp add`): only the policy half of the +// judgement is left to make. Keeps the opt-outs in one place rather than +// re-tested at each call site. +RefreshDecision decide_for_miss(const RefreshPolicy& policy, + const mcpp::xlings::Env& env, + std::string_view subject); + +// Run the sync a decision asked for. At most ONE per process: a build whose +// deps miss for the same reason should pay one sync, not one per dep. +// Returns an error only when the sync itself failed; callers decide whether +// that is fatal (it is not, if the build can still resolve locally). +std::expected apply(const RefreshDecision& d, + const mcpp::xlings::Env& env); + +// Explicit user intent (`mcpp update`): ignores debounce and the once-per- +// process guard. Still refuses when offline, and says so. +std::expected force_refresh(const mcpp::xlings::Env& env); + +// One line of advisory context for a resolution failure: what the index is and +// how old it is. Never a gate — only ever appended to an error the user is +// already seeing. +std::string staleness_note(const mcpp::xlings::Env& env); + +} // namespace mcpp::pm + +namespace mcpp::pm { + +namespace { + +// One sync per process, however many dependencies ask for one. +bool g_refreshed_this_process = false; + +std::string age_phrase(std::int64_t s) { + if (s < 0) return "never refreshed"; + if (s < 90) return std::format("refreshed {}s ago", s); + if (s < 5400) return std::format("refreshed {}m ago", s / 60); + if (s < 172800) return std::format("refreshed {}h ago", s / 3600); + return std::format("refreshed {}d ago", s / 86400); +} + +} // namespace + +std::string_view reason_text(RefreshReason r) { + switch (r) { + case RefreshReason::None: return "resolvable locally"; + case RefreshReason::IndexAbsent: return "no local package index"; + case RefreshReason::DescriptorMiss: return "not found locally"; + case RefreshReason::VersionMiss: return "not satisfiable locally"; + case RefreshReason::SuppressedOffline: return "offline mode"; + case RefreshReason::SuppressedDisabled: return "[index] auto_refresh = false"; + case RefreshReason::SuppressedDebounce: return "index was just refreshed"; + case RefreshReason::SuppressedInconclusive: return "no index can refute this"; + case RefreshReason::SuppressedIndexUnusable: + return "an index requires a newer mcpp — refreshing cannot help"; + case RefreshReason::SuppressedMalformedDescriptor: + return "descriptor identity is malformed — refreshing cannot help"; + } + return ""; +} + +RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg) { + RefreshPolicy p; + p.offline = mcpp::platform::env::offline_mode(); + p.autoRefresh = cfg.indexAutoRefresh; + return p; +} + +RefreshDecision decide_for_miss(const RefreshPolicy& policy, + const mcpp::xlings::Env& env, + std::string_view subject) { + RefreshDecision d; + d.subject = std::string(subject); + // See decide(): a refresh cannot fix an index this binary cannot read. + if (mcpp::pm::any_index_unusable()) { + d.reason = RefreshReason::SuppressedIndexUnusable; + return d; + } + if (policy.offline) { d.reason = RefreshReason::SuppressedOffline; return d; } + if (!policy.autoRefresh) { d.reason = RefreshReason::SuppressedDisabled; return d; } + // Same debounce as the build path: two `mcpp add` typos in a row should not + // buy two multi-repo syncs, for the same reason a build with two missing + // packages does not. + auto status = mcpp::xlings::default_index_status(env, policy.debounceSeconds); + if (status.present && status.ageSeconds >= 0 + && status.ageSeconds < policy.debounceSeconds) { + d.reason = RefreshReason::SuppressedDebounce; + return d; + } + d.reason = RefreshReason::DescriptorMiss; + d.shouldRefresh = true; + return d; +} + +namespace { + +std::expected run_sync(const mcpp::xlings::Env& env, + std::string_view banner) { + auto before = mcpp::xlings::default_index_status(env, 0).rev; + mcpp::ui::status("Refreshing", banner); + int rc = mcpp::xlings::update_index(env, /*quiet=*/true); + if (rc != 0) + return std::unexpected(std::format("package index refresh failed (rc {})", rc)); + auto after = mcpp::xlings::default_index_status(env, 0).rev; + if (before && after && *before != *after) + mcpp::ui::status("Updated", std::format("package index {} → {}", *before, *after)); + else if (after) + mcpp::log::verbose("index", std::format("package index still at {}", *after)); + return {}; +} + +} // namespace + +std::expected apply(const RefreshDecision& d, + const mcpp::xlings::Env& env) +{ + if (!d.shouldRefresh) { + mcpp::log::verbose("index", std::format( + "skip refresh for {}: {}", d.subject, reason_text(d.reason))); + return {}; + } + if (g_refreshed_this_process) { + mcpp::log::verbose("index", std::format( + "skip refresh for {}: already refreshed in this run", d.subject)); + return {}; + } + g_refreshed_this_process = true; + return run_sync(env, std::format("package index — `{}` {} (one-time)", + d.subject, reason_text(d.reason))); +} + +std::expected force_refresh(const mcpp::xlings::Env& env) { + if (mcpp::platform::env::offline_mode()) + return std::unexpected( + "offline mode is on — cannot refresh the package index " + "(unset MCPP_OFFLINE or drop --offline)"); + g_refreshed_this_process = true; + return run_sync(env, "package index (requested)"); +} + +std::string staleness_note(const mcpp::xlings::Env& env) { + auto st = mcpp::xlings::default_index_status(env, 0); + if (!st.present) return "no local package index"; + return st.rev + ? std::format("local index {} ({})", *st.rev, age_phrase(st.ageSeconds)) + : std::format("local index {}", age_phrase(st.ageSeconds)); +} + +} // namespace mcpp::pm diff --git a/src/runtime/elf.cppm b/src/runtime/elf.cppm index 8deaef932..e8b4a0640 100644 --- a/src/runtime/elf.cppm +++ b/src/runtime/elf.cppm @@ -137,16 +137,38 @@ struct DynamicSymbol { // dynamic symbol table therefore has exactly one cause — the linker // exported it so that some shared object's reference would bind to it. bool isFunc = false; - // STB_WEAK. A vague-linkage definition -- a template instantiation, an - // inline function, a vtable -- which the C++ ABI emits into every - // translation unit that needs it and expects the loader to unify across - // the process. That is the intended behaviour, not a leak, so a caller - // asking "is this image providing something twice" has to be able to tell - // it from a strong definition that displaces a library's own. + // A vague-linkage definition -- a template instantiation, an inline + // function, a vtable, or the static data member of such a template -- + // which the C++ ABI emits into every translation unit that needs it and + // expects the loader to unify across the process. That is the intended + // behaviour, not a leak, so a caller asking "is this image providing + // something twice" has to be able to tell it from a strong definition that + // displaces a library's own. + // + // TWO BINDINGS SAY IT, NOT ONE. `STB_WEAK` is the portable spelling; + // `STB_GNU_UNIQUE` is what GCC emits for the static data of an inline + // entity so that the loader keeps exactly one copy even across RTLD_LOCAL + // (libstdc++'s `__from_chars_alnum_to_val_table::value`, measured). + // Reading only the first reported seven such objects as a program + // displacing libstdc++ (#646 F3). See `is_vague_linkage_binding`. bool isWeak = false; std::uint64_t value = 0; // st_value; the key a copy relocation matches }; +// Is a symbol of this binding a vague-linkage definition the loader unifies? +// `STB_WEAK` (2) and `STB_GNU_UNIQUE` (10); every other binding is either +// strong (`STB_GLOBAL`) or not visible to the loader. +constexpr bool is_vague_linkage_binding(unsigned char binding) { + return binding == 2 || binding == 10; +} + +// The names an ELF RELOCATABLE object (`.o`) defines with external linkage: +// GLOBAL, WEAK and GNU_UNIQUE entries of `.symtab` whose section index is not +// SHN_UNDEF. Used to attribute a definition to the object that contributed it, +// which a linked image no longer records. +std::expected, std::string> +defined_object_symbols(const std::filesystem::path& object); + // What an ELF object's DYNAMIC symbol table says it provides. // // Deliberately not folded into ElfRuntimeFacts: every caller of @@ -739,7 +761,7 @@ inspect_dynamic_symbols(const std::filesystem::path& object) { out.defined.push_back(DynamicSymbol{ .name = std::move(*name), .isFunc = (type == detail::kSttFunc || type == detail::kSttGnuIfunc), - .isWeak = (bind == detail::kStbWeak), + .isWeak = is_vague_linkage_binding(bind), .value = *value, }); } @@ -770,6 +792,73 @@ inspect_dynamic_symbols(const std::filesystem::path& object) { return out; } +std::expected, std::string> +defined_object_symbols(const std::filesystem::path& object) { + detail::Reader reader; + std::ifstream input(object, std::ios::binary); + if (!input) return std::unexpected(std::format( + "cannot open ELF object '{}'", object.string())); + reader.bytes.assign(std::istreambuf_iterator(input), {}); + + if (reader.bytes.size() < 0x40 + || reader.bytes[0] != 0x7f || reader.bytes[1] != 'E' + || reader.bytes[2] != 'L' || reader.bytes[3] != 'F') + return std::unexpected(std::format( + "object '{}' is not ELF", object.string())); + if (reader.bytes[4] != 2 || reader.bytes[5] != 1) + return std::unexpected(std::format( + "object '{}' is not ELF64 little-endian", object.string())); + + // Section headers: e_shoff at 0x28, e_shentsize at 0x3A, e_shnum at 0x3C. + auto shoff = reader.u64(0x28); + auto shentsize = reader.u16(0x3A); + auto shnum = reader.u16(0x3C); + if (!shoff || !shentsize || !shnum || *shentsize < 0x40 || *shnum > 65000 + || !reader.range(*shoff, static_cast(*shentsize) * *shnum)) + return std::unexpected(std::format( + "object '{}' has a truncated section table", object.string())); + + constexpr std::uint32_t kShtSymtab = 2; + std::vector names; + for (std::uint16_t i = 0; i < *shnum; ++i) { + const auto at = *shoff + static_cast(i) * *shentsize; + auto type = reader.u32(at + 0x04); + if (!type || *type != kShtSymtab) continue; + auto offset = reader.u64(at + 0x18); + auto size = reader.u64(at + 0x20); + auto link = reader.u32(at + 0x28); + auto entsize = reader.u64(at + 0x38); + if (!offset || !size || !link || !entsize + || *entsize != detail::kSymEntrySize || *link >= *shnum + || !reader.range(*offset, *size)) + return std::unexpected(std::format( + "object '{}' has a malformed symbol table", object.string())); + const auto strAt = *shoff + static_cast(*link) * *shentsize; + auto strOffset = reader.u64(strAt + 0x18); + auto strSize = reader.u64(strAt + 0x20); + if (!strOffset || !strSize || !reader.range(*strOffset, *strSize)) + return std::unexpected(std::format( + "object '{}' has a malformed string table", object.string())); + for (std::uint64_t off = 0; off + detail::kSymEntrySize <= *size; + off += detail::kSymEntrySize) { + const auto sym = *offset + off; + auto nameOffset = reader.u32(sym); + auto shndx = reader.u16(sym + 6); + if (!nameOffset || !shndx) break; + if (*shndx == 0) continue; // SHN_UNDEF + const unsigned char bind = + static_cast(reader.bytes[sym + 4] >> 4); + if (bind == detail::kStbLocal) continue; + if (*nameOffset >= *strSize) continue; + auto name = reader.cstr(*strOffset + *nameOffset, *strSize - *nameOffset); + if (name && !name->empty()) names.push_back(std::move(*name)); + } + } + std::ranges::sort(names); + names.erase(std::ranges::unique(names).begin(), names.end()); + return names; +} + std::expected inspect_elf_runtime(const std::filesystem::path& artifact) { detail::Reader reader; diff --git a/src/wire.cppm b/src/wire.cppm index df94a93dc..63fce2479 100644 --- a/src/wire.cppm +++ b/src/wire.cppm @@ -50,6 +50,7 @@ export module mcpp.wire; import std; import mcpp.version; import mcpp.libs.json; +import mcpp.platform; // env::network_accessed (#648 A4) export namespace mcpp::wire { @@ -67,7 +68,7 @@ inline constexpr int kEnvelopeVersion = 1; // from what comes back. struct KindVersion { std::string_view kind; int version; }; -inline constexpr std::array kKinds{{ +inline constexpr std::array kKinds{{ {"mcpp.env", 1}, {"mcpp.xpkg", 1}, {"mcpp.cache", 1}, @@ -84,6 +85,12 @@ inline constexpr std::array kKinds{{ // (`data.watch`) and their digest (`data.inputs-fingerprint`). Written // into nothing; see docs/specs/build-database.md. {"mcpp.build-database", 1}, + // `mcpp pack --message-format json`: what the pack produced -- each + // artifact with its absolute path, whether it is a file or a directory, + // the format and the triples of its legs -- and the staged tree it was + // made from (`data.stage`). `--format` names the package format on that + // command, hence the `test`-style spelling (docs/50 §3). + {"mcpp.pack", 1}, }}; // What running a command does, beyond writing to stdout. @@ -181,6 +188,12 @@ inline nlohmann::json to_json(const Diagnostic& d) { inline nlohmann::json to_json(const Envelope& e) { nlohmann::json effects = nlohmann::json::array(); for (auto f : e.effects) effects.push_back(std::string(effect_name(f))); + // `effects` states what running the command did (docs/50 section 2). Network + // access is observed where it happens rather than declared by each command, + // so every enveloped command reports it the same way. + if (mcpp::platform::env::network_accessed() + && std::ranges::find(e.effects, Effect::Network) == e.effects.end()) + effects.push_back(std::string(effect_name(Effect::Network))); nlohmann::json diags = nlohmann::json::array(); for (auto const& d : e.diagnostics) diags.push_back(to_json(d)); diff --git a/src/xlings/xlings.cppm b/src/xlings/xlings.cppm index 20eca1e77..8089bb872 100644 --- a/src/xlings/xlings.cppm +++ b/src/xlings/xlings.cppm @@ -470,6 +470,9 @@ struct SeedRepo { std::string url; std::string artifact; // artifact source base, e.g. https://github.com/xlings-res/mcpp-index std::string source; // "auto" | "artifact" | "git" + // Non-empty: `artifact` is written as the region object + // {"GLOBAL": artifact, "CN": artifactCn} (xlings #377). + std::string artifactCn; }; void seed_xlings_json(const Env& env, @@ -521,6 +524,36 @@ std::optional find_sandbox_nasm(const Env& env); // imports this module — the reverse would be a cycle. inline constexpr std::int64_t kIndexRefreshDebounceSeconds = 120; +// ─── Bounds on the xlings children mcpp starts (#648 A3) ─────────────── +// +// Every xlings invocation on the planning path runs through +// `run_streaming_bounded`: the child owns a process group (a job object on +// Windows) that dies with mcpp, and it is bounded. Before, an `xlings update` +// on a connection that never answered held `emit build-database` for eleven +// minutes, and outlived the mcpp its caller had killed. +// +// Which bound fits depends on what the child says while it works: +// - the index refresh prints little and is small: a TOTAL bound, configured +// by `[index] refresh_timeout`. A timed-out refresh is a failed refresh, +// which the build already survives when the local index can answer; +// - an install through the NDJSON interface emits a heartbeat after five +// seconds of silence (xlings interface.cpp), so no output for +// kInterfaceIdleTimeout means the xlings process itself is wedged, while a +// slow download that makes progress is never cut short: an IDLE bound; +// - a direct `xlings install -y` has its output sent to the null device, so +// only a total bound can apply, and it is generous: a toolchain archive on a +// slow link is not a hang; +// - a local command (the one-time sandbox init) gets a total bound. +inline constexpr std::chrono::seconds kDefaultIndexRefreshTimeout{120}; +inline constexpr std::chrono::seconds kInterfaceIdleTimeout{300}; +inline constexpr std::chrono::seconds kDirectInstallTimeout{3 * 3600}; +inline constexpr std::chrono::seconds kLocalCommandTimeout{600}; + +// Set once from `[index] refresh_timeout` when the configuration is loaded; +// zero or negative restores the default. +void set_index_refresh_timeout(std::chrono::seconds timeout); +std::chrono::seconds index_refresh_timeout(); + // Check whether the default mcpplibs index data exists and is fresh // (within ttlSeconds). // Returns true if index is present and fresh, false otherwise. @@ -547,14 +580,12 @@ int update_index(const Env& env, bool quiet = false); // when no update is needed. void ensure_index_fresh(const Env& env, std::int64_t ttlSeconds, bool quiet = false); -// Ensure xlings' official xim index is present and fresh. -void ensure_official_index_fresh(const Env& env, std::int64_t ttlSeconds, bool quiet = false); - -// Ensure a specific package file exists in xlings' official xim index. -void ensure_official_package_index_fresh(const Env& env, - std::string_view packageName, - std::int64_t ttlSeconds, - bool quiet = false); +// Whether xlings' official xim index on disk carries a descriptor for +// `packageName`. Offline and read-only. Whether a miss may refresh the index is +// not decided here: that is mcpp.pm.refresh_policy's `decide_for_miss`, so +// `[index] auto_refresh`, `--offline` and the debounce apply to it as they do +// to every other refresh (mcpp-community/mcpp#648 A5). +bool official_package_present(const Env& env, std::string_view packageName); // ─── Index status (read-only, offline) ────────────────────────────── // Snapshot of a local index directory — computed without touching the @@ -1417,8 +1448,13 @@ call(const Env& env, std::string_view capability, mcpp::log::verbose("xlings", std::format("interface {} exec: {}", capability, cmd)); + // An install reaches the network; the query capabilities read local state. + if (capability == "install_packages" || capability == "update_packages") + mcpp::platform::env::note_network_access(); + CallResult result; - int rc = mcpp::platform::process::run_streaming(cmd, + bool timedOut = false; + int rc = mcpp::platform::process::run_streaming_bounded(cmd, [&](std::string_view line) { if (line.empty()) return; @@ -1443,7 +1479,16 @@ call(const Env& env, std::string_view capability, if (handler) handler->on_result(e); } }, *ev); - }); + }, + std::chrono::milliseconds{0}, + std::chrono::duration_cast(kInterfaceIdleTimeout), + &timedOut); + if (timedOut) { + result.exitCode = result.exitCode != 0 ? result.exitCode : 124; + result.stderrTail.push_back(std::format( + "xlings interface {} wrote nothing for {} seconds (not even its " + "heartbeat) and was stopped", capability, kInterfaceIdleTimeout.count())); + } if (rc != 0 && result.exitCode == 0) result.exitCode = rc; if (result.exitCode != 0) { // Error-level lines only, the last 20: enough to name a rejection, and @@ -1522,10 +1567,19 @@ int install_with_progress(const Env& env, std::string_view target, // Only when interactive (not quiet, stderr/stdout is a TTY). const bool showSpinner = !quiet && mcpp::platform::terminal::is_tty(); + mcpp::platform::env::note_network_access(); std::atomic done{false}; - int directRaw = 0; + int directRc = 0; std::thread worker([&] { - directRaw = std::system(directCmd.c_str()); + bool timedOut = false; + directRc = mcpp::platform::process::run_streaming_bounded( + directCmd, [](std::string_view) {}, + std::chrono::duration_cast(kDirectInstallTimeout), + std::chrono::milliseconds{0}, &timedOut); + if (timedOut) + mcpp::log::warn("xlings", std::format( + "`xlings install {}` did not finish within {} hours and was stopped", + target, kDirectInstallTimeout.count() / 3600)); done.store(true, std::memory_order_release); }); @@ -1552,7 +1606,6 @@ int install_with_progress(const Env& env, std::string_view target, } worker.join(); - int directRc = mcpp::platform::process::extract_exit_code(directRaw); if (directRc == 0) return 0; } @@ -1617,7 +1670,17 @@ int install_with_progress(const Env& env, std::string_view target, if (!prog.files.empty()) cb(prog); }; - int closeRc = mcpp::platform::process::run_streaming(cmd, handle_line); + bool idleTimedOut = false; + int closeRc = mcpp::platform::process::run_streaming_bounded( + cmd, handle_line, std::chrono::milliseconds{0}, + std::chrono::duration_cast(kInterfaceIdleTimeout), + &idleTimedOut); + if (idleTimedOut) { + mcpp::log::warn("xlings", std::format( + "xlings interface install_packages wrote nothing for {} seconds and was stopped", + kInterfaceIdleTimeout.count())); + return resultExitCode > 0 ? resultExitCode : 124; + } return (resultExitCode != -1) ? resultExitCode : closeRc; } @@ -1629,12 +1692,19 @@ int install_direct(const Env& env, std::string_view target, bool quiet) { cmd += " "; cmd += std::string(mcpp::platform::shell::silent_redirect); } - if constexpr (mcpp::platform::is_windows) { - cmd += " (kDirectInstallTimeout), + std::chrono::milliseconds{0}, &timedOut); + if (timedOut) + mcpp::log::warn("xlings", std::format( + "`xlings install {}` did not finish within {} hours and was stopped", + target, kDirectInstallTimeout.count() / 3600)); + return rc; } // ─── Sandbox lifecycle ────────────────────────────────────────────── @@ -1651,7 +1721,11 @@ void seed_xlings_json(const Env& env, json += std::format(" {{ \"name\": \"{}\", \"url\": \"{}\"", json_escape(repos[i].name), json_escape(repos[i].url)); - if (!repos[i].artifact.empty()) + if (!repos[i].artifact.empty() && !repos[i].artifactCn.empty()) + json += std::format(", \"artifact\": {{ \"GLOBAL\": \"{}\", \"CN\": \"{}\" }}", + json_escape(repos[i].artifact), + json_escape(repos[i].artifactCn)); + else if (!repos[i].artifact.empty()) json += std::format(", \"artifact\": \"{}\"", json_escape(repos[i].artifact)); if (!repos[i].source.empty()) @@ -1721,7 +1795,11 @@ void ensure_init(const Env& env, bool quiet) { ScopedInvocationEnv scope(globalEnv); // #614 std::string cmd = build_command_prefix(globalEnv) + " self init " + std::string(mcpp::platform::shell::silent_redirect); - int rc = mcpp::platform::process::run_silent(cmd); + bool initTimedOut = false; + int rc = mcpp::platform::process::run_streaming_bounded( + cmd, [](std::string_view) {}, + std::chrono::duration_cast(kLocalCommandTimeout), + std::chrono::milliseconds{0}, &initTimedOut); if (rc != 0 && !quiet) { std::println(stderr, "warning: `xlings self init` failed for sandbox at '{}'", @@ -1879,6 +1957,20 @@ int update_index_unguarded(const Env& env, bool quiet); // back — the refresh itself was the thing that broke the machine. See // mcpp.pm.index_snapshot for why the shape is archive/judge/restore rather // than the stage-and-swap the original design assumed. +namespace { +std::atomic g_index_refresh_timeout_s{kDefaultIndexRefreshTimeout.count()}; +} + +void set_index_refresh_timeout(std::chrono::seconds timeout) { + g_index_refresh_timeout_s.store( + timeout.count() > 0 ? timeout.count() : kDefaultIndexRefreshTimeout.count(), + std::memory_order_relaxed); +} + +std::chrono::seconds index_refresh_timeout() { + return std::chrono::seconds{g_index_refresh_timeout_s.load(std::memory_order_relaxed)}; +} + int update_index(const Env& env, bool quiet) { namespace snap = mcpp::pm::index_snapshot; const auto dataRoot = paths::index_data(env); @@ -1944,6 +2036,7 @@ int update_index_unguarded(const Env& env, bool quiet) { } std::string cmd = build_command_prefix(env) + " update 2>&1"; + mcpp::platform::env::note_network_access(); // The index sync is a network git operation; a single transient blip (DNS, // TLS reset, a mirror hiccup) otherwise fails a cold `mcpp self env` / // first-run init outright (e.g. CI's index/sandbox bootstrap). Retry with @@ -1951,12 +2044,28 @@ int update_index_unguarded(const Env& env, bool quiet) { // added latency in steady state; only a genuine failure pays the backoff. constexpr int kMaxAttempts = 3; int rc = 0; + const auto refreshBound = index_refresh_timeout(); for (int attempt = 1; attempt <= kMaxAttempts; ++attempt) { - rc = mcpp::platform::process::run_streaming(cmd, + bool timedOut = false; + rc = mcpp::platform::process::run_streaming_bounded(cmd, [quiet](std::string_view line) { if (!quiet) std::println("{}", line); - }); - if (rc == 0) { mark_known_indexes_refreshed(env); return 0; } + }, + std::chrono::duration_cast(refreshBound), + std::chrono::milliseconds{0}, &timedOut); + if (rc == 0 && !timedOut) { mark_known_indexes_refreshed(env); return 0; } + // A refresh that exceeded its bound is not retried: the retries exist for + // a transient failure that ends, and a connection that never answers + // would only be waited on three times. The caller treats the refresh as + // failed and resolves from the local index, as it does for any failure. + if (timedOut) { + std::println(stderr, + "warning: the package index refresh did not finish within {} seconds and was " + "stopped; continuing with the local index " + "(the bound is [index] refresh_timeout in mcpp's config.toml)", + refreshBound.count()); + return rc != 0 ? rc : 124; + } if (attempt < kMaxAttempts) { int delay = attempt * 2; // 2s, then 4s mcpp::log::verbose("index", std::format( @@ -1978,43 +2087,10 @@ void ensure_index_fresh(const Env& env, std::int64_t ttlSeconds, bool quiet) { update_index(env, /*quiet=*/true); } -void ensure_official_index_fresh(const Env& env, std::int64_t ttlSeconds, bool quiet) { - if (is_official_index_fresh(env, ttlSeconds)) return; - if (!quiet) - print_status("Updating", "package index (auto-refresh)"); - update_index(env, /*quiet=*/true); -} - -void ensure_official_package_index_fresh(const Env& env, - std::string_view packageName, - [[maybe_unused]] std::int64_t ttlSeconds, - bool quiet) { - // Offline-first, miss-triggered. We do NOT auto-update just because a TTL - // expired — that runs a network `xlings update` (git-syncs several index - // repos) that stalls for minutes on slow/blocked networks (the Termux - // first-run / build hang). But fully offline is too strict: if a requested - // dependency is NOT in the local index, we DO refresh once to discover it. - // - // present locally → use as-is, zero network (the common build case). - // missing locally → refresh once to try to fetch it. - // - // Routine, deps-already-present refresh stays the user's explicit - // `mcpp index update` / `xlings update`. +bool official_package_present(const Env& env, std::string_view packageName) { auto pkg = official_package_file(env, packageName); - if (!pkg.empty() && std::filesystem::exists(pkg)) return; - - // The package is missing locally. Refresh once — but guard against a build - // that resolves several genuinely-absent packages re-running the heavy - // `xlings update` per package: if the index was refreshed moments ago and - // the package is STILL missing, upstream simply lacks it; re-pulling won't - // help. (A package added upstream before this run lands in that one pull.) - if (is_official_index_fresh(env, kIndexRefreshDebounceSeconds)) return; - - if (!quiet) - print_status("Refreshing", - std::format("package index — `{}` not found locally (one-time)", - packageName)); - update_index(env, /*quiet=*/quiet); + std::error_code ec; + return !pkg.empty() && std::filesystem::exists(pkg, ec); } } // namespace mcpp::xlings diff --git a/tests/e2e/151_index_url_artifact_migration.sh b/tests/e2e/151_index_url_artifact_migration.sh index 1ad8c7453..21c9728ae 100755 --- a/tests/e2e/151_index_url_artifact_migration.sh +++ b/tests/e2e/151_index_url_artifact_migration.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# requires: python3 # #267/#269: fresh init seeds the mcpplibs-org index URL + artifact source; # legacy config.toml / .xlings.json (old org URL, no artifact) are healed in # place, idempotently, in both pretty (mcpp writer) and compact (xlings @@ -13,6 +14,10 @@ export MCPP_HOME="$TMP/mcpp-home" NEW_URL='https://github.com/mcpplibs/mcpp-index.git' OLD_URL='https://github.com/mcpp-community/mcpp-index.git' ART='https://github.com/xlings-res/mcpp-index' +ART_CN='https://gitcode.com/xlings-res/mcpp-index' +# The default artifact is a region object since 2026.9.16.1 (#648 A6): the +# GitHub base for every mirror and the GitCode base a CN mirror asks first. +REGION="{ \"GLOBAL\": \"$ART\", \"CN\": \"$ART_CN\" }" CFG="$MCPP_HOME/config.toml" XJ="$MCPP_HOME/registry/.xlings.json" @@ -21,11 +26,16 @@ XJ="$MCPP_HOME/registry/.xlings.json" # assert on the key/value pairs, not on mcpp's seed line layout.) "$MCPP" self env > /dev/null grep -q "url = \"$NEW_URL\"" "$CFG" || { echo "config.toml missing new url"; exit 1; } -grep -q "artifact = \"$ART\"" "$CFG" || { echo "config.toml missing artifact"; exit 1; } +grep -qF "artifact = { GLOBAL = \"$ART\", CN = \"$ART_CN\" }" "$CFG" \ + || { echo "config.toml missing the region artifact"; cat "$CFG"; exit 1; } grep -q "\"url\": \"$NEW_URL\"" "$XJ" \ || { echo "seeded .xlings.json missing new url"; cat "$XJ"; exit 1; } -grep -q "\"artifact\": \"$ART\"" "$XJ" \ - || { echo "seeded .xlings.json missing artifact"; cat "$XJ"; exit 1; } +python3 - "$XJ" "$ART" "$ART_CN" <<'PYEOF' || { echo "seeded .xlings.json missing the region artifact"; cat "$XJ"; exit 1; } +import json, sys +doc = json.load(open(sys.argv[1])) +entry = next(r for r in doc["index_repos"] if r["name"] == "mcpplibs") +assert entry["artifact"] == {"GLOBAL": sys.argv[2], "CN": sys.argv[3]}, entry +PYEOF if grep -q 'mcpp-community/mcpp-index' "$CFG" "$XJ"; then echo "old org URL leaked into fresh seed"; exit 1 fi @@ -54,7 +64,7 @@ cat > "$XJ" < /dev/null grep -q "url = \"$NEW_URL\"" "$CFG" || { echo "config.toml url not healed"; cat "$CFG"; exit 1; } -grep -q "\"url\": \"$NEW_URL\", \"artifact\": \"$ART\"" "$XJ" \ +grep -qF "\"url\": \"$NEW_URL\", \"artifact\": $REGION" "$XJ" \ || { echo ".xlings.json not healed"; cat "$XJ"; exit 1; } grep -q '"subos": "default"' "$XJ" || { echo "unrelated .xlings.json key lost"; exit 1; } if grep -q 'mcpp-community/mcpp-index' "$CFG" "$XJ"; then @@ -72,6 +82,27 @@ printf '{"index_repos":[{"name":"mcpp-index","url":"%s"}],"mirror":"auto"}' "$OL "$MCPP" self env > /dev/null grep -q '"name":"mcpplibs"' "$XJ" || { echo "compact name not healed"; cat "$XJ"; exit 1; } grep -q 'xlings-res/mcpp-index' "$XJ" || { echo "compact artifact not injected"; cat "$XJ"; exit 1; } +grep -q 'gitcode.com/xlings-res/mcpp-index' "$XJ" || { echo "compact artifact has no CN half"; cat "$XJ"; exit 1; } + +# 4. A home seeded before the region object (flat GitHub base, both spacings) +# gains the CN half, once; a base the user wrote is left alone. +printf '{\n "index_repos": [\n { "name": "mcpplibs", "url": "%s", "artifact": "%s" }\n ],\n "mirror": "CN"\n}\n' "$NEW_URL" "$ART" > "$XJ" +"$MCPP" self env > /dev/null +grep -qF "\"artifact\": $REGION" "$XJ" || { echo "flat default artifact not upgraded"; cat "$XJ"; exit 1; } +cp "$XJ" "$TMP/xj4"; "$MCPP" self env > /dev/null +cmp -s "$XJ" "$TMP/xj4" || { echo "region upgrade not idempotent"; diff "$TMP/xj4" "$XJ"; exit 1; } +# A base the user states in config.toml owns the entry, and has no CN half. +cat > "$CFG" < /dev/null +grep -q 'https://mirror.example/idx' "$XJ" || { echo "a user artifact base was rewritten"; cat "$XJ"; exit 1; } +if grep -q 'gitcode.com/xlings-res/mcpp-index' "$XJ"; then echo "a user artifact base gained a CN half"; cat "$XJ"; exit 1; fi if grep -q 'mcpp-community' "$XJ"; then echo "compact old org URL survived"; exit 1 fi diff --git a/tests/e2e/187_dep_host_tool.sh b/tests/e2e/187_dep_host_tool.sh index 9ebb23680..af9b99958 100755 --- a/tests/e2e/187_dep_host_tool.sh +++ b/tests/e2e/187_dep_host_tool.sh @@ -221,4 +221,54 @@ cd nsapp out="$("$MCPP" run 2>&1 | grep '^G=' | tail -1)" [[ "$out" == "G=5" ]] || { echo "FAIL: namespaced tool did not generate: $out"; exit 1; } +# ── a package that writes `namespace` and `name` apart (#647 E4.3) ───────── +# `myns.tp` above is the legacy dotted name, whose `package.name` is already the +# qualified spelling. A manifest writing `namespace = "myns2"`, `name = "tp2"` +# has `package.name == "tp2"`, and the tool used to be published under that +# spelling alone, so `dep_bin("myns2.tp2", ...)` read nothing while +# `dep_dir("myns2.tp2")` answered. +cd "$TMP" +mkdir -p ns2/src +cat > ns2/mcpp.toml <<'EOF' +[package] +name = "tp2" +namespace = "myns2" +version = "0.1.0" + +[targets.gen] +kind = "bin" +main = "src/gen.cpp" +EOF +cp ns/src/gen.cpp ns2/src/gen.cpp +mkdir -p ns2app/src +cat > ns2app/mcpp.toml <<'EOF' +[package] +name = "ns2app" +version = "0.1.0" + +[dependencies] +myns2.tp2 = { path = "../ns2", tools = ["gen"] } +EOF +printf '#include \nint gv();\nint main(){std::printf("G2=%%d\\n",gv());}\n' > ns2app/src/main.cpp +cat > ns2app/build.mcpp <<'EOF' +#include +#include +#include +import mcpp; +int main() { + const char* full = mcpp::dep_bin("myns2.tp2", "gen"); + const char* brief = mcpp::dep_bin("tp2", "gen"); + if (!*full) { std::fprintf(stderr, "qualified spelling did not resolve\n"); return 1; } + if (!*brief) { std::fprintf(stderr, "name spelling did not resolve\n"); return 1; } + std::string out = std::string(mcpp::out_dir()) + "/g.cpp"; + std::string cmd = std::string("\"") + full + "\" \"" + out + "\""; + if (std::system(cmd.c_str()) != 0) return 1; + mcpp::generated(out.c_str()); +} +EOF +cd ns2app +"$MCPP" build > b7.log 2>&1 || { cat b7.log; echo "FAIL: a namespace + name package's tool was not addressable by its qualified name"; exit 1; } +out="$("$MCPP" run 2>&1 | grep '^G2=' | tail -1)" +[[ "$out" == "G2=5" ]] || { echo "FAIL: the namespace + name tool did not generate: $out"; exit 1; } + echo "OK" diff --git a/tests/e2e/307_symbol_provision.sh b/tests/e2e/307_symbol_provision.sh index c62678919..11ffbc208 100755 --- a/tests/e2e/307_symbol_provision.sh +++ b/tests/e2e/307_symbol_provision.sh @@ -1,22 +1,23 @@ #!/usr/bin/env bash -# requires: elf +# requires: elf python3 # 307_symbol_provision.sh — one library, one provider (issue #519). # # THE SILENT CASE IS THE IMPORTANT ONE, and it is why this test has two -# halves that differ by a single package. +# halves. # -# mcpp's own `kind = "shared"` mechanism produces the exact shape the -# diagnostic looks for: a shared dependency's link unit takes only ITS OWN -# objects, so a static package underneath it lands in the consumer's -# executable and the shared library binds back to it at run time. There is one -# copy of the code in the process and nothing is wrong. A check that reported -# "this image exports a symbol a library it loads binds to" would fire on -# every correct build of that shape, and the user could do nothing about it. +# A shared dependency over a static package used to put the static package in +# the consumer's executable, and the shared library bound back to it at run +# time; this test then asserted that the executable's export stayed silent. +# Since #646 F1 the static package is linked INTO the shared library, which is +# the only arrangement that works on every format, so the executable exports +# nothing for it and the check still measures the image (half 1). # -# The finding is the SECOND provider, not the export. So: +# The finding is a SECOND provider of a name the executable holds. So: # -# half 1 wrap.so (shared) → core (static) must stay SILENT -# half 2 ... plus alt.so, which defines the same name must REPORT both +# half 1 app -> wrap.so (shared) -> core (static) must stay SILENT, +# with core linked into wrap.so +# half 2 app -> core (static) and alt.so, which defines the same name and +# calls it must REPORT both # # Asserted against `resolution.json` rather than stdout: the message is free # to improve, the recorded verdict is the contract. @@ -105,16 +106,20 @@ status="$(verdict "$res" status)" echo "FAIL: the legitimate shared→static arrangement reported '$status'" cat "$res"; exit 1; } -# It must have been MEASURED, not skipped: the export is real and the -# denominator is real. "0 findings" and "never looked" must not read alike. +# It must have been MEASURED, not skipped: the denominator is real. "0 +# findings" and "never looked" must not read alike. exported="$(verdict "$res" exported)" total="$(verdict "$res" dynamic_symbols)" -[[ "$exported" -ge 1 ]] || { - echo "FAIL: expected the executable to export shared_answer, got $exported" - exit 1; } [[ "$total" -gt "$exported" ]] || { echo "FAIL: implausible denominator ($exported of $total)"; exit 1; } +# core lives in wrap.so now, not in the executable (#646 F1): a host that never +# linked core loads it. +wrapso="$(ls "$(dirname "$res")"/bin/libwrap.so 2>/dev/null | head -1)" +[[ -n "$wrapso" ]] || { echo "FAIL: no libwrap.so"; exit 1; } +"$PYTHON" -c 'import ctypes, os, sys; assert ctypes.CDLL(sys.argv[1], mode=os.RTLD_NOW).wrap_call() == 8' "$wrapso" \ + || { echo "FAIL: a host that did not link core cannot load libwrap.so"; exit 1; } + grep -qi "also provided by" silent.log && { echo "FAIL: a correct build printed a conflict warning"; exit 1; } @@ -144,14 +149,14 @@ c_standard = "c11" kind = "bin" main = "src/main.c" [dependencies] -wrap = { path = "../wrap" } +core = { path = "../core" } alt = { path = "../alt" } EOF cat > src/main.c <<'EOF' #include -int wrap_call(void); +int shared_answer(void); int alt_ping(void); -int main(void) { printf("%d %d\n", wrap_call(), alt_ping()); return 0; } +int main(void) { printf("%d %d\n", shared_answer(), alt_ping()); return 0; } EOF "$MCPP" build > conflict.log 2>&1 || { cat conflict.log; exit 1; } @@ -186,11 +191,12 @@ done cat conflict.log; exit 1; } # The defect is real, not theoretical: alt's 999 is unreachable because the -# executable's merged copy wins for every caller in the process. +# executable's copy wins for every caller in the process, alt's own call +# included. cbin="$conflict_dir/bin/app" [[ -x "$cbin" ]] || { echo "FAIL: no executable in $conflict_dir"; exit 1; } -"$cbin" | grep -qx "8 7" || { - echo "FAIL: expected '8 7' (the merged copy winning), got: $("$cbin")"; exit 1; } +"$cbin" | grep -qx "7 7" || { + echo "FAIL: expected '7 7' (the executable's copy winning), got: $("$cbin")"; exit 1; } # ── --strict turns the finding into a failure, in ONE place ──────────────── touch src/main.c diff --git a/tests/e2e/700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh b/tests/e2e/700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh new file mode 100755 index 000000000..be2af16dc --- /dev/null +++ b/tests/e2e/700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh @@ -0,0 +1,114 @@ +#!/usr/bin/env bash +# requires: llvm elf +# 700 -- one process, one C++ runtime (#646 F3a), read with llvm@22.1.8. +# +# The ELF defaults gave a program `self-contained` and a shared library +# `toolchain-coupled`. A program that loads a C++ shared library therefore held +# a static libc++ and `libc++.so.1`: the executable exported the runtime symbols +# the library referenced, the library bound some of them there and kept the +# rest, and the program aborted the first time the library formatted a string: +# +# libc++abi: terminating due to uncaught exception of type std::bad_cast +# +# (measured on 2026.9.15.2, exit 134, the build itself green). An unstated +# program contract now takes the shared library's when the program loads a C++ +# shared library of the build. Three legs: the default shape runs, with the +# program coupled to the same runtime and the record saying so; `--strict` +# passes, so the symbol-provision check no longer reports the module +# initialiser both images link; a stated self-contained program over the +# coupled library is refused before compiling, with its reason. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME=${MCPP_HOME:-$HOME/.mcpp} + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +READELF=$(command -v readelf || true) +[ -n "$READELF" ] || READELF=$(ls "$MCPP_HOME"/registry/data/xpkgs/xim-x-llvm/22.1.8/bin/llvm-readelf 2>/dev/null | head -1) +[ -n "$READELF" ] || fail "no readelf and no llvm-readelf to read NEEDED with" + +cd "$TMP" +mkdir -p lib/src app/src +cat > lib/mcpp.toml <<'TOML' +[package] +name = "lib" +version = "0.1.0" + +[targets.lib] +kind = "shared" +TOML +cat > lib/src/lib.cppm <<'CPP' +export module lib; +import std; +export [[gnu::visibility("default")]] std::string lib_greet(int n); +CPP +cat > lib/src/lib.cpp <<'CPP' +module lib; +import std; +std::string lib_greet(int n) { return std::format("lib-{}", n); } +CPP + +write_app() { # $1 = extra [build] lines + cat > app/mcpp.toml < app/src/main.cpp <<'CPP' +import std; +import lib; +int main() { std::println("{}", lib_greet(3)); } +CPP + +# ── The default shape runs, coupled to one runtime ────────────────────────── +write_app '' +cd app +"$MCPP" build --strict > default.log 2>&1 \ + || fail "the default shape did not build under --strict" default.log +dir=$(ls -d target/*/*/ | head -1) +out=$("$dir/bin/app" 2>&1) || fail "the program aborted: $out" default.log +[ "$out" = "lib-3" ] || fail "expected lib-3, got: $out" +"$READELF" -d "$dir/bin/app" | grep -q 'Shared library: \[libc++.so.1\]' \ + || fail "the program does not load libc++.so.1, so it carries its own runtime" default.log +grep -q '"distributable": *"toolchain-coupled"' "$dir/resolution.json" \ + || fail "resolution.json does not record the program as toolchain-coupled" "$dir/resolution.json" +grep -q 'also provided by a library it loads' default.log \ + && fail "the symbol-provision check reported the shared shape" default.log +echo "ok: a program over a C++ shared library runs on one C++ runtime" +cd .. + +# ── A stated self-contained program over the coupled library is refused ───── +write_app 'cxx_runtime = { default = "self-contained", shared = "toolchain-coupled" }' +cd app +rm -rf target +if "$MCPP" build > split.log 2>&1; then + fail "a stated self-contained program over a coupled C++ shared library was not refused" split.log +fi +grep -q "state a self-contained C++ runtime and load the C++ shared library 'lib'" split.log \ + || fail "the refusal does not name the statement and the library" split.log +grep -q 'cxx_runtime = { shared = "self-contained" }' split.log \ + || fail "the refusal does not name the private-copy remedy" split.log +ninja=$(ls target/*/*/build.ninja 2>/dev/null | head -1 || true) +[ -z "$ninja" ] || ! grep -q '^build bin/liblib.so' "$ninja" \ + || fail "a build graph was written before the refusal" "$ninja" +if command -v jq >/dev/null 2>&1; then + "$MCPP" why toolchain --format json > why.json 2>/dev/null || true + reason=$(jq -r '.data.reason // empty' why.json 2>/dev/null || true) + [ "$reason" = "program-cxx-runtime-split" ] \ + || fail "the machine reason was '$reason', expected program-cxx-runtime-split" why.json +fi +echo "ok: a stated self-contained program over a coupled C++ shared library is refused" + +echo "PASS: 700 a program over a C++ shared library has one C++ runtime" diff --git a/tests/e2e/701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh b/tests/e2e/701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh new file mode 100755 index 000000000..e9c2c1387 --- /dev/null +++ b/tests/e2e/701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +# requires: elf python3 +# 701 -- the symbol-provision check reports what the build did not intend +# (#646 F3), read with the default toolchain. +# +# A program over a C++ shared library, both importing `std`, reported symbols +# "provided twice" that nothing in the project had done wrong: +# +# * 882 to 900 libstdc++ symbols under the default contracts: the program +# carried a static libstdc++ beside the library's libstdc++.so.6 (#646 +# F3a, now one runtime); +# * under one runtime, the `std` module initialiser (`_ZGIW3std`), which +# every C++ image importing `std` links from one `std.o`, and which GCC 16's +# libstdc++.so.6 exports as well; +# * seven STB_GNU_UNIQUE objects, which the loader unifies by design. +# +# Every finding is a degradation, so `--strict` failed the build. This asserts +# the default shape is clean in the record, passes `--strict` and runs; the +# real duplicate the check exists for is 307's second half. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +PYTHON="$(command -v python3 || command -v python || true)" +[[ -n "$PYTHON" ]] || { echo "skip: no python for JSON assertions"; exit 0; } + +cd "$TMP" +mkdir -p lib/src app/src +cat > lib/mcpp.toml <<'TOML' +[package] +name = "lib" +version = "0.1.0" + +[targets.lib] +kind = "shared" +TOML +cat > lib/src/lib.cppm <<'CPP' +export module lib; +import std; +export [[gnu::visibility("default")]] std::string lib_greet(int n); +CPP +cat > lib/src/lib.cpp <<'CPP' +module lib; +import std; +std::string lib_greet(int n) { return std::format("lib-{}", n); } +CPP +cat > app/mcpp.toml <<'TOML' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +lib = { path = "../lib" } +TOML +cat > app/src/main.cpp <<'CPP' +import std; +import lib; +int main() { std::println("{}", lib_greet(3)); } +CPP + +cd app +"$MCPP" build --strict > strict.log 2>&1 \ + || fail "a program over a C++ shared library failed --strict" strict.log +grep -q 'also provided by a library it loads' strict.log \ + && fail "the check reported the shape the build creates by construction" strict.log +dir=$(ls -d target/*/*/ | head -1) +out=$("$dir/bin/app" 2>&1) || fail "the program exited non-zero: $out" +[ "$out" = "lib-3" ] || fail "expected lib-3, got: $out" + +verdict=$("$PYTHON" - "$dir/resolution.json" <<'PY' +import json, sys +doc = json.load(open(sys.argv[1])) +entries = doc.get("runtime", {}).get("symbol_provision") or [] +app = [e for e in entries if e["path"].endswith("bin/app")] +if not app: + print("NO-APP-ENTRY") +else: + e = app[0] + print(e.get("status"), e.get("exported", 0), e.get("dynamic_symbols", 0), + len(e.get("conflicts", []))) +PY +) +set -- $verdict +[ "$1" = "clean" ] || fail "the record says '$verdict', expected a clean verdict" "$dir/resolution.json" +# Measured, not skipped: the program does export what the library binds to, and +# the denominator is real. "clean" must not be the reading of a check that did +# not look. +[ "${2:-0}" -ge 1 ] || fail "the program exports nothing, so the check did not measure the shape" "$dir/resolution.json" +[ "${3:-0}" -gt "${2:-0}" ] || fail "implausible denominator: $verdict" "$dir/resolution.json" +echo "ok: the default shape is clean in the record and passes --strict" + +echo "PASS: 701 symbol provision does not report what the build shares by construction" diff --git a/tests/e2e/702_a_static_package_under_a_shared_library_is_linked_into_it.sh b/tests/e2e/702_a_static_package_under_a_shared_library_is_linked_into_it.sh new file mode 100755 index 000000000..bdef35978 --- /dev/null +++ b/tests/e2e/702_a_static_package_under_a_shared_library_is_linked_into_it.sh @@ -0,0 +1,152 @@ +#!/usr/bin/env bash +# requires: unix-shell python3 +# 702 -- one static package, one image (#646 F1). +# +# A dependency's shared library was linked from its own package's objects only. +# A static package beneath it went into the PROGRAM, and on ELF the library +# bound to the program's copy at run time. That worked for that program alone: +# measured on Linux, the same `libfw.so` refused `-Wl,-z,defs` and a host that +# did not link the package could not load it (`undefined symbol: x_answer`); +# Mach-O and PE resolve every reference at link time, so the library did not +# link there at all. +# +# Three legs: +# A. fw (shared) over x (static): x is linked into libfw, which then loads +# from a host that never linked x (Python's ctypes, RTLD_NOW); +# B. the program depends on x as well: no single image can own x. On ELF the +# build proceeds as it always did and says so (`--strict` fails); on +# Mach-O the build is refused before compiling, naming the remedy; +# C. the remedy the message names, `linkage = "shared"` on x, builds under +# `--strict` and runs. +set -e + +case "$(uname -s)" in + MINGW*|MSYS*|CYGWIN*) echo "skip: the PE legs of 702 are not wired on this shard"; exit 0 ;; +esac + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +PYTHON="$(command -v python3 || command -v python || true)" +[[ -n "$PYTHON" ]] || fail "no python to load the library from a foreign host" + +MACHO=0 +[ "$(uname -s)" = "Darwin" ] && MACHO=1 +SO=so +[ $MACHO = 1 ] && SO=dylib + +cd "$TMP" +mkdir -p x/src fw/src app/src +cat > x/mcpp.toml <<'TOML' +[package] +name = "x" +version = "0.1.0" + +[build] +sources = ["src/*.c"] + +[targets.x] +kind = "lib" +TOML +cat > x/src/x.c <<'C' +int x_answer(void) { return 41; } +C +cat > fw/mcpp.toml <<'TOML' +[package] +name = "fw" +version = "0.1.0" + +[build] +sources = ["src/*.c"] + +[targets.fw] +kind = "shared" + +[dependencies] +x = { path = "../x" } +TOML +cat > fw/src/fw.c <<'C' +extern int x_answer(void); +__attribute__((visibility("default"))) int fw_answer(void) { return x_answer() + 1; } +C + +write_app() { # $1 = extra [dependencies] lines, $2 = main body + cat > app/mcpp.toml < app/src/main.cpp +} + +# ── A. x belongs in libfw ──────────────────────────────────────────────────── +write_app '' 'extern "C" int fw_answer(void); +int main() { return fw_answer() == 42 ? 0 : 1; }' +cd app +"$MCPP" build > a.log 2>&1 || fail "a shared library over a static package did not build" a.log +dir=$(ls -d target/*/*/ | head -1) +"$dir/bin/app" || fail "the program over libfw exited non-zero" a.log +lib="$dir/bin/libfw.$SO" +[ -f "$lib" ] || fail "no libfw.$SO" a.log +loaded=$("$PYTHON" - "$lib" <<'PY' 2>&1 +import ctypes, os, sys +lib = ctypes.CDLL(sys.argv[1], mode=os.RTLD_NOW) +print(lib.fw_answer()) +PY +) || fail "a host that never linked x cannot load libfw: $loaded" a.log +[ "$loaded" = "42" ] || fail "expected 42 from libfw in a foreign host, got: $loaded" +echo "ok: a static package reachable only through a shared library is linked into it" +cd .. + +# ── B. the program reaches x too ──────────────────────────────────────────── +write_app 'x = { path = "../x" }' 'extern "C" int fw_answer(void); +extern "C" int x_answer(void); +int main() { return fw_answer() == 42 && x_answer() == 41 ? 0 : 1; }' +cd app +rm -rf target +if [ $MACHO = 1 ]; then + if "$MCPP" build > b.log 2>&1; then + fail "a static package in two Mach-O images was not refused" b.log + fi + grep -q "a static package is linked into more than one image" b.log \ + || fail "the refusal does not say what it refuses" b.log + grep -q "'mcpplibs.x' is reached by the program, 'fw'" b.log \ + || fail "the refusal does not name the package and its images" b.log + grep -q 'linkage = "shared"' b.log || fail "the refusal does not name the remedy" b.log + echo "ok: a static package in two Mach-O images is refused before compiling" +else + "$MCPP" build > b.log 2>&1 || fail "the two-image graph no longer builds on ELF" b.log + grep -q "a static package is reachable from more than one image" b.log \ + || fail "the ELF build does not report the two-image package" b.log + grep -q "'mcpplibs.x' is reached by the program, 'fw'" b.log \ + || fail "the report does not name the package and its images" b.log + dir=$(ls -d target/*/*/ | head -1) + "$dir/bin/app" || fail "the two-image program exited non-zero" b.log + rm -rf target + if "$MCPP" build --strict > b-strict.log 2>&1; then + fail "--strict accepted a static package in two images" b-strict.log + fi + echo "ok: a static package in two ELF images builds as before and is reported" +fi +cd .. + +# ── C. the remedy the message names ───────────────────────────────────────── +write_app 'x = { path = "../x", linkage = "shared" }' 'extern "C" int fw_answer(void); +extern "C" int x_answer(void); +int main() { return fw_answer() == 42 && x_answer() == 41 ? 0 : 1; }' +cd app +rm -rf target +"$MCPP" build --strict > c.log 2>&1 || fail "linkage = \"shared\" on x did not build under --strict" c.log +grep -q "more than one image" c.log && fail "the shared form still reports the two-image package" c.log +dir=$(ls -d target/*/*/ | head -1) +[ -f "$dir/bin/libx.$SO" ] || fail "no libx.$SO under linkage = \"shared\"" c.log +"$dir/bin/app" || fail "the program over the shared x exited non-zero" c.log +echo "ok: linkage = \"shared\" gives the package one image" + +echo "PASS: 702 a static package under a shared library is linked into it" diff --git a/tests/e2e/703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh b/tests/e2e/703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh new file mode 100755 index 000000000..a5017bc22 --- /dev/null +++ b/tests/e2e/703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +# requires: windows +# 703 -- the C++ runtime record of clang on the MSVC ABI (#649 E10). +# +# mcpp emits a CRT model (`/MT` or `/MD`) only for cl.exe. clang on the MSVC ABI +# receives none, and its driver links the static CRT (`-defaultlib:libcmt`), so +# the program imports no vcruntime or ucrt DLL. The contract table was written +# for cl.exe and recorded `host-coupled` for that artifact; an explicit +# `cxx_runtime = "host-coupled"` was recorded and not delivered, with no word. +# The record now states `self-contained`, and the explicit request prints that +# the row does not deliver it. Nothing about the artifact changes. +# +# Read only when the default toolchain on this runner is the llvm row: a runner +# whose default is msvc@system prints that and asserts nothing, because the +# cl.exe cells are unchanged and covered elsewhere. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'CPP' +import std; +int main() { std::println("crt"); } +CPP +write_app() { # $1 = [build] lines + cat > app/mcpp.toml < default.log 2>&1 || fail "the default build failed" default.log +if ! grep -q "Resolved llvm@" default.log; then + echo "READING E10: the default toolchain here is not the llvm row: $(grep -m1 'Resolved' default.log)" + echo "PASS: 703 the llvm row on the MSVC ABI records its static CRT (not the llvm row; nothing to assert)" + exit 0 +fi +contract=$(record_of) +echo "READING E10 default record: distributable=$contract" +[ "$contract" = "self-contained" ] \ + || fail "the llvm row recorded '$contract' for a program linked with the static CRT" default.log + +cd .. +write_app 'cxx_runtime = "host-coupled"' +cd app +rm -rf target +"$MCPP" build > host.log 2>&1 || fail "the host-coupled build failed" host.log +grep -q 'is not delivered for clang on the MSVC ABI' host.log \ + || fail "an explicit host-coupled request on the llvm row printed nothing" host.log +contract=$(record_of) +echo "READING E10 host-coupled record: distributable=$contract" +[ "$contract" = "self-contained" ] \ + || fail "the undelivered request was recorded as '$contract'" host.log +echo "ok: the llvm row records the static CRT it links, and says an explicit dynamic request is not delivered" + +echo "PASS: 703 the llvm row on the MSVC ABI records its static CRT" diff --git a/tests/e2e/704_cxx_identity_across_macho_images.sh b/tests/e2e/704_cxx_identity_across_macho_images.sh new file mode 100755 index 000000000..efbefe4a7 --- /dev/null +++ b/tests/e2e/704_cxx_identity_across_macho_images.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash +# requires: macos +# 704 -- a measurement leg (#646 F2): on Mach-O, the payload's default contract +# gives every dylib a private libc++ linked with `-load_hidden`. This prints, +# for a program over a C++ dylib under three runtime statements, whether a +# standard exception thrown in the dylib is caught by its class, whether the +# dylib's own exception class is, and whether an `std::error_code` compares +# equal across the boundary. See `_cxx_identity_across_images_body.sh`. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +IMAGE_FORMAT=macho +source "$(dirname "$0")/_cxx_identity_across_images_body.sh" + +echo "PASS: 704 C++ identity across Mach-O images was measured" diff --git a/tests/e2e/705_cxx_identity_across_pe_images.sh b/tests/e2e/705_cxx_identity_across_pe_images.sh new file mode 100755 index 000000000..5747e6f90 --- /dev/null +++ b/tests/e2e/705_cxx_identity_across_pe_images.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash +# requires: windows +# 705 -- a measurement leg (#646 F2, #649 E10): on the MSVC ABI with the default +# toolchain, every DLL links its own C and C++ runtime. This prints, for a +# program over a C++ DLL under three runtime statements, whether a standard +# exception thrown in the DLL is caught by its class, whether the DLL's own +# exception class is, and whether an `std::error_code` compares equal across the +# boundary. See `_cxx_identity_across_images_body.sh`. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +IMAGE_FORMAT=pe +source "$(dirname "$0")/_cxx_identity_across_images_body.sh" + +echo "PASS: 705 C++ identity across PE images was measured" diff --git a/tests/e2e/710_a_forward_reaches_every_dependency_table.sh b/tests/e2e/710_a_forward_reaches_every_dependency_table.sh new file mode 100755 index 000000000..2d391a70f --- /dev/null +++ b/tests/e2e/710_a_forward_reaches_every_dependency_table.sh @@ -0,0 +1,167 @@ +#!/usr/bin/env bash +# requires: gcc +# 710 -- a feature forward is validated against every dependency table of its +# manifest (#647 E4.1). +# +# `injectForwards` applies a `[features]` forward to `[build-dependencies]` +# edges, while the validator used to look the key up in `[dependencies]` and +# `[dev-dependencies]` only. A two-level forward along build-dependency edges +# therefore opened the leaf's feature (its tool was built) and printed +# "not declared" for each level in the same run, and `--strict` refused the +# build whose forward had worked. A key declared only for another row is also +# declared: on this row the forward reaches no edge. +# +# Readings: the two-level forward builds the tool under `--strict` with no +# forward warning; a forward to a dependency declared for another row builds +# under `--strict`; a forward to a key no table declares still warns, and +# `--strict` refuses it. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p kt/src leaf/src rules/src fw/src app/src win/src +cat > kt/mcpp.toml <<'EOF' +[package] +name = "kt" +namespace = "spike" +version = "0.1.0" + +[targets.kt] +kind = "bin" +main = "src/main.cpp" +EOF +echo 'int main() { return 0; }' > kt/src/main.cpp + +cat > leaf/mcpp.toml <<'EOF' +[package] +name = "leaf" +namespace = "spike" +version = "0.1.0" + +[targets.leaf] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[features] +kt = [] + +[feature-deps.kt] +spike.kt = { path = "../kt", tools = ["kt"] } +EOF +echo 'int leaf_answer() { return 1; }' > leaf/src/leaf.cpp + +cat > rules/mcpp.toml <<'EOF' +[package] +name = "rules" +namespace = "spike" +version = "0.1.0" + +[targets.rules] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[build-dependencies] +spike.leaf = { path = "../leaf" } + +[features] +kotlin = ["spike.leaf/kt"] +EOF +echo 'int rules_answer() { return 1; }' > rules/src/rules.cpp + +cat > win/mcpp.toml <<'EOF' +[package] +name = "win" +namespace = "spike" +version = "0.1.0" + +[targets.win] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[features] +x = [] +EOF +echo 'int win_answer() { return 1; }' > win/src/win.cpp + +cat > fw/mcpp.toml <<'EOF' +[package] +name = "fw" +namespace = "spike" +version = "0.1.0" + +[targets.fw] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[build-dependencies] +spike.rules = { path = "../rules" } + +[target.'cfg(os = "windows")'.dependencies] +spike.win = { path = "../win" } + +[features] +kotlin = ["spike.rules/kotlin", "spike.win/x"] +EOF +echo 'int fw_answer() { return 42; }' > fw/src/fw.cpp + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw = { path = "../fw", features = ["kotlin"] } +EOF +cat > app/src/main.cpp <<'EOF' +int fw_answer(); +int main() { return fw_answer() == 42 ? 0 : 1; } +EOF + +# ── 1. two levels along [build-dependencies], strict, no forward warning ── +cd "$TMP/app" +"$MCPP" build --strict > b1.log 2>&1 || fail "the two-level build-dependency forward was refused under --strict" b1.log +grep -q "forwards to dependency" b1.log \ + && fail "a forward along a build-dependency edge was reported as undeclared" b1.log +grep -q "Building host tool kt" b1.log \ + || fail "the forwarded feature did not reach the leaf (its tool was not built)" b1.log + +# ── 2. a forward to a dependency declared only for another row ─────────── +# `spike.win` is declared for Windows only (this script runs where gcc is the +# host compiler, which is never a Windows row). The forward names it, reaches +# no edge here, and must be accepted: the manifest is portable, not wrong. +grep -q "spike.win" b1.log && fail "a row-declared dependency's forward was reported" b1.log + +# ── 3. a key no table declares is still named, and --strict refuses it ─── +sed -i.bak 's|kotlin = \["spike.rules/kotlin", "spike.win/x"\]|kotlin = ["spike.rules/kotlin", "spike.nowhere/x"]|' "$TMP/fw/mcpp.toml" +rm -rf target +"$MCPP" build > b3.log 2>&1 || fail "an undeclared forward failed a non-strict build" b3.log +grep -q "forwards to dependency 'spike.nowhere'" b3.log \ + || fail "a forward to an undeclared key was not named" b3.log +if "$MCPP" build --strict > b4.log 2>&1; then + fail "--strict accepted a forward to a key no table declares" b4.log +fi +grep -q "spike.nowhere" b4.log || fail "the --strict refusal does not name the key" b4.log + +echo "PASS: 710 a forward is validated against every dependency table" diff --git a/tests/e2e/711_a_feature_deps_restatement_names_one_source.sh b/tests/e2e/711_a_feature_deps_restatement_names_one_source.sh new file mode 100755 index 000000000..bdb924d8b --- /dev/null +++ b/tests/e2e/711_a_feature_deps_restatement_names_one_source.sh @@ -0,0 +1,112 @@ +#!/usr/bin/env bash +# requires: gcc +# 711 -- a `[feature-deps.]` entry that adds `tools` to a dependency declared +# unconditionally restates that dependency's source, and a restatement naming +# another source is refused (#647 E4.2). A build program addresses the tool of a +# package written `namespace = "spike"`, `name = "installer"` by the qualified +# name as well as the bare one (#647 E4.3). +# +# The grammar requires the source in a dependency table, so a source-less entry +# is refused by the parser, whose message prescribes restating it. The merge +# took only the additive fields of a restatement: one naming another path was +# ignored without a word, under `--strict` too. And the tool variable was +# published under `package.name` alone, so `dep_bin("spike.installer", ...)` +# read nothing while `dep_dir("spike.installer")` answered. +# +# Readings: the source-less entry is refused and names the remedy; the +# restatement with the same source builds the tool only under the feature, and +# the build program sees both spellings; a restatement naming another path is +# refused under `--strict` naming both sources. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p installer/src other/src app/src +for d in installer other; do +cat > $d/mcpp.toml <<'EOF' +[package] +name = "installer" +namespace = "spike" +version = "0.1.0" + +[targets.installer] +kind = "bin" +main = "src/main.cpp" +EOF +cat > $d/src/main.cpp <<'EOF' +#include +int main() { std::puts("installer ran"); return 0; } +EOF +done + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.installer = { path = "../installer" } + +[features] +installer = [] + +[feature-deps.installer] +spike.installer = { tools = ["installer"] } +EOF +echo 'int main() { return 0; }' > app/src/main.cpp +cat > app/build.mcpp <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("installer", "installer"); + const char* b = mcpp::dep_bin("spike.installer", "installer"); + mcpp::warning((std::string("DEPBIN short=[") + (a ? a : "") + "] qualified=[" + + (b ? b : "") + "]").c_str()); + return 0; +} +EOF + +cd "$TMP/app" + +# ── 1. a source-less entry is refused, and the refusal names the remedy ── +if "$MCPP" build --features installer > b1.log 2>&1; then + fail "a source-less [feature-deps] entry was accepted" b1.log +fi +grep -q "source" b1.log || fail "the refusal does not say to restate the source" b1.log + +# ── 2. the same source restated: the tool, both spellings, only under the feature +sed -i.bak 's|spike.installer = { tools = \["installer"\] }|spike.installer = { path = "../installer", tools = ["installer"] }|' mcpp.toml +"$MCPP" build --strict --features installer > b2.log 2>&1 \ + || fail "the restated feature-deps entry was refused under --strict" b2.log +grep -q 'DEPBIN short=\[[^]]' b2.log || fail "dep_bin(\"installer\") read nothing" b2.log +grep -q 'qualified=\[[^]]' b2.log \ + || fail "dep_bin(\"spike.installer\") read nothing for a namespace + name package" b2.log +rm -rf target +"$MCPP" build --strict > b3.log 2>&1 || fail "the build without the feature failed" b3.log +grep -q 'DEPBIN short=\[\] qualified=\[\]' b3.log \ + || fail "the tool was provided without its feature" b3.log + +# ── 3. a restatement naming another source is refused, naming both ─────── +sed -i.bak 's|spike.installer = { path = "../installer", tools|spike.installer = { path = "../other", tools|' mcpp.toml +rm -rf target +if "$MCPP" build --strict --features installer > b4.log 2>&1; then + fail "a restatement naming another path was accepted" b4.log +fi +grep -q '../other' b4.log && grep -q '../installer' b4.log \ + || fail "the refusal does not name both sources" b4.log + +echo "PASS: 711 a feature-deps restatement names one source; dep_bin answers both spellings" diff --git a/tests/e2e/712_a_tool_package_contributes_nothing_to_its_consumer.sh b/tests/e2e/712_a_tool_package_contributes_nothing_to_its_consumer.sh new file mode 100755 index 000000000..8a59f1aa6 --- /dev/null +++ b/tests/e2e/712_a_tool_package_contributes_nothing_to_its_consumer.sh @@ -0,0 +1,200 @@ +#!/usr/bin/env bash +# requires: gcc +# 712 -- a package whose declared targets are all programs contributes nothing +# to a consumer's graph; its tools come from the tool sub-build (#649 E6). +# +# An SDK provides a program behind a feature, and the program depends on the +# SDK. The tool sub-build resolves the program's package as its own root, so +# the program and the SDK it links are built there. The consumer's graph used +# to walk the program package's `[dependencies]` as well, which made the +# declaration a cycle of the consumer's graph ("dependency cycle through +# package 'fw' while computing its build-cache key", default cache only) and +# linked a tool's own library into the application. +# +# Readings: the SDK fixture builds with the default cache, the tool runs, and +# the build program reads its path under both spellings; a tool's own library +# is not linked into the application; a cycle among libraries is refused with +# the same message under both cache modes; a tool whose own build activates +# the feature that requests it is refused once, naming the repetition. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +# fixture +fixture() { + local W=$1 + mkdir -p "$W/fw/src" "$W/fw/tool/src" "$W/app/src" "$W/z/src" + cat > "$W/fw/mcpp.toml" <<'EOF' +[package] +name = "fw" +namespace = "spike" +version = "0.1.0" + +[targets.fw] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[features] +installer = [] + +[feature-deps.installer] +spike.fw-installer = { path = "tool", tools = ["fw-installer"], reexport = true } +EOF + echo 'int fw_answer() { return 42; }' > "$W/fw/src/fw.cpp" + cat > "$W/fw/tool/mcpp.toml" < "$W/z/mcpp.toml" <<'EOF' +[package] +name = "z" +namespace = "spike" +version = "0.1.0" + +[targets.z] +kind = "lib" + +[build] +sources = ["src/*.cpp"] +EOF + echo 'int z_only_in_the_tool() { return 7; }' > "$W/z/src/z.cpp" + cat > "$W/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw = { path = "../fw" } + +[features] +windows-installer = ["spike.fw/installer"] +EOF + cat > "$W/app/src/main.cpp" <<'EOF' +int fw_answer(); +int main() { return fw_answer() == 42 ? 0 : 1; } +EOF + cat > "$W/app/build.mcpp" <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("fw-installer", "fw-installer"); + const char* b = mcpp::dep_bin("spike.fw-installer", "fw-installer"); + mcpp::warning((std::string("DEPBIN short=[") + (a ? a : "") + "] qualified=[" + + (b ? b : "") + "]").c_str()); + return 0; +} +EOF +} + +# ── 1. the SDK shape: the tool depends on the package declaring it ──────── +fixture "$TMP/s1" 'spike.fw = { path = ".." }' +cat > "$TMP/s1/fw/tool/src/main.cpp" <<'EOF' +#include +int fw_answer(); +int main() { std::printf("fw-installer ran %d\n", fw_answer()); return 0; } +EOF +cd "$TMP/s1/app" +"$MCPP" build --features windows-installer > b1.log 2>&1 \ + || fail "a feature tool depending on its declaring package did not build" b1.log +tool=$(grep -o 'DEPBIN short=\[[^]]*' b1.log | sed 's/DEPBIN short=\[//') +[ -n "$tool" ] || fail "dep_bin(\"fw-installer\") read nothing" b1.log +grep -q 'qualified=\[[^]]' b1.log || fail "dep_bin(\"spike.fw-installer\") read nothing" b1.log +out=$("$tool") || fail "the tool did not run: $out" +[ "$out" = "fw-installer ran 42" ] || fail "the tool printed '$out'" +"$MCPP" run > r1.log 2>&1 || fail "the application does not run" r1.log + +# ── 2. a tool's own library does not reach the application ────────────── +fixture "$TMP/s2" 'spike.z = { path = "../../z" }' +cat > "$TMP/s2/fw/tool/src/main.cpp" <<'EOF' +#include +int z_only_in_the_tool(); +int main() { std::printf("tool %d\n", z_only_in_the_tool()); return 0; } +EOF +cd "$TMP/s2/app" +"$MCPP" build --features windows-installer > b2.log 2>&1 || fail "the tool-with-a-library build failed" b2.log +ninja_file=$(find target -name build.ninja | head -1) +grep '^build bin/app' "$ninja_file" | grep -q 'spike_z' \ + && fail "the tool's own library was linked into the application" "$ninja_file" +grep -q 'spike_fw-installer' "$ninja_file" \ + && fail "the tool package's sources were compiled in the consumer's build" "$ninja_file" + +# ── 3. a cycle among libraries is refused under both cache modes ───────── +mkdir -p "$TMP/s3/a/src" "$TMP/s3/b/src" "$TMP/s3/app/src" +for p in a b; do + q=$([ "$p" = a ] && echo b || echo a) + cat > "$TMP/s3/$p/mcpp.toml" < "$TMP/s3/$p/src/$p.cpp" +done +cat > "$TMP/s3/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.a = { path = "../a" } +EOF +echo 'int main() { return 0; }' > "$TMP/s3/app/src/main.cpp" +cd "$TMP/s3/app" +for mode in "" "--cache=local"; do + if "$MCPP" build $mode > b3.log 2>&1; then + fail "a cycle of packages was accepted (${mode:-default cache})" b3.log + fi + grep -q "dependency cycle: 'spike.a' -> 'spike.b' -> 'spike.a'" b3.log \ + || fail "the cycle refusal does not name its edges (${mode:-default cache})" b3.log +done + +# ── 4. a tool whose own build requests it again is refused once ────────── +fixture "$TMP/s4" 'spike.fw = { path = "..", features = ["installer"] }' +echo 'int main() { return 0; }' > "$TMP/s4/fw/tool/src/main.cpp" +cd "$TMP/s4/app" +if "$MCPP" build --features windows-installer > b4.log 2>&1; then + fail "a tool requesting itself through its own build was accepted" b4.log +fi +grep -q "is requested by its own build" b4.log || fail "the repetition is not named" b4.log +[ "$(grep -o 'building host tool' b4.log | wc -l | tr -d ' ')" -le 1 ] \ + || fail "the repetition was refused only after nested sub-builds" b4.log + +echo "PASS: 712 a package of programs contributes nothing to its consumer's graph" diff --git a/tests/e2e/713_a_git_dependency_selects_a_repository_member.sh b/tests/e2e/713_a_git_dependency_selects_a_repository_member.sh new file mode 100755 index 000000000..695dbfe61 --- /dev/null +++ b/tests/e2e/713_a_git_dependency_selects_a_repository_member.sh @@ -0,0 +1,172 @@ +#!/usr/bin/env bash +# requires: gcc +# 713 -- a git dependency whose key names a package other than the repository's +# root package selects the `[workspace] members` entry that declares it +# (#649 E7). A second declaration of one dependency by one consumer keeps its +# requests, and the compile banner of a git dependency names its reference. +# +# A framework repository holds the framework, its rules and its tools, and a +# consumer pins it by revision before it is published. The git source always +# yielded the root package, so a key naming a member was either refused as a +# name mismatch or, beside a key naming the root, adopted the root's identity +# with a warning and lost its `tools` request. The same early return dropped +# the requests of a dependency named in `[dependencies]` and again in +# `[build-dependencies]`. +# +# Readings: the member alone builds its tool; the root and the member by the +# same revision build without an identity warning and with the member's tool; +# the banner prints the reference instead of an empty version; a dependency +# declared in both tables gets the tool the second declaration requests. +set -e + +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +command -v git >/dev/null || { echo "SKIP: git is not installed"; exit 0; } + +mkdir -p repo/src repo/tool/src +cat > repo/mcpp.toml <<'EOF' +[package] +name = "fw" +namespace = "spike" +version = "0.1.0" + +[targets.fw] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[workspace] +members = ["tool"] +EOF +echo 'int fw_answer() { return 42; }' > repo/src/fw.cpp +cat > repo/tool/mcpp.toml <<'EOF' +[package] +name = "fw-installer" +namespace = "spike" +version = "0.1.0" + +[targets.fw-installer] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw = { path = ".." } +EOF +cat > repo/tool/src/main.cpp <<'EOF' +#include +int fw_answer(); +int main() { std::printf("fw-installer ran %d\n", fw_answer()); return 0; } +EOF +( + cd repo + git init -q + git -c user.email=e2e@mcpp.dev -c user.name=e2e add -A + git -c user.email=e2e@mcpp.dev -c user.name=e2e commit -qm fixture +) +REV=$(git -C repo rev-parse HEAD) +REPO_HOST=$(host_path "$TMP/repo") + +write_app() { + mkdir -p "$TMP/$1/src" + cat > "$TMP/$1/mcpp.toml" + echo 'int main() { return 0; }' > "$TMP/$1/src/main.cpp" + cat > "$TMP/$1/build.mcpp" <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("spike.fw-installer", "fw-installer"); + mcpp::warning((std::string("DEPBIN tool=[") + (a ? a : "") + "]").c_str()); + return 0; +} +EOF +} + +# ── 1. the member alone ───────────────────────────────────────────────── +write_app member < b1.log 2>&1 || fail "a git dependency naming a repository member was refused" b1.log +tool=$(grep -o 'DEPBIN tool=\[[^]]*' b1.log | sed 's/DEPBIN tool=\[//') +[ -n "$tool" ] || fail "the member's tool was not provided" b1.log +[ "$("$tool")" = "fw-installer ran 42" ] || fail "the member's tool did not run" + +# ── 2. the root and the member by the same revision ───────────────────── +write_app both < "$TMP/both/src/main.cpp" <<'EOF' +int fw_answer(); +int main() { return fw_answer() == 42 ? 0 : 1; } +EOF +cd "$TMP/both" +"$MCPP" build > b2.log 2>&1 || fail "the root and a member by one revision did not build" b2.log +grep -q "that identity is used" b2.log \ + && fail "the member's key adopted the root's identity" b2.log +grep -q 'DEPBIN tool=\[[^]]' b2.log || fail "the member's tools request was lost" b2.log +grep -q "Compiling spike.fw v$" b2.log && fail "the git banner printed an empty version" b2.log +grep -q "spike.fw (git rev ${REV:0:12})" b2.log \ + || fail "the git banner does not name the reference" b2.log +"$MCPP" run > r2.log 2>&1 || fail "the application over the git root does not run" r2.log + +# ── 3. one dependency in [dependencies] and in [build-dependencies] ───── +mkdir -p "$TMP/local/src" +cp -r "$TMP/repo/tool/." "$TMP/local/" +sed -i.bak 's|spike.fw = { path = ".." }||' "$TMP/local/mcpp.toml" +cat > "$TMP/local/src/main.cpp" <<'EOF' +#include +int main() { std::puts("local tool"); return 0; } +EOF +write_app twice <<'EOF' +[package] +name = "twice" +version = "0.1.0" + +[targets.twice] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw-installer = { path = "../local" } + +[build-dependencies] +spike.fw-installer = { path = "../local", tools = ["fw-installer"] } +EOF +cd "$TMP/twice" +"$MCPP" build > b3.log 2>&1 || fail "a dependency declared in two tables did not build" b3.log +grep -q 'DEPBIN tool=\[[^]]' b3.log \ + || fail "the second declaration's tools request was dropped" b3.log + +echo "PASS: 713 a git dependency selects a repository member; a second declaration keeps its requests" diff --git a/tests/e2e/714_features_names_a_dependency_feature.sh b/tests/e2e/714_features_names_a_dependency_feature.sh new file mode 100755 index 000000000..4a9df96e0 --- /dev/null +++ b/tests/e2e/714_features_names_a_dependency_feature.sh @@ -0,0 +1,121 @@ +#!/usr/bin/env bash +# requires: gcc +# 714 -- `--features /` opens a feature of a dependency, as +# the same token does inside `[features]` (#649 E8). +# +# The command line accepted root features only. With no `[features]` table the +# token became the macro `-DMCPP_FEATURE_SPIKE_FW_INSTALLER` and opened nothing; +# with the table it was reported as an undeclared feature of the root. A plain +# name without a `[features]` table keeps its documented meaning, a macro. +# +# Readings: without a `[features]` table the dependency's tool is built and no +# macro is made of the token; with the table the build is clean under +# `--strict`; a token naming no dependency is refused under `--strict`; and +# `mcpp why deps --features` reports the graph of the feature build. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p fw/src fw/tool/src app/src +cat > fw/mcpp.toml <<'EOF' +[package] +name = "fw" +namespace = "spike" +version = "0.1.0" + +[targets.fw] +kind = "lib" + +[build] +sources = ["src/*.cpp"] + +[features] +installer = [] + +[feature-deps.installer] +spike.fw-installer = { path = "tool", tools = ["fw-installer"], reexport = true } +EOF +echo 'int fw_answer() { return 42; }' > fw/src/fw.cpp +cat > fw/tool/mcpp.toml <<'EOF' +[package] +name = "fw-installer" +namespace = "spike" +version = "0.1.0" + +[targets.fw-installer] +kind = "bin" +main = "src/main.cpp" +EOF +echo 'int main() { return 0; }' > fw/tool/src/main.cpp + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +spike.fw = { path = "../fw" } +EOF +cat > app/src/main.cpp <<'EOF' +int fw_answer(); +int main() { return fw_answer() == 42 ? 0 : 1; } +EOF +cat > app/build.mcpp <<'EOF' +import std; +import mcpp; +int main() { + const char* a = mcpp::dep_bin("spike.fw-installer", "fw-installer"); + mcpp::warning((std::string("DEPBIN tool=[") + (a ? a : "") + "]").c_str()); + return 0; +} +EOF +cd "$TMP/app" + +# ── 1. no [features] table: the dependency's feature opens, no macro ───── +"$MCPP" build --features spike.fw/installer > b1.log 2>&1 || fail "the dependency feature build failed" b1.log +grep -q 'DEPBIN tool=\[[^]]' b1.log || fail "--features spike.fw/installer did not open the feature" b1.log +ninja_file=$(find target -name build.ninja | head -1) +grep -q 'MCPP_FEATURE_SPIKE_FW_INSTALLER' "$ninja_file" \ + && fail "the dependency token became a macro of the root" "$ninja_file" + +# A plain undeclared name without a [features] table is documented as a macro. +rm -rf target +"$MCPP" build --strict --features nothing-here > b2.log 2>&1 \ + || fail "a plain name without a [features] table was refused" b2.log + +# ── 2. a token naming no dependency is refused under --strict ─────────── +if "$MCPP" build --strict --features nope/x > b3.log 2>&1; then + fail "--features nope/x was accepted under --strict" b3.log +fi +grep -q "nope" b3.log || fail "the refusal does not name the token" b3.log + +# ── 3. with a [features] table the token is not reported as undeclared ── +cat >> mcpp.toml <<'EOF' + +[features] +windows-installer = [] +EOF +rm -rf target +"$MCPP" build --strict --features spike.fw/installer > b4.log 2>&1 \ + || fail "the dependency token was refused beside a [features] table" b4.log +grep -q 'DEPBIN tool=\[[^]]' b4.log || fail "the feature did not open beside a [features] table" b4.log + +# ── 4. `why deps` reads the graph of the feature build ────────────────── +"$MCPP" why deps --features spike.fw/installer > w1.log 2>&1 || fail "why deps --features failed" w1.log +grep -q 'spike.fw-installer' w1.log || fail "why deps does not show the feature's dependency" w1.log + +echo "PASS: 714 --features opens a dependency's feature" diff --git a/tests/e2e/720_the_root_build_program_reads_the_resolved_graph.sh b/tests/e2e/720_the_root_build_program_reads_the_resolved_graph.sh new file mode 100755 index 000000000..8561ae34d --- /dev/null +++ b/tests/e2e/720_the_root_build_program_reads_the_resolved_graph.sh @@ -0,0 +1,176 @@ +#!/usr/bin/env bash +# requires: python3 +# 720 -- the root package's build program reads the resolved dependency graph +# and each package's `[package.metadata]` through `mcpp::graph_file()` (#647 E1). +# +# A framework merges what its libraries contribute (resources, platform +# sources) at build time, in dependency order, including libraries the +# application does not name. `dep_dir` answers by name for a direct dependency +# only, so a transitive library had no name to be asked for. +# +# Legs: +# A. app -> spike.a -> spike.b: the document lists b before a before app, the +# root last; b's entry carries its absolute manifest directory, its target +# kind and `metadata.demo.resources = "res"` verbatim. +# B. A second build with nothing changed does not re-run the root program. +# C. Editing b's `[package.metadata]` re-runs the root program, and the +# document states the new value. +# D. Editing b's source does not re-run the root program. +# E. A dependency's program reads "" from graph_file(). +# F. `[package]` reports an unknown key the way `[build]` does: a warning, +# an error under --strict; `metadata` is a known key. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p "$TMP/b/src" "$TMP/a/src" "$TMP/app/src" + +write_b() { # write_b + cat > "$TMP/b/mcpp.toml" < "$TMP/b/src/b.cpp" + +cat > "$TMP/a/mcpp.toml" <<'EOF' +[package] +name = "a" +namespace = "spike" +version = "0.1.0" + +[dependencies] +spike.b = { path = "../b" } + +[targets.a] +kind = "lib" +EOF +printf 'int b_answer();\nint a_answer() { return b_answer() + 1; }\n' > "$TMP/a/src/a.cpp" +# The dependency's program: graph_file() is not offered to it. +cat > "$TMP/a/build.mcpp" <<'EOF' +#include +import mcpp; +int main() { + const char* g = mcpp::graph_file(); + if (g != nullptr && g[0] != '\0') { + std::fprintf(stderr, "a: a dependency's program was offered graph_file()=%s\n", g); + return 1; + } + return 0; +} +EOF + +cat > "$TMP/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +spike.a = { path = "../a" } +EOF +printf 'int a_answer();\nint main() { return a_answer() == 41 ? 0 : 1; }\n' > "$TMP/app/src/main.cpp" +# The root's program copies the document it was given, one copy per run. +cat > "$TMP/app/build.mcpp" <<'EOF' +#include +#include +#include +#include +import mcpp; +int main() { + const char* g = mcpp::graph_file(); + if (g == nullptr || g[0] == '\0') { + std::fprintf(stderr, "app: graph_file() is empty\n"); + return 1; + } + std::ifstream in(g, std::ios::binary); + std::stringstream ss; + ss << in.rdbuf(); + const std::string dir = mcpp::manifest_dir(); + int n = 0; + for (;; ++n) { + std::ifstream probe(dir + "/graph-" + std::to_string(n) + ".json"); + if (!probe) break; + } + std::ofstream(dir + "/graph-" + std::to_string(n) + ".json", std::ios::binary) << ss.str(); + return 0; +} +EOF + +cd "$TMP/app" +runs() { ls graph-*.json 2>/dev/null | wc -l | tr -d ' '; } + +# ── A, E ──────────────────────────────────────────────────────────────────── +"$MCPP" build > a.log 2>&1 || fail "A: build failed" a.log +[ "$(runs)" -eq 1 ] || fail "A: the root program ran $(runs) times" a.log +python3 - "$TMP" graph-0.json res <<'EOF' || fail "A: the graph document" graph-0.json +import json, os, sys +tmp, path, want = sys.argv[1], sys.argv[2], sys.argv[3] +d = json.load(open(path)) +names = [p["package"]["canonical"] for p in d["packages"]] +assert names == ["spike.b@0.2.0", "spike.a@0.1.0", "mcpplibs.app@0.1.0"], names +b, a, app = d["packages"] +assert app["root"] is True and b["root"] is False, (app["root"], b["root"]) +assert os.path.isabs(b["manifest_dir"]), b["manifest_dir"] +assert os.path.realpath(b["manifest_dir"]) == os.path.realpath(os.path.join(tmp, "b")), b["manifest_dir"] +assert b["targets"] == [{"name": "b", "kind": "lib"}], b["targets"] +assert b["metadata"] == {"demo": {"resources": want, "languages": ["en", "zh"]}}, b["metadata"] +assert a["metadata"] == {}, a["metadata"] +assert [r["requester"] for r in b["requested_by"]] == ["spike.a@0.1.0"], b["requested_by"] +assert isinstance(b["features"], list) +EOF +echo "ok: A, the document lists the graph in dependency order with metadata" +echo "ok: E, a dependency's program is not offered the document" + +# ── B ─────────────────────────────────────────────────────────────────────── +"$MCPP" build > b.log 2>&1 || fail "B: build failed" b.log +[ "$(runs)" -eq 1 ] || fail "B: an unchanged build re-ran the root program" b.log +echo "ok: B, an unchanged build does not re-run the root program" + +# ── C ─────────────────────────────────────────────────────────────────────── +write_b assets +"$MCPP" build > c.log 2>&1 || fail "C: build failed" c.log +[ "$(runs)" -eq 2 ] || fail "C: editing a dependency's metadata did not re-run the root program" c.log +python3 - "$TMP" graph-1.json assets <<'EOF' || fail "C: the document after the edit" graph-1.json +import json, sys +d = json.load(open(sys.argv[2])) +b = d["packages"][0] +assert b["metadata"]["demo"]["resources"] == sys.argv[3], b["metadata"] +EOF +echo "ok: C, a metadata edit re-runs the root program with the new value" + +# ── D ─────────────────────────────────────────────────────────────────────── +printf 'int b_answer() { return 39 + 1; }\n' > "$TMP/b/src/b.cpp" +"$MCPP" build > d.log 2>&1 || fail "D: build failed" d.log +[ "$(runs)" -eq 2 ] || fail "D: editing a dependency's source re-ran the root program" d.log +echo "ok: D, a source edit does not re-run the root program" + +# ── F ─────────────────────────────────────────────────────────────────────── +cat >> "$TMP/app/mcpp.toml" <<'EOF' +EOF +python3 - "$TMP/app/mcpp.toml" <<'EOF' +import sys +p = sys.argv[1] +s = open(p).read().replace('version = "0.1.0"\n', 'version = "0.1.0"\nlicence = "MIT"\n', 1) +open(p, "w").write(s) +EOF +"$MCPP" build > f.log 2>&1 || fail "F: build failed on a warning" f.log +grep -q "\[package\] has unsupported key 'licence'" f.log || fail "F: no warning for an unknown [package] key" f.log +if "$MCPP" build --strict > f2.log 2>&1; then fail "F: --strict accepted an unknown [package] key" f2.log; fi +grep -q "\[package\] has unsupported key 'licence'" f2.log || fail "F: --strict failed for another reason" f2.log +grep -q "unsupported key 'metadata'" a.log c.log f.log && fail "F: metadata reported as unknown" a.log +echo "ok: F, [package] reports an unknown key and knows metadata" + +echo "PASS: 720" diff --git a/tests/e2e/721_an_android_row_links_on_a_macos_host.sh b/tests/e2e/721_an_android_row_links_on_a_macos_host.sh new file mode 100755 index 000000000..77dafb578 --- /dev/null +++ b/tests/e2e/721_an_android_row_links_on_a_macos_host.sh @@ -0,0 +1,102 @@ +#!/usr/bin/env bash +# requires: macos +# 721 -- an Android row links on a macOS host (#647 E3). +# +# The link line used to be chosen by the HOST: on macOS every target received +# the Apple SDK line, which carries `-isysroot ` and +# `-mmacosx-version-min` and no `--target`. The NDK's clang then linked an +# Android object as the host, `-fuse-ld=lld` selected `ld64.lld`, and the link +# failed with +# +# ld64.lld: error: unknown argument '-soname' +# +# Measured on macos-15 (mcpp-plugins run 34968851641). The line is now chosen +# by the host AND the target's object format (`link_shape`, unit +# `LinkShape.*`), so an ELF target takes the line that names its target. +# +# The NDK (`xim:android-ndk`) is installed on first use when this machine does +# not have it; on a runner without network access the leg reports that it +# could not install rather than passing. +# +# Legs: +# A. `mcpp build --target x86_64-linux-android` for a `kind = "app"` target +# over a shared dependency succeeds, and both `libapp.so` and `libdep.so` +# are ELF shared objects for x86-64. +# B. The build log names no `ld64.lld`, no `-mmacosx-version-min` and no +# `MacOSX.sdk` on the Android link. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; tail -60 "$f" 2>/dev/null; done; exit 1; } + +mkdir -p dep/src app/src +cat > dep/mcpp.toml <<'EOF' +[package] +name = "dep" +version = "0.1.0" +standard = "c++20" + +[targets.dep] +kind = "shared" + +[build] +sources = ["src/*.cpp"] +EOF +printf '[[gnu::visibility("default")]] int dep_answer() { return 42; }\n' > dep/src/dep.cpp + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +standard = "c++20" + +[dependencies] +dep = { path = "../dep" } + +[targets.app] +kind = "app" +main = "src/main.cpp" + +[target.x86_64-linux-android] +min_api_level = 24 +EOF +cat > app/src/main.cpp <<'EOF' +int dep_answer(); +[[gnu::visibility("default")]] int app_entry() { return dep_answer() + 1; } +int main() { return app_entry() == 43 ? 0 : 1; } +EOF + +cd app +# ── A ─────────────────────────────────────────────────────────────────────── +if ! "$MCPP" build --target x86_64-linux-android --verbose > a.log 2>&1; then + if grep -qi "cannot be downloaded\|download.*failed\|not installed" a.log \ + && ! grep -q "ld64.lld\|unknown argument '-soname'" a.log; then + fail "A: the NDK could not be installed on this runner" a.log + fi + fail "A: the Android row did not link on a macOS host" a.log +fi +app=$(find target -name libapp.so -path '*/bin/*' | head -1) +dep=$(find target -name libdep.so -path '*/bin/*' | head -1) +[ -n "$app" ] && [ -n "$dep" ] || fail "A: no libapp.so / libdep.so" a.log +for f in "$app" "$dep"; do + kind=$(file -b "$f") + echo "reading A: $(basename "$f"): $kind" + case "$kind" in *ELF*"shared object"*x86-64*) ;; *) fail "A: $(basename "$f") is not an x86-64 ELF shared object: $kind" a.log ;; esac +done +echo "ok: A, the Android row links on a macOS host" + +# ── B ─────────────────────────────────────────────────────────────────────── +if grep -q "ld64.lld" a.log; then fail "B: ld64.lld appears in the build" a.log; fi +graph=$(ls target/x86_64-linux-android/*/build.ninja 2>/dev/null | head -1 || true) +[ -n "$graph" ] || fail "B: no build graph for the Android row" a.log +# The link line only: `ldflags` is what the host branch assembled. +if grep -E "^[[:space:]]*ldflags[[:space:]]*=" "$graph" | grep -q "mmacosx-version-min\|MacOSX.sdk"; then + grep -nE "^[[:space:]]*ldflags[[:space:]]*=" "$graph" | head -3 + fail "B: the Android link line carries the macOS SDK or deployment floor" a.log +fi +echo "ok: B, the Android link carries no Apple SDK token" + +echo "PASS: 721" diff --git a/tests/e2e/722_pack_strips_what_the_graph_built.sh b/tests/e2e/722_pack_strips_what_the_graph_built.sh new file mode 100755 index 000000000..97b14633f --- /dev/null +++ b/tests/e2e/722_pack_strips_what_the_graph_built.sh @@ -0,0 +1,210 @@ +#!/usr/bin/env bash +# requires: pack python3 +# 722 -- `mcpp pack` strips what the graph built on every row that strips, and +# says so only when it does (#649 E5). +# +# `strip_program` used to strip the staged PROGRAM and nothing else, on the ELF +# and PE paths only. A shared library the same graph compiled from source +# shipped its `.symtab` and `.debug_*`, and the Android row, which `run` +# dispatches before either path, stripped nothing at all while the `Packing` +# line said "stripped". A member that stages libraries of its own had no way to +# learn the decision, so `--no-strip` did not reach its files. +# +# Legs: +# A. Linux desktop, `bin` over a shared dependency: `bin/hostapp` and +# `lib/libdep.so` carry no `.symtab` and no `.debug_*`; `libdep.so` still +# exports `dep_answer` in `.dynsym`; the unpacked program runs. +# B. `--no-strip`: `lib/libdep.so` keeps its `.symtab`, and the `Packing` +# line does not say "stripped". +# C. `--debug-symbols DIR`: `DIR/libdep.so.debug` exists and the packed +# library names it through `.gnu_debuglink`. +# D. A dispatched format's build program reads `pack_strip()` = "1" and +# `pack_debug_symbols_dir()` = DIR, and "0" under `--no-strip`. +# E. When the NDK is installed: `--target x86_64-linux-android` gives +# `lib/libapp.so`, `lib/libdep.so` and `lib/libc++_shared.so` with no +# `.symtab`, and `.dynsym` still names `app_entry` and `dep_answer`. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +# Sections and dynamic symbols, read from the ELF itself. +elf_facts() { # elf_facts -> "symtab=N debug=N dynsym=" +python3 - "$1" <<'PY' +import struct, sys +d = open(sys.argv[1], 'rb').read() +assert d[:4] == b'\x7fELF' and d[4] == 2, "not ELF64" +shoff, = struct.unpack_from(' dep/mcpp.toml <<'EOF' +[package] +name = "dep" +namespace = "spike" +version = "0.1.0" +standard = "c++20" + +[targets.dep] +kind = "shared" + +[build] +sources = ["src/*.cpp"] +EOF +cat > dep/src/dep.cpp <<'EOF' +#include +[[gnu::visibility("default")]] int dep_answer() { return static_cast(std::to_string(42).size()) + 40; } +EOF + +cat > hostapp/mcpp.toml <<'EOF' +[package] +name = "hostapp" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.dep = { path = "../dep" } + +[targets.hostapp] +kind = "bin" +main = "src/main.cpp" +EOF +printf 'int dep_answer();\nint main() { return dep_answer() == 42 ? 0 : 1; }\n' > hostapp/src/main.cpp +cd hostapp + +# ── A ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --format tar > a.log 2>&1 || fail "A: pack failed" a.log +grep -q "Packing hostapp v0.1.0 (vendored, stripped)" a.log || fail "A: the Packing line" a.log +rm -rf xa && mkdir xa && tar -xzf target/dist/hostapp-0.1.0-x86_64-linux-gnu.tar.gz -C xa +lib=$(ls xa/*/lib/libdep.so); prog=$(ls xa/*/bin/hostapp) +fa=$(elf_facts "$lib"); fp=$(elf_facts "$prog") +echo "reading A: libdep.so $fa" +echo "reading A: hostapp $fp" +case "$fa" in "symtab=0 debug=0 "*) ;; *) fail "A: lib/libdep.so is not stripped: $fa" ;; esac +case "$fa" in *dep_answer*) ;; *) fail "A: lib/libdep.so lost its export: $fa" ;; esac +case "$fp" in "symtab=0 debug=0 "*) ;; *) fail "A: bin/hostapp is not stripped: $fp" ;; esac +"$(ls -d xa/*)/hostapp" || fail "A: the unpacked program does not run" a.log +echo "ok: A, the program and the graph-built library are stripped and run" + +# ── B ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --format tar --no-strip > b.log 2>&1 || fail "B: pack failed" b.log +grep -q "Packing hostapp v0.1.0 (vendored)" b.log || fail "B: the Packing line" b.log +rm -rf xb && mkdir xb && tar -xzf target/dist/hostapp-0.1.0-x86_64-linux-gnu.tar.gz -C xb +fb=$(elf_facts "$(ls xb/*/lib/libdep.so)") +echo "reading B: libdep.so $fb" +case "$fb" in "symtab=1 "*) ;; *) fail "B: --no-strip did not reach lib/libdep.so: $fb" ;; esac +echo "ok: B, --no-strip reaches the graph-built library" + +# ── C ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --format tar --debug-symbols "$TMP/dbg" > c.log 2>&1 || fail "C: pack failed" c.log +[ -s "$TMP/dbg/libdep.so.debug" ] || fail "C: no libdep.so.debug" c.log +rm -rf xc && mkdir xc && tar -xzf target/dist/hostapp-0.1.0-x86_64-linux-gnu.tar.gz -C xc +fc=$(elf_facts "$(ls xc/*/lib/libdep.so)") +case "$fc" in *"debuglink=1"*) ;; *) fail "C: lib/libdep.so names no debug file: $fc" ;; esac +echo "ok: C, --debug-symbols separates the graph-built library's debug information" + +# ── D ─────────────────────────────────────────────────────────────────────── +# Written only now, so the legs above measure the strip itself on any engine. +# D's provider: records what a dispatched format's program is told. +cat > build.mcpp <<'EOF' +import mcpp; +#include +#include +#include +int main() { + mcpp::provides_pack_format("probe"); + if (std::string_view(mcpp::pack_format()) != "probe") return 0; + const std::string out = std::string(mcpp::out_dir()) + "/probe.txt"; + const std::string text = std::string("strip=") + mcpp::pack_strip() + + " debug=" + mcpp::pack_debug_symbols_dir(); + mcpp::action a; + a.id = "probe"; + a.role = "artifact"; + a.description = "probe"; + a.arg("/bin/sh").arg("-c").arg(("printf '%s' '" + text + "' > " + out).c_str()) + .input("${mcpp.target_file:hostapp}") + .output(out.c_str()) + .submit(); + return 0; +} +EOF + +"$MCPP" pack --format probe --debug-symbols "$TMP/dbg2" > d.log 2>&1 || fail "D: pack failed" d.log +probe=$(find target -name probe.txt | head -1) +[ "$(cat "$probe")" = "strip=1 debug=$TMP/dbg2" ] || fail "D: the program read '$(cat "$probe" 2>/dev/null)'" d.log +"$MCPP" pack --format probe --no-strip > d2.log 2>&1 || fail "D: pack --no-strip failed" d2.log +[ "$(cat "$probe")" = "strip=0 debug=" ] || fail "D: under --no-strip the program read '$(cat "$probe")'" d2.log +echo "ok: D, a dispatched format's program reads the strip decision" + +# ── E ─────────────────────────────────────────────────────────────────────── +ndk=$(ls -d "${MCPP_HOME:-$HOME/.mcpp}"/registry/data/xpkgs/xim-x-android-ndk/*/ 2>/dev/null | head -1 || true) +if [ -z "$ndk" ]; then + echo "skip: E, xim:android-ndk is not installed on this machine" +else + cd "$TMP" + cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +standard = "c++20" + +[dependencies] +spike.dep = { path = "../dep" } + +[targets.app] +kind = "app" +main = "src/main.cpp" + +[target.x86_64-linux-android] +min_api_level = 23 +EOF + cat > app/src/main.cpp <<'EOF' +#include +int dep_answer(); +[[gnu::visibility("default")]] int app_entry() { return dep_answer() + static_cast(std::to_string(1).size()); } +int main() { return app_entry() == 43 ? 0 : 1; } +EOF + cd app + "$MCPP" pack --target x86_64-linux-android --format tar > e.log 2>&1 || fail "E: pack failed" e.log + grep -q "Packing app v0.1.0 (vendored, stripped)" e.log || fail "E: the Packing line" e.log + rm -rf xe && mkdir xe && tar -xzf target/dist/app-0.1.0-x86_64-linux-android.tar.gz -C xe + for f in libapp.so libdep.so libc++_shared.so; do + fe=$(elf_facts "$(ls xe/*/lib/$f)") + echo "reading E: $f $(echo "$fe" | cut -d' ' -f1-3)" + case "$fe" in "symtab=0 debug=0 "*) ;; *) fail "E: lib/$f is not stripped: $fe" e.log ;; esac + done + case "$(elf_facts "$(ls xe/*/lib/libapp.so)")" in *app_entry*) ;; *) fail "E: libapp.so lost app_entry" ;; esac + case "$(elf_facts "$(ls xe/*/lib/libdep.so)")" in *dep_answer*) ;; *) fail "E: libdep.so lost dep_answer" ;; esac + echo "ok: E, the Android row strips the program, the graph's library and the NDK runtime" +fi + +echo "PASS: 722" diff --git a/tests/e2e/723_pack_reports_what_it_produced.sh b/tests/e2e/723_pack_reports_what_it_produced.sh new file mode 100755 index 000000000..dd3bba3ac --- /dev/null +++ b/tests/e2e/723_pack_reports_what_it_produced.sh @@ -0,0 +1,118 @@ +#!/usr/bin/env bash +# requires: pack python3 +# 723 -- `mcpp pack --message-format json` reports what it produced, and `pack` +# takes the profile shorthands `build` and `run` take, with one precedence +# (#649 E9). +# +# A driver that publishes a pack's artifacts used to parse the human `Packed` +# lines, whose paths are shortened (`@mcpp/`, `~/`, project-relative), and a +# driver mapping one profile switch onto three verbs found `--release` refused +# on the third. `mcpp run --profile dev --release` built `release` while +# `mcpp build` given the same line built `dev`. +# +# Legs: +# A. `pack --format tar --message-format json`: stdout is one `mcpp.pack` +# envelope and nothing else; the artifact path is absolute and exists, its +# type is `file`, its format `tar`, its target the host triple; the staged +# tree and its manifest exist, and the closure is `walked`. +# B. `pack --format dir --message-format json`: the artifact is a directory. +# C. `pack --release` and `pack --dev` are accepted; `--dev` builds `dev`. +# D. `run --profile dev --release` builds `dev`, as `build` does. +# E. `pack --message-format yaml` is refused with exit 2. +# F. `--protocol-version` lists the `mcpp.pack` kind and the `pack` command. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p app/src +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.3.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +printf '#include \nint main() { std::puts("app ran"); return 0; }\n' > app/src/main.cpp +cd app + +# ── A ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --format tar --message-format json > a.json 2> a.err || fail "A: pack failed" a.err a.json +python3 - a.json <<'EOF' || fail "A: the envelope" a.json a.err +import json, os, sys +text = open(sys.argv[1]).read() +e = json.loads(text) # the whole of stdout is one document +assert e["kind"] == "mcpp.pack" and e["kindVersion"] == 1, (e["kind"], e.get("kindVersion")) +assert "write-project" in e["effects"], e["effects"] +arts = e["data"]["artifacts"] +assert len(arts) == 1, arts +a = arts[0] +assert os.path.isabs(a["path"]) and os.path.isfile(a["path"]), a +assert a["path"].endswith(".tar.gz") and a["type"] == "file" and a["format"] == "tar", a +assert len(a["targets"]) == 1 and a["targets"][0].endswith("linux-gnu"), a["targets"] +st = e["data"]["stage"] +assert os.path.isdir(st["dir"]) and os.path.isfile(st["manifest"]), st +assert st["closure"] == "walked", st +assert e["diagnostics"] == [], e["diagnostics"] +EOF +grep -q "Packed" a.err || fail "A: the human lines did not go to stderr" a.err +echo "ok: A, one envelope on stdout names the archive, its legs and the staged tree" + +# ── B ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --format dir --message-format json > b.json 2> b.err || fail "B: pack failed" b.err +python3 - b.json <<'EOF' || fail "B: the envelope" b.json +import json, os, sys +a = json.load(open(sys.argv[1]))["data"]["artifacts"][0] +assert a["type"] == "directory" and os.path.isdir(a["path"]) and a["format"] == "dir", a +EOF +echo "ok: B, a directory artifact is reported as one" + +# ── C ─────────────────────────────────────────────────────────────────────── +# `pack` narrates no `Finished` line, so the profile is read from the graph the +# pack built: `dev` compiles at `-O0`, `release` does not. Each sub-leg starts +# from an empty `target/`, so the one graph there is the one this pack wrote. +built_dev() { grep -q -- " -O0" "$(ls target/*-linux-gnu/*/build.ninja | head -1)"; } +rm -rf target +"$MCPP" pack --release --format tar > c1.log 2>&1 || fail "C: pack --release was refused" c1.log +built_dev && fail "C: pack --release built dev" c1.log +rm -rf target +"$MCPP" pack --dev --format tar > c2.log 2>&1 || fail "C: pack --dev was refused" c2.log +built_dev || fail "C: pack --dev did not build dev" c2.log +rm -rf target +"$MCPP" pack --profile release --dev --format tar > c3.log 2>&1 || fail "C: pack --profile --dev failed" c3.log +built_dev && fail "C: --dev won over --profile release on pack" c3.log +echo "ok: C, pack takes --release and --dev, and --profile wins" + +# ── D ─────────────────────────────────────────────────────────────────────── +"$MCPP" run --profile dev --release > d.log 2>&1 || fail "D: run failed" d.log +grep -q "Finished dev" d.log || fail "D: run --profile dev --release did not build dev" d.log +"$MCPP" build --profile dev --release > d2.log 2>&1 || fail "D: build failed" d2.log +grep -q "Finished dev" d2.log || fail "D: build --profile dev --release did not build dev" d2.log +echo "ok: D, run and build resolve --profile over the shorthands alike" + +# ── E ─────────────────────────────────────────────────────────────────────── +set +e +"$MCPP" pack --format tar --message-format yaml > e.out 2> e.err +rc=$? +set -e +[ "$rc" -eq 2 ] || fail "E: --message-format yaml exited $rc" e.err +grep -q "unknown --message-format 'yaml'" e.err || fail "E: the refusal does not name the value" e.err +echo "ok: E, an unknown message format is refused" + +# ── F ─────────────────────────────────────────────────────────────────────── +"$MCPP" --protocol-version > f.json +python3 - f.json <<'EOF' || fail "F: the protocol document" f.json +import json, sys +p = json.load(open(sys.argv[1])) +assert p["kinds"]["mcpp.pack"] == 1, p["kinds"] +fx = p["commands"]["pack"]["effects"] +assert "write-project" in fx and "exec-build-script" in fx, fx +EOF +echo "ok: F, the protocol document declares the kind and the command" + +echo "PASS: 723" diff --git a/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh b/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh new file mode 100755 index 000000000..720b53e33 --- /dev/null +++ b/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# requires: elf +# 730 -- a dependency the resolver reaches through the deprecated bare-name rung +# is not a descriptor miss for the index refresh decision (#648 L1). +# +# `ftxui = "6.1.9"` omits the namespace, which means mcpplibs; the package is +# published as compat.ftxui. The resolver tries compat after the exact miss and +# resolves it from disk. The refresh decision used to stop at the exact +# coordinate and asked for a network refresh, which `-v` shows offline as a +# decision suppressed by offline mode. Criteria, planned offline with `-v`: +# A. the resolver takes the rung (its deprecation warning is printed), so the +# fixture exercises the path at all; +# B. no refresh decision for that dependency is suppressed by offline mode. +# Skipped when the local mcpplibs index has no compat descriptor to reach. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +INDEX_ROOT="${MCPP_HOME:-$HOME/.mcpp}/registry/data/mcpplibs/pkgs/c" +DESC="" +for cand in cjson argparse gtest ftxui; do + if [ -f "$INDEX_ROOT/compat.$cand.lua" ]; then DESC="$cand"; break; fi +done +if [ -z "$DESC" ]; then + echo "SKIP: no compat descriptor in the local mcpplibs index" + exit 0 +fi +VER=$(grep -o '\["[0-9][0-9.]*"\]' "$INDEX_ROOT/compat.$DESC.lua" | head -1 | tr -d '[]"') +[ -n "$VER" ] || fail "could not read a version of compat.$DESC" + +mkdir -p "$TMP/app/src" +cat > "$TMP/app/mcpp.toml" < "$TMP/app/src/main.cpp" + +cd "$TMP/app" +MCPP_OFFLINE=1 "$MCPP" emit build-database --format json -v > out.json 2> err.txt || true + +grep -q "resolved to 'compat.$DESC' through the deprecated bare-name search" err.txt \ + || fail "A: the resolver did not take the bare-name rung for $DESC" err.txt +if grep -a "index: $DESC@$VER: offline mode" err.txt; then + fail "B: the refresh decision asked for a network refresh of a dependency the resolver found" err.txt +fi +echo "PASS: 730 bare compat dependency ($DESC@$VER) is not a refresh miss" diff --git a/tests/e2e/731_planning_children_do_not_inherit_the_callers_pipe.sh b/tests/e2e/731_planning_children_do_not_inherit_the_callers_pipe.sh new file mode 100755 index 000000000..03bca566e --- /dev/null +++ b/tests/e2e/731_planning_children_do_not_inherit_the_callers_pipe.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# requires: elf +# 731 -- a child started while `emit build-database` plans does not hold the +# caller's pipe (#648 L2). +# +# `emit build-database` sends planning narration to stderr by redirecting +# descriptor 1 and saving the original. The saved copy was inheritable, so a +# build program, a hook or an xlings process started during planning held the +# caller's pipe open, and a caller reading mcpp's output waited for that child +# rather than for mcpp. Criterion: a build program lists its descriptors, and +# none of them names the pipe the reader of mcpp's standard output reads. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +mkdir -p "$TMP/app/src" +cat > "$TMP/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +echo 'int main() { return 0; }' > "$TMP/app/src/main.cpp" +cat > "$TMP/app/build.mcpp" < err.txt \ + | { readlink /proc/self/fd/0 > "$TMP/reader.txt"; cat > "$TMP/out.json"; } +[ -s "$TMP/fds.txt" ] || fail "the build program did not run" err.txt +PIPE=$(cat "$TMP/reader.txt") +case "$PIPE" in pipe:*) ;; *) fail "the reader's stdin is not a pipe: $PIPE" ;; esac +if grep -F " $PIPE" "$TMP/fds.txt"; then + fail "a build program holds the caller's pipe $PIPE" "$TMP/fds.txt" +fi +echo "PASS: 731 planning children do not inherit the caller's pipe" diff --git a/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh b/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh new file mode 100755 index 000000000..b65625aac --- /dev/null +++ b/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh @@ -0,0 +1,77 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 732 -- an xlings child is bounded, and it does not outlive mcpp (#648 L3). +# +# The xlings binary is a stub named through `[xlings] binary`; its `update` +# writes its pid and sleeps. Criteria: +# A. `mcpp index update` under `[index] refresh_timeout = 3` returns within the +# bound plus a margin, says which bound stopped it, and the stub is gone; +# B. with a long bound, SIGTERM to mcpp takes the stub with it: the stub's +# process group is registered with mcpp's signal guard. +set -e + +TMP=$(mktemp -d) +cleanup() { + for f in "$TMP"/stub.pid "$TMP"/stub2.pid; do + [ -s "$f" ] && kill -9 "$(cat "$f")" 2>/dev/null || true + done + rm -rf "$TMP" +} +trap cleanup EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +# `timeout` is GNU coreutils and is absent on macOS; `gtimeout` is there when +# coreutils is installed. Without either, the bound under test is the engine's +# own, and the suite's per-test bound is the backstop. +bounded() { # + local secs="$1"; shift + if command -v timeout >/dev/null 2>&1; then timeout "$secs" "$@"; return $?; fi + if command -v gtimeout >/dev/null 2>&1; then gtimeout "$secs" "$@"; return $?; fi + "$@" +} + +mkdir -p "$TMP/bin" "$TMP/home" +cat > "$TMP/bin/xlings" <<'EOF' +#!/usr/bin/env bash +case "${1:-}" in + update) echo "$$" > "${STUB_PID:?}"; exec sleep 1000 ;; +esac +exit 0 +EOF +chmod +x "$TMP/bin/xlings" + +write_config() { + cat > "$TMP/home/config.toml" < "$TMP/a.log" 2>&1 +set -e +elapsed=$(( $(date +%s) - start )) +[ -s "$TMP/stub.pid" ] || fail "A: the stub's update never ran" "$TMP/a.log" +[ "$elapsed" -lt 30 ] || fail "A: index update returned after ${elapsed}s under a 3 s bound" "$TMP/a.log" +grep -q "did not finish within 3 seconds" "$TMP/a.log" || fail "A: no line names the bound" "$TMP/a.log" +sleep 1 +if kill -0 "$(cat "$TMP/stub.pid")" 2>/dev/null; then fail "A: the stub outlived its bound"; fi + +# B. ownership +write_config 600 +STUB_PID="$TMP/stub2.pid" MCPP_HOME="$TMP/home" "$MCPP" index update > "$TMP/b.log" 2>&1 & +mcpp_pid=$! +for _ in $(seq 1 150); do [ -s "$TMP/stub2.pid" ] && break; sleep 0.1; done +[ -s "$TMP/stub2.pid" ] || { kill -9 "$mcpp_pid" 2>/dev/null; fail "B: the stub's update never ran" "$TMP/b.log"; } +kill -TERM "$mcpp_pid" +wait "$mcpp_pid" 2>/dev/null || true +for _ in $(seq 1 30); do kill -0 "$(cat "$TMP/stub2.pid")" 2>/dev/null || break; sleep 0.1; done +if kill -0 "$(cat "$TMP/stub2.pid")" 2>/dev/null; then fail "B: the stub outlived mcpp after SIGTERM"; fi + +echo "PASS: 732 xlings refresh is bounded and owned (A ${elapsed}s)" diff --git a/tests/e2e/733_envelope_reports_observed_network_access.sh b/tests/e2e/733_envelope_reports_observed_network_access.sh new file mode 100755 index 000000000..9fb4f1989 --- /dev/null +++ b/tests/e2e/733_envelope_reports_observed_network_access.sh @@ -0,0 +1,63 @@ +#!/usr/bin/env bash +# requires: unix-shell python3 +# 733 -- the envelope's `effects` report the network access that happened, and +# an offline plan that needs the index has its own code (#648 L4, L6). +# +# docs/50 defines `effects` as what running the command did. A home with no +# package index plans a registry dependency: online, the plan refreshes the +# index (the xlings stub answers at once and fetches nothing); offline, it does +# not. Criteria: +# A. online, the failure envelope lists `network`; +# B. offline, it does not, and its code is MCPP_OFFLINE_DOWNLOAD_REQUIRED. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +export MCPP_HOME="$TMP/home" +source "$(dirname "$0")/_inherit_toolchain.sh" +rm -rf "$MCPP_HOME/registry/data/mcpplibs" + +mkdir -p "$TMP/bin" +cat > "$TMP/bin/xlings" <<'EOF' +#!/usr/bin/env bash +echo "$*" >> "${STUB_LOG:?}" +exit 0 +EOF +chmod +x "$TMP/bin/xlings" +STUB_HOST="$(host_path "$TMP/bin/xlings")" +{ + grep -v '^binary' "$MCPP_HOME/config.toml" 2>/dev/null | sed '/^\[xlings\]/d' + printf '\n[xlings]\nbinary = "%s"\n' "$STUB_HOST" +} > "$TMP/config.toml" +mv "$TMP/config.toml" "$MCPP_HOME/config.toml" + +mkdir -p "$TMP/app/src" +cat > "$TMP/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[dependencies.mcpplibs] +probe-not-published = "1.0.0" +EOF +echo 'int main() { return 0; }' > "$TMP/app/src/main.cpp" +cd "$TMP/app" + +effects() { python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print(" ".join(d["effects"])); print(" ".join(x["code"] for x in d["diagnostics"]))' "$1"; } + +STUB_LOG="$TMP/online.log" "$MCPP" emit build-database --format json > online.json 2> online.err || true +[ -s online.json ] || fail "A: no envelope" online.err +grep -q '^update' "$TMP/online.log" 2>/dev/null || fail "A: the plan did not refresh the index" online.err +effects online.json > online.txt +head -1 online.txt | grep -qw network || fail "A: a plan that refreshed the index reports no network effect" online.txt + +STUB_LOG="$TMP/offline.log" MCPP_OFFLINE=1 "$MCPP" emit build-database --format json > offline.json 2> offline.err || true +[ -s offline.json ] || fail "B: no envelope" offline.err +effects offline.json > offline.txt +if head -1 offline.txt | grep -qw network; then fail "B: an offline plan reports a network effect" offline.txt; fi +sed -n 2p offline.txt | grep -qw MCPP_OFFLINE_DOWNLOAD_REQUIRED \ + || fail "B: an offline plan with no index is not MCPP_OFFLINE_DOWNLOAD_REQUIRED" offline.txt +echo "PASS: 733 the envelope reports observed network access" diff --git a/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh b/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh new file mode 100755 index 000000000..44cd442ed --- /dev/null +++ b/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 734 -- `[index] auto_refresh = false` governs the implicit refreshes that used +# to bypass the refresh policy (#648 L5). +# +# docs/05: auto_refresh = false means the index is never refreshed +# automatically. A project whose custom index has never been synced used to be +# synced on its first build regardless. Criteria, with an xlings stub that logs: +# A. the build refuses before syncing, and names `mcpp index update`; +# B. the stub was never asked to `update`. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +export MCPP_HOME="$TMP/home" +source "$(dirname "$0")/_inherit_toolchain.sh" +source "$(dirname "$0")/_host_path.sh" + +mkdir -p "$TMP/bin" +cat > "$TMP/bin/xlings" <<'EOF' +#!/usr/bin/env bash +echo "$*" >> "${STUB_LOG:?}" +exit 0 +EOF +chmod +x "$TMP/bin/xlings" +STUB_HOST="$(host_path "$TMP/bin/xlings")" +{ + grep -v '^binary\|^auto_refresh' "$MCPP_HOME/config.toml" 2>/dev/null \ + | sed '/^\[xlings\]/d; /^\[index\]$/d' + printf '\n[xlings]\nbinary = "%s"\n\n[index]\nauto_refresh = false\n' "$STUB_HOST" +} > "$TMP/config.toml" +mv "$TMP/config.toml" "$MCPP_HOME/config.toml" + +mkdir -p "$TMP/app/src" +cat > "$TMP/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[indices] +acme = { url = "https://example.invalid/acme-index.git" } + +[dependencies.acme] +widget = "1.0.0" +EOF +echo 'int main() { return 0; }' > "$TMP/app/src/main.cpp" +cd "$TMP/app" + +set +e +STUB_LOG="$TMP/stub.log" "$MCPP" build > build.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || fail "A: the build succeeded with an unsynced custom index" build.log +grep -q "mcpp index update" build.log || fail "A: the refusal does not name mcpp index update" build.log +grep -q "auto_refresh = false" build.log || fail "A: the refusal does not name the setting" build.log +if grep -q '^update' "$TMP/stub.log" 2>/dev/null; then + fail "B: the custom index was synced although auto_refresh = false" "$TMP/stub.log" +fi +echo "PASS: 734 auto_refresh = false governs the first custom index sync" diff --git a/tests/e2e/735_offline_missing_package_has_its_own_code.sh b/tests/e2e/735_offline_missing_package_has_its_own_code.sh new file mode 100755 index 000000000..c36d8e8a5 --- /dev/null +++ b/tests/e2e/735_offline_missing_package_has_its_own_code.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# requires: python3 +# 735 -- an offline plan that needs a package download reports +# MCPP_OFFLINE_DOWNLOAD_REQUIRED, and names the package (#648 L6). +# +# The package comes from a local path index, so the descriptor is present and +# the download is the only thing missing. Criteria: +# A. offline, the envelope's code is MCPP_OFFLINE_DOWNLOAD_REQUIRED, its +# message names the package and its version, and the exit status is 1; +# B. a plan that fails for another reason keeps MCPP_BUILD_DATABASE_PLAN_FAILED. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +mkdir -p "$TMP/index/pkgs/w" "$TMP/app/src" +cat > "$TMP/index/pkgs/w/widget.lua" <<'EOF' +package = { + spec = "1", + namespace = "acme", + name = "widget", + description = "a package whose archive is not installed", + licenses = {"MIT"}, + type = "package", + xpm = { + linux = { ["1.0.0"] = { url = "https://example.invalid/widget-1.0.0.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + macosx = { ["1.0.0"] = { url = "https://example.invalid/widget-1.0.0.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + windows = { ["1.0.0"] = { url = "https://example.invalid/widget-1.0.0.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + }, + mcpp = { language = "c++23", sources = { "src/*.cpp" }, targets = { ["widget"] = { kind = "lib" } } }, +} +EOF +INDEX_HOST="$(host_path "$TMP/index")" +cat > "$TMP/app/mcpp.toml" < "$TMP/app/src/main.cpp" +cd "$TMP/app" + +code_of() { python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print(d["diagnostics"][0]["code"] if d["diagnostics"] else ""); print(d["diagnostics"][0]["message"] if d["diagnostics"] else "")' "$1"; } + +set +e +MCPP_OFFLINE=1 "$MCPP" emit build-database --format json > a.json 2> a.err +rc=$? +set -e +[ "$rc" -eq 1 ] || fail "A: exit status $rc, expected 1" a.err +code_of a.json > a.txt +sed -n 1p a.txt | grep -qx MCPP_OFFLINE_DOWNLOAD_REQUIRED || fail "A: wrong code" a.txt +grep -q "widget" a.txt && grep -q "1.0.0" a.txt || fail "A: the message does not name the package and version" a.txt + +# B. another failure keeps the general code: a dependency no index declares. +sed -i.bak 's/^widget = "1.0.0"/absent-package = "1.0.0"/' mcpp.toml +set +e +MCPP_OFFLINE=1 "$MCPP" emit build-database --format json > b.json 2> b.err +set -e +code_of b.json > b.txt +sed -n 1p b.txt | grep -qx MCPP_BUILD_DATABASE_PLAN_FAILED || fail "B: a selector error took the offline code" b.txt +echo "PASS: 735 an offline plan that needs a download has its own code" diff --git a/tests/e2e/_cxx_identity_across_images_body.sh b/tests/e2e/_cxx_identity_across_images_body.sh new file mode 100644 index 000000000..c5d82d67e --- /dev/null +++ b/tests/e2e/_cxx_identity_across_images_body.sh @@ -0,0 +1,103 @@ +# tests/e2e/_cxx_identity_across_images_body.sh -- the measurement body shared +# by 704 (Mach-O) and 705 (PE). Sourced, not run: the caller sets `set -e`, +# `TMP`, `fail`, and `IMAGE_FORMAT` (a label for the READING lines). +# +# WHAT IS MEASURED (#646 F2, #649 E10). Each image that embeds a private copy +# of the C++ runtime holds its own type information and its own error +# categories. Whether a `std::runtime_error` thrown in a shared library is +# caught by its class in the program, and whether an `std::error_code` made in +# the library compares equal to an `std::errc` in the program, depends on how +# the platform compares those identities across images. libc++ documents +# address comparison for such types on Apple platforms; MSVC matches catch +# clauses by decorated name. This body builds one fixture under three runtime +# statements and PRINTS what each one does. It does not fail on the value it +# reads: the decision taken from the reading is recorded in the design record, +# not here. It fails only when the default leg cannot be built or run, since +# then nothing was measured. + +mkdir -p "$TMP/fw/src" "$TMP/app/src" +cat > "$TMP/fw/mcpp.toml" <<'TOML' +[package] +name = "fw" +version = "0.1.0" + +[targets.fw] +kind = "shared" +TOML +cat > "$TMP/fw/src/fw.cppm" <<'CPP' +export module fw; +import std; +export struct fw_error : std::runtime_error { + using std::runtime_error::runtime_error; +}; +export [[gnu::visibility("default")]] void fw_throw_standard(); +export [[gnu::visibility("default")]] void fw_throw_own(); +export [[gnu::visibility("default")]] std::error_code fw_code(); +CPP +cat > "$TMP/fw/src/fw.cpp" <<'CPP' +module fw; +import std; +void fw_throw_standard() { throw std::runtime_error("from fw"); } +void fw_throw_own() { throw fw_error("own"); } +std::error_code fw_code() { return std::make_error_code(std::errc::no_such_file_or_directory); } +CPP +cat > "$TMP/app/src/main.cpp" <<'CPP' +import std; +import fw; +int main() { + const char* standard = "not-matched"; + try { fw_throw_standard(); } + catch (const std::runtime_error&) { standard = "caught"; } + catch (...) {} + const char* own = "not-matched"; + try { fw_throw_own(); } + catch (const fw_error&) { own = "caught"; } + catch (...) {} + const bool equal = fw_code() == std::errc::no_such_file_or_directory; + std::println("runtime_error={} own_error={} errc={}", standard, own, + equal ? "equal" : "unequal"); +} +CPP + +measure_leg() { # $1 = leg name, $2 = [build] lines + cat > "$TMP/app/mcpp.toml" < "build-$1.log" 2>&1; then + echo "READING $IMAGE_FORMAT $1: build-failed $(grep -m1 -E 'error' "build-$1.log" | cut -c1-160)" + exit 3 + fi + local exe + exe=$(find target -type f \( -name app -o -name app.exe \) -path '*bin*' | head -1) + local out + if out=$("$exe" 2>&1); then + echo "READING $IMAGE_FORMAT $1: $out" + else + echo "READING $IMAGE_FORMAT $1: run-failed exit=$? $out" + exit 4 + fi + local contracts + contracts=$(tr -d ' \n\r' < "$(find target -name resolution.json | head -1)" 2>/dev/null \ + | grep -o '"cxx_runtime_by_role":{[^}]*}' || true) + echo "READING $IMAGE_FORMAT $1 record: ${contracts:-none}" + ) +} + +set +e +measure_leg default '' +default_rc=$? +measure_leg host-coupled 'cxx_runtime = "host-coupled"' +measure_leg shared-host-coupled 'cxx_runtime = { shared = "host-coupled" }' +set -e +[ "$default_rc" -eq 0 ] || fail "the default leg could not be measured (rc $default_rc)" "$TMP/app/build-default.log" diff --git a/tests/unit/test_build_flags.cpp b/tests/unit/test_build_flags.cpp index b9d1f6448..dbc088b49 100644 --- a/tests/unit/test_build_flags.cpp +++ b/tests/unit/test_build_flags.cpp @@ -2,6 +2,7 @@ import std; import mcpp.build.flags; +import mcpp.build.distribution; import mcpp.modgraph.scanner; namespace { @@ -165,3 +166,45 @@ TEST(BuildFlagsLoaderTokens, OrdinaryPathsAreNotTokens) { EXPECT_FALSE(mcpp::build::is_loader_relative_search_path(entry)) << entry; } } + +// #647 E3: which link line a (host, target) pair takes. The branch used to be +// chosen by `if constexpr` on the host, so an Android row built on a macOS host +// received the Apple SDK line with no `--target` and linked through +// `ld64.lld`. The choice is a function of the host AND the target's object +// format, stated here for every host on every CI machine. +namespace { +using mcpp::build::LinkHost; +using mcpp::build::LinkShape; +using mcpp::build::link_shape; +using Fmt = mcpp::build::dist::Format; +} // namespace + +TEST(LinkShape, LinuxHostTakesTheGenericLineForEveryTarget) { + for (auto f : {Fmt::Elf, Fmt::MachO, Fmt::Pe, Fmt::Wasm}) { + EXPECT_EQ(link_shape(LinkHost::Linux, f, false, true), LinkShape::Generic); + EXPECT_EQ(link_shape(LinkHost::Linux, f, false, false), LinkShape::Generic); + } +} + +TEST(LinkShape, MacOSHostKeepsTheAppleLineForMachOOnly) { + // macOS for macOS, and the iOS rows: the Apple SDK line, as before. + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::MachO, false, false), LinkShape::AppleSdk); + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::MachO, false, true), LinkShape::AppleSdk); + // x86_64-linux-android and wasm32-emscripten on a Mac: the line that + // carries `--target`. + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::Elf, false, true), LinkShape::Generic); + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::Wasm, false, true), LinkShape::Generic); + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::Elf, false, false), LinkShape::Generic); + EXPECT_EQ(link_shape(LinkHost::MacOS, Fmt::Pe, false, true), LinkShape::Generic); +} + +TEST(LinkShape, WindowsHostSeparatesPeFromTargetsNamedByFlag) { + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Pe, true, false), LinkShape::MsvcLinkExe); + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Pe, false, true), LinkShape::PeLld); + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Pe, false, false), LinkShape::PeLld); + // An SDK or retargetable clang aimed at an ELF row: named by `--target`. + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Elf, false, true), LinkShape::Generic); + // The canadian GCC cross to x86_64-linux-musl names its target by prefix + // and keeps the line its CI job verifies. + EXPECT_EQ(link_shape(LinkHost::Windows, Fmt::Elf, false, false), LinkShape::PeLld); +} diff --git a/tests/unit/test_build_profile.cpp b/tests/unit/test_build_profile.cpp index df1cc3b8c..18c306e21 100644 --- a/tests/unit/test_build_profile.cpp +++ b/tests/unit/test_build_profile.cpp @@ -170,3 +170,18 @@ TEST(BuildProfile, UnknownManifestCacheModeFallsThroughToDefault) { EXPECT_EQ(mcpp::build::resolve_cache_mode(m, "bogus"), mcpp::build::CacheMode::Local); } + +// ── #649 E9: the override the command line states, one rule for every verb ─── +// +// `build` preferred `--profile` over the shorthands and `run` preferred the +// shorthands, so one command line built two profiles. Every verb now reads this. +TEST(BuildProfile, ProfileOptionBeatsTheShorthands) { + EXPECT_EQ(mcpp::build::profile_override_from_flags("dev", true, false), "dev"); + EXPECT_EQ(mcpp::build::profile_override_from_flags("release", false, true), "release"); +} + +TEST(BuildProfile, ShorthandsNameTheirProfile) { + EXPECT_EQ(mcpp::build::profile_override_from_flags("", true, false), "release"); + EXPECT_EQ(mcpp::build::profile_override_from_flags("", false, true), "dev"); + EXPECT_EQ(mcpp::build::profile_override_from_flags("", false, false), ""); +} diff --git a/tests/unit/test_config.cpp b/tests/unit/test_config.cpp index f3d1f2d16..e036fbf77 100644 --- a/tests/unit/test_config.cpp +++ b/tests/unit/test_config.cpp @@ -265,7 +265,8 @@ TEST(ConfigIndexMigration, XlingsJsonHealsPrettyLegacyEntry) { auto text = read_all(p); EXPECT_NE(text.find( "\"url\": \"https://github.com/mcpplibs/mcpp-index.git\", " - "\"artifact\": \"https://github.com/xlings-res/mcpp-index\""), + "\"artifact\": { \"GLOBAL\": \"https://github.com/xlings-res/mcpp-index\", " + "\"CN\": \"https://gitcode.com/xlings-res/mcpp-index\" }"), std::string::npos) << text; EXPECT_NE(text.find("\"subos\": \"default\""), std::string::npos); EXPECT_EQ(text.find("mcpp-community/mcpp-index"), std::string::npos); @@ -288,20 +289,44 @@ TEST(ConfigIndexMigration, XlingsJsonHealsCompactLegacyNameAndUrl) { EXPECT_NE(text.find("\"name\":\"mcpplibs\""), std::string::npos); EXPECT_NE(text.find( "\"url\":\"https://github.com/mcpplibs/mcpp-index.git\", " - "\"artifact\": \"https://github.com/xlings-res/mcpp-index\""), + "\"artifact\": { \"GLOBAL\": \"https://github.com/xlings-res/mcpp-index\", " + "\"CN\": \"https://gitcode.com/xlings-res/mcpp-index\" }"), std::string::npos) << text; EXPECT_FALSE(mcpp::fallback::migrate_xlings_json_index_names(p)); std::filesystem::remove_all(dir); } -TEST(ConfigIndexMigration, XlingsJsonSkipsInjectionWhenArtifactPresent) { +// #648 A6. A home seeded with the GitHub-only default artifact gains the CN half +// of the region object once; the migration is idempotent afterwards. +TEST(ConfigIndexMigration, XlingsJsonUpgradesTheFlatDefaultArtifactToTheRegionObject) { auto dir = make_tempdir("mcpp-migrate-xja"); auto p = dir / ".xlings.json"; + { + std::ofstream os(p); + os << "{\n \"index_repos\": [\n" + " { \"name\": \"mcpplibs\", \"url\": " + "\"https://github.com/mcpplibs/mcpp-index.git\", " + "\"artifact\": \"https://github.com/xlings-res/mcpp-index\" }\n" + " ]\n}\n"; + } + EXPECT_TRUE(mcpp::fallback::migrate_xlings_json_index_names(p)); + auto text = read_all(p); + EXPECT_NE(text.find("\"artifact\": { \"GLOBAL\": \"https://github.com/xlings-res/mcpp-index\", " + "\"CN\": \"https://gitcode.com/xlings-res/mcpp-index\" }"), std::string::npos) << text; + EXPECT_FALSE(mcpp::fallback::migrate_xlings_json_index_names(p)); + EXPECT_EQ(read_all(p), text); + std::filesystem::remove_all(dir); +} + +// An artifact base the user wrote is theirs: no CN half is added to it. +TEST(ConfigIndexMigration, XlingsJsonLeavesAUserArtifactBaseAlone) { + auto dir = make_tempdir("mcpp-migrate-xju"); + auto p = dir / ".xlings.json"; std::string body = "{\n \"index_repos\": [\n" " { \"name\": \"mcpplibs\", \"url\": " "\"https://github.com/mcpplibs/mcpp-index.git\", " - "\"artifact\": \"https://github.com/xlings-res/mcpp-index\" }\n" + "\"artifact\": \"https://mirror.example/xlings-res/mcpp-index\" }\n" " ]\n}\n"; { std::ofstream os(p); diff --git a/tests/unit/test_distribution.cpp b/tests/unit/test_distribution.cpp index c4210eb04..34d96f255 100644 --- a/tests/unit/test_distribution.cpp +++ b/tests/unit/test_distribution.cpp @@ -817,3 +817,120 @@ TEST(Distribution, WasmIsSelfContainedByConstructionAndSaysNothing) { // second answer to a question the driver has already answered. EXPECT_TRUE(m.unitFlags.empty()) << m.unitFlags; } + +// --------------------------------------------------------------------------- +// #646 F3a -- one process, one C++ runtime. +// +// The ELF defaults give a program `self-contained` and a shared library +// `toolchain-coupled`. Each is right alone; a program that LOADS such a library +// held two C++ runtimes, and with llvm@22.1.8 it aborted with std::bad_cast. +// An unstated program or test contract takes the shared library's when its +// image loads a C++ shared library of the build. +TEST(Distribution, AProgramThatLoadsACxxSharedLibraryTakesItsContractOnElf) { + const dist::ContractStatement nothing{}; + auto c = dist::role_contracts(nothing, dist::Format::Elf, + {.program = true, .tests = true}); + EXPECT_EQ(c.shared, dist::Contract::ToolchainCoupled); + EXPECT_EQ(c.program, dist::Contract::ToolchainCoupled); + EXPECT_EQ(c.tests, dist::Contract::ToolchainCoupled); + // The intermediate role links no runtime and keeps the program default. + EXPECT_EQ(c.intermediate, dist::Contract::SelfContained); + EXPECT_FALSE(c.programStated); + EXPECT_FALSE(dist::runtime_split(c, dist::Format::Elf, + {.program = true, .tests = true}).has_value()); +} + +// Nothing moves for a program that loads no C++ shared library: the default +// that makes a shipped binary portable is untouched. +TEST(Distribution, AProgramThatLoadsNoCxxSharedLibraryKeepsItsDefault) { + auto c = dist::role_contracts({}, dist::Format::Elf, {}); + EXPECT_EQ(c.program, dist::Contract::SelfContained); + EXPECT_EQ(c.tests, dist::Contract::SelfContained); + auto onlyTests = dist::role_contracts({}, dist::Format::Elf, {.tests = true}); + EXPECT_EQ(onlyTests.program, dist::Contract::SelfContained); + EXPECT_EQ(onlyTests.tests, dist::Contract::ToolchainCoupled); +} + +// Mach-O and PE keep their table: Mach-O's shared default is already +// self-contained (the F2 question is measured separately), and PE resolves +// imports per DLL. +TEST(Distribution, TheFollowingRuleIsElfOnly) { + for (auto f : {dist::Format::MachO, dist::Format::Pe, dist::Format::Wasm}) { + auto c = dist::role_contracts({}, f, {.program = true, .tests = true}); + EXPECT_EQ(c.program, dist::Contract::SelfContained); + EXPECT_FALSE(dist::runtime_split(c, f, {.program = true}).has_value()); + } +} + +// A STATED contract is never moved. A stated self-contained program over a +// coupled C++ shared library is the split, and is named for the caller to +// refuse; a stated self-contained project (the library self-contained too) is +// the documented private-copy arrangement and is not. +TEST(Distribution, AStatedSelfContainedProgramOverACoupledLibraryIsASplit) { + const dist::ContractStatement split{ + .cxxRuntime = "self-contained", .cxxRuntimeShared = "toolchain-coupled"}; + auto c = dist::role_contracts(split, dist::Format::Elf, {.program = true}); + EXPECT_EQ(c.program, dist::Contract::SelfContained); + EXPECT_TRUE(c.programStated); + auto role = dist::runtime_split(c, dist::Format::Elf, {.program = true}); + ASSERT_TRUE(role.has_value()); + EXPECT_EQ(*role, dist::Role::Distributable); + + const dist::ContractStatement whole{.cxxRuntime = "self-contained"}; + auto w = dist::role_contracts(whole, dist::Format::Elf, {.program = true}); + EXPECT_EQ(w.shared, dist::Contract::SelfContained); + EXPECT_FALSE(dist::runtime_split(w, dist::Format::Elf, {.program = true}).has_value()); + + const dist::ContractStatement tests{.cxxRuntimeTests = "self-contained"}; + auto t = dist::role_contracts(tests, dist::Format::Elf, {.tests = true}); + auto testRole = dist::runtime_split(t, dist::Format::Elf, {.tests = true}); + ASSERT_TRUE(testRole.has_value()); + EXPECT_EQ(*testRole, dist::Role::Test); +} + +// `static_stdlib = false` is a statement for every role: all of them are +// host-coupled and nothing splits. +TEST(Distribution, TheOlderSpellingStatesEveryRole) { + const dist::ContractStatement host{.staticStdlib = false}; + auto c = dist::role_contracts(host, dist::Format::Elf, {.program = true, .tests = true}); + EXPECT_EQ(c.program, dist::Contract::HostCoupled); + EXPECT_EQ(c.shared, dist::Contract::HostCoupled); + EXPECT_FALSE(dist::runtime_split(c, dist::Format::Elf, {.program = true}).has_value()); +} + +// --------------------------------------------------------------------------- +// #649 E10 -- clang on the MSVC ABI is given no CRT model, and its driver links +// the static CRT (`-defaultlib:libcmt`). The table recorded `host-coupled` +// beside an artifact that imports no vcruntime DLL. It now records what the +// row delivers, and an explicit request the row does not deliver says so. +TEST(Distribution, ClangOnTheMsvcAbiRecordsTheStaticCrtItsDriverLinks) { + dist::MechanismInput in; + in.format = dist::Format::Pe; + in.stdlibId = "msvc"; + in.msvcCrtModelEmitted = false; + + for (auto requested : {dist::Contract::SelfContained, + dist::Contract::HostCoupled, + dist::Contract::ToolchainCoupled}) { + in.requested = requested; + in.explicitRequest = false; + auto quiet = dist::resolve(in); + EXPECT_EQ(quiet.effective, dist::Contract::SelfContained); + EXPECT_FALSE(quiet.degraded); + EXPECT_FALSE(quiet.deployToolchainRuntime); + EXPECT_TRUE(quiet.unitFlags.empty()); + } + + in.explicitRequest = true; + in.requested = dist::Contract::HostCoupled; + auto undelivered = dist::resolve(in); + EXPECT_EQ(undelivered.effective, dist::Contract::SelfContained); + EXPECT_TRUE(undelivered.degraded); + EXPECT_NE(undelivered.diagnostic.find("not delivered"), std::string::npos) + << undelivered.diagnostic; + EXPECT_NE(undelivered.diagnostic.find("libcmt"), std::string::npos) + << undelivered.diagnostic; + + in.requested = dist::Contract::SelfContained; + EXPECT_FALSE(dist::resolve(in).degraded); +} diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 144ed77df..86ace8691 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -5686,3 +5686,106 @@ kind = "shared" EXPECT_EQ(m.targets[0].kind, mcpp::manifest::Target::Library); EXPECT_EQ(m.targets[0].linkageDefault, "shared"); } + +// #649 E6: a consumer reads "every declared target is a program" as "a tool +// provider that contributes nothing to my graph". A target list the loader +// INFERRED from the tree is not that statement, so the loader says which of +// the two it produced. +TEST(Manifest, TargetsInferredIsDistinguishedFromDeclared) { + auto dir = std::filesystem::temp_directory_path() + / std::format("mcpp_targets_inferred_{}", std::random_device{}()); + std::filesystem::create_directories(dir / "src"); + { + std::ofstream(dir / "src" / "main.cpp") << "int main() {}\n"; + std::ofstream(dir / "mcpp.toml") << "[package]\nname = \"inferred\"\nversion = \"0.1.0\"\n"; + } + auto inferred = mcpp::manifest::load(dir / "mcpp.toml"); + ASSERT_TRUE(inferred) << (inferred ? "" : inferred.error().message); + ASSERT_EQ(inferred->targets.size(), 1u); + EXPECT_EQ(inferred->targets[0].kind, mcpp::manifest::Target::Binary); + EXPECT_TRUE(inferred->targetsInferred); + + { + std::ofstream(dir / "mcpp.toml") + << "[package]\nname = \"declared\"\nversion = \"0.1.0\"\n\n" + "[targets.declared]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n"; + } + auto declared = mcpp::manifest::load(dir / "mcpp.toml"); + ASSERT_TRUE(declared) << (declared ? "" : declared.error().message); + ASSERT_EQ(declared->targets.size(), 1u); + EXPECT_FALSE(declared->targetsInferred); + + std::error_code ec; + std::filesystem::remove_all(dir, ec); +} + +// ── #647 E1 / #649 X4: `[package.metadata]` and the `[package]` key check ──── +// +// The metadata table is kept verbatim for the graph document, as JSON text, and +// `metadata` is a known key. Any other key the parser does not read is reported +// the way `[build]` reports one, so `licence` is no longer accepted in silence. +TEST(Manifest, PackageMetadataIsKeptVerbatimAsJson) { + constexpr auto src = R"( +[package] +name = "x" +version = "0.1.0" +[package.metadata.demo] +resources = "res" +langs = ["en", "zh"] +count = 3 +enabled = true +note = "a \"quoted\" value" +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + EXPECT_EQ(m->packageMetadataJson, + R"({"demo":{"count":3,"enabled":true,"langs":["en","zh"],"note":"a \"quoted\" value","resources":"res"}})"); + EXPECT_TRUE(m->schemaWarnings.empty()) + << (m->schemaWarnings.empty() ? "" : m->schemaWarnings[0]); +} + +TEST(Manifest, PackageWithoutMetadataHasNone) { + auto m = mcpp::manifest::parse_string("[package]\nname = \"x\"\nversion = \"0.1.0\"\n"); + ASSERT_TRUE(m.has_value()) << m.error().format(); + EXPECT_TRUE(m->packageMetadataJson.empty()); +} + +TEST(Manifest, UnknownPackageKeyIsReported) { + constexpr auto src = R"( +[package] +name = "x" +version = "0.1.0" +licence = "MIT" +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + ASSERT_EQ(m->schemaWarnings.size(), 1u); + EXPECT_NE(m->schemaWarnings[0].find("[package] has unsupported key 'licence'"), + std::string::npos) << m->schemaWarnings[0]; +} + +TEST(Manifest, EveryReadPackageKeyIsKnown) { + constexpr auto src = R"( +[package] +name = "x" +namespace = "ns" +version = "0.1.0" +standard = "c++23" +description = "d" +license = "MIT" +repo = "https://example.org" +authors = ["a"] +platforms = ["linux"] +accelerators = ["cuda"] +provides = ["cap"] +requires = ["other"] +exclusive = ["cap"] +std-module = "m" +std-compat-module = "c" +std-module-flags = ["-x"] +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + for (auto const& w : m->schemaWarnings) + EXPECT_EQ(w.find("[package] has unsupported key"), std::string::npos) << w; +} diff --git a/tests/unit/test_pm_index_refresh.cpp b/tests/unit/test_pm_index_refresh.cpp index a4a912a94..c7604be14 100644 --- a/tests/unit/test_pm_index_refresh.cpp +++ b/tests/unit/test_pm_index_refresh.cpp @@ -62,6 +62,27 @@ package = {{ )", name, versions, versions, versions); } + // A descriptor under another namespace, laid out as the index lays out + // `compat.*` packages: pkgs//..lua. + void publish_namespaced(std::string_view ns, std::string_view name, + std::string_view versions) { + auto dir = index_dir() / "pkgs" / std::string(1, ns.front()); + std::filesystem::create_directories(dir); + std::ofstream(dir / std::format("{}.{}.lua", ns, name)) << std::format(R"( +package = {{ + spec = "1", + namespace = "{}", + name = "{}", + type = "package", + xpm = {{ + linux = {{ {} }}, + macosx = {{ {} }}, + windows = {{ {} }}, + }}, +}} +)", ns, name, versions, versions, versions); + } + // Age the refresh marker by writing it and back-dating it. void mark_refreshed(std::chrono::seconds ago = std::chrono::seconds{0}) { auto marker = index_dir() / ".mcpp-index-updated"; @@ -147,6 +168,35 @@ TEST(PmIndexRefresh, OfflineSuppressesEvenAGenuineMiss) { EXPECT_FALSE(refreshes(route, spec, reg.env(), offline)); } +// mcpp-community/mcpp#648. `ftxui = "6.1.9"` omits the namespace, so the +// exact coordinate is (mcpplibs, ftxui), which the index does not carry; the +// resolver then reaches compat.ftxui through the deprecated bare-name rung and +// the build succeeds from disk. The decision must walk the same rung, or every +// build of such a manifest refreshes the index once the debounce has passed. +TEST(PmIndexRefresh, BareNameResolvedThroughTheLegacyRungIsNotAMiss) { + FakeRegistry reg("legacyrung"); + reg.publish_namespaced("compat", "ftxui", R"(["6.1.9"] = { url = "u", sha256 = "s" },)"); + reg.mark_refreshed(std::chrono::hours{99}); + mcpp::pm::IndexMap indices; + auto cfg = reg.config(); + mcpp::pm::IndexRoute route{ &indices, "/nowhere", &cfg }; + + auto bare = version_dep("mcpplibs", "ftxui", "6.1.9"); + bare.namespaceOmitted = true; + EXPECT_EQ(decide(route, bare, reg.env()), RefreshReason::None); + EXPECT_FALSE(refreshes(route, bare, reg.env())); + + // A constraint is checked against the descriptor the rung found. + auto bareRange = version_dep("mcpplibs", "ftxui", "^7"); + bareRange.namespaceOmitted = true; + EXPECT_EQ(decide(route, bareRange, reg.env()), RefreshReason::VersionMiss); + + // The namespace written out states an identity: the rung does not apply, + // exactly as in the resolver, and the miss stands. + auto stated = version_dep("mcpplibs", "ftxui", "6.1.9"); + EXPECT_EQ(decide(route, stated, reg.env()), RefreshReason::DescriptorMiss); +} + TEST(PmIndexRefresh, OfflineDoesNotMisreportAResolvableDependency) { // The opt-outs are applied AFTER the local analysis, so a dependency that // resolved fine is reported as such even offline. Short-circuiting on the diff --git a/tests/unit/test_static_placement.cpp b/tests/unit/test_static_placement.cpp new file mode 100644 index 000000000..a457f8b1a --- /dev/null +++ b/tests/unit/test_static_placement.cpp @@ -0,0 +1,110 @@ +// #646 F1 -- where a static package under a shared library is linked. +// +// A dependency's shared library used to take only its own package's objects, +// so a static package beneath it went into the program and the library bound +// to the program's copy at run time. That works on ELF alone, and only for a +// program that links the package. `place_static_packages` decides the image +// each static package belongs in; these are the graph shapes the plan record +// enumerates (§1.2 of the implementation plan), asserted before any link line +// depends on them. +// +// Package 0 is the root. `shared` maps each shared package to its image count. + +#include + +import std; +import mcpp.build.plan; + +using mcpp::build::place_static_packages; + +namespace { + +using Edges = std::map>; + +} // namespace + +// root -> fw (shared) -> x: x belongs in fw. +TEST(StaticPlacement, AStaticPackageUnderOneSharedLibraryGoesIntoIt) { + Edges edges{{0, {1}}, {1, {2}}}; + auto p = place_static_packages(edges, {{1, 1}}, {}); + ASSERT_EQ(p.intoImage.size(), 1u); + EXPECT_EQ(p.intoImage.at(2), 1u); + EXPECT_TRUE(p.conflicts.empty()); +} + +// root -> fw -> a -> x, fw -> x: the whole static closure goes into fw. +TEST(StaticPlacement, ADiamondInsideOneImageStaysInThatImage) { + Edges edges{{0, {1}}, {1, {2, 3}}, {2, {3}}}; + auto p = place_static_packages(edges, {{1, 1}}, {}); + EXPECT_EQ(p.intoImage.size(), 2u); + EXPECT_EQ(p.intoImage.at(2), 1u); + EXPECT_EQ(p.intoImage.at(3), 1u); + EXPECT_TRUE(p.conflicts.empty()); +} + +// root -> fw1 -> x, root -> fw2 -> x: x has no single image. +TEST(StaticPlacement, TwoImagesReachingOnePackageIsAConflict) { + Edges edges{{0, {1, 2}}, {1, {3}}, {2, {3}}}; + auto p = place_static_packages(edges, {{1, 1}, {2, 1}}, {}); + EXPECT_TRUE(p.intoImage.empty()); + ASSERT_EQ(p.conflicts.size(), 1u); + EXPECT_EQ(p.conflicts[0].package, 3u); + EXPECT_EQ(p.conflicts[0].images, (std::vector{1, 2})); + EXPECT_FALSE(p.conflicts[0].root); +} + +// root -> fw -> x and root -> x: the program and the library both need it. +TEST(StaticPlacement, ThePackageTheRootAlsoLinksIsAConflictNamingTheRoot) { + Edges edges{{0, {1, 2}}, {1, {2}}}; + auto p = place_static_packages(edges, {{1, 1}}, {}); + EXPECT_TRUE(p.intoImage.empty()); + ASSERT_EQ(p.conflicts.size(), 1u); + EXPECT_EQ(p.conflicts[0].package, 2u); + EXPECT_TRUE(p.conflicts[0].root); +} + +// root (itself the shared image) -> x: nothing moves; the root's images +// already take every static package they reach. +TEST(StaticPlacement, ARootOwnedImageKeepsItsStaticPackages) { + Edges edges{{0, {1}}}; + auto p = place_static_packages(edges, {}, {}); + EXPECT_TRUE(p.intoImage.empty()); + EXPECT_TRUE(p.conflicts.empty()); +} + +// root -> fw1 -> fw2 -> x: a closure stops at the next shared package, so x +// belongs to fw2 and not to fw1. +TEST(StaticPlacement, AClosureStopsAtTheNextSharedLibrary) { + Edges edges{{0, {1}}, {1, {2}}, {2, {3}}}; + auto p = place_static_packages(edges, {{1, 1}, {2, 1}}, {}); + ASSERT_EQ(p.intoImage.size(), 1u); + EXPECT_EQ(p.intoImage.at(3), 2u); + EXPECT_TRUE(p.conflicts.empty()); +} + +// A package with two shared targets is two images: its static closure has no +// single home. +TEST(StaticPlacement, OnePackageWithTwoSharedTargetsIsTwoImages) { + Edges edges{{0, {1}}, {1, {2}}}; + auto p = place_static_packages(edges, {{1, 2}}, {}); + EXPECT_TRUE(p.intoImage.empty()); + ASSERT_EQ(p.conflicts.size(), 1u); + EXPECT_EQ(p.conflicts[0].package, 2u); +} + +// A boundary (a distribution package, or a package providing a target layer) +// is neither placed nor walked through. +TEST(StaticPlacement, ABoundaryIsNeitherPlacedNorTraversed) { + Edges edges{{0, {1}}, {1, {2}}, {2, {3}}}; + auto p = place_static_packages(edges, {{1, 1}}, {2}); + EXPECT_TRUE(p.intoImage.empty()); + EXPECT_TRUE(p.conflicts.empty()); +} + +// A header-only package placed into an image contributes no objects, and the +// placement is still well defined. +TEST(StaticPlacement, APackageWithNoObjectsIsPlacedLikeAnyOther) { + Edges edges{{0, {1}}, {1, {2}}}; + auto p = place_static_packages(edges, {{1, 1}}, {}); + EXPECT_EQ(p.intoImage.at(2), 1u); +} diff --git a/tests/unit/test_symbol_provision.cpp b/tests/unit/test_symbol_provision.cpp index eefd60bde..a0db4eeb7 100644 --- a/tests/unit/test_symbol_provision.cpp +++ b/tests/unit/test_symbol_provision.cpp @@ -250,3 +250,46 @@ TEST(SymbolProvision, AConflictRemembersWhetherItsDefinitionIsWeak) { EXPECT_TRUE(conflicts[0].isWeak); EXPECT_FALSE(conflicts[1].isWeak); } + +// #646 F3. GCC emits the static data of an inline entity with STB_GNU_UNIQUE +// (10) so that the loader keeps one copy across RTLD_LOCAL. It is vague linkage +// exactly as STB_WEAK is; reading only the latter reported seven libstdc++ +// objects as a program displacing the library. +TEST(SymbolProvision, TheUniqueBindingIsVagueLinkageAsTheWeakOneIs) { + EXPECT_TRUE(elf::is_vague_linkage_binding(2)); // STB_WEAK + EXPECT_TRUE(elf::is_vague_linkage_binding(10)); // STB_GNU_UNIQUE + EXPECT_FALSE(elf::is_vague_linkage_binding(1)); // STB_GLOBAL + EXPECT_FALSE(elf::is_vague_linkage_binding(0)); // STB_LOCAL +} + +// #646 F3. The std module's initialiser is linked into every C++ image that +// imports `std`, from one object. A provider that defines a name only through +// an object this build linked into both images is removed from that conflict. +TEST(SymbolProvision, ADefinitionBothImagesTakeFromOneObjectIsNotAConflict) { + std::vector conflicts{ + { .name = "_ZGIW3std", .isFunc = true, + .alsoProvidedBy = {"bin/liblib.so"} }, + { .name = "inflate", .isFunc = true, + .alsoProvidedBy = {"bin/liblib.so", "/usr/lib/libz.so.1"} }, + }; + const std::map> shared{ + { "bin/liblib.so", {"_ZGIW3std"} }, + }; + EXPECT_EQ(sp::drop_shared_plan_definitions(conflicts, shared), 1u); + ASSERT_EQ(conflicts.size(), 1u); + EXPECT_EQ(conflicts[0].name, "inflate"); + // The library still defines `inflate` from its OWN object, so it stays a + // provider: attribution is by object, never by the library as a whole. + EXPECT_EQ(conflicts[0].alsoProvidedBy, + (std::vector{"bin/liblib.so", "/usr/lib/libz.so.1"})); +} + +// The rule is provenance, not a name pattern: an initialiser-shaped name that +// no shared object defines is still a finding. +TEST(SymbolProvision, AnInitialiserShapedNameWithoutSharedProvenanceIsStillReported) { + std::vector conflicts{ + { .name = "_ZGIW5other", .isFunc = true, .alsoProvidedBy = {"bin/libother.so"} }, + }; + EXPECT_EQ(sp::drop_shared_plan_definitions(conflicts, {}), 0u); + EXPECT_EQ(conflicts.size(), 1u); +}