diff --git a/.agents/docs/2026-09-14-634-cmake-parity-items-by-home.md b/.agents/docs/2026-09-14-634-cmake-parity-items-by-home.md new file mode 100644 index 000000000..1c4d7c502 --- /dev/null +++ b/.agents/docs/2026-09-14-634-cmake-parity-items-by-home.md @@ -0,0 +1,859 @@ +--- +subject: triage +status: active +--- + +# A framework's CMake parity list: the twenty-one items of #634, read against the code and routed to where each one belongs + +**Status:** active, revision 3. Revision 3 is the implementation's: the plan +and ledger are `2026-09-14-634-implementation-plan.md`, whose §1 states the +refinements adopted before and while implementing; the sections below carry +them where a decision's wording changed (§5.2, §5.3, §5.6). Revision 1 classified the items; revision 2 held every +decision to six properties (simple, low in new surface, observable, +cross-platform, stable for existing manifests, consistent with the rules its +neighbours already follow) and measured every premise revision 1 had left +unmeasured. §1 states the review and §2 lists what it changed. + +Engine code was read at `b8d96844` (mcpp 2026.9.14.1, the version the issue +measured), mcpp-plugins at `9301832` (0.9.3), xim-pkgindex at `19e264e6`, +xlings at `59068d6`, mcpp-index at `f68512d`, HuxerUI at `5356991` (the head of +Sunrisepeak/HuxerUI#7). A statement marked *measured* names where it was run: +*local* is a Linux x86_64 host with the released mcpp 2026.9.14.1, +`gcc@16.1.0`, `llvm@22.1.8`, `xim:android-ndk@30.0.16248370` and mcpp:plugins +0.9.3 (probes in `2026-09-14-634-probes.sh`); *CI* is the temporary pull +request mcpp-community/mcpp#635, which carries only a measurement workflow +(`.github/verify-634/` on that branch) and ran on `macos-15`, `ubuntu-24.04` +(fresh homes, and an API 34 x86_64 emulator) and `windows-2022` against the +released 2026.9.14.1 (runs 34806507856, 34808499807, 34809272832 and 34809856621). A statement marked *read* names a file +and line and was not executed. + +§0 is the ledger. §1 is the review and §2 its changes. §3 lists where the +measured state differs from the issue. §4 states the routing rule and the +rules the decisions share. §5 to §7 take the items by home. §8 is what the +project can do today, §9 the defects found beyond the issue, §10 the order and +tasks, §11 every reading with its source, §12 the self-review. + +## 0. The ledger + +"Home" is where the change belongs under the routing rule of §4: **engine**, +**official plugin** (mcpp-plugins), **project** (HuxerUI or Lib-Live2D), +**payload** (xim-pkgindex), **xlings**, **index** (mcpp-index). + +| # | real | home | decision (§) | evidence | +|---|---|---|---|---| +| A1 | defect | engine | a matching conditional dependency declaration replaces the unconditional one on its rows; a package states its form on a row as `[target..targets.] kind` (§5.1) | measured local: the conditional `linkage` is dropped on the host row; no manifest among 509 declares one key in both tables; an unconditional `kind = "shared"` already selects the shared form | +| A2 | defect | engine; nine manifests | a `path`/`git` dependency's identity is its manifest's; a differing key warns and adopts it; two identities over one source are refused before scanning (§5.2) | measured local | +| A3 | defect | engine; bundle placement in the plugin (B1) | the Android and Mach-O rows read their closure with the PE row's function; Android stages into `lib/`, Mach-O beside the program; `walked` means complete (§5.3) | measured local (Android), CI (Mach-O) | +| A4 | defect | engine | an ELF shared library's default SONAME is its file name (§5.4) | measured local; bionic's documented requirement | +| A5 | gap | engine (one key); suites as packages | `[test] discover` globs; a suite built from several sources is a workspace member, as every divergent program already is (§5.5) | read | +| A6 | defect, and a question the defect was hiding | engine | the static C++ runtime is located by asking the driver for the effective target, so test programs on the Android rows are self-contained; every runner receives the artifact's runtime files as `MCPP_RUNTIME_FILES`, which `adb-run` transfers (§5.6) | measured CI (emulator), local | +| A7 | resolved by the ecosystem's existing surface | engine (one accessor); one recipe | the SubOS pkg-config view is complete on a fresh home; the engine names it with `mcpp::pkg_config_libdir()` (§5.7) | measured CI (fresh home), local | +| A8 | no engine change now | project | library resources already reach the application; a graph API waits for a need transitive deploy cannot answer (§5.8) | measured local | +| A9 | gap | engine | the engine states the target's platform floor as a fact and dependencies use the existing `version-floor` requirement, which refuses (§5.9) | measured local | +| A10 | gap | engine | `run`, `test` and `pack` declare `--toolchain` (§5.10) | measured local | +| X | observability gap revision 1 missed | engine | the resolved dependency graph is recorded in `resolution.json` and printed by `mcpp why deps` (§5.11) | measured local | +| B1 | real; needs one engine defect fixed (§9 item 11) | official plugin | `dist-apple` places staged dylibs in the bundle's framework directory; the rpath arrives at link time through `mcpp::link_flag` (§6.1) | measured local (`link_flag` reaches the link), CI (run 2 found the link-time rpath anchored to the package directory, an engine defect, and an ad-hoc signed bundle verifying; run 4 measured a bundle whose program carries the literal rpath from its link: it verifies, runs, and fails without the framework) | +| B2 | feature | official plugin | a `dmg` format of `dist-apple` (§6.2) | measured CI (create, verify, mount with the `Applications` link, detach) | +| B3 | small engine gap, plugin, payload | engine; official plugin; payload | `--format ` runs through the named runner `` when one exists (§6.3) | measured CI (`--format app --runner app` exits 7 with the program's output and arguments; without a runner the bundle directory is executed and refused, exit 126) | +| B4 | feature | official plugin; payload | a `setup` format of `dist-wix` (§6.4) | measured CI: the archives link on both MSVC-ABI toolchains and a program calling `DutilInitialize` runs, exit 0, on both | +| B5 | two plugin defects, then a feature | official plugin; payload | fix the second-pack loss and the multi-ABI read, report refusals through `mcpp::warning`, then `aab` (§6.5) | measured local | +| B6 | feature | `rules-metal`: official plugin; `tools-esbuild`: project | promote the Metal rule; keep esbuild in the project (§6.6) | read | +| C1 | feature | payload | `macapp-run` execs the bundle executable (§7.1) | measured CI (the main bundle and its resources resolve when `Contents/MacOS/` is executed directly) | +| C2 | real | payload; plugins CI | a bundle whose executable does not load UIKit is installed and its installed executable spawned, which returns the application's status and kept its output every time; a UIKit application keeps `simctl launch`, whose status simctl does not report, and the runner says so (§7.2) | measured CI | +| C3 | already answered | payload (close-out) | anchor `mbanative.dll`, verify installs, add the `.wixext`, name the ABI (§7.3) | measured CI, and the payload reader's hashes | +| C4 | real, and usage-solvable today | engine (small); payload CI | a fresh home's `config.toml` override already serves a branch checkout; the engine makes the same key reach an existing home (§7.4) | measured CI | +| D1 | ecosystem data | index | after HuxerUI's release (§7.5) | read | + +## 1. The review + +Each decision was held to the same six questions. The table records the +answers; §2 lists the decisions the review changed. + +| decision | new surface | observable through | rows measured | what stays byte-identical | consistent with | +|---|---|---|---|---|---| +| A1 replacement | none | the graph record (X): the declaring table | host (local; the issue measured Android) | every manifest that does not declare one key twice (509 of 509 scanned) | every other conditional scalar: last matching section wins | +| A1 per-row kind | one conditional sub-table, one key | the graph record: form and reason | host (local) | packages without the sub-table | `kind = "shared"` as a package constraint (`linkage_form.cppm:232-233`) | +| A2 | one warning | the warning; the graph record: key and identity | host | manifests whose keys match | SPEC-001 §1.2 (identity is what the content declares); the short-name check already refuses | +| A3 | additive manifest lines | `closure =` and `needs` lines | Android (Linux host), macOS | the ELF host tree; PE | `pe_closure`; PE's layout beside the program | +| A4 | none | `readelf -d` | host, Android | consumers' `DT_NEEDED` | Mach-O's default install name | +| A5 | one key | `mcpp test --list` | host | the default glob's names | `[build] sources` globs; the target model's rule for divergent programs | +| A6 static runtime | none | test programs' `DT_NEEDED` | Android emulator | the LLVM payload's archives (same answer by query) | `clang.cppm`'s driver query | +| A7 | one accessor | the accessor's value | fresh home | no environment default | the `toolchain_*()` accessor family | +| A9 | three fact names | the refusal names the fact and the key | Android (local) | no manifest key | `version-floor`; `requires_abi` refuses | +| A10 | three option declarations | the `Resolved` line | host | the side channel | `build --toolchain` | +| X | one JSON section, one `why` topic | itself | host | existing `resolution.json` fields | `resolution.json`'s own `package_json` | +| B3 | one lookup rule | the `Running` line names the runner | macOS | plain `run`, `test` | named runners (docs/41) | + +## 2. What revision 2 changed, and why + +1. **A1: key-wise refinement became whole replacement.** Revision 1 merged the + keys a conditional table writes and unioned `features`; that needed the + parser to record which keys were written. A scan of 509 manifests (mcpp's + examples and tests, the plugins, HuxerUI, Lib-Live2D, the mcpplibs + checkouts) found none that declares one dependency in both tables, so the + richer rule has no user, and replacement is the rule every conditional + scalar already follows. +2. **A1: `requires_linkage` became a per-row `kind`.** Everywhere else a + `requires_*` key refuses (`requires_abi`, `version-floor`); a + `requires_linkage` that *selects* would give one prefix two meanings. The + per-row form of the existing constraint needs no resolver change, since + `kind = "shared"` already selects the shared form under a default request + (measured, §5.1). +3. **A3: Mach-O stages beside the program instead of changing the link.** + Revision 1 added `@loader_path/../lib` to every Mach-O link with a shared + dependency, which changes command lines and fingerprints for every such + macOS build. The consumer already carries `@loader_path`, so a dylib staged + beside the program loads with no link change; that is PE's layout. + Measured on `macos-15`: the staged program runs, and fails without the staged dylib (§5.3). +4. **A5: named test targets and labels became one glob key.** Exclusive + sources per target contradict the target model, which states that objects + are compiled once and linked into every target and that divergent programs + are workspace members (`types.cppm:175-180`). A multi-source suite is + therefore a package; the one missing piece is pointing discovery away from + a `tests/` another build system owns. +5. **A6: a defect replaced the question.** On the emulator no test loaded: + every test program needed `libc++_shared.so`, because the engine's static + libc++ lookup does not find the NDK's archives and the contract degrades + (measured, §5.6). The deployed-file question was then measured with the + runtime linked statically. +6. **A7: "the view is incomplete" was this machine's history.** On a fresh + home the view resolves `gtk4` completely; the one absent `.pc` belongs to a + recipe with no pkg-config handling (measured, §5.7). +7. **A8: the graph file is not built now.** The observability half moved to X, + where it serves every resolution change in this record. +8. **A9: `requires_platform` with raising became engine facts with refusal.** + The floor is fixed in the compiler's `--target` at toolchain resolution + (`prepare.cppm:3670-3673`), before the dependency worklist starts + (`:6499`); raising it from the graph would re-resolve the toolchain. A + refusal reuses `version-floor`, which knows no meanings, and leaves the + application's floor the application's decision. +9. **B6: the optional Apple SDK accessor was dropped.** A rule can ask + `xcrun --sdk --show-sdk-path`; no engine surface is needed. +10. **C4: no xlings change.** A fresh home's override serves a branch checkout + and survives an index update (measured, §7.4). The engine's gap is that the + same key in an existing home is ignored without a word. +11. **X was added**, because `mcpp why deps` reads only `mcpp.lock`'s text and + `resolution.json` holds no graph (measured, §5.11), so three of this + record's decisions would otherwise be observable only through warnings. +12. **New defects were measured**: two in `dist-apk` (§6.5); in the engine, the + Android static runtime (§5.6), Mach-O loader tokens anchored in rpaths + (§9 item 11, found by B1's measurement), the link-form degradation's + missing reason (§5.1), and the version-floor message's wording (§5.9). + +## 3. Where the measured state differs from the issue + +1. **A1 is not Android-specific**, and the issue's proposed requirement form + is weaker than what the engine already does for `kind = "shared"`. +2. **A2 is silent even without a duplicate**, and mcpp's own curriculum + carries the shape six times. +3. **A3 is worse than reported**: the engine's own `--format dir` on the + Android row ships an archive without its closure and says `walked`. +4. **A4 is wider than reported**: the Android application object lacks + `DT_SONAME` too; the Mach-O half is already done. +5. **A6 cannot run at all today on the emulator**, for a reason the issue did + not reach: the C++ runtime, not the data files. +6. **A7's 37 pins are unnecessary on a fresh home**: the view already resolves + the GTK closure. +7. **A8's resources already reach the application.** +8. **A9 has a working form today** through a dependency's build program. +9. **A10 works today** through `MCPP_TOOLCHAIN`. +10. **B3 works today** through a named runner, and HuxerUI's `open -W` default + runner also wraps a plain `mcpp run` (measured: a plain `mcpp run` handed `-W /rapp` to `open` and exited 0 for a program that exits 7). +11. **B5's premise does not hold**: `dist-apk` 0.9.3 refuses the multi-ABI + stage, and a second pack drops the dependency's library (measured). +12. **C3 is already answered**, and the archives link under both MSVC-ABI + toolchains once the system import libraries are named. +13. **C4 needs no xlings change**: the override works on a fresh home. + +## 4. The routing rule, and the rules the decisions share + +**The routing rule (given by the maintainer for this triage).** mcpp is a +general build engine and takes general capabilities only; special +functionality is built on the plugin system, a plugin a second project would +use belongs to the official plugins, a project-specific one to the project. +Each item gets one home, tried in this order, and its section says why the +cheaper homes do not suffice: usage; project plugin; official plugin; +ecosystem data (payload, runner program, xlings, index); engine, stated as the +general capability rather than the product feature. The rule filters +features, not defects. + +**Rule 1. A declaration never disappears.** It is applied or reported (A1, A2, +the runner tokens of §9, C4's existing home). + +**Rule 2. A closure is read, not run, and its manifest says what happened** +(A3; `pe_closure` is the complete instance). + +**Rule 3. The engine asks the toolchain instead of guessing its layout** (A6's +archives; `clang.cppm:130-150` already asks `--print-file-name` for the same +reason), and it does not rewrite what belongs to the loader (the rpath tokens +of §9 item 11). + +**Rule 4. Name the existing mechanism before adding one** (A7's view, A8's +deploy, A9's floor, A10's variable, B3's named runners, C3's payloads, C4's +override). + +**Rule 5. A change to resolution leaves a record a test can read** (X): a +criterion reads state, not a warning's wording. + +## 5. Engine items + +### 5.1 A1: one identity, a different link form per row + +**What the code does.** `merge_conditional_config` folds matching +`[target.]` sections before resolution: build inputs are appended +after the base, so the conditional rule wins (`prepare.cppm:395-398`); tools +replace the base entry (`:266-292`); scalars follow the last matching section +(`:428-438`). Dependencies do the opposite, `m.dependencies.insert(...)` +keeping the unconditional entry (`:458-463`), a rule dating from Phase 1b +whose record gives no reason +(`2026-06-29-manifest-environment-and-platform-design.md:279-287`) and which +#359 moved into the funnel every package's manifest passes through +(`df4f75e2`; `prepare.cppm:7227-7230`). A modifier-only table is TOML's nested +table: `huxerui.huxerui = { linkage = "shared" }` names no source +(`toml.cppm:1347-1353`), reaches `load_selector_dep_table` as the selector +`huxerui.huxerui.linkage` (`:1599-1629`), and the conditional tables are loaded +under the section string `"dependencies"` (`:3015-3024`), which is why the +warning names `[dependencies]`. + +**Measured (local, `x86_64-linux-gnu`).** The first spelling exits 0 with no +`libfw.so` and no diagnostic; the second exits 2 with `[dependencies] +demo.fw.linkage = 'shared' is not a version RANGE` and `tried: (demo.fw, +linkage)`; two complementary predicates produce `bin/libfw.so` and `NEEDED +libfw.so` (the control). A dependency whose own target is `kind = "shared"`, +consumed by a root that writes no `linkage`, is linked shared; a root that +writes `linkage = "static"` gets `warning: demo.fw@0.1.0 is linked as a +shared library: the requested form is not available here`, a sentence that +does not name the package's own statement. A scan of 509 manifests found none +declaring one dependency key in both an unconditional and a conditional table. + +**Decision.** + +1. *Replacement.* A matching conditional declaration of a key replaces the + unconditional declaration of that key on the rows its predicate selects; + several matching sections apply in manifest order, last wins. No warning: + this is what every conditional scalar does. The declaring table is recorded + in the graph record (§5.11). The same rule applies to `dev-dependencies`, + `build-dependencies` and `feature-deps`. +2. *The right words for the modifier-only table.* Conditional dependency + tables are labelled with their full section name, separately from the + lookup path `is_namespace_table` builds (`toml.cppm:1541-1546`). When a + selector's last segment is a dependency option (`linkage`, `features`, + `default-features`, `visibility`, `tools`, `host-module`, `reexport`, + `backend`) and its value is a string, the warning at that one site says + that the table names no source and gives the restated form, instead of the + version-range warning. The grammar does not change, so a package named like + an option remains addressable. +3. *A package states its form on a row.* + `[target.'cfg(env = "android")'.targets.huxerui] kind = "shared"` is the + per-row form of `[targets.huxerui] kind = "shared"`: merged in the first + pass, accepted only for a target the package declares unconditionally, only + with the library forms `lib` and `shared`. Resolution does not change: on a + matching row the package is `declaredShared` and the shared form is + selected; an explicit `linkage = "static"` on the root's edge gets the + existing degradation, whose sentence now names the package's statement and, + for a row, its selector. An engine that predates this sub-table skips it + without a word (the `[target.]` sweep exempts every table, + `toml.cppm:2756`), as `requires_abi` records for itself; a package relying + on it states its engine floor, and the sweep learns the sub-table names the + parser reads, so the next new sub-table is reported by engines from this + one on. + +HuxerUI's applications then write one unconditional line and the framework +writes the Android form once. + +**Criteria.** The first spelling on the host row gives `bin/libfw.so` and +`NEEDED libfw.so`, on a row the predicate does not select a static link, and +the graph record names the conditional table. The modifier-only table exits +non-zero naming the conditional table and the restated form. A package with +`[target.'cfg(os = "linux")'.targets.fw] kind = "shared"` consumed by a root +that writes no `linkage` gives `bin/libfw.so` on Linux and a static link on a +row the selector does not match; with `linkage = "static"` on the root's edge +the degradation names the selector and `--strict` exits non-zero. Negative +direction: e2e 86, 195 and 328 are unchanged, and the manifests of the scan +build with identical command lines. + +### 5.2 A2: a dependency key that names another identity than its manifest + +**What the code does.** The worklist dedups by the identity the consumer's key +normalises to (`prepare.cppm:6531-6534`). After a `path` or `git` manifest is +loaded, only the short name is compared (`:7242-7260`), so a namespace +mismatch passes and the package's own identity and the resolver's key +disagree from then on; every reader that builds the qualified name from the +manifest (`:11315-11317`, `:5620-5630`) sees the manifest's answer. SPEC-001 +takes identity from what the content declares (§1.2, §3.4). + +**Measured (local).** Two edges to one directory, keyed `huxdemo.fw` and `fw`, +fail in the scanner naming one file twice; one edge keyed `fw` builds with no +diagnostic. Nine `path` edges among 46 scanned write a key the target manifest +does not declare: six in mcpp's `examples/04`, `08` and `12`, three in +HuxerUI's `huxerui-build-rules`, `huxerui-tools` and `huxerui-tests`. + +**Decision.** A `path` or `git` dependency's identity is the one its manifest +declares. A key that normalises to another identity adopts the declared one +and warns, naming the requester, the key, its normalisation and the +declaration. Implemented by recording the identity resolved from each +canonical source, so a second key over the same directory or commit takes it +without the manifest being loaded again (revision 3: a key-to-identity alias +would also have captured a `version` dependency written with the same key); an +identity already resolved from a different source follows the #630 decision +table (`docs/05-dependencies.md:126-133`). Two identities over one canonical +source (possible only for manifests without a namespace, which keep taking the +key's) are refused before scanning, naming both. The scanner names the two +packages when one module has two providers. The nine manifests are corrected +with the change (mcpp's six in the same PR). Adoption rather than refusal: +nine builds that work today would break for a defect whose resolution is +known, and SPEC-001's reason against candidate search (index state +retargeting a dependency) does not apply to a source the manifest line fixes. + +**Criteria.** The two-edge fixture builds, compiles `fw.cppm` once and warns +once with a sentence naming `fw`, `mcpplibs.fw` and `huxdemo.fw` (the domain +string is not rendered and is not asserted); the graph record shows one +package `huxdemo.fw` with both keys. Two keys `a.fw` and `b.fw` over a manifest +without a namespace are refused before scanning. Negative direction: matching +keys produce no warning. + +### 5.3 A3: the staged closure on the Android and Mach-O rows + +**What the code does.** `pack::run` dispatches the Android shared-object row, +PE and wasm before the closure walk (`pack.cppm:1359-1377`). PE reads its +closure: a breadth-first walk over `needed_names`, resolved against +`searchDirs`, which start at the artifact's `bin/` (`:999-1053`, `:497`). The +Android row copies the application object and the deployed files and returns +`ClosureResult{}`, whose `walked` defaults to `true` (`:1281-1337`, `:247`). +Mach-O programs reach `finish_without_closure` (`:1423-1438`) although the +reader landed in #630. Consumers of a graph-built dylib link with +`-Wl,-rpath,@loader_path` (`plan.cppm:583`), and dylibs are named +`@rpath/` (`ninja_backend.cppm:288`). + +**Measured.** Local, `mcpp pack --target x86_64-linux-android --format dir`: +exit 0, `Packed`, a tree holding only `lib/libapp.so`, whose `DT_NEEDED` names +`libfw.so` and `libc++_shared.so`, and a manifest `closure = walked`. The NDK +sysroot separates the two kinds of name: `usr/lib///` holds the +platform stubs, `usr/lib//libc++_shared.so` ships with the +application. CI, `macos-15`: the program in the build tree runs (exit 7) with `LC_RPATH @loader_path`, and the dylib is named `@rpath/libfw.dylib`; copied with its dylib into a fresh `stage/bin/` and run after the build tree was moved away, it exits 7; with the staged dylib removed it stops with `dyld: Library not loaded: @rpath/libfw.dylib` (exit 134). Today `mcpp pack --format dir` refuses with `cannot package the Mach-O program 'app' yet`. + +**Decision.** + +1. `pe_closure` generalises to one reader-driven function for PE, the Android + rows and Mach-O, with a per-format rule for platform names: PE as today; + Android, a name present as a stub in the payload sysroot's API-level + directory; Mach-O, `/usr/lib/` and `/System/Library/`, with `@rpath`, + `@loader_path` and `@executable_path` resolved by `resolve_macho_names`. +2. The Android row stages resolved libraries into `lib/` (per ABI on the + several-triple route), `libc++_shared.so` included. +3. Mach-O stages resolved dylibs beside the program in `bin/`, where the + existing `@loader_path` finds them: no link change, no load-command edit, + no re-signing. Measured on `macos-15` (§11, C5). +4. `walked` is written only when every needed name resolved to a staged file + or a platform name; otherwise `not-walked` with the names, and `dir` and + `tar` refuse. +5. The stage manifest gains one additive line per needed name, + `needs`, `needsplatform` or + `needsunresolved` (revision 3: TAB-separated, because an + install name or a directory can contain a space, and a third value so a + provider reads a failure as state). + +The ELF host row keeps `ldd_parse`; moving it onto the reader is a separate +change with its own byte-identical criterion. `dist-apk` then reads `lib/` +from the stage and deletes its own walk, which removes the defect of §6.5 by +construction. + +**Criteria.** Android: the `dir` tree holds `lib/libapp.so`, `lib/libfw.so` +and `lib/libc++_shared.so`, and the manifest says `walked` with `libc.so`, +`libm.so`, `libdl.so` as platform; a needed library deleted after the build +makes `dir` refuse naming it. macOS: the staged program runs after the build +tree is moved, and fails with "Library not loaded" when the staged dylib is +removed. Negative direction: the ELF host tree is byte-identical and its +manifest differs only by `needs` lines. + +### 5.4 A4: the default SONAME of an ELF shared library + +**What the code does.** `shared_soname_flag` names `@rpath/` on Mach-O +always, nothing on PE, and a SONAME on ELF only when one is declared +(`ninja_backend.cppm:274-290`); the ELF default was never decided (`93398f40`). + +**Measured (local).** No `DT_SONAME` on `libfw.so` built through `linkage = +"shared"` on the host and Android rows, nor on the Android application object +`libapp.so`. bionic's `android-changes-for-ndk-developers.md` lists "Missing +SONAME (Enforced for API level >= 23)": each ELF shared object must carry one, +and the file name is used when it is missing. + +**Decision.** An ELF shared library without a declared `soname` is linked with +`-Wl,-soname,`. Consumers link with `-l` +(`plan.cppm:582-584`) and already record that name. + +**Criteria.** `SONAME lib.so` on the host and Android rows, and on the +application object. Negative direction: a consumer's dynamic section is +byte-identical, and a declared `soname` keeps its value and its alias. + +### 5.5 A5: tests from several sources, and tests that live elsewhere + +**What the code does.** The test set is the literal pattern `tests/**/*.cpp`, +one program per file (`test_targets.cppm:38`). `[targets]` has no test kind +(`toml.cppm:1111-1124`). The target model compiles objects once, links them +into every target, and sends a program that must diverge to a workspace +member (`types.cppm:175-180`). + +**Decision.** `[test] discover = ["tests/**/*.cpp"]` is the default, a list of +globs in the vocabulary `[build] sources` already uses, `!` exclusions +included; `[]` disables discovery; a test's name is its path relative to its +glob's fixed prefix, so the default yields today's names. A suite compiled +from several sources is a package, as any divergent program is: a support +library carrying Catch2 and the include directories, one package per suite +with its own `[build] sources` and one `tests/main.cpp`, selected with +`mcpp test -p `. No `[[test]]` table, no per-target exclusive sources, +no labels. + +**Criteria.** `[test] discover = ["checks/**/*.cpp"]` runs `checks/a.cpp` and +not a failing `tests/b.cpp`; `discover = []` runs none. Negative direction: a +package without the key lists the same names as today. + +### 5.6 A6: test programs on the emulator and the simulator + +**What the code does.** `mcpp test` resolves its runner as `mcpp run` does +(`execute.cppm:643-711`, `2423`); a runner receives the artifact path only +(`:1691`); `adb-run` pushes that one file and runs it from `/` +(`xim-pkgindex:pkgs/a/android-platform-tools.lua:596-623`). The static libc++ +of the self-contained contract is looked for only under the LLVM root's `lib/` +(`flags.cppm:1064-1079`, `1182-1183`), and a miss degrades the contract to +toolchain-coupled (`distribution.cppm:718-726`). + +**Measured.** CI, API 34 emulator, run 1: `adb devices` lists +`emulator-5554`; all three tests fail with `CANNOT LINK EXECUTABLE +"/data/local/tmp/runs": library "libc++_shared.so" not found`, after the +engine warned `cxx_runtime: test target: this toolchain ships no +libc++.a/libc++abi.a; using toolchain-coupled`. Local: the same tests need +`libc++_shared.so`; with `-static-libstdc++` they need only `libm.so`, +`libdl.so` and `libc.so`. The drivers answer where their archives are for the +effective target: the NDK names its API-level `libc++.a` (a linker script, +`INPUT(-lc++_static -lc++abi)`) and `libc++abi.a`, the LLVM payload names +`lib/x86_64-unknown-linux-gnu/libc++.a`. CI run 2, with the runtime static: +`runs` passes; `reads_relative` fails with `open failed: /data/local/tmp/data/data.txt`, because `adb-run` pushed the executable and not its deployed file; `reads_host_path` fails with `open failed: `. CI, iOS simulator: all three tests pass through `simctl-run`, `reads_relative` (the deployed file beside the build-tree program) and `reads_host_path` (the build machine's path) included: the simulator reads the host's filesystem, so the runtime-files contract below serves device-like runners such as `adb-run` and changes nothing on the simulator. + +**Decision.** + +1. *The static runtime is located by asking the driver.* `find_archive` asks + the compiler `--print-file-name=` with the effective target flag, + the query `clang.cppm:130-150` already runs to find the std module, and + falls back to today's directory search. The self-contained contract then + holds on the Android rows for tests and programs; the shared-object role + keeps `libc++_shared.so`, which `dist-apk` packages. +2. *Runners learn the artifact's runtime files.* Every runner, for `mcpp run` and `mcpp test`, receives `MCPP_RUNTIME_FILES`: the path of a file of `` lines, taken from the plan's deployed files and the shared libraries the artifact links (revision 3: TAB-separated, the file always exists, and a test on a row whose dependency is shared needs that library on the device too). The argv grammar does not change and a runner that needs no files ignores the variable. `adb-run` pushes each file beside the pushed program and runs the program from that directory. A test locates its data relative to its own directory; a compiled-in build-machine path is the project's to change. + +**Criteria.** On the emulator the fixture without any `ldflags` passes `runs`, +and its test programs name no `libc++_shared.so`; with `adb-run` reading +`MCPP_RUNTIME_FILES`, `reads_relative` passes too. Negative direction: the LLVM +payload's link line on the host row is unchanged, and on the iOS simulator, +whose runner ignores the variable, the three tests still pass. + +### 5.7 A7: a payload's recipe closure and pkg-config + +**What the code does.** `xpkg_dir` answers for the addresses the owning +manifest and its compiled-in host modules declare (`prepare.cppm:5720-5791`). +`gtk4` declares its dependencies as floors, relocates its `.pc` files and +declares them into `/usr/lib/pkgconfig` (`xim-pkgindex: +pkgs/g/gtk4.lua:60-84`, `110`, `132`). Payloads are provisioned into the global +registry so that its SubOS is the sysroot (`prepare.cppm:1508-1530`), but +`toolchain_sysroot()` is set only in the `Sysroot` link mode +(`prepare.cppm:1272-1275`), and the bundled LLVM takes `PayloadFirst` +(`linkmodel.cppm:469-476`). + +**Measured.** CI, a fresh home declaring only `xim:gtk4` (xlings 2026.9.5.1, +50 payloads installed): the view holds 143 entries and +`PKG_CONFIG_LIBDIR= pkg-config --cflags --libs gtk4` exits 0 with every +path under the payload store; of every `.pc` an installed payload ships, only +`libxml-2.0.pc` is absent from the view, and `libxml2.lua` has no pkg-config +handling. Local, a registry with older installs: the same command fails on +`zlib`, `x11` and five more; that is machine history, not a property of +provisioning. + +**Decision.** `mcpp::pkg_config_libdir()` (`MCPP_PKG_CONFIG_LIBDIR`) returns +the registry SubOS's pkg-config directories, independent of the link mode; an +accessor, not an environment default, so a package using the host's +pkg-config (#493) is unaffected. `libxml2` declares its `.pc` files. No +`xpkg_dir` over a closure and no payload versions in `mcpp.lock`: the view +answers the need, and locking payloads is a decision about the payload plane +as a whole. + +**Criteria.** A fresh home declaring only `xim:gtk4` runs `pkg-config +--cflags --libs gtk4` with the accessor's value and exits 0. Negative +direction: with the value replaced by an empty directory it exits non-zero. + +### 5.8 A8: the resolved graph and package metadata, from a build program + +**Measured (local).** Application, `liba`, `libb`, where `libb`'s build program +deploys `strings.txt`: the file is in the application's `bin/resources/libb/` +after `mcpp build` and in the `dir` pack. + +**Decision.** No engine change now. A library's resources reach the +application; merging them into one package is a rule in HuxerUI's own rule +module, or a load-time merge. A build-program channel to the graph waits for a +need that transitive deploy and re-exported provisions cannot answer; when it +comes, it reads the record of §5.11. + +### 5.9 A9: a platform floor a dependency needs + +**What the code does.** `min_platform_version` reads the root manifest, then +the payload's `platform_floor`, then the NDK's metadata +(`prepare.cppm:1619-1670`), and the answer enters the compiler's `--target` at +toolchain resolution (`:3670-3673`), before the dependency worklist (`:6499`). +`version-floor` requirements are compared with stated facts before compiling +(`:7804-7849`); the module that parses them knows no meanings +(`version_floor.cppm`, header). + +**Measured (local, Android row).** A dependency whose build program states +`mcpp::fact("demo.android-api", mcpp::min_platform_version())` and +`mcpp::floor("demo.android-api >= 23")`: a root stating 21 is refused before +compiling (`error: \`fw\` requires demo.android-api >= 23, and this machine has +21`), a root stating nothing likewise (the NDK's default is 21), a root stating +24 builds with `x86_64-unknown-linux-android24`, and the host row is silent. + +**Decision.** The engine states the target's floor as a fact, in the +platform's words: `android.api-level`, `ios.deployment-target`, +`macos.deployment-target`. A dependency writes `[[runtime.requirements]] kind += "version-floor" value = "android.api-level >= 23"`; a floor whose fact the +row does not state is silent, so no selector is needed. The refusal names the +manifest key that sets the fact, and the message says "this build targets" +where it says "this machine has". No raising, for the ordering reason above and +because an application's floor is a product decision. + +**Criteria.** Android row: a dependency requiring 23 and a root stating 21, or +nothing, is refused naming `min_api_level`; a root stating 24 builds. Negative +direction: the requirement is silent on the host and iOS rows. + +### 5.10 A10: `--toolchain` on `run`, `test` and `pack` + +**What the code does.** The pre-parse loop publishes `--toolchain` as +`MCPP_TOOLCHAIN` for every command (`cli.cppm:195-199`); only `build` and `why` +declare the option (`:364`, `:538`). + +**Measured (local).** `MCPP_TOOLCHAIN=llvm@22.1.8` makes `mcpp test` compile +with `Clang 22.1.8` and `mcpp pack` resolve `llvm@22.1.8`; the three commands +refuse `--toolchain`. + +**Decision.** Declare the option on the three subcommands, not globally +(`why --toolchain` is a query with its own declaration). + +**Criteria.** `mcpp test --toolchain llvm@22.1.8` builds with clang while the +default is gcc. Negative direction: without the flag the default resolves. + +### 5.11 X: the resolved dependency graph, recorded + +**What the code does.** `mcpp why deps` prints the lines of `mcpp.lock` +(`doctor.cppm:1089-1110`); `path` dependencies are not locked, so a project of +path dependencies shows "no mcpp.lock". `resolution.json` holds `toolchain` +and `runtime` (`prepare.cppm:12700-12870`). + +**Measured (local).** For an application with one `path` dependency, +`mcpp why deps` prints `(no mcpp.lock — run mcpp build or mcpp update)` after a +build, and `resolution.json` has the keys `runtime`, `schema_version` and +`toolchain` only. + +**Decision.** `resolution.json` gains `graph`: for each package the existing +`package_json` (identity, version, source provenance), the keys its requesters +wrote with the requester and the declaring table, and its link form with the +reason (`default`, `requested`, `package kind`, `row kind`). `mcpp why deps` +prints the same before the lock's lines. An additive field under the +machine-output guarantees (docs/50 §7). + +**Criteria.** The A1, A2 and per-row-kind fixtures assert on `graph` rather +than on warnings. Negative direction: every existing `resolution.json` field +is unchanged. + +## 6. Official plugin items + +### 6.1 B1: `dist-apple` carries the graph's dylibs + +**What the plugin does** (read). It registers `app` (`dist/apple.cppm:952`), +copies the launcher and maps the rest of `/bin/` into the resource +directory (`:734-777`); `/lib/` is never read; it signs only with an +identity (`:815-841`). + +**Measured.** Local: `mcpp::link_flag("-Wl,-rpath,/m634/marker/Frameworks")` +emitted by a host module and called from the root's build program appears in +the root program's `RPATH`. CI, `macos-15`: the rpath a host module adds with `mcpp::link_flag("-Wl,-rpath,@executable_path/../Frameworks")` arrives in the binary as `/@executable_path/../Frameworks`. `normalize_ldflag` anchors every relative rpath to the package root and exempts only a leading `$` (`flags.cppm:266-283`, with a copy for dependencies at `prepare.cppm:6141-6158`), so ELF's `$ORIGIN` survives and Mach-O's `@executable_path`, `@loader_path` and `@rpath` do not; a local probe shows the same on Linux. The bundle therefore cannot load its framework, exactly as the control built without the flag. An unsigned bundle fails `codesign --verify --deep --strict` (`code has no resources but signature indicates they must be present`); signed ad hoc, dylib first and bundle second, it verifies (`valid on disk`, `satisfies its Designated Requirement`). Run 3 tried to add the literal rpath to a copy after link, and the edit failed; run 4 recorded why and measured both remaining routes. `install_name_tool -add_rpath` refuses an mcpp-built program: `larger updated load commands do not fit (the program must be relinked, and you may need to use -headerpad or -headerpad_max_install_names)`. A program linked with `-rpath @executable_path/../Frameworks` and bundled with its dylib in `Contents/Frameworks` verifies after ad-hoc signing, runs (exit 7), and stops with `Library not loaded` without the framework (exit 134). An mcpp-built program linked with `-Wl,-headerpad_max_install_names` through `mcpp::link_flag` accepts the edit and then behaves the same after re-signing. + +**Decision.** After §5.3, `dist-apple` copies the staged dylibs into +`Contents/Frameworks/` (macOS) or `Frameworks/` (iOS); the member emits +`mcpp::link_flag("-Wl,-rpath,@executable_path/../Frameworks")` on `*-macos` +(`@executable_path/Frameworks` on iOS), so no file is modified after link; +this needs the engine to treat `@executable_path`, `@loader_path` and `@rpath` in an rpath as loader tokens, as it already treats `$ORIGIN` (§9 item 11, task T3). The link-time route is chosen over an edit after link: the edit needs header padding at link time and a re-signature of the edited program, and the link-time route needs neither (both measured, run 4). Until an identity is given the member signs ad hoc, dylibs first and the bundle second. + +### 6.2 B2: `dist-apple` produces a DMG + +**Measured (CI, `macos-15`).** `hdiutil create -volname Demo -srcfolder -format UDZO` exits 0; `hdiutil verify` reports the checksum valid; the image attaches read-only with `Demo.app/Contents/MacOS/app` and an `Applications` link to `/Applications`, and detaches. + +**Decision.** `dmg` is a second format of `dist-apple`: the `.app` action's +output and an `Applications` symlink in a staging directory, then one +`hdiutil create -format UDZO` action whose `.dmg` is the terminal artifact. No +engine change. + +### 6.3 B3: the runner for a distributable + +**What the engine does** (read). `mcpp run --format ` packs and then +prepares again without a pack format (`execute.cppm:1873-1885`); the default +runner slot is keyed by triple and shared with `mcpp test` (`:2423`); a named +slot is chosen by `--runner ` for a distributable as for a program +(`:1602-1627`). + +**Measured (CI, `macos-15`).** With `[target.aarch64-macos.runners] app = ["/bin/sh", "-c", "exec …", "{}"]`, `mcpp run --format app --runner app -- extra` prints `1-2-3 argc=2` and exits 7; `mcpp run --format app` alone reports `…/RApp.app could not be started: Permission denied (error 13)` and exits 126; a plain `mcpp run` runs the program bare and exits 7; with `[target.aarch64-macos] runner = ["open", "-W"]` a plain `mcpp run` hands `-W …/bin/rapp` to `open` and exits with `open`'s 0. + +**Decision.** *Engine:* `mcpp run --format ` without `--runner` uses the +named runner `` when the graph or the manifest supplies one, and the +default runner otherwise; a distributable that is a directory and meets no +runner is refused before the spawn, naming the runner `` that would reach +it, rather than executed and answered with `Permission denied`. *Official plugin:* `dist-apple` supplies +`mcpp::runner("app", "macapp-run")` on `*-macos` and declares the payload. +*Payload:* `macapp-run` (§7.1). + +**Criteria.** A bundle whose program prints `1-2-3` and exits 7, with no runner +in the manifest: `mcpp run --format app` prints `1-2-3` and exits 7. Without +the member, `mcpp run --format app` is refused before the spawn, naming the +runner `app`. Negative direction: a plain `mcpp run` executes the program with +no runner line. + +### 6.4 B4: `dist-wix` produces a Burn bundle + +`dist-wix` produces `msi` from one `wix build` action (`dist/wix.cppm:584-601`, +`672`). The native SDK archives ship in `xim:wix` (§7.3) and link under both +MSVC-ABI toolchains (measured, §7.3). `setup` chains the MSI; the stock +bootstrapper needs `WixToolset.BootstrapperApplications.wixext`, which the +payload does not carry; a custom bootstrapper is the project's program, +reached as `${mcpp.target_file:}`. No engine change. + +### 6.5 B5: `dist-apk`'s two defects, then an Android App Bundle + +**Measured (local, mcpp 2026.9.14.1, plugins 0.9.3, `tests/apk-consumer-shared`).** +A first `mcpp pack --target x86_64-linux-android --format apk` produces an APK +whose `lib/x86_64/` holds `libapk-consumer-dep.so`, the application object and +`libc++_shared.so`. A second, with no edit, exits 0 and its APK lacks +`libapk-consumer-dep.so`: the plan wipes the member's stage while the +`apk:needed` stamp lives outside it (`dist/apk.cppm:892-893`, `1164`), so the +walk does not run again. With `--target x86_64-linux-android --target +aarch64-linux-android` the engine stages `lib/x86_64/` and `lib/arm64-v8a/`, +and the pack exits 1 with the engine's `no action claimed --format 'apk'`: the +member reads `/lib/*.so` without descending (`:626-648`) and prints its +reason to `std::cerr`, which a build program's success does not surface. + +**Decision.** Read the staged closure of §5.3 and delete the member's walk +(which removes the stamp); read `lib//`; report every refusal through +`mcpp::warning`. Then `aab` (`aapt2 link --proto-format`, `bundletool +build-bundle` from a new `xim:bundletool`, `jarsigner`). No engine change. + +**Criteria.** Two packs in a row carry the dependency's library both times; +two triples give one APK listing both ABIs; a member refusal prints its reason. + +### 6.6 B6: `rules-metal` and `tools-esbuild` + +Neither exists in the plugins; Lib-Live2D writes both in-project. `rules-metal` +is general (the Apple platforms' shader language, the counterpart of +`rules-spirv`, whose one-action-per-shader shape it copies, +`rules/spirv.cppm:751-813`) and belongs to the official plugins now; +`tools-esbuild` serves one project's Web bridge and stays there until a second +consumer. A rule locates an SDK with `xcrun --sdk --show-sdk-path`. + +## 7. Payload, xlings and index items + +### 7.1 C1: a macOS app-bundle runner program + +**Measured (CI, `macos-15`).** A program run as `N.app/Contents/MacOS/probe` reports `bundle=…/N.app` and `resource=…/N.app/Contents/Resources/greeting.txt` and exits 7; the same binary copied out of the bundle reports its own directory and `resource=none`. + +**Decision.** `macapp-run [args…]` in `apple-simulator-tools` (or a new +`apple-app-tools`): require `Contents/Info.plist` (exit 2 naming what is +missing), read `CFBundleExecutable`, `exec` `Contents/MacOS/` with the +arguments, so stdio and the status are the program's by construction. + +### 7.2 C2: `simctl-run`'s output and status + +**What the recipe does** (read). A bare executable goes through `simctl spawn`; +a `.app` through `simctl install` and `simctl launch --console-pty +--terminate-running-process … ; exit $?`, whose status the recipe marks +unmeasured (`pkgs/a/apple-simulator-tools.lua:200-236`). The plugins CI run +34770579266 lost the output twice in three attempts; its fixture exits 0 and +its diagnostic uses `cat -A`, which BSD `cat` refuses. + +**Measured (CI, `macos-15`, 20 launches per cell).** Run 2, one install and then twenty launches per cell. For an application that exits at once, `--console-pty`, `--console` and `--stdout`/`--stderr` files each kept the marker 20 of 20 times; for one that sleeps a second first, 20, 19 and 20. Every one of the 120 launches returned 0 for an application that exits 7, and the application is absent from `launchctl list` once it exits, so neither simctl nor launchctl reports its status. Run 3: with an install before every launch, `--console-pty` kept the marker 19 of 20 times and returned 0 every time (the plugins CI's condition, reproduced); `simctl spawn` of the installed application's executable, whose path `simctl get_app_container` gives, kept the marker 20 of 20 times and returned the application's 7 every time; for an application that calls `abort()`, `simctl launch` returned 0 and `simctl spawn` returned 134 (`terminated with signal 6`). + +**Decision.** For a bundle, `simctl-run` installs it and reads the executable's load +commands. When the executable does not load `UIKit.framework`, the program +asks `simctl get_app_container` for the installed path and runs `simctl spawn + / `: measured to return the +application's own status, a signal as a shell status, and its output every +time. When it loads UIKit, the application needs the launch path, so +`simctl launch --console-pty` stays, and the program prints that the status it +returns is simctl's, since no simctl spelling and no launchctl listing reports +the application's (measured). The rule reads the executable, not a guess about +its behaviour. The recipe's UNMEASURED notes are replaced with these readings. +The plugins CI fixture exits non-zero, and its diagnostic uses `od -c`. A +UIKit application under `spawn` is not measured here and is not proposed. + +### 7.3 C3: WiX Burn native SDK payloads + +Already shipped: `xim:wix` 5.0.2 installs `WixToolset.BootstrapperApplicationApi` +and `WixToolset.DUtil` beside `wix` under HuxerUI's own sha256 pins +(`pkgs/w/wix.lua:1-20`, `83-106`, `186-227`). **Measured (CI, `windows-2022`).** +Run 1 linked `dutil.lib` and `balutil.lib` under the default toolchain +(`lld-link`, MSVC ABI) and under `msvc@system`, and every unresolved symbol was +a Windows import (`MessageBoxA`, `RegOpenKeyExW`, `CoInitializeEx`), none a +WiX symbol; run 2, with the system import libraries named: both toolchains build the program and it runs with exit 0 (the default resolves the `x86_64-windows-msvc` row; `msvc@system` resolves MSVC 19.44.35228). A MinGW +link of the same archives fails (the payload reader's measurement: MSVC's +security-cookie and mangled `StringCch*` symbols). **Decision:** anchor +`runtimes/win-x64/native/mbanative.dll`, add a Windows install-verify job, add +the `.wixext`, and state in the recipe that the archives need an MSVC-ABI +link and the Windows import libraries. + +### 7.4 C4: resolving an unmerged recipe from a consumer's CI + +**What the pieces do** (read). xlings resolves `xim:` against the index named +`xim`; `--add-xpkg` lands under `local` (`xlings:src/core/xim/catalog.cpp: +397-426`). mcpp seeds the registry's `.xlings.json` from `config.toml +[index.repos.]` only when that file does not exist (`src/config.cppm: +588-596`). + +**Measured (CI, `ubuntu-24.04`).** A fresh home whose `config.toml` sets +`[index.repos.xim] url` to a checkout of xim-pkgindex with a recipe added on a +branch: the build installs `xim-x-m634-probe/0.0.1`; the registry's +`data/xim-pkgindex` is a symlink to the checkout and `.xlings.json` names it; +`mcpp index update` exits 0 and both stay; a version committed to the branch after the update (`0.0.2`) then resolves and installs, so the index follows the live checkout. A fresh home without the +override refuses: `package 'xim:m634-probe@0.0.1' not found in the synced index +(xim@artifact:19e264e, …)`. An existing home whose `config.toml` gains the same +table after its first build refuses the same way, and its `.xlings.json` never +names the checkout. + +**Decision.** *Usage, today:* a consumer's CI creates a fresh home with the +override before its first command. *Engine:* `[index.repos.]` in +`config.toml` reaches an existing registry too (reconciled on load, adding and +updating the entries it names), so the key never silently does nothing; a +payload installed from an overridden index names that source in the +provisioning line. *Payload CI:* xim-pkgindex's pull-request workflow builds a +consumer fixture through the override. *xlings:* nothing. + +### 7.5 D1: six-row descriptors + +`pkgs/h/huxerui.huxerui.lua` carries 0.3.0 on three rows; `huxerui.live2d` is +absent; `[package] platforms` accepts the other rows +(`docs/04-mcpp-toml.md:1273-1280`). It follows HuxerUI's release. + +## 8. What the project can do today + +| item | today | +|---|---| +| A1 | keep the two complementary predicates until §5.1 lands | +| A2 | write the namespaced key in `huxerui-build-rules`, `huxerui-tools` and `huxerui-tests` | +| A5 | one package per suite over a shared support library, selected with `mcpp test -p` | +| A6 | on the Android rows, `[target..build] ldflags = ["-static-libstdc++"]` until §5.6 lands | +| A7 | declare the four direct payloads and point pkg-config at `/subos/default/usr/lib/pkgconfig`; the 37 pins are unnecessary | +| A8 | each library deploys its resource package under a common directory; the framework loads them all | +| A9 | the framework's build program states a fact from `mcpp::min_platform_version()` and a floor against it | +| A10 | `MCPP_TOOLCHAIN=llvm@22.1.8` on the clang job's steps | +| B3 | drop the `open -W` default runner; declare `[target.aarch64-macos.runners] app = [...]` and run `mcpp run --format app --runner app` (measured on `macos-15`: exit 7, and the output and arguments reach the terminal) | +| C4 | a fresh `MCPP_HOME` with `[index.repos.xim] url = ""` in `config.toml` before the first command | + +## 9. Defects found beyond the issue + +1. The Android `dir` and `tar` archives omit their closure and say `walked` + (measured, §5.3). +2. Test programs on the Android rows cannot load on a device: the static + libc++ is not found and the contract degrades (measured, §5.6). +3. A second `dist-apk` pack drops the dependency's library (measured, §6.5). +4. `dist-apk` refuses the multi-ABI stage and its reason is not shown + (measured, §6.5). +5. The link-form degradation for a package that declares `kind = "shared"` + does not name that statement (measured, §5.1). +6. The version-floor refusal says "this machine has" for any fact (measured, + §5.9). +7. `[index.repos.]` added to an existing home is ignored without a word + (measured, §7.4). +8. Two runner emitters in one build program become one argv + (`modules/buildmcpp/src/directives.cppm:870`; the one-supplier check runs + across dependencies only, `prepare.cppm:9285-9306`; read). +9. `libxml2.lua` declares no `.pc` into the view (measured, §5.7). +10. The plugins iOS CI step cannot record what it lost, nor measure a status + (§7.2). +11. An rpath beginning with `@executable_path`, `@loader_path` or `@rpath`, + given through `mcpp::link_flag` or `[build] ldflags`, is anchored to the + package directory, because the normalisation exempts only a leading `$` + (`flags.cppm:266-283`; its copy for dependencies, `prepare.cppm:6141-6158`). + Measured on `macos-15` and on Linux (§6.1). + +## 10. Order and tasks + +| id | task | home | depends on | +|---|---|---|---| +| T1 | A1 replacement, labels and option warning; per-row `kind`; the degradation names its statement; the sweep's sub-table names | engine | - | +| T2 | A2 adoption, the canonical-source refusal, the scanner message; mcpp's six manifests | engine | - | +| T3 | A3 reader-driven closure for Android and Mach-O, staging, `walked`, `needs` lines; loader tokens (`$ORIGIN`, `@executable_path`, `@loader_path`, `@rpath`) exempt from rpath anchoring, in one function shared by the two copies | engine | - | +| T4 | A4 default SONAME | engine | - | +| T5 | A6 static runtime by driver query; `MCPP_RUNTIME_FILES` for every runner | engine | - | +| T6 | A10 option declarations | engine | - | +| T7 | X the graph record and `why deps` | engine | T1, T2 (criteria read it) | +| T8 | A9 platform facts and message wording; C4 index repos reach an existing home | engine | - | +| T9 | A5 `[test] discover`; A7 accessor; B3 format-named runner | engine | - | +| T10 | docs (`docs/04`, `05`, `06`, `08`, `09`, `10`, `30`, `50`) and zh mirrors; release; sandbox | engine | T1-T9 | +| P1 | `dist-apk`: staged closure, `lib//`, `mcpp::warning` refusals | official plugin | T3 released | +| P2 | `dist-apple`: frameworks placement, link-time rpath, signing; `app` runner; `dmg` | official plugin | T3, T9 released; X1 | +| P3 | `aab`; `setup`; `rules-metal` | official plugin | P1; X3 | +| P4 | plugins CI: the iOS fixture exits non-zero; portable diagnostic | official plugin | - | +| X1 | `macapp-run` | payload | - | +| X2 | `simctl-run` per §7.2; `adb-run` transfers `MCPP_RUNTIME_FILES` | payload | T5 released | +| X3 | `xim:wix` close-out; `xim:bundletool`; `libxml2` pkg-config | payload | - | +| X4 | xim-pkgindex PR workflow through the index override | payload CI | T8 released (or a fresh home today) | +| H1 | HuxerUI: the A2 keys, `open -W`, `MCPP_TOOLCHAIN`, the four GTK payloads | project | - | +| I1 | D1 | index | upstream release | + +T1 to T9 are independent engine changes and form one PR, each with its own +criterion; T7's fixtures are written against T1 and T2. + +## 11. Readings + +Every measurement this record uses, with its source. Local probes are in +`2026-09-14-634-probes.sh`; CI scripts are on the branch of mcpp#635. + +| id | question | where | reading | decides | +|---|---|---|---|---| +| L1 | is a conditional `linkage` applied | local, host row | the second declaration: exit 0, no `libfw.so`, no diagnostic; the modifier-only table: exit 2 under the label `[dependencies]`, `tried: (demo.fw, linkage)`; two complementary predicates: `libfw.so` and `NEEDED libfw.so` | A1 | +| L2 | does any manifest declare one dependency key in both tables | local scan of 509 manifests | none | A1 replacement | +| L3 | what does `kind = "shared"` do to a default request | local | linked shared; an explicit `static` request warns without naming the package's statement | A1 per-row kind; §9 item 5 | +| L4 | is a key's namespace checked against the manifest | local | two edges: the scanner names one file twice; one edge: builds silently; 9 of 46 `path` edges disagree (6 in mcpp's examples, 3 in HuxerUI) | A2 | +| L5 | what does the Android `dir` pack stage | local | only `lib/libapp.so`, whose `DT_NEEDED` names `libfw.so` and `libc++_shared.so`; the manifest says `closure = walked` | A3 | +| L6 | does an ELF shared library carry a SONAME | local | none on the host `libfw.so`, the Android `libfw.so` or `libapp.so` | A4 | +| L7 | what do Android test programs need | local | `libc++_shared.so` by default; nothing but `libm.so`, `libdl.so`, `libc.so` with `-static-libstdc++`; the NDK and LLVM drivers name their static archives for the effective target | A6 | +| L8 | do a dependency's deployed files reach the application | local | yes, from two levels down, into `bin/` and the `dir` pack | A8 | +| L9 | can a dependency refuse a low platform floor today | local, Android row | a build-program fact and floor refuse 21 and the unset default, accept 24, and are silent on the host | A9 | +| L10 | does `MCPP_TOOLCHAIN` reach `test` and `pack` | local | yes (clang compiles the test; pack resolves `llvm@22.1.8`); `--toolchain` is refused by `run`, `test`, `pack` | A10 | +| L11 | what shows the resolved dependencies | local | `mcpp why deps` prints only `mcpp.lock` lines ("no mcpp.lock" for path dependencies); `resolution.json` has `runtime`, `schema_version`, `toolchain` | X | +| L12 | does a host module's `link_flag` reach the root's link | local | yes for an absolute rpath; an `@executable_path` rpath is anchored to the package directory | B1; §9 item 11 | +| L13 | does `dist-apk` 0.9.3 pack twice, and for two ABIs | local, `tests/apk-consumer-shared` | the second APK lacks `libapk-consumer-dep.so`; two ABIs exit 1 with `no action claimed --format 'apk'` | B5 | +| L14 | does this machine's registry view resolve `gtk4` | local | no (`zlib`, `x11` and five more absent); a history reading, superseded by C1 | A7 | +| C1 | does a fresh home's view resolve `gtk4` | CI `ubuntu-24.04`, runs 1 and 2 | 143 entries, `pkg-config --cflags --libs gtk4` exits 0; only `libxml-2.0.pc` of every shipped `.pc` is absent | A7 | +| C2 | does a `config.toml` index override serve a branch checkout | CI `ubuntu-24.04`, runs 1 and 2 | fresh home: `0.0.1` installs, the index is a symlink to the checkout, `index update` keeps it, `0.0.2` committed afterwards installs; no override: refused naming the address; existing home with the override added later: refused, `.xlings.json` never names the checkout | C4 | +| C3 | do the WiX `v14` archives link under mcpp's Windows toolchains | CI `windows-2022`, runs 1 and 2 | run 1: only Windows imports unresolved on both; run 2 with the import libraries: both build and the program exits 0 | C3, B4 | +| C4 | do test programs run on the emulator | CI API 34 emulator, runs 1 and 2 | run 1: `CANNOT LINK EXECUTABLE … libc++_shared.so not found` for every test; run 2 with `-static-libstdc++`: `runs` passes, `reads_relative` cannot open `/data/local/tmp/data/data.txt`, the host path is absent | A6 | +| C5 | does a Mach-O program run staged beside its dylib | CI `macos-15`, run 2 | exit 7; exit 134 with `Library not loaded` when the dylib is removed; `pack --format dir` refuses today | A3 | +| C6 | does a Frameworks bundle load and verify | CI `macos-15`, runs 2 to 4 | the `link_flag` rpath is anchored and the bundle cannot load; unsigned verification fails, ad-hoc signing verifies; with the literal rpath from the link (run 4): verifies, exit 7, exit 134 without the framework; an rpath edit of an unpadded mcpp build fails (`larger updated load commands do not fit`), of one linked with `-headerpad_max_install_names` succeeds | B1 | +| C7 | can the base system make and mount a DMG | CI `macos-15`, run 2 | create, verify, attach with the `Applications` link, detach: all exit 0 | B2 | +| C8 | does the main bundle resolve when its executable is run directly | CI `macos-15`, run 2 | yes, with its resource; a copy outside the bundle does not | C1 | +| C9 | which runner reaches a distributable | CI `macos-15`, run 2 | `--format app --runner app`: exit 7, `1-2-3 argc=2`; `--format app` alone: exit 126; plain `run`: exit 7; an `open -W` default runner receives a plain run's program and exits 0 | B3 | +| C10 | which `simctl launch` spelling keeps output and status | CI `macos-15`, run 2 | run 2: the marker in 119 of 120 launches (one loss, `--console`); every launch exits 0 for an application that exits 7; run 3: an install before each launch: `--console-pty` 19 of 20, exit 0; `simctl spawn` of the installed executable: 20 of 20, exit 7; an aborting application: `launch` 0, `spawn` 134 | C2 | +| C11 | do test programs run on the iOS simulator | CI `macos-15`, run 2 | all three pass, both file reads included | A6 | + +Run 1's macOS readings were lost: its artifact upload hung after the measurements finished, and cancelling the job discarded its log; run 2 split the job and dropped the upload. + +## 12. Self-review + +- **What is measured.** Every decision's premise in §5 to §7 has a reading in + §11, except where a section says *read*: A5's target-model citation, B4's + and B6's plugin steps, D1, and §9 item 8. The CI readings measure the + released 2026.9.14.1; the engine changes themselves are not implemented. +- **What was not taken.** A modifier-only dependency table (ambiguous with a + namespace table); a selecting `requires_linkage` (a second meaning for + `requires_*`); named test targets with exclusive sources and labels (the + target model's rule); a closure `xpkg_dir` and payload versions in the lock; + a graph API for build programs now; raising a floor from the graph; an xlings + overlay; a Mach-O load-command editor, and an rpath edit after link for B1 + (it needs header padding and a re-signature, measured in run 4); `simctl + spawn` for a UIKit application; an Apple SDK accessor. +- **Earlier decisions revisited.** #622's "the Android closure is the + provider's" (§5.3); Phase 1b's `insert()` (§5.1); #630 §4's `LC_RPATH` + question (§5.3); revision 1's six decisions listed in §2. +- **Negative directions.** Every criterion has one. diff --git a/.agents/docs/2026-09-14-634-implementation-plan.md b/.agents/docs/2026-09-14-634-implementation-plan.md new file mode 100644 index 000000000..d810ce1d0 --- /dev/null +++ b/.agents/docs/2026-09-14-634-implementation-plan.md @@ -0,0 +1,578 @@ +--- +subject: triage +status: active +--- + +# #634 implemented across five repositories: the plan, its review, and the ledger that tracks it + +**Status:** active. This record turns the decisions of +`2026-09-14-634-cmake-parity-items-by-home.md` (revision 2, "the triage +record" below) into work: one pull request per repository, the order in which +they merge and release, the criteria each task is held to, and a ledger whose +rows are closed only by a reading. §1 reviews revision 2 from the angles an +implementation adds and states the refinements adopted before any code was +written (revision 3). A refinement found while implementing is appended to §1 +and folded into the triage record's §2 before the engine pull request merges. + +Base commits: mcpp `b8d96844` (2026.9.14.1), mcpp-plugins `9301832` (0.9.3), +xim-pkgindex `19e264e6`, mcpplibs/mcpp-index `f68512d`, xlings `59068d6`. +HuxerUI and Lib-Live2D are not modified by this work; their changes are listed +for the project in the reply on #634. + +## 0. The ledger + +Status is one of `todo`, `doing`, `branch` (implemented on the pull request's +branch, with the reading that measured it), `done` (merged, with the reading +that closed it), `dropped` (with the reason). Owner `lead` is the integrating session; +`W1` to `W4` are the parallel work trees of §8. + +### 0.1 Engine: mcpp-community/mcpp, one pull request (`feat/634-cmake-parity`) + +| id | task (triage §) | owner | depends on | status | +|---|---|---|---|---| +| E1 | a matching conditional dependency declaration replaces the unconditional one, compared by identity; the same for `dev-dependencies`, `build-dependencies`, `feature-deps` (§5.1.1) | lead | - | branch: e2e 677 A, B (fails on 2026.9.14.1) | +| E2 | conditional dependency tables labelled by their full section; an option-named selector with a string value warns with the restated form (§5.1.2) | lead | - | branch: e2e 677 C | +| E3 | `[target..targets.] kind` for declared library targets; the `[target.]` sweep reports sub-tables the parser does not read (§5.1.3) | lead | - | branch: e2e 677 D, 678 D; unit `TargetScalarKeys.EveryParsedSubTableIsKnownToTheSweep` | +| E4 | the link-form degradation names the package's statement and, for a row, its selector (§5.1.3, §9 item 5) | lead | E3 | branch: e2e 678 C | +| E5 | a `path`/`git` dependency adopts its manifest's identity and warns once; the record is registered under both keys (§5.2) | lead | - | branch: e2e 679 A, B | +| E6 | two identities over one canonical source are refused before scanning; the scanner's duplicate-provider message names the packages (§5.2) | lead | E5 | branch: e2e 679 C | +| E7 | the six edges in mcpp's `examples/` write the declared identity (§5.2) | lead | E5 | branch: the three examples build with no identity warning, and still build on 2026.9.14.1 | +| E8 | one reader-driven closure for PE, the Android rows and Mach-O, with per-format platform rules (§5.3.1) | W1 | - | branch: unit `test_pack_closure` (11 cases); e2e 667, 668 (macOS CI) | +| E9 | Android stages `lib/` (per ABI), Mach-O stages beside the program; `walked` only when complete, else `not-walked` naming the names and `dir`/`tar` refuse; `needs` lines on every row (§5.3.2-5) | W1 | E8 | branch: e2e 266, 666, 668 (macOS CI), 667, 669 | +| E10 | rpath entries that begin with a loader token (`$ORIGIN`, `@executable_path`, `@loader_path`, `@rpath`) are not anchored, through one function shared by both normalisers (§9 item 11) | W1 | - | branch: unit `test_build_flags`; e2e 670 | +| E11 | an ELF shared library without a declared `soname` is linked with its file name as SONAME (§5.4) | W1 | - | branch: e2e 667, 669 | +| E12 | the static C++ runtime archives are located by asking the driver for the effective target (§5.6.1) | W2 | - | branch: e2e 675 | +| E13 | every runner of `mcpp run` and `mcpp test` receives `MCPP_RUNTIME_FILES` (§5.6.2, format refined in §1) | W2 | - | branch: e2e 672; the `android` job's emulator step | +| E14 | `run`, `test` and `pack` declare `--toolchain` (§5.10) | W2 | - | branch: e2e 671 | +| E15 | `[test] discover` (§5.5) | W2 | - | branch: e2e 673; unit `test_test_targets` | +| E16 | `mcpp run --format ` uses the named runner `` when one exists; a directory distributable that meets no runner is refused before the spawn (§6.3) | W2 | - | branch: e2e 674 | +| E17 | two emissions of one named runner: measured, then decided (§9 item 8; see §1) | W2 | - | branch: decided in §1.9, e2e 684 | +| E18 | `resolution.json` records `graph`; `mcpp why deps` prints it (§5.11) | lead | E1-E6 | branch: e2e 682; 677, 678 and 679 read `graph` | +| E19 | the engine states `android.api-level`, `ios.deployment-target`, `macos.deployment-target`; the refusal names the fact's key and says "this build targets" (§5.9, §9 item 6) | lead | - | branch: e2e 680 | +| E20 | `config.toml [index.repos.]` reaches an existing registry; a payload installed from an overridden index names the source (§7.4, §9 item 7) | lead | - | branch: e2e 681; an existing home re-pointed its index and restored the entry (local probe, §1.9) | +| E21 | `mcpp::pkg_config_libdir()` (§5.7) | lead | - | branch: e2e 683 | +| E22 | CI: the `android-ndk` e2e tests run on a Linux job, and one emulator step runs `mcpp test` on the x86_64 Android row (§1, test coverage) | lead | E9, E12, E13 | branch: `ci-linux-e2e.yml` job `android` | +| E23 | user documentation and its Chinese mirror; normative specification changes (§5 below) | lead, W1, W2 | E1-E21 | doing | +| E24 | triage record revision 3 folded in; this ledger closed for the engine rows; CHANGELOG; version group 1 | lead | E1-E23 | doing | + +### 0.2 Payloads: openxlings/xim-pkgindex, one pull request (`feat/634-runners-and-payloads`) + +| id | task (triage §) | owner | depends on | status | +|---|---|---|---|---| +| X1 | `macapp-run`: a macOS application bundle runner (§7.1) | W3 | - | done: xim-pkgindex#838 (8f67d875), job `macapp-run` | +| X2 | `simctl-run` spawns the installed executable of a bundle that does not load UIKit; UNMEASURED notes replaced by readings (§7.2) | W3 | - | done: xim-pkgindex#838, job `simctl-run` | +| X3 | `adb-run` transfers the files `MCPP_RUNTIME_FILES` names and runs the program from their directory (§5.6.2) | W3 | E13 merged in the branch | done: xim-pkgindex#838, job `adb-run` on the API 34 emulator | +| X4 | `xim:wix` close-out: anchor `mbanative.dll`, a Windows install-verify job, `WixToolset.BootstrapperApplications.wixext`, the ABI statement (§7.3) | W3 | - | done: xim-pkgindex#838, job `wix` | +| X5 | `xim:bundletool` (§6.5) | W3 | - | done: xim-pkgindex#838, workflow `bundletool` on three hosts | +| X6 | `libxml2` declares its `.pc` files into the view (§5.7, §9 item 9) | W3 | - | done: xim-pkgindex#838, job `consumers` | +| X7 | the pull-request workflow builds a consumer fixture through the `config.toml` index override (§7.4) | W3 | - | done: xim-pkgindex#838, workflow `consumer-through-index-override` | + +### 0.3 Official plugins: mcpp-community/mcpp-plugins, one pull request (`feat/634-closure-bundles`, 0.10.0) + +| id | task (triage §) | owner | depends on | status | +|---|---|---|---|---| +| P1 | `dist-apk` reads the staged closure (`lib/`, `lib//`), deletes its walk and stamp, packs several ABIs into one APK, reports refusals through `mcpp::warning`, and refuses a stage without `needs` lines naming the engine floor (§6.5) | W4 | E9 | doing | +| P2 | `dist-apple` places staged dylibs in the framework directory, adds the link-time rpath through `mcpp::link_flag`, signs ad hoc when no identity is given, and keeps closure members out of the resource directory (§6.1) | W4 | E9, E10 | doing | +| P3 | `dist-apple` supplies `mcpp::runner("app", "macapp-run")` on `*-macos` and declares the payload (§6.3) | W4 | E16, X1 | doing | +| P4 | `dist-apple` `dmg` format (§6.2) | W4 | - | doing | +| P5 | `dist-apk` `aab` format (§6.5) | W4 | P1, X5 | doing | +| P6 | `dist-wix` `setup` format (§6.4) | W4 | X4 | doing | +| P7 | `rules-metal` (§6.6) | W4 | - | doing | +| P8 | CI: the iOS fixture exits non-zero, the diagnostic is portable, P1-P7 have jobs, `MCPP_VERSION` names the released engine (§7.2, §9 item 10) | W4 | R1 | doing | +| P9 | documentation, version 0.10.0, release, GitCode assets (§4 below) | lead | P1-P8 | todo | + +### 0.4 Index: mcpplibs/mcpp-index, one pull request + +| id | task | owner | depends on | status | +|---|---|---|---|---| +| I1 | `mcpp.plugins` 0.10.0 descriptor entry and its note | lead | R2 | todo | +| I2 | D1, HuxerUI's six-row descriptors | project | HuxerUI's release | dropped: follows the project's release, and HuxerUI is not modified here | + +### 0.5 Release, verification, reply + +| id | task | depends on | status | +|---|---|---|---| +| R1 | engine: merge, tag, `release.yml` on four hosts, `publish-ecosystem`, the bot's index pull request merged, GitCode assets checked (local `gtc` for any missing), bootstrap pin (version group 2) | E24, CI green | todo | +| R2 | plugins: merge, tag `v0.10.0`, release assets on GitHub and GitCode (`gtc`), byte comparison of each downloaded asset | P9, CI green | todo | +| R3 | xim-pkgindex pull request merged, and its artifact read back from a client | X1-X7, CI green | done: a local client installed `xim:android-platform-tools@37.0.1-4` from the index after the merge | +| R4 | mcpp-index pull request merged; `latest` of `mcpp.plugins` read back from a client | I1 | todo | +| V1 | sandbox verification of the released engine (§6.2) | R1, R3 | todo | +| V2 | sandbox verification of plugins 0.10.0 through the index (§6.2) | R2, R4 | todo | +| V3 | the ecosystem review (§7) | V1, V2 | todo | +| V4 | the reply on #634 | V3 | todo | + +## 1. Review of revision 2 from the implementation's side, and revision 3 + +Revision 2 held each decision to six properties. An implementation adds +angles the triage could not: what a later reader of each new record needs, +what an engine one release older does with a manifest written for this one, +which rows CI can reach, and where a test would stay green over a regression. +Each angle below names what it checked and what it changed. + +### 1.1 Architecture + +- **E1 compares identities, not map keys.** The dependency maps are keyed by + `selector.stableMapKey` (`toml.cppm:1509`); an unconditional `fw` and a + conditional `mcpplibs.fw` are one identity under two keys. Replacement + therefore removes every unconditional entry whose normalised identity equals + the conditional entry's before inserting it. A map-key comparison would + leave both entries, and the resolver would see two declarations of one + package. +- **E8 keeps one function and one data flow.** The closure function takes the + artifact, a list of search directories and a platform predicate, and + returns resolved members, platform names and unresolved names. The three + rows differ only in the predicate and in the stage layout; the manifest + writer, the refusal and the `needs` lines are shared. The ELF host row keeps + `ldd_parse` (triage §5.3) and feeds the same writer. +- **E18 records at the point of decision.** The declaring table of an edge is + a field of `DependencySpec` set by the parser; the adoption of E5 and the + link-form reason of E4 are recorded where they are decided and read by the + writer. No reader reconstructs a decision from output text. + +### 1.2 Stability + +- **E8, Mach-O names outside the loader's reach.** A dependency dylib whose + install name is an absolute path outside `/usr/lib/` and `/System/Library/` + resolves to a file, but a staged copy beside the program is not what dyld + loads. Revision 2 would have staged it and written `walked`. Revision 3: a + Mach-O name is a closure member only when it is `@rpath/` and the + loading image carries an rpath of exactly `@loader_path` or + `@executable_path`, or when it is `@loader_path/` or + `@executable_path/`; any other non-platform name makes the closure + `not-walked`, naming it. The tree then says what a machine without the build + tree will do. +- **E12 normalises the driver's answer.** A driver answers + `/bin/../lib//libc++.a`; the path is made lexically normal + before it enters a link line, so the LLVM payload's host link line is + byte-identical (the triage's negative criterion). A driver that does not + know the file echoes the bare name; that answer is treated as a miss and the + directory search runs. +- **E20 is measured before it is implemented.** Rewriting the registry's + `.xlings.json` changes the file xlings reads, not necessarily the index + checkout xlings already cloned. The task starts with a local reading: after + the file names a checkout on an existing home, does `mcpp index update` + re-point `data/xim-pkgindex`? The implementation follows the reading; if the + checkout is not re-pointed, the engine names the stale checkout and the + command that replaces it, and never deletes it silently. + +### 1.3 Simplicity + +- **E13, one file format with a separator paths do not contain in practice.** + Revision 2 wrote ` ` separated by a space; a + Windows user directory commonly contains one. Revision 3: one line per file, + destination and source separated by a TAB, destination relative to the + artifact's directory with `/` separators. The file exists for every runner + invocation, empty when there is nothing to transfer, so a runner can tell an + engine that states "nothing" from one that predates the variable. +- **E17, measure before adding a mechanism.** `mcpp::runner(name, token)` + appends one token per call (`hostprogram.cppm:78`), and the tokens are the + argv by contract (`directives.cppm:866-878`); two modules in one build + program cannot be told apart in the directive stream. Adding attribution to + the stream is a larger change than the defect. The task first measures the + case that matters to B3: a manifest's `[target..runners] ` and a + graph-supplied runner of the same name. If the two argvs are concatenated, + the manifest's declaration wins and the graph's is reported, which is the + precedence every other manifest-versus-graph conflict follows; the + in-program case is stated in docs/30 as the contract it is. + +### 1.4 User experience + +- Every new refusal and warning names the file, the key as written, and the + form that works (E2, E4, E5, E6, E9, E16, E19, P1). A warning is written + once per build, not once per edge. +- The graph record (E18) is the single place a user reads why a package has + its identity, its declaring table and its link form; `mcpp why deps` prints + it in the same order `resolution.json` stores it. + +### 1.5 Compatibility and upgrade without notice + +| change | an existing manifest or package | an engine one release older, given a manifest written for this one | +|---|---|---| +| E1 | no manifest among 509 declares one key in both tables; builds unchanged | ignores the conditional declaration, as today | +| E3 | packages without the sub-table unchanged | skips the sub-table without a word (the triage records this); the package states its floor in its documentation | +| E3 sweep | a sub-table the parser reads is never reported; the list is taken from the parser, and a unit test fails when a new reader is added without the list | not applicable | +| E5 | nine edges gain one warning and keep building | unchanged | +| E9 | an Android `dir`/`tar` that shipped an incomplete tree now carries its closure, or refuses naming what is missing; a Mach-O `dir` that refused now succeeds | stage manifests gain lines; no official plugin reads the stage manifest today (`grep` over mcpp-plugins at `9301832`) | +| E11 | every ELF shared library gains `DT_SONAME` equal to its file name; consumers' `DT_NEEDED` is unchanged because they link with `-l` | not applicable | +| E12 | host LLVM link lines byte-identical; Android test programs become self-contained | not applicable | +| E13 | runners that ignore the variable are unchanged | the variable is absent; `adb-run` then transfers nothing, as today | +| E16 | a plain `mcpp run` is unchanged; `--format ` without a runner named `` is unchanged except that a directory is refused before the spawn instead of failing in it | not applicable | +| E18 | every existing `resolution.json` field unchanged; `schema_version` follows docs/50 §7's rule for additive fields | not applicable | +| E19 | the refusal's wording changes; no e2e test asserts the old sentence (`grep "this machine has" tests/e2e`: none in a floor test) | not applicable | +| E20 | a home whose `config.toml` carries an override that was silently ignored now takes it, and the reconciliation prints one line naming the index and both URLs | not applicable | +| P1 | a stage without `needs` lines comes from an engine below the floor; the member refuses naming the floor instead of packing an APK without the dependency's library | not applicable | + +### 1.6 Cross-platform + +| task | Linux x86_64 (local) | Linux CI | macOS CI | Windows CI | device | +|---|---|---|---|---|---| +| E1-E7, E18-E21 | e2e | e2e shards | e2e | e2e | - | +| E8-E9 Android | e2e (`android-ndk`) | E22 job | - | - | E22 emulator | +| E8-E9 Mach-O | - | - | e2e 666 and a new test | - | - | +| E8 PE | existing e2e | - | - | e2e | - | +| E10 | unit test for all four tokens; e2e with `$ORIGIN` and `@executable_path` spellings | e2e | e2e reads `LC_RPATH` | - | - | +| E11 | e2e (`readelf`) | e2e | - | - | - | +| E12-E13 | e2e (`android-ndk`: `DT_NEEDED`; a fake runner reads the file) | E22 | e2e (fake runner) | e2e (fake runner) | E22 emulator | +| E16 | e2e (`--format dir` refusal; fake runner) | e2e | e2e | e2e | - | +| P1, P5 | local | plugins CI | - | - | plugins CI emulator | +| P2-P4, P7 | - | - | plugins CI | - | iOS simulator | +| P6, X4 | - | - | - | plugins and xim-pkgindex CI | - | + +A cell marked `-` is a row the task does not change. + +### 1.7 Consistency + +- The new keys use existing vocabularies: `kind` is `[targets.] kind`, + `discover` takes `[build] sources` globs, the facts are `version-floor` + facts, the accessor joins the `toolchain_*()` family, `--toolchain` is + `build --toolchain`. +- Every new record line follows its file's existing grammar: `needs` is one + more line kind in `.stage-manifest`, `graph` is one more top-level member of + `resolution.json`. + +### 1.8 Test coverage, and where a test would stay green over a regression + +- **Every criterion is run once with the fix removed.** For each engine task + the test is run against the released 2026.9.14.1 and must fail there; the + reading is recorded in the pull request. A test that passes on both engines + measures nothing. +- **The Android e2e tests never ran in CI.** `run_all.sh` detects + `android-ndk` from a payload path, and no workflow installs the payload, so + e2e 652b and 664 are local-only. E22 adds the job that installs the NDK and + runs them, plus the new Android tests, and one emulator step for E12/E13. +- **The criteria read state.** A1, A2 and the per-row kind assert on the + `graph` record (E18); a warning's sentence is asserted only where the + sentence is the deliverable (E2, E4, E5, E19). +- **Negative directions** are the triage record's, per task, and are tests, + not prose. + +### 1.9 Refinements found while implementing + +Each item was found by a measurement on the branch, and each has a test. + +- **A2 is keyed by source, not by key.** A map from a key an edge wrote to the + identity its manifest declared would also have captured a `version` + dependency written with the same key, which is a different package. The + resolver records the identity resolved from each canonical source (a + directory, or a repository and reference); a second key over that source + takes it, and a manifest that declares no namespace makes the second key a + refusal (e2e 679). +- **The `[target.]` sweep reports sub-tables.** The sweep skipped every + table-valued key because a hand-written list of sections had drifted + twice. The list now exists beside the scalar lists and a unit test derives + the parsed sections from the parser's source in both directions, so the + drift is a failing test rather than a false warning. +- **A row's `kind` is checked after target inference.** A library target a + package does not declare exists only after `load` infers it, so the name + is validated there rather than where the row is parsed. +- **The stage manifest's `needs` lines are TAB-separated and carry a third + value.** `needs`: a Mach-O + install name or a Windows directory can contain a space, and a provider + reads a failure as state. The ELF host row applies §5.3.4 too: a name the + loader reports as not found makes the closure `not-walked`. +- **The Android search set includes `[runtime] link_library_dirs` and the + transitive needed directories.** Without them a valid prebuilt library was + refused (measured). +- **The default SONAME precedes `$ldflags`.** A project that names its SONAME + through `[build] ldflags` keeps it, because the linker takes the last + `-soname` (e2e 220). +- **The NDK is asked with the API level, and its `libc++.a` is a linker + script.** Without the level the NDK answers with the host's archive. With + the script's archive names passed to `--exclude-libs`, a self-contained + Android shared library exports 4 dynamic symbols instead of 161. +- **`MCPP_RUNTIME_FILES` lists the linked shared libraries too.** A test on a + row whose dependency is shared (A1's per-row kind) needs the library beside + it on the device; a nested test's destinations start with `../`. +- **A directory distributable that meets no runner exits 126.** The + exit-code specification forbids moving a published failure to another + category, and the kernel's refusal of the same case was 126 (docs/50 §6). +- **§9 item 8 (E17).** Measured on 2026.9.14.1: a manifest runner and a + build-program runner of one name do not concatenate (the manifest wins); + two emissions in one build program form one argv by the directive contract, + now stated in docs/30; a runner a dependency's build program supplies and + the root's build program also emits became one argv (`run-A.sh run-B.sh + `). The last is refused naming both, unless the manifest declares + the name, which is the precedence every other runner conflict follows + (e2e 684). +- **A test built from a subdirectory could not load a graph-built shared + library.** Its search path named its own directory (`bin/sub/`), and the + post-link closure check refused the build on 2026.9.14.1. A consumer outside + the library's directory also searches the relative path to it (e2e 685). +- **A refused `--toolchain` value was credited to `[toolchain].`.** + The message names where the value was written (e2e 671). +- **C4, measured before implementing.** On an existing home, once the + registry's `.xlings.json` names a checkout, `mcpp index update` re-points + `data/xim-pkgindex` at it. Removing the table restores the previous entry, + through a record of what mcpp wrote (`.mcpp-index-overrides.json`), unless + the entry was changed after mcpp wrote it (e2e 681). +- **C4 reconciles tables only (found by the independent review of the pull + request).** The reconciliation compared every index repository the + configuration resolved, including the `mcpplibs` entry mcpp adds when + `config.toml` has no table for it, so a registry copy of that entry which + differed from the default was rewritten with a line naming a table that + did not exist, and a later `[index.repos.mcpplibs]` table could not be + undone by removing it. The review's premise was partly wrong: a home mcpp + creates writes the `mcpplibs` table into its own `config.toml`, where the + line is accurate; the defect needs that table removed. Only a table is now + reconciled, and a name counts as configured only while a table names it + (e2e 681 E and F, which fail without the change). + +## 2. Repositories, branches, versions + +| repository | branch | version | merges after | +|---|---|---|---| +| mcpp-community/mcpp | `feat/634-cmake-parity` | 2026.9.14.2, or the first free number of the release date | its CI | +| openxlings/xim-pkgindex | `feat/634-runners-and-payloads` | `macapp-run` 0.1.0; `apple-simulator-tools` 0.3.0; `android-platform-tools` next revision; `wix` 5.0.2 with the extension; `bundletool` upstream's current release; `libxml2` a revision that re-runs the install hook | its CI; independent of the engine release | +| mcpp-community/mcpp-plugins | `feat/634-closure-bundles` | 0.10.0 | R1 (its CI pins the released engine) | +| mcpplibs/mcpp-index | `feat/634-plugins-0.10.0` | - | R2 | +| mcpp-community/mcpp (pin) | `ci/bootstrap-pin-` | version group 2 | R1 and the bot's index pull request | + +Why the plugins pull request merges after the engine release: P1 and P2 read +the stage E9 writes, P3 needs E16, and its CI must measure the released engine +rather than a branch. During development its CI is pointed at the engine +branch through a temporary `MCPP_SOURCE_REF` channel on every step that +installs mcpp, removed before merge. + +## 3. Engine tasks in detail + +Paths are relative to the mcpp repository. "Test" names the e2e test added or +changed; numbers are assigned per work tree (W1: 667-670, W2: 671-676, lead: +677-690) so that parallel branches do not collide. + +### E1-E4 (A1) + +- *Code.* `merge_conditional_config` (`src/build/prepare.cppm:367-472`): for + each conditional map, erase unconditional entries of the same identity, + then insert. `ConditionalConfig` (`modules/manifest/src/types.cppm:1276`) + gains the per-row target kinds; `DependencySpec` gains the declaring table. + The parser (`modules/manifest/src/toml.cppm`): the conditional dependency + label (`:3015-3024`), the option-named selector warning in + `load_selector_dep_table` (`:1599-1629`), the `targets` sub-table, and the + sweep's list (`:2748-2760`). The degradation sentence in the linkage pass + (`prepare.cppm:11280-11375`). +- *Tests.* Host row, both directions: the conditional `linkage = "shared"` + gives `NEEDED libfw.so` on the matching row and a static link on a + non-matching one; the modifier-only table exits non-zero naming the + conditional section and the restated form; a per-row `kind` gives + `bin/libfw.so` on Linux and a static link under a selector that does not + match; `linkage = "static"` on the root's edge warns naming the selector and + `--strict` exits non-zero. e2e 86, 195, 328 unchanged. + +### E5-E7 (A2) + +- *Code.* The resolve loop (`prepare.cppm:6499+`, dedup `:6531-6534`, name + check `:7242-7260`, namespace fill `:7267-7270`); the scanner's + duplicate-provider message. +- *Tests.* Two edges keyed `fw` and `huxdemo.fw` over one directory build, + compile the unit once, warn once naming the requester, `fw`, `mcpplibs.fw` + and `huxdemo.fw`; `graph` shows one package with both keys. Two keys `a.fw` + and `b.fw` over a manifest without a namespace are refused before scanning. + Matching keys warn nothing. `examples/04`, `08`, `12` build without the + warning. + +### E8-E11 (A3, A4, §9 item 11): work tree W1 + +- *Code.* `src/pack/pack.cppm` (`pe_closure` `:999-1053` generalised; + `run_shared_program` `:1281-1337`; the Mach-O branch `:1423-1438`; search + directories `:497`; `ClosureResult` `:247`), `src/pack/stage_tree.cppm` + (`:126`), `src/pack/pipeline.cppm` (`:438-478`), the binary readers; + `normalize_ldflag` (`src/build/flags.cppm:266-283`) and its copy + (`prepare.cppm:6141-6158`) call one shared predicate; + `shared_soname_flag` (`src/build/ninja_backend.cppm:274-290`), with the + declared-soname alias logic unchanged. +- *Tests.* Android (`# requires: elf gcc android-ndk`): the `dir` tree holds + `lib/libapp.so`, `lib/libfw.so`, `lib/libc++_shared.so`, the manifest says + `walked` and lists `libc.so`, `libm.so`, `libdl.so` as platform; a needed + library deleted after the build makes `dir` refuse naming it; two triples + stage `lib//` each with its closure. macOS: e2e 666's `dir` direction + changes to the staged closure; the staged program runs after the build tree + is moved and fails with "Library not loaded" when the staged dylib is + removed. ELF host: the tree is byte-identical to 2026.9.14.1's and the + manifest differs by `needs` lines. SONAME: `readelf -d` shows `SONAME + libfw.so` on the host and Android rows and on the application object; a + declared `soname` keeps its value and alias. rpath: a unit test over + `$ORIGIN`, `${ORIGIN}`, `@executable_path/..`, `@loader_path`, `@rpath/x`, a + relative path and an absolute path; an e2e shows the literal token in the + program's rpath. + +### E12-E17 (A6, A10, A5, B3, §9 item 8): work tree W2 + +- *Code.* `find_archive` (`src/build/flags.cppm:1064-1079`, `1182-1183`) asks + the driver through the toolchain module (`src/toolchain/clang.cppm:130-150` + is the existing query) with the effective target flag, cached with the + toolchain probe; the degradation (`src/build/distribution.cppm:718-726`). + Runner spawns in `src/build/execute.cppm` (`:643-711`, `:1602`, `:1691`, + `:1873-1904`, `:2423`) write the runtime-files list from the plan's + `runtimeDeployFiles` and set the variable. `src/cli.cppm` declares + `--toolchain` on `run` (`:391`), `test` (`:480`), `pack` (`:574`). + `src/build/test_targets.cppm:38` takes `[test] discover`, parsed by a new + reader in the manifest module. The named-runner lookup for `--format`. +- *Tests.* Android row without `ldflags`: test programs name no + `libc++_shared.so`; the host LLVM link line is unchanged. A fake runner + script prints the variable's file; `mcpp test` and `mcpp run` hand it a file + whose lines are `data/data.txt`; an artifact with no deployed + files gets an empty file. `mcpp test --toolchain llvm@22.1.8` compiles with + clang when the default is gcc. `discover = ["checks/**/*.cpp"]` runs + `checks/a.cpp` and not a failing `tests/b.cpp`; `discover = []` runs none; + without the key the names equal today's. `run --format ` with a manifest + runner named `` runs it; a directory distributable without one is refused + before the spawn naming the runner name. + +### E18-E21 + +- *E18 code.* The `resolution.json` writer (`prepare.cppm:12700-12870`), + `mcpp why deps` (`src/doctor.cppm:1089-1110`). *Test.* The A1, A2 and + per-row kind fixtures read `graph`; the existing fields are unchanged. +- *E19 code.* `min_platform_version` (`prepare.cppm:1619-1670`), + `checkVersionFloors` (`:7804-7849`). *Test.* Android row: a dependency + requiring `android.api-level >= 23` is refused for a root stating 21 and for + a root stating nothing, naming `min_api_level`; 24 builds; the host row is + silent. +- *E20 code.* `src/config.cppm:535-549`, `:588-596`, the provisioning line. + *Test.* An existing home whose `config.toml` gains `[index.repos.xim]` + pointing at a local index checkout with an extra recipe installs it on the + next build and prints the reconciliation line once. +- *E21 code.* `src/build/hostprogram.cppm` (the accessor and its contract + entry), the environment the build program receives. *Test.* A build program + prints the accessor; the value names `/subos/default/usr/lib/pkgconfig`. + +### E22 (CI) + +A job in `ci-linux-e2e.yml` installs `xim:android-ndk` into the job's home +with a cache keyed by the payload version, runs the e2e tests that require +`android-ndk`, then starts an API 34 x86_64 emulator +(`reactivecircus/android-emulator-runner@v2`, KVM enabled, the configuration +measured on mcpp#635) and runs `mcpp test` on a fixture whose test programs +read a deployed file, through `adb-run`. The emulator step is marked with the +xim-pkgindex revision it needs; until X3 is published it asserts only the +program's loading (`runs`), and the file criterion is added when the index +carries X3. + +### E23 (documentation) + +| document (and `docs/zh/` mirror) | changes | owner | +|---|---|---| +| `docs/04-mcpp-toml.md` | conditional dependency replacement; `[target..targets.] kind`; `[test] discover` | lead, W2 | +| `docs/05-dependencies.md` | identity adoption for `path`/`git`; the graph record's link-form reasons | lead | +| `docs/08-testing.md` | `discover`; test programs on device rows; `--toolchain` | W2 | +| `docs/09-commands-by-scenario.md` | `--toolchain` on `run`/`test`/`pack`; `why deps` | W2, lead | +| `docs/10-pack-and-release.md`, `docs/12-binary-distribution.md` | the Android and Mach-O closure; `needs` lines; `not-walked` | W1 | +| `docs/22-target-side.md` | default SONAME, rpath loader tokens | W1 | +| `docs/23-the-project-environment.md` | `config.toml` index overrides on an existing home | lead | +| `docs/30-build-mcpp.md` | `pkg_config_libdir()`; platform facts; runner emission contract | lead, W2 | +| `docs/41-devices.md` | `MCPP_RUNTIME_FILES`; format-named runners | W2 | +| `docs/50-machine-output.md` | `graph`; `needs` | lead, W1 | +| `docs/specs/manifest-semantics.md`, `package-identity.md` | replacement; adoption | lead | + +The prose follows `.agents/skills/mcpp-docs-style`: usage for implemented +behaviour, declarative, no design reasoning, bilingual parity. + +## 4. Payload, plugin and index tasks in detail + +- **X1** `pkgs/m/macapp-run.lua` (macOS): the program requires + `Contents/Info.plist` (exit 2 naming the missing piece), reads + `CFBundleExecutable` with `plutil`, and `exec`s `Contents/MacOS/` with + the arguments. CI: a bundle whose program prints its arguments and exits 7 + returns 7 with the output. +- **X2** `pkgs/a/apple-simulator-tools.lua` 0.3.0: after `simctl install`, + `otool -L` on the executable decides; without UIKit, `simctl spawn + / `; with UIKit, `simctl launch --console-pty` and one + line saying the status is simctl's. CI: exit 7 and abort 134 are returned + for a non-UIKit bundle, twenty runs each keep the marker. +- **X3** `pkgs/a/android-platform-tools.lua`: when `MCPP_RUNTIME_FILES` names + a file, every line's source is pushed to `/` and + the program runs with that directory as its working directory. CI on the + emulator: a program reading `data/data.txt` relative to its directory exits + 0. +- **X4** `pkgs/w/wix.lua`: `mbanative.dll` anchor, the `.wixext` package, + a Windows job that installs the payload and checks the anchors, and the ABI + statement in the recipe's notes. +- **X5** `pkgs/b/bundletool.lua`: the upstream jar and a launcher through the + JDK payload `dist-apk` already uses. +- **X6** `pkgs/l/libxml2.lua`: relocate and declare `libxml-2.0.pc` as + `gtk4.lua` does; a revision so an installed copy re-runs the hook. +- **X7** The pull-request workflow creates a fresh home whose `config.toml` + names the pull request's checkout and builds a consumer fixture that + declares a recipe the pull request adds or changes. +- **P1-P9** as §0.3; each has a job in the plugins CI, and P1's criteria are + two packs in a row carrying the dependency's library, two triples giving one + APK listing both ABIs, and a refusal whose reason is printed. +- **I1** `pkgs/m/mcpp.plugins.lua`: a 0.10.0 entry in the bot's shape (one + `latest` line replaced), and a note stating the engine floor 2026.9.14.2 and + what an older engine does (P1's refusal). + +## 5. Order + +``` +X1..X7 ─────────────────────────────► R3 ──┐ +E1..E7, E19..E21 (lead) ─┐ │ +E8..E11 (W1) ────────────┼─► E18 ─► E22 ─► E23 ─► E24 ─► R1 ─┬─► V1 ─┐ +E12..E17 (W2) ───────────┘ │ │ +P1..P7 (W4, against the engine branch) ──────────────► P8 ◄──┘ │ + P9 ─► R2 ─► I1 ─► R4 ─► V2 ─► V3 ─► V4 +``` + +The engine pull request merges only with every CI workflow green on its last +commit and the triage record's revision 3 folded in. The release follows +`.agents/skills/mcpp-release`: version group 1 in the pull request, tag, +`release.yml`, `publish-ecosystem`, the bot's index pull request merged with +`--admin`, `latest` read back, GitCode assets compared, version group 2 in a +separate pull request. + +## 6. Verification + +### 6.1 In CI + +Every pull request's CI is green on its last commit, read from the run of that +commit's SHA. For the engine, each new e2e test also has a recorded failing +run against 2026.9.14.1 (§1.8). + +### 6.2 In a sandbox, after release + +A SubOS created with `xlings subos new m634`, entered with `xlings subos use +m634 --sandbox --cmd "..."`, with `xlings config --mirror CN` and `mcpp self +config --mirror CN` set inside it. Each check reads the installed version +first. + +| id | check | criterion | +|---|---|---| +| S1 | `mcpp --version` | the released version | +| S2 | A1 fixture on the host row | `NEEDED libfw.so`; `graph` names the conditional table | +| S3 | A2 fixture | one warning; `graph` one package, two keys | +| S4 | A4 | `SONAME libfw.so` | +| S5 | A5 | `discover` runs `checks/a.cpp` only | +| S6 | A10 | `mcpp test --toolchain llvm@22.1.8` compiles with clang | +| S7 | X | `mcpp why deps` prints the graph for a path dependency | +| S8 | A3 and A6 on `x86_64-linux-android` | `dir` tree with `libfw.so` and `libc++_shared.so`, `walked`; test programs name no `libc++_shared.so` | +| S9 | A9 | the floor refusal names `min_api_level` | +| S10 | A7 | `pkg-config --cflags --libs gtk4` with the accessor's value exits 0 | +| S11 | C4 | an existing home takes a `config.toml` override on the next build | +| S12 | B3 | `mcpp run --format dir` is refused before the spawn naming the runner | +| S13 | P1 through the index | two packs carry the dependency's library; two triples give one APK with both ABIs | +| S14 | I1 | `mcpp.plugins` resolves to 0.10.0 | +| S15 | X6 | the view holds `libxml-2.0.pc` after installing `xim:libxml2` | + +## 7. The ecosystem review (V3) + +After V1 and V2: every ledger row is `done` or `dropped` with a reading or a +reason; the four repositories' `main` heads build green; the index's `latest` +entries point at what was released; the documentation describes what the +sandbox measured; the triage record and this record are marked landed; the +reply on #634 lists, per item, what landed, where, and what the project can +now remove. + +## 8. Parallel work trees + +| tree | path | branch | owns | +|---|---|---|---| +| lead | `../mcpp-634` | `feat/634-cmake-parity` | everything not listed below; integration | +| W1 | `../mcpp-634-pack` | `feat/634-pack` | `src/pack/**`; `normalize_ldflag` and its copy; `shared_soname_flag`; e2e 667-670; the unit test for the rpath predicate; docs 10, 12, 22 | +| W2 | `../mcpp-634-run` | `feat/634-run` | `find_archive` and the driver query; `src/build/execute.cppm`; `src/build/test_targets.cppm`; `src/cli.cppm` option declarations; the `[test]` reader, inserted as a new function; `modules/buildmcpp/src/directives.cppm`; e2e 671-676; docs 08, 41, the `--toolchain` lines of 09 | +| W3 | xim-pkgindex work tree | `feat/634-runners-and-payloads` | X1-X7 | +| W4 | mcpp-plugins work tree | `feat/634-closure-bundles` | P1-P8 | + +Each engine work tree builds its own mcpp and runs only its own tests and the +e2e tests that cover the files it changes; a built binary is copied to a fixed +path before a suite runs against it. The lead merges W1 and W2 into the +feature branch, resolves conflicts, and runs the full suite once on the +merged tree. No work tree kills processes by pattern. diff --git a/.agents/docs/2026-09-14-634-probes.sh b/.agents/docs/2026-09-14-634-probes.sh new file mode 100644 index 000000000..cc4cfdd44 --- /dev/null +++ b/.agents/docs/2026-09-14-634-probes.sh @@ -0,0 +1,363 @@ +#!/usr/bin/env bash +# Probes behind the measured statements of 2026-09-14-634-cmake-parity-items-by-home.md. +# +# Each section builds a small fixture in a fresh directory and prints one line +# per claim: `holds: ` when the reading matches what the record states +# about mcpp 2026.9.14.1, `differs: ()` otherwise. After a fix +# lands, the corresponding claim is expected to read `differs`. +# +# Usage: MCPP=/path/to/mcpp bash 2026-09-14-634-probes.sh [workdir] +# Needs: gcc@16.1.0 (the default toolchain), llvm@22.1.8 and +# xim:android-ndk installed in the mcpp home the binary uses; readelf. +# +# Deliberately NOT `set -e`: several probes run a command that is expected to +# fail and read its output afterwards. +set -u + +MCPP=${MCPP:-mcpp} +WORK=${1:-$(mktemp -d)} +mkdir -p "$WORK" +WORK=$(cd "$WORK" && pwd) +echo "mcpp: $("$MCPP" --version)" +echo "work: $WORK" + +verdict() { # verdict <0|1> + if [ "$1" -eq 0 ]; then echo "holds: $2"; else echo "differs: $2 ($3)"; fi +} + +# A library `demo.fw` exporting module `fw`, reused by A1, A3 and A4. +make_fw() { + mkdir -p "$1/src" + cat > "$1/mcpp.toml" <<'T' +[package] +namespace = "demo" +name = "fw" +version = "0.1.0" +[targets.fw] +kind = "lib" +T + printf 'export module fw;\nexport int fw_anchor() { return 41; }\n' > "$1/src/fw.cppm" +} + +# ── A1: one identity, a different linkage per row (host row) ────────────────── +A1=$WORK/a1; rm -rf "$A1"; make_fw "$A1/fw" +for n in 1 2 3; do + mkdir -p "$A1/app$n/src" + printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > "$A1/app$n/src/main.cpp" +done +cat > "$A1/app1/mcpp.toml" <<'T' +[package] +name = "app1" +version = "0.1.0" +[dependencies] +demo.fw = { path = "../fw" } +[target.'cfg(os = "linux")'.dependencies] +demo.fw = { path = "../fw", linkage = "shared" } +T +cat > "$A1/app2/mcpp.toml" <<'T' +[package] +name = "app2" +version = "0.1.0" +[dependencies] +demo.fw = { path = "../fw" } +[target.'cfg(os = "linux")'.dependencies] +demo.fw = { linkage = "shared" } +T +# The control: two complementary predicates, which the record says work. +# Without it, "no libfw.so" above could mean the shared form is broken in +# general rather than that the declaration was dropped. +cat > "$A1/app3/mcpp.toml" <<'T' +[package] +name = "app3" +version = "0.1.0" +[target.'cfg(os = "linux")'.dependencies] +demo.fw = { path = "../fw", linkage = "shared" } +[target.'cfg(not(os = "linux"))'.dependencies] +demo.fw = { path = "../fw" } +T +(cd "$A1/app3" && "$MCPP" build > build.log 2>&1); rc3=$? +so3=$(find "$A1/app3/target" -name 'libfw.so' 2>/dev/null | head -1) +exe3=$(find "$A1/app3/target" -type f -name app3 -perm -u+x 2>/dev/null | head -1) +needed3=$(readelf -d "$exe3" 2>/dev/null | grep NEEDED | tr '\n' ' ') +[ "$rc3" -eq 0 ] && [ -n "$so3" ] && [[ "$needed3" == *"libfw.so"* ]] +verdict $? "A1 control: two complementary predicates build libfw.so and link it" "exit=$rc3 so=${so3:-none}" +(cd "$A1/app1" && "$MCPP" build > build.log 2>&1); rc1=$? +so1=$(find "$A1/app1/target" -name 'libfw.so' 2>/dev/null | head -1) +[ "$rc1" -eq 0 ] && [ -z "$so1" ] +verdict $? "A1 spelling 1 exits 0 and builds no libfw.so" "exit=$rc1 so=${so1:-none}" +(cd "$A1/app2" && "$MCPP" build > build.log 2>&1); rc2=$? +log2=$(cat "$A1/app2/build.log") +[ "$rc2" -ne 0 ] && [[ "$log2" == *"[dependencies] demo.fw.linkage = 'shared'"* ]] \ + && [[ "$log2" == *"(demo.fw, linkage)"* ]] +verdict $? "A1 spelling 2 is read as the selector (demo.fw, linkage) under the label [dependencies]" "exit=$rc2" + +# ── A2: a key normalising to another identity than its manifest ─────────────── +A2=$WORK/a2; rm -rf "$A2"; mkdir -p "$A2/fw/src" "$A2/comp/src" "$A2/app/src" "$A2/single/src" +cat > "$A2/fw/mcpp.toml" <<'T' +[package] +namespace = "huxdemo" +name = "fw" +version = "0.1.0" +[targets.fw] +kind = "lib" +T +printf 'export module fw;\nexport int fw_anchor() { return 7; }\n' > "$A2/fw/src/fw.cppm" +cat > "$A2/comp/mcpp.toml" <<'T' +[package] +namespace = "huxdemo" +name = "comp" +version = "0.1.0" +[targets.comp] +kind = "lib" +[dependencies] +fw = { path = "../fw" } +T +printf 'export module comp;\nimport fw;\nexport int comp_anchor() { return fw_anchor(); }\n' > "$A2/comp/src/comp.cppm" +cat > "$A2/app/mcpp.toml" <<'T' +[package] +name = "app" +version = "0.1.0" +[dependencies] +huxdemo.fw = { path = "../fw" } +huxdemo.comp = { path = "../comp" } +T +printf 'import fw;\nimport comp;\nint main() { return fw_anchor() + comp_anchor() == 14 ? 0 : 1; }\n' > "$A2/app/src/main.cpp" +cat > "$A2/single/mcpp.toml" <<'T' +[package] +name = "single" +version = "0.1.0" +[dependencies] +fw = { path = "../fw" } +T +printf 'import fw;\nint main() { return fw_anchor() == 7 ? 0 : 1; }\n' > "$A2/single/src/main.cpp" +(cd "$A2/app" && "$MCPP" build > build.log 2>&1); rc=$? +log=$(cat "$A2/app/build.log") +[ "$rc" -ne 0 ] && [[ "$log" == *"module 'fw' already provided by"* ]] +verdict $? "A2 two edges to one directory fail in the scanner" "exit=$rc" +(cd "$A2/single" && "$MCPP" build > build.log 2>&1); rc=$? +log=$(cat "$A2/single/build.log") +[ "$rc" -eq 0 ] && [[ "$log" != *"identity"* ]] +verdict $? "A2 one mismatched edge builds with no identity diagnostic" "exit=$rc" + +# ── A3 and A4: the Android shared-object row ────────────────────────────────── +A3=$WORK/a3; rm -rf "$A3"; make_fw "$A3/fw"; mkdir -p "$A3/app/src" +cat > "$A3/app/mcpp.toml" <<'T' +[package] +name = "app" +version = "0.1.0" +[targets.app] +kind = "app" +main = "src/main.cpp" +[target.'cfg(env = "android")'.dependencies] +demo.fw = { path = "../fw", linkage = "shared" } +[target.'cfg(not(env = "android"))'.dependencies] +demo.fw = { path = "../fw" } +T +printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > "$A3/app/src/main.cpp" +(cd "$A3/app" && "$MCPP" pack --target x86_64-linux-android --format dir > pack.log 2>&1); rc=$? +tree=$(find "$A3/app/target/dist" -mindepth 1 -maxdepth 1 -type d 2>/dev/null | head -1) +files=$(cd "$tree" 2>/dev/null && find . -type f | sort | tr '\n' ' ') +manifest=$(cat "$tree.stage-manifest" 2>/dev/null) +needed=$(readelf -d "$tree/lib/libapp.so" 2>/dev/null | grep NEEDED | tr -s ' ' | tr '\n' ' ') +[ "$rc" -eq 0 ] && [ "$files" = "./lib/libapp.so " ] && [[ "$manifest" == "closure = walked"* ]] \ + && [[ "$needed" == *"libfw.so"* ]] && [[ "$needed" == *"libc++_shared.so"* ]] +verdict $? "A3 --format dir stages only lib/libapp.so, needs libfw.so and libc++_shared.so, says walked" \ + "exit=$rc files=$files" +built=$(find "$A3/app/target/x86_64-linux-android" -path '*/bin/*.so' 2>/dev/null) +nosoname=1 +for so in $built; do + readelf -d "$so" | grep -q SONAME && nosoname=0 +done +[ -n "$built" ] && [ "$nosoname" -eq 1 ] +verdict $? "A4 no DT_SONAME on the Android libfw.so or libapp.so" "libs=$(echo $built | wc -w)" + +# ── A7: the SubOS pkg-config view (reads the registry; builds nothing) ──────── +# `mcpp self env` prints `xlings home = `; the SubOS view is under it. +registry=$("$MCPP" self env 2>/dev/null | sed -n 's/^xlings home *= *//p' | head -1) +view="${registry:-$HOME/.mcpp/registry}/subos/default/usr/lib/pkgconfig" +if [ -f "$view/gtk4.pc" ]; then + out=$(env -u PKG_CONFIG_PATH PKG_CONFIG_LIBDIR="$view" pkg-config --cflags --libs gtk4 2>&1); rc=$? + [ "$rc" -ne 0 ] && [[ "$out" == *"Package 'zlib'"* ]] + # A reading of THIS registry's history, not of provisioning: on a fresh + # home the same command exits 0 (CI, revision 2 of the record, section 5.7). + verdict $? "A7 (this registry only) pkg-config gtk4 against the SubOS view fails on zlib" "exit=$rc" +else + echo "skipped: A7 (no gtk4.pc in $view)" +fi + +# ── A8: a transitive dependency's deploy reaches the application ───────────── +A8=$WORK/a8; rm -rf "$A8"; mkdir -p "$A8/libb/src" "$A8/liba/src" "$A8/app/src" +cat > "$A8/libb/mcpp.toml" <<'T' +[package] +namespace = "demo" +name = "libb" +version = "0.1.0" +[targets.libb] +kind = "lib" +T +printf 'export module libb;\nexport int b() { return 2; }\n' > "$A8/libb/src/libb.cppm" +echo "libb strings" > "$A8/libb/strings.txt" +cat > "$A8/libb/build.mcpp" <<'T' +import std; +import mcpp; +int main() { + mcpp::deploy((std::string(mcpp::manifest_dir()) + "/strings.txt").c_str(), "resources/libb"); + return 0; +} +T +cat > "$A8/liba/mcpp.toml" <<'T' +[package] +namespace = "demo" +name = "liba" +version = "0.1.0" +[targets.liba] +kind = "lib" +[dependencies] +demo.libb = { path = "../libb" } +T +printf 'export module liba;\nimport libb;\nexport int a() { return b() + 1; }\n' > "$A8/liba/src/liba.cppm" +cat > "$A8/app/mcpp.toml" <<'T' +[package] +name = "app" +version = "0.1.0" +[dependencies] +demo.liba = { path = "../liba" } +T +printf 'import liba;\nint main() { return a() == 3 ? 0 : 1; }\n' > "$A8/app/src/main.cpp" +(cd "$A8/app" && "$MCPP" build > build.log 2>&1 && "$MCPP" pack --format dir > pack.log 2>&1); rc=$? +inbin=$(find "$A8/app/target" -path '*/bin/resources/libb/strings.txt' 2>/dev/null | head -1) +instage=$(find "$A8/app/target/dist" -path '*/resources/libb/strings.txt' 2>/dev/null | head -1) +[ "$rc" -eq 0 ] && [ -n "$inbin" ] && [ -n "$instage" ] +verdict $? "A8 a dependency two levels down deploys into the app's bin/ and staged tree" "exit=$rc" + +# ── A10: MCPP_TOOLCHAIN reaches test and pack; the flag does not ───────────── +A10=$WORK/a10; rm -rf "$A10"; mkdir -p "$A10/src" "$A10/tests" +printf '[package]\nname = "a10"\nversion = "0.1.0"\n' > "$A10/mcpp.toml" +printf 'int main() { return 0; }\n' > "$A10/src/main.cpp" +printf '#include \nint main() { std::printf("compiler=%%s\\n", __VERSION__); return 0; }\n' > "$A10/tests/t.cpp" +out=$(cd "$A10" && MCPP_TOOLCHAIN=llvm@22.1.8 "$MCPP" test 2>&1); rc=$? +[ "$rc" -eq 0 ] && [[ "$out" == *"compiler=Clang 22.1.8"* ]] +verdict $? "A10 MCPP_TOOLCHAIN=llvm@22.1.8 mcpp test compiles the test with clang" "exit=$rc" +out=$(cd "$A10" && MCPP_TOOLCHAIN=llvm@22.1.8 "$MCPP" pack --format dir 2>&1); rc=$? +[ "$rc" -eq 0 ] && [[ "$out" == *"Resolved llvm@22.1.8"* ]] +verdict $? "A10 MCPP_TOOLCHAIN=llvm@22.1.8 mcpp pack resolves llvm" "exit=$rc" +for c in run test pack; do + out=$(cd "$A10" && "$MCPP" "$c" --toolchain llvm@22.1.8 2>&1) + [[ "$out" == *"unknown option: --toolchain"* ]] + verdict $? "A10 mcpp $c refuses --toolchain" "$(printf '%s' "$out" | head -1)" +done + +# ── Revision 2 ──────────────────────────────────────────────────────────────── + +# A1: a dependency that declares `kind = "shared"` is linked shared with no +# request from the root, and the degradation for an explicit `static` request +# does not name the package's statement. +R1=$WORK/r2-kind; rm -rf "$R1"; mkdir -p "$R1/fw/src" "$R1/app/src" "$R1/appstatic/src" +cat > "$R1/fw/mcpp.toml" <<'T' +[package] +namespace = "demo" +name = "fw" +version = "0.1.0" +[targets.fw] +kind = "shared" +T +printf 'export module fw;\nexport int fw_anchor() { return 41; }\n' > "$R1/fw/src/fw.cppm" +for a in app appstatic; do + printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > "$R1/$a/src/main.cpp" +done +printf '[package]\nname = "app"\nversion = "0.1.0"\n[dependencies]\ndemo.fw = { path = "../fw" }\n' > "$R1/app/mcpp.toml" +printf '[package]\nname = "appstatic"\nversion = "0.1.0"\n[dependencies]\ndemo.fw = { path = "../fw", linkage = "static" }\n' > "$R1/appstatic/mcpp.toml" +(cd "$R1/app" && "$MCPP" build > build.log 2>&1); rc=$? +so=$(find "$R1/app/target" -name libfw.so 2>/dev/null | head -1) +[ "$rc" -eq 0 ] && [ -n "$so" ] +verdict $? "A1 a kind = \"shared\" dependency is linked shared under a default request" "exit=$rc so=${so:-none}" +out=$(cd "$R1/appstatic" && "$MCPP" build 2>&1) +[[ "$out" == *"is linked as a shared library: the requested form is not available here"* ]] +verdict $? "A1 the explicit-static degradation does not name the package's kind" "$(printf '%s' "$out" | grep -m1 warning)" + +# A6: test programs on the Android row need libc++_shared.so; -static-libstdc++ +# removes it; the driver names its static archives for the effective target. +R6=$WORK/r2-a6; rm -rf "$R6" +for v in default static; do + d="$R6/$v"; mkdir -p "$d/src" "$d/tests" + printf '[package]\nname = "t6"\nversion = "0.1.0"\n[target.x86_64-linux-android]\nmin_api_level = 24\nrunner = ["/bin/false"]\n' > "$d/mcpp.toml" + [ "$v" = static ] && printf '[target.x86_64-linux-android.build]\nldflags = ["-static-libstdc++"]\n' >> "$d/mcpp.toml" + printf 'int main() { return 0; }\n' > "$d/src/main.cpp" + printf '#include \nint main() { std::string s = "x"; return s.size() == 1 ? 0 : 1; }\n' > "$d/tests/t.cpp" + (cd "$d" && "$MCPP" test --target x86_64-linux-android > test.log 2>&1) +done +nd=$(readelf -d "$(find "$R6/default/target" -type f -name t | head -1)" 2>/dev/null | grep -c 'libc++_shared.so') +ns=$(readelf -d "$(find "$R6/static/target" -type f -name t | head -1)" 2>/dev/null | grep -c 'libc++_shared.so') +[ "$nd" -eq 1 ] && [ "$ns" -eq 0 ] +verdict $? "A6 Android test programs need libc++_shared.so unless -static-libstdc++" "default=$nd static=$ns" +NDKCXX=$(find "$HOME/.mcpp/registry/data/xpkgs/xim-x-android-ndk" -path '*/bin/clang++' 2>/dev/null | head -1) +if [ -n "$NDKCXX" ]; then + a=$("$NDKCXX" --target=x86_64-linux-android24 -print-file-name=libc++.a) + [[ "$a" == */24/libc++.a ]] && grep -q 'c++_static' "$a" + verdict $? "A6 the NDK driver names its API-level libc++.a linker script" "$a" +fi + +# A9: a dependency's build program states a fact and a floor; the engine +# refuses a lower root floor before compiling. +R9=$WORK/r2-a9; rm -rf "$R9"; make_fw "$R9/fw"; mkdir -p "$R9/app/src" +cat > "$R9/fw/build.mcpp" <<'T' +import std; +import mcpp; +int main() { + if (std::string_view(mcpp::target_env()).starts_with("android")) { + mcpp::fact("demo.android-api", mcpp::min_platform_version()); + mcpp::floor("demo.android-api >= 23"); + } + return 0; +} +T +printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > "$R9/app/src/main.cpp" +printf '[package]\nname = "app"\nversion = "0.1.0"\n[dependencies]\ndemo.fw = { path = "../fw" }\n[target.x86_64-linux-android]\nmin_api_level = 21\n' > "$R9/app/mcpp.toml" +out=$(cd "$R9/app" && "$MCPP" build --target x86_64-linux-android 2>&1); rc=$? +[ "$rc" -ne 0 ] && [[ "$out" == *"requires demo.android-api >= 23, and this machine has 21"* ]] +verdict $? "A9 a dependency's fact and floor refuse a root floor of 21" "exit=$rc" + +# B1: `mcpp::link_flag` from a host module reaches the root program's link. +RB=$WORK/r2-b1; rm -rf "$RB"; mkdir -p "$RB/rules/src" "$RB/app/src" +printf '[package]\nnamespace = "demo"\nname = "rules"\nversion = "0.1.0"\n[targets.rules]\nkind = "lib"\n' > "$RB/rules/mcpp.toml" +printf 'export module demo.rules;\nimport std;\nimport mcpp;\nexport namespace demo::rules {\ninline void add() { mcpp::link_flag("-Wl,-rpath,/m634/marker/Frameworks"); }\n}\n' > "$RB/rules/src/rules.cppm" +printf '[package]\nname = "app"\nversion = "0.1.0"\n[build-dependencies]\ndemo.rules = { path = "../rules", host-module = true }\n' > "$RB/app/mcpp.toml" +printf 'import std;\nimport mcpp;\nimport demo.rules;\nint main() { demo::rules::add(); return 0; }\n' > "$RB/app/build.mcpp" +printf 'int main() { return 0; }\n' > "$RB/app/src/main.cpp" +(cd "$RB/app" && "$MCPP" build > build.log 2>&1); rc=$? +rp=$(readelf -d "$(find "$RB/app/target" -type f -name app -perm -u+x | head -1)" 2>/dev/null | grep -E 'RPATH|RUNPATH') +[ "$rc" -eq 0 ] && [[ "$rp" == */m634/marker/Frameworks* ]] +verdict $? "B1 a host module's link_flag reaches the root program's link" "exit=$rc" + +# X: `mcpp why deps` shows no path dependency, and resolution.json holds no graph. +out=$(cd "$R1/app" && "$MCPP" why deps 2>&1) +rj=$(find "$R1/app/target" -name resolution.json | head -1) +keys=$(python3 -c "import json,sys; print(sorted(json.load(open(sys.argv[1])).keys()))" "$rj" 2>/dev/null) +[[ "$out" == *"no mcpp.lock"* ]] && [[ "$keys" != *graph* ]] +verdict $? "X why deps reads only mcpp.lock and resolution.json has no graph" "keys=$keys" + +# B5: dist-apk 0.9.3, when a mcpp-plugins checkout at 0.9.3 is given: +# a second pack drops the dependency's library; two ABIs are refused. +if [ -n "${PLUGINS_DIR:-}" ] && [ -d "$PLUGINS_DIR/tests/apk-consumer-shared" ]; then + RP=$WORK/r2-b5; rm -rf "$RP"; mkdir -p "$RP"; cp -r "$PLUGINS_DIR" "$RP/plugins" + F="$RP/plugins/tests/apk-consumer-shared" + list() { unzip -l "$(find "$F/target" -name withlibs.apk | head -1)" 2>/dev/null | grep -c 'libapk-consumer-dep.so'; } + (cd "$F" && "$MCPP" pack --target x86_64-linux-android --format apk > p1.log 2>&1); r1=$?; l1=$(list) + (cd "$F" && "$MCPP" pack --target x86_64-linux-android --format apk > p2.log 2>&1); r2=$?; l2=$(list) + [ "$r1" -eq 0 ] && [ "$r2" -eq 0 ] && [ "$l1" -eq 1 ] && [ "$l2" -eq 0 ] + verdict $? "B5 a second dist-apk pack drops libapk-consumer-dep.so" "exits=$r1,$r2 libs=$l1,$l2" + out=$(cd "$F" && "$MCPP" pack --target x86_64-linux-android --target aarch64-linux-android --format apk 2>&1); rc=$? + [ "$rc" -ne 0 ] && [[ "$out" == *"no action claimed --format 'apk'"* ]] + verdict $? "B5 two ABIs are refused with the engine's generic sentence" "exit=$rc" +else + echo "skipped: B5 (set PLUGINS_DIR to a mcpp-plugins 0.9.3 checkout)" +fi + +# B1 / A3: a link_flag rpath that begins with a Mach-O loader token is anchored +# to the package root, while `$ORIGIN` is not (the same code runs on every host). +RR=$WORK/r2-rpath; rm -rf "$RR"; cp -r "$RB" "$RR"; rm -rf "$RR/app/target" +sed -i 's|/m634/marker/Frameworks|@executable_path/../Frameworks|' "$RR/rules/src/rules.cppm" +(cd "$RR/app" && "$MCPP" build > build.log 2>&1); rc=$? +rp=$(readelf -d "$(find "$RR/app/target" -type f -name app -perm -u+x | head -1)" 2>/dev/null | grep -E 'RPATH|RUNPATH') +[ "$rc" -eq 0 ] && [[ "$rp" == *"$RR/app/@executable_path/../Frameworks"* ]] +verdict $? "B1 an @executable_path rpath from link_flag is anchored to the package root" "$(printf '%s' "$rp" | tr -s ' ' | cut -c1-200)" diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 0824381d9..dc059599f 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -284 records. +286 records. ## By subject @@ -58,6 +58,8 @@ Records that declare one. Everything else is listed by date below. ### triage +- [#634 implemented across five repositories: the plan, its review, and the ledger that tracks it](2026-09-14-634-implementation-plan.md) — active +- [A framework's CMake parity list: the twenty-one items of #634, read against the code and routed to where each one belongs](2026-09-14-634-cmake-parity-items-by-home.md) — active - [What a framework and its ecosystem library still hit in the engine: the ten items of #630, read against the code](2026-09-13-630-what-a-framework-still-hits-in-the-engine.md) — landed - [The engine gaps left open after the SDK batch](2026-09-12-engine-gaps-after-the-sdk-batch.md) — landed - [Six open issues: what each one actually is, and what would answer it](2026-09-11-six-open-issues-analysis.md) — active @@ -66,6 +68,8 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 +- [#634 implemented across five repositories: the plan, its review, and the ledger that tracks it](2026-09-14-634-implementation-plan.md) — active +- [A framework's CMake parity list: the twenty-one items of #634, read against the code and routed to where each one belongs](2026-09-14-634-cmake-parity-items-by-home.md) — active - [Four upstream asks from a UI framework: what each one is under mcpp's design, and the combined plan](2026-09-13-four-upstream-asks-from-a-ui-framework.md) — landed - [What a framework and its ecosystem library still hit in the engine: the ten items of #630, read against the code](2026-09-13-630-what-a-framework-still-hits-in-the-engine.md) — landed - [The engine gaps left open after the SDK batch](2026-09-12-engine-gaps-after-the-sdk-batch.md) — landed diff --git a/.github/tools/android_emulator_test.sh b/.github/tools/android_emulator_test.sh new file mode 100755 index 000000000..bf1138499 --- /dev/null +++ b/.github/tools/android_emulator_test.sh @@ -0,0 +1,128 @@ +#!/usr/bin/env bash +# `mcpp test` on the x86_64 Android row, run through `adb-run` on an emulator. +# +# android_emulator_test.sh prebuild before the emulator: write the +# fixture, provision the row and +# compile its test programs +# android_emulator_test.sh run inside the emulator step: run +# them and assert on the stream +# +# The fixture's tests measure the two properties #634 A6 decided (triage +# record §5.6), each of which failed on 2026.9.14.1 on an API 34 emulator: +# +# runs the program loads: on this row it needs no +# `libc++_shared.so`, because the engine finds the NDK's +# static C++ runtime by asking the driver +# reads_relative a file `[runtime] deploy` placed beside the program is +# readable there, because the runner receives +# MCPP_RUNTIME_FILES and `adb-run` pushes what it lists +# +# `MCPP` names the engine under test; the assertion reads the JSON test stream +# (docs/50 §8), not the human summary. +set -euo pipefail + +mode="${1:?usage: android_emulator_test.sh prebuild|run }" +dir="${2:?usage: android_emulator_test.sh prebuild|run }" +: "${MCPP:?set MCPP to the engine under test}" +TARGET=x86_64-linux-android + +write_fixture() { + rm -rf "$dir" + mkdir -p "$dir/src" "$dir/tests" "$dir/share" + echo "m634-data" > "$dir/share/data.txt" + cat > "$dir/mcpp.toml" <<'TOML' +[package] +name = "droidtest" +version = "0.1.0" + +[targets.droidtest] +kind = "bin" +main = "src/main.cpp" + +[runtime] +deploy = [ { from = "share/data.txt", to = "data" } ] + +[target.x86_64-linux-android] +min_api_level = 24 +runner = ["adb-run"] + +[target.x86_64-linux-android.xlings.workspace] +"xim:android-platform-tools" = "37.0.1-4" +TOML + printf 'int main() { return 0; }\n' > "$dir/src/main.cpp" + printf 'int main() { return 0; }\n' > "$dir/tests/runs.cpp" + cat > "$dir/tests/reads_relative.cpp" <<'CPP' +#include +#include +#include +int main(int, char** argv) { + std::string p = argv[0]; + auto s = p.rfind('/'); + std::string f = (s == std::string::npos ? std::string(".") : p.substr(0, s)) + "/data/data.txt"; + FILE* fp = std::fopen(f.c_str(), "r"); + if (!fp) { std::printf("open failed: %s\n", f.c_str()); return 3; } + char buf[64] = {0}; + std::fgets(buf, sizeof buf, fp); + std::fclose(fp); + std::printf("read %s: %s", f.c_str(), buf); + return std::strncmp(buf, "m634-data", 9) == 0 ? 0 : 4; +} +CPP +} + +status_of() { # status_of + python3 - "$1" "$2" <<'PY' +import json, sys +for line in open(sys.argv[1]): + line = line.strip() + if not line.startswith("{"): + continue + rec = json.loads(line) + if rec.get("test") == sys.argv[2]: + print(rec.get("status"), rec.get("exit_code")) + break +else: + print("absent") +PY +} + +case "$mode" in + prebuild) + write_fixture + cd "$dir" + # Compiles the tests and provisions the NDK and the platform tools. No + # device is attached yet, so the runs are reported, not asserted. + "$MCPP" test --target "$TARGET" --message-format json > prebuild.json 2> prebuild.err || true + tail -5 prebuild.err + # The property the runner step depends on, checked on the build + # machine: no test program names the shared C++ runtime. + found=0 + while IFS= read -r prog; do + found=$((found + 1)) + if readelf -d "$prog" | grep -q 'NEEDED.*libc++_shared.so'; then + echo "FAIL: $prog needs libc++_shared.so" + exit 1 + fi + done < <(find target -type f \( -name runs -o -name reads_relative \) -path '*/bin/*') + [ "$found" -ge 2 ] || { echo "FAIL: expected two test programs, found $found"; exit 1; } + echo "prebuild: $found test programs, none needs libc++_shared.so" + ;; + run) + cd "$dir" + adb devices + "$MCPP" test --target "$TARGET" --message-format json > test.json 2> test.err || true + tail -5 test.err + cat test.json + runs=$(status_of test.json runs) + reads=$(status_of test.json reads_relative) + echo "runs: $runs" + echo "reads_relative: $reads" + [ "$runs" = "pass 0" ] || { echo "FAIL: runs did not pass on the emulator"; exit 1; } + [ "$reads" = "pass 0" ] || { echo "FAIL: reads_relative did not pass on the emulator"; exit 1; } + echo "PASS: mcpp test on the x86_64 Android emulator" + ;; + *) + echo "usage: android_emulator_test.sh prebuild|run " >&2 + exit 2 + ;; +esac diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 6e427b99f..7ba672c7d 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -257,6 +257,93 @@ jobs: cat 338_cortex_m_picolibc_sysroot.sh.log echo "338 neither passed nor reported why it did not run"; exit 1; } + # ────────────────────────────────────────────────────────────────── + # Android: the rows no sharded runner can reach. + # + # `run_all.sh` adds the `android-ndk` capability only when the NDK payload + # is already installed, and no other job installs it, so every + # `# requires: android-ndk` test skipped on every CI runner while reporting + # green. This job installs the payload, runs those tests directly and demands + # each PASS line, then runs `mcpp test` on an API 34 emulator through + # `adb-run` (the configuration measured on mcpp#635), which is the only + # place a test program's loading on the device is observed. + # ────────────────────────────────────────────────────────────────── + android: + name: android e2e (NDK rows + x86_64 emulator) + runs-on: ubuntu-24.04 + timeout-minutes: 90 + env: + MCPP_HOME: /home/runner/.mcpp + steps: + - uses: actions/checkout@v4 + - uses: ./.github/actions/bootstrap-mcpp + + - name: Build mcpp from source (self-host) + run: | + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true + "$MCPP" self config --mirror GLOBAL 2>/dev/null || true + "$MCPP" build + + - name: Install the NDK (xim:android-ndk) + timeout-minutes: 20 + run: | + XLINGS_HOME="$MCPP_HOME/registry" "$XLINGS_BIN" install xim:android-ndk@30.0.16248370 -y + # Present before the tests, so a missing payload fails this step + # rather than an assertion deep inside a test. + ls "$MCPP_HOME"/registry/data/xpkgs/xim-x-android-ndk/*/toolchains/llvm/prebuilt/*/bin/clang++ + + - name: Android e2e (no device) + timeout-minutes: 40 + run: | + MCPP=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") + test -x "$MCPP" + export MCPP + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL + "$MCPP" self config --mirror "$MCPP_E2E_TOOLCHAIN_MIRROR" + "$MCPP" toolchain install gcc 16.1.0 + "$MCPP" toolchain default gcc@16.1.0 + # Run directly, not through run_all.sh, so a skip cannot pass as a + # green: each script either prints its PASS line or this step fails. + declare -A pass=( + [652b_an_application_on_android_is_a_shared_library]='652b: kind = "app" on Android is a shared library OK' + [664_a_universal_apk_is_two_legs_in_one_tree]='664: the universal APK is the library route applied to an app OK' + [667_an_android_pack_stages_its_closure]='667: an Android pack stages its closure OK' + [675_android_test_programs_carry_their_cxx_runtime]='PASS: 675_android_test_programs_carry_their_cxx_runtime' + [680_a_dependency_floor_on_the_android_api_level]='PASS: 680_a_dependency_floor_on_the_android_api_level' + ) + for t in "${!pass[@]}"; do + echo "=== $t ===" + bash "tests/e2e/$t.sh" 2>&1 | tee "$t.log" + rc=${PIPESTATUS[0]} + [ "$rc" = "0" ] || { echo "$t failed (exit $rc)"; exit 1; } + grep -qF "${pass[$t]}" "$t.log" || { echo "$t did not print its PASS line"; exit 1; } + done + echo "MCPP=$MCPP" >> "$GITHUB_ENV" + + - name: Enable KVM + run: | + echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules + sudo udevadm control --reload-rules + sudo udevadm trigger --name-match=kvm + + - name: Compile the device tests before the emulator starts + timeout-minutes: 20 + run: bash .github/tools/android_emulator_test.sh prebuild "$RUNNER_TEMP/droidtest" + + - name: mcpp test on an API 34 emulator + timeout-minutes: 30 + uses: reactivecircus/android-emulator-runner@v2 + with: + api-level: 34 + arch: x86_64 + target: google_apis + force-avd-creation: false + emulator-options: -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none + disable-animations: true + script: bash .github/tools/android_emulator_test.sh run "$RUNNER_TEMP/droidtest" + # ────────────────────────────────────────────────────────────────── # Hermetic (no host toolchain): the ONLY environment class that # faithfully reproduces issue #195. Standard runners ship gcc + diff --git a/CHANGELOG.md b/CHANGELOG.md index 0ccff9c45..4a27711e5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,60 @@ ## [Unreleased] +### 一个框架的 CMake 对齐清单:#634 的二十一项中引擎的部分(2026.9.14.2) + +#634 列出 HuxerUI 从 CMake 迁到 mcpp 时仍缺的二十一项。分类与决定见设计记录 +`2026-09-14-634-cmake-parity-items-by-home.md`,实现计划与台账见 +`2026-09-14-634-implementation-plan.md`;引擎只承接通用能力,专有功能归官方插件 +(mcpp-plugins 0.10.0)、载荷(xim-pkgindex#838)或工程本身。 + +**条件依赖声明替换无条件声明(A1)。** 选择器命中的行上, +`[target..dependencies]` 中某个身份的声明替换 `[dependencies]` 中同一身份的 +声明,按规范化身份比较,多段按清单顺序、后者为准;`dev-dependencies`、 +`build-dependencies`、`feature-deps` 同理。此前保留的是无条件那条,条件表里的 +`linkage = "shared"` 在它自己的行上被静默丢弃。只写选项、不写来源的条件表给出 +补全来源后的写法;`[target.]` 下 mcpp 不读取的子表被报出(已知表由单测从 +解析点双向核对)。`[target..targets.] kind` 是库目标形态的按行形式, +消费者的 `linkage = "static"` 被拒时,告警点名这一行。 + +**`path`/`git` 依赖的身份取自其清单(A2)。** 键规范化到另一身份时采用清单声明并 +每条声明边告警一次;同一来源上两个命名空间不同的键在扫描前拒绝。mcpp 自己的三个 +示例改写为声明的身份。 + +**Android 与 Mach-O 的闭包被读取并暂存(A3)。** `mcpp pack` 从文件读闭包:Android +暂存到 `lib/`(多 triple 为 `lib//`),含图里的共享库与 `libc++_shared.so`; +Mach-O 的 dylib 暂存在程序旁,不改 load command、不重签。stage manifest 增加 +`needs<路径|platform|unresolved>` 行;闭包不完整时写 `not-walked`, +`dir`/`tar` 拒绝并点名。rpath 中以 `@executable_path`、`@loader_path`、`@rpath` +开头的项不再被锚定到包目录。 + +**ELF 共享库默认以文件名为 SONAME(A4)。** 位置在 `$ldflags` 之前,工程自己写的 +`-soname` 仍生效。 + +**测试与运行器(A5、A6、A10、B3)。** `[test] discover` 指定测试从哪些 glob 发现; +静态 C++ 运行时通过询问驱动(带 API level)定位,Android 行的测试程序不再依赖 +`libc++_shared.so`,NDK 的 `libc++.a` 链接脚本里的归档名进入 `--exclude-libs`; +每个 runner 收到 `MCPP_RUNTIME_FILES`(部署文件与链接的共享库,TAB 分隔); +`run`、`test`、`pack` 接受 `--toolchain`;`mcpp run --format ` 使用名为 `` 的 +runner,没有 runner 能到达的目录分发物在启动前以 126 拒绝。依赖的构建程序与根的 +构建程序提供同名 runner 时拒绝并点名二者。 + +**解析记录与平台下限(X、A9)。** `resolution.json` 增加 `graph`(每个包、每条请求 +的键与声明表、库的链接形态与原因),`mcpp why deps` 打印它;引擎把 +`android.api-level`、`ios.deployment-target`、`macos.deployment-target` 陈述为事实, +依赖以 `version-floor` 要求拒绝过低的下限,拒绝写明「this build targets」与设定来源。 + +**配置与访问器(C4、A7)。** `config.toml` 的 `[index.repos.]` 对已存在的 home +生效,删去后恢复原条目;从被覆盖的索引安装时打印来源。构建程序新增 +`mcpp::pkg_config_libdir()`。 + +**实现中发现并修复的缺陷。** 嵌套在 `tests/` 子目录中的测试加载不到图构建的共享库; +被拒的 `--toolchain` 值被记在 `[toolchain].` 名下。 + +- 判据:`tests/e2e/266`、`666`–`685`(677–685 在 2026.9.14.1 上失败), + 单测 `test_pack_closure`、`TargetScalarKeys.EveryParsedSubTableIsKnownToTheSweep`; + `ci-linux-e2e` 新增 `android` job,在 API 34 模拟器上跑 `mcpp test`。 + ### 图里的 libc++ 之下,预编译 C 库的头文件也要进 std 模块的命令;iOS 行未写的部署下限取 SDK 的版本 `llvm.libcxx` 的 CI 扩到它声明的每一行后量到两处引擎缺口。包提供的 std 模块在 diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 27a79986f..fca9b8009 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -147,7 +147,10 @@ Autotools/CMake. On Linux, mcpp passes `-Wl,-soname,` to the linker and generates a ` -> lib.so` alias in the output directory, so that downstream programs can load the library via its standard ABI name through `DT_NEEDED` or `dlopen()`. This field only applies to `kind = "shared"`, and the -value must be a filename basename. +value must be a filename basename. An ELF shared library that declares no +`soname` records its output file name as its SONAME (2026.9.14.2+), which is +the name its consumers already record in `DT_NEEDED`; bionic requires one from +API level 23. Shared-library targets work on all three binary formats. ELF gets a `.so` with its `soname` and a `$ORIGIN` search path; Mach-O gets a `.dylib` whose install @@ -440,6 +443,7 @@ whose C library is linked statically — which is the **default for musl** — | The package writes | mcpp reads it as | |---|---| | `[targets.] kind = "shared"` | *must* be shared — something else in the process will `dlopen` it, so there may only be one copy (X11, a Vulkan loader) | +| `[target..targets.] kind = "shared"` *(2026.9.14.2+)* | *must* be shared on the rows the selector matches, and either form elsewhere ([22 — The Target Side](22-target-side.md)) | | `ldflags` containing `-L` | *must* be static — the package ships prebuilt archives mcpp did not compile and cannot place inside a shared object it builds | | a packaged library (`mcpp pack`) | whichever legs it actually ships, from `[[runtime.artifacts]] role` | | anything else | either form | @@ -448,6 +452,13 @@ whose C library is linked statically — which is the **default for musl** — packages write it without choosing anything. Absence of a statement is not a statement. +A request the constraint refuses is linked in the form the package allows, +with a warning that names the package's statement (`its manifest states +[targets.fw] kind = "shared", ...`); `--strict` turns the warning into an +error. `mcpp why deps` reports each dependency's form and the reason for it: +`default`, `requested`, `package-kind`, `row-kind`, `packaged`, `no-sources`, +`prebuilt-inputs`, `no-loader` or `static-libc` (2026.9.14.2+). + A per-dependency `linkage` is honoured **only in the root project's** `[dependencies]`. A package deep in the graph does not get to decide how the final program is laid out; one that genuinely must be a single shared copy says @@ -1434,6 +1445,24 @@ do`. Moved to [09 — Commands by Scenario](09-commands-by-scenario.md). +### 2.17 `[test]` — Where Test Programs Are + +```toml +[test] +discover = ["tests/**/*.cpp"] # the default +``` + +| Key | Type | Meaning | +|---|---|---| +| `discover` | array of globs | every file a glob matches is one test program; a glob beginning with `!` removes the files it matches; `[]` discovers none | + +The globs use the vocabulary of `[build] sources`. A test's name is its path +relative to the fixed directory of the first glob that matched it, without the +extension. Two files with one name are refused, naming both. A value that is not +an array of non-empty strings is an error; any other key in `[test]` is a +warning, and an error under `--strict`. [08 — Testing](08-testing.md) describes +the test model. + ## 3. Worked Examples @@ -1495,7 +1524,7 @@ kind = "bin" | C standard | `c11` | `.c` files go through the C compiler automatically | | Static stdlib | `true` | Portable binary | | Headers | `include/` (if present) | Added to `-I` automatically | -| Tests | `tests/**/*.cpp` | Discovered automatically by `mcpp test` | +| Tests | `tests/**/*.cpp` | Discovered automatically by `mcpp test`; `[test] discover` replaces the set | | Dependency namespace | `mcpplibs` (default) | A bare selector means only this exact namespace | ### 4.1 Legacy `[language]` Compatibility Layer diff --git a/docs/05-dependencies.md b/docs/05-dependencies.md index f5ff8dd91..3ca040567 100644 --- a/docs/05-dependencies.md +++ b/docs/05-dependencies.md @@ -140,6 +140,33 @@ disagrees with another dependency, with neither being the root, is never settled by guessing which one was declared first — that is exactly the "accident of queue order" this section replaces. +### The identity of a `path` or `git` dependency (mcpp 2026.9.14.2+) + +A `path` or `git` dependency is the package its manifest declares, whatever key +reaches it. A key that normalises to another identity than the manifest's +`[package] namespace` and `name` takes the declared identity, and mcpp warns +once for each declaring edge, naming the requester, the key, the identity the +key names and the identity the manifest declares: + +```toml +# comp/mcpp.toml; fw/mcpp.toml declares namespace = "huxdemo" +[dependencies] +fw = { path = "../fw" } # names mcpplibs.fw; huxdemo.fw is used +``` + +``` +warning: 'huxdemo.comp@path' declares the dependency 'fw', which names mcpplibs.fw; the manifest '.../fw/mcpp.toml' declares huxdemo.fw, and that identity is used. + hint: write 'huxdemo.fw' in 'huxdemo.comp@path' to state the identity the manifest declares. +``` + +Two edges written `fw` and `huxdemo.fw` over one directory are therefore one +package, compiled once, and `mcpp why deps` lists both keys under it. A +manifest that declares no namespace takes the key's, so two keys with +different namespaces over one such directory are two identities over one +source: the build is refused before scanning, naming both, and the fix is a +`namespace` in that manifest or one key in both places. A `version` dependency +is unaffected; its identity is the key. + ### Namespace resolution rules Every package has a two-part identity: a **namespace** and a **name**. Every diff --git a/docs/06-features-and-capabilities.md b/docs/06-features-and-capabilities.md index 5b280fe7f..ba15baa5a 100644 --- a/docs/06-features-and-capabilities.md +++ b/docs/06-features-and-capabilities.md @@ -318,7 +318,7 @@ mcpp compares them when capabilities are bound and refuses before anything is compiled, reporting `version-floor-unmet`: ``` -error: `toolkitnew` requires cuda.driver >= 13.0, and this machine has 12.4. +error: `toolkitnew` requires cuda.driver >= 13.0, and cuda.driver is stated as 12.4. stated by: driverfact ``` @@ -332,6 +332,34 @@ has is not a machine that fails the floor — it is one nobody asked. Turning asserted directly: `tests/e2e/603_version_floor.sh` builds a project whose floor names something no package provides. +**The engine states the target's platform floor** (mcpp 2026.9.14.2+). On a +row whose compiler takes a minimum platform version, the engine states that +version as a fact in the platform's own words, and a package writes a floor +against it as against any other fact: + +| fact | rows | set by | +|---|---|---| +| `android.api-level` | `*-linux-android` | `[target.] min_api_level`, else the toolchain's lowest supported level | +| `ios.deployment-target` | the iOS device and simulator rows | `[build] ios_deployment_target`, else the located SDK's version | +| `macos.deployment-target` | the macOS rows | `[build] macos_deployment_target`, else mcpp's default for macOS | + +```toml +[[runtime.requirements]] +kind = "version-floor" +value = "android.api-level >= 23" +``` + +``` +error: `fw` requires android.api-level >= 23, and this build targets 21. + set by: [target.x86_64-linux-android] min_api_level +``` + +A row that states no such fact leaves the requirement silent, so the +requirement needs no selector. The floor is not raised on the dependency's +behalf: the value the key sets is the one the compiler targets, and which +devices an application installs on is the application's decision. A package +that states a fact of the same name does not replace the engine's. + **It is a claim about this package's own symbols**, so an entry that names a capability the package does not provide is reported as a schema warning: there is nothing to be exclusive about. And a capability nobody declares exclusive diff --git a/docs/07-workspace.md b/docs/07-workspace.md index 6f5df7c9a..b6af2339e 100644 --- a/docs/07-workspace.md +++ b/docs/07-workspace.md @@ -68,7 +68,7 @@ name = "myapp" version = "0.1.0" [dependencies] -core = { path = "libs/core" } +myproject.core = { path = "libs/core" } ``` ### 2.3 Member Project Files @@ -96,7 +96,7 @@ name = "http" version = "0.1.0" [dependencies] -core = { path = "../core" } +myproject.core = { path = "../core" } [dependencies.compat] mbedtls.workspace = true diff --git a/docs/08-testing.md b/docs/08-testing.md index 95415712b..1ce9abe6d 100644 --- a/docs/08-testing.md +++ b/docs/08-testing.md @@ -36,6 +36,27 @@ also why a test written for another framework needs no adapter. `mcpp new` scaffolds `tests/test_smoke.cpp` so a project starts with the directory in place. +### Where tests are + +`tests/**/*.cpp` is the default of one key: + +```toml +[test] +discover = ["checks/**/*.cpp", "!checks/fixtures/**"] +``` + +`discover` takes globs in the vocabulary of `[build] sources`: every file a +glob matches is one test program, and a glob beginning with `!` removes the +files it matches from the set, whichever glob found them. A test's name is its +path relative to the fixed directory of the first glob that matched it, without +the extension, so the default names `tests/unit/test_span.cpp` `unit/test_span`. +`discover = []` discovers no test. Two files whose names coincide are refused, +naming both. + +A suite compiled from several sources is a package of its own: a workspace +member whose `[build] sources` carries the suite and whose one test program +drives it, selected with `mcpp test -p `. + ## Running them ```bash @@ -55,6 +76,7 @@ configuration it is meant to check rather than against the default one: | `--target ` | a target other than the host | | `--accel ` / `--no-accel` | the device backends the build targets | | `--cap ` | pin a capability provider | +| `--toolchain ` | the toolchain for this invocation, e.g. `llvm@22.1.8` | `--timeout ` kills a test still running (default 300; `0` disables it) and `--build-timeout ` bounds the compile. A test that hangs is reported as a @@ -92,6 +114,13 @@ or a QEMU exit code is what a bare-metal runner is chosen to produce. `--no-runner` exists for a host that can execute the binaries natively and should not pay for an emulator. +A test program carries the files it reads beside it: the runner receives +`MCPP_RUNTIME_FILES`, the list of its deployed files and the shared libraries it +loads, and a runner that moves the program to a device copies them with it. A +test locates such a file relative to its own directory. On the Android rows a +test program links the C++ runtime statically unless `cxx_runtime` says +otherwise, so it needs no `libc++_shared.so` on the device. + The runner itself, named runners, and what a board declares are [41 — Reaching a Device](41-devices.md). @@ -119,6 +148,8 @@ here is only that the flag exists and that the human format is the default. - A test is one `.cpp` producing one program. mcpp does not discover cases inside a file, so a framework's per-case selection happens inside the program, through arguments after `--`. +- `mcpp test --list` over a manifest that does not load lists + `tests/**/*.cpp`, not the `[test] discover` set it cannot read. - `--build-timeout` is POSIX-only. - `--workspace-timeout` bounds a `--workspace` fan-out and reports what did run; it does not attribute the timeout to a member. diff --git a/docs/09-commands-by-scenario.md b/docs/09-commands-by-scenario.md index 4d9f3f55d..6d88751e0 100644 --- a/docs/09-commands-by-scenario.md +++ b/docs/09-commands-by-scenario.md @@ -107,6 +107,22 @@ This scan runs only after a lookup has already failed, and its result reaches error text and search output only. A bare name never resolves across namespaces on the strength of it. +## Another toolchain for one invocation + +`mcpp build`, `mcpp run`, `mcpp test` and `mcpp pack` take `--toolchain `, +which selects the compiler for that invocation and writes nothing: + +```bash +mcpp test --toolchain llvm@22.1.8 +mcpp run --toolchain gcc@16.1.0 +mcpp pack --toolchain llvm@22.1.8 --format dir +``` + +For that invocation the option takes the place of `[toolchain] default` in +`mcpp.toml`, at the rank [20 — Toolchain Management](20-toolchains.md) gives +`MCPP_TOOLCHAIN`. Each toolchain builds into its own output directory, and a +recorded build is replayed only for the toolchain request that recorded it. + ## Explaining a resolution `mcpp why` reports what a build would resolve, and builds nothing: @@ -118,6 +134,27 @@ toolchain: gcc 16.1.0 (x86_64-linux-gnu) reason: [toolchain] in mcpp.toml if set, else platform-native default ``` +`mcpp why deps` lists the resolved dependency graph before the lines of +`mcpp.lock` (2026.9.14.2+): every package, the key and the table each request +was written with, and a library's link form with the reason for it. A `path` +dependency, which the lock does not record, is listed too: + +``` +$ mcpp why deps +dependency graph: + mcpplibs.app@0.1.0 (root) path+/work/app + huxdemo.fw@0.1.0 path+/work/fw + requested by mcpplibs.app@0.1.0 as 'huxdemo.fw' in [dependencies] + requested by huxdemo.comp@0.1.0 as 'fw' in [dependencies] + linked static (default) +``` + +The same graph is recorded under `graph` in +`target///resolution.json`, one entry per package with the root +first: `package` (canonical identity, namespace, name, version, source), +`root`, `requested_by` (`requester`, `key`, `table`), and, for a library, +`link` (`form`, `reason`). + The topic is `toolchain`, `runtime`, `deps` or `runners`, and all four report when none is named. `--target` and `--toolchain` turn the report into a query about a pair the current directory does not use, which is how a target matrix diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 38d6f95f3..5da44dd94 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -197,7 +197,9 @@ one `--target`** (mcpp 2026.9.13.2+): on every Android row an application aarch64-linux-android --target x86_64-linux-android` builds and stages both legs into one tree, exactly as a library package's several triples already do. Each leg lands at `lib//lib.so` (`aarch64` → `arm64-v8a`, -`x86_64` → `x86_64`), the declared deploy files are staged once, and one +`x86_64` → `x86_64`) with its own closure beside it (see +[Android](#android-the-application-object-and-its-closure-under-lib)), the +declared deploy files are staged once, and one dispatch runs against the combined tree — which is what lets a member such as `dist-apk` build one universal APK. A single `--target` keeps today's flat `lib/lib.so` layout unchanged. A target whose artifact is an executable @@ -299,6 +301,18 @@ matches the tarball filename (minus the `.tar.gz`) —— this way both a GUI self-contained directory, instead of scattering the contents across the current path. +**The closure is recorded, and an incomplete one refuses the archive** +(mcpp 2026.9.14.2+). Beside the staged tree, `.stage-manifest` states +every library name the closure resolved: the staged path of a library the tree +carries, `platform` for one the target provides, and `unresolved` for one found +nowhere (the line format is in [50](50-machine-output.md#the-stage-manifest)). +When a name is unresolved, `--format tar` and `--format dir` fail naming it, +because a tree without that library does not start where it is installed; a +dispatched format still receives the tree, with `closure = not-walked`. A +library the target provides and this machine does not have is named in +`[pack.bundle-project] also_skip`. The modes that bundle nothing (`system`, +`static`) state no closure. + ### Mode `static` ``` @@ -477,37 +491,70 @@ Two consequences worth knowing: - The archive is **deterministic**: no timestamps are read, so two packs of the same tree are byte-identical and a published checksum means something. -The reverse direction — packing a Linux or macOS artifact *from* Windows — -still does not work, and for the original reason: that closure is resolved by -the target's own dynamic linker, which a Windows host has no way to run. - -#### Packing a Mach-O program is refused — on every host, including macOS - -The same closure step asks the dynamic linker for the dependency list by running -the artifact with `LD_TRACE_LOADED_OBJECTS=1`. That variable is glibc's; dyld -has never heard of it. So on a Mac the command does not trace anything — **it -runs the program**, and whatever the program prints is then parsed as a -dependency table. mcpp refuses instead, and says which mechanism is missing. - -The refusal is keyed on the artifact's **format**, not on the host, for the same -reason the Windows one is: `LD_TRACE_LOADED_OBJECTS` cannot trace a Mach-O from -Linux either. - -A `kind = "lib"` / `"shared"` target packs normally on macOS — a library package -never runs the artifact. This restriction is only for programs. - -The dependency closure can now be **read** rather than run — `mcpp.pack.binfmt` -walks a Mach-O's load commands (`LC_LOAD_DYLIB` and its weak/re-export/upward -siblings for names, `LC_RPATH` for search entries) the same way it already -reads a PE's import table, and resolves `@executable_path`, `@loader_path` and -`@rpath` the way `dyld` would, without loading anything. `mcpp pack` does not -call it yet: bundling a resolved dylib beside the program needs an editor for -`LC_RPATH` (a load command has no free space to grow into), and that editor is -designed once resolution is measured on a real macOS build, not before. Until -then a Mach-O program still stages without its closure — see [Producing a -distributable](30-build-mcpp.md#producing-a-distributable-pack_format--stage_dir-20269111) -for what a dispatched format (`.app`, `.ipa`) can already do with a tree that -has a program and no closure. +The reverse direction — packing a Linux artifact *from* Windows — still does +not work: an ELF program's closure is resolved by the target's own dynamic +linker, which a Windows host has no way to run. A Mach-O program's closure is +read from the file, so a macOS program packs from any host. + +### macOS (Mach-O): the dylibs beside the program + +``` +target/dist/myapp-0.1.0-aarch64-macos.tar.gz +└── myapp-0.1.0-aarch64-macos/ + ├── bin/myapp + ├── bin/libmydep.dylib ← a dependency's dylib, beside the program + ├── myapp ← entry-point wrapper + ├── README.md + └── LICENSE +``` + +A Mach-O program's closure is read from its load commands (mcpp 2026.9.14.2+), +so packing it runs nothing. Its dylibs are staged beside it in `bin/`, which is +where the `@loader_path` rpath a consumer of a graph-built dylib is linked with +already looks: no load command is edited, nothing is re-signed, and the program +is not stripped. + +A needed name is staged when the loader resolves it to the program's +directory: + +| Name | Treatment | +|---|---| +| `/usr/lib/…`, `/System/Library/…` | not staged; the OS provides it | +| `@rpath/` | staged when an rpath of the image that needs it, or of the program, is exactly `@loader_path` or `@executable_path` | +| `@loader_path/`, `@executable_path/` | staged at that path, which must stay inside the program's directory | + +Any other name leaves the closure incomplete, and `tar` and `dir` refuse naming +it. An absolute install name outside the OS's directories is the common case: +the loader reads it from that path on every machine, whatever the tree carries. +`--mode system` and `--mode static` bundle nothing on this row, as on PE. A +`kind = "lib"` / `"shared"` target is packaged by the library route, which is +unaffected. + +### Android: the application object and its closure under `lib/` + +``` +target/dist/myapp-0.1.0-x86_64-linux-android/ +├── lib/libmyapp.so ← the application object +├── lib/libmydep.so ← a dependency's shared library +├── lib/libc++_shared.so ← the NDK's C++ runtime, when the object needs it +└── bin//… ← files `[runtime] deploy` placed +``` + +On an Android row the application is a shared object staged under `lib/`, and +its closure is read from the files and staged beside it (mcpp 2026.9.14.2+). A +name the device provides is not staged: one present in the API level's stub +directory, which mcpp takes from the row's own compiler (the directory it finds +`libc.so` in) — `libc.so`, `libm.so`, `libdl.so`, `liblog.so` and the other +platform libraries. Every other name must resolve in a directory the link used: +the build's output directory, the `[runtime] library_dirs` and +`link_library_dirs` of the packages in the graph, and the compiler's library +search path, which holds `libc++_shared.so`. A name that resolves nowhere makes +`tar` and `dir` refuse, naming it and the directories searched. + +With several `--target` triples each leg is staged under `lib//` with its +own closure. `--mode` does not apply on this row: the device provides exactly +the platform libraries, so there is no host set to leave out and no loader to +carry. ## Configuration @@ -540,12 +587,7 @@ The `static` mode additionally requires a musl toolchain configured under ## Planned Support -macOS **program** bundling is still on the roadmap. The closure is read now -(`mcpp.pack.binfmt`'s Mach-O load-command walk — no `otool` needed), but -bundling a resolved dylib beside the program and rewriting `LC_RPATH` for it -is not; until that lands `mcpp pack ` still refuses on that format -rather than producing something that only looks like a bundle. Windows DLL -bundling beyond the current `.zip` is also on the roadmap. +Windows DLL bundling beyond the current `.zip` is on the roadmap. Distribution formats such as `.deb`, `.rpm`, AppImage and `.msi` are **not** on this list, and that is a decision rather than an omission: they live in diff --git a/docs/12-binary-distribution.md b/docs/12-binary-distribution.md index 15b2e1be1..0d7d6642d 100644 --- a/docs/12-binary-distribution.md +++ b/docs/12-binary-distribution.md @@ -85,9 +85,11 @@ mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23/ produce `libfoo.a` and `foo.lib` respectively. A **shared** package carries the library under *both* of its names: a consumer -links `lib.so` and the loader then asks for the `SONAME`, and those are -different filenames. Shipping only the built file links cleanly and then fails -to start. +links `lib.so` and the loader then asks for the `SONAME`. A declared +`soname` makes those two different filenames, and shipping only the built file +links cleanly and then fails to start. A library that declares no `soname` +records its file name as its `SONAME` (mcpp 2026.9.14.2+), so its two names are +one file. ### The reason neither set may be trimmed diff --git a/docs/22-target-side.md b/docs/22-target-side.md index da2a93b50..0d1aa7235 100644 --- a/docs/22-target-side.md +++ b/docs/22-target-side.md @@ -401,7 +401,36 @@ for arch/env conditions and combinators. `include_dirs` / `include_dirs_after` (mcpp 0.0.102+), plus `private_include_dirs` and `std-module-flags` (mcpp 2026.9.1.1+), and `runtime` with `frameworks` / `libraries` / `link_library_dirs` - (mcpp 2026.8.29.1+; `frameworks` since 2026.9.12.3). + (mcpp 2026.8.29.1+; `frameworks` since 2026.9.12.3), and `targets.` + with `kind` (mcpp 2026.9.14.2+; see + [`targets. kind`](#targetsname-kind--a-librarys-form-on-one-row-mcpp-20269142)). +- **A sub-table that mcpp does not read is reported** (mcpp 2026.9.14.2+): a + misspelled `[target..dependecies]` is a warning naming the sections a + `[target.]` table has, and `--strict` makes it an error. +- **A conditional declaration of a dependency replaces the unconditional one** + (mcpp 2026.9.14.2+). On a row the selector matches, the declaration of an + identity in `[target..dependencies]` is the declaration of that + identity, and the unconditional declaration does not apply there. A + dependency that one row links differently is written twice, each time with + its source: + + ```toml + [dependencies] + huxerui.huxerui = { version = "0.3.0" } + + [target.'cfg(env = "android")'.dependencies] + huxerui.huxerui = { version = "0.3.0", linkage = "shared" } + ``` + + The same rule applies to `dev-dependencies`, `build-dependencies` and + `feature-deps.`, and several matching sections apply in manifest + order, the last one winning. `mcpp why deps` names the table each request + came from ([09 — Commands by Scenario](09-commands-by-scenario.md)). A table + that writes options without a source, `huxerui.huxerui = { linkage = + "shared" }`, declares a dependency on a package named + `huxerui.huxerui.linkage`: mcpp reports the line with the dependency + restated, and the resolution fails. An engine before 2026.9.14.2 keeps the + unconditional declaration. - **`runtime` is the dialect-neutral half of a link line.** `build.ldflags` is spelled the GNU way, and a native `cl.exe` rejects `-L`. These keys say the same thing without committing to a spelling: mcpp renders `libraries` / @@ -425,6 +454,13 @@ for arch/env conditions and combinators. A predicate whose only content is this table is applied like any other. Until mcpp 2026.9.9.1 it was not: the block was parsed and then discarded unless something else appeared under the same predicate. +- **A relative search path in `build.ldflags` belongs to the package that wrote + it.** `-L` and `-Wl,-rpath,` — here, in the top-level + `[build] ldflags`, and from `mcpp::link_flag` — reach the link as absolute + paths under that package's directory, also when a dependency's flags reach + its consumer. An entry that begins with a token the loader expands is passed + as written: `$ORIGIN` and every other `$`-token, and `@executable_path`, + `@loader_path` and `@rpath` (mcpp 2026.9.14.2+). - **What `build` accepts is exactly the set of *additive build inputs*** — the things that combine by appending and are consumed after the predicate is evaluated, which is the member list of `BuildInputs`. `linkage`, `target`, @@ -456,7 +492,8 @@ for arch/env conditions and combinators. build rather than an answer from the graph, so `[target.'cfg(accelerator = "cuda")'.dependencies]` applies. - **Precedence**: an exact-triple table wins over a `cfg`/alias table; multiple - matching predicate tables have their flags concatenated. Conditional entries + matching predicate tables have their flags concatenated, and their dependency + declarations applied in manifest order. Conditional entries are appended **after** the unconditional `[build]` ones, so under GNU "last flag wins" a conditional rule overrides a broader unconditional one. That is what makes a per-OS **removal** expressible: @@ -638,6 +675,37 @@ mcpp interprets; it is an ordinary `[target.'cfg(os = "emscripten")'.build] ldflags` entry, and `main` keeps naming a translation unit regardless — see [21 — The Target Triple](21-the-target-triple.md#the-wasm-artifact-contract). +### `targets. kind` — a library's form on one row (mcpp 2026.9.14.2+) + +```toml +[targets.huxerui] +kind = "lib" + +[target.'cfg(env = "android")'.targets.huxerui] +kind = "shared" +``` + +The per-row form of `[targets.] kind` +([04 — mcpp.toml](04-mcpp-toml.md) §2.2). A framework that is linked into its +application on the desktop rows and must be one shared library on Android +states it once, in its own manifest; every consumer keeps one unconditional +dependency line. + +- `kind` is the only key, and it chooses between the two library forms, `lib` + and `shared`. A name that is not a library target of the package (declared + or inferred), a program target, or another kind is refused. +- On a matching row the package is constrained to the shared form exactly as + `[targets.] kind = "shared"` constrains it (`dependency_linkage` in + [04 — mcpp.toml](04-mcpp-toml.md)): a consumer that writes no `linkage` + receives the shared library, and a consumer that writes `linkage = + "static"` receives a warning naming this line, which `--strict` turns into + an error. `mcpp why deps` reports the form with the reason `row-kind`. +- A selector that names a target-side layer cannot carry it; the table is + reported and ignored, because a library's form is decided while the graph + that answers the layer is resolved. +- An engine before 2026.9.14.2 does not read the table and reports nothing. A + package that relies on it states that engine floor. + ## Current limitations - **A dependency cannot be conditioned on the accelerator.** The accelerator diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 5360e8506..6d02a4ad9 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -243,6 +243,14 @@ both claiming to know how to run the artifact is a configuration error, and mcpp reports it naming both rather than merging them into an argv that is neither one's. +**Within one build program, the tokens are one argv per name.** Every +`mcpp::runner(tok)` call appends to the default runner, and every +`mcpp::runner(name, tok)` call appends to the runner `name`, in emission order, +whichever host module compiled into the program makes the call. A program that +chooses between two runners chooses before it emits either. A runner named +after a pack format reaches that format's distributable under `mcpp run +--format ` ([41 — Reaching a Device](41-devices.md)). + ### Asking instead of declaring: `toolchain_dir` / `sysroot_dir` (2026.8.19.4+) ```cpp @@ -294,6 +302,28 @@ the first one sees. and is empty on a hosted target, which is exactly the case this pair exists for. Either of these two is empty when mcpp passes no such flag. +### The payloads' pkg-config view: `pkg_config_libdir` (2026.9.14.2+) + +```cpp +const char* dirs = mcpp::pkg_config_libdir(); +// /subos/default/usr/lib/pkgconfig:/subos/default/usr/share/pkgconfig +``` + +The pkg-config search path of the payloads mcpp installed, joined with the +platform's path-list separator. A payload recipe declares its `.pc` files into +this view, so a library a payload provides resolves with its whole pkg-config +closure: + +```cpp +std::string cmd = std::string("PKG_CONFIG_LIBDIR=") + mcpp::pkg_config_libdir() + + " pkg-config --cflags --libs gtk4"; +``` + +**An accessor, not an environment default.** A build program's environment +does not carry `PKG_CONFIG_LIBDIR`, so a package that means the host's own +pkg-config database keeps it, and a package that means the payloads states so. +The value does not depend on the link mode or on which toolchain resolved. + ### The resolved C++ standard library: `cxx_stdlib` (2026.9.6.3+) ```cpp @@ -711,14 +741,15 @@ reported as up to date. **The manifest's first line is `closure = walked` or `closure = not-walked`.** `mcpp pack` stages the program and its declared runtime -files before it asks whether this host can resolve the artifact's dependency -closure, so the tree can exist without one — a Mach-O program today, or a -non-PE artifact packed from a Windows host. `--format tar` and `--format dir` -still fail the command in that case, since the archive IS the closure; a -dispatched format receives the tree regardless, with a second manifest line, -`reason = `, naming the mechanism that was unavailable. A provider that -needs the closure reads the field rather than inferring a gap from an empty -`lib/`. +files before it resolves the artifact's dependency closure, so the tree can +exist without a complete one: a closure with a name that resolves to no file +the tree can carry (2026.9.14.2+), or a non-PE, non-Mach-O artifact packed from +a Windows host. `--format tar` and `--format dir` fail the command in that +case, since the archive IS the closure; a dispatched format receives the tree +regardless, with a second manifest line, `reason = `. The `needs` lines +that follow the header state each needed name and what satisfies it +([50 — Machine Output](50-machine-output.md)), so a provider that places +libraries itself reads them rather than inferring the closure from `lib/`. Commands are an **argv, not a shell string** (no shell is assumed — Windows has none to rely on), and the only interpolations are a closed set: diff --git a/docs/32-authoring-a-payload.md b/docs/32-authoring-a-payload.md index db9c7ba21..657862589 100644 --- a/docs/32-authoring-a-payload.md +++ b/docs/32-authoring-a-payload.md @@ -145,6 +145,31 @@ which is what makes a stack of payloads resolve without anyone setting the SubOS sysroot view. **Declared rather than copied** — xlings removes them with the package, and a copy would outlive its owner. +## Resolving a recipe from a checkout (2026.9.14.2+) + +A recipe on an unmerged branch is resolved by a consumer through an index +override in the mcpp home's `config.toml`, before the recipe is published: + +```toml +# $MCPP_HOME/config.toml +[index.repos.xim] +url = "/work/xim-pkgindex" # a checkout of the branch +``` + +The next command writes the entry into the registry's `.xlings.json` and +prints one line; the index then follows the checkout, including commits made +after the first build: + +``` + Index xim -> /work/xim-pkgindex ([index.repos.xim] in config.toml) +``` + +Installations from that index repeat the line beside `Provisioning`. Removing +the table restores the registry's previous entry on the next command, unless +the entry was changed after mcpp wrote it. Both hold for a home that has +already run; an mcpp before 2026.9.14.2 applies the table only when the home +is created. + ## The tier: the commands that need a payload ```toml diff --git a/docs/41-devices.md b/docs/41-devices.md index 5dedf1ae1..9328f7c85 100644 --- a/docs/41-devices.md +++ b/docs/41-devices.md @@ -94,6 +94,54 @@ a dependency supplied, and the override is reported rather than applied in silence. Exactly one dependency may supply a given name; a second is an error naming both packages. +## What a runner receives + +A runner receives an argv — its own tokens, with the artifact's path appended +or substituted for `{}` — and one environment variable, `MCPP_RUNTIME_FILES`. +The variable is set for every runner that `mcpp run` and `mcpp test` start, +the default and the named ones alike. `mcpp run --no-runner` starts no runner +and sets nothing. + +`MCPP_RUNTIME_FILES` names a file that lists what the artifact reads or loads +from its own directory: the `[runtime] deploy` and `deploy_files` entries, and +the shared libraries the build links. One line per file: + +``` +data/data.txt/abs/project/target///bin/data/data.txt +libfw.so/abs/project/target///bin/libfw.so +``` + +The first field is the destination relative to the artifact's directory, with +`/` separators; it begins with `../` for a test program discovered in a +subdirectory. The second is the absolute path of the file in the output tree. A +TAB separates them. The file exists for every runner invocation and is empty +when there is nothing to carry; a distributable run with `--format` receives +an empty file, because the distributable holds its own files. + +A runner that executes the artifact where it lies ignores the variable. A +runner that moves the artifact — to a device, a container, a remote host — +copies each listed file to the destination beside the moved artifact. + +## Running a distributable + +`mcpp run --format ` packs the distributable `` and runs it. Without +`--runner`, it is reached by the runner named `` when the graph or the +manifest supplies one, and by the default runner otherwise: + +```cpp +mcpp::runner("app", ""); // a package names a runner after its format +``` + +```bash +mcpp run --format app # through the runner named `app` +mcpp run --format app --runner other # a typed --runner still wins +``` + +A distributable that is a directory, such as an application bundle, is not +executed directly. When no runner reaches it, `mcpp run --format ` refuses +before starting anything, with exit status `126`, and names the runner `` +and the manifest table that would declare it. + ## Termination is declared, not inferred | | Meaning | diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 3387159b3..a9c79d93d 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -193,7 +193,7 @@ and only the first belongs to the program: | range | meaning | |---|---| | `0`–`124` | the program ran; this is its own status, passed through unchanged | -| `125`–`127` | the spawn was attempted and refused — `127` not found, `126` found but not executable, `125` anything else | +| `125`–`127` | the spawn was attempted and refused — `127` not found, `126` found but not executable, `125` anything else; `126` also answers `mcpp run --format ` for a distributable that is a directory and meets no runner, refused before the spawn with the same meaning (2026.9.14.2+) | | `2` | mcpp refused before attempting anything: a usage, configuration or resolution error | Until 2026.9.4.3 every non-zero status was folded to `1`, so that `2` could mean @@ -454,6 +454,29 @@ must also read `not_run`. existing `not_run` list, which continues to name members the `--workspace-timeout` stopped before they started. +### The stage manifest + +`mcpp pack` writes `.stage-manifest` beside the tree it stages, +and an `artifact` action that names `${mcpp.stage_dir}` depends on it (see +[30](30-build-mcpp.md#producing-a-distributable-pack_format--stage_dir-20269111)). +It is a line-oriented text file rather than a JSON envelope, and its lines form +three blocks in this order: + +| Line | Content | +|---|---| +| `closure = walked` or `closure = not-walked` | the first line: whether every library the tree needs was resolved | +| `reason = ` | only with `closure = not-walked`; one line | +| `needs` *(mcpp 2026.9.14.2+)* | one line per library name the closure read, sorted; `` is the path of the staged library relative to the tree, `platform` for a library the target provides, or `unresolved` | +| ` ` or `link ` | one line per staged file or symbolic link, sorted | + +`` is spelled as the needing object spells it: a `DT_NEEDED` entry, a PE +import name, or a Mach-O install name. The fields of a `needs` line are +separated by TAB characters, because a name and a path may each contain a space. +A tree staged by a mode that bundles nothing (`system`, `static`) carries no +`needs` lines, and neither does a tree staged by an earlier mcpp; a reader that +places libraries itself reads these lines rather than inferring the closure from +the files under `lib/`. + ## Current limitations - **The exit-code table is scoped to the commands it names.** A code another diff --git a/docs/README.md b/docs/README.md index cb1ecc7fa..b54e8c9e6 100644 --- a/docs/README.md +++ b/docs/README.md @@ -126,6 +126,8 @@ token in front of a reader to the chapter that owns it. | `[package] platforms`, `[build] cache` | [04](04-mcpp-toml.md) | `[targets.]`, `[profile.]` | [04](04-mcpp-toml.md) | | `runner`, `[target..runners]` | [41](41-devices.md) | `rule_module` | [31](31-authoring-a-rule-package.md) | | `min_api_level` | [04](04-mcpp-toml.md) | `macos_deployment_target` | [04](04-mcpp-toml.md) | +| `[target..targets.] kind` | [22](22-target-side.md) | `[index.repos.]` in `config.toml` | [32](32-authoring-a-payload.md) | +| `[test] discover` | [04](04-mcpp-toml.md), [08](08-testing.md) | `MCPP_RUNTIME_FILES` | [41](41-devices.md) | **Commands** diff --git a/docs/specs/manifest-semantics.md b/docs/specs/manifest-semantics.md index fcc9f411e..ecbcaa98e 100644 --- a/docs/specs/manifest-semantics.md +++ b/docs/specs/manifest-semantics.md @@ -5,8 +5,8 @@ | **规范编号** | SPEC-004 | | **标题** | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | | **状态** | **草案(Draft)** | -| **版本** | 1.1 | -| **最后修改** | 2026-09-07 | +| **版本** | 1.3 | +| **最后修改** | 2026-09-14 | | **最低实现版本** | 条件化形状:mcpp **2026.8.29.1**(`[target..build-dependencies]` 起齐备);目标轴:mcpp **2026.9.6.4** | | **作者/维护** | mcpp-community | | **相关设计文档** | `.agents/docs/2026-09-07-mcpp-toml-unified-semantics-design.md`
`.agents/docs/2026-06-04-manifest-schema-ownership.md`
`.agents/docs/2026-09-03-xlings-workspace-as-the-one-table.md` | @@ -74,10 +74,29 @@ Principle)规定,本规范不重复它,只在 §6 引用并补充一条。 (`[xlings.workspace.linux]`)或值的兄弟键。 今天接受 `
` 为:`build`、`dependencies`、`dev-dependencies`、 -`build-dependencies`、`feature-deps.`、`runtime`、`xlings`、`feature-xlings.`。 +`build-dependencies`、`feature-deps.`、`runtime`、`xlings`、`feature-xlings.`、 +`targets.`(mcpp 2026.9.14.2+),以及 `abi`、`requires_abi`、`feature-requires-abi`、 +`runners`。实现不读取的 `
` **必须**报出,不得静默忽略(mcpp 2026.9.14.2+)。 **状态:已实现**(上列 section)。 +### 3.1.1 条件声明替换同一身份的无条件声明 + +在 `` 命中的行上,`[target..dependencies]` 中某个身份的声明 +**替换**该身份在 `[dependencies]` 中的声明;身份按键规范化后的 `(namespace, name)` +比较,而非按键的字面。多个命中的 section 按清单顺序生效,后者替换前者。同一规则适用于 +`dev-dependencies`、`build-dependencies` 与 `feature-deps.`。这与条件化标量 +「最后一个命中者为准」是同一条规则;可叠加的构建输入(`build`)仍按追加合并。 + +只写选项而不写来源(`path`/`version`/`git`/`workspace`)的条件表不是对既有依赖的 +修饰,按文法它声明的是另一个包;实现**必须**把这种写法报出,并给出补全来源后的声明。 + +`[target..targets.] kind` 是 `[targets.] kind` 的按行形式: +只接受库目标,只在 `lib` 与 `shared` 之间选择,在命中的行上约束该包的链接形态, +与无条件的 `kind = "shared"` 相同。 + +**状态:已实现**(mcpp 2026.9.14.2)。 + ### 3.2 门可以嵌进条件 `[target..feature-deps.]` 合法:条件决定这个门**拉进什么**,不决定 @@ -291,6 +310,11 @@ feature-deps feature-xlings ← 限定词是门 7. §4.5 的拒绝判据**必须**带反向腿:把钉抬到满足要求后同一份工程构建通过。否则一个 「凡工程与依赖同时声明同一个包就拒绝」的实现也会通过 (`tests/e2e/628_a_pin_below_a_stated_floor_is_refused.sh`)。 +8. §3.1.1 的判据:同一身份在无条件表与命中的条件表中各声明一次、只有条件表写 + `linkage = "shared"` 时,命中的行链接共享库,不命中的行静态链接,解析记录给出 + 声明所在的表(`tests/e2e/677_a_conditional_dependency_replaces_the_unconditional_one.sh`); + 按行 `kind` 在命中行上给出共享库与原因 `row-kind`,不命中行为 `default` + (`tests/e2e/678_a_row_states_a_library_form.sh`)。 ## 变更记录 @@ -299,3 +323,4 @@ feature-deps feature-xlings ← 限定词是门 | 1.0 | 2026-09-07 | 首版。平面(§2)、条件化唯一形状(§3)、两条解析轴(§4)、命名规约(§5)、条件化准入(§6)。目标轴列为未实现。 | | 1.1 | 2026-09-07 | 目标轴落地(mcpp 2026.9.6.4):§4.3.1 工具 selector 禁止命名目标侧层;`[target..xlings…]` 与 `[target..feature-xlings.]` 转为已实现;§4.3 补两条轴同时命名一个包时的取舍与按包去重;§4.4 转为已实现;§7 补第 4 条判据。 | | 1.2 | 2026-09-07 | 一个包一个版本(mcpp 2026.9.6.6):新增 §4.5(身份=`(namespace, name)`,版本是约束;裁决与校验两步;范围必须双向可解且可被 `xpkg_dir` 回答);§4.3.1 改为「禁止命名**被解析的**层」,`accelerator` 明确被接受(2026.9.6.5);§7 补第 5 条的反向腿与第 6、7 条判据。 | +| 1.3 | 2026-09-14 | 条件依赖声明替换同一身份的无条件声明,`targets.` 成为可条件化的 section,不读取的 section 必须报出(mcpp 2026.9.14.2):新增 §3.1.1 与 §7 第 8 条判据。 | diff --git a/docs/specs/package-identity.md b/docs/specs/package-identity.md index 0da69892e..30b013ab7 100644 --- a/docs/specs/package-identity.md +++ b/docs/specs/package-identity.md @@ -5,8 +5,8 @@ | **规范编号** | SPEC-001 | | **标题** | 包身份(`package.namespace` / `package.name`)、`[dependencies]` 选择器与匹配机制 | | **状态** | **评审中(Review)** —— 已实现 | -| **版本** | 1.2 | -| **最后修改** | 2026-08-09 | +| **版本** | 1.4 | +| **最后修改** | 2026-09-14 | | **最低实现版本** | 描述符身份:mcpp **0.0.106**;精确 selector:mcpp **2026.8.10.1**(xlings >= 0.4.69) | | **作者/维护** | mcpp-community | | **相关设计文档** | `.agents/docs/2026-06-20-package-resolution-architecture.md` §4
`.agents/docs/2026-06-26-identity-first-resolution-no-filename.md`
`.agents/docs/2026-07-25-issue278-descriptor-name-form-canonicalization-design.md`
`.agents/docs/2026-07-25-name-namespace-bidirectional-verification-report.md`
`.agents/docs/2026-07-25-name-namespace-canonical-implementation-spec.md` | @@ -298,6 +298,25 @@ warning，规范列举命令是 `mcpp new --list-templates pkg`。 **已实现**(0.0.105)。此前空命名空间会流入 lockfile 与安装层。 +### 5.4 `path` 与 `git` 依赖的身份取自其清单 + +`path` 与 `git` 依赖没有描述符查找,来源由声明行直接给定。它的身份**必须**是其清单 +`[package]` 声明的 `(namespace, name)`,与指向它的键无关;键规范化后的身份与之不同时, +实现**必须**采用清单声明的身份,并对每条声明边给出一次告警,点名请求方、键、键规范化 +后的身份与清单声明的身份。同一来源(规范化后的目录,或仓库与引用)经不同的键到达时 +**必须**解析为同一个包。 + +采用而非拒绝:来源已由声明行固定,§4.2 反对候选搜索的理由(索引状态改变依赖的指向) +在这里不成立。 + +清单未声明 `namespace` 时,身份的命名空间取键的命名空间(§5.3 的兼容边界)。此时同一 +来源经两个命名空间不同的键到达,就是同一来源上的两个身份;实现**必须**在扫描源码之前 +拒绝,点名二者。 + +`version` 依赖不受本条影响:它的身份就是键,由 §5.2 校验。 + +**已实现**(mcpp 2026.9.14.2)。判据:`tests/e2e/679_a_path_dependency_takes_its_manifests_identity.sh`。 + --- ## 6. 派生量 @@ -456,6 +475,7 @@ lua = "0.0.3" | 版本 | 日期 | 变更 | |---|---|---| +| 1.4 | 2026-09-14 | 新增 §5.4:`path` 与 `git` 依赖的身份取自其清单,键规范化到另一身份时采用清单声明并告警,同一来源上的两个身份在扫描前拒绝(mcpp 2026.9.14.2) | | 1.3 | 2026-09-07 | 新增 §10:同一条身份规则扩展到 xlings 工具地址(mcpp 2026.9.6.6)。此前工具侧有两套定义,同一个包被两处以不同版本声明时两份都装、只用一份 | | 1.2 | 2026-08-09 | selector 收敛为唯一精确 PackageId:裸名只表示默认 mcpplibs,dotted 以最后一段为 name;移除 compat/空 namespace 隐式候选,加入 lock 保持与一个 release train 的双 selector 迁移 warning | | 1.1 | 2026-08-03 | 按当前实现复核：澄清文件名发现是快路径加身份回退扫描，修正 legacy `package.name` 的 wire key 示例，并将 0.0.106 明确为最低实现版本 | diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 1ac7b4fa8..a1d5177a5 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -139,7 +139,9 @@ soname = "libmylib.so.1" # 可选: Linux/ELF ABI 名称,运行时会生成同 `SOVERSION`/`SONAME`。在 Linux 上,mcpp 会向链接器传递 `-Wl,-soname,`,并在输出目录生成 ` -> lib.so` alias, 让下游程序可通过标准 ABI 名称 `DT_NEEDED` 或 `dlopen()` 加载该库。 -该字段只对 `kind = "shared"` 有效,值必须是文件名 basename。 +该字段只对 `kind = "shared"` 有效,值必须是文件名 basename。未声明 `soname` +的 ELF 共享库以输出文件名作为 SONAME(2026.9.14.2+),这正是消费者已经记录在 +`DT_NEEDED` 中的名字;bionic 自 API level 23 起要求共享库带有 SONAME。 共享库目标在三种二进制格式上都可用。ELF 产出带 `soname` 的 `.so` 与 `$ORIGIN` 搜索路径;Mach-O 产出 install name 为 `@rpath/` 的 `.dylib`,因此移动后 @@ -392,6 +394,7 @@ dependency_linkage = "shared" # 按 profile 覆盖 | 包写了 | mcpp 读作 | |---|---| | `[targets.] kind = "shared"` | *必须* shared —— 进程里会有别人 `dlopen` 它,因此只能有一份(X11、Vulkan loader) | +| `[target..targets.] kind = "shared"` *(2026.9.14.2+)* | 在选择器命中的行上*必须* shared,其余行两种形态都可以([22 —— 目标侧](22-target-side.md)) | | `ldflags` 里含 `-L` | *必须* static —— 包携带了 mcpp 没有编译的预构建归档,放不进 mcpp 自己构建的共享对象 | | 分发包(`mcpp pack`) | 它实际随包的那些腿,取自 `[[runtime.artifacts]] role` | | 其他 | 两种形态都可以 | @@ -399,6 +402,12 @@ dependency_linkage = "shared" # 按 profile 覆盖 `kind = "lib"` **不是**约束:它是默认值,大多数包写下它并没有做任何选择。 **没有陈述不等于一条陈述。** +约束拒绝的请求按包允许的形态链接,并给出一条点名包的陈述的警告 +(`its manifest states [targets.fw] kind = "shared", ...`);`--strict` 下该警告 +成为错误。`mcpp why deps` 报告每个依赖的形态及其原因:`default`、`requested`、 +`package-kind`、`row-kind`、`packaged`、`no-sources`、`prebuilt-inputs`、 +`no-loader` 或 `static-libc`(2026.9.14.2+)。 + 依赖边上的 `linkage` 只在**根工程**的 `[dependencies]` 里生效。依赖图深处的包 无权决定最终程序的布局;真正必须只有一份共享副本的包,应当在自己的 target 上 声明。 @@ -1236,6 +1245,22 @@ o.arg("./mkblob.sh").arg("blob.bin").arg("${mcpp.out_dir}/blob.o") 已移入 [09 —— 按场景选命令](09-commands-by-scenario.md)。 +### 2.17 `[test]` —— 测试程序的位置 + +```toml +[test] +discover = ["tests/**/*.cpp"] # 默认值 +``` + +| 键 | 类型 | 含义 | +|---|---|---| +| `discover` | glob 数组 | glob 匹配到的每个文件都是一个测试程序;以 `!` 开头的 glob 去掉它匹配到的文件;`[]` 不发现任何测试 | + +glob 使用 `[build] sources` 的词汇。测试的名字是它相对于第一个匹配它的 glob 的固定 +目录的路径,去掉扩展名。同名的两个文件会被拒绝,并点名两者。值不是非空字符串数组时 +报错;`[test]` 中的其他键给出警告,在 `--strict` 下为错误。测试模型见 +[08 —— 测试](08-testing.md)。 + ## 3. 实战示例 @@ -1296,7 +1321,7 @@ kind = "bin" | C 标准 | `c11` | `.c` 文件自动走 C 编译器 | | 静态 stdlib | `true` | 便携二进制 | | 头文件 | `include/`(如果存在） | 自动加到 `-I` | -| 测试 | `tests/**/*.cpp` | `mcpp test` 自动发现 | +| 测试 | `tests/**/*.cpp` | `mcpp test` 自动发现;`[test] discover` 替换这个集合 | | 依赖命名空间 | `mcpplibs`(默认) | 裸 selector 只表示该精确 ns | ### 4.1 旧 `[language]` 兼容层 diff --git a/docs/zh/05-dependencies.md b/docs/zh/05-dependencies.md index e02141945..c8363d795 100644 --- a/docs/zh/05-dependencies.md +++ b/docs/zh/05-dependencies.md @@ -127,6 +127,29 @@ qux = ">=1.0, <2.0" # 范围组合 突、且都不是根的情形,绝不会靠猜哪个先被声明来解决 —— 那正是本节要替换掉的 "队列顺序的意外"。 +### `path` 与 `git` 依赖的身份(mcpp 2026.9.14.2+) + +`path` 或 `git` 依赖就是它的清单所声明的那个包,与指向它的键无关。一个键规范化后的 +身份若不同于清单 `[package]` 的 `namespace` 与 `name`,就采用清单声明的身份;mcpp +对每条声明边告警一次,点名请求方、键、键所指的身份与清单声明的身份: + +```toml +# comp/mcpp.toml; fw/mcpp.toml declares namespace = "huxdemo" +[dependencies] +fw = { path = "../fw" } # names mcpplibs.fw; huxdemo.fw is used +``` + +``` +warning: 'huxdemo.comp@path' declares the dependency 'fw', which names mcpplibs.fw; the manifest '.../fw/mcpp.toml' declares huxdemo.fw, and that identity is used. + hint: write 'huxdemo.fw' in 'huxdemo.comp@path' to state the identity the manifest declares. +``` + +因此同一目录上分别写作 `fw` 与 `huxdemo.fw` 的两条边是同一个包,只编译一次, +`mcpp why deps` 在它下面列出两个键。未声明命名空间的清单取键的命名空间,于是在这样 +一个目录上用两个不同命名空间的键,就是同一来源上的两个身份:构建在扫描之前被拒绝, +点名二者;修正方式是在该清单中声明 `namespace`,或两处写同一个键。`version` 依赖 +不受影响,它的身份就是键。 + ### 命名空间解析规则 每个包的身份是**命名空间 + 名字**二元组。每个 selector 都只规范化成一个身份: diff --git a/docs/zh/06-features-and-capabilities.md b/docs/zh/06-features-and-capabilities.md index b945cf30a..21360dc29 100644 --- a/docs/zh/06-features-and-capabilities.md +++ b/docs/zh/06-features-and-capabilities.md @@ -273,7 +273,7 @@ mcpp 在绑定 capability 时比较二者,并在任何东西被编译之前拒 报 `version-floor-unmet`: ``` -error: `toolkitnew` requires cuda.driver >= 13.0, and this machine has 12.4. +error: `toolkitnew` requires cuda.driver >= 13.0, and cuda.driver is stated as 12.4. stated by: driverfact ``` @@ -284,6 +284,30 @@ error: `toolkitnew` requires cuda.driver >= 13.0, and this machine has 12.4. 而是「没人问过」的机器。把「我们不知道」变成「不行」正是这个机制要避免的失败, 并且有直接判据:`tests/e2e/603_version_floor.sh` 会构建一个下界指向无人提供之物的工程。 +**引擎陈述目标的平台下限**(mcpp 2026.9.14.2+)。在编译器接受最低平台版本的行上, +引擎以平台自己的术语把该版本陈述为一项事实,包像对待其他事实一样对它写下界: + +| 事实 | 行 | 设定来源 | +|---|---|---| +| `android.api-level` | `*-linux-android` | `[target.] min_api_level`,否则为工具链支持的最低级别 | +| `ios.deployment-target` | iOS 真机与模拟器各行 | `[build] ios_deployment_target`,否则为定位到的 SDK 版本 | +| `macos.deployment-target` | macOS 各行 | `[build] macos_deployment_target`,否则为 mcpp 的 macOS 默认值 | + +```toml +[[runtime.requirements]] +kind = "version-floor" +value = "android.api-level >= 23" +``` + +``` +error: `fw` requires android.api-level >= 23, and this build targets 21. + set by: [target.x86_64-linux-android] min_api_level +``` + +不陈述这类事实的行上,该要求保持沉默,因此要求本身不需要选择器。下限不会替依赖 +抬高:键设定的值就是编译器面向的值,应用安装到哪些设备上由应用决定。包陈述同名 +事实不会替换引擎陈述的那一项。 + **它是关于这个包自己的符号的声明**,所以一条指向本包并不提供的能力的条目会被报为 schema 警告:那里没有可独占的东西。而无人声明独占的能力行为完全不变 —— 两个 BLAS 实现照常共存,既有的「两个或更多、未 pin」报错也仍然只在**有人 require** 该能力时出现。 diff --git a/docs/zh/07-workspace.md b/docs/zh/07-workspace.md index 45f4652f7..449f47107 100644 --- a/docs/zh/07-workspace.md +++ b/docs/zh/07-workspace.md @@ -66,7 +66,7 @@ name = "myapp" version = "0.1.0" [dependencies] -core = { path = "libs/core" } +myproject.core = { path = "libs/core" } ``` ### 2.3 成员工程文件 @@ -94,7 +94,7 @@ name = "http" version = "0.1.0" [dependencies] -core = { path = "../core" } +myproject.core = { path = "../core" } [dependencies.compat] mbedtls.workspace = true diff --git a/docs/zh/08-testing.md b/docs/zh/08-testing.md index 8d44470af..520edd95a 100644 --- a/docs/zh/08-testing.md +++ b/docs/zh/08-testing.md @@ -33,6 +33,24 @@ myproject/ `mcpp new` 会生成 `tests/test_smoke.cpp`,让工程一开始就有这个目录。 +### 测试的位置 + +`tests/**/*.cpp` 是一个键的默认值: + +```toml +[test] +discover = ["checks/**/*.cpp", "!checks/fixtures/**"] +``` + +`discover` 接受与 `[build] sources` 同一套词汇的 glob:glob 匹配到的每个文件都是一个 +测试程序,以 `!` 开头的 glob 把它匹配到的文件从集合中去掉,无论是哪个 glob 找到的。 +测试的名字是它相对于第一个匹配它的 glob 的固定目录的路径,去掉扩展名,因此默认值 +把 `tests/unit/test_span.cpp` 命名为 `unit/test_span`。`discover = []` 不发现任何测试。 +名字相同的两个文件会被拒绝,并点名两者。 + +由多个源文件编译成的测试套件是一个独立的包:一个工作区成员,它的 `[build] sources` +承载套件,它唯一的测试程序驱动套件,用 `mcpp test -p ` 选中。 + ## 运行它们 ```bash @@ -52,6 +70,7 @@ mcpp test -- --verbose # `--` 之后的一切传给每个测试程序 | `--target ` | 宿主以外的目标 | | `--accel ` / `--no-accel` | 本次构建面向的设备后端 | | `--cap ` | 钉住某个能力的 provider | +| `--toolchain ` | 本次调用使用的工具链,例如 `llvm@22.1.8` | `--timeout ` 杀掉仍在运行的测试(默认 300;`0` 关闭),`--build-timeout ` 限制编译。一个挂住的测试被报为**以它自己的名字失败**,而不是一个停下来的任务。 @@ -84,6 +103,11 @@ mcpp test --no-runner # 忽略 runner,直接执行 `--no-runner` 是给「本机就能原生执行这些二进制、不该为模拟器付代价」的宿主准备的。 +测试程序把它读取的文件带在身边:runner 收到 `MCPP_RUNTIME_FILES`,即它部署的文件与 +它加载的共享库的清单,把程序移到设备上的 runner 连同这些文件一起复制。测试按相对于 +自身所在目录的路径定位这类文件。在 Android 行上,除非 `cxx_runtime` 另有声明,测试 +程序静态链接 C++ 运行时,因此设备上不需要 `libc++_shared.so`。 + runner 本身、具名 runner,以及一块板子声明什么,见 [41 —— 抵达一台设备](41-devices.md)。 @@ -110,6 +134,8 @@ mcpp test --message-format json - 一个测试是一个 `.cpp` 产出一个程序。mcpp 不发现文件内部的用例,因此框架的 逐用例选择发生在程序内部,经由 `--` 之后的参数。 +- 清单无法加载时,`mcpp test --list` 列出的是 `tests/**/*.cpp`,而不是它读不到的 + `[test] discover` 集合。 - `--build-timeout` 只在 POSIX 上有效。 - `--workspace-timeout` 限制 `--workspace` 的扇出并报告跑到了哪些;它不把超时 归因到某个成员。 diff --git a/docs/zh/09-commands-by-scenario.md b/docs/zh/09-commands-by-scenario.md index 0de5ae288..8a7cc9cb7 100644 --- a/docs/zh/09-commands-by-scenario.md +++ b/docs/zh/09-commands-by-scenario.md @@ -90,6 +90,21 @@ $ mcpp search imgui 这次扫描只在查找已经失败之后进行,结果只进入错误文本与 search 输出。裸名不会因此跨 命名空间解析。 +## 为一次调用换一个工具链 + +`mcpp build`、`mcpp run`、`mcpp test` 与 `mcpp pack` 接受 `--toolchain `, +它为这一次调用选择编译器,不写入任何东西: + +```bash +mcpp test --toolchain llvm@22.1.8 +mcpp run --toolchain gcc@16.1.0 +mcpp pack --toolchain llvm@22.1.8 --format dir +``` + +对这一次调用,这个选项取代 `mcpp.toml` 中的 `[toolchain] default`,其优先级即 +[20 —— 工具链管理](20-toolchains.md) 给 `MCPP_TOOLCHAIN` 的那一级。每个工具链构建到 +它自己的输出目录,一次记录下来的构建只为记录它的那个工具链请求重放。 + ## 解释一次解析 `mcpp why` 报告一次构建会解析出什么,并且不构建任何东西: @@ -101,6 +116,24 @@ toolchain: gcc 16.1.0 (x86_64-linux-gnu) reason: [toolchain] in mcpp.toml if set, else platform-native default ``` +`mcpp why deps` 在 `mcpp.lock` 的各行之前列出解析出的依赖图(2026.9.14.2+):每个包、 +每条请求书写时用的键与所在的表,以及库的链接形态与其原因。锁文件不记录的 `path` +依赖同样列出: + +``` +$ mcpp why deps +dependency graph: + mcpplibs.app@0.1.0 (root) path+/work/app + huxdemo.fw@0.1.0 path+/work/fw + requested by mcpplibs.app@0.1.0 as 'huxdemo.fw' in [dependencies] + requested by huxdemo.comp@0.1.0 as 'fw' in [dependencies] + linked static (default) +``` + +同一张图记录在 `target///resolution.json` 的 `graph` 下,每个包一条, +根在最前:`package`(规范身份、命名空间、名字、版本、来源)、`root`、 +`requested_by`(`requester`、`key`、`table`),库还有 `link`(`form`、`reason`)。 + 话题是 `toolchain`、`runtime`、`deps` 或 `runners`,不给话题时四者全报。`--target` 与 `--toolchain` 把报告变成对当前目录并不使用的那一对的查询,目标矩阵正是这样逐格提问的。 diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index b5a3b3755..ee1b065f9 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -156,7 +156,8 @@ error: unknown --format 'bogus'. (mcpp 2026.9.13.2+):在每一行 Android 上,一个应用*就是*平台加载的那个共享库, 所以 `mcpp pack myapp --target aarch64-linux-android --target x86_64-linux-android` 会构建并把两条腿暂存进同一棵树里,与库包的多三元组做法完全一致。每条腿落在 -`lib//lib.so`(`aarch64` → `arm64-v8a`,`x86_64` → `x86_64`),声明的 +`lib//lib.so`(`aarch64` → `arm64-v8a`,`x86_64` → `x86_64`),各自的 +闭包暂存在它旁边(见 [Android](#android应用目标文件与它的闭包在-lib-下)),声明的 部署文件只暂存一次,随后对这棵合并后的树只跑一次分派——这正是 `dist-apk` 这样的 成员能构建出一个通用 APK 的原因。只给一个 `--target` 时,今天这种扁平的 `lib/lib.so` 布局保持不变。产物在任何被请求的一行上是可执行文件的 target, @@ -240,6 +241,15 @@ tarball 内容包在一个顶层目录里,该目录的名字与 tarball 文件 `.tar.gz`)保持一致 —— 这样图形界面"右键解压"和命令行 `tar -xzf` 都 得到同一个自包含的目录,不会把内容散到当前路径。 +**闭包会被记录,不完整的闭包使归档被拒绝**(mcpp 2026.9.14.2+)。暂存树旁边的 +`.stage-manifest` 列出闭包解析过的每一个库名:树里携带的库给出其暂存路径, +目标自己提供的库记为 `platform`,哪里都找不到的记为 `unresolved`(行格式见 +[50](50-machine-output.md#暂存清单))。只要有一个名字是 unresolved, +`--format tar` 与 `--format dir` 就失败并点名它,因为缺了这个库的树在安装处无法启动; +被分派的格式仍然收到这棵树,并带 `closure = not-walked`。目标提供、而本机没有的库, +写进 `[pack.bundle-project] also_skip`。不打包任何东西的模式(`system`、`static`) +不记录闭包。 + ### Mode `static` ``` @@ -402,33 +412,61 @@ mcpp pack --target x86_64-windows-gnu # 在 Linux 宿主上 - 压缩包是**确定性**的:不读取任何时间戳,同一棵树打两次字节一致,公布的校验和 才有意义。 -反方向 —— 在 Windows 上给 Linux / macOS 产物打包 —— 仍然不支持,原因还是最初 -那个:那条闭包要由目标自己的动态链接器解析,而 Windows 宿主没有办法运行它。 +反方向 —— 在 Windows 上给 Linux 产物打包 —— 仍然不支持:ELF 程序的闭包要由目标 +自己的动态链接器解析,而 Windows 宿主没有办法运行它。Mach-O 程序的闭包是从文件里 +读出来的,所以 macOS 程序在任何宿主上都能打包。 + +### macOS(Mach-O):dylib 与程序同目录 + +``` +target/dist/myapp-0.1.0-aarch64-macos.tar.gz +└── myapp-0.1.0-aarch64-macos/ + ├── bin/myapp + ├── bin/libmydep.dylib ← 依赖的 dylib,与程序同目录 + ├── myapp ← 入口 wrapper + ├── README.md + └── LICENSE +``` + +Mach-O 程序的闭包从它的 load command 里读出(mcpp 2026.9.14.2+),所以打包时不运行 +任何东西。它的 dylib 暂存在它旁边的 `bin/` 中,而图内构建的 dylib 的消费者链接时带的 +`@loader_path` rpath 本来就指向那里:不编辑任何 load command,不重新签名,也不 strip +这个程序。 -#### Mach-O 程序会被拒绝 —— 在所有宿主上,包括 macOS +一个被需要的名字,在加载器能把它解析到程序所在目录时才会被暂存: + +| 名字 | 处理 | +|---|---| +| `/usr/lib/…`、`/System/Library/…` | 不暂存;由操作系统提供 | +| `@rpath/` | 需要它的映像或程序本身有一条恰为 `@loader_path` 或 `@executable_path` 的 rpath 时暂存 | +| `@loader_path/`、`@executable_path/` | 暂存到该路径,该路径必须留在程序所在目录之内 | -同一步闭包解析是靠 `LD_TRACE_LOADED_OBJECTS=1` **运行产物**来问动态链接器要 -依赖表的。这个变量属于 glibc 的 ld.so,dyld 从来不认(它的对应物是 -`DYLD_PRINT_LIBRARIES`)。所以在 Mac 上这条命令不会 trace 任何东西 —— -**它会把用户的程序跑起来**,然后把程序的输出当成依赖表解析。mcpp 现在直接拒绝, -并在信息里点名缺的是哪个机制。 +其他名字都使闭包不完整,`tar` 与 `dir` 拒绝并点名它。最常见的是操作系统目录之外的 +绝对 install name:加载器在每台机器上都从那个路径读取它,不论树里带了什么。 +`--mode system` 与 `--mode static` 在这一行上不打包任何东西,与 PE 相同。 +`kind = "lib"` / `"shared"` 目标走库打包路线,不受影响。 -判定按产物的**格式**而不是宿主,理由与 Windows 那条完全相同: -`LD_TRACE_LOADED_OBJECTS` 在 Linux 上也 trace 不了一个 Mach-O。 +### Android:应用目标文件与它的闭包在 `lib/` 下 + +``` +target/dist/myapp-0.1.0-x86_64-linux-android/ +├── lib/libmyapp.so ← 应用目标文件 +├── lib/libmydep.so ← 依赖的共享库 +├── lib/libc++_shared.so ← NDK 的 C++ 运行时,目标文件需要它时才有 +└── bin//… ← `[runtime] deploy` 放置的文件 +``` -`kind = "lib"` / `"shared"` 目标在 macOS 上照常打包 —— 库打包从不运行产物。 -这条限制只针对程序。 +在 Android 行上,应用是暂存在 `lib/` 下的共享目标文件,它的闭包从文件里读出并暂存在 +它旁边(mcpp 2026.9.14.2+)。设备提供的名字不暂存:即出现在该 API level 桩目录中的 +名字,这个目录由 mcpp 向这一行自己的编译器询问得到(编译器找到 `libc.so` 的目录)—— +`libc.so`、`libm.so`、`libdl.so`、`liblog.so` 以及其他平台库。其余每个名字都必须在 +链接用过的目录中解析到:构建的输出目录、图中各包的 `[runtime] library_dirs` 与 +`link_library_dirs`,以及编译器的库搜索路径(`libc++_shared.so` 就在其中)。 +哪里都解析不到的名字使 `tar` 与 `dir` 拒绝,并点名它和搜索过的目录。 -依赖闭包现在可以被**读出来**而不必运行了 —— `mcpp.pack.binfmt` 走一遍 Mach-O 的 -load command(`LC_LOAD_DYLIB` 及其 weak/re-export/upward 三个变体给出名字, -`LC_RPATH` 给出搜索项),读法与它读 PE 导入表一样;并按 `dyld` 的规则解析 -`@executable_path`、`@loader_path`、`@rpath`,全程不加载任何东西。`mcpp pack` -还没有调用它:把解析到的 dylib 拷到程序旁边、再重写它的 `LC_RPATH`,需要一个 -load command 编辑器(一条 load command 里没有空间可以塞进更长的路径),这个编辑器 -要等在真实的 macOS 构建上量过解析结果之后才设计,而不是先设计。在那之前,Mach-O -程序仍然是"暂存但没有闭包"——一个被分发出去的格式(`.app`、`.ipa`)已经能拿这样一棵 -"有程序、没闭包"的树做什么,见 -[产出可分发物](30-build-mcpp.md#产出可分发物pack_format-与-stage_dir20269111)。 +给出多个 `--target` 三元组时,每条腿暂存在 `lib//` 下,各带自己的闭包。 +`--mode` 不适用于这一行:设备恰好提供平台库,所以既没有可以留给宿主的库集合, +也没有需要携带的加载器。 ## 配置项 @@ -457,11 +495,7 @@ force_bundle = ["libfoo.so"] # 即使命中 PEP 600 名单也强制打包 ## 待支持 -macOS **程序** bundling 仍在规划中。闭包现在已经能读出来了(`mcpp.pack.binfmt` -走一遍 Mach-O 的 load command,不需要 `otool`),但把解析到的 dylib 拷到程序旁边、 -再重写它的 `LC_RPATH` 还没做;在这落地之前,`mcpp pack <程序>` 仍会在该格式上拒绝, -而不是产出一个只是看起来像 bundle 的东西。当前 `.zip` 之外的 Windows DLL 分发, -同样在规划中。 +当前 `.zip` 之外的 Windows DLL 分发在规划中。 `.deb`、`.rpm`、AppImage、`.msi` 这些分发格式**不在**这份清单上,而这是一个决定而不是 一处遗漏:它们住在包里,经 `--format ` 到达用户,理由见上一节。`[pack]` 的内建 diff --git a/docs/zh/12-binary-distribution.md b/docs/zh/12-binary-distribution.md index 9030f58a9..3f1da9c7d 100644 --- a/docs/zh/12-binary-distribution.md +++ b/docs/zh/12-binary-distribution.md @@ -77,6 +77,11 @@ mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23/ `lib/` 按**三元组**分目录,不按 OS 分:MinGW 与 MSVC 同为 Windows, 一个产 `libfoo.a` 一个产 `foo.lib`。 +**共享**包以库的*两个*名字携带它:消费者链接 `lib.so`,加载器随后按 +`SONAME` 查找。声明了 `soname` 时这是两个不同的文件名,只发构建出的那个文件能链接 +成功、随后启动失败。未声明 `soname` 的库把自己的文件名记为 `SONAME` +(mcpp 2026.9.14.2+),它的两个名字就是同一个文件。 + ### 两个集合不可裁剪的原因 同一个包的**源码**分发会把 `include_dirs` 里的每一个头都放到消费者的 include diff --git a/docs/zh/22-target-side.md b/docs/zh/22-target-side.md index 435ef1ce7..512debdd5 100644 --- a/docs/zh/22-target-side.md +++ b/docs/zh/22-target-side.md @@ -341,7 +341,30 @@ cxxflags = ["-march=x86-64-v2"] `include_dirs` / `include_dirs_after`(mcpp 0.0.102+), 以及 `private_include_dirs` 与 `std-module-flags`(mcpp 2026.9.1.1+), 还有带 `frameworks` / `libraries` / `link_library_dirs` 的 `runtime` - (mcpp 2026.8.29.1+;`frameworks` 自 2026.9.12.3 起)。 + (mcpp 2026.8.29.1+;`frameworks` 自 2026.9.12.3 起),以及带 `kind` 的 + `targets.`(mcpp 2026.9.14.2+;见 + [`targets. kind`](#targetsname-kind--一个库在某一行上的形态mcpp-20269142))。 +- **mcpp 不读取的子表会被报出**(mcpp 2026.9.14.2+):拼错的 + `[target..dependecies]` 是一条警告,列出 `[target.]` 表拥有的各个段, + `--strict` 下成为错误。 +- **条件依赖声明替换无条件声明**(mcpp 2026.9.14.2+)。在选择器命中的行上, + `[target..dependencies]` 中某个身份的声明就是该身份的声明,无条件声明在 + 该行上不生效。某一行上以不同形态链接的依赖写两次,每次都带来源: + + ```toml + [dependencies] + huxerui.huxerui = { version = "0.3.0" } + + [target.'cfg(env = "android")'.dependencies] + huxerui.huxerui = { version = "0.3.0", linkage = "shared" } + ``` + + 同一规则适用于 `dev-dependencies`、`build-dependencies` 与 + `feature-deps.`;多个命中的段按清单顺序生效,最后一个为准。 + `mcpp why deps` 给出每条请求来自哪张表([09 —— 按场景的命令](09-commands-by-scenario.md))。 + 只写选项、不写来源的表 `huxerui.huxerui = { linkage = "shared" }` 声明的是 + 名为 `huxerui.huxerui.linkage` 的包:mcpp 报出这一行并给出补全来源后的声明, + 解析随即失败。2026.9.14.2 之前的引擎保留无条件声明。 - **`runtime` 是链接行中与方言无关的那一半。** `build.ldflags` 按 GNU 拼法书写, 而原生 `cl.exe` 不接受 `-L`。这些键表达同一件事而不承诺拼法:mcpp 把 `libraries` / `link_library_dirs` 渲染成 `-L` + `-l` 或 @@ -361,6 +384,11 @@ cxxflags = ["-march=x86-64-v2"] 一个谓词下只写这一张表时,它与其他情形一样生效。在 mcpp 2026.9.9.1 之前并非 如此:除非同一谓词下还写了别的东西,该块会被解析后丢弃。 +- **`build.ldflags` 中的相对搜索路径属于写下它的包。** `-L` 与 + `-Wl,-rpath,` —— 无论写在这里、写在顶层 `[build] ldflags`,还是来自 + `mcpp::link_flag` —— 以该包目录下的绝对路径进入链接,依赖的 flag 传给消费者时 + 也是如此。以加载器展开的记号开头的项按原样传递:`$ORIGIN` 及其他任何 `$` 记号, + 以及 `@executable_path`、`@loader_path`、`@rpath`(mcpp 2026.9.14.2+)。 - **`build` 接受的恰好是*可叠加的构建输入*集合** —— 那些以追加方式合并、 并在谓词求值之后被消费的东西,也就是 `BuildInputs` 的成员表。`linkage`、`target` 与档案开关刻意不在其中:它们是**目标选择的输入**(用一个针对 `target` 求值的谓词 @@ -382,7 +410,8 @@ cxxflags = ["-march=x86-64-v2"] 会被报出并忽略;同一谓词下的 `build` 输入照常生效。`accelerator` 不在此列 (mcpp 2026.9.6.5):它是构建的输入而不是图给出的答案,所以 `[target.'cfg(accelerator = "cuda")'.dependencies]` 生效。 -- **优先级**:精确三元组表胜过 `cfg`/别名表;多个命中的谓词表,其 flag 按序拼接。 +- **优先级**:精确三元组表胜过 `cfg`/别名表;多个命中的谓词表,其 flag 按序拼接, + 其依赖声明按清单顺序生效。 条件项追加在无条件 `[build]` 项**之后**,因此在 GNU「最后一个 flag 生效」的 规则下,条件规则会覆盖更宽的无条件规则。这正是让按 OS **移除**成为可表达的原因: @@ -535,6 +564,30 @@ schema 清扫会跳过每一个取值为表的键,理由是它假定表就是条 它是一条普通的 `[target.'cfg(os = "emscripten")'.build] ldflags` 条目,`main` 照样只是指出一个翻译单元——见[21 —— 目标三元组](21-the-target-triple.md#wasm-产物契约)。 +### `targets. kind` —— 一个库在某一行上的形态(mcpp 2026.9.14.2+) + +```toml +[targets.huxerui] +kind = "lib" + +[target.'cfg(env = "android")'.targets.huxerui] +kind = "shared" +``` + +`[targets.] kind` 的按行形式([04 —— mcpp.toml](04-mcpp-toml.md) §2.2)。 +在桌面各行上链接进应用、在 Android 上必须是唯一一份共享库的框架,在它自己的清单里 +声明一次;每个消费者都只保留一行无条件依赖。 + +- `kind` 是唯一的键,只在两种库形态 `lib` 与 `shared` 之间选择。不是该包库目标 + (声明的或推断的)的名字、程序目标以及其他形态都被拒绝。 +- 在命中的行上,该包被约束为共享形态,与 `[targets.] kind = "shared"` + 的约束完全相同([04 —— mcpp.toml](04-mcpp-toml.md) 中的 `dependency_linkage`): + 不写 `linkage` 的消费者得到共享库,写 `linkage = "static"` 的消费者得到一条点名 + 这一行的警告,`--strict` 下成为错误。`mcpp why deps` 以原因 `row-kind` 报告该形态。 +- 点名目标侧层的选择器不能承载这张表;该表被报出并忽略,因为库的形态是在解析 + 回答该层的那张图时决定的。 +- 2026.9.14.2 之前的引擎不读取这张表,也不报告。依赖它的包要写明这一引擎下限。 + ## 当前边界 - **依赖不能以加速器为条件。** 加速器这一层是从依赖图解析出来的,因此由它选择的依赖 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index 5f496aa81..10cfbd496 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -213,6 +213,12 @@ for (auto a : {"-machine","virt","-nographic","-no-reboot","-kernel"}) **只能有一个依赖提供 runner。** 两个板级支持包都声称知道怎么跑这个产物是配置 错误;mcpp 会**同时点名两个**并报错,而不是把它们并成一个谁也不是的 argv。 +**在一个构建程序之内,每个名字的 token 组成一个 argv。** 每次 `mcpp::runner(tok)` +调用追加到默认 runner,每次 `mcpp::runner(name, tok)` 调用追加到名为 `name` 的 +runner,按发出顺序,无论调用来自编译进这个程序的哪个 host module。要在两个 runner +之间选择的程序,在发出任何一个之前完成选择。以打包格式命名的 runner 在 +`mcpp run --format ` 下抵达该格式的可分发物([41 —— 抵达一台设备](41-devices.md))。 + ### 问,而不是声明:`toolchain_dir` / `sysroot_dir`(2026.8.19.4+) ```cpp @@ -256,6 +262,25 @@ crt/host_config.h:218: fatal error: features.h: No such file or directory **不是 `sysroot_dir()`。** 那个回答的是目标**档位**的问题,在宿主目标上为空, 而宿主目标恰恰是这一对存在的场合。mcpp 不传某个开关时,对应的那个为空串。 +### 载荷的 pkg-config 视图:`pkg_config_libdir`(2026.9.14.2+) + +```cpp +const char* dirs = mcpp::pkg_config_libdir(); +// /subos/default/usr/lib/pkgconfig:/subos/default/usr/share/pkgconfig +``` + +mcpp 安装的载荷的 pkg-config 搜索路径,以平台的路径列表分隔符连接。载荷配方把自己的 +`.pc` 文件声明进这个视图,因此由载荷提供的库连同它的整个 pkg-config 闭包都能解析: + +```cpp +std::string cmd = std::string("PKG_CONFIG_LIBDIR=") + mcpp::pkg_config_libdir() + + " pkg-config --cflags --libs gtk4"; +``` + +**这是访问器,不是环境默认值。** 构建程序的环境不携带 `PKG_CONFIG_LIBDIR`,因此 +指宿主自己 pkg-config 数据库的包照旧,指载荷的包自己写明。该值与链接模式以及解析出 +哪个工具链无关。 + ### 解析出的 C++ 标准库:`cxx_stdlib`(2026.9.6.3+) ```cpp @@ -607,12 +632,13 @@ mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件,永不是 已是最新。 **这份 manifest 的第一行是 `closure = walked` 或 `closure = not-walked`。** -`mcpp pack` 会先暂存程序本身与它声明过的运行期文件,再去问这台宿主机能不能解析该产物的 -依赖闭包 —— 所以这棵树可以在没有闭包的情况下存在:今天是一个 Mach-O 程序,或者在 -Windows 宿主上打包一个非 PE 产物。`--format tar` 与 `--format dir` 在这种情况下仍然让 -命令失败,因为归档本身就是闭包;一个被分发出去的格式无论如何都会拿到这棵树,manifest 上 -多出第二行 `reason = <原因>`,点名是哪种机制在这台宿主上不可用。需要闭包的提供方读这个 -字段,而不是从一个空的 `lib/` 里去猜测缺口。 +`mcpp pack` 会先暂存程序本身与它声明过的运行期文件,再解析该产物的依赖闭包 —— 所以这棵 +树可以在闭包不完整的情况下存在:闭包里有一个名字解析不到树能携带的文件(2026.9.14.2+), +或者在 Windows 宿主上打包一个既非 PE 也非 Mach-O 的产物。`--format tar` 与 +`--format dir` 在这种情况下让命令失败,因为归档本身就是闭包;一个被分发出去的格式无论如何 +都会拿到这棵树,manifest 上多出第二行 `reason = <原因>`。头部之后的 `needs` 行给出每个 +被需要的名字以及满足它的东西([50 —— 机器输出](50-machine-output.md)),自行放置库的 +提供方读这些行,而不是从 `lib/` 里推断闭包。 命令是 **argv 而不是 shell 字符串**(不假设存在 shell —— Windows 没有能依赖的那个), 插值只有封闭的一组: diff --git a/docs/zh/32-authoring-a-payload.md b/docs/zh/32-authoring-a-payload.md index 58cc4a4ed..02b7f32eb 100644 --- a/docs/zh/32-authoring-a-payload.md +++ b/docs/zh/32-authoring-a-payload.md @@ -127,6 +127,28 @@ exports = { 头文件声明把载荷放进 SubOS 的 sysroot 视图。**是声明而不是复制** —— xlings 会随包 一起移除它们,而一份复制会比它的主人活得更久。 +## 从检出目录解析配方(2026.9.14.2+) + +未合入分支上的配方,在发布之前可由消费者通过 mcpp home 的 `config.toml` 中的索引覆盖 +来解析: + +```toml +# $MCPP_HOME/config.toml +[index.repos.xim] +url = "/work/xim-pkgindex" # a checkout of the branch +``` + +下一条命令把该条目写入 registry 的 `.xlings.json` 并打印一行;此后索引跟随该检出 +目录,包括第一次构建之后才提交的内容: + +``` + Index xim -> /work/xim-pkgindex ([index.repos.xim] in config.toml) +``` + +从该索引安装时,`Provisioning` 旁边会重复这一行。删去这张表后,下一条命令恢复 +registry 原先的条目,除非该条目在 mcpp 写入之后被改动过。以上对已经运行过的 home +同样成立;2026.9.14.2 之前的 mcpp 只在创建 home 时应用这张表。 + ## 档位:需要载荷的命令 ```toml diff --git a/docs/zh/41-devices.md b/docs/zh/41-devices.md index a56d48ab8..1ad8c5224 100644 --- a/docs/zh/41-devices.md +++ b/docs/zh/41-devices.md @@ -77,6 +77,47 @@ monitor = ["probe-rs", "attach", "--chip", "STM32L475VG"] 优先级是通常那个:工程作者写的胜过依赖提供的,且覆盖会被报告。一个名字只允许一个 依赖提供,第二个是点名两个包的错误。 +## runner 收到的内容 + +runner 收到一段 argv —— 它自己的 token,产物路径被追加或替换 `{}` —— 以及一个环境 +变量 `MCPP_RUNTIME_FILES`。`mcpp run` 与 `mcpp test` 启动的每一个 runner 都会收到 +这个变量,默认 runner 与具名 runner 相同。`mcpp run --no-runner` 不启动 runner, +也不设置它。 + +`MCPP_RUNTIME_FILES` 指向一个文件,列出产物从自己所在目录读取或加载的东西: +`[runtime] deploy` 与 `deploy_files` 的条目,以及构建链接的共享库。每个文件一行: + +``` +data/data.txt/abs/project/target///bin/data/data.txt +libfw.so/abs/project/target///bin/libfw.so +``` + +第一个字段是相对于产物所在目录的目的路径,分隔符为 `/`;在子目录中发现的测试程序, +其目的路径以 `../` 开头。第二个字段是该文件在输出树中的绝对路径。两者以 TAB 分隔。 +每次启动 runner 时这个文件都存在,没有需要携带的东西时为空;以 `--format` 运行的 +可分发物收到空文件,因为可分发物自带其文件。 + +就地执行产物的 runner 忽略这个变量。把产物移走的 runner —— 移到设备、容器或远程 +主机 —— 把列出的每个文件复制到被移走的产物旁边对应的目的路径。 + +## 运行一个可分发物 + +`mcpp run --format ` 打包可分发物 `` 并运行它。不带 `--runner` 时,若图或 +清单提供了名为 `` 的 runner,就由它抵达,否则由默认 runner 抵达: + +```cpp +mcpp::runner("app", ""); // 包以格式名命名 runner +``` + +```bash +mcpp run --format app # 经名为 `app` 的 runner +mcpp run --format app --runner other # 显式的 --runner 仍然优先 +``` + +目录形式的可分发物(例如应用包)不被直接执行。没有 runner 抵达它时, +`mcpp run --format ` 在启动任何东西之前拒绝,退出状态为 `126`,并点名能抵达它的 +runner `` 以及声明它的清单表。 + ## 终止由声明决定,不由推断 | | 含义 | diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index f0a2793a0..6699fe798 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -166,7 +166,7 @@ mcpp cache list --json -> {"root": …, "entries": [ … ]} | 区间 | 含义 | |---|---| | `0`–`124` | 程序跑过了,这是它自己的退出码,原样透传 | -| `125`–`127` | 尝试启动但被拒绝 —— `127` 找不到,`126` 找到但不可执行,`125` 其他 | +| `125`–`127` | 尝试启动但被拒绝 —— `127` 找不到,`126` 找到但不可执行,`125` 其他;对 `mcpp run --format ` 而言,一个是目录、且没有任何 runner 能到达的分发物也得到 `126`,它在启动之前被拒绝,含义相同(2026.9.14.2+) | | `2` | mcpp 在尝试启动之前就拒绝了:用法、配置或解析错误 | 2026.9.4.3 之前,所有非零退出码都被折成 `1`,为的是让 `2` 表示「起不来」以区别于 @@ -399,6 +399,25 @@ mcpp test [pattern] [--workspace] --message-format json 测试都 `not_run` 的成员),与既有的 `not_run` 列表并列;后者仍然指 `--workspace-timeout` 到达时尚未开始的成员。 +### 暂存清单 + +`mcpp pack` 在它暂存的树旁边写出 `.stage-manifest`,点名 +`${mcpp.stage_dir}` 的 `artifact` action 依赖这个文件(见 +[30](30-build-mcpp.md#产出可分发物pack_format-与-stage_dir20269111))。它是按行 +组织的文本文件而不是 JSON 信封,各行按以下顺序构成三块: + +| 行 | 内容 | +|---|---| +| `closure = walked` 或 `closure = not-walked` | 第一行:树所需的每一个库是否都已解析 | +| `reason = ` | 仅在 `closure = not-walked` 时出现;一行 | +| `needs` *(mcpp 2026.9.14.2+)* | 闭包读到的每个库名一行,已排序;`` 是暂存库相对于树的路径、目标提供的库记为 `platform`,或 `unresolved` | +| ` ` 或 `link ` | 每个暂存文件或符号链接一行,已排序 | + +`` 按需要它的对象的拼写给出:`DT_NEEDED` 条目、PE 导入名或 Mach-O install +name。`needs` 行的字段以 TAB 分隔,因为名字与路径都可能含空格。不打包任何东西的 +模式(`system`、`static`)暂存的树不带 `needs` 行,更早的 mcpp 暂存的树同样不带; +自行放置库的读者读这些行,而不是从 `lib/` 下的文件推断闭包。 + ## 当前边界 - **退出码表的作用域只到它点名的那些命令。** 别的命令返回的码不在表里,而把它加进来 diff --git a/docs/zh/README.md b/docs/zh/README.md index f44cb1d43..95542eda5 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -122,6 +122,8 @@ | `[package] platforms`、`[build] cache` | [04](04-mcpp-toml.md) | `[targets.]`、`[profile.]` | [04](04-mcpp-toml.md) | | `runner`、`[target..runners]` | [41](41-devices.md) | `rule_module` | [31](31-authoring-a-rule-package.md) | | `min_api_level` | [04](04-mcpp-toml.md) | `macos_deployment_target` | [04](04-mcpp-toml.md) | +| `[target..targets.] kind` | [22](22-target-side.md) | `config.toml` 中的 `[index.repos.]` | [32](32-authoring-a-payload.md) | +| `[test] discover` | [04](04-mcpp-toml.md), [08](08-testing.md) | `MCPP_RUNTIME_FILES` | [41](41-devices.md) | **命令** diff --git a/examples/04-workspace/apps/hello/mcpp.toml b/examples/04-workspace/apps/hello/mcpp.toml index 4b3e1d005..bd4a74b2b 100644 --- a/examples/04-workspace/apps/hello/mcpp.toml +++ b/examples/04-workspace/apps/hello/mcpp.toml @@ -4,4 +4,4 @@ name = "hello" version = "0.1.0" [dependencies] -greeter = { path = "../../libs/greeter" } +demo.greeter = { path = "../../libs/greeter" } diff --git a/examples/04-workspace/libs/greeter/mcpp.toml b/examples/04-workspace/libs/greeter/mcpp.toml index f67325371..71f6d954b 100644 --- a/examples/04-workspace/libs/greeter/mcpp.toml +++ b/examples/04-workspace/libs/greeter/mcpp.toml @@ -7,4 +7,4 @@ version = "0.1.0" kind = "lib" [dependencies] -core = { path = "../core" } +demo.core = { path = "../core" } diff --git a/examples/08-build-rules/app/mcpp.toml b/examples/08-build-rules/app/mcpp.toml index b127b401e..f7798182e 100644 --- a/examples/08-build-rules/app/mcpp.toml +++ b/examples/08-build-rules/app/mcpp.toml @@ -5,5 +5,5 @@ version = "0.1.0" # Both rules are BUILD-time only: neither reaches this binary, and neither does # anything they depend on. [dependencies] -rules-tidy = { path = "../rules-tidy", host-module = true } -rules-embed = { path = "../rules-embed", host-module = true } +example.rules-tidy = { path = "../rules-tidy", host-module = true } +example.rules-embed = { path = "../rules-embed", host-module = true } diff --git a/examples/08-build-rules/rules-tidy/mcpp.toml b/examples/08-build-rules/rules-tidy/mcpp.toml index 1db75cc4f..6615f1e70 100644 --- a/examples/08-build-rules/rules-tidy/mcpp.toml +++ b/examples/08-build-rules/rules-tidy/mcpp.toml @@ -10,4 +10,4 @@ license = "Apache-2.0" # written by the CONSUMER on an edge, so without this a rule could not request # anything on its own behalf — which is why rules used to be leaves. [build-dependencies] -rules-embed = { path = "../rules-embed", host-module = true } +example.rules-embed = { path = "../rules-embed", host-module = true } diff --git a/examples/12-a-new-device-language/README.md b/examples/12-a-new-device-language/README.md index d6998d1c0..daf4703cf 100644 --- a/examples/12-a-new-device-language/README.md +++ b/examples/12-a-new-device-language/README.md @@ -129,7 +129,7 @@ because the manifest names it: ```toml [dependencies] -rules-toy = { path = "../rules-toy", features = ["rules-toy"] } +example.rules-toy = { path = "../rules-toy", features = ["rules-toy"] } [build] sources = ["src/*.cpp", "src/kernels/*.toy"] @@ -142,7 +142,7 @@ The rule contains no compiler. `toyc` is an ordinary mcpp package with a ```toml [feature-deps.rules-toy] -toyc = { path = "../toyc", tools = ["toyc"], reexport = true } +example.toyc = { path = "../toyc", tools = ["toyc"], reexport = true } ``` | part | what it does | diff --git a/examples/12-a-new-device-language/app/mcpp.toml b/examples/12-a-new-device-language/app/mcpp.toml index 009366e6d..0db8696f2 100644 --- a/examples/12-a-new-device-language/app/mcpp.toml +++ b/examples/12-a-new-device-language/app/mcpp.toml @@ -9,7 +9,7 @@ standard = "c++23" # feature's own `[feature-deps]` brings the compiler, so the consumer names the # feature and nothing else. [dependencies] -rules-toy = { path = "../rules-toy", features = ["rules-toy"] } +example.rules-toy = { path = "../rules-toy", features = ["rules-toy"] } # `.toy` IS NOT IN THE DEFAULT SOURCE GLOB. Device sources are opted into by # naming them -- a package that vendors one it builds elsewhere must not start diff --git a/examples/12-a-new-device-language/rules-toy/mcpp.toml b/examples/12-a-new-device-language/rules-toy/mcpp.toml index d2a116bf6..70bfa76c7 100644 --- a/examples/12-a-new-device-language/rules-toy/mcpp.toml +++ b/examples/12-a-new-device-language/rules-toy/mcpp.toml @@ -37,4 +37,4 @@ device_extensions = [".toy"] # The request hangs on the FEATURE, so a project that depends on this package # without activating the rule builds no compiler. [feature-deps.rules-toy] -toyc = { path = "../toyc", tools = ["toyc"], reexport = true } +example.toyc = { path = "../toyc", tools = ["toyc"], reexport = true } diff --git a/mcpp.toml b/mcpp.toml index 88dabf194..f12c17bd9 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.14.1" +version = "2026.9.14.2" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/manifest/src/dep_spec.cppm b/modules/manifest/src/dep_spec.cppm index 37131f7cd..e7137b31c 100644 --- a/modules/manifest/src/dep_spec.cppm +++ b/modules/manifest/src/dep_spec.cppm @@ -118,6 +118,14 @@ struct DependencySpec { // author wrote. Only such a selector is eligible for the one-release // bare-name fallback: `mcpplibs.gtest` states an identity and must miss. bool namespaceOmitted = false; + // The table that declared this edge, spelled as its TOML header: + // `[dependencies]`, `[target.'cfg(os = "android")'.dependencies]`, + // `[feature-deps.gpu]`. A conditional declaration replaces an + // unconditional one on the rows its selector matches, so which table a + // resolved edge came from is a fact the resolution record states rather + // than one a reader reconstructs. Empty for an edge the parser did not + // create (a synthesised or xpkg-derived dependency). + std::string declaredIn; bool isPath() const { return !path.empty(); } bool isGit() const { return !git.empty(); } diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 7e7611248..8098b935c 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -396,6 +396,25 @@ bool is_array_of_tables(const t::Value& v) { return true; } +// A key as a TOML header has to spell it: bare when every character is one a +// bare key admits, quoted otherwise. Messages name a conditional section with +// it, so that the name a reader sees can be pasted back into a manifest: +// `[target.'cfg(os = "android")'.dependencies]`, not +// `[target.cfg(os = "android").dependencies]`, which does not parse. +std::string toml_header_key(std::string_view key) { + const bool bare = !key.empty() && std::ranges::all_of(key, [](char c) { + return std::isalnum(static_cast(c)) || c == '_' || c == '-'; + }); + if (bare) return std::string(key); + if (key.find('\'') == std::string_view::npos) return std::format("'{}'", key); + std::string out = "\""; + for (char c : key) { + if (c == '"' || c == '\\') out += '\\'; + out += c; + } + return out + "\""; +} + // #253: shared parser for the per-glob flags array shape // `[{ glob = "...", cflags/cxxflags/asmflags/defines = [...] }, ...]` — // one entry grammar for `[build].flags` and `[features]..flags`. @@ -1122,6 +1141,7 @@ std::expected parse_string(std::string_view content, else if (kind_s == "app" || kind_s == "application") t.kind = Target::Application; else return std::unexpected(error(origin, std::format("targets.{}.kind must be 'bin', 'app', 'lib' or 'shared'; got '{}'", tname, kind_s))); + t.kindDeclaredBy = std::format("[targets.{}] kind = \"{}\"", tname, kind_s); // `main` is required for `bin` and for `app`: on every row but // Android it is the executable's entry, exactly as it is for `bin`; @@ -1352,6 +1372,78 @@ std::expected parse_string(std::string_view content, return false; }; + // A selector whose last segment is a dependency OPTION rather than a + // package: `huxerui.huxerui.linkage = "shared"`. TOML reads it as a table + // that names no source, so the grammar sees a dependency on a package + // called `linkage` in the namespace `huxerui.huxerui`, and every later + // message is about that package. The grammar does not change -- a package + // named like an option stays addressable -- but the message does: it says + // what the line declares and restates the dependency with its source, + // taken from the unconditional declaration of the same key when there is + // one. Empty when the last segment is not an option. + auto option_restatement = [&](std::string_view section, + std::string_view selectorText, + std::string_view renderedValue) -> std::string { + auto dot = selectorText.rfind('.'); + if (dot == std::string_view::npos) return {}; + const auto option = selectorText.substr(dot + 1); + if (!is_dep_spec_key(option) || option == "path" || option == "version" + || option == "git" || option == "workspace") + return {}; + const std::string dependency(selectorText.substr(0, dot)); + const auto& family = section.ends_with("dev-dependencies") ? m.devDependencies + : section.ends_with("build-dependencies") ? m.buildDependencies + : m.dependencies; + auto it = family.find(dependency); + const std::string defaultPrefix = std::format("{}.", kDefaultNamespace); + if (it == family.end() && dependency.starts_with(defaultPrefix)) + it = family.find(dependency.substr(defaultPrefix.size())); + std::string source = "version = \"\""; + if (it != family.end()) { + auto const& s = it->second; + if (s.isPath()) + source = std::format("path = \"{}\"", s.path); + else if (s.isGit()) + source = std::format("git = \"{}\", {} = \"{}\"", s.git, + s.gitRefKind.empty() ? "rev" : s.gitRefKind, s.gitRev); + else if (s.inheritWorkspace) + source = "workspace = true"; + else if (!s.version.empty()) + source = std::format("version = \"{}\"", s.version); + } + const bool conditional = section.starts_with("target."); + return std::format( + "'{}' is an option of a dependency, not a package: a dependency " + "table names its source (path, version, git or workspace) beside " + "its options. Restate the dependency with its source: " + "{} = {{ {}, {} = {} }}.{}", + option, dependency, source, option, renderedValue, + conditional ? " On the rows this selector matches, that declaration " + "replaces the unconditional one." + : ""); + }; + // The same restatement for a value that reached an error site rather than + // the version warning (`features = [...]`, `default-features = false`), + // as a clause appended to that error. Empty when the key is not an option. + auto with_restatement = [&](std::string_view section, + std::string_view selectorText, + const t::Value& value) -> std::string { + std::string rendered = "..."; + if (value.is_bool()) { + rendered = value.as_bool() ? "true" : "false"; + } else if (value.is_array()) { + rendered = "["; + for (auto const& e : value.as_array()) { + if (rendered.size() > 1) rendered += ", "; + rendered += e.is_string() ? std::format("\"{}\"", e.as_string()) + : std::string("..."); + } + rendered += "]"; + } + auto restated = option_restatement(section, selectorText, rendered); + return restated.empty() ? std::string{} : std::format(". {}", restated); + }; + auto fill_inline_spec = [&](DependencySpec& spec, std::string_view section, std::string_view fqName, @@ -1505,11 +1597,22 @@ std::expected parse_string(std::string_view content, spec.legacyDottedKey = legacyDottedKey; spec.legacyCandidateSearch = legacyCandidateSearch; spec.namespaceOmitted = selector.namespaceOmitted; + spec.declaredIn = std::format("[{}]", section); auto key = selector.stableMapKey; if (value.is_string()) { spec.version = value.as_string(); - if (auto why = version_req_problem(spec.version); !why.empty()) + auto why = version_req_problem(spec.version); + auto restated = why.empty() + ? std::string{} + : option_restatement(section, key, + std::format("\"{}\"", spec.version)); + if (!restated.empty()) + m.schemaWarnings.push_back(std::format( + "[{}] {} = \"{}\" declares a dependency on a package named " + "'{}'. {}", + section, key, spec.version, key, restated)); + else if (!why.empty()) m.schemaWarnings.push_back(std::format( "[{}] {} = '{}' is not a version RANGE ({}), so it is used " "as an exact index key: the index must carry it verbatim, " @@ -1576,8 +1679,9 @@ std::expected parse_string(std::string_view content, } if (!v.is_table()) { return std::unexpected(error(origin, std::format( - "[{}].{}.{} must be a string, inline dep table, or nested table", - section, ns, k))); + "[{}].{}.{} must be a string, inline dep table, or nested table{}", + section, ns, k, + with_restatement(section, std::format("{}.{}", ns, k), v)))); } auto childNs = std::format("{}.{}", ns, k); auto childMapPrefix = mapPrefix.empty() @@ -1618,8 +1722,9 @@ std::expected parse_string(std::string_view content, } if (!v.is_table()) { return std::unexpected(error(origin, std::format( - "[{}].{} must be a string, inline dep table, or nested table", - section, selectorText))); + "[{}].{} must be a string, inline dep table, or nested table{}", + section, selectorText, + with_restatement(section, selectorText, v)))); } if (auto r = load_selector_dep_table( section, out, selectorText, v.as_table()); !r) @@ -1631,10 +1736,20 @@ std::expected parse_string(std::string_view content, // Parse a dependency table (already obtained) into `out`. Factored out of // load_deps so the same logic serves both [dependencies] (via doc->get_table) // and [target.'cfg(...)'.dependencies] (a nested table the dotted getter - // can't address). `section` is the logical section name, used for error - // messages and namespace/selector resolution. + // can't address). + // + // TWO NAMES, BECAUSE THEY ANSWER TWO QUESTIONS. `section` is the table's + // name as a reader writes it, used in every message and recorded on each + // edge (`DependencySpec::declaredIn`): `target.'cfg(os = "android")'. + // dependencies` for a conditional table. `lookup` is the path the + // explicit-namespace-table test builds (`is_namespace_table`), which a + // conditional table has always resolved against `dependencies`; empty + // means the same as `section`. Before the two were separated, a + // conditional table was loaded under the name `dependencies`, so its + // messages named the unconditional table. auto load_deps_table = [&](std::string_view section, auto& tt, - std::map& out) + std::map& out, + std::string_view lookup = {}) -> std::expected { for (auto& [k, v] : tt) { @@ -1678,7 +1793,7 @@ std::expected parse_string(std::string_view content, // Explicit tables such as `[dependencies.acme]` are namespace // roots. Dotted keys inside the single dependency table are exact // selectors too: `capi.lua` means only `(capi, lua)`. - if (is_namespace_table(section, k)) { + if (is_namespace_table(lookup.empty() ? section : lookup, k)) { if (auto r = load_nested_dep_table(section, out, k, k, sub); !r) return r; } else if (auto r = load_selector_dep_table(section, out, k, sub); !r) { @@ -2362,6 +2477,44 @@ std::expected parse_string(std::string_view content, } } + // [test] — which files are test programs (#634 A5). + // + // One key. A suite compiled from several sources is a package of its own, + // as any program that must diverge from its siblings is (see `Target`); + // what a project could not say before was only WHERE its test programs + // are, which mattered once `tests/` belonged to another build system. + // A wrong type is an error, because the author's value is in front of + // them; an unknown key is a warning, so a manifest written for a later + // mcpp still loads on this one. + if (auto* testValue = doc->get("test"); testValue && !testValue->is_table()) { + return std::unexpected(error(origin, + "[test] must be a table, e.g. [test] discover = [\"tests/**/*.cpp\"]")); + } + if (auto* tt = doc->get_table("test")) { + for (auto& [key, value] : *tt) { + if (key != "discover") { + m.schemaWarnings.push_back(std::format( + "[test] has unsupported key '{}' (ignored). Supported keys: " + "discover.", key)); + continue; + } + bool ok = value.is_array(); + if (ok) { + for (auto& e : value.as_array()) + if (!e.is_string() || e.as_string().empty()) { ok = false; break; } + } + if (!ok) { + return std::unexpected(error(origin, + "[test] discover must be an array of non-empty glob strings, " + "e.g. [\"tests/**/*.cpp\", \"!tests/fixtures/**\"]; an empty " + "array discovers no test")); + } + m.testDiscover.clear(); + for (auto& e : value.as_array()) m.testDiscover.push_back(e.as_string()); + m.testDiscoverDeclared = true; + } + } + // [hooks] — project build lifecycle commands (#496). Parsed HERE rather // than by the module that runs them, for the reason Appendix A of // docs/04-mcpp-toml.md states: mcpp.toml has one grammar and one parser. @@ -2717,21 +2870,22 @@ std::expected parse_string(std::string_view content, // so a key that looks plausible — `cxx_runtime_tests` was the real // one — was accepted in silence and had no effect (#418). // - // NO SUB-TABLES, AND THAT IS THE POINT. The sub-TABLES here are the - // conditional channel (`[target..build]`, `.dependencies`, - // `.dev-dependencies`, `.build-dependencies`, `.feature-deps`) and - // TOML presents each as a key of this table. A hand-written list of - // "known keys" therefore has to enumerate that channel too — and - // that list is exactly the thing this codebase has watched drift - // twice already (see ConditionalConfig's comments on #258 and #359, - // both "the conditional reader kept its own subset and fell - // behind"). The first version of this check did hand-list them and - // warned about `[target.'cfg(unix)'.dependencies]`, a documented - // feature with its own e2e. - // - // Restricting the check to non-tables removes the coupling entirely: - // new conditional sections need no change here, and the reported - // case — a key that does nothing — is still caught. + // SUB-TABLES ARE CHECKED AGAINST A LIST A TEST DERIVES FROM THE + // PARSER. The sub-tables here are the conditional channel + // (`[target..build]`, `.dependencies`, `.targets`, ...) and + // TOML presents each as a key of this table. The first version of + // this check hand-listed them and warned about + // `[target.'cfg(unix)'.dependencies]`, a documented feature with + // its own e2e, and the next version skipped every table-valued + // key. That made a misspelled section (`.dependecies`) and a + // section an older engine does not know (`.targets`, #634) do + // nothing without a word, which is the failure this sweep exists + // to report. `kKnownTargetTables` is written next to the scalar + // lists, and tests/unit/test_target_scalar_keys.cpp reads the + // parse sites of the whole loop body by source text and checks the + // list in both directions, so a new section added to the parser + // without the list fails that test instead of being reported as + // unsupported. // // Scalars AND arrays (#544). The sweep used to skip arrays, which // kept it from reporting `runner` as unsupported while honouring @@ -2752,8 +2906,28 @@ std::expected parse_string(std::string_view content, "cxx_runtime", "linkage", "min_api_level", "sysroot", "toolchain", }; static constexpr std::string_view kKnownTargetArrays[] = { "runner" }; + static constexpr std::string_view kKnownTargetTables[] = { + "abi", "build", "build-dependencies", "dependencies", + "dev-dependencies", "feature-deps", "feature-requires-abi", + "feature-xlings", "requires_abi", "runners", "runtime", + "targets", "xlings", + }; for (auto& [key, value] : body) { - if (value.is_table()) continue; // the conditional channel + if (value.is_table()) { + if (std::ranges::find(kKnownTargetTables, key) + != std::end(kKnownTargetTables)) + continue; + std::string tables; + for (auto name : kKnownTargetTables) { + if (!tables.empty()) tables += ", "; + tables += name; + } + m.schemaWarnings.push_back(std::format( + "[target.{}.{}] is not a section mcpp reads (ignored). " + "The sections of a [target.] table are: {}.", + toml_header_key(triple), key, tables)); + continue; + } const std::span known = value.is_array() ? std::span(kKnownTargetArrays) : std::span(kKnownTargetScalars); @@ -2803,6 +2977,56 @@ std::expected parse_string(std::string_view content, } } } + // `[target..targets.] kind` (#634, A1) -- the per-row + // form of `[targets.] kind`: only `kind`, and only between + // the two library forms. A row may decide how a library is linked; + // it may not decide which targets exist or what a program is. + // Bad values are refused, as `[targets.] kind` refuses its + // own, because a package that states a form and silently does not + // get it is the defect this table exists to remove. Whether the + // name is a library target of this package is checked in `load`, + // after target inference, where the target list is complete. + if (auto tit = body.find("targets"); tit != body.end()) { + if (!tit->second.is_table()) + return std::unexpected(error(origin, std::format( + "[target.{0}].targets must be a table of target names, " + "e.g. `[target.{0}.targets.]` with `kind = \"shared\"`", + toml_header_key(triple)))); + for (auto& [tname, tval] : tit->second.as_table()) { + const auto header = std::format("[target.{}.targets.{}]", + toml_header_key(triple), tname); + if (!tval.is_table()) + return std::unexpected(error(origin, std::format( + "{} must be a table with `kind = \"lib\"` or " + "`kind = \"shared\"`", header))); + auto& row = tval.as_table(); + for (auto& [rk, rv] : row) + if (rk != "kind") + m.schemaWarnings.push_back(std::format( + "{} has unsupported key '{}' (ignored). A row " + "states only `kind`; every other key of a target " + "is declared once, under [targets.{}].", + header, rk, tname)); + auto kit = row.find("kind"); + if (kit == row.end() || !kit->second.is_string()) + return std::unexpected(error(origin, std::format( + "{} must set `kind = \"lib\"` or `kind = \"shared\"`", + header))); + const auto& kindText = kit->second.as_string(); + RowTargetKind rowKind; + if (kindText == "lib" || kindText == "library") + rowKind.kind = Target::Library; + else if (kindText == "shared" || kindText == "dylib" + || kindText == "so" || kindText == "shlib") + rowKind.kind = Target::SharedLibrary; + else + return std::unexpected(error(origin, std::format( + "{} kind = \"{}\": a row chooses between the library " + "forms only, `lib` and `shared`", header, kindText))); + rowKind.statement = std::format("{} kind = \"{}\"", header, kindText); + cc.targetKinds[std::string(tname)] = std::move(rowKind); + } + } // `[target.] requires_abi = { ... }` -- design 2026-09-12 // (the UI framework record), section 2.6, A6: the requirement can // sit on the TARGET axis, because the sources it gates @@ -3011,12 +3235,16 @@ std::expected parse_string(std::string_view content, } // [target..{dependencies,dev-dependencies,build-dependencies}] // parsed via the shared table-based loader (same selectors/namespaces - // as the global [dependencies]) into the deferred config. + // as the global [dependencies]) into the deferred config, under the + // table's full name so that a message and the resolution record + // name the conditional table rather than the unconditional one. auto read_deps = [&](const char* key, std::map& out) -> std::expected { if (auto f = body.find(key); f != body.end() && f->second.is_table()) - return load_deps_table(key, f->second.as_table(), out); + return load_deps_table( + std::format("target.{}.{}", toml_header_key(triple), key), + f->second.as_table(), out, key); return {}; }; if (auto r = read_deps("dependencies", cc.dependencies); !r) return std::unexpected(r.error()); @@ -3031,9 +3259,14 @@ std::expected parse_string(std::string_view content, f != body.end() && f->second.is_table()) { for (auto& [fname, fval] : f->second.as_table()) { if (!fval.is_table()) continue; + // The lookup keeps the spelling this table has always + // been resolved against; only the name a reader sees + // changes, which used to arrive in double brackets. if (auto r = load_deps_table( - std::format("[target.{}.feature-deps.{}]", triple, fname), - fval.as_table(), cc.featureDeps[std::string(fname)]); !r) + std::format("target.{}.feature-deps.{}", + toml_header_key(triple), fname), + fval.as_table(), cc.featureDeps[std::string(fname)], + std::format("[target.{}.feature-deps.{}]", triple, fname)); !r) return std::unexpected(r.error()); m.featuresMap.try_emplace(std::string(fname), std::vector{}); @@ -3551,6 +3784,42 @@ std::expected load(const std::filesystem::path& path, // M5.0: defaults + target inference (uses filesystem context relative to mcpp.toml). apply_defaults_and_infer(*m, path.parent_path()); + + // A `[target..targets.]` row names a target, and an undeclared + // library target exists only after the inference above, so the name is + // checked here rather than where the row is parsed. + for (auto const& cc : m->conditionalConfigs) { + // Plain loops, no structured binding and no projection: clang 20.1.7 on + // the MSVC ABI crashed generating this function (0xC0000005 in LLVM IR + // generation) with `for (auto const& [name, row] : ...)` and a + // member-pointer projection, the class the 2026-09-12 directives.cppm + // crash recorded without isolating the trigger. + for (auto it = cc.targetKinds.begin(); it != cc.targetKinds.end(); ++it) { + const std::string& name = it->first; + const RowTargetKind& row = it->second; + const Target* target = nullptr; + for (std::size_t i = 0; i < m->targets.size(); ++i) + if (m->targets[i].name == name) { target = &m->targets[i]; break; } + if (target == nullptr) { + std::string names; + for (std::size_t i = 0; i < m->targets.size(); ++i) { + if (!names.empty()) names += ", "; + names += m->targets[i].name; + } + return std::unexpected(ManifestError{std::format( + "{} names no target of this package (its targets: {})", + row.statement, names.empty() ? std::string("none") : names), + path, 0, 0}); + } + if (target->kind != Target::Library + && target->kind != Target::SharedLibrary) + return std::unexpected(ManifestError{std::format( + "{}: '{}' is a program target, and a row chooses between " + "the library forms only, `lib` and `shared`", + row.statement, name), + path, 0, 0}); + } + } return m; } diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index fcc950f6b..161e88eeb 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -192,6 +192,21 @@ struct Target { // inert on every object format that is not PE. std::string windowsSubsystem; std::string windowsEntry; + // Where `kind` was stated, as the manifest line that states it: + // `[targets.fw] kind = "shared"`, or, when a row states it, + // `[target.'cfg(os = "android")'.targets.fw] kind = "shared"`. Read by + // the link-form resolution, whose refusal names the statement that + // constrains a package instead of saying that a form "is not available". + std::string kindDeclaredBy; + bool kindFromRow = false; +}; + +// One `[target..targets.] kind` statement: the form a row gives a +// library target, and the manifest line that gives it, which the link-form +// resolution names when it has to refuse a request against it. +struct RowTargetKind { + Target::Kind kind = Target::Library; + std::string statement; }; // `DependencySpec` and `kDefaultNamespace` have moved to mcpp.pm.dep_spec. @@ -1348,6 +1363,14 @@ struct ConditionalConfig { // declaration turns an unsupported platform into a hard error raised from // inside the LIBRARY's manifest, which its user cannot work around. std::map> featureDeps; + // `[target..targets.] kind` -- the per-row form of + // `[targets.] kind`, for a library target the package declares + // unconditionally and only between the two library forms (`Library`, + // `SharedLibrary`). Keyed by target name. A framework that is linked + // statically on the desktop rows and must be one shared copy on Android + // states that once, in its own manifest, and every consumer keeps a + // single unconditional dependency line. + std::map targetKinds; // SPEC-004 §4: `[target..xlings…]` — the TARGET axis of the tool plane. // // THE WHOLE TYPE, NOT A HAND-PICKED SUBSET, for the reason `inputs` above @@ -1397,6 +1420,7 @@ inline bool is_empty(const ConditionalConfig& c) { && c.frameworks.empty() && c.dependencies.empty() && c.devDependencies.empty() && c.buildDependencies.empty() && c.featureDeps.empty() + && c.targetKinds.empty() && c.xlings.empty() && !c.abiThreadsDeclared && !c.abiExceptionsDeclared && !c.requiresAbiThreads && !c.requiresAbiExceptions && c.featureRequiresAbiThreads.empty() && c.featureRequiresAbiExceptions.empty(); @@ -1702,6 +1726,14 @@ struct Manifest { RuntimeConfig runtimeConfig; XlingsConfig xlings; // [xlings] build environment (L-1) Hooks hooks; // [hooks] lifecycle commands (#496) + // `[test] discover`: the globs whose every match is one test program, in + // the vocabulary of `[build] sources` (a leading `!` excludes). A test's + // name is its path relative to the fixed prefix of the first glob that + // matched it. Not declared means `["tests/**/*.cpp"]`; declared and empty + // means no test is discovered (#634 A5). Two members rather than an + // optional vector, for the reason `TargetEntry::sysrootDeclared` records. + std::vector testDiscover; + bool testDiscoverDeclared = false; std::vector conditionalConfigs; // [target.'cfg(...)'.build], deferred std::map profiles; // [profile.] // [features] — feature name → implied features ("default" = default set). diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index b571002a5..bbe75676d 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.14.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.14.2"; } // namespace mcpp diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index a3e1d2e50..b5a209567 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -123,6 +123,10 @@ struct BuildProgramEnv { // a member's own options, where `dist/apple.cppm:140-143` measured it // drifting. std::string minPlatformVersion; + // #634, A7: the pkg-config directories of the registry SubOS, the view + // payloads declare their `.pc` files into. Joined with the platform's + // path-list separator; empty when no registry was loaded. + std::string pkgConfigLibdir; std::string profile; // effective profile name (dev/release/…) std::vector features; // active feature closure of the package // The device axis of this build, in the wire form `mcpp.pack.abi_tag` @@ -583,6 +587,7 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv // contract value here is. Rides this vector, so it joins the re-run key // like every other value contract_hash folds in. e.emplace_back("MCPP_TARGET_MIN_PLATFORM_VERSION", env.minPlatformVersion); + e.emplace_back("MCPP_PKG_CONFIG_LIBDIR", env.pkgConfigLibdir); e.emplace_back("MCPP_PROFILE", env.profile); e.emplace_back("MCPP_ACCEL", env.accel); e.emplace_back("MCPP_LANGUAGE_MODULES", env.languageModules ? "1" : "0"); diff --git a/src/build/distribution.cppm b/src/build/distribution.cppm index c7377cbf8..c9ddcbccb 100644 --- a/src/build/distribution.cppm +++ b/src/build/distribution.cppm @@ -305,6 +305,14 @@ struct MechanismInput { std::string libcxxArchive; std::string libcxxAbiArchive; std::string libunwindArchive; + // The archive FILE NAMES the linker opens for the libc++ pair, when they + // are not `libc++.a` and `libc++abi.a`. `--exclude-libs` matches the + // archive a member was taken from, and the Android NDK's `libc++.a` is a + // linker script, `INPUT(-lc++_static -lc++abi)`, so its members come from + // `libc++_static.a`. Measured on a self-contained shared library for + // `x86_64-linux-android`: 161 dynamic symbols with only the two default + // names, 4 with `libc++_static.a` named. Empty means the two defaults. + std::vector libcxxLinkedArchiveNames; // macOS only: the libc++ archive actually defines the ABI symbol the // initializer-ordering shim binds to. Checked against the archive rather // than assumed, so an unexpected spelling disables the shim instead of @@ -732,8 +740,14 @@ Mechanism resolve(const MechanismInput& in) { // is part of the mechanism, not an optional extra. m.unitFlags = " -nostdlib++ " + in.libcxxArchive + " " + in.libcxxAbiArchive; - m.unitFlags += detail::hide_static_cxx_runtime( - in.role, in.foreignCxxRuntime, {"libc++.a", "libc++abi.a"}); + if (in.libcxxLinkedArchiveNames.empty()) { + m.unitFlags += detail::hide_static_cxx_runtime( + in.role, in.foreignCxxRuntime, {"libc++.a", "libc++abi.a"}); + } else { + for (auto const& name : in.libcxxLinkedArchiveNames) + m.unitFlags += detail::hide_static_cxx_runtime( + in.role, in.foreignCxxRuntime, {std::string_view(name)}); + } if (in.foreignCxxRuntime) { // ONE UNWINDER PER PROCESS. // diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 592d0cb26..9c3d7dafa 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -582,6 +582,83 @@ compute_subos_env(const mcpp::build::BuildPlan& plan) { }); } +// THE FILES A RUNNER HAS TO CARRY WITH THE ARTIFACT (#634 A6). +// +// A runner receives the artifact's path and nothing else, and for a runner +// that executes the artifact on this machine that is enough: the files beside +// it are beside it. A runner that moves the artifact -- `adb-run` pushes the +// program to a device -- moved only the program, and a test reading its +// deployed data then failed on the emulator with `open failed: +// /data/local/tmp/data/data.txt` while passing on the host and on the iOS +// simulator, which reads the host's filesystem. +// +// THE LIST IS WHAT THE ARTIFACT LOADS OR READS FROM ITS OWN DIRECTORY, AS THE +// BUILD LAID IT OUT: every `[runtime] deploy` and `deploy_files` entry, and +// every shared library the plan links, which consumers find beside them +// through `$ORIGIN` or `@loader_path`. A test of a package whose dependency is +// shared on a row (#634 A1) needs that library on the device as much as its +// data. The staged copy in the output tree is named, not the declared source: +// it is the file the artifact reads when it runs here. +// +// One line per file: the destination relative to the artifact's directory, +// with `/` separators, a TAB, and the absolute path of the file. A TAB, +// because a Windows user directory commonly contains a space. A destination +// begins with `../` when the artifact sits below the tree's root, as a test +// discovered in a subdirectory does. The file exists for every runner +// invocation and is empty when there is nothing to carry, so a runner can +// tell an engine that states "nothing" from one that predates the variable. +constexpr std::string_view kRuntimeFilesEnv = "MCPP_RUNTIME_FILES"; + +std::vector> +runtime_files_for(const mcpp::build::BuildContext& ctx, + const std::filesystem::path& artifact) { + std::vector> out; + const auto artifactDir = artifact.parent_path().lexically_normal(); + const auto artifactNorm = artifact.lexically_normal(); + std::set seen; + auto add = [&](const std::filesystem::path& relToOutputDir) { + const auto staged = (ctx.outputDir / relToOutputDir).lexically_normal(); + if (staged == artifactNorm) return; + auto dest = staged.lexically_relative(artifactDir).generic_string(); + if (dest.empty() || !seen.insert(dest).second) return; + out.emplace_back(std::move(dest), staged); + }; + for (auto const& d : ctx.plan.runtimeDeployFiles) add(d.dest); + for (auto const& lu : ctx.plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::SharedLibrary) continue; + add(lu.output); + for (auto const& alias : lu.runtimeAliases) add(alias); + } + return out; +} + +// Writes the list for `artifact` under the output tree and returns its path. +// `carried` is empty for a distributable, which holds its own files. +std::expected +write_runtime_files_list( + const mcpp::build::BuildContext& ctx, + const std::filesystem::path& artifact, + const std::vector>& carried) { + namespace fs = std::filesystem; + std::error_code ec; + auto rel = artifact.lexically_normal().lexically_relative( + ctx.outputDir.lexically_normal()); + if (rel.empty() || *rel.begin() == "..") + rel = fs::path("distributable") / artifact.filename(); + auto listPath = ctx.outputDir / ".mcpp-runtime-files" / rel; + listPath += ".tsv"; + fs::create_directories(listPath.parent_path(), ec); + std::ofstream os(listPath, std::ios::binary | std::ios::trunc); + for (auto const& [dest, source] : carried) + os << dest << '\t' << source.string() << '\n'; + os.flush(); + if (!os) + return std::unexpected(std::format( + "could not write the runtime-files list '{}' a runner receives as {}", + listPath.string(), kRuntimeFilesEnv)); + return listPath; +} + // Compile a prepared BuildContext. Shared between `mcpp build` and `mcpp run` // so the latter doesn't call prepare_build twice (and re-print the toolchain // resolution banner). @@ -796,6 +873,12 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, for (auto& [name, spec] : ctx.manifest.dependencies) { if (announced.contains(name)) continue; announced.insert(name); + // Two keys that resolved to one identity (#634, A2) are one package + // and one compile, so they are announced once. + if (!spec.shortName.empty() + && !announced.insert(std::format("identity:{}.{}", spec.namespace_, + spec.shortName)).second) + continue; // `spec.version` is the constraint the manifest WROTE. Announcing it // printed "Compiling compat.imgui v^1.92.8" — a banner naming a version // that does not exist (mcpp#363). prepare_build hands the resolution @@ -1603,7 +1686,13 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, const std::filesystem::path& exe, std::span passthrough, bool no_runner, - std::string_view runner_name) { + std::string_view runner_name, + // Non-empty when `exe` is the distributable a + // `--format` pack reported; `runner_from_format` + // says the named runner was chosen by that name + // rather than typed with `--runner`. + std::string_view format_name = {}, + bool runner_from_format = false) { auto pathCtx = mcpp::fetcher::make_path_ctx(/*cfg=*/nullptr, ctx.projectRoot); std::vector argv; // An artifact this machine cannot execute — a freestanding image by @@ -1669,6 +1758,29 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, mcpp::freestanding::no_runner_message(choice.tripleKey)); return 2; } + // A DISTRIBUTABLE THAT IS A DIRECTORY, AND NOTHING TO RUN IT (#634 B3). + // + // An application bundle is a directory. Handed to the kernel, it came back + // as `could not be started: Permission denied (error 13)` with status 126 + // (measured on macos-15), a sentence about permissions for a request that + // lacked a runner. The status stays 126 -- found, and not executable -- and + // the sentence names the runner that would reach it, before any spawn. + std::error_code dirEc; + if (!format_name.empty() && choice.tmpl.empty() + && std::filesystem::is_directory(exe, dirEc)) { + std::println(stderr, + "error: --format {} produced a directory, '{}', and no runner reaches it.\n" + " A directory is not executed directly: a runner named '{}' runs\n" + " it. A package supplies one with `mcpp::runner(\"{}\", …)`, or the\n" + " project declares it:\n" + "\n" + " [target.{}.runners]\n" + " {} = [\"\", \"{{}}\"]", + format_name, mcpp::ui::shorten_path(exe, pathCtx), + format_name, format_name, choice.tripleKey, format_name); + return 126; + } + std::optional runtimeFilesList; if (!choice.tmpl.empty()) { // The program is located by mcpp, not by posix_spawnp: a declared // payload's bin/ and then its root, then PATH — see runner_lookup for the shim @@ -1690,12 +1802,25 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, tmpl.front() = found.program->string(); argv = mcpp::freestanding::expand(tmpl, exe); for (auto& a : passthrough) argv.push_back(a); + // What the runner has to carry with the artifact. A distributable + // holds its own files, so its list is empty and still exists. + auto listed = write_runtime_files_list( + ctx, exe, + format_name.empty() ? runtime_files_for(ctx, exe) + : std::vector>{}); + if (!listed) { + std::println(stderr, "error: {}", listed.error()); + return 1; + } + runtimeFilesList = *listed; // The status word is the NAME the package chose, capitalised. The // engine has no table of verbs to look one up in, which is the point: // `Serve`, `Submit` and `Flash` all read correctly and none is known - // here. - std::string verb = isRunSlot ? std::string("Running") : slotName; - if (!isRunSlot && !verb.empty()) + // here. A runner the format's own name selected is still `mcpp run` + // running something, and says so. + std::string verb = (isRunSlot || runner_from_format) ? std::string("Running") + : slotName; + if (!isRunSlot && !runner_from_format && !verb.empty()) verb[0] = static_cast(std::toupper(verb[0])); mcpp::ui::status(verb, std::format("`{} … {}`", choice.tmpl.front(), mcpp::ui::shorten_path(exe, pathCtx))); @@ -1714,6 +1839,8 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, childEnv.emplace_back(runEnvKey, runEnvValue); // ...plus whatever the subos declares for the programs it hosts (#352). for (auto& kv : compute_subos_env(ctx.plan)) childEnv.push_back(std::move(kv)); + if (runtimeFilesList) + childEnv.emplace_back(std::string(kRuntimeFilesEnv), runtimeFilesList->string()); // Direct exec (no /bin/sh): the loader env reaches ONLY the target child, // never mcpp or a host shell. Fixes the bundled-glibc-vs-host-libtinfo @@ -1901,8 +2028,25 @@ export int build_run_target(const std::optional& targetName, "exactly one to hand to the runner", format, outcome.artifacts.size(), names); return 1; } + // THE NAMED RUNNER THE FORMAT'S OWN NAME SELECTS (#634 B3). + // + // A distributable is reached the way its format is reached, and the + // name a package gives that way is the format's own: `dist-apple` + // supplies `mcpp::runner("app", …)` for `--format app`. Without this, + // `mcpp run --format app` took the DEFAULT runner -- the one a plain + // `mcpp run` hands the link output -- and a project had to repeat the + // format as `--runner app`. A typed `--runner` still wins, and a + // format no runner is named after keeps the default runner. + std::string effectiveRunner{runner_name}; + bool runnerFromFormat = false; + if (effectiveRunner.empty() + && !choose_device_action(*ctx2, format).tmpl.empty()) { + effectiveRunner = format; + runnerFromFormat = true; + } return run_artifact_via_runner(*ctx2, outcome.artifacts.front(), - passthrough, no_runner, runner_name); + passthrough, no_runner, effectiveRunner, + format, runnerFromFormat); } // Build first. Single prepare_build → drive build → reuse ctx to locate @@ -2103,7 +2247,16 @@ export int run_tests(std::span passthrough, auto testRoot = discovered->packageRoot; auto testTargets = std::move(discovered->targets); if (testTargets.empty()) { - std::println("no tests found in tests/"); + // Names where it looked when the manifest chose the place, so that a + // glob that matches nothing is not read as a project without tests. + if (discovered->discoverDeclared) { + std::string globs; + for (auto const& g : discovered->discover) + globs += std::format("{}\"{}\"", globs.empty() ? "" : ", ", g); + std::println("no tests found ([test] discover = [{}])", globs); + } else { + std::println("no tests found in tests/"); + } return 0; } // --list: enumerate (filtered) tests and stop — no toolchain resolution, @@ -2669,6 +2822,16 @@ export int run_tests(std::span passthrough, // A GL test that cannot find a driver fails the same way a GL program // does, so it must be told the same things. for (auto& kv : subosEnv) childEnv.push_back(kv); + // ...and, through a runner, the files the test carries with it, as + // `mcpp run` hands them over (see `runtime_files_for`). Written here, + // in the single-threaded pass, one list per test program. + if (!runnerTmpl.empty()) { + if (auto listed = write_runtime_files_list(*ctx, exe, + runtime_files_for(*ctx, exe))) + childEnv.emplace_back(std::string(kRuntimeFilesEnv), listed->string()); + else if (invocationNotRunReason.empty()) + invocationNotRunReason = listed.error(); + } // Prepend the sandbox's subos/default/bin to the CHILD PATH so test // binaries that shell out to bootstrapped tools (patchelf, ninja) find diff --git a/src/build/flags.cppm b/src/build/flags.cppm index acf1b2be3..3dea5ab71 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -228,6 +228,16 @@ std::string include_token(const mcpp::toolchain::CommandDialect& d, std::string_view prefixOverride = {}, PathForm form = PathForm::Native); +// Does a search-path entry begin with a token the dynamic loader expands +// (#634, item 11 of the triage record)? ELF's `$ORIGIN` and every other +// `$`-token (`${ORIGIN}`, `$LIB`), and Mach-O's `@executable_path`, +// `@loader_path` and `@rpath`. Such an entry is relative to an object the +// loader has loaded, not to the package that wrote it, so the ldflag +// normalisers leave it as written. One predicate for both of them: the second +// copy exempted `$` alone, and an `@executable_path` rpath arrived in the binary +// as `/@executable_path/..`. +bool is_loader_relative_search_path(std::string_view entry); + } // namespace mcpp::build namespace mcpp::build { @@ -252,6 +262,18 @@ std::string escape_ninja_chars(std::string_view s) { return out; } +bool is_loader_relative_search_path(std::string_view entry) { + if (entry.starts_with('$')) return true; + for (std::string_view token : {std::string_view("@executable_path"), + std::string_view("@loader_path"), + std::string_view("@rpath")}) { + if (entry.starts_with(token) + && (entry.size() == token.size() || entry[token.size()] == '/')) + return true; + } + return false; +} + namespace { std::filesystem::path staged_std_bmi_path(const BuildPlan& plan) { @@ -266,7 +288,7 @@ std::string escape_path(const std::filesystem::path& p) { std::string normalize_ldflag(const std::filesystem::path& root, const std::string& flag) { auto absolute_path = [&](std::string_view raw) { std::filesystem::path p{std::string(raw)}; - if (p.is_absolute() || raw.starts_with("$")) return p; + if (p.is_absolute() || is_loader_relative_search_path(raw)) return p; return root / p; }; @@ -1057,11 +1079,11 @@ CompileFlags compute_flags(const BuildPlan& plan) { ? base : dist::default_contract(dist::Role::SharedLibrary, format)); - // Archive lookup. LLVM lays these out either directly under lib/ (the - // macOS packages) or under lib// (the Linux ones), so try - // both rather than hard-coding one layout. Sorted so the choice cannot - // depend on directory iteration order. - auto find_archive = [&](std::string_view name) -> std::filesystem::path { + // Archive lookup, directory half. LLVM lays these out either directly + // under lib/ (the macOS packages) or under lib// (the + // Linux ones), so try both rather than hard-coding one layout. Sorted + // so the choice cannot depend on directory iteration order. + auto search_llvm_root = [&](std::string_view name) -> std::filesystem::path { if (llvmRootForStdlib.empty()) return {}; std::error_code ec; auto libDir = llvmRootForStdlib / "lib"; @@ -1173,16 +1195,112 @@ CompileFlags compute_flags(const BuildPlan& plan) { // compiled against the package's headers. mi.graphCxxRuntime = plan.targetSide.cxx.fromGraph(); + // AND THE DRIVER'S HALF, WHICH IS ASKED FIRST (#634 A6). + // + // The search above knows one layout; the Android NDK keeps these + // archives in its sysroot per API level, the search missed them, and + // every Android test program degraded to needing `libc++_shared.so`. + // `runtime_archive_path` asks the driver with the flags this link + // carries -- the target with its API level, and the configuration + // bypass when the payload has a cfg -- which is the question the link + // itself will ask. + // + // The same file found both ways keeps the SEARCH's spelling, so a host + // link line the search already served stays byte-identical (a cache key + // and a ninja command both read it). A different file means the search + // guessed: a payload carrying several triples' archives sorts another + // triple first. + // + // Asked only where an answer is used: ELF and Mach-O links of a hosted + // target whose C++ runtime is the toolchain's. Every other cell of the + // mechanism table ignores these archives, and a query costs a process. + const bool driverPlacesArchives = + (format == dist::Format::Elf || format == dist::Format::MachO) + && !mi.freestanding && !mi.graphCxxRuntime && !mi.appleCrossTarget; + const std::string driverFlags = + crossTarget + (isClangWithCfg ? " --no-default-config" : ""); + auto find_archive = [&](std::string_view name) -> std::filesystem::path { + auto searched = search_llvm_root(name); + if (!driverPlacesArchives) return searched; + auto answered = mcpp::toolchain::clang::runtime_archive_path( + plan.toolchain, driverFlags, name); + if (answered.empty()) return searched; + std::error_code ec; + if (!searched.empty() && std::filesystem::equivalent(searched, answered, ec)) + return searched; + return answered; + }; + const bool wantsArchives = (base == dist::Contract::SelfContained || testsContract == dist::Contract::SelfContained || sharedContract == dist::Contract::SelfContained) && caps.stdlib_id == "libc++"; + // THE ARCHIVES A LINKER SCRIPT OPENS, by file name. + // + // A driver may place an archive name on a linker script rather than on + // an archive: the Android NDK's per-API `libc++.a` is + // `INPUT(-lc++_static -lc++abi)`. Linking it is correct -- lld reads a + // script given as an input -- but `--exclude-libs` names the archive a + // member came from, so hiding `libc++.a` hid nothing (see + // `MechanismInput::libcxxLinkedArchiveNames`). An archive starts with + // `!` and yields nothing here, so a payload whose archives are + // archives keeps the two default names and its link line. + auto script_archive_names = [](const std::filesystem::path& p) + -> std::vector { + std::vector names; + std::error_code ec; + if (p.empty() || std::filesystem::file_size(p, ec) > 4096 || ec) return names; + std::ifstream is(p, std::ios::binary); + std::string text((std::istreambuf_iterator(is)), + std::istreambuf_iterator()); + if (text.starts_with("!")) return names; + auto open = text.find('('); + auto close = text.rfind(')'); + if (open == std::string::npos || close == std::string::npos || close < open) + return names; + auto keyword = text.substr(0, open); + std::erase_if(keyword, [](unsigned char c) { return std::isspace(c); }); + if (keyword != "INPUT" && keyword != "GROUP") return names; + std::string token; + auto flush = [&] { + if (token.empty() || token == "AS_NEEDED") { token.clear(); return; } + std::string name; + if (token.starts_with("-l:")) name = token.substr(3); + else if (token.starts_with("-l")) name = "lib" + token.substr(2) + ".a"; + else name = std::filesystem::path(token).filename().string(); + if (!name.empty() && std::ranges::find(names, name) == names.end()) + names.push_back(std::move(name)); + token.clear(); + }; + for (auto c : std::string_view(text).substr(open + 1, close - open - 1)) { + if (std::isspace(static_cast(c)) || c == ',' || c == '(' + || c == ')') + flush(); + else + token.push_back(c); + } + flush(); + return names; + }; + if (wantsArchives) { auto libcxxA = find_archive("libc++.a"); auto libcxxAbiA = find_archive("libc++abi.a"); mi.libcxxArchive = libcxxA.empty() ? std::string{} : escape_path(libcxxA); mi.libcxxAbiArchive = libcxxAbiA.empty() ? std::string{} : escape_path(libcxxAbiA); + { + auto fromLibcxx = script_archive_names(libcxxA); + auto fromAbi = script_archive_names(libcxxAbiA); + if (!fromLibcxx.empty() || !fromAbi.empty()) { + std::vector names{"libc++.a", "libc++abi.a"}; + for (auto* list : {&fromLibcxx, &fromAbi}) + for (auto& n : *list) + if (std::ranges::find(names, n) == names.end()) + names.push_back(n); + mi.libcxxLinkedArchiveNames = std::move(names); + } + } // ELF only: without it the "self-contained" binary still pulls // libunwind.so.1. Mach-O's libc++abi.a carries its own unwinder. if (mi.format == dist::Format::Elf) { diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 31a6b9557..e1e677c3d 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -401,6 +401,17 @@ inline const char* toolchain_dir() { return env_or("MCPP_TOOLCHAI inline const char* toolchain_sysroot() { return env_or("MCPP_TOOLCHAIN_SYSROOT"); } inline const char* toolchain_binutils_dir() { return env_or("MCPP_TOOLCHAIN_BINUTILS_DIR"); } +// The pkg-config search path of the payloads mcpp installed: the registry +// SubOS's `usr/lib/pkgconfig` and `usr/share/pkgconfig`, joined with the +// platform's path-list separator, for a build program that runs `pkg-config` +// over a library a payload provides (`PKG_CONFIG_LIBDIR= pkg-config +// --cflags --libs gtk4`). Payload recipes declare their `.pc` files into that +// view, so it resolves a payload's whole pkg-config closure. +// +// AN ACCESSOR AND NOT AN ENVIRONMENT DEFAULT: a package that means the host's +// own pkg-config database keeps it, and one that means the payloads says so. +inline const char* pkg_config_libdir() { return env_or("MCPP_PKG_CONFIG_LIBDIR"); } + // Which compiler resolved: "gcc", "clang", "msvc", or "" if none did. // // Ask this rather than inferring it from `toolchain_dir()`. The two questions diff --git a/src/build/linkage_form.cppm b/src/build/linkage_form.cppm index b28e34d9f..80af6d38a 100644 --- a/src/build/linkage_form.cppm +++ b/src/build/linkage_form.cppm @@ -73,6 +73,16 @@ struct Admissible { // Why not, when `sharedOk` is false. Always populated in that case: a // refusal a user cannot act on is worse than no feature. std::string sharedRefusal; + // Why not, when `staticOk` is false: the manifest line that constrains + // the package to the shared form. Before it existed, a refused `static` + // request was answered with "the requested form is not available here", + // a sentence that does not name the statement that decided. + std::string staticRefusal; + // Which constraint narrowed the set, as the token the resolution record + // stores (`package-kind`, `row-kind`, `no-loader`, `static-libc`, + // `packaged`, `no-sources`, `prebuilt-inputs`). Empty when both forms are + // admissible. + std::string constraint; bool allows(DepLinkage linkage) const { return linkage == DepLinkage::Static ? staticOk : sharedOk; @@ -96,6 +106,11 @@ struct PackageFacts { // as "must be static" would freeze the entire ecosystem out of this axis. // Absence of a constraint is not a constraint. bool declaredShared = false; + // The line that states it, `[targets.fw] kind = "shared"` or its per-row + // form, and whether it is the per-row form. Read only for the refusal and + // the resolution record. + std::string declaredSharedBy; + bool declaredSharedByRow = false; // The package's resolved `ldflags` name link inputs mcpp did not compile // (see `carries_foreign_link_inputs`). @@ -151,6 +166,10 @@ struct Resolution { DepLinkage linkage = DepLinkage::Static; // Non-empty exactly when the answer differs from an EXPLICIT request. std::string diagnostic; + // Why this form, for the resolution record: `default` (nobody asked and + // nothing constrains), `requested` (an explicit request was honoured), or + // the `Admissible::constraint` token that overrode the request. + std::string reason; }; Resolution resolve(const PackageFacts& package, const Admissible& admissible, @@ -205,14 +224,16 @@ Admissible admissible(const PackageFacts& package, const TargetFacts& target) { if (!target.hasLoader) { return Admissible{ .staticOk = true, .sharedOk = false, .sharedRefusal = "this target has no dynamic loader, so there is " - "nothing that could load a shared library" }; + "nothing that could load a shared library", + .constraint = "no-loader" }; } if (target.fullStaticLibc) { return Admissible{ .staticOk = true, .sharedOk = false, .sharedRefusal = "this image links its C library statically " "(`linkage = \"static\"`), and a static " "executable has no interpreter to load a shared " - "library with" }; + "library with", + .constraint = "static-libc" }; } if (package.isDistribution) { @@ -226,24 +247,37 @@ Admissible admissible(const PackageFacts& package, const TargetFacts& target) { out.sharedRefusal = std::format( "{} is a packaged library and ships only a static leg", package.label); + if (!out.staticOk) + out.staticRefusal = std::format( + "{} is a packaged library and ships only a shared leg", + package.label); + if (!out.staticOk || !out.sharedOk) out.constraint = "packaged"; return out; } if (package.declaredShared) - return Admissible{ .staticOk = false, .sharedOk = true }; + return Admissible{ .staticOk = false, .sharedOk = true, + .staticRefusal = package.declaredSharedBy.empty() + ? std::string("its manifest declares a shared library target") + : std::format("its manifest states {}, which constrains the " + "package to the shared form", + package.declaredSharedBy), + .constraint = package.declaredSharedByRow ? "row-kind" : "package-kind" }; if (!package.hasSources) { return Admissible{ .staticOk = true, .sharedOk = false, .sharedRefusal = std::format( "{} builds none of its own sources, so mcpp has no objects to " - "make a shared library from", package.label) }; + "make a shared library from", package.label), + .constraint = "no-sources" }; } if (package.carriesForeignLinkInputs) { return Admissible{ .staticOk = true, .sharedOk = false, .sharedRefusal = std::format( "{} brings its own prebuilt link inputs (its `ldflags` carry a " "`-L`), which mcpp cannot place inside a shared library it " - "builds", package.label) }; + "builds", package.label), + .constraint = "prebuilt-inputs" }; } return Admissible{ .staticOk = true, .sharedOk = true }; } @@ -259,26 +293,33 @@ Resolution resolve(const PackageFacts& package, const Admissible& admissible, explicitRequest = true; } - if (admissible.allows(wanted)) return Resolution{ .linkage = wanted }; + if (admissible.allows(wanted)) + return Resolution{ .linkage = wanted, + .reason = explicitRequest ? "requested" : "default" }; // Not allowed. There is exactly one other form, and the admissible set is // never empty by construction — `staticOk` is false only for a package the // author constrained to shared, and that case allows Shared. const DepLinkage fallback = admissible.sharedOk ? DepLinkage::Shared : DepLinkage::Static; - Resolution out{ .linkage = fallback }; + Resolution out{ .linkage = fallback, .reason = admissible.constraint }; // SPEAK ONLY FOR A BROKEN PROMISE. When the whole-graph value is mcpp's // own default, nobody asked for anything and there is nothing to report; // saying so on every build would put a warning on correct manifests that // their authors cannot act on. Same rule mcpp.build.distribution applies // to its contract defaults. if (explicitRequest && wanted != fallback) { + // The refusal of the form that was ASKED FOR: a refused `static` + // request is explained by what constrains the package to `shared`. + const auto& refusal = wanted == DepLinkage::Static + ? admissible.staticRefusal + : admissible.sharedRefusal; out.diagnostic = std::format( "{} is linked as a {} library: {}", package.label, to_string(fallback), - admissible.sharedRefusal.empty() + refusal.empty() ? std::string("the requested form is not available here") - : admissible.sharedRefusal); + : refusal); } return out; } diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 8d2ade095..6741bc36c 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -240,6 +240,16 @@ std::string join_flags(const std::vector& flags) { // default install name is the path it was LINKED at, so a package built in // /tmp/build-xyz records /tmp/build-xyz and cannot be relocated — which is // every distributed dylib. `@rpath/` is the only default that travels. +// * On ELF likewise (#634 A4): a library linked without `-soname` carries no +// DT_SONAME, and bionic enforces one from API level 23 ("Missing SONAME"). +// The default is the output file name, which is the name every consumer +// already records in DT_NEEDED because it links with `-l`. It is a +// DEFAULT and not a declaration, so it is `shared_soname_default` below and +// reaches the link BEFORE `$ldflags`: the linker takes the last `-soname`, +// and a name a project passes through `[build] ldflags` or +// `mcpp::link_flag` has to keep winning, as it did when there was no +// default (e2e 220 names its library that way). A declared `soname` keeps +// its value, its position and its alias. // Whether a PE link speaks the MSVC ABI. Asked of the target triple, and of the // compiler's own answer only when there is no triple; `pe_link_flag` below says // why the compiler binary is the wrong question. One definition, so the import @@ -289,6 +299,19 @@ std::string shared_soname_flag(const LinkUnit& lu, const BuildPlan& plan) { return lu.soname.empty() ? "" : "-Wl,-soname," + lu.soname; } +// The ELF default SONAME, for a shared library that declares none: its output +// file name. See the comment above `shared_soname_flag` for why it is a +// separate variable placed before `$ldflags`. A wasm row refuses a shared +// target before a link is planned, so a non-PE, non-Mach-O triple here is ELF. +std::string shared_soname_default(const LinkUnit& lu, const BuildPlan& plan) { + if (lu.kind != LinkUnit::SharedLibrary || !lu.soname.empty()) return ""; + const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); + const bool pe = t ? t->is_pe() : bool(mcpp::platform::is_windows); + const bool macho = t ? t->is_mach_o() : bool(mcpp::platform::is_macos); + if (pe || macho) return ""; + return "-Wl,-soname," + lu.output.filename().string(); +} + // WHICH SYMBOLS A SHARED LIBRARY PUBLISHES, rendered per platform from one // neutral list. // @@ -1378,7 +1401,7 @@ std::string emit_ninja_string(const BuildPlan& plan) { "$cxx $in -o $out $ldflags $unit_ldflags", "LINK"); link_rule("cxx_archive", std::string(dial.archiveCmd), "AR"); link_rule("cxx_shared", - "$cxx -shared $in -o $out $ldflags $soname_flag " + "$cxx -shared $in -o $out $soname_default $ldflags $soname_flag " "$implib_flag $def_flag $unit_ldflags", "SHARED"); // mcpp#426: a link unit with no C++ translation unit in it is @@ -1391,7 +1414,7 @@ std::string emit_ninja_string(const BuildPlan& plan) { link_rule("c_link", "$cc $in -o $out $c_ldflags $unit_ldflags", "LINK"); link_rule("c_shared", - "$cc -shared $in -o $out $c_ldflags $soname_flag " + "$cc -shared $in -o $out $soname_default $c_ldflags $soname_flag " "$implib_flag $def_flag $unit_ldflags", "SHARED"); } @@ -2273,6 +2296,8 @@ std::string emit_ninja_string(const BuildPlan& plan) { implicit.empty() ? std::string{} : " |" + implicit); if (auto flag = shared_soname_flag(lu, plan); !flag.empty()) out_line += " soname_flag = " + flag + "\n"; + if (auto flag = shared_soname_default(lu, plan); !flag.empty()) + out_line += " soname_default = " + flag + "\n"; // The export set, written beside the artifact and named on the link. // // Folded into `soname_flag` rather than given a rule variable of its diff --git a/src/build/plan.cppm b/src/build/plan.cppm index fd47a1e6b..ee6555a2d 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -1731,6 +1731,26 @@ make_plan(const mcpp::manifest::Manifest& manifest, lu.implicitInputs.push_back(alias); auto flags = shared_library_link_flags(dep.target, naming, targetTriple); lu.linkFlags.insert(lu.linkFlags.end(), flags.begin(), flags.end()); + // A CONSUMER IN ANOTHER DIRECTORY NEEDS THE WAY BACK. The + // search path above is the consumer's own directory, which is + // where the library is for a program in `bin/` and is not for + // a test built from a subdirectory of `tests/` + // (`bin/tests/sub/`): measured, such a test could not load a + // graph-built shared library and exited 127. The relative + // path from the consumer's directory to the library's is + // added for that case only, so every consumer that shares the + // library's directory keeps its link line. + const auto libDir = dep.output.parent_path().lexically_normal(); + const auto unitDir = lu.output.parent_path().lexically_normal(); + if (!naming.sharedNeedsImportLib && !unitDir.empty() && libDir != unitDir) { + const auto rel = libDir.lexically_relative(unitDir).generic_string(); + const bool macho = targetTriple.empty() ? bool(mcpp::platform::is_macos) + : targetTriple.is_mach_o(); + if (!rel.empty() && rel != ".") + lu.linkFlags.push_back(macho + ? "-Wl,-rpath,@loader_path/" + rel + : "-Wl,-rpath,'$$ORIGIN/" + rel + "'"); + } } } }; diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 3ca2c746f..820a69124 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -364,6 +364,28 @@ layer_predicated_xlings_refusal(const mcpp::manifest::Manifest& m) return std::nullopt; } +// Two declarations of one dependency, compared by the identity their keys +// normalise to rather than by the keys themselves: `fw` and `mcpplibs.fw` are +// one package under two map keys (`selector.stableMapKey`), and a comparison +// of keys would leave both entries in the map for the resolver to see. +bool same_dependency_identity(const mcpp::manifest::DependencySpec& a, + const mcpp::manifest::DependencySpec& b) { + if (a.shortName.empty() || b.shortName.empty()) return false; + return a.namespace_ == b.namespace_ && a.shortName == b.shortName; +} + +void replace_dependencies( + std::map& into, + const std::map& from) +{ + for (auto const& [key, spec] : from) { + std::erase_if(into, [&](auto const& entry) { + return entry.first == key || same_dependency_identity(entry.second, spec); + }); + into[key] = spec; + } +} + export void merge_conditional_config(mcpp::manifest::Manifest& m, const cfgpred::Ctx& ctx) { @@ -455,18 +477,34 @@ export void merge_conditional_config(mcpp::manifest::Manifest& m, // BuildInputs, so conditional sources are mirrored into it here. for (auto const& s : cc.inputs.sources) m.modules.sources.push_back(s); - // insert() keeps an existing unconditional entry: a conditional - // section adds a dependency, it never silently overrides one. - m.dependencies.insert(cc.dependencies.begin(), cc.dependencies.end()); - m.devDependencies.insert(cc.devDependencies.begin(), cc.devDependencies.end()); - m.buildDependencies.insert(cc.buildDependencies.begin(), - cc.buildDependencies.end()); + // A matching conditional declaration of a dependency REPLACES the + // declaration of the same identity, and a later matching section + // replaces an earlier one: the rule every conditional scalar above + // follows (#634, A1). This used to be `insert()`, which kept the + // unconditional entry, so `linkage = "shared"` written for one row was + // dropped on that row without a word. No manifest among 509 scanned + // declared one dependency in both tables, so no build that worked + // changes; the declaring table rides on the spec (`declaredIn`) into + // the resolution record. + replace_dependencies(m.dependencies, cc.dependencies); + replace_dependencies(m.devDependencies, cc.devDependencies); + replace_dependencies(m.buildDependencies, cc.buildDependencies); // #359: `[target..feature-deps.]`. The feature is // registered by the parser regardless of the predicate; only what it // pulls in is conditional. - for (auto const& [fname, deps] : cc.featureDeps) { - auto& dst = m.featureDeps[fname]; - dst.insert(deps.begin(), deps.end()); + for (auto const& [fname, deps] : cc.featureDeps) + replace_dependencies(m.featureDeps[fname], deps); + // `[target..targets.] kind`: the row's form of a library + // target, applied before resolution, so the link-form resolution + // reads it exactly as it reads `[targets.] kind`. `load` has + // already refused a name that is not a library target. + for (auto const& [name, row] : cc.targetKinds) { + for (auto& t : m.targets) { + if (t.name != name) continue; + t.kind = row.kind; + t.kindDeclaredBy = row.statement; + t.kindFromRow = true; + } } } } @@ -1245,8 +1283,18 @@ void fill_package_build_env(mcpp::build::BuildProgramEnv& e, void fill_target_build_env(mcpp::build::BuildProgramEnv& e, const mcpp::manifest::Manifest& m, - const mcpp::toolchain::Toolchain* tc) + const mcpp::toolchain::Toolchain* tc, + const mcpp::config::GlobalConfig* cfg) { + // The registry SubOS is where payloads are installed, whichever + // toolchain or link mode resolved, so this is set before the toolchain + // gate below. + if (cfg) { + const auto view = mcpp::xlings::paths::sysroot(mcpp::config::make_xlings_env(*cfg)); + e.pkgConfigLibdir = (view / "usr" / "lib" / "pkgconfig").generic_string() + + mcpp::platform::env::path_list_separator() + + (view / "usr" / "share" / "pkgconfig").generic_string(); + } e.toolchainDir = (tc && !tc->binaryPath.empty()) ? tc->binaryPath.parent_path().parent_path().string() : std::string{}; e.targetSysroot = tc ? tc->targetSysrootRoot.string() : std::string{}; @@ -1505,6 +1553,25 @@ provision_xlings_addresses(const mcpp::config::GlobalConfig& cfg, if (needProvision) { mcpp::ui::status("Provisioning", std::format("{} ({})", label, join_deps(", "))); + // An address whose index a `[index.repos.]` table + // redirects is installed from that source, and says so: an + // installation from a branch checkout must not read as one + // from the published index (#634, C4). + std::set redirected; + for (auto const& d : declaredDeps) { + const auto colon = d.find(':'); + if (colon == std::string::npos) continue; + const auto index = d.substr(0, colon); + for (auto const& r : cfg.indexRepos) { + if (!r.fromConfig || r.name != index) continue; + if (r.name == "mcpplibs" && r.url == mcpp::config::kMcpplibsIndexUrl) + continue; + if (redirected.insert(index).second) + mcpp::ui::status("Index", std::format( + "{} -> {} ([index.repos.{}] in config.toml)", + r.name, r.url, r.name)); + } + } // GLOBAL scope, and the scope is the whole point. // // The obvious alternative -- `install_packages` against @@ -1709,6 +1776,10 @@ prepare_build(bool print_fingerprint, // developer directory can be switched between them -- and this repository // has a standing rule that a value crossing two sites is resolved at one. std::optional appleSdkLocated; + // The iOS floor was not written and was taken from the located SDK. The + // refusal of a dependency's platform floor names where the value came + // from, and after the fill below the manifest no longer says. + bool iosFloorFromSdk = false; // Non-empty when a target row's convention replaced a toolchain the user // had set with `mcpp toolchain default`. Reported on the status line, // because a substitution nobody is told about is a rule that can only be @@ -2053,6 +2124,17 @@ prepare_build(bool print_fingerprint, "condition it on the triple instead.", cc.predicate)); } + // The same reason holds for a row's library form: whether a package + // is linked shared is decided while the graph is resolved, before a + // layer has an answer. + if (cfgpred::uses_layer(cc.predicate) && !cc.targetKinds.empty()) { + m->schemaWarnings.push_back(std::format( + "[target.'{}'] conditions a target's kind on a target-side " + "layer (ignored). A layer is resolved from the dependency " + "graph, and a library's form is decided while that graph is " + "resolved; condition the kind on the triple instead.", + cc.predicate)); + } } // Surface non-fatal manifest schema warnings (e.g. unsupported [targets.*] @@ -2340,6 +2422,26 @@ prepare_build(bool print_fingerprint, // silently overrule one the user wrote down. auto tcOrigin = tcSpec.has_value() ? TcOrigin::ManifestToolchain : TcOrigin::None; + // `--toolchain` shares `ManifestToolchain`'s precedence and not its + // spelling: the messages that refuse a spec name where it was written, and + // a value from the command line credited to a manifest key sends the + // reader to a file that does not contain it. + bool tcFromCommandLine = false; + auto tcSpecSource = [&]() -> std::string { + if (tcOrigin == TcOrigin::ManifestToolchain && tcFromCommandLine) + return "--toolchain"; + switch (tcOrigin) { + case TcOrigin::ManifestToolchain: + return std::format("[toolchain].{}", kCurrentPlatform); + case TcOrigin::TargetSection: + return std::format("[target.{}].toolchain", overrides.target_triple); + case TcOrigin::GlobalDefault: + return "the default toolchain (`mcpp toolchain default`)"; + default: + return std::format("the toolchain mcpp chose ({})", + tc_origin_name(tcOrigin)); + } + }; // `--toolchain` (arriving as MCPP_TOOLCHAIN, the same side channel // `--offline` and `--jobs` use) beats everything, including the manifest. // @@ -2354,6 +2456,7 @@ prepare_build(bool print_fingerprint, if (const char* tcEnv = std::getenv("MCPP_TOOLCHAIN"); tcEnv && *tcEnv) { tcSpec = std::string(tcEnv); tcOrigin = TcOrigin::ManifestToolchain; + tcFromCommandLine = true; } if (!tcSpec.has_value()) { auto cfg = get_cfg(); @@ -2605,8 +2708,10 @@ prepare_build(bool print_fingerprint, // that the fingerprint slot, the effective triple and every // report read one value. if (appleSdkLocated && m->buildConfig.iosDeploymentTarget.empty()) { - if (auto v = mcpp::platform::macos::sdk_version(which)) + if (auto v = mcpp::platform::macos::sdk_version(which)) { m->buildConfig.iosDeploymentTarget = *v; + iosFloorFromSdk = true; + } } if (!appleSdkLocated) { // A CODE, BECAUSE THE MATRIX COMPARES REASONS AND NOT ONLY @@ -3091,7 +3196,7 @@ prepare_build(bool print_fingerprint, // where it used to happen — somewhere else, saying something else. auto s = mcpp::toolchain::parse_toolchain_spec(*tcSpec); if (!s) return std::unexpected(std::format( - "[toolchain].{} = '{}': {}", kCurrentPlatform, *tcSpec, s.error())); + "{} = '{}': {}", tcSpecSource(), *tcSpec, s.error())); parsedSpec = std::move(*s); tcOriginAxis = mcpp::toolchain::origin_of(*parsedSpec); } @@ -3118,8 +3223,8 @@ prepare_build(bool print_fingerprint, auto spec = parsedSpec; if (spec->version.empty()) { return std::unexpected(std::format( - "[toolchain].{} = '{}' is invalid; expected '@'", - kCurrentPlatform, *tcSpec)); + "{} = '{}' is invalid; expected '@'", + tcSpecSource(), *tcSpec)); } // A `--target ` build carries the (already canonical) triple // into the spec's target axis: the payload mapping then resolves the @@ -4688,6 +4793,51 @@ prepare_build(bool print_fingerprint, }; std::deque worklist; + // #634, A2. A `path` or `git` dependency's identity is the one its manifest + // declares (SPEC-001 §1.2), and the key a consumer wrote is one way of + // reaching it. `identityBySource` maps a canonical source (the directory, + // or the repository and reference) to the identity resolved from it, so a + // second key over the same source finds that record without loading the + // manifest again. `declaringManifest` holds the manifest each such + // identity came from, and records whether that manifest named its + // namespace: one that does not takes the key's, so two keys over it would + // be two identities over one source. + struct DeclaringManifest { + std::string path; + bool namespaceDeclared = false; + }; + std::map identityBySource; + std::map declaringManifest; + std::set> adoptionsReported; + auto qualifiedKey = [](const ResolvedKey& k) { + return k.ns.empty() ? k.shortName : std::format("{}.{}", k.ns, k.shortName); + }; + // A root edge that adopted an identity states it on the root's own + // declaration too, which is what every later reader of the root manifest + // (the build banner, the resolution record) sees. + auto stateAdoptedIdentity = [&](const WorkItem& item, const ResolvedKey& declared) { + if (item.consumerDepIndex != kMainConsumer) return; + if (auto it = m->dependencies.find(item.name); it != m->dependencies.end()) { + it->second.namespace_ = declared.ns; + it->second.shortName = declared.shortName; + } + }; + // One warning per declaring edge: each names a line someone can correct. + auto reportAdoption = [&](const std::string& requestedBy, const std::string& written, + const ResolvedKey& normalised, const ResolvedKey& declared, + const std::string& manifestPath) { + if (!adoptionsReported.emplace(requestedBy, written).second) return; + mcpp::diag::warning("dependency/identity", std::format( + "'{}' declares the dependency '{}', which names {}; the manifest " + "'{}' declares {}, and that identity is used.", + requestedBy, written, qualifiedKey(normalised), manifestPath, + qualifiedKey(declared)), + std::format("write '{}' in '{}' to state the identity the " + "manifest declares.", + qualifiedKey(declared), requestedBy)); + }; + + // Index routing — WHICH index answers for a namespace and how its // descriptors are read — lives in mcpp.pm.index_route, shared with the // `mcpp add` existence gate so the two cannot disagree about which @@ -5359,7 +5509,7 @@ prepare_build(bool print_fingerprint, // version. Scoped: restored when this dependency's install returns, // compat retries below included. mcpp::build::BuildProgramEnv hookEnv; - fill_target_build_env(hookEnv, *m, tc ? &*tc : nullptr); + fill_target_build_env(hookEnv, *m, tc ? &*tc : nullptr, cfg_opt ? &*cfg_opt : nullptr); hookEnv.targetTriple = overrides.target_triple; // Six names, fixed by install_hook_env; one guard each. const auto hookVars = mcpp::build::install_hook_env(hookEnv); @@ -5573,6 +5723,22 @@ prepare_build(bool print_fingerprint, bool buildOnly = false; }; std::vector dependencyEdges; + // #634, X: every request that reached a package, as the requester wrote + // it, for the `graph` section of resolution.json. Kept apart from + // `dependencyEdges`, which merges two requests of one consumer for one + // dependency into one edge; the record has to keep both keys, because two + // keys over one identity (A2) and the table a declaration came from (A1) + // are what it exists to show. + struct GraphRequest { + std::size_t consumerPackageIndex = 0; + std::size_t dependencyPackageIndex = 0; + std::string key; // the dependency key as the requester wrote it + std::string table; // `DependencySpec::declaredIn` + }; + std::vector graphRequests; + // The link form each dependency took, and why (`linkage_form::Resolution`), + // by package index. + std::map> graphLinkForms; namespace dg = mcpp::build::dep_graph; // #355: consumer package index → (env var, absolute path) for each host // tool that consumer requested. Filled by the provisioning pass below; @@ -6034,13 +6200,25 @@ prepare_build(bool print_fingerprint, [&](std::size_t consumerDepIndex, std::size_t dependencyPackageIndex, const mcpp::manifest::DependencySpec& spec, - bool buildOnly = false) + bool buildOnly, + const std::string& writtenKey) { const auto consumerPackageIndex = packageIndexForConsumer(consumerDepIndex); if (consumerPackageIndex >= packages.size() || dependencyPackageIndex >= packages.size()) { return; } + if (std::ranges::none_of(graphRequests, [&](const GraphRequest& r) { + return r.consumerPackageIndex == consumerPackageIndex + && r.dependencyPackageIndex == dependencyPackageIndex + && r.key == writtenKey && r.table == spec.declaredIn; + })) + graphRequests.push_back(GraphRequest{ + .consumerPackageIndex = consumerPackageIndex, + .dependencyPackageIndex = dependencyPackageIndex, + .key = writtenKey, + .table = spec.declaredIn, + }); const auto visibility = parseVisibility(spec.visibility); auto same = [&](const DependencyEdge& edge) { return edge.consumerPackageIndex == consumerPackageIndex @@ -6141,7 +6319,9 @@ prepare_build(bool print_fingerprint, const std::string& flag) { auto absolute_path = [&](std::string_view raw) { std::filesystem::path p{std::string(raw)}; - if (p.is_absolute() || raw.starts_with("$")) return p; + // A loader token stays as written; see the predicate. + if (p.is_absolute() || mcpp::build::is_loader_relative_search_path(raw)) + return p; return depRoot / p; }; @@ -6540,6 +6720,34 @@ prepare_build(bool print_fingerprint, // branch below for the cache identity. std::string sourceCommit; + // A second key over a source that is already resolved takes the + // identity resolved there; its manifest is not loaded again. + if (sourceKind != "version") { + const auto source = sourceRefOf(sourceKind, spec, item.resolveRoot, + item.originalConstraint); + if (auto bySource = identityBySource.find(source); + bySource != identityBySource.end() && !(bySource->second == key)) { + const auto& existing = resolved.at(bySource->second); + const auto& declaring = declaringManifest.at(bySource->second); + if (!declaring.namespaceDeclared) { + return std::unexpected(std::format( + "one source is reached as two packages: '{}' names it {} " + "and '{}' names it {}, and its manifest '{}' declares no " + "namespace, so each key gives it its own identity and " + "its modules would be compiled twice.\n" + " fix: declare `namespace` in '{}', or write the " + "same key in both places.", + existing.requestedBy, qualifiedKey(bySource->second), + item.requestedBy, qualifiedKey(key), + declaring.path, declaring.path)); + } + reportAdoption(item.requestedBy, name, key, bySource->second, + declaring.path); + key = bySource->second; + stateAdoptedIdentity(item, key); + } + } + if (auto it = resolved.find(key); it != resolved.end()) { // A package is dev-only until some non-dev consumer wants it. Order // of arrival must not decide, so this is an AND over every request. @@ -6641,7 +6849,7 @@ prepare_build(bool print_fingerprint, if (it->second.depIndex + 1 < packages.size()) { recordDependencyEdge(item.consumerDepIndex, it->second.depIndex + 1, - spec, item.buildOnly); + spec, item.buildOnly, name); } continue; } @@ -6806,7 +7014,7 @@ prepare_build(bool print_fingerprint, const auto depPackageIndex = packages.size(); packages.push_back(makePackageRoot(secStage, *dep_manifests.back())); recordDependencyEdge(item.consumerDepIndex, depPackageIndex, - spec, item.buildOnly); + spec, item.buildOnly, name); auto linkFlagsAdded = propagateLinkFlags(secStage, *dep_manifests.back()); ResolvedKey mangledKey{key.ns, mangledPackage}; @@ -6849,7 +7057,7 @@ prepare_build(bool print_fingerprint, // no re-fetch needed; just record this consumer edge. recordDependencyEdge(item.consumerDepIndex, it->second.depIndex + 1, - spec, item.buildOnly); + spec, item.buildOnly, name); continue; } @@ -6907,7 +7115,7 @@ prepare_build(bool print_fingerprint, makePackageRoot(newRoot, *dep_manifests[it->second.depIndex]); recordDependencyEdge(item.consumerDepIndex, it->second.depIndex + 1, - spec, item.buildOnly); + spec, item.buildOnly, name); it->second.version = *merged; it->second.linkFlagsAdded = std::move(linkFlagsAdded); @@ -6986,7 +7194,7 @@ prepare_build(bool print_fingerprint, if (it->second.depIndex + 1 < packages.size()) { recordDependencyEdge(item.consumerDepIndex, it->second.depIndex + 1, - spec, item.buildOnly); + spec, item.buildOnly, name); } continue; } @@ -7259,6 +7467,44 @@ prepare_build(bool print_fingerprint, } } + // The identity a `path` or `git` manifest declares is the package's, + // whatever key reached it (#634, A2). Before this, only the short name + // was compared, so `fw` reaching a manifest that declares `huxdemo.fw` + // resolved as `mcpplibs.fw` while every reader that builds a name from + // the manifest saw `huxdemo.fw`, and a second edge written + // `huxdemo.fw` put the same sources into the build twice. + const bool namespaceDeclared = !dep_manifest->package.namespace_.empty(); + const std::string manifestPath = sourceKind == "version" + ? std::string{} + : (dep_root / "mcpp.toml").lexically_normal().generic_string(); + if (sourceKind != "version" && namespaceDeclared) { + auto declaredName = mcpp::pm::compat::resolve_package_name( + dep_manifest->package.name, dep_manifest->package.namespace_); + ResolvedKey declared{ dep_manifest->package.namespace_, + declaredName.shortName }; + if (!(declared == key)) { + reportAdoption(item.requestedBy, name, key, declared, manifestPath); + stateAdoptedIdentity(item, declared); + if (resolved.contains(declared)) { + // Another source already resolved the declared identity, + // and the rules for two declarations of one identity + // decide (the #630 decision table, at the resolved-record + // hit above). The edge is queued again stating that + // identity, which sends it there. + item.spec.namespace_ = declared.ns; + item.spec.shortName = declared.shortName; + item.spec.candidates = {{ .namespace_ = declared.ns, + .shortName = declared.shortName }}; + item.spec.namespaceOmitted = false; + item.spec.legacyCandidateSearch = false; + item.spec.legacyDottedKey = false; + worklist.push_front(std::move(item)); + continue; + } + key = declared; + } + } + // Stamp the identity with the resolver's exact coordinate and source. // A descriptor that omitted namespace inherits the coordinate that // answered it; otherwise two indices containing the same short name @@ -7305,10 +7551,16 @@ prepare_build(bool print_fingerprint, const auto depPackageIndex = packages.size(); packages.push_back(makePackageRoot(dep_root, *dep_manifests.back())); recordDependencyEdge(item.consumerDepIndex, depPackageIndex, spec, - item.buildOnly); + item.buildOnly, name); // Record this dep as resolved so future encounters of the same // (ns, name) hit the fast path (skip / merge / conflict). + if (sourceKind != "version") { + identityBySource.emplace( + sourceRefOf(sourceKind, spec, item.resolveRoot, item.originalConstraint), + key); + declaringManifest[key] = DeclaringManifest{ manifestPath, namespaceDeclared }; + } resolved[key] = ResolvedRecord{ .version = sourceKind == "version" ? spec.version : "", .constraint = sourceKind == "version" ? item.originalConstraint : "", @@ -7803,6 +8055,48 @@ prepare_build(bool print_fingerprint, std::map> deviceSourcesByPackage; auto checkVersionFloors = [&]() -> std::optional { std::map> facts; // name -> (version, who) + // #634, A9: THE TARGET'S PLATFORM FLOOR IS A FACT THE ENGINE STATES, + // in the platform's own words. A dependency that needs Android API 23 + // writes `android.api-level >= 23` as an ordinary `version-floor` + // requirement and is refused before compiling when the application + // targets less. The floor is not raised for it: the value is already + // inside the compiler's `--target` by now, and which devices an + // application installs on is the application's decision. A row that + // states no such fact (a desktop Linux build) leaves the requirement + // silent, so a requirement needs no selector. The engine's value is + // entered first, so a package stating the same name cannot replace it. + std::map platformFactOrigin; // name -> the key that sets it + if (tc) { + if (auto t = mcpp::toolchain::triple::parse(tc->targetTriple); + t && (t->is_android() || t->is_apple())) { + const auto value = min_platform_version(*m, *t, tc->binaryPath); + std::string name, origin; + if (t->is_android()) { + auto row = m->targetOverrides.find(t->str()); + name = "android.api-level"; + origin = row != m->targetOverrides.end() && row->second.minApiLevel > 0 + ? std::format("[target.{}] min_api_level", t->str()) + : std::format("the toolchain's lowest supported level, because " + "[target.{}] min_api_level is not set", t->str()); + } else if (t->is_ios()) { + name = "ios.deployment-target"; + origin = iosFloorFromSdk + ? std::string("the located SDK's version, because [build] " + "ios_deployment_target is not set") + : std::string("[build] ios_deployment_target"); + } else { + name = "macos.deployment-target"; + origin = m->buildConfig.macosDeploymentTarget.empty() + ? std::string("mcpp's default for macOS, because [build] " + "macos_deployment_target is not set") + : std::string("[build] macos_deployment_target"); + } + if (!value.empty()) { + facts.emplace(name, std::pair{value, std::string{}}); + platformFactOrigin.emplace(name, std::move(origin)); + } + } + } for (std::size_t pi = 0; pi < packages.size(); ++pi) { // The root's claims live in *m: its build program mutates // *m, and packages[0] is a snapshot taken before it ran. @@ -7834,15 +8128,26 @@ prepare_build(bool print_fingerprint, floor.version); if (!met || *met) continue; refusal::record(refusal::Code::VersionFloorUnmet); + if (auto origin = platformFactOrigin.find(floor.name); + origin != platformFactOrigin.end()) + return std::format( + "`{}` requires {} >= {}, and this build targets {}.\n" + " set by: {}\n" + " This is checked before anything is compiled " + "because the failure it prevents is not:\n" + " a library that needs a newer platform links " + "cleanly and fails on the device that lacks it.", + who, floor.name, floor.version, it->second.first, + origin->second); return std::format( - "`{}` requires {} >= {}, and this machine has {}.\n" + "`{}` requires {} >= {}, and {} is stated as {}.\n" " stated by: {}\n" " This is checked before anything is compiled " "because the failure it prevents is not:\n" " a build against too-new a runtime links " "cleanly and fails at first use.", - who, floor.name, floor.version, it->second.first, - it->second.second); + who, floor.name, floor.version, floor.name, + it->second.first, it->second.second); } } return std::nullopt; @@ -9196,7 +9501,7 @@ prepare_build(bool print_fingerprint, // library resolved, and the three answers that keep a board // package from naming a toolchain. One call, so a new answer // reaches every build program at once — see fill_target_build_env. - fill_target_build_env(bpEnv, *m, tc ? &*tc : nullptr); + fill_target_build_env(bpEnv, *m, tc ? &*tc : nullptr, cfg_opt ? &*cfg_opt : nullptr); bpEnv.toolsBin = projectSubosBin; bpEnv.profile = effectiveProfile; bpEnv.accel = resolvedAccel(); @@ -10238,7 +10543,7 @@ prepare_build(bool print_fingerprint, // C library, which compiler and which C++ standard library resolved, // and the three answers that keep a board package from naming a // toolchain. One call — see fill_target_build_env. - fill_target_build_env(bpEnv, *m, tc ? &*tc : nullptr); + fill_target_build_env(bpEnv, *m, tc ? &*tc : nullptr, cfg_opt ? &*cfg_opt : nullptr); bpEnv.toolsBin = projectSubosBin; bpEnv.profile = effectiveProfile; bpEnv.accel = resolvedAccel(); @@ -10281,12 +10586,63 @@ prepare_build(bool print_fingerprint, // snapshot. Anything past this index is a `mcpp::deploy()` residue // that needs the same mirror the flag/source tails get below. const auto rdeployN = m->runtimeConfig.linkIntent.deploy.size(); + // What the dependencies supplied as runners, before the root's program + // speaks. The root's emissions are appended to the same slots, so a + // name both supply becomes one argv joining the two (#634, §9 item 8, + // measured: `run-A.sh run-B.sh `). + const auto runnerBeforeRoot = bcRoot.runner; + const auto namedBeforeRoot = bcRoot.namedRunners; if (auto bp = mcpp::build::run_build_program( *m, *root, host->first, host->second, m->cppStandard, bpEnv); !bp) { return std::unexpected(bp.error()); } + // THE SAME RULE THE DEPENDENCIES ARE HELD TO, WITH THE ROOT AS A PARTY. + // Two suppliers of one runner are refused naming both, and the + // manifest is the way to choose: a `[target.]` runner the + // project writes outranks every supplied one where the runner is + // looked up, so a name the manifest declares is not refused here. + { + const auto rowKey = [&]() -> std::string { + if (!tc) return {}; + auto t = mcpp::toolchain::triple::parse(tc->targetTriple); + return t ? t->str() : tc->targetTriple; + }(); + const auto row = m->targetOverrides.find(rowKey); + const auto manifestNames = [&](std::string_view name) { + if (row == m->targetOverrides.end()) return false; + if (name.empty()) return !row->second.runner.empty(); + return row->second.namedRunners.contains(std::string(name)); + }; + if (!runnerProvider.empty() && !runnerBeforeRoot.empty() + && bcRoot.runner.size() > runnerBeforeRoot.size() + && !manifestNames({})) { + return std::unexpected(std::format( + "the dependency '{}' and this project's build program both " + "supply the runner for this target, and the two would be " + "joined into one argv.\n" + " Drop one of them, or state the runner in " + "[target.{}].runner.", + runnerProvider, rowKey)); + } + for (auto const& [name, nr] : bcRoot.namedRunners) { + auto before = namedBeforeRoot.find(name); + auto who = namedRunnerProvider.find(name); + if (before == namedBeforeRoot.end() || before->second.argv.empty() + || who == namedRunnerProvider.end() || who->second.empty()) + continue; + if (nr.argv.size() <= before->second.argv.size()) continue; + if (manifestNames(name)) continue; + return std::unexpected(std::format( + "the dependency '{}' and this project's build program both " + "supply a runner named '{}' for this target, and the two " + "would be joined into one argv.\n" + " Drop one of them, or state it in " + "[target.{}.runners].{}.", + who->second, name, rowKey, name)); + } + } auto& pkg0 = packages[0]; // Compile-visible tail → privateBuild: the shared fold (same owner // as the dep loop; the root's TUs read privateBuild). @@ -11329,8 +11685,12 @@ prepare_build(bool print_fingerprint, } std::vector libraryTargets; for (auto& t : pkg.targets) { - if (t.kind == mcpp::manifest::Target::SharedLibrary) + if (t.kind == mcpp::manifest::Target::SharedLibrary + && !facts.declaredShared) { facts.declaredShared = true; + facts.declaredSharedBy = t.kindDeclaredBy; + facts.declaredSharedByRow = t.kindFromRow; + } if (t.kind == mcpp::manifest::Target::Library) libraryTargets.push_back(&t); } @@ -11350,6 +11710,11 @@ prepare_build(bool print_fingerprint, } auto allowed = lf::admissible(facts, targetFacts); auto answer = lf::resolve(facts, allowed, request); + // Recorded for a package that has a library to link; a package of + // programs or rules has no form to report. + if (facts.isDistribution || facts.declaredShared || !libraryTargets.empty()) + graphLinkForms[i] = { std::string(lf::to_string(answer.linkage)), + answer.reason }; if (!answer.diagnostic.empty()) mcpp::diag::degraded("build/dependency-linkage", answer.diagnostic, @@ -12821,6 +13186,39 @@ prepare_build(bool print_fingerprint, std::string(mcpp::build::dist::to_string(roleFlags.contractByRole[i])); } + // #634, X: the resolved dependency graph. One entry per package, the + // root first: its identity as `runtime` records identities, every + // request that reached it with the key as written and the table that + // declared it, and for a library the link form with its reason. It is + // what `mcpp why deps` prints, and what a test of a resolution rule + // reads instead of a warning's wording. + { + nlohmann::json graphPackages = nlohmann::json::array(); + for (std::size_t i = 0; i < packages.size(); ++i) { + nlohmann::json entry = { + {"package", package_json( + mcpp::manifest::package_id(packages[i].manifest.package))}, + {"root", i == 0}, + }; + nlohmann::json requests = nlohmann::json::array(); + for (auto const& r : graphRequests) { + if (r.dependencyPackageIndex != i) continue; + requests.push_back({ + {"requester", mcpp::manifest::package_id( + packages[r.consumerPackageIndex].manifest.package).canonical()}, + {"key", r.key}, + {"table", r.table}, + }); + } + entry["requested_by"] = std::move(requests); + if (auto form = graphLinkForms.find(i); form != graphLinkForms.end()) + entry["link"] = { {"form", form->second.first}, + {"reason", form->second.second} }; + graphPackages.push_back(std::move(entry)); + } + j["graph"] = { {"packages", std::move(graphPackages)} }; + } + j["runtime"] = { {"cxx_runtime_by_role", contracts}, {"library_dirs", dirs}, diff --git a/src/build/test_targets.cppm b/src/build/test_targets.cppm index a769742da..4d3de90ed 100644 --- a/src/build/test_targets.cppm +++ b/src/build/test_targets.cppm @@ -12,8 +12,15 @@ export namespace mcpp::build { struct TestTargetSet { std::filesystem::path packageRoot; std::vector targets; + // The globs discovery used, and whether the manifest wrote them. Carried + // so that "no tests found" can name where it looked. + std::vector discover; + bool discoverDeclared = false; }; +// `[test] discover` when the manifest says nothing. +inline constexpr std::string_view kDefaultTestDiscover = "tests/**/*.cpp"; + // 损坏 manifest 时仅提供 best-effort 文件清单；严格调用方仍须走 prepare/validate。 std::expected discover_test_targets(const std::filesystem::path& manifestRoot, @@ -35,7 +42,40 @@ discover_test_targets(const std::filesystem::path& manifestRoot, std::vector globFlags; if (packageManifest) globFlags = packageManifest->buildConfig.globFlags; - const auto testFiles = mcpp::modgraph::expand_glob(packageRoot, "tests/**/*.cpp"); + TestTargetSet result{packageRoot, {}, {}, false}; + if (packageManifest && packageManifest->testDiscoverDeclared) { + result.discover = packageManifest->testDiscover; + result.discoverDeclared = true; + } else { + result.discover = { std::string(kDefaultTestDiscover) }; + } + + // `[test] discover` IN THE VOCABULARY OF `[build] sources` (#634 A5): + // every positive glob is expanded, then every `!` glob, and an excluded + // file is excluded whichever positive glob found it. A file keeps the + // FIRST positive glob that found it, because that glob's fixed prefix is + // what its name is relative to -- which is how the default, + // `tests/**/*.cpp`, names `tests/unit/test_span.cpp` `unit/test_span`, the + // name every earlier release gave it. + std::vector> testFiles; + { + std::set excluded; + for (auto const& g : result.discover) { + if (g.starts_with('!')) + for (auto& p : mcpp::modgraph::expand_glob(packageRoot, g.substr(1))) + excluded.insert(p); + } + std::set seenFiles; + for (auto const& g : result.discover) { + if (g.starts_with('!')) continue; + const auto prefix = mcpp::modgraph::glob_literal_prefix(g); + const auto base = prefix.empty() ? packageRoot : packageRoot / prefix; + for (auto& p : mcpp::modgraph::expand_glob(packageRoot, g)) { + if (excluded.contains(p) || !seenFiles.insert(p).second) continue; + testFiles.emplace_back(p, base); + } + } + } std::vector> globHits; globHits.reserve(globFlags.size()); for (auto const& gf : globFlags) { @@ -43,10 +83,9 @@ discover_test_targets(const std::filesystem::path& manifestRoot, globHits.emplace_back(hits.begin(), hits.end()); } - TestTargetSet result{packageRoot, {}}; result.targets.reserve(testFiles.size()); - std::set seenNames; - for (auto const& file : testFiles) { + std::map seenNames; + for (auto const& [file, discoverBase] : testFiles) { auto lexical_relative = [&](const std::filesystem::path& base, std::string_view boundary) -> std::expected { @@ -61,15 +100,18 @@ discover_test_targets(const std::filesystem::path& manifestRoot, return relative; }; - auto testRelative = lexical_relative(packageRoot / "tests", "tests root"); + auto testRelative = lexical_relative(discoverBase, "its discover glob's directory"); if (!testRelative) return std::unexpected(testRelative.error()); auto mainRelative = lexical_relative(packageRoot, "package root"); if (!mainRelative) return std::unexpected(mainRelative.error()); auto name = testRelative->replace_extension("").generic_string(); - if (!seenNames.insert(name).second) { + if (auto [it, fresh] = seenNames.emplace(name, *mainRelative); !fresh) { return std::unexpected(std::format( - "duplicate test name '{}' (two test files map to the same name)", name)); + "duplicate test name '{}': '{}' and '{}' map to the same name " + "(a test's name is its path relative to the fixed directory of " + "the [test] discover glob that found it)", + name, it->second.generic_string(), mainRelative->generic_string())); } mcpp::manifest::Target target; diff --git a/src/cli.cppm b/src/cli.cppm index e2d8e010a..b057a3599 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -419,6 +419,14 @@ int run(int argc, char** argv) { .help("Alias for --target")) .option(cl::Option("package").short_name('p').takes_value().value_name("NAME") .help("Run only the named workspace member (single-member; no --workspace fan-out)")) + // DECLARED ON THE THREE COMMANDS THAT BUILD BEFORE THEY ACT, AS ON + // `build`. The value has always reached them: the pre-parse loop + // above publishes it as MCPP_TOOLCHAIN for every command, and + // `MCPP_TOOLCHAIN=llvm@22.1.8 mcpp test` compiled with clang, while + // the spelling the help of `build` teaches was refused here as an + // unknown option (#634 A10). + .option(cl::Option("toolchain").takes_value().value_name("SPEC") + .help("Build with this toolchain for one invocation, e.g. llvm@22.1.8")) .option(cl::Option("cache").takes_value().value_name("MODE") .help("Global dependency cache: global (default) | local | off")) .option(cl::Option("no-cache") @@ -509,6 +517,8 @@ int run(int argc, char** argv) { .help("Treat manifest schema warnings (unknown feature/platform) as errors")) .option(cl::Option("package").short_name('p').takes_value().value_name("NAME") .help("Run tests only for the named workspace member")) + .option(cl::Option("toolchain").takes_value().value_name("SPEC") + .help("Build the tests with this toolchain for one invocation, e.g. llvm@22.1.8")) .option(cl::Option("cache").takes_value().value_name("MODE") .help("Global dependency cache: global (default) | local | off")) .option(cl::Option("no-cache") @@ -609,6 +619,8 @@ int run(int argc, char** argv) { // this only replaces the "dev" fallback every other command uses. .option(cl::Option("profile").takes_value() .help("Build profile (default: [build] default-profile, else release)")) + .option(cl::Option("toolchain").takes_value().value_name("SPEC") + .help("Build with this toolchain for one invocation, e.g. llvm@22.1.8")) .option(cl::Option("no-strip") .help("Ship the artifacts as built (default: strip debug info)")) .option(cl::Option("debug-symbols").takes_value().value_name("DIR") diff --git a/src/config.cppm b/src/config.cppm index 08a1f5ef4..14578f0b6 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -23,6 +23,7 @@ export module mcpp.config; import std; import mcpp.home; import mcpp.libs.toml; +import mcpp.libs.json; import mcpp.pm.index_spec; import mcpp.xlings; import mcpp.platform; @@ -40,6 +41,10 @@ struct IndexRepo { std::string url; std::string artifact; // optional artifact source base (xlings >= 0.4.68, #269) std::string source; // optional "auto" | "artifact" | "git" ("" = xlings default auto) + // Read from a `[index.repos.]` table rather than added by mcpp as a + // default. The provisioning line names an index that such a table + // redirected (#634, C4). + bool fromConfig = false; }; // Canonical mcpplibs index coordinates. The index repository moved from the @@ -384,6 +389,151 @@ bool write_default_xlings_json(const std::filesystem::path& path, return std::filesystem::exists(path); } +// ── `[index.repos.]` reaches an existing registry (#634, C4) ────────── +// +// The table used to seed the registry's `.xlings.json` only when that file did +// not exist, so a table added to the config of a home that had already run +// once did nothing, without a word; the measured case is a consumer's CI that +// points `xim` at a checkout of an unmerged recipe branch. Every index +// repository a table declares is now written into `index_repos` when it differs +// from the entry of the same name, and xlings re-points the index at the next +// sync (measured: an existing home's `data/xim-pkgindex` became the checkout +// after `mcpp index update`). The default entry mcpp adds without a table is +// not reconciled: the file's copy of it is left as the file has it. +// +// WHAT MCPP WROTE IS RECORDED, so that removing the table undoes it. The record +// (`.mcpp-index-overrides.json`, beside the file) keeps, per name, the entry a +// table replaced (`null` when there was none) and the entry mcpp wrote. A name +// whose table leaves the configuration gets its previous entry back, but only +// while the file still holds what mcpp wrote: an entry changed since belongs +// to whoever changed it. Each change prints one line. +namespace { + +std::filesystem::path index_override_record(const std::filesystem::path& xjson) { + return xjson.parent_path() / ".mcpp-index-overrides.json"; +} + +std::optional read_json_file(const std::filesystem::path& path) { + std::ifstream in(path); + if (!in) return std::nullopt; + auto doc = nlohmann::json::parse(in, nullptr, false); + if (doc.is_discarded()) return std::nullopt; + return doc; +} + +bool write_json_file(const std::filesystem::path& path, const nlohmann::json& doc) { + auto tmp = path; + tmp += ".tmp"; + { + std::ofstream out(tmp); + if (!out) return false; + out << doc.dump(2) << "\n"; + } + std::error_code ec; + std::filesystem::rename(tmp, path, ec); + return !ec; +} + +nlohmann::json index_repo_entry(const IndexRepo& r) { + nlohmann::json e = { {"name", r.name}, {"url", r.url} }; + if (!r.artifact.empty()) e["artifact"] = r.artifact; + if (!r.source.empty()) e["source"] = r.source; + return e; +} + +std::string describe_index_repo(const IndexRepo& r) { + return std::format("{} -> {} ([index.repos.{}] in config.toml)", r.name, r.url, r.name); +} + +} // namespace + +// A fresh home was seeded with every configured repository. The ones a table +// declared are recorded as written over nothing, so that removing the table +// later restores the default instead of leaving the redirection behind. +void record_seeded_index_repos(const std::filesystem::path& xjson, + const std::vector& repos, + bool quiet) +{ + nlohmann::json record = nlohmann::json::object(); + for (auto const& r : repos) { + if (!r.fromConfig) continue; + if (r.name == "mcpplibs" && r.url == kMcpplibsIndexUrl) continue; // the default + record[r.name] = { {"previous", nullptr}, {"written", index_repo_entry(r)} }; + if (!quiet) print_status("Index", describe_index_repo(r)); + } + if (!record.empty()) (void)write_json_file(index_override_record(xjson), record); +} + +void reconcile_index_repos(const std::filesystem::path& xjson, + const std::vector& repos, + bool quiet) +{ + auto doc = read_json_file(xjson); + if (!doc || !doc->is_object()) return; + auto& entries = (*doc)["index_repos"]; + if (!entries.is_array()) entries = nlohmann::json::array(); + const auto recordPath = index_override_record(xjson); + auto record = read_json_file(recordPath).value_or(nlohmann::json::object()); + if (!record.is_object()) record = nlohmann::json::object(); + + auto find = [&](const std::string& name) -> nlohmann::json* { + for (auto& e : entries) + if (e.is_object() && e.value("name", std::string{}) == name) return &e; + return nullptr; + }; + // ONLY A TABLE IS RECONCILED. The entry mcpp adds as its default was + // seeded with the file, and an existing home's copy of it belongs to the + // file: rewriting a copy that differs would report a table config.toml + // does not have, and record a change no removal can undo. The same holds + // below, where a name counts as configured only while a table names it. + const auto fromTable = [&](const std::string& name) { + return std::ranges::any_of(repos, + [&](const IndexRepo& r) { return r.fromConfig && r.name == name; }); + }; + bool fileChanged = false, recordChanged = false; + for (auto const& r : repos) { + if (!r.fromConfig) continue; + const auto want = index_repo_entry(r); + auto* have = find(r.name); + if (have && *have == want) continue; + if (!record.contains(r.name)) + record[r.name] = { {"previous", have ? *have : nlohmann::json(nullptr)} }; + record[r.name]["written"] = want; + recordChanged = true; + if (have) *have = want; + else entries.insert(entries.begin(), want); + fileChanged = true; + if (!quiet) print_status("Index", describe_index_repo(r)); + } + for (auto it = record.begin(); it != record.end();) { + const std::string name = it.key(); + if (fromTable(name)) { ++it; continue; } + auto* have = find(name); + if (have && it.value().contains("written") && *have == it.value()["written"]) { + const auto& previous = it.value()["previous"]; + if (previous.is_null()) { + for (auto e = entries.begin(); e != entries.end(); ++e) + if (&*e == have) { entries.erase(e); break; } + } else { + *have = previous; + } + fileChanged = true; + if (!quiet) + print_status("Index", std::format( + "{} restored (its [index.repos.{}] table is no longer in config.toml)", + name, name)); + } + it = record.erase(it); + recordChanged = true; + } + if (fileChanged) (void)write_json_file(xjson, *doc); + if (recordChanged) { + std::error_code ec; + if (record.empty()) std::filesystem::remove(recordPath, ec); + else (void)write_json_file(recordPath, record); + } +} + // Migration helpers delegated to mcpp.fallback.config_migration. // canonicalize_legacy_index_names is exported (declared above, defined after // this helper namespace) so its ordering rules stay under unit test. @@ -540,6 +690,7 @@ std::expected load_or_init( auto it = tt.find("url"); if (it == tt.end() || !it->second.is_string()) continue; IndexRepo r{ name, it->second.as_string() }; + r.fromConfig = true; if (auto a = tt.find("artifact"); a != tt.end() && a->second.is_string()) r.artifact = a->second.as_string(); if (auto s = tt.find("source"); s != tt.end() && s->second.is_string()) @@ -591,8 +742,10 @@ std::expected load_or_init( auto xjson = cfg.xlingsHome() / ".xlings.json"; if (!std::filesystem::exists(xjson)) { write_default_xlings_json(xjson, cfg.indexRepos, initial_mirror); + record_seeded_index_repos(xjson, cfg.indexRepos, quiet); } else { mcpp::fallback::migrate_xlings_json_index_names(xjson); + reconcile_index_repos(xjson, cfg.indexRepos, quiet); } // 6. Acquire xlings binary if needed diff --git a/src/doctor.cppm b/src/doctor.cppm index 167b8a130..7cd87e482 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -1086,6 +1086,40 @@ export int why_report(const std::string& topic) { std::println("package index: {}", mcpp::pm::staleness_note(mcpp::config::make_xlings_env(*cfgW))); } + // The resolved graph first (#634, X). It is read back from the + // resolution record the preparation above just wrote, so what is + // printed is what a tool reading `resolution.json` sees, and a `path` + // dependency, which `mcpp.lock` does not record, is listed too. + std::println("dependency graph:"); + { + const auto recordPath = ctx->plan.outputDir / "resolution.json"; + std::ifstream record(recordPath); + auto doc = record ? nlohmann::json::parse(record, nullptr, false) + : nlohmann::json{}; + const auto graph = doc.is_object() ? doc.find("graph") : doc.end(); + if (!doc.is_object() || graph == doc.end() || !graph->is_object() + || !graph->contains("packages")) { + std::println(" (no graph in {})", recordPath.generic_string()); + } else { + for (auto const& entry : (*graph)["packages"]) { + auto const& pkg = entry["package"]; + const auto source = pkg.value("source", std::string{}); + std::println(" {}{}{}", + pkg.value("canonical", std::string("?")), + entry.value("root", false) ? " (root)" : "", + source.empty() ? std::string{} : " " + source); + for (auto const& r : entry.value("requested_by", nlohmann::json::array())) + std::println(" requested by {} as '{}' in {}", + r.value("requester", std::string("?")), + r.value("key", std::string("?")), + r.value("table", std::string("?"))); + if (auto link = entry.find("link"); link != entry.end()) + std::println(" linked {} ({})", + link->value("form", std::string("?")), + link->value("reason", std::string("?"))); + } + } + } std::println("dependencies (mcpp.lock):"); std::ifstream in(ctx->projectRoot / "mcpp.lock"); if (!in) { diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index 62169f998..5014465be 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -1293,11 +1293,21 @@ void resolve_graph(ScanResult& result) { if (u.provides) { auto [it, inserted] = g.producerOf.emplace(u.provides->logicalName, i); if (!inserted) { + // Name both packages: the same file reached as two packages + // and two packages that happen to pick one module name are + // different defects, and only the package names tell them + // apart. + auto const& first = g.units[it->second]; result.errors.push_back(ScanError{ u.path, 0, - std::format("module '{}' already provided by {}", + std::format("module '{}' is provided by package '{}' ({}) " + "and by package '{}' ({}){}", u.provides->logicalName, - g.units[it->second].path.string())}); + first.packageName, first.path.string(), + u.packageName, u.path.string(), + first.path == u.path + ? "; one file is reached as two packages" + : "")}); } } } diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 1ac5d0ad4..dda50d0d2 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -1,10 +1,12 @@ // mcpp.pack — bundle a built binary into a self-contained release archive. // -// TWO OUTPUT FAMILIES, ONE PIPELINE. An ELF/Mach-O artifact becomes a -// `.tar.gz` whose libraries live in `lib/` and are reached through a rewritten -// RUNPATH; a PE artifact becomes a `.zip` whose DLLs sit BESIDE the .exe, -// because that is where the Win32 loader looks and PE has no rpath to rewrite. -// Same contract, same modes, different mechanism — see +// TWO OUTPUT FAMILIES, ONE PIPELINE. An ELF artifact becomes a `.tar.gz` whose +// libraries live in `lib/` and are reached through a rewritten RUNPATH; a PE +// artifact becomes a `.zip` whose DLLs sit BESIDE the .exe, because that is +// where the Win32 loader looks and PE has no rpath to rewrite. A Mach-O program +// takes PE's placement inside the ELF family's archive: its dylibs sit beside +// it in `bin/`, where the `@loader_path` rpath it was linked with finds them +// (#634 A3). Same contract, same modes, different mechanism — see // .agents/docs/2026-08-16-windows-toolchain-three-axes-design.md §4. // // The PE path runs on ANY host. That is not a portability nicety: the ELF @@ -103,8 +105,9 @@ struct Options { std::string targetTriple; // empty = host // Where a dependency NAME may be resolved to a file. // - // Only used where the closure is read STATICALLY (PE): on ELF the loader - // hands back resolved paths and no search is performed here. Deliberately + // Only used where the closure is read STATICALLY (PE, the Android row): on + // the ELF host row the loader hands back resolved paths and no search is + // performed here, and a Mach-O name carries its own search rule. Deliberately // never the target's own system directories — a DLL that resolves only // there is the target's to provide, and copying one is a broken program // rather than a heavier one. @@ -113,6 +116,13 @@ struct Options { // toolset's `VC\Redist\MSVC\\\Microsoft.VC*.CRT\` for cl. // Searched ONLY under the toolchain-coupled contract — see make_plan. std::vector toolchainRuntimeDirs; + // #634 A3: the Android row's closure is read against the directories the + // row's own driver links from (`-print-search-dirs`), and a name found in + // `platformLibraryDirs` -- the directory the driver finds bionic's + // `libc.so` in, which is the API level's stub directory -- is the + // device's. Both are empty on every other row. + std::vector toolchainLibraryDirs; + std::vector platformLibraryDirs; // What the build placed relative to the executable, from // `runtime.deploy_files` and `runtime.deploy` (#615), as paths relative to // the executable's directory. Each is staged at the same relative path @@ -167,6 +177,18 @@ std::filesystem::path resolve_debug_dir(const Options& opts, const mcpp::manifest::PackConfig& cfg, const std::filesystem::path& projectRoot); +// #630 A9: one more leg of a several-triple application pack, built by +// `pack::pipeline::build_extra_android_legs`: the Android ABI it is staged +// under and the artifact that triple produced. #634 A3 adds where that leg's +// closure is read from, because each triple resolves its libraries against +// its own directories (see `Options::toolchainLibraryDirs`). +struct SharedLeg { + std::string abi; + std::filesystem::path artifact; + std::vector searchDirs; + std::vector platformDirs; +}; + // Resolved plan — all paths absolute, all decisions baked in. struct Plan { Options opts; @@ -211,16 +233,14 @@ struct Plan { // walk below, which asks the file to name its own needs by executing it. bool programIsSharedObject = false; // #630 A9: the OTHER triples this `app` was packed for, each already - // built by the caller (`pack::pipeline::build_extra_android_legs`) as - // (android abi, its artifact path). Set AFTER `make_plan`, the way - // `strip`/`debugDir` are: what the request came out as once more than - // one `--target` was resolved, which `make_plan` itself has no way to - // know from a single triple. Empty means "an ordinary single-triple - // pack", which is every caller before this item and keeps - // `run_shared_program`'s layout byte-identical for it. - std::vector> extraSharedLegs; - // The search set the PE closure resolves names against, after the - // contract has had its say (see make_plan). + // built by the caller (`pack::pipeline::build_extra_android_legs`). Set + // AFTER `make_plan`, the way `strip`/`debugDir` are: what the request came + // out as once more than one `--target` was resolved, which `make_plan` + // itself has no way to know from a single triple. Empty means "an ordinary + // single-triple pack", which stages the flat `lib/` layout. + std::vector extraSharedLegs; + // The search set the PE and Android closures resolve names against, after + // the contract has had its say (see make_plan). std::vector searchDirs; // ── debug information: the RESOLVED decision, not the request ───── // @@ -243,11 +263,78 @@ struct Error { std::string message; }; // hard refusal used to carry — moved from "before staging" to "step 4's // outcome", per the design record's decision (§3 of // 2026-09-13-630-what-a-framework-still-hits-in-the-engine.md). +// +// `needs` (#634 A3) is every name the closure resolved or failed to, for the +// stage manifest's `needs` lines; it is empty when no closure was read (a mode +// that bundles nothing, the wasm launcher). `walked` is true only when no +// entry of it is unresolved. struct ClosureResult { - bool walked = true; - std::string reason; + bool walked = true; + std::string reason; + std::vector needs; +}; + +// ─── the closure, read from the files (#634 A3) ─────────────────────────── +// +// THREE ROWS, ONE READER. PE, the Android shared-object row and Mach-O have +// their closure read from the files rather than traced by a loader, and they +// differ in two answers only: which names the target itself provides, and +// where a member is staged relative to the object that needs it. Those two +// answers are the rule; the walk is shared, and so are the refusal and the +// `needs` lines that the stagers in `run` derive from its result. +// +// Pe a name `binfmt::is_system_lib` knows (unless `[pack] +// force_bundle` names it), or one found in no search directory, +// is the target's -- as before this reader existed: a Windows +// component has no file to find on another host, and only the +// system directory holds it on a Windows one. +// Android a name found in a platform directory (the API level's stub +// directory the driver links against) is the device's; any other +// name must resolve in a search directory, or the closure is +// incomplete. +// MachO a name under `/usr/lib/` or `/System/Library/` is the OS's. +// Members are staged beside the program, so a name is a member only +// when the loader resolves it there: `@executable_path/`, +// `@loader_path/`, or `@rpath/` when an rpath of the +// image that needs it, or of the program, is exactly `@loader_path` +// or `@executable_path`. Any other name -- an absolute install name +// outside the OS's roots is read from that path on the target, +// whatever the tree carries -- leaves the closure incomplete. +enum class ClosureRule { Pe, Android, MachO }; + +struct ClosureReadInput { + std::filesystem::path object; // the program or application object + ClosureRule rule = ClosureRule::Pe; + std::vector searchDirs; // Pe, Android + std::vector platformDirs; // Android + std::vector forceBundle; // Pe: overrides the system list + std::string arch; // MachO: the slice of a fat file +}; + +// A name that resolved to a file, and where that file is staged: `dest` is +// relative to the directory the object itself is staged in. +struct ClosureMember { + std::string name; + std::filesystem::path source; + std::filesystem::path dest; }; +struct ClosureUnresolved { + std::string name; + std::string why; +}; + +struct ClosureRead { + std::vector members; // sorted by `dest` + std::vector platform; // sorted + std::vector unresolved; // sorted by name +}; + +// Read `in.object`'s closure under `in.rule`, transitively. Runs nothing; the +// only file-system access is reading the objects and asking whether a file +// exists. +ClosureRead read_closure(const ClosureReadInput& in); + // Build a Plan from already-resolved inputs. Caller is expected to have // already run `mcpp build` (or equivalent) and pass the resulting // binary path in. @@ -590,7 +677,15 @@ bool soname_matches(std::string_view soname, return false; } -std::expected, std::string> +// What the loader reported: the names it found a file for, and the names it +// found none for (#634 A3). A name in `notFound` used to be dropped here, so a +// bundle that could not start said `closure = walked`. +struct LddClosure { + std::vector found; + std::vector notFound; +}; + +std::expected ldd_parse(const std::filesystem::path& binary) { // Don't shell out to `ldd` directly — many distros (and our own @@ -606,7 +701,7 @@ ldd_parse(const std::filesystem::path& binary) auto out = run_capture(cmd); if (!out) return std::unexpected(out.error()); - std::vector deps; + LddClosure deps; std::istringstream is{*out}; std::string line; while (std::getline(is, line)) { @@ -626,8 +721,9 @@ ldd_parse(const std::filesystem::path& binary) // Trim "(0x...)" tail. if (auto paren = rest.find(" ("); paren != std::string::npos) rest = rest.substr(0, paren); - // "not found" → mcpp can't ship a lib it can't see. - if (rest == "not found") continue; + // "not found": mcpp cannot ship a library it cannot see, and the + // caller decides whether the mode leaves it to the target. + if (rest == "not found") { deps.notFound.push_back(d.soname); continue; } d.path = rest; } else if (line.starts_with('/')) { // Absolute-path line (typically the dynamic linker itself). @@ -639,7 +735,7 @@ ldd_parse(const std::filesystem::path& binary) } else { continue; } - deps.push_back(std::move(d)); + deps.found.push_back(std::move(d)); } return deps; } @@ -987,73 +1083,94 @@ stage_declared(const Plan& plan) // What `run` does with a reason the closure could not be resolved, as a // function of the plan's requested format — see `closure_unavailable_outcome` // for the two outcomes and why they differ. +// +// `needs` is what a closure that was read but is incomplete did resolve +// (#634 A3); a provider receives those lines with `closure = not-walked`. std::expected -finish_without_closure(const Plan& plan, std::string reason) +finish_without_closure(const Plan& plan, std::string reason, + std::vector needs = {}) { if (closure_unavailable_outcome(plan.opts.format) == ClosureUnavailableOutcome::StageWithoutClosure) - return ClosureResult{false, std::move(reason)}; + return ClosureResult{false, std::move(reason), std::move(needs)}; return std::unexpected(Error{std::move(reason)}); } -// ─── PE: the closure, read rather than executed ───────────────────────── -// -// BFS over the import tables, resolving each name against `searchDirs`. A -// name that resolves NOWHERE is deliberately not an error: on a Linux host -// `kernel32.dll` has no file to find, and on a Windows host it would resolve -// only in the system directory, which is not searched. Both are the same -// answer — the target provides it — and both are correct. -std::vector -pe_closure(const std::filesystem::path& binary, - std::span searchDirs, - const std::vector& forceBundle) +// The refusal text for a closure with unresolved names: one line per name, +// under the object it was read for, then what the row accepts as a way out. +// `hint` is that row's sentence, since what makes a name resolve differs: a +// directory the link declared (Android), the program's run-time search path +// (ELF), an install name the loader resolves beside the program (Mach-O). +std::string unresolved_reason(std::string_view object, + const std::vector& lines, + std::string_view hint) { - namespace bf = mcpp::pack::binfmt; - std::vector out; - std::set seen; // lowercased: PE names are not - std::vector queue{binary}; + std::string out = std::format( + "the dependency closure of '{}' is incomplete, so the staged tree would " + "not load where it is installed:\n", object); + for (auto const& l : lines) out += std::format(" {}\n", l); + out += std::format(" {}\n", hint); + out += " A library the target provides is named in " + "[pack.bundle-project] also_skip."; + return out; +} - auto lower = [](std::string_view s) { - std::string l(s); - std::ranges::transform(l, l.begin(), - [](unsigned char c) { return std::tolower(c); }); - return l; +// Copy a read closure's members under `dir`, the directory the object itself +// is staged in, and describe every name it read as a `needs` line whose staged +// path is relative to the staging root (#634 A3). `[pack] also_skip` keeps a +// member out of the tree and states it as the target's, as it always has; it +// is matched against a name's last path component too, so a Mach-O +// `@rpath/libfoo.dylib` answers to `libfoo`. `prefix` leads each refusal line +// (a several-ABI tree names the leg). +std::expected +stage_closure(const Plan& plan, const ClosureRead& read, + const std::filesystem::path& dir, + std::vector& needs, + std::vector& unresolvedLines, + std::string_view prefix) +{ + auto skipped = [&](const std::string& name) { + const auto leaf = std::filesystem::path(name).filename().string(); + const bool skip = soname_matches(name, plan.alsoSkipLibs) + || soname_matches(leaf, plan.alsoSkipLibs); + const bool force = soname_matches(name, plan.forceBundleLibs) + || soname_matches(leaf, plan.forceBundleLibs); + return skip && !force; }; - - while (!queue.empty()) { - auto current = queue.back(); - queue.pop_back(); - auto names = bf::needed_names(current); - if (!names) continue; // unreadable: nothing to add - for (auto const& name : *names) { - auto key = lower(name); - if (!seen.insert(key).second) continue; - // `[pack] force_bundle` is the escape hatch, and it has to reach - // the SYSTEM list too — on ELF it always did. Shipping a Windows - // component is normally a broken program rather than a heavier - // one, so this is a decision a human has to make explicitly; when - // they have, mcpp does not know better than them. - if (bf::is_system_lib(bf::Format::Pe, name) - && !soname_matches(name, forceBundle)) - continue; - std::error_code ec; - for (auto const& dir : searchDirs) { - auto cand = dir / name; - if (!std::filesystem::is_regular_file(cand, ec)) continue; - out.push_back({name, cand}); - // Transitive: a bundled DLL brings its own imports, and a - // closure that stops at depth one ships a package that starts - // failing one link further in. - queue.push_back(cand); - break; - } + for (auto const& name : read.platform) + needs.push_back({name, ClosureNeed::Kind::Platform, {}}); + for (auto const& m : read.members) { + if (skipped(m.name)) { + needs.push_back({m.name, ClosureNeed::Kind::Platform, {}}); + continue; } + const auto dst = dir / m.dest; + std::error_code ec; + std::filesystem::create_directories(dst.parent_path(), ec); + // The build may already have placed it where it is staged, in which + // case source and destination are the same file. + if (!std::filesystem::equivalent(m.source, dst, ec)) { + std::error_code cec; + std::filesystem::copy_file(m.source, dst, + std::filesystem::copy_options::overwrite_existing, cec); + if (cec) return std::unexpected(Error{std::format( + "failed to copy {} -> {}: {}", + m.source.string(), dst.string(), cec.message())}); + } + needs.push_back({m.name, ClosureNeed::Kind::Staged, + dst.lexically_relative(plan.stagingRoot).generic_string()}); } - std::sort(out.begin(), out.end(), - [](const ResolvedDep& a, const ResolvedDep& b) { - return a.soname < b.soname; - }); - return out; + for (auto const& u : read.unresolved) { + // Named as the target's: the tree need not carry what this machine + // does not have. + if (skipped(u.name)) { + needs.push_back({u.name, ClosureNeed::Kind::Platform, {}}); + continue; + } + needs.push_back({u.name, ClosureNeed::Kind::Unresolved, {}}); + unresolvedLines.push_back(std::format("{}{}: {}", prefix, u.name, u.why)); + } + return {}; } std::expected @@ -1086,6 +1203,213 @@ make_tarball(const std::filesystem::path& stagingRoot, } // namespace detail +ClosureRead read_closure(const ClosureReadInput& in) +{ + namespace bf = mcpp::pack::binfmt; + ClosureRead out; + + auto is_file = [](const std::filesystem::path& p) { + std::error_code ec; + return std::filesystem::is_regular_file(p, ec); + }; + auto lower = [](std::string_view s) { + std::string l(s); + std::ranges::transform(l, l.begin(), + [](unsigned char c) { return std::tolower(c); }); + return l; + }; + auto dir_list = [](const std::vector& dirs) { + std::string s; + for (auto const& d : dirs) s += (s.empty() ? "" : ", ") + d.string(); + return s.empty() ? std::string("(no directory)") : s; + }; + + // Names are unique per process on all three formats: a second request for + // one is the same library. PE compares them case-insensitively, as its + // loader does. + std::set seen; + + if (in.rule != ClosureRule::MachO) { + std::vector queue{in.object}; + while (!queue.empty()) { + auto current = queue.back(); + queue.pop_back(); + auto names = bf::needed_names(current); + if (!names) { + // The object itself was chosen by the caller, so an unreadable + // one is reported; a member that cannot be read was already + // staged and contributes nothing further. + if (current == in.object) + out.unresolved.push_back({current.filename().string(), names.error()}); + continue; + } + for (auto const& name : *names) { + auto key = in.rule == ClosureRule::Pe ? lower(name) : name; + if (!seen.insert(key).second) continue; + // `[pack] force_bundle` is the escape hatch, and it has to reach + // the SYSTEM list too -- on ELF it always did. Shipping a + // Windows component is normally a broken program rather than a + // heavier one, so this is a decision a human has to make + // explicitly; when they have, mcpp does not know better. + if (in.rule == ClosureRule::Pe + && bf::is_system_lib(bf::Format::Pe, name) + && !detail::soname_matches(name, in.forceBundle)) { + out.platform.push_back(name); + continue; + } + if (in.rule == ClosureRule::Android + && std::ranges::any_of(in.platformDirs, [&](auto const& d) { + return is_file(d / name); })) { + out.platform.push_back(name); + continue; + } + std::optional hit; + for (auto const& dir : in.searchDirs) + if (is_file(dir / name)) { hit = dir / name; break; } + if (hit) { + out.members.push_back({name, *hit, std::filesystem::path(name)}); + // Transitive: a staged library brings its own needs, and a + // closure that stops at depth one ships a package that + // starts failing one load further in. + queue.push_back(*hit); + continue; + } + if (in.rule == ClosureRule::Pe) { + // Found nowhere: the target provides it (see the rule). + out.platform.push_back(name); + continue; + } + out.unresolved.push_back({name, std::format( + "found in none of {}", dir_list(in.searchDirs))}); + } + } + } else { + struct Image { + std::filesystem::path source; // on this machine + std::filesystem::path stagedDir; // relative to the program's directory + std::vector names; + std::vector rpaths; + }; + auto first = bf::macho_needed(in.object, in.arch); + if (!first) { + out.unresolved.push_back({in.object.filename().string(), first.error()}); + return out; + } + const auto programDir = in.object.parent_path(); + const auto programRpaths = first->rpaths; + std::set seenDest; + std::vector queue{ + Image{in.object, {}, std::move(first->names), std::move(first->rpaths)}}; + + // `@loader_path/` -> ``; nullopt when `s` does not begin + // with the token followed by a separator. + auto after = [](std::string_view s, std::string_view token) + -> std::optional { + if (s.size() > token.size() + 1 && s.starts_with(token) + && s[token.size()] == '/') + return std::string(s.substr(token.size() + 1)); + return std::nullopt; + }; + // An rpath entry that places a library in the carrying image's own + // directory (`@loader_path`) or the program's (`@executable_path`). + auto is_exactly = [](std::string_view rp, std::string_view token) { + return rp == token + || (rp.size() == token.size() + 1 && rp.starts_with(token) + && rp.back() == '/'); + }; + + while (!queue.empty()) { + auto image = std::move(queue.back()); + queue.pop_back(); + const auto imageDir = image.source.parent_path(); + for (auto const& name : image.names) { + if (bf::is_system_lib(bf::Format::MachO, name)) { + if (seen.insert(name).second) out.platform.push_back(name); + continue; + } + // WHERE THE LOADER FINDS IT ON THIS MACHINE, by dyld's own + // substitutions (`resolve_macho_names`): the needing image's + // rpaths first, then the program's, which every image it loads + // inherits. + std::optional source; + const std::array one{name}; + auto own = bf::resolve_macho_names(one, image.rpaths, programDir, imageDir); + if (!own.front().unresolved) source = own.front().path; + if (!source && image.source != in.object && name.starts_with("@rpath/")) { + auto inherited = bf::resolve_macho_names(one, programRpaths, + programDir, programDir); + if (!inherited.front().unresolved) source = inherited.front().path; + } + // `resolve_macho_names` asks only whether a path exists; a + // member is a file to copy. + if (source && !is_file(*source)) source.reset(); + + // WHERE A COPY BESIDE THE PROGRAM IS WHAT THE LOADER FINDS. + std::optional dest; + std::string why; + if (auto rest = after(name, "@executable_path")) { + dest = std::filesystem::path(*rest).lexically_normal(); + } else if (auto rest = after(name, "@loader_path")) { + dest = (image.stagedDir / *rest).lexically_normal(); + } else if (auto rest = after(name, "@rpath")) { + for (auto const& rp : image.rpaths) { + if (is_exactly(rp, "@loader_path")) + { dest = (image.stagedDir / *rest).lexically_normal(); break; } + if (is_exactly(rp, "@executable_path")) + { dest = std::filesystem::path(*rest).lexically_normal(); break; } + } + if (!dest && image.source != in.object) { + for (auto const& rp : programRpaths) { + if (is_exactly(rp, "@loader_path") || is_exactly(rp, "@executable_path")) + { dest = std::filesystem::path(*rest).lexically_normal(); break; } + } + } + if (!dest) + why = "no rpath of the image that needs it is exactly " + "@loader_path or @executable_path, so a copy beside " + "the program is not where the loader looks"; + } else if (name.starts_with("/")) { + why = "an absolute install name outside /usr/lib and " + "/System/Library, which the loader reads from that path " + "rather than from the staged tree"; + } else { + why = "an install name with no loader token"; + } + if (dest && (dest->empty() || dest->begin()->string() == "..")) { + dest.reset(); + why = "it names a location outside the program's directory"; + } + if (dest && !source) + why = "the loader finds no file for it on this machine"; + + if (dest && source) { + if (!seenDest.insert(dest->generic_string()).second) continue; + out.members.push_back({name, *source, *dest}); + if (auto sub = bf::macho_needed(*source, in.arch)) { + queue.push_back(Image{*source, dest->parent_path(), + std::move(sub->names), + std::move(sub->rpaths)}); + } + continue; + } + if (seen.insert(name).second) out.unresolved.push_back({name, why}); + } + } + } + + std::ranges::sort(out.members, [](const ClosureMember& a, const ClosureMember& b) { + return a.dest.generic_string() < b.dest.generic_string(); + }); + std::ranges::sort(out.platform); + out.platform.erase(std::unique(out.platform.begin(), out.platform.end()), + out.platform.end()); + std::ranges::sort(out.unresolved, [](const ClosureUnresolved& a, + const ClosureUnresolved& b) { + return a.name < b.name; + }); + return out; +} + namespace detail { // The PE half of `run`. Flat layout, deliberately: the Win32 loader resolves @@ -1118,7 +1442,7 @@ namespace detail { // namespace comment above, and hostflags.cppm on a neighbouring function // being miscompiled by an unrelated addition). Each replacement is also // simpler than what it replaced, so nothing is being paid for the avoidance. -std::expected +std::expected, Error> run_pe(const Plan& plan) { std::error_code ec; @@ -1132,7 +1456,8 @@ run_pe(const Plan& plan) std::filesystem::copy_options::overwrite_existing, ec); if (ec) return std::unexpected(Error{std::format( "copy binary failed: {}", ec.message())}); - if (auto r = stage_runtime_files(plan, stagedExe.parent_path()); !r) return r; + if (auto r = stage_runtime_files(plan, stagedExe.parent_path()); !r) + return std::unexpected(r.error()); copy_if_exists(plan.projectRoot / "README.md", plan.stagingRoot); copy_if_exists(plan.projectRoot / "LICENSE", plan.stagingRoot); @@ -1143,7 +1468,7 @@ run_pe(const Plan& plan) out << render(plan.hostRequirements); } - std::vector deps; + std::vector needs; if (plan.opts.mode != Mode::None && plan.opts.mode != Mode::Static) { // `vendored` and `self-contained` collect the same set here, and that // is a property of the PLATFORM rather than a simplification. @@ -1153,27 +1478,26 @@ run_pe(const Plan& plan) // have two of something that must be unique. So the ceiling on // "everything" is the same for both modes: every dependency mcpp is // ALLOWED to carry. - deps = pe_closure(stagedExe, plan.searchDirs, plan.forceBundleLibs); - for (auto const& d : deps) { - if (soname_matches(d.soname, plan.alsoSkipLibs) - && !soname_matches(d.soname, plan.forceBundleLibs)) - continue; - auto dst = plan.stagingRoot / d.soname; - // The build may already have staged it beside the artifact, in - // which case source and destination are the same file. - std::error_code cec; - if (std::filesystem::equivalent(d.path, dst, cec)) continue; - std::filesystem::copy_file(d.path, dst, - std::filesystem::copy_options::overwrite_existing, cec); - if (cec) return std::unexpected(Error{std::format( - "failed to copy {} → {}: {}", - d.path.string(), dst.string(), cec.message())}); - } + ClosureReadInput in; + in.object = stagedExe; + in.rule = ClosureRule::Pe; + in.searchDirs = plan.searchDirs; + in.forceBundle = plan.forceBundleLibs; + const auto read = read_closure(in); + // PE's rule resolves every name (see `ClosureRule`); what can remain + // is an executable the reader could not parse, which no format can + // package. + std::vector unresolved; + if (auto r = stage_closure(plan, read, plan.stagingRoot, needs, unresolved, {}); !r) + return std::unexpected(r.error()); + if (!unresolved.empty()) + return std::unexpected(Error{unresolved_reason(plan.binaryName, unresolved, + "The executable's import table could not be read.")}); } - if (auto r = strip_program(plan, stagedExe); !r) return r; + if (auto r = strip_program(plan, stagedExe); !r) return std::unexpected(r.error()); - if (plan.opts.format != Format::Tar) return {}; + if (plan.opts.format != Format::Tar) return needs; std::vector entries; const auto wrapper = plan.stagingRoot.filename().string(); @@ -1202,7 +1526,7 @@ run_pe(const Plan& plan) const mcpp::pack::zip::Entry& b) { return a.name < b.name; }); if (auto r = mcpp::pack::zip::write(plan.archivePath, entries); !r) return std::unexpected(Error{r.error()}); - return {}; + return needs; } // The wasm32-emscripten half of `run`. No dependency closure: the ordinary @@ -1264,20 +1588,25 @@ run_wasm(const Plan& plan) } // #622 A3/A10: the Application half whose form on this row is a shared -// object (`*-linux-android`). No dependency closure: the ELF closure below -// asks the file to name its own needs by executing it under -// `LD_TRACE_LOADED_OBJECTS`, and a cross-compiled Android object names an -// interpreter this host does not have (`/system/bin/linker64`) -- it is not -// runnable here at all, on any host architecture. The file is staged the way -// a dependency .so is staged below (`lib/`), because that is where a closure -// conventionally puts a shared object; a provider that wants the object's -// own dependency set bundled (`dist-apk`) reads `${mcpp.target_file:}` -// and resolves that itself, out of the engine's closure entirely. +// object (`*-linux-android`). The object is staged under `lib/`, where every +// Android packaging tool expects native libraries, and it is never executed: +// a cross-compiled Android object names an interpreter this host does not have +// (`/system/bin/linker64`). // -// #630 A9: one triple stages flat (`lib/.so`, unchanged); more than -// one triple stages one `lib//.so` per leg into the SAME tree — -// see `Plan::extraSharedLegs`. -std::expected +// #634 A3: ITS CLOSURE IS READ AND STAGED BESIDE IT. It used to be left to a +// provider (`dist-apk` walked `DT_NEEDED` itself), which meant the engine's +// own `dir` and `tar` shipped an object without the libraries it loads and said +// `closure = walked`. The rule is `ClosureRule::Android`: a name the API +// level's stub directory holds is the device's, and every other name -- +// a graph-built dependency, the NDK's `libc++_shared.so` -- travels. The mode +// does not apply on this row: the device provides exactly the platform's +// libraries, so there is no `system` set to leave out and no loader or libc to +// carry. +// +// #630 A9: one triple stages flat (`lib/.so`); more than one triple +// stages one `lib//` per leg into the SAME tree, each with its own +// closure — see `Plan::extraSharedLegs`. +std::expected run_shared_program(const Plan& plan) { std::error_code ec; @@ -1286,38 +1615,67 @@ run_shared_program(const Plan& plan) if (ec) return std::unexpected(Error{std::format( "cannot create staging '{}': {}", plan.stagingRoot.string(), ec.message())}); + struct Leg { + std::filesystem::path dir; // where the object is staged + std::string abi; // empty on the flat layout + std::filesystem::path artifact; + std::vector searchDirs; + std::vector platformDirs; + }; + auto primarySearch = plan.searchDirs; + for (auto const& d : plan.opts.toolchainLibraryDirs) primarySearch.push_back(d); + // #630 A9: MORE THAN ONE TRIPLE MEANS MORE THAN ONE `lib//`, since a // flat `lib/.so` cannot hold two architectures' bytes under one // name. `extraSharedLegs` is non-empty ONLY when the caller is the // several-`--target` route (`cmd_pack`, via `build_and_pack`'s trailing - // parameter) — every pre-existing single-triple caller leaves it empty - // and keeps the flat layout below byte-identical to before this item. + // parameter). + std::vector legs; if (!plan.extraSharedLegs.empty()) { - auto stage_leg = [&](std::string_view abi, const std::filesystem::path& artifact) - -> std::expected - { - auto dir = plan.stagingRoot / "lib" / abi; - std::error_code dec; - std::filesystem::create_directories(dir, dec); - if (dec) return std::unexpected(Error{std::format( - "cannot create staging '{}': {}", dir.string(), dec.message())}); - std::filesystem::copy_file(artifact, dir / plan.binaryName, - std::filesystem::copy_options::overwrite_existing, dec); - if (dec) return std::unexpected(Error{std::format( - "copy binary failed: {}", dec.message())}); - return {}; - }; auto t = mcpp::toolchain::triple::parse(plan.triple); auto primaryAbi = t ? mcpp::toolchain::triple::android_abi(*t) : plan.triple; - if (auto r = stage_leg(primaryAbi, plan.builtBinary); !r) return r; - for (auto const& [legAbi, legArtifact] : plan.extraSharedLegs) - if (auto r = stage_leg(legAbi, legArtifact); !r) return r; + legs.push_back({plan.stagingRoot / "lib" / primaryAbi, primaryAbi, + plan.builtBinary, primarySearch, plan.opts.platformLibraryDirs}); + for (auto const& leg : plan.extraSharedLegs) + legs.push_back({plan.stagingRoot / "lib" / leg.abi, leg.abi, leg.artifact, + leg.searchDirs, leg.platformDirs}); } else { - auto staged = plan.stagingRoot / "lib" / plan.binaryName; - std::filesystem::copy_file(plan.builtBinary, staged, - std::filesystem::copy_options::overwrite_existing, ec); - if (ec) return std::unexpected(Error{std::format( - "copy binary failed: {}", ec.message())}); + legs.push_back({plan.stagingRoot / "lib", {}, plan.builtBinary, primarySearch, + plan.opts.platformLibraryDirs}); + } + + std::vector needs; + std::vector unresolved; + for (auto const& leg : legs) { + std::error_code dec; + std::filesystem::create_directories(leg.dir, dec); + if (dec) return std::unexpected(Error{std::format( + "cannot create staging '{}': {}", leg.dir.string(), dec.message())}); + std::filesystem::copy_file(leg.artifact, leg.dir / plan.binaryName, + std::filesystem::copy_options::overwrite_existing, dec); + if (dec) return std::unexpected(Error{std::format( + "copy binary failed: {}", dec.message())}); + + const std::string prefix = leg.abi.empty() ? std::string{} : leg.abi + ": "; + // WITHOUT A PLATFORM DIRECTORY EVERY NAME WOULD BE A MEMBER: bionic's + // `libc.so` stub is on the driver's search path too, and staging it + // would ship a library with no code in it. The driver is what names + // the directory, so a driver that did not answer is reported. + if (leg.platformDirs.empty()) { + unresolved.push_back(prefix + std::format( + "the driver named no directory holding the platform's libc.so, so " + "the device's libraries cannot be told from the ones '{}' carries", + plan.binaryName)); + continue; + } + ClosureReadInput in; + in.object = leg.artifact; + in.rule = ClosureRule::Android; + in.searchDirs = leg.searchDirs; + in.platformDirs = leg.platformDirs; + const auto read = read_closure(in); + if (auto r = stage_closure(plan, read, leg.dir, needs, unresolved, prefix); !r) + return std::unexpected(r.error()); } // THE RUNTIME FILES TRAVEL AS ON EVERY OTHER ROW. `deploy` placed them @@ -1327,13 +1685,25 @@ run_shared_program(const Plan& plan) // 2026-09-12: without this the Android staged tree carried the library // alone and a deploy'd resource never reached the APK. if (!plan.opts.runtimeFiles.empty()) - if (auto r = stage_runtime_files(plan, plan.stagingRoot / "bin"); !r) return r; + if (auto r = stage_runtime_files(plan, plan.stagingRoot / "bin"); !r) + return std::unexpected(r.error()); copy_if_exists(plan.projectRoot / "README.md", plan.stagingRoot); copy_if_exists(plan.projectRoot / "LICENSE", plan.stagingRoot); - if (plan.opts.format != Format::Tar) return {}; - return make_tarball(plan.stagingRoot, plan.archivePath); + if (!unresolved.empty()) + return finish_without_closure( + plan, + unresolved_reason(plan.binaryName, unresolved, + "A library the link used is found through the build's output " + "directory, [runtime] library_dirs\n" + " or link_library_dirs, or the compiler's library search path."), + std::move(needs)); + + if (plan.opts.format == Format::Tar) + if (auto r = make_tarball(plan.stagingRoot, plan.archivePath); !r) + return std::unexpected(r.error()); + return ClosureResult{true, {}, std::move(needs)}; } } // namespace detail @@ -1351,19 +1721,16 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // closure walk that follows and try to execute an object this host // cannot load at all. See the field comment on `programIsSharedObject`. // - // These three dispatchers read their own closure (PE) or need none at - // all (the Android shared-object row; the wasm32-emscripten launcher, a - // static image with everything embedded) — none of them is the "declare - // before discover" reorder below, so each reports a walked closure on - // success. - if (plan.programIsSharedObject) { - if (auto r = detail::run_shared_program(plan); !r) return std::unexpected(r.error()); - return ClosureResult{}; - } + // These three dispatchers stage their own tree: PE and the Android + // shared-object row read their closure from the files (#634 A3), and the + // wasm32-emscripten launcher is a static image with everything embedded. + // None of them is the "declare before discover" reorder below. + if (plan.programIsSharedObject) return detail::run_shared_program(plan); if (plan.targetIsPe) { - if (auto r = detail::run_pe(plan); !r) return std::unexpected(r.error()); - return ClosureResult{}; + auto r = detail::run_pe(plan); + if (!r) return std::unexpected(r.error()); + return ClosureResult{true, {}, std::move(*r)}; } // wasm32-emscripten, before the Mach-O refusal and the ELF closure below: @@ -1388,53 +1755,51 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) if (!staged) return std::unexpected(staged.error()); auto bundledBinary = *staged; - // STEP 4, first mechanism: a Mach-O artifact's closure is never walked, - // on every host including macOS. + // STEP 4 FOR A MACH-O PROGRAM: ITS CLOSURE IS READ, ON EVERY HOST (#634 A3). // - // The closure below asks the dynamic linker for the dependency list by + // The ELF closure below asks the dynamic linker for the dependency list by // running the artifact with `LD_TRACE_LOADED_OBJECTS=1`. That variable - // belongs to glibc's ld.so; dyld has never heard of it (its counterpart is - // `DYLD_PRINT_LIBRARIES`), so on macOS the command does not trace anything - // — IT RUNS THE USER'S PROGRAM. Whatever that program prints is then parsed - // as a dependency table, which yields nothing, and the bundle is written - // and reported as `Packed`. A program with side effects performs them; an - // interactive one hangs the packer. + // belongs to glibc's ld.so; dyld has never heard of it, so on macOS the + // command would RUN THE USER'S PROGRAM and parse whatever it printed. So a + // Mach-O program never reaches that path: its load commands are read + // (`ClosureRule::MachO`), which needs no loader and therefore no macOS + // host either, and its dylibs are staged beside it in `bin/`, where the + // `@loader_path` rpath a consumer of a graph-built dylib links with + // already finds them. No load command is edited and nothing is re-signed, + // which is also why the program is not stripped on this row. // - // ASKED OF THE FORMAT, NOT OF THE HOST — the same correction the `_WIN32` - // branch below already carries. `LD_TRACE_LOADED_OBJECTS` cannot trace a - // Mach-O from Linux either, and a macOS host is not the thing that makes - // this impossible. - // - // docs/02 lists macOS bundling under "Planned Support"; until it lands, - // saying so is strictly better than producing an empty bundle that claims - // to be one. - // - // NAMES THE ARTIFACT. Not decoration: `mcpp pack ` routes on the - // target's kind, and a refusal that does not say WHICH program it got to is - // indistinguishable from one that resolved the wrong target — which is the - // exact defect `route_pack_target` exists to prevent. It is also the only - // way an e2e can check that routing on macOS, where no program bundle can - // be produced to inspect. - // - // THIS USED TO BE A HARD REFUSAL BEFORE ANY STAGING RAN. It is now step - // 4's outcome: the reason text is unchanged (it is accurate), but for a - // dispatched format the tree staged above is handed to the provider - // regardless — see `finish_without_closure`. + // THE BUILT PROGRAM IS READ, NOT THE STAGED COPY: `@loader_path` must + // expand to the directory the dylibs were built into. if (mcpp::pack::binfmt::identify(plan.builtBinary).format == mcpp::pack::binfmt::Format::MachO) { - return finish_without_closure(plan, std::format( - "cannot package the Mach-O program '{}' yet.\n", plan.binaryName) + - " The dependency closure for that format is resolved by running the " - "artifact under\n" - " the target's own dynamic linker, and the mechanism mcpp uses " - "(LD_TRACE_LOADED_OBJECTS)\n" - " is glibc's — dyld ignores it and simply RUNS the program, which is " - "why this is\n" - " refused rather than attempted.\n" - " A `kind = \"lib\"` / `\"shared\"` target packs normally on macOS " - "(`mcpp pack `);\n" - " for a program, ship the build tree or use a platform bundler until " - "macOS support lands."); + std::vector needs; + std::vector unresolved; + if (plan.opts.mode != Mode::None && plan.opts.mode != Mode::Static) { + ClosureReadInput in; + in.object = plan.builtBinary; + in.rule = ClosureRule::MachO; + auto t = mcpp::toolchain::triple::parse(plan.triple); + in.arch = t ? t->arch : std::string(mcpp::platform::host_arch); + const auto read = read_closure(in); + if (auto r = stage_closure(plan, read, plan.stagingRoot / "bin", needs, + unresolved, {}); !r) + return std::unexpected(r.error()); + } + if (!unresolved.empty()) + return finish_without_closure( + plan, + unresolved_reason(plan.binaryName, unresolved, + "A dylib is staged beside the program when its install name " + "is @rpath/ and the program\n" + " carries an @loader_path or @executable_path rpath, or " + "when it is @loader_path/."), + std::move(needs)); + if (auto r = write_topentry_wrapper(plan.stagingRoot, plan.binaryName); !r) + return std::unexpected(Error{r.error()}); + if (plan.opts.format == Format::Tar) + if (auto r = make_tarball(plan.stagingRoot, plan.archivePath); !r) + return std::unexpected(r.error()); + return ClosureResult{true, {}, std::move(needs)}; } #if defined(_WIN32) @@ -1473,6 +1838,10 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // BundleAll — ship every dep including the dynamic linker; entry // point becomes `run.sh` which invokes the bundled ld with // --library-path → fully portable across glibc versions. + // #634 A3: what step 4 resolved, for the stage manifest, and the refusal + // text when it is incomplete. Both stay empty under `--mode static`. + std::vector needs; + std::string unresolvedText; if (plan.opts.mode != Mode::Static) { // THE BUILT BINARY, NOT THE STAGED COPY, and the difference is // `$ORIGIN`. @@ -1524,19 +1893,59 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) if (!deps) return std::unexpected(Error{std::format( "ldd failed on {}: {}", plan.builtBinary.string(), deps.error())}); - std::vector toBundle; - for (auto& d : *deps) { + auto skipped_by_mode = [&](std::string_view soname) { bool skip = false; if (plan.opts.mode == Mode::None) { skip = true; // system: host provides every .so, bundle nothing } else if (plan.opts.mode == Mode::BundleProject) { - if (is_system_lib(d.soname)) skip = true; - if (soname_matches(d.soname, plan.alsoSkipLibs)) skip = true; - if (soname_matches(d.soname, plan.forceBundleLibs)) skip = false; // override + if (is_system_lib(soname)) skip = true; + if (soname_matches(soname, plan.alsoSkipLibs)) skip = true; + if (soname_matches(soname, plan.forceBundleLibs)) skip = false; // override } // Mode::BundleAll: skip nothing — we want the loader too. - if (!skip) toBundle.push_back(d); + return skip; + }; + + // #634 A3: the `needs` lines, from the same decisions that fill the + // tree. A mode that bundles nothing states no closure, as on every + // other row. + const bool bundling = plan.opts.mode != Mode::None; + std::vector toBundle; + for (auto& d : deps->found) { + if (skipped_by_mode(d.soname)) { + if (bundling) needs.push_back({d.soname, ClosureNeed::Kind::Platform, {}}); + continue; + } + toBundle.push_back(d); + needs.push_back({d.soname, ClosureNeed::Kind::Staged, "lib/" + d.soname}); } + // A name the loader found no file for is one the tree cannot carry. + // When the mode leaves it to the target anyway it is the target's; + // otherwise the closure is incomplete, and an archive that says + // `walked` would not start where it is installed. + std::vector unresolved; + for (auto const& name : deps->notFound) { + if (!bundling) continue; + // `also_skip` names the target's libraries in every bundling mode + // here, `self-contained` included: the tree cannot carry a file + // this machine does not have. + if (skipped_by_mode(name) || (soname_matches(name, plan.alsoSkipLibs) + && !soname_matches(name, plan.forceBundleLibs))) { + needs.push_back({name, ClosureNeed::Kind::Platform, {}}); + continue; + } + needs.push_back({name, ClosureNeed::Kind::Unresolved, {}}); + unresolved.push_back(name + ": the loader finds no file for it"); + } + if (!unresolved.empty()) + unresolvedText = unresolved_reason(plan.binaryName, unresolved, + "The loader looks through the program's run-time search path " + "(DT_RPATH, DT_RUNPATH) and\n" + " its own default directories."); + // An archive refuses before any file is edited; a dispatched format + // still receives the tree the resolved names allow. + if (!unresolved.empty() && plan.opts.format != Format::Dispatched) + return finish_without_closure(plan, unresolvedText, needs); if (auto r = bundle_libs(toBundle, plan.stagingRoot); !r) return std::unexpected(Error{r.error()}); @@ -1615,7 +2024,7 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // ignores it and launches via the bundled // loader directly. if (plan.opts.mode == Mode::BundleProject || plan.opts.mode == Mode::None) { - if (auto soname = find_loader_soname(*deps); !soname.empty()) { + if (auto soname = find_loader_soname(deps->found); !soname.empty()) { auto distroLoader = (soname == "ld-linux-x86-64.so.2" ? "/lib64/" : "/lib/") + soname; @@ -1664,12 +2073,17 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // at its leg loop. if (auto r = strip_program(plan, bundledBinary); !r) return std::unexpected(r.error()); + // A dispatched format reaches this point with an incomplete closure; an + // archive was refused before any file was edited. + if (!unresolvedText.empty()) + return finish_without_closure(plan, unresolvedText, std::move(needs)); + // Output. if (plan.opts.format == Format::Tar) { if (auto r = make_tarball(plan.stagingRoot, plan.archivePath); !r) return std::unexpected(r.error()); } - return ClosureResult{}; + return ClosureResult{true, {}, std::move(needs)}; #endif // !_WIN32 } diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 7d10ff49e..8da9ea255 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -21,10 +21,14 @@ import mcpp.manifest; import mcpp.pack; import mcpp.pack.stage_tree; import mcpp.pack.strip; +import mcpp.platform; +import mcpp.platform.env; import mcpp.toolchain.model; +import mcpp.toolchain.probe; import mcpp.toolchain.registry; import mcpp.toolchain.triple; import mcpp.ui; +import mcpp.xlings; namespace mcpp::pack { @@ -41,6 +45,56 @@ export struct PackOutcome { std::vector artifacts; }; +// #634 A3: the two directory lists an Android row's closure is read against, +// asked of the row's own driver rather than derived from the NDK's layout: +// where the link searches for a library (`-print-search-dirs`), and the +// directory the driver finds bionic's `libc.so` in, which is the API level's +// stub directory -- the set of names the device itself provides. +// +// A driver that answers neither leaves both lists empty; `pack::run` then +// reports that it cannot tell the device's libraries from the program's rather +// than staging stubs. +struct DriverLibraryDirs { + std::vector search; + std::vector platform; +}; + +DriverLibraryDirs driver_library_dirs(const mcpp::toolchain::Toolchain& tc) +{ + DriverLibraryDirs out; + const auto base = std::format("{} {}", mcpp::xlings::shq(tc.binaryPath.string()), + tc.crossTargetFlag); + if (auto r = mcpp::toolchain::run_capture(std::format( + "{} -print-search-dirs {}", base, mcpp::platform::null_redirect))) { + std::istringstream is{*r}; + std::string line; + const std::string_view key = "libraries: ="; + const auto sep = mcpp::platform::env::path_list_separator(); + while (std::getline(is, line)) { + if (!line.starts_with(key)) continue; + std::string rest = mcpp::toolchain::trim_line(line.substr(key.size())); + std::size_t at = 0; + while (at <= rest.size()) { + auto next = rest.find(sep, at); + if (next == std::string::npos) next = rest.size(); + if (next > at) + out.search.push_back( + std::filesystem::path(rest.substr(at, next - at)).lexically_normal()); + at = next + sep.size(); + } + } + } + if (auto r = mcpp::toolchain::run_capture(std::format( + "{} -print-file-name=libc.so {}", base, mcpp::platform::null_redirect))) { + // A driver that cannot place the library echoes the bare name back. + std::filesystem::path lib(mcpp::toolchain::trim_line(*r)); + std::error_code ec; + if (lib.has_parent_path() && std::filesystem::is_regular_file(lib, ec)) + out.platform.push_back(lib.parent_path().lexically_normal()); + } + return out; +} + // #630 A9: build every triple but the PRIMARY one for a `kind = "app"` // target whose form is a shared object on every requested row (Android). // Each leg is its own `prepare_build` + `ninja` build, the same shape @@ -55,12 +109,12 @@ export struct PackOutcome { // Returns `nullopt` on the first leg that fails, having already printed the // error — the same contract `build_and_pack_library` and `build_and_pack` // use, so `cmd_pack` only has to check the outcome and return. -export std::optional>> +export std::optional> build_extra_android_legs(const std::string& targetName, std::span triples, const std::string& profile) { - std::vector> out; + std::vector out; for (auto const& triple : triples) { mcpp::build::BuildOverrides ov; ov.target_triple = triple; @@ -107,7 +161,24 @@ build_extra_android_legs(const std::string& targetName, auto canonical = t ? t->str() : triple; auto abi = t ? mcpp::toolchain::triple::android_abi(*t) : triple; mcpp::ui::status("Packed leg", std::format("{} [{}]", canonical, abi)); - out.emplace_back(std::move(abi), ctx->outputDir / lu->output); + // #634 A3: where THIS triple's closure is read from -- the same + // channels the primary leg's plan names (`build_and_pack` below), from + // this leg's own plan and driver. + SharedLeg leg; + leg.abi = std::move(abi); + leg.artifact = ctx->outputDir / lu->output; + leg.searchDirs.push_back(leg.artifact.parent_path()); + for (auto const& d : ctx->plan.runtimeLibraryDirs) leg.searchDirs.push_back(d); + for (auto const& d : ctx->plan.linkIntent.runtimeSearchDirs) + leg.searchDirs.push_back(d); + for (auto const& d : ctx->plan.linkIntent.linkLibraryDirs) + leg.searchDirs.push_back(d); + for (auto const& d : ctx->plan.linkIntent.transitiveNeededDirs) + leg.searchDirs.push_back(d); + auto dirs = driver_library_dirs(ctx->tc); + for (auto const& d : dirs.search) leg.searchDirs.push_back(d); + leg.platformDirs = std::move(dirs.platform); + out.push_back(std::move(leg)); } return out; } @@ -122,19 +193,16 @@ build_extra_android_legs(const std::string& targetName, // is wrong. // // `extraLegs` (#630 A9) is every OTHER triple a several-`--target` app-pack -// request named, already built by `build_extra_android_legs` below as -// (android abi, its artifact path). This call still does exactly one build — -// of `opts.targetTriple`, the PRIMARY leg — and stages the primary's own -// artifact and the declared deploy files as it always has; `extraLegs`, when -// non-empty, only changes WHERE the primary's shared object lands -// (`Plan::extraSharedLegs`, read by `run_shared_program`) and adds the other -// legs beside it in the same staged tree, before the one dispatch pass runs. -// Empty for every caller before this item, which is what keeps a -// single-`--target` pack byte-identical. +// request named, already built by `build_extra_android_legs` above. This call +// still does exactly one build — of `opts.targetTriple`, the PRIMARY leg — and +// stages the primary's own artifact and the declared deploy files as it always +// has; `extraLegs`, when non-empty, only changes WHERE the primary's shared +// object lands (`Plan::extraSharedLegs`, read by `run_shared_program`) and adds +// the other legs, each with its own closure, beside it in the same staged tree, +// before the one dispatch pass runs. Empty for a single-`--target` pack. export PackOutcome build_and_pack(Options opts, bool modeFromUser, const std::string& wantTarget = {}, - std::vector> - extraLegs = {}) { + std::vector extraLegs = {}) { // `--target *-linux-musl` without an explicit `--mode` implies // `--mode static` — packaging a musl-static ELF as bundle-project // would feed patchelf a static binary and crash. The docs treat @@ -373,6 +441,19 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, // destinations are `bin//`, and the executable is in `bin/`. for (auto const& d : ctx->plan.runtimeDeployFiles) opts.runtimeFiles.push_back(d.dest.lexically_relative("bin")); + // #634 A3: the Android row reads its closure against the directories + // its link declared -- a prebuilt library named through `[runtime] + // link_library_dirs` is a file the link used and the device does not + // have -- and against its driver's; see `driver_library_dirs`. + if (programIsSharedObject) { + for (auto const& d : ctx->plan.linkIntent.linkLibraryDirs) + opts.depSearchDirs.push_back(d); + for (auto const& d : ctx->plan.linkIntent.transitiveNeededDirs) + opts.depSearchDirs.push_back(d); + auto dirs = driver_library_dirs(ctx->tc); + opts.toolchainLibraryDirs = std::move(dirs.search); + opts.platformLibraryDirs = std::move(dirs.platform); + } } // ─── Build the plan + run ──────────────────────────────────────── @@ -458,7 +539,7 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, } else if (!r->walked) { // The tree exists; only its dependency closure does not. Distinct // warning text -- "staged" is true here, unlike the branch above. - closure = mcpp::pack::ClosureStatus{false, r->reason}; + closure = mcpp::pack::ClosureStatus{false, r->reason, r->needs}; mcpp::ui::warning(std::format( "staged without its dependency closure: {}\n" " A format that consumes ${{mcpp.stage_dir}} sees the program and its " @@ -467,6 +548,8 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, "built file\n" " with ${{mcpp.target_file:}} is unaffected.", closure.reason)); + } else { + closure.needs = r->needs; } // The staged tree is now on disk and final -- past the closure (walked or diff --git a/src/pack/stage_tree.cppm b/src/pack/stage_tree.cppm index 09089b553..c38ea83a8 100644 --- a/src/pack/stage_tree.cppm +++ b/src/pack/stage_tree.cppm @@ -40,6 +40,19 @@ // the manifest states that the sorted file list cannot: a provider that needs // the closure reads this line and decides for itself rather than discovering // the gap by what is absent from `lib/`. +// +// THE `needs` LINES (#634 A3). `closure = walked` says the closure was +// resolved; it does not say what the closure was, and a provider that places +// libraries itself (`dist-apk` under `lib//`, `dist-apple` under +// `Frameworks/`) would otherwise have to tell a staged library from a staged +// resource by its extension. One line per needed name states it: +// +// needs the tree carries it at that path +// needsplatform the target provides it +// needsunresolved neither; the closure is not-walked +// +// TAB-separated because both fields are names a loader reads and either may +// contain a space (a Mach-O install name, a Windows directory). module; #include @@ -71,6 +84,17 @@ std::filesystem::path stage_manifest_path(const std::filesystem::path& stagingRo // manifest only for a DISPATCHED format -- `pack::closure_unavailable_outcome` // turns the same condition into a hard failure for `--format tar` and // `--format dir`, so those two never write a `not-walked` manifest. +// One name the staged program or a staged library needs, and what satisfies +// it. See the `needs` lines in the header comment. +struct ClosureNeed { + enum class Kind { Staged, Platform, Unresolved }; + std::string name; // as the needing object spells it + Kind kind = Kind::Staged; + // `Kind::Staged` only: the member's path relative to the staged tree, + // `/`-separated on every host. + std::string staged; +}; + struct ClosureStatus { bool walked = true; // Populated only when `!walked`. A SINGLE LINE: the manifest is a plain @@ -78,8 +102,25 @@ struct ClosureStatus { // carries its own embedded newlines (it doubles as a CLI diagnostic), so // `write_stage_manifest` folds them to spaces before writing. std::string reason; + // Every name the closure resolved, or failed to. Empty when no closure was + // read: a mode that bundles nothing (`system`, `static`), or a row whose + // closure this host cannot read. + std::vector needs; }; +// The `needs` line for `need`, without its newline. +std::string render_closure_need(const ClosureNeed& need) { + switch (need.kind) { + case ClosureNeed::Kind::Staged: + return std::format("needs\t{}\t{}", need.name, need.staged); + case ClosureNeed::Kind::Platform: + return std::format("needs\t{}\tplatform", need.name); + case ClosureNeed::Kind::Unresolved: + break; + } + return std::format("needs\t{}\tunresolved", need.name); +} + // Write the manifest for the tree now on disk at `stagingRoot`. // // Best-effort by construction and deliberately so: the manifest is a @@ -140,6 +181,16 @@ bool write_stage_manifest(const std::filesystem::path& stagingRoot, while (!folded.empty() && folded.back() == ' ') folded.pop_back(); text += std::format("reason = {}\n", folded); } + // The closure's lines follow the header and precede the file list, as a + // block of their own for the reason the header is one: they describe what + // the tree's files are for, and sorting them into the file list would + // interleave the two. Sorted and deduplicated, because a several-ABI tree + // reads the same platform name once per leg. + std::vector needs; + for (auto const& n : closure.needs) needs.push_back(render_closure_need(n)); + std::ranges::sort(needs); + needs.erase(std::unique(needs.begin(), needs.end()), needs.end()); + for (auto const& l : needs) { text += l; text.push_back('\n'); } for (auto const& l : lines) { text += l; text.push_back('\n'); } auto out = stage_manifest_path(stagingRoot); diff --git a/src/toolchain/clang.cppm b/src/toolchain/clang.cppm index 27b5515b6..97af17325 100644 --- a/src/toolchain/clang.cppm +++ b/src/toolchain/clang.cppm @@ -47,6 +47,14 @@ std::vector std_compat_build_commands(const Toolchain& tc, // Locate clang-scan-deps in the same bin/ directory as clang++. std::optional find_scan_deps(const Toolchain& tc); +// The file the driver links for `archive` (`libc++.a`, `libc++abi.a`, +// `libunwind.a`) under `driverFlags`, which carry the target the link names. +// Empty when the driver cannot place it. Asked once per process for each +// (compiler, flags, archive) triple. +std::filesystem::path runtime_archive_path(const Toolchain& tc, + std::string_view driverFlags, + std::string_view archive); + } // namespace mcpp::toolchain::clang namespace mcpp::toolchain::clang { @@ -384,6 +392,61 @@ std::optional find_scan_deps(const Toolchain& tc) { return std::nullopt; } +// THE DRIVER KNOWS WHERE ITS ARCHIVES ARE FOR A TARGET, AND A DIRECTORY +// SEARCH ONLY KNOWS ONE LAYOUT. +// +// The self-contained contract used to look for `libc++.a` under the LLVM +// payload's `lib/` alone. The Android NDK keeps its archives in the sysroot, +// per API level, so the search found nothing, the contract degraded to +// toolchain-coupled, and every test program on an Android row needed +// `libc++_shared.so`, which a device does not have (#634 A6, measured on an +// API 34 emulator: `CANNOT LINK EXECUTABLE ... library "libc++_shared.so" not +// found` for every test). The drivers answer directly, measured 2026-09-14: +// +// xim:android-ndk 30 --target=x86_64-linux-android24 +// libc++.a /usr/lib/x86_64-linux-android/24/libc++.a +// (a linker script: INPUT(-lc++_static -lc++abi)) +// libc++abi.a /usr/lib/x86_64-linux-android/libc++abi.a +// libunwind.a /lib/clang/21/lib/linux/x86_64/libunwind.a +// xim:llvm 22.1.8 libc++.a /bin/../lib/x86_64-unknown-linux-gnu/libc++.a +// +// THE TARGET IN THE QUESTION IS THE ONE THE LINK NAMES. The same NDK asked +// with `--target=x86_64-linux-android`, no API level, answers +// `/lib/clang/21/../../libc++.a`, a symlink to the HOST's +// `x86_64-unknown-linux-gnu/libc++.a`. The caller passes the flags its link +// line carries, API level included. +// +// A driver that cannot place a file echoes the bare name back, which is a +// miss rather than a relative path. The answer is made lexically normal: the +// `bin/..` in it is a spelling of the driver's, and a link line that spells +// one file two ways is two commands to ninja. +std::filesystem::path runtime_archive_path(const Toolchain& tc, + std::string_view driverFlags, + std::string_view archive) { + static std::mutex mu; + static std::map answered; + auto key = std::format("{}\n{}\n{}", tc.binaryPath.string(), driverFlags, archive); + { + std::scoped_lock lock(mu); + if (auto it = answered.find(key); it != answered.end()) return it->second; + } + std::filesystem::path out; + if (auto r = mcpp::toolchain::run_capture(std::format( + "{}{}{} --print-file-name={} {}", + mcpp::toolchain::compiler_env_prefix(tc), + mcpp::xlings::shq(tc.binaryPath.string()), + driverFlags, archive, mcpp::platform::null_redirect))) { + std::filesystem::path lib(mcpp::toolchain::trim_line(*r)); + std::error_code ec; + if (lib.has_parent_path() && lib.is_absolute() + && std::filesystem::is_regular_file(lib, ec)) + out = lib.lexically_normal(); + } + std::scoped_lock lock(mu); + answered.emplace(std::move(key), out); + return out; +} + std::optional find_libcxx_std_compat_source( const std::filesystem::path& cxx_binary, const std::string& envPrefix) diff --git a/tests/e2e/249_pack_workspace_root_unchanged.sh b/tests/e2e/249_pack_workspace_root_unchanged.sh index bc8f3cd14..c2a982830 100755 --- a/tests/e2e/249_pack_workspace_root_unchanged.sh +++ b/tests/e2e/249_pack_workspace_root_unchanged.sh @@ -53,30 +53,9 @@ EOF cd ws -# ── Mach-O: the routing is the claim, and a bundle cannot be produced ── -# -# `mcpp pack` of a PROGRAM is refused on Mach-O (its dependency closure would -# be resolved by RUNNING the artifact under a linker that ignores the tracing -# variable — see 266). So on macOS the question this file asks becomes: did the -# workspace root still hand through to the MEMBER'S PROGRAM? The refusal names -# the artifact, which is exactly the evidence for that — and it is a different -# message from "this package declares no program and no library to pack", which -# is what a routing regression would produce. -if [[ "$(uname -s)" == "Darwin" ]]; then - if "$MCPP" pack --mode system > pack.log 2>&1; then - cat pack.log - echo "FAIL: a Mach-O program bundle was produced; 266 says it must be refused" - exit 1 - fi - grep -q "Mach-O program 'hello'" pack.log || { - cat pack.log - echo "FAIL: the workspace root did not route to the member's program 'hello'." - echo " (A refusal naming some other artifact, or a 'declares no program'" - echo " error, is the routing regression this file exists to catch.)" - exit 1; } - echo "PASS: a workspace root still routes to its member's program" - exit 0 -fi +# The same legs on every host. A Mach-O program packs as the others do +# (2026.9.14.2+, its closure read from its load commands); the macOS branch +# that asserted the earlier refusal naming the member's program is gone. "$MCPP" pack --mode system > pack.log 2>&1 || { cat pack.log diff --git a/tests/e2e/250_pack_names_the_target.sh b/tests/e2e/250_pack_names_the_target.sh index a35b72369..bb7c4264a 100755 --- a/tests/e2e/250_pack_names_the_target.sh +++ b/tests/e2e/250_pack_names_the_target.sh @@ -42,37 +42,9 @@ EOF cd two -# ── Mach-O: which target was CHOSEN is still checkable ───────────────── -# -# A program bundle cannot be produced on Mach-O (266), so "look inside the -# staging dir" is unavailable here. The refusal names the artifact it got to, -# which answers the same question: `pack beta` must reach beta and `pack alpha` -# must reach alpha. Without this branch the file would simply not run on macOS, -# and the defect it exists for — `mcpp pack app2` bundling app1 — is not a -# platform-specific one. -if [[ "$(uname -s)" == "Darwin" ]]; then - for want in beta alpha; do - other=$([[ "$want" == beta ]] && echo alpha || echo beta) - if "$MCPP" pack "$want" --mode system > "$want.log" 2>&1; then - cat "$want.log"; echo "FAIL: a Mach-O program bundle was produced"; exit 1 - fi - grep -q "Mach-O program '$want'" "$want.log" || { - cat "$want.log" - echo "FAIL: 'mcpp pack $want' did not reach $want"; exit 1; } - grep -q "Mach-O program '$other'" "$want.log" && { - cat "$want.log" - echo "FAIL: 'mcpp pack $want' reached $other instead"; exit 1; } - done - # An unknown name must still fail EARLIER and differently — otherwise the - # two assertions above would pass for a build that refuses everything. - if "$MCPP" pack nosuch --mode system > bad.log 2>&1; then - cat bad.log; echo "FAIL: an unknown target name was accepted"; exit 1 - fi - grep -q "no target named 'nosuch'" bad.log || { - cat bad.log; echo "wrong refusal for an unknown name"; exit 1; } - echo "PASS: mcpp pack reaches the target it is given" - exit 0 -fi +# The same legs on every host. A Mach-O program packs as the others do +# (2026.9.14.2+), so the named program is looked for in the staged tree on +# macOS too, rather than in the name an earlier refusal carried. # ── the named program is the one that gets bundled ───────────────────── "$MCPP" pack beta --mode system > beta.log 2>&1 || { cat beta.log; echo "pack beta failed"; exit 1; } diff --git a/tests/e2e/266_pack_reads_a_macho_program_without_running_it.sh b/tests/e2e/266_pack_reads_a_macho_program_without_running_it.sh new file mode 100755 index 000000000..0b567a7da --- /dev/null +++ b/tests/e2e/266_pack_reads_a_macho_program_without_running_it.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +# requires: macos +# 266_pack_reads_a_macho_program_without_running_it.sh — `mcpp pack` of a +# PROGRAM on macOS packs it, and never runs it. +# +# WHAT IT ONCE DID, WHICH IS WHY THE CRITERION IS A SIDE EFFECT +# +# The non-PE path resolves an artifact's dependency closure by asking the +# dynamic linker: +# +# LD_TRACE_LOADED_OBJECTS=1 '' +# +# That variable is glibc's. dyld has never heard of it, so on macOS this did +# not trace anything — IT RAN THE USER'S PROGRAM, and a bundle containing just +# the binary was written and reported as `Packed`. mcpp then refused a Mach-O +# program outright, which this test asserted. Since #634 A3 the closure is +# read from the program's load commands, so the program packs; what must stay +# true is that packing it runs nothing. The program writes a marker file when +# it runs, and the marker must not exist after the pack. +# +# BOTH SIDES, ON THE SAME HOST: the same run also packs a LIBRARY target, +# which takes a different pipeline and must still succeed. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p proj/src +cat > proj/src/mathkit.cppm <<'CPP' +export module mathkit; +export namespace mk { int answer(); } +CPP +cat > proj/src/impl.cpp <<'CPP' +module mathkit; +namespace mk { int answer() { return 42; } } +CPP +cat > proj/src/main.cpp <<'CPP' +#include +import mathkit; +int main() { + // The side effect the criterion reads: a file beside the working directory. + if (std::FILE* f = std::fopen("ran.marker", "w")) std::fclose(f); + std::printf("ok=%d\n", mk::answer()); + return 0; +} +CPP +cat > proj/mcpp.toml <<'TOML' +[package] +name = "proj" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.proj] +kind = "bin" +main = "src/main.cpp" +[targets.mathkit] +kind = "lib" +TOML + +cd proj + +# ── 1. the program packs, and packing it ran nothing ──────────────────────── +"$MCPP" pack proj > pack.log 2>&1 || { + cat pack.log + echo "FAIL: mcpp pack refused a Mach-O program whose closure is the OS's alone." + exit 1; } +if [ -e ran.marker ] || [ -n "$(find target -name ran.marker | head -1)" ]; then + cat pack.log + echo "FAIL: packing the program RAN it (ran.marker exists)." + exit 1 +fi +archive="$(find target/dist -maxdepth 1 -name 'proj-0.1.0-*.tar.gz' | head -1)" +[[ -n "$archive" ]] || { cat pack.log; echo "FAIL: no archive under target/dist"; exit 1; } +tar -tzf "$archive" | grep -q '/bin/proj$' || { + tar -tzf "$archive"; echo "FAIL: the archive has no bin/proj"; exit 1; } +echo " a Mach-O program packs, and the pack did not run it" + +# ── 2. …and a library target on the same host still packs ────────────────── +"$MCPP" pack mathkit > packlib.log 2>&1 || { + cat packlib.log + echo "FAIL: a library target does not pack on macOS." + exit 1; } +# Searched by content: the program's staged tree above shares the package's +# name prefix, so a directory match could pick either. +[[ -n "$(find target/dist -name '*.a' | head -1)" ]] || { + find target/dist -type f; echo "FAIL: no archive in the library package"; exit 1; } +echo " a library target still packs" + +echo "PASS: mcpp pack reads a Mach-O program without running it, and library packaging is unaffected" diff --git a/tests/e2e/266_pack_refuses_a_macho_program.sh b/tests/e2e/266_pack_refuses_a_macho_program.sh deleted file mode 100755 index 75c21a00a..000000000 --- a/tests/e2e/266_pack_refuses_a_macho_program.sh +++ /dev/null @@ -1,94 +0,0 @@ -#!/usr/bin/env bash -# requires: macos -# 266_pack_refuses_a_macho_program.sh — `mcpp pack` of a PROGRAM on macOS is -# refused, and refused for the stated reason. -# -# WHAT IT USED TO DO INSTEAD, WHICH IS WORSE THAN FAILING -# -# The non-PE path resolves an artifact's dependency closure by asking the -# dynamic linker: -# -# LD_TRACE_LOADED_OBJECTS=1 '' -# -# That variable is glibc's. dyld has never heard of it (its counterpart is -# `DYLD_PRINT_LIBRARIES`), so on macOS this does not trace anything — IT RUNS -# THE USER'S PROGRAM. Whatever the program prints is then parsed as a -# dependency table, yields nothing, and a bundle containing just the binary is -# written and reported as `Packed`. A program with side effects performs them. -# An interactive one hangs the packer. -# -# The `_WIN32` branch beside it has refused the same class since it was -# written; macOS was simply never checked, because the e2e harness only grants -# the `pack` capability where `elf` + `patchelf` are both present — i.e. Linux. -# So no job in this suite has ever run `mcpp pack` on a Mac. -# -# BOTH SIDES, ON THE SAME HOST. Asserting only the refusal cannot tell "the -# gate works" from "pack is broken on this machine". So the same run also packs -# a LIBRARY target, which takes a different pipeline and must still succeed. -set -e - -TMP=$(mktemp -d) -trap "rm -rf $TMP" EXIT -cd "$TMP" - -mkdir -p proj/src -cat > proj/src/mathkit.cppm <<'EOF' -export module mathkit; -export namespace mk { int answer(); } -EOF -cat > proj/src/impl.cpp <<'EOF' -module mathkit; -namespace mk { int answer() { return 42; } } -EOF -cat > proj/src/main.cpp <<'EOF' -#include -import mathkit; -int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } -EOF -cat > proj/mcpp.toml <<'EOF' -[package] -name = "proj" -version = "0.1.0" -[build] -sources = ["src/*.cppm", "src/*.cpp"] -[targets.proj] -kind = "bin" -main = "src/main.cpp" -[targets.mathkit] -kind = "lib" -EOF - -cd proj - -# ── 1. the program is refused, by name ───────────────────────────────────── -if "$MCPP" pack proj > pack.log 2>&1; then - cat pack.log - echo "FAIL: mcpp pack produced a bundle for a Mach-O program." - echo " Its dependency closure cannot be resolved on this platform, so" - echo " whatever it produced is not one — and producing it RAN the program." - exit 1 -fi -grep -qi "Mach-O" pack.log || { - cat pack.log - echo "FAIL: pack failed, but not with the Mach-O refusal — so this test is" - echo " observing some other failure and proves nothing about the gate." - exit 1; } -grep -q "LD_TRACE_LOADED_OBJECTS" pack.log || { - cat pack.log - echo "FAIL: the refusal does not say WHY. A reader has to be able to tell" - echo " this from 'macOS is unsupported in general'." - exit 1; } -echo " a Mach-O program is refused, and the message names the mechanism" - -# ── 2. …and a library target on the same host still packs ────────────────── -"$MCPP" pack mathkit > packlib.log 2>&1 || { - cat packlib.log - echo "FAIL: a library target does not pack on macOS either — the refusal above" - echo " is therefore not evidence of a working gate." - exit 1; } -pkg="$(find target/dist -maxdepth 1 -type d -name 'proj-0.1.0-*' | head -1)" -[[ -n "$(find "$pkg" -name '*.a' | head -1)" ]] || { - find "$pkg" -type f; echo "FAIL: no archive in the library package"; exit 1; } -echo " a library target still packs" - -echo "PASS: mcpp pack refuses a Mach-O program with the reason, and library packaging is unaffected" diff --git a/tests/e2e/666_a_macho_program_is_staged_without_its_closure.sh b/tests/e2e/666_a_macho_program_reaches_a_dispatched_format_with_its_tree.sh similarity index 54% rename from tests/e2e/666_a_macho_program_is_staged_without_its_closure.sh rename to tests/e2e/666_a_macho_program_reaches_a_dispatched_format_with_its_tree.sh index bcb59dddb..9b81c0742 100755 --- a/tests/e2e/666_a_macho_program_is_staged_without_its_closure.sh +++ b/tests/e2e/666_a_macho_program_reaches_a_dispatched_format_with_its_tree.sh @@ -2,10 +2,13 @@ # requires: macos # 666 -- on the one host that produces a Mach-O program, `mcpp pack` stages # the program and its declared files and hands the tree to a dispatched -# format with `closure = not-walked` and a reason (#630, item 3a). Before -# this, the Mach-O refusal preceded staging, and a bundler reached its action -# with no tree at all. The negative direction stays: `--format dir` and -# `--format tar`, whose product is the closure, still refuse. +# format (#630, item 3a). Until #634 A3 the tree arrived with +# `closure = not-walked`, because the Mach-O closure was never read, and +# `--format dir` and `--format tar` refused the program. The closure is read +# from the load commands now, so both directions changed: the built-in +# archive stages the program, and the dispatched format sees +# `closure = walked` with the OS's libraries stated as `platform`. A program +# with a dylib of its own is 668. set -e TMP=$(mktemp -d) @@ -59,22 +62,26 @@ int main() { } EOF -# The negative direction first: the built-in archive still refuses, since -# its product IS the closure. -if "$MCPP" pack --format dir > dir.log 2>&1; then - fail "--format dir of a Mach-O program was accepted" dir.log -fi -grep -q 'Mach-O' dir.log || fail "the refusal does not name the format" dir.log -echo " negative: --format dir still refuses a Mach-O program" +# The built-in archive stages the program: its closure is only the OS's. +"$MCPP" pack --format dir > dir.log 2>&1 || fail "--format dir of a Mach-O program failed" dir.log +D=$(find target/dist -mindepth 1 -maxdepth 1 -type d -name 'zapapp-0.1.0-*' | head -1) +[ -n "$D" ] || fail "--format dir staged no tree" dir.log +[ -x "$D/bin/zapapp" ] || fail "the staged tree has no bin/zapapp" dir.log +[ "$(sed -n '1p' "$D.stage-manifest")" = "closure = walked" ] \ + || fail "the dir manifest does not say closure = walked" "$D.stage-manifest" +echo " --format dir stages a Mach-O program with closure = walked" "$MCPP" pack --format zap > zap.log 2>&1 || fail "mcpp pack --format zap failed" zap.log -grep -q 'staged without its dependency closure' zap.log \ - || fail "pack did not report the tree as staged without its closure" zap.log +if grep -q 'staged without its dependency closure' zap.log; then + fail "pack still reports the tree as staged without its closure" zap.log +fi Z=$(find target -name 'app.zap' | head -1) [ -n "$Z" ] || fail "--format zap produced nothing" zap.log grep -qx './bin/zapapp' "$Z" || fail "the action's view of the tree lacks bin/zapapp" "$Z" zap.log grep -qx './bin/data/notes.txt' "$Z" || fail "the action's view of the tree lacks the deployed file" "$Z" zap.log -grep -qx 'closure = not-walked' "$Z" || fail "the manifest does not say closure = not-walked" "$Z" zap.log -grep -q '^reason = ' "$Z" || fail "the manifest carries no reason line" "$Z" zap.log -echo " positive: the dispatched format sees the program, the deployed file, and closure = not-walked" -echo "PASS: 666_a_macho_program_is_staged_without_its_closure" +grep -qx 'closure = walked' "$Z" || fail "the manifest does not say closure = walked" "$Z" zap.log +if grep -q '^reason = ' "$Z"; then + fail "a walked closure carries a reason line" "$Z" zap.log +fi +echo " the dispatched format sees the program, the deployed file, and closure = walked" +echo "PASS: 666_a_macho_program_reaches_a_dispatched_format_with_its_tree" diff --git a/tests/e2e/667_an_android_pack_stages_its_closure.sh b/tests/e2e/667_an_android_pack_stages_its_closure.sh new file mode 100755 index 000000000..0f66d6c81 --- /dev/null +++ b/tests/e2e/667_an_android_pack_stages_its_closure.sh @@ -0,0 +1,184 @@ +#!/usr/bin/env bash +# requires: elf gcc android-ndk +# 667 -- #634 A3 and A4 on the Android rows. `mcpp pack` reads the application +# object's closure from the files and stages it beside the object under `lib/`: +# a graph-built dependency and the NDK's `libc++_shared.so` travel, and a name +# the API level's stub directory holds (`libc.so`, `libm.so`, `libdl.so`) is the +# device's. The stage manifest says so in `needs` lines. Before this, the tree +# held `lib/libapp.so` alone and the manifest said `closure = walked`. +# +# Every ELF shared library carries a SONAME equal to its file name (A4), which +# bionic enforces from API level 23. +# +# The negative directions: a library the link used and that is gone by the +# time of the pack makes `dir` and `tar` refuse naming it, and a dispatched +# format receives `closure = not-walked` with the name as `unresolved`. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +export MCPP_HOME=${MCPP_HOME:-$HOME/.mcpp} + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +TAB=$(printf '\t') + +cd "$TMP" +mkdir -p fw/src app/src +cat > fw/mcpp.toml <<'EOF' +[package] +namespace = "demo" +name = "fw" +version = "0.1.0" +[targets.fw] +kind = "lib" +EOF +printf 'export module fw;\nexport int fw_anchor() { return 41; }\n' > fw/src/fw.cppm + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +[targets.app] +kind = "app" +main = "src/main.cpp" +[dependencies] +demo.fw = { path = "../fw", linkage = "shared" } +EOF +printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > app/src/main.cpp +cd app + +# ── 1. one triple: the closure is staged under lib/ ─────────────────────── +"$MCPP" pack --target x86_64-linux-android --format dir > dir.log 2>&1 \ + || fail "mcpp pack --format dir failed" dir.log +tree=target/dist/app-0.1.0-x86_64-linux-android +[ -d "$tree" ] || fail "no staged tree at $tree" dir.log +got=$(cd "$tree/lib" && find . -type f | sort | tr '\n' ' ') +[ "$got" = "./libapp.so ./libc++_shared.so ./libfw.so " ] \ + || fail "lib/ holds '$got', not the object and its two libraries" dir.log + +manifest="$tree.stage-manifest" +[ "$(sed -n '1p' "$manifest")" = "closure = walked" ] \ + || fail "the manifest's first line is not 'closure = walked'" "$manifest" +for line in "needs${TAB}libfw.so${TAB}lib/libfw.so" \ + "needs${TAB}libc++_shared.so${TAB}lib/libc++_shared.so" \ + "needs${TAB}libc.so${TAB}platform" \ + "needs${TAB}libm.so${TAB}platform" \ + "needs${TAB}libdl.so${TAB}platform"; do + grep -qxF "$line" "$manifest" || fail "the manifest lacks the line '$line'" "$manifest" +done +echo " ok: lib/ carries libfw.so and libc++_shared.so; the stubs are the platform's" + +for so in libapp.so libfw.so; do + soname=$(readelf -d "$tree/lib/$so" | sed -n 's/.*(SONAME).*\[\(.*\)\].*/\1/p') + [ "$soname" = "$so" ] || fail "lib/$so carries SONAME '$soname'" dir.log +done +readelf -d "$tree/lib/libapp.so" | grep -q 'NEEDED.*\[libfw.so\]' \ + || fail "libapp.so no longer needs libfw.so by its file name" dir.log +echo " ok: libapp.so and libfw.so carry their file names as SONAME" + +# ── 2. a prebuilt library the link named travels, and its absence refuses ── +NDKCC=$(ls "$MCPP_HOME"/registry/data/xpkgs/xim-x-android-ndk/*/toolchains/llvm/prebuilt/*/bin/clang 2>/dev/null | head -1) +[ -x "$NDKCC" ] || fail "no NDK clang under $MCPP_HOME/registry" +mkdir -p ext/lib +printf 'int ext_value(void) { return 1; }\n' > ext/ext.c +"$NDKCC" --target=x86_64-linux-android24 -shared -fPIC -Wl,-soname,libext.so \ + -o ext/lib/libext.so ext/ext.c +cat >> mcpp.toml <<'EOF' +[target.'cfg(env = "android")'.runtime] +link_library_dirs = ["ext/lib"] +libraries = ["ext"] +EOF +printf 'import fw;\nextern "C" int ext_value();\nint main() { return fw_anchor() == 41 && ext_value() == 1 ? 0 : 1; }\n' > src/main.cpp + +"$MCPP" pack --target x86_64-linux-android --format dir > ext.log 2>&1 \ + || fail "the pack with a prebuilt library failed" ext.log +[ -f "$tree/lib/libext.so" ] || fail "the prebuilt libext.so is not staged" ext.log +grep -qxF "needs${TAB}libext.so${TAB}lib/libext.so" "$manifest" \ + || fail "the manifest does not state libext.so as staged" "$manifest" +echo " ok: a library from [runtime] link_library_dirs is staged" + +# The build is up to date, so the pack does not relink; the file is simply gone. +rm ext/lib/libext.so +for fmt in dir tar; do + if "$MCPP" pack --target x86_64-linux-android --format "$fmt" > "gone-$fmt.log" 2>&1; then + fail "--format $fmt succeeded with libext.so gone" "gone-$fmt.log" + fi + grep -q "libext.so: found in none of" "gone-$fmt.log" \ + || fail "--format $fmt did not name libext.so" "gone-$fmt.log" + grep -q "incomplete" "gone-$fmt.log" \ + || fail "--format $fmt did not say the closure is incomplete" "gone-$fmt.log" +done +echo " ok: dir and tar refuse, naming the library that is gone" + +# ── 3. a dispatched format receives the incomplete closure ──────────────── +cat > dist.sh <<'EOF' +set -e +manifest="$1"; out="$2" +cp "$manifest" "$out" +EOF +chmod +x dist.sh +cat > build.mcpp <<'EOF' +import std; +import mcpp; +int main() { + mcpp::provides_pack_format("copy"); + if (std::string_view(mcpp::pack_format()) != "copy") return 0; + const std::string root = mcpp::manifest_dir(); + const std::string stage = std::string("${mcpp.stage_dir}"); + const std::string out = std::string(mcpp::out_dir()) + "/stage.txt"; + mcpp::action a; + a.id = "copy"; + a.role = "artifact"; + a.description = "copy"; + a.arg("/bin/sh") + .arg((root + "/dist.sh").c_str()) + .arg((stage + ".stage-manifest").c_str()) + .arg(out.c_str()) + .input("${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +EOF +"$MCPP" pack --target x86_64-linux-android --format copy > copy.log 2>&1 \ + || fail "the dispatched format failed" copy.log +seen=$(find target -name stage.txt | head -1) +[ -f "$seen" ] || fail "the provider wrote nothing" copy.log +[ "$(sed -n '1p' "$seen")" = "closure = not-walked" ] \ + || fail "the provider did not receive 'closure = not-walked'" "$seen" +grep -qxF "needs${TAB}libext.so${TAB}unresolved" "$seen" \ + || fail "the provider did not receive libext.so as unresolved" "$seen" +grep -qxF "needs${TAB}libfw.so${TAB}lib/libfw.so" "$seen" \ + || fail "the provider lost the members that did resolve" "$seen" +echo " ok: a dispatched format receives not-walked, the unresolved name and the members" +rm -f build.mcpp dist.sh + +# ── 4. two triples: each lib// carries its own closure ─────────────── +cat > mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +[targets.app] +kind = "app" +main = "src/main.cpp" +[dependencies] +demo.fw = { path = "../fw", linkage = "shared" } +EOF +printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > src/main.cpp +rm -rf target/dist +"$MCPP" pack --target x86_64-linux-android --target aarch64-linux-android --format dir \ + > two.log 2>&1 || fail "the two-triple pack failed" two.log +tree2=$(find target/dist -mindepth 1 -maxdepth 1 -type d | head -1) +[ -n "$tree2" ] || fail "no staged tree for two triples" two.log +for abi in x86_64 arm64-v8a; do + got=$(cd "$tree2/lib/$abi" && find . -type f | sort | tr '\n' ' ') + [ "$got" = "./libapp.so ./libc++_shared.so ./libfw.so " ] \ + || fail "lib/$abi holds '$got'" two.log + grep -qxF "needs${TAB}libfw.so${TAB}lib/$abi/libfw.so" "$tree2.stage-manifest" \ + || fail "the manifest lacks lib/$abi/libfw.so" "$tree2.stage-manifest" +done +file "$tree2/lib/arm64-v8a/libc++_shared.so" | grep -qi "aarch64" \ + || fail "lib/arm64-v8a/libc++_shared.so is not the aarch64 runtime" two.log +echo " ok: lib/x86_64/ and lib/arm64-v8a/ each carry their own closure" + +echo "667: an Android pack stages its closure OK" diff --git a/tests/e2e/668_a_macho_program_is_staged_with_its_closure.sh b/tests/e2e/668_a_macho_program_is_staged_with_its_closure.sh new file mode 100755 index 000000000..3fb15c4b6 --- /dev/null +++ b/tests/e2e/668_a_macho_program_is_staged_with_its_closure.sh @@ -0,0 +1,103 @@ +#!/usr/bin/env bash +# requires: macos +# 668 -- #634 A3 on the Mach-O row. `mcpp pack` reads a Mach-O program's load +# commands and stages the dylibs it needs beside it in `bin/`, where the +# `@loader_path` rpath a consumer of a graph-built dylib links with finds them: +# no load command is edited and nothing is re-signed. The stage manifest names +# each dylib in a `needs` line, and the OS's own libraries as `platform`. +# +# The criterion is the program running from the staged tree after the build +# tree is gone, and failing with "Library not loaded" once the staged dylib is +# removed -- a tree that ran only while the build directory existed would pass +# every check but that one. Measured on macos-15 before the change (the +# triage record's reading C5): exit 7 staged by hand, exit 134 without the +# dylib, and `mcpp pack --format dir` refused the program. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +export MCPP_HOME=${MCPP_HOME:-$HOME/.mcpp} + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +TAB=$(printf '\t') + +cd "$TMP" +mkdir -p fw/src app/src +cat > fw/mcpp.toml <<'EOF' +[package] +namespace = "demo" +name = "fw" +version = "0.1.0" +[targets.fw] +kind = "lib" +EOF +printf 'export module fw;\nexport int fw_anchor() { return 41; }\n' > fw/src/fw.cppm + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +[dependencies] +demo.fw = { path = "../fw", linkage = "shared" } +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +cat > app/src/main.cpp <<'EOF' +#include +import fw; +int main() { std::puts("1-2-3"); return fw_anchor() == 41 ? 7 : 1; } +EOF +cd app + +# ── 1. --format dir stages the program and its dylib ────────────────────── +"$MCPP" pack --format dir > dir.log 2>&1 || fail "mcpp pack --format dir failed" dir.log +tree=$(find target/dist -mindepth 1 -maxdepth 1 -type d -name 'app-0.1.0-*' | head -1) +[ -n "$tree" ] || fail "no staged tree under target/dist" dir.log +[ -x "$tree/bin/app" ] || fail "the staged tree has no bin/app" dir.log +[ -f "$tree/bin/libfw.dylib" ] || fail "the staged tree has no bin/libfw.dylib beside the program" dir.log + +manifest="$tree.stage-manifest" +[ "$(sed -n '1p' "$manifest")" = "closure = walked" ] \ + || fail "the manifest's first line is not 'closure = walked'" "$manifest" +grep -qxF "needs${TAB}@rpath/libfw.dylib${TAB}bin/libfw.dylib" "$manifest" \ + || fail "the manifest does not state the dylib beside the program" "$manifest" +grep -q "^needs${TAB}/usr/lib/libSystem.B.dylib${TAB}platform\$" "$manifest" \ + || fail "the manifest does not state libSystem as the platform's" "$manifest" +echo " ok: bin/libfw.dylib is staged and the manifest names both kinds" + +# ── 2. the staged program runs with the build tree gone ─────────────────── +mv target "$TMP/moved-target" +dest="$TMP/installed" +mkdir -p "$dest" +cp -R "$TMP/moved-target/dist/$(basename "$tree")" "$dest/" +staged="$dest/$(basename "$tree")" +rm -rf "$TMP/moved-target" + +set +e +"$staged/bin/app" > run.log 2>&1; rc=$? +set -e +[ "$rc" -eq 7 ] || fail "the staged program exited $rc, not 7" run.log +grep -q '^1-2-3$' run.log || fail "the staged program did not print 1-2-3" run.log +echo " ok: the staged program runs with the build tree deleted (exit 7)" + +# ── 3. and it is the staged dylib that loaded ───────────────────────────── +rm "$staged/bin/libfw.dylib" +set +e +"$staged/bin/app" > gone.log 2>&1; rc=$? +set -e +[ "$rc" -ne 0 ] && [ "$rc" -ne 7 ] || fail "the program still ran without its staged dylib (exit $rc)" gone.log +grep -q "Library not loaded" gone.log || fail "the failure is not a missing library" gone.log +echo " ok: without the staged dylib the program stops with Library not loaded" + +# ── 4. --format tar carries the same two files ──────────────────────────── +cd "$TMP/app" +"$MCPP" pack > tar.log 2>&1 || fail "mcpp pack (tar) failed" tar.log +archive=$(find target/dist -maxdepth 1 -name 'app-0.1.0-*.tar.gz' | head -1) +[ -n "$archive" ] || fail "no archive under target/dist" tar.log +listing=$(tar -tzf "$archive") +grep -q '/bin/app$' <<<"$listing" || fail "the archive has no bin/app" tar.log +grep -q '/bin/libfw.dylib$' <<<"$listing" || fail "the archive has no bin/libfw.dylib" tar.log +echo " ok: the archive carries bin/app and bin/libfw.dylib" + +echo "668: a Mach-O program is staged with its closure OK" diff --git a/tests/e2e/669_an_elf_host_pack_states_its_closure.sh b/tests/e2e/669_an_elf_host_pack_states_its_closure.sh new file mode 100755 index 000000000..7302d8f97 --- /dev/null +++ b/tests/e2e/669_an_elf_host_pack_states_its_closure.sh @@ -0,0 +1,126 @@ +#!/usr/bin/env bash +# requires: elf gcc +# 669 -- #634 A3 and A4 on the ELF host row. +# +# A4: an ELF shared library built without a declared `soname` records its file +# name as DT_SONAME; its consumer's DT_NEEDED is the same name it was before, +# and a declared `soname` keeps its value. +# +# A3: the host row keeps its loader-traced closure, and the stage manifest +# gains the `needs` lines every other row writes: a bundled library at its +# staged path, a library the target provides as `platform`. A name the loader +# finds no file for used to be dropped, so an archive that could not start +# said `closure = walked`; now `dir` refuses naming it. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +export MCPP_HOME=${MCPP_HOME:-$HOME/.mcpp} + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +TAB=$(printf '\t') +soname_of() { readelf -d "$1" | sed -n 's/.*(SONAME).*\[\(.*\)\].*/\1/p'; } + +cd "$TMP" +mkdir -p fw/src named/src app/src +cat > fw/mcpp.toml <<'EOF' +[package] +namespace = "demo" +name = "fw" +version = "0.1.0" +[targets.fw] +kind = "lib" +EOF +printf 'export module fw;\nexport int fw_anchor() { return 41; }\n' > fw/src/fw.cppm + +cat > named/mcpp.toml <<'EOF' +[package] +namespace = "demo" +name = "named" +version = "0.1.0" +[targets.named] +kind = "shared" +soname = "libnamed.so.1" +EOF +printf 'export module named;\nexport int named_anchor() { return 1; }\n' > named/src/named.cppm + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +[toolchain] +linux = "gcc@16.1.0" +[dependencies] +demo.fw = { path = "../fw", linkage = "shared" } +demo.named = { path = "../named" } +EOF +cat > app/src/main.cpp <<'EOF' +#include +import fw; +import named; +int main() { std::puts("1-2-3"); return fw_anchor() == 41 && named_anchor() == 1 ? 0 : 1; } +EOF +cd app + +# ── 1. A4: the default SONAME, and the declared one ─────────────────────── +"$MCPP" build > build.log 2>&1 || fail "mcpp build failed" build.log +bin=$(dirname "$(find target -path '*/bin/app' -type f | head -1)") +[ -f "$bin/libfw.so" ] || fail "no bin/libfw.so" build.log +[ "$(soname_of "$bin/libfw.so")" = "libfw.so" ] \ + || fail "libfw.so carries SONAME '$(soname_of "$bin/libfw.so")', not its file name" build.log +readelf -d "$bin/app" | grep -q 'NEEDED.*\[libfw.so\]' \ + || fail "the consumer no longer needs libfw.so by its file name" build.log +named=$(find "$bin" -maxdepth 1 -name 'libnamed.so' -type f | head -1) +[ -n "$named" ] || fail "no bin/libnamed.so" build.log +[ "$(soname_of "$named")" = "libnamed.so.1" ] \ + || fail "the declared soname became '$(soname_of "$named")'" build.log +echo " ok: libfw.so records its file name; a declared soname keeps libnamed.so.1" + +# ── 2. A3: the manifest states the closure the tree carries ─────────────── +"$MCPP" pack --format dir > dir.log 2>&1 || fail "mcpp pack --format dir failed" dir.log +tree=$(find target/dist -mindepth 1 -maxdepth 1 -type d -name 'app-0.1.0-*' | head -1) +[ -n "$tree" ] || fail "no staged tree" dir.log +manifest="$tree.stage-manifest" +[ "$(sed -n '1p' "$manifest")" = "closure = walked" ] \ + || fail "the manifest's first line is not 'closure = walked'" "$manifest" +[ -f "$tree/lib/libfw.so" ] || fail "lib/libfw.so is not bundled" dir.log +grep -qxF "needs${TAB}libfw.so${TAB}lib/libfw.so" "$manifest" \ + || fail "the manifest does not state libfw.so at lib/libfw.so" "$manifest" +grep -qxF "needs${TAB}libc.so.6${TAB}platform" "$manifest" \ + || fail "the manifest does not state libc.so.6 as the platform's" "$manifest" +# The needs block sits between the header and the file list. +awk 'NR==1{next} /^needs\t/{ if (files) bad=1; next } { files=1 } END{ exit bad }' "$manifest" \ + || fail "a needs line follows a file line" "$manifest" +"$tree/bin/app" > run.log 2>&1 || fail "the staged program does not run" run.log +grep -q '^1-2-3$' run.log || fail "the staged program did not print 1-2-3" run.log +echo " ok: needs lines for the bundled and the platform libraries; the tree runs" + +# ── 3. a library the loader cannot find refuses the archive ─────────────── +cd "$TMP" +mkdir -p ext/lib extapp/src +printf 'int ext_value() { return 1; }\n' > ext/ext.cpp +g++ -shared -fPIC -Wl,-soname,libext.so -o ext/lib/libext.so ext/ext.cpp +cat > extapp/mcpp.toml < extapp/src/main.cpp +cd extapp +"$MCPP" pack --format dir > ext.log 2>&1 || fail "the pack with a prebuilt library failed" ext.log +etree=$(find target/dist -mindepth 1 -maxdepth 1 -type d -name 'extapp-0.1.0-*' | head -1) +grep -qxF "needs${TAB}libext.so${TAB}lib/libext.so" "$etree.stage-manifest" \ + || fail "libext.so is not stated as bundled" "$etree.stage-manifest" +rm "$TMP/ext/lib/libext.so" +if "$MCPP" pack --format dir > gone.log 2>&1; then + fail "--format dir succeeded with libext.so gone" gone.log +fi +grep -q "libext.so: the loader finds no file for it" gone.log \ + || fail "the refusal does not name libext.so" gone.log +echo " ok: dir refuses, naming the library the loader cannot find" + +echo "669: an ELF host pack states its closure OK" diff --git a/tests/e2e/670_an_rpath_loader_token_is_left_as_written.sh b/tests/e2e/670_an_rpath_loader_token_is_left_as_written.sh new file mode 100755 index 000000000..d8b5f56dd --- /dev/null +++ b/tests/e2e/670_an_rpath_loader_token_is_left_as_written.sh @@ -0,0 +1,71 @@ +#!/usr/bin/env bash +# requires: elf gcc +# 670 -- #634, item 11 of the triage record. A search path that begins with a +# token the loader expands is relative to a loaded object, not to the package +# that wrote it, so `mcpp` leaves it as written: ELF's `$ORIGIN` always was, +# and Mach-O's `@executable_path`, `@loader_path` and `@rpath` were anchored to +# the package directory instead -- `-Wl,-rpath,@executable_path/../Frameworks` +# reached the binary as `/@executable_path/../Frameworks`, which +# is how a bundle's framework rpath could not load (measured on macos-15). +# +# The normalisation is host-independent, so Linux measures it: the root's +# `[build] ldflags` and a dependency's, which reach the consumer through a +# second copy of the normaliser. The negative direction is an ordinary +# relative rpath, which is still anchored to the package that wrote it. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +export MCPP_HOME=${MCPP_HOME:-$HOME/.mcpp} + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +cd "$TMP" +mkdir -p dep/src app/src +cat > dep/mcpp.toml <<'EOF' +[package] +namespace = "demo" +name = "dep" +version = "0.1.0" +[targets.dep] +kind = "lib" +[build] +ldflags = ["-Wl,-rpath,@loader_path/../lib"] +EOF +printf 'export module dep;\nexport int dep_anchor() { return 3; }\n' > dep/src/dep.cppm + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +[toolchain] +linux = "gcc@16.1.0" +[dependencies] +demo.dep = { path = "../dep" } +[build] +ldflags = ["-Wl,-rpath,@executable_path/../Frameworks", "-Wl,-rpath,vendor/lib"] +EOF +printf 'import dep;\nint main() { return dep_anchor() == 3 ? 0 : 1; }\n' > app/src/main.cpp +cd app + +"$MCPP" build > build.log 2>&1 || fail "mcpp build failed" build.log +bin=$(find target -path '*/bin/app' -type f | head -1) +[ -n "$bin" ] || fail "no bin/app" build.log +rpath=$(readelf -d "$bin" | grep -E 'RPATH|RUNPATH' | sed 's/.*\[\(.*\)\]/\1/') +[ -n "$rpath" ] || fail "the program carries no RPATH" build.log + +tr ':' '\n' <<<"$rpath" > entries.txt +grep -qx '@executable_path/../Frameworks' entries.txt \ + || fail "the root's @executable_path rpath is not literal" entries.txt +grep -qx '@loader_path/../lib' entries.txt \ + || fail "the dependency's @loader_path rpath is not literal" entries.txt +if grep -q '/@executable_path\|/@loader_path' entries.txt; then + fail "a loader token was anchored to a package directory" entries.txt +fi +echo " ok: @executable_path and @loader_path reach the binary as written" + +grep -qx "$(pwd -P)/vendor/lib" entries.txt || grep -qx "$(pwd)/vendor/lib" entries.txt \ + || fail "an ordinary relative rpath is no longer anchored to the package" entries.txt +echo " ok: an ordinary relative rpath is still anchored to the package directory" + +echo "670: an rpath loader token is left as written OK" diff --git a/tests/e2e/671_toolchain_is_an_option_of_run_test_and_pack.sh b/tests/e2e/671_toolchain_is_an_option_of_run_test_and_pack.sh new file mode 100644 index 000000000..52046acc5 --- /dev/null +++ b/tests/e2e/671_toolchain_is_an_option_of_run_test_and_pack.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash +# requires: +# 671_toolchain_is_an_option_of_run_test_and_pack.sh -- `--toolchain` is +# declared on `run`, `test` and `pack` as it is on `build` (#634 A10). +# +# The value always reached the three commands: the pre-parse loop publishes it +# as MCPP_TOOLCHAIN for every command, and `MCPP_TOOLCHAIN=llvm@22.1.8 mcpp +# test` compiled with clang. The spelling the help of `build` teaches was +# refused by the option parser of the other three (2026.9.14.1): +# +# error: unknown option: --toolchain +# +# Criteria: +# 1. each of `run`, `test` and `pack` accepts the option and hands its value +# to toolchain resolution: an unknown family is refused by RESOLUTION, +# naming the value, and not by the option parser; +# 2. positive direction: the toolchain this host resolves by default, named +# with the option, builds and runs the program and passes the test; +# 3. with a second toolchain installed (llvm@22.1.8 while the default is +# another family), `mcpp test --toolchain llvm@22.1.8` compiles the tests +# with that compiler, read from the build graph it wrote. Reported as not +# measured when the payload is absent, because installing it here would +# make the criterion a download. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +"$MCPP" new tcopt > /dev/null +cd tcopt + +# ── 1. the option is accepted and its value reaches resolution ───────────── +for spelling in "run" "test" "pack --format dir"; do + read -ra words <<<"$spelling" + set +e + "$MCPP" "${words[@]}" --toolchain nosuch@0.0.1 > c1.log 2>&1 + rc=$? + set -e + [ "$rc" -ne 0 ] || fail "'mcpp $spelling --toolchain nosuch@0.0.1' succeeded" c1.log + grep -q "unknown option" c1.log \ + && fail "'mcpp $spelling' refused --toolchain as an unknown option" c1.log + grep -q "unknown toolchain 'nosuch'" c1.log \ + || fail "'mcpp $spelling' did not hand the value to toolchain resolution" c1.log + echo "mcpp $spelling --toolchain reaches resolution OK" +done + +# ── 2. the default toolchain, named with the option, works ───────────────── +"$MCPP" build > b.log 2>&1 || fail "the initial build failed" b.log +own=$(sed -n 's/.*Resolved \([^ ]*\) .*/\1/p' b.log | head -1) +[ -n "$own" ] || fail "could not learn this host's toolchain from the build" b.log + +"$MCPP" test --toolchain "$own" > t2.log 2>&1 || fail "mcpp test --toolchain $own failed" t2.log +grep -q "test result ok" t2.log || fail "mcpp test --toolchain $own did not pass" t2.log +"$MCPP" run --toolchain "$own" > r2.log 2>&1 || fail "mcpp run --toolchain $own failed" r2.log +grep -q "Hello from tcopt" r2.log || fail "mcpp run --toolchain $own did not run the program" r2.log +echo "the default toolchain named with --toolchain builds, runs and tests OK" + +# ── 3. a second installed toolchain compiles the tests ──────────────────── +store="${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-llvm/22.1.8/bin" +case "$own" in + llvm@*) + echo "NOT MEASURED: the default toolchain is already $own" + ;; + *) + if [ -x "$store/clang++" ] || [ -x "$store/clang++.exe" ]; then + "$MCPP" test --toolchain llvm@22.1.8 > t3.log 2>&1 \ + || fail "mcpp test --toolchain llvm@22.1.8 failed" t3.log + grep -q "Resolved llvm@22.1.8" t3.log \ + || fail "the test build did not resolve llvm@22.1.8" t3.log + grep -lq "xim-x-llvm/22.1.8/bin/clang++" target/*/*/build.ninja \ + || fail "no build graph compiles with the llvm payload's clang++" t3.log + echo "mcpp test --toolchain llvm@22.1.8 compiles with clang OK" + else + echo "NOT MEASURED: llvm@22.1.8 is not installed in this home" + fi + ;; +esac + +# ── 4. a spec the command line gave is refused as the command line's ───── +if "$MCPP" build --toolchain nosuch@1.0 > t4.log 2>&1; then + fail "an unknown toolchain from --toolchain was accepted" t4.log +fi +grep -q "^error: --toolchain = 'nosuch@1.0'" t4.log \ + || fail "the refusal does not name --toolchain as where the value was written" t4.log +if grep -q "\[toolchain\]\." t4.log; then + fail "the refusal credits a manifest key with a command-line value" t4.log +fi +echo "an unknown toolchain from --toolchain is refused naming --toolchain OK" + +echo "PASS: 671_toolchain_is_an_option_of_run_test_and_pack" diff --git a/tests/e2e/672_a_runner_receives_the_files_the_artifact_carries.sh b/tests/e2e/672_a_runner_receives_the_files_the_artifact_carries.sh new file mode 100644 index 000000000..24374053e --- /dev/null +++ b/tests/e2e/672_a_runner_receives_the_files_the_artifact_carries.sh @@ -0,0 +1,218 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 672_a_runner_receives_the_files_the_artifact_carries.sh -- every runner of +# `mcpp run` and `mcpp test` receives MCPP_RUNTIME_FILES (#634 A6). +# +# A runner receives the artifact's path and nothing else. A runner that moves +# the artifact -- `adb-run` pushes the program to a device -- moved only the +# program, and a test reading its deployed data failed on an API 34 emulator +# with `open failed: /data/local/tmp/data/data.txt` while it passed on the host. +# +# The variable names a file with one line per file the artifact reads or loads +# from its own directory: the destination relative to the artifact's +# directory, a TAB, and the absolute path of the staged file. It lists the +# `[runtime] deploy` entries and the shared libraries the plan links. +# +# No device is needed and none is used: the runner is a script that records +# the variable and the list it names, then executes the program. +# +# Criteria: +# 1. `mcpp run` through a runner: the list names `data/data.txt` and the +# dependency's shared library, each with the absolute path of an existing +# file in the output tree; +# 2. `mcpp test` through the same runner: one list per test program, and a +# test discovered in a subdirectory names its files with `../` (in a +# project without a shared dependency: a test in a subdirectory does not +# load a graph-built shared library on 2026.9.14.1 either, because the +# consumer's `$ORIGIN` rpath names the subdirectory); +# 3. a project that deploys nothing and links no shared library still +# receives the variable, naming an existing empty file; +# 4. negative direction: `--no-runner` executes the program without the +# variable, and a distributable run with `--format` receives an empty +# list, because a distributable holds its own files. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p rec fw/src app/src app/share app/tests + +cat > "$TMP/runner.sh" < "$TMP/rec/\$name.rec" +else + echo "UNSET" > "$TMP/rec/\$name.rec" +fi +exec "\$@" +EOF +chmod +x "$TMP/runner.sh" + +cat > fw/mcpp.toml <<'EOF' +[package] +name = "fw" +version = "0.1.0" + +[targets.fw] +kind = "shared" +EOF +cat > fw/src/fw.cppm <<'EOF' +export module fw; +export int fw_value() { return 42; } +EOF + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +fw = { path = "../fw" } + +[runtime] +deploy = [ { from = "share/data.txt", to = "data" } ] +EOF +printf 'payload\n' > app/share/data.txt +cat > app/src/main.cpp <<'EOF' +import std; +import fw; +int main() { + std::println("APP-RAN {}", fw_value()); + std::println("RUNTIME_FILES={}", std::getenv("MCPP_RUNTIME_FILES") ? "set" : "unset"); + return 0; +} +EOF +cat > app/tests/reads.cpp <<'EOF' +import std; +import fw; +int main() { return fw_value() == 42 ? 0 : 1; } +EOF + +cd app +"$MCPP" build > b0.log 2>&1 || fail "the initial build failed" b0.log +HOST=$(ls target | head -1) +[ -n "$HOST" ] || fail "could not determine the host triple from target/" b0.log +printf '\n[target.%s]\nrunner = ["%s"]\n' "$HOST" "$TMP/runner.sh" >> mcpp.toml + +# field -> the absolute path the record pairs with it +field() { awk -F'\t' -v d="$2" '$1 == d { print $2 }' "$1"; } + +# ── 1. mcpp run ──────────────────────────────────────────────────────────── +"$MCPP" run > r1.log 2>&1 || fail "mcpp run through the runner failed" r1.log +grep -q "APP-RAN 42" r1.log || fail "the program did not run through the runner" r1.log +rec="$TMP/rec/app.rec" +[ -f "$rec" ] || fail "the runner recorded nothing" r1.log +grep -q '^UNSET$' "$rec" && fail "the runner did not receive MCPP_RUNTIME_FILES" "$rec" +list=$(sed -n 's/^LIST=//p' "$rec") +[ -f "$list" ] || fail "MCPP_RUNTIME_FILES does not name an existing file" "$rec" +data=$(field "$rec" "data/data.txt") +[ -n "$data" ] || fail "the list does not name data/data.txt" "$rec" +case "$data" in /*) ;; *) fail "the source of data/data.txt is not absolute: $data" "$rec" ;; esac +[ "$(cat "$data")" = "payload" ] || fail "data/data.txt pairs with a file that is not the deployed one" "$rec" +lib=$(awk -F'\t' '$1 ~ /^libfw\.(so|dylib)$/ { print $2 }' "$rec") +[ -n "$lib" ] && [ -f "$lib" ] || fail "the list does not name the dependency's shared library" "$rec" +tabs=$(awk -F'\t' 'NF != 2' "$rec" | grep -vc '^LIST=' || true) +[ "$tabs" -eq 0 ] || fail "a line of the list is not " "$rec" +grep -q "RUNTIME_FILES=set" r1.log || fail "the program under the runner did not inherit the variable" r1.log +echo "mcpp run hands the runner data/data.txt and libfw OK" + +# ── 2a. mcpp test ────────────────────────────────────────────────────────── +rm -f "$TMP"/rec/*.rec +"$MCPP" test > t2.log 2>&1 || fail "mcpp test through the runner failed" t2.log +[ -f "$TMP/rec/reads.rec" ] || fail "the runner recorded nothing for test reads" t2.log +grep -q '^UNSET$' "$TMP/rec/reads.rec" && fail "test reads' runner did not receive the variable" "$TMP/rec/reads.rec" +[ -n "$(field "$TMP/rec/reads.rec" "data/data.txt")" ] \ + || fail "test reads does not carry data/data.txt" "$TMP/rec/reads.rec" +awk -F'\t' '$1 ~ /^libfw\.(so|dylib)$/' "$TMP/rec/reads.rec" | grep -q . \ + || fail "test reads does not carry the shared library" "$TMP/rec/reads.rec" +echo "mcpp test hands the test program its list OK" + +# ── 4a. --no-runner: no variable ─────────────────────────────────────────── +"$MCPP" run --no-runner > r4.log 2>&1 || fail "mcpp run --no-runner failed" r4.log +grep -q "RUNTIME_FILES=unset" r4.log || fail "a run without a runner received the variable" r4.log +echo "--no-runner executes the program without the variable OK" + +# ── 2b. a test in a subdirectory names its files with ../ ───────────────── +cd "$TMP" +"$MCPP" new deep > /dev/null +cd deep +mkdir -p share tests/unit +printf 'payload\n' > share/data.txt +printf '\n[runtime]\ndeploy = [ { from = "share/data.txt", to = "data" } ]\n' >> mcpp.toml +printf 'int main() { return 0; }\n' > tests/unit/nested.cpp +"$MCPP" build > b2b.log 2>&1 || fail "the project with a nested test did not build" b2b.log +printf '\n[target.%s]\nrunner = ["%s"]\n' "$HOST" "$TMP/runner.sh" >> mcpp.toml +rm -f "$TMP"/rec/*.rec +"$MCPP" test > t2b.log 2>&1 || fail "mcpp test of the nested test failed" t2b.log +for t in nested test_smoke; do + [ -f "$TMP/rec/$t.rec" ] && ! grep -q '^UNSET$' "$TMP/rec/$t.rec" \ + || fail "test $t's runner did not receive the variable" t2b.log "$TMP/rec/$t.rec" +done +nested_data=$(field "$TMP/rec/nested.rec" "../data/data.txt") +[ -n "$nested_data" ] && [ -f "$nested_data" ] \ + || fail "test unit/nested does not name its data with ../" "$TMP/rec/nested.rec" +[ -n "$(field "$TMP/rec/test_smoke.rec" "data/data.txt")" ] \ + || fail "the top-level test does not name its data without ../" "$TMP/rec/test_smoke.rec" +[ "$(sed -n 's/^LIST=//p' "$TMP/rec/nested.rec")" != "$(sed -n 's/^LIST=//p' "$TMP/rec/test_smoke.rec")" ] \ + || fail "two test programs share one list" "$TMP/rec/nested.rec" "$TMP/rec/test_smoke.rec" +echo "a test in a subdirectory names its files with ../, one list per test OK" + +# ── 3. nothing to carry: an empty list, still named ──────────────────────── +cd "$TMP" +"$MCPP" new lone > /dev/null +cd lone +"$MCPP" build > b3.log 2>&1 || fail "the plain project did not build" b3.log +printf '\n[target.%s]\nrunner = ["%s"]\n' "$HOST" "$TMP/runner.sh" >> mcpp.toml +"$MCPP" run > r3.log 2>&1 || fail "mcpp run of the plain project failed" r3.log +rec="$TMP/rec/lone.rec" +[ -f "$rec" ] && ! grep -q '^UNSET$' "$rec" || fail "the plain project's runner did not receive the variable" "$rec" +list=$(sed -n 's/^LIST=//p' "$rec") +[ -f "$list" ] && [ ! -s "$list" ] || fail "the plain project's list is not an existing empty file" "$rec" +echo "a project with nothing to carry receives an empty list OK" + +# ── 4b. a distributable holds its own files: an empty list ──────────────── +cat > copy.sh <<'EOF' +#!/usr/bin/env bash +set -e +cp "$1" "$2" +chmod +x "$2" +EOF +chmod +x copy.sh +cat > build.mcpp <<'EOF' +import mcpp; +#include +#include +int main() { + mcpp::provides_pack_format("blob"); + if (std::string_view(mcpp::pack_format()) != "blob") return 0; + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/lone.blob"; + mcpp::action a; + a.id = "blob"; + a.role = "artifact"; + a.description = "blob"; + a.arg((root + "/copy.sh").c_str()) + .arg("${mcpp.target_file:lone}") + .arg(out.c_str()) + .input("${mcpp.target_file:lone}") + .output(out.c_str()) + .submit(); + return 0; +} +EOF +"$MCPP" run --format blob > r5.log 2>&1 || fail "mcpp run --format blob failed" r5.log +rec="$TMP/rec/lone.blob.rec" +[ -f "$rec" ] && ! grep -q '^UNSET$' "$rec" || fail "the distributable's runner did not receive the variable" r5.log "$rec" +list=$(sed -n 's/^LIST=//p' "$rec") +[ -f "$list" ] && [ ! -s "$list" ] || fail "the distributable's list is not an existing empty file" "$rec" +echo "a distributable run with --format receives an empty list OK" + +echo "PASS: 672_a_runner_receives_the_files_the_artifact_carries" diff --git a/tests/e2e/673_test_discover_names_where_tests_are.sh b/tests/e2e/673_test_discover_names_where_tests_are.sh new file mode 100644 index 000000000..b0b5649dc --- /dev/null +++ b/tests/e2e/673_test_discover_names_where_tests_are.sh @@ -0,0 +1,97 @@ +#!/usr/bin/env bash +# requires: +# 673_test_discover_names_where_tests_are.sh -- `[test] discover` (#634 A5). +# +# The test set was the literal pattern `tests/**/*.cpp`, one program per file. +# A project whose `tests/` belongs to another build system could not point +# discovery elsewhere. `discover` takes globs in the vocabulary of `[build] +# sources`, `!` exclusions included; a test's name is its path relative to the +# fixed directory of the glob that found it, so the default names every test +# as every earlier release did. +# +# Criteria: +# 1. negative direction first: without the key, `--list` names +# `tests/a_ok.cpp` and `tests/unit/b_ok.cpp` as `a_ok` and `unit/b_ok`; +# 2. `discover = ["checks/**/*.cpp"]` runs `checks/c_ok.cpp` and not the +# failing `tests/fails.cpp`, and names it `c_ok`; +# 3. a `!` glob excludes a file another glob found; +# 4. `discover = []` discovers nothing and says where it looked; +# 5. two files that map to one name are refused, naming both; +# 6. a value that is not an array of strings is refused, naming the key. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +"$MCPP" new disc > /dev/null +cd disc +rm -f tests/*.cpp +mkdir -p tests/unit checks/deep +printf 'int main() { return 0; }\n' > tests/a_ok.cpp +printf 'int main() { return 0; }\n' > tests/unit/b_ok.cpp +printf 'int main() { return 1; }\n' > tests/fails.cpp +printf 'int main() { return 0; }\n' > checks/c_ok.cpp +printf 'int main() { return 1; }\n' > checks/deep/skipped.cpp +cp mcpp.toml mcpp.toml.base + +# ── 1. no key: today's names ─────────────────────────────────────────────── +"$MCPP" test --list > l1.log 2>&1 || fail "mcpp test --list failed" l1.log +grep -qx "a_ok" l1.log || fail "the default does not name tests/a_ok.cpp 'a_ok'" l1.log +grep -qx "unit/b_ok" l1.log || fail "the default does not name tests/unit/b_ok.cpp 'unit/b_ok'" l1.log +grep -qx "c_ok" l1.log && fail "the default discovered a file outside tests/" l1.log +echo "without [test] the names are unchanged OK" + +# ── 2 and 3. discover elsewhere, with an exclusion ──────────────────────── +cat mcpp.toml.base > mcpp.toml +cat >> mcpp.toml <<'EOF' + +[test] +discover = ["checks/**/*.cpp", "!checks/deep/**"] +EOF +"$MCPP" test --list > l2.log 2>&1 || fail "mcpp test --list with discover failed" l2.log +grep -qx "c_ok" l2.log || fail "discover did not name checks/c_ok.cpp 'c_ok'" l2.log +grep -q "fails\|a_ok" l2.log && fail "discover still discovered tests/" l2.log +grep -q "skipped" l2.log && fail "the ! glob did not exclude checks/deep/skipped.cpp" l2.log +"$MCPP" test > t2.log 2>&1 || fail "mcpp test with discover failed (a file outside the set ran?)" t2.log +grep -q "c_ok ... ok" t2.log || fail "checks/c_ok.cpp did not run" t2.log +grep -q "fails" t2.log && fail "tests/fails.cpp ran" t2.log +echo "discover runs checks/c_ok.cpp, excludes checks/deep, ignores tests/ OK" + +# ── 4. an empty list discovers nothing, and says where it looked ────────── +cat mcpp.toml.base > mcpp.toml +printf '\n[test]\ndiscover = []\n' >> mcpp.toml +"$MCPP" test > t4.log 2>&1 || fail "mcpp test with discover = [] failed" t4.log +grep -q "no tests found (\[test\] discover = \[\])" t4.log \ + || fail "an empty discover did not say where it looked" t4.log +echo "discover = [] discovers nothing and names the key OK" + +# ── 5. two files, one name ──────────────────────────────────────────────── +cat mcpp.toml.base > mcpp.toml +printf '\n[test]\ndiscover = ["tests/*.cpp", "checks/*.cpp"]\n' >> mcpp.toml +printf 'int main() { return 0; }\n' > checks/a_ok.cpp +set +e +"$MCPP" test --list > t5.log 2>&1; rc=$? +set -e +[ "$rc" -ne 0 ] || fail "two files mapping to 'a_ok' were accepted" t5.log +grep -q "duplicate test name 'a_ok'" t5.log || fail "the refusal does not name the test" t5.log +grep -q "tests/a_ok.cpp" t5.log && grep -q "checks/a_ok.cpp" t5.log \ + || fail "the refusal does not name both files" t5.log +rm -f checks/a_ok.cpp +echo "two files mapping to one name are refused, naming both OK" + +# ── 6. a wrong type ──────────────────────────────────────────────────────── +cat mcpp.toml.base > mcpp.toml +printf '\n[test]\ndiscover = "tests/**/*.cpp"\n' >> mcpp.toml +# `mcpp test`, not `--list`: the listing is best effort over a manifest that +# does not load, so it lists the default set rather than failing. +set +e +"$MCPP" test > t6.log 2>&1; rc=$? +set -e +[ "$rc" -ne 0 ] || fail "a string discover was accepted" t6.log +grep -q "\[test\] discover must be an array" t6.log || fail "the refusal does not name the key" t6.log +echo "a discover that is not an array is refused OK" + +echo "PASS: 673_test_discover_names_where_tests_are" diff --git a/tests/e2e/674_a_format_is_run_by_the_runner_named_after_it.sh b/tests/e2e/674_a_format_is_run_by_the_runner_named_after_it.sh new file mode 100644 index 000000000..a5ecc9196 --- /dev/null +++ b/tests/e2e/674_a_format_is_run_by_the_runner_named_after_it.sh @@ -0,0 +1,175 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 674_a_format_is_run_by_the_runner_named_after_it.sh -- `mcpp run --format +# ` without `--runner` runs the distributable through the named runner `` +# when one exists, and refuses a directory that no runner reaches before any +# spawn (#634 B3). +# +# Measured on macos-15 against 2026.9.14.1: with a named runner `app` declared, +# `mcpp run --format app --runner app` ran the bundle and returned its status, +# while `mcpp run --format app` alone handed the bundle directory to the kernel: +# +# ...RApp.app could not be started: Permission denied (error 13) exit 126 +# +# The runner is modelled by scripts that record their operand and then run the +# program, and the bundle by a directory holding a copy of the program, so the +# criteria hold on every POSIX host. +# +# Criteria: +# 1. a runner a build program names after the format (`mcpp::runner("blob", +# ...)`, as `dist-apple` does for `app`) receives the distributable, the +# status line says Running, and the program's status comes back; +# 2. a typed `--runner` still wins over the format's name; +# 3. negative direction: a plain `mcpp run` executes the link output without +# the format's runner; +# 4. a directory distributable that no runner reaches is refused before the +# spawn, with status 126, naming the runner `bundle` that would reach it; +# 5. a manifest runner named `bundle` then receives the directory; +# 6. negative direction of 4: a default runner reaches the directory, and no +# refusal is printed. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +for r in blob other bundle default; do + cat > "$TMP/runner-$r.sh" < mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +cat > src/main.cpp <<'EOF' +#include +int main() { std::puts("PROGRAM-RAN"); return 7; } +EOF +cat > copy.sh <<'EOF' +#!/usr/bin/env bash +set -e +cp "$1" "$2" +chmod +x "$2" +EOF +cat > bundle.sh <<'EOF' +#!/usr/bin/env bash +set -e +rm -rf "$2" +mkdir -p "$2/Contents/MacOS" +cp "$1" "$2/Contents/MacOS/app" +chmod +x "$2/Contents/MacOS/app" +EOF +chmod +x copy.sh bundle.sh +cat > build.mcpp < +#include +int main() { + mcpp::provides_pack_format("blob"); + mcpp::provides_pack_format("bundle"); + mcpp::runner("blob", "$TMP/runner-blob.sh"); + const std::string_view fmt = mcpp::pack_format(); + if (fmt != "blob" && fmt != "bundle") return 0; + const std::string root = mcpp::manifest_dir(); + const bool dir = fmt == "bundle"; + const std::string out = std::string(mcpp::out_dir()) + (dir ? "/app.bundle" : "/app.blob"); + mcpp::action a; + a.id = dir ? "bundle" : "blob"; + a.role = "artifact"; + a.description = a.id; + a.arg((root + (dir ? "/bundle.sh" : "/copy.sh")).c_str()) + .arg("\${mcpp.target_file:app}") + .arg(out.c_str()) + .input("\${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +EOF + +set +e +"$MCPP" build > b0.log 2>&1 || fail "the initial build failed" b0.log +set -e +HOST=$(ls target | head -1) +[ -n "$HOST" ] || fail "could not determine the host triple from target/" b0.log +printf '\n[target.%s.runners]\nother = ["%s"]\n' "$HOST" "$TMP/runner-other.sh" >> mcpp.toml +cp mcpp.toml mcpp.toml.base + +# ── 1. the runner named after the format ────────────────────────────────── +set +e +"$MCPP" run --format blob > r1.log 2>&1; rc=$? +set -e +grep -q "RUNNER-blob: .*/app\.blob$" r1.log \ + || fail "the runner named 'blob' did not receive the distributable" r1.log +grep -q "PROGRAM-RAN" r1.log || fail "the program did not run through the runner" r1.log +[ "$rc" -eq 7 ] || fail "the program's status 7 did not come back (got $rc)" r1.log +grep -q "Running \`.*runner-blob.sh" r1.log \ + || fail "the status line does not say Running for the format's runner" r1.log +echo "mcpp run --format blob runs through the runner named blob OK" + +# ── 2. a typed --runner wins ─────────────────────────────────────────────── +set +e +"$MCPP" run --format blob --runner other > r2.log 2>&1; rc=$? +set -e +grep -q "RUNNER-other: .*/app\.blob$" r2.log || fail "--runner other did not win" r2.log +grep -q "RUNNER-blob" r2.log && fail "the format's runner ran although --runner was typed" r2.log +[ "$rc" -eq 7 ] || fail "--runner other lost the program's status (got $rc)" r2.log +echo "a typed --runner wins over the format's name OK" + +# ── 3. a plain run is unaffected ─────────────────────────────────────────── +set +e +"$MCPP" run > r3.log 2>&1; rc=$? +set -e +grep -q "RUNNER-" r3.log && fail "a plain run went through a runner" r3.log +grep -q "PROGRAM-RAN" r3.log || fail "a plain run did not run the program" r3.log +[ "$rc" -eq 7 ] || fail "a plain run lost the program's status (got $rc)" r3.log +echo "a plain mcpp run executes the link output directly OK" + +# ── 4. a directory no runner reaches ─────────────────────────────────────── +set +e +"$MCPP" run --format bundle > r4.log 2>&1; rc=$? +set -e +[ "$rc" -eq 126 ] || fail "the directory without a runner did not exit 126 (got $rc)" r4.log +grep -q "produced a directory" r4.log || fail "the refusal does not say the distributable is a directory" r4.log +grep -q "a runner named 'bundle'" r4.log || fail "the refusal does not name the runner 'bundle'" r4.log +grep -q "\[target\.$HOST\.runners\]" r4.log || fail "the refusal does not name the manifest table" r4.log +grep -q "Permission denied\|Is a directory" r4.log && fail "the directory reached the kernel" r4.log +grep -q "PROGRAM-RAN\|RUNNER-" r4.log && fail "something ran despite the refusal" r4.log +echo "a directory no runner reaches is refused before the spawn OK" + +# ── 5. a manifest runner named after the format reaches the directory ───── +printf 'bundle = ["%s"]\n' "$TMP/runner-bundle.sh" >> mcpp.toml +set +e +"$MCPP" run --format bundle > r5.log 2>&1; rc=$? +set -e +grep -q "RUNNER-bundle: .*/app\.bundle$" r5.log || fail "the runner named 'bundle' did not receive the directory" r5.log +[ "$rc" -eq 7 ] || fail "the bundle's program status did not come back (got $rc)" r5.log +echo "a manifest runner named bundle runs the directory OK" + +# ── 6. a default runner reaches the directory without a refusal ─────────── +cp mcpp.toml.base mcpp.toml +printf '\n[target.%s]\nrunner = ["%s"]\n' "$HOST" "$TMP/runner-default.sh" >> mcpp.toml +set +e +"$MCPP" run --format bundle > r6.log 2>&1; rc=$? +set -e +grep -q "RUNNER-default: .*/app\.bundle$" r6.log || fail "the default runner did not receive the directory" r6.log +grep -q "no runner reaches it" r6.log && fail "a refusal was printed although a default runner exists" r6.log +[ "$rc" -eq 7 ] || fail "the default runner lost the status (got $rc)" r6.log +echo "a default runner reaches the directory, no refusal OK" + +echo "PASS: 674_a_format_is_run_by_the_runner_named_after_it" diff --git a/tests/e2e/675_android_test_programs_carry_their_cxx_runtime.sh b/tests/e2e/675_android_test_programs_carry_their_cxx_runtime.sh new file mode 100644 index 000000000..d4f3f2b2b --- /dev/null +++ b/tests/e2e/675_android_test_programs_carry_their_cxx_runtime.sh @@ -0,0 +1,102 @@ +#!/usr/bin/env bash +# requires: elf gcc android-ndk +# 675_android_test_programs_carry_their_cxx_runtime.sh -- the static C++ +# runtime is located by asking the driver for the effective target, so the +# self-contained contract holds on the Android rows (#634 A6). +# +# The archive lookup searched the LLVM payload's `lib/` alone. The NDK keeps +# its archives in the sysroot per API level, the search found nothing, and the +# contract degraded: +# +# warning: cxx_runtime: test target: this toolchain ships no +# libc++.a/libc++abi.a; using toolchain-coupled +# +# Every test program then needed `libc++_shared.so`, and on an API 34 emulator +# every one of them stopped with `CANNOT LINK EXECUTABLE ... library +# "libc++_shared.so" not found`. The driver answers directly for the target +# the link names, API level included; the NDK's per-API `libc++.a` is a linker +# script, `INPUT(-lc++_static -lc++abi)`. +# +# No device is needed: the runner is a script that exits 0, and the criteria +# read the programs' dynamic sections and the build graph. +# +# Criteria: +# 1. test programs on `x86_64-linux-android` name no `libc++_shared.so`, and +# the degradation warning is not printed; +# 2. their link line names the NDK sysroot's per-API `libc++.a`; +# 3. a shared library that asks for the self-contained contract hides the +# runtime it embeds: its link names `--exclude-libs,libc++_static.a`, the +# archive the script opens, and `std::__ndk1::to_string(int)` is not among +# its dynamic symbols, while it still names no `libc++_shared.so`. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +readelf_bin="" +for d in "$HOME"/.xlings/data/xpkgs/xim-x-android-ndk/*/toolchains/llvm/prebuilt/*/bin \ + "${MCPP_HOME:-$HOME/.mcpp}"/registry/data/xpkgs/xim-x-android-ndk/*/toolchains/llvm/prebuilt/*/bin; do + if [ -x "$d/llvm-readelf" ] && [ -x "$d/llvm-nm" ]; then readelf_bin="$d"; fi +done +[ -n "$readelf_bin" ] || fail "the android-ndk capability was detected but no llvm-readelf was found" + +printf '#!/bin/sh\nexit 0\n' > "$TMP/ok-runner.sh" +chmod +x "$TMP/ok-runner.sh" + +# ── 1 and 2. test programs ──────────────────────────────────────────────── +"$MCPP" new droid > /dev/null +cd droid +cat > tests/strings.cpp <<'EOF' +#include +#include +int main() { std::string s = std::to_string(42); std::puts(s.c_str()); return s == "42" ? 0 : 1; } +EOF +printf '\n[target.x86_64-linux-android]\nrunner = ["%s"]\n' "$TMP/ok-runner.sh" >> mcpp.toml +"$MCPP" test --target x86_64-linux-android > t1.log 2>&1 || fail "mcpp test on the Android row failed" t1.log +grep -q "ships no libc++.a" t1.log && fail "the contract still degraded to toolchain-coupled" t1.log +prog=$(find target/x86_64-linux-android -path '*/bin/strings' -type f | head -1) +[ -n "$prog" ] || fail "no test program was built for x86_64-linux-android" t1.log +"$readelf_bin/llvm-readelf" -d "$prog" > needed1.txt +grep -q "NEEDED" needed1.txt || fail "the test program's dynamic section could not be read" needed1.txt +grep -q "libc++_shared.so" needed1.txt && fail "the test program still needs libc++_shared.so" needed1.txt +grep -q "sysroot/usr/lib/x86_64-linux-android/[0-9]*/libc++.a" target/x86_64-linux-android/*/build.ninja \ + || fail "the link line does not name the sysroot's per-API libc++.a" t1.log +echo "Android test programs carry their C++ runtime OK" + +# ── 3. a self-contained shared library hides what it embeds ─────────────── +cd "$TMP" +mkdir -p fw/src +cat > fw/mcpp.toml <<'EOF' +[package] +name = "fw" +version = "0.1.0" + +[build] +cxx_runtime = { shared = "self-contained" } + +[targets.fw] +kind = "shared" +EOF +cat > fw/src/fw.cppm <<'EOF' +module; +#include +export module fw; +export std::string fw_name() { return "fw-" + std::to_string(7); } +EOF +cd fw +"$MCPP" build --target x86_64-linux-android > b3.log 2>&1 || fail "the shared library did not build" b3.log +so=$(find target/x86_64-linux-android -name 'libfw.so' -type f | head -1) +[ -n "$so" ] || fail "no libfw.so was built" b3.log +grep -q "exclude-libs,libc++_static.a" target/x86_64-linux-android/*/build.ninja \ + || fail "the link does not hide the archive the linker script opens" b3.log +"$readelf_bin/llvm-readelf" -d "$so" > needed3.txt +grep -q "libc++_shared.so" needed3.txt && fail "the self-contained shared library needs libc++_shared.so" needed3.txt +"$readelf_bin/llvm-nm" -D --defined-only -C "$so" > dyn3.txt +grep -q "std::__ndk1::to_string(int)" dyn3.txt \ + && fail "the shared library exports the C++ runtime it embeds" dyn3.txt +echo "a self-contained Android shared library hides the embedded runtime OK" + +echo "PASS: 675_android_test_programs_carry_their_cxx_runtime" diff --git a/tests/e2e/677_a_conditional_dependency_replaces_the_unconditional_one.sh b/tests/e2e/677_a_conditional_dependency_replaces_the_unconditional_one.sh new file mode 100755 index 000000000..5b8d8e296 --- /dev/null +++ b/tests/e2e/677_a_conditional_dependency_replaces_the_unconditional_one.sh @@ -0,0 +1,104 @@ +#!/usr/bin/env bash +# requires: elf +# 677 -- a matching `[target..dependencies]` declaration REPLACES the +# unconditional declaration of the same identity on the rows the selector +# matches (#634, A1). Before, the merge kept the unconditional entry, so a +# conditional `linkage = "shared"` was dropped on its own row without a word. +# +# Legs: +# A. The conditional table matches this row: `libfw.so` is built and the +# program NEEDs it, and the resolution record names the conditional table. +# B. The conditional table does not match: the unconditional declaration +# stands (a static link), and the record names `[dependencies]`. +# C. The modifier-only spelling `demo.fw = { linkage = "shared" }` names no +# source. It still fails, and the warning names the conditional table and +# restates the dependency with the source the unconditional table wrote. +# D. A misspelled sub-table of `[target.]` is reported instead of +# doing nothing. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +# table_of -> the tables its requests came from +table_of() { + python3 - "$1" "$2" <<'PY' +import json, sys +doc = json.load(open(sys.argv[1])) +for p in doc["graph"]["packages"]: + if p["package"]["canonical"] == sys.argv[2]: + print(" ".join(r["table"] for r in p["requested_by"])) +PY +} + +mkdir -p "$TMP/fw/src" +cat > "$TMP/fw/mcpp.toml" <<'TOML' +[package] +namespace = "demo" +name = "fw" +version = "0.1.0" +[targets.fw] +kind = "lib" +TOML +printf 'export module fw;\nexport int fw_anchor() { return 41; }\n' > "$TMP/fw/src/fw.cppm" + +make_app() { # make_app + mkdir -p "$TMP/$1/src" + printf '[package]\nname = "%s"\nversion = "0.1.0"\n%s' "$1" "$2" > "$TMP/$1/mcpp.toml" + printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > "$TMP/$1/src/main.cpp" +} + +# ── A ────────────────────────────────────────────────────────────────────── +make_app matching '[dependencies] +demo.fw = { path = "../fw" } +[target.'"'"'cfg(os = "linux")'"'"'.dependencies] +demo.fw = { path = "../fw", linkage = "shared" } +' +( cd "$TMP/matching" && "$MCPP" build > build.log 2>&1 ) || fail "A: build failed" "$TMP/matching/build.log" +so=$(find "$TMP/matching/target" -name libfw.so | head -1) +[ -n "$so" ] || fail "A: the conditional linkage did not produce libfw.so" "$TMP/matching/build.log" +exe=$(find "$TMP/matching/target" -type f -name matching -perm -u+x | head -1) +readelf -d "$exe" | grep -q 'NEEDED.*libfw.so' || fail "A: the program does not NEED libfw.so" +json=$(find "$TMP/matching/target" -name resolution.json | head -1) +t=$(table_of "$json" "demo.fw@0.1.0") +[ "$t" = "[target.'cfg(os = \"linux\")'.dependencies]" ] || fail "A: graph table is '$t'" "$json" + +# ── B ────────────────────────────────────────────────────────────────────── +make_app other '[dependencies] +demo.fw = { path = "../fw" } +[target.'"'"'cfg(os = "windows")'"'"'.dependencies] +demo.fw = { path = "../fw", linkage = "shared" } +' +( cd "$TMP/other" && "$MCPP" build > build.log 2>&1 ) || fail "B: build failed" "$TMP/other/build.log" +[ -z "$(find "$TMP/other/target" -name libfw.so | head -1)" ] || fail "B: a non-matching selector produced libfw.so" +json=$(find "$TMP/other/target" -name resolution.json | head -1) +t=$(table_of "$json" "demo.fw@0.1.0") +[ "$t" = "[dependencies]" ] || fail "B: graph table is '$t'" "$json" + +# ── C ────────────────────────────────────────────────────────────────────── +make_app modifier '[dependencies] +demo.fw = { path = "../fw" } +[target.'"'"'cfg(os = "linux")'"'"'.dependencies] +demo.fw = { linkage = "shared" } +' +if ( cd "$TMP/modifier" && "$MCPP" build > build.log 2>&1 ); then + fail "C: a table that names no source built" "$TMP/modifier/build.log" +fi +grep -qF "[target.'cfg(os = \"linux\")'.dependencies] demo.fw.linkage = \"shared\"" "$TMP/modifier/build.log" \ + || fail "C: the warning does not name the conditional table" "$TMP/modifier/build.log" +grep -qF 'demo.fw = { path = "../fw", linkage = "shared" }' "$TMP/modifier/build.log" \ + || fail "C: the warning does not restate the dependency with its source" "$TMP/modifier/build.log" + +# ── D ────────────────────────────────────────────────────────────────────── +make_app typo '[dependencies] +demo.fw = { path = "../fw" } +[target.'"'"'cfg(os = "linux")'"'"'.dependecies] +demo.fw = { path = "../fw", linkage = "shared" } +' +( cd "$TMP/typo" && "$MCPP" build > build.log 2>&1 ) || fail "D: build failed" "$TMP/typo/build.log" +grep -qF "[target.'cfg(os = \"linux\")'.dependecies] is not a section mcpp reads" "$TMP/typo/build.log" \ + || fail "D: the misspelled section was not reported" "$TMP/typo/build.log" + +echo "OK" diff --git a/tests/e2e/678_a_row_states_a_library_form.sh b/tests/e2e/678_a_row_states_a_library_form.sh new file mode 100755 index 000000000..cbbc44a59 --- /dev/null +++ b/tests/e2e/678_a_row_states_a_library_form.sh @@ -0,0 +1,116 @@ +#!/usr/bin/env bash +# requires: elf +# 678 -- `[target..targets.] kind` is the per-row form of +# `[targets.] kind` (#634, A1): a framework linked statically on one row +# states on another that it must be one shared copy, in its own manifest, +# and every consumer keeps a single unconditional dependency line. +# +# Legs: +# A. The row matches: a consumer that asks for nothing gets `libfw.so`, and +# the resolution record gives the reason `row-kind`. +# B. The row does not match: a static link, reason `default`. +# C. A consumer that asks for `linkage = "static"` is warned, and the warning +# names the row's statement; `--strict` turns it into a failure. +# D. A row that names no target, a program target, or a non-library kind is +# refused. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +reason_of() { # reason_of + python3 - "$1" "$2" <<'PY' +import json, sys +doc = json.load(open(sys.argv[1])) +for p in doc["graph"]["packages"]: + if p["package"]["canonical"] == sys.argv[2]: + print(p.get("link", {}).get("form", ""), p.get("link", {}).get("reason", "")) +PY +} + +make_fw() { # make_fw [extra manifest text] + mkdir -p "$TMP/$1/src" + cat > "$TMP/$1/mcpp.toml" < "$TMP/$1/src/fw.cppm" +} +make_app() { # make_app [edge options] + mkdir -p "$TMP/$1/src" + printf '[package]\nname = "%s"\nversion = "0.1.0"\n[dependencies]\ndemo.fw = { path = "../%s"%s }\n' \ + "$1" "$2" "$3" > "$TMP/$1/mcpp.toml" + printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > "$TMP/$1/src/main.cpp" +} + +make_fw fwlinux linux +make_fw fwwin windows + +# ── A ────────────────────────────────────────────────────────────────────── +make_app a fwlinux +( cd "$TMP/a" && "$MCPP" build > build.log 2>&1 ) || fail "A: build failed" "$TMP/a/build.log" +[ -n "$(find "$TMP/a/target" -name libfw.so | head -1)" ] || fail "A: no libfw.so" "$TMP/a/build.log" +r=$(reason_of "$(find "$TMP/a/target" -name resolution.json | head -1)" "demo.fw@0.1.0") +[ "$r" = "shared row-kind" ] || fail "A: link record is '$r'" + +# ── B ────────────────────────────────────────────────────────────────────── +make_app b fwwin +( cd "$TMP/b" && "$MCPP" build > build.log 2>&1 ) || fail "B: build failed" "$TMP/b/build.log" +[ -z "$(find "$TMP/b/target" -name libfw.so | head -1)" ] || fail "B: a non-matching row built libfw.so" +r=$(reason_of "$(find "$TMP/b/target" -name resolution.json | head -1)" "demo.fw@0.1.0") +[ "$r" = "static default" ] || fail "B: link record is '$r'" + +# ── C ────────────────────────────────────────────────────────────────────── +make_app c fwlinux ', linkage = "static"' +( cd "$TMP/c" && "$MCPP" build > build.log 2>&1 ) || fail "C: build failed" "$TMP/c/build.log" +grep -qF "its manifest states [target.'cfg(os = \"linux\")'.targets.fw] kind = \"shared\"" "$TMP/c/build.log" \ + || fail "C: the degradation does not name the row's statement" "$TMP/c/build.log" +rm -rf "$TMP/c/target" +if ( cd "$TMP/c" && "$MCPP" build --strict > strict.log 2>&1 ); then + fail "C: --strict accepted the degradation" "$TMP/c/strict.log" +fi + +# ── D ────────────────────────────────────────────────────────────────────── +make_fw nosuch linux '[target.'"'"'cfg(os = "linux")'"'"'.targets.nope] +kind = "shared"' +make_app d1 nosuch +if ( cd "$TMP/d1" && "$MCPP" build > build.log 2>&1 ); then fail "D: a row naming no target was accepted"; fi +grep -qF "names no target of this package" "$TMP/d1/build.log" || fail "D: refusal wording" "$TMP/d1/build.log" + +mkdir -p "$TMP/prog/src" +cat > "$TMP/prog/mcpp.toml" <<'TOML' +[package] +name = "prog" +version = "0.1.0" +[targets.prog] +kind = "bin" +main = "src/main.cpp" +[target.'cfg(os = "linux")'.targets.prog] +kind = "shared" +TOML +printf 'int main() { return 0; }\n' > "$TMP/prog/src/main.cpp" +if ( cd "$TMP/prog" && "$MCPP" build > build.log 2>&1 ); then fail "D: a row made a program shared"; fi +grep -qF "is a program target" "$TMP/prog/build.log" || fail "D: program refusal wording" "$TMP/prog/build.log" + +cat > "$TMP/prog/mcpp.toml" <<'TOML' +[package] +name = "prog" +version = "0.1.0" +[targets.lib] +kind = "lib" +[target.'cfg(os = "linux")'.targets.lib] +kind = "bin" +TOML +if ( cd "$TMP/prog" && "$MCPP" build > build.log 2>&1 ); then fail "D: a row kind 'bin' was accepted"; fi +grep -qF "a row chooses between the library forms only" "$TMP/prog/build.log" || fail "D: kind refusal wording" "$TMP/prog/build.log" + +echo "OK" diff --git a/tests/e2e/679_a_path_dependency_takes_its_manifests_identity.sh b/tests/e2e/679_a_path_dependency_takes_its_manifests_identity.sh new file mode 100755 index 000000000..205d06334 --- /dev/null +++ b/tests/e2e/679_a_path_dependency_takes_its_manifests_identity.sh @@ -0,0 +1,101 @@ +#!/usr/bin/env bash +# 679 -- a `path` or `git` dependency's identity is the one its manifest +# declares, whatever key reached it (#634, A2; SPEC-001 §1.2). +# +# Before, only the short name was compared: `fw` reaching a manifest that +# declares `huxdemo.fw` resolved as `mcpplibs.fw`, and a second edge keyed +# `huxdemo.fw` over the same directory put the same module into the build +# twice, which the scanner then refused naming one file twice. +# +# Legs: +# A. Two edges over one directory, keyed `huxdemo.fw` (the application) and +# `fw` (a component): the build succeeds, the unit is compiled once, one +# warning names the component's key, its normalisation and the declared +# identity, and the resolution record shows one package with both keys. +# B. Keys that match the declaration warn nothing. +# C. Two keys over a manifest that declares no namespace are two identities +# over one source, and are refused before scanning. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p "$TMP/fw/src" "$TMP/comp/src" "$TMP/app/src" "$TMP/plain/src" +cat > "$TMP/fw/mcpp.toml" <<'TOML' +[package] +namespace = "huxdemo" +name = "fw" +version = "0.1.0" +[targets.fw] +kind = "lib" +TOML +printf 'export module fw;\nexport int fw_anchor() { return 7; }\n' > "$TMP/fw/src/fw.cppm" +cat > "$TMP/comp/mcpp.toml" <<'TOML' +[package] +namespace = "huxdemo" +name = "comp" +version = "0.1.0" +[targets.comp] +kind = "lib" +[dependencies] +fw = { path = "../fw" } +TOML +printf 'export module comp;\nimport fw;\nexport int comp_anchor() { return fw_anchor(); }\n' > "$TMP/comp/src/comp.cppm" +cat > "$TMP/app/mcpp.toml" <<'TOML' +[package] +name = "app" +version = "0.1.0" +[dependencies] +huxdemo.fw = { path = "../fw" } +huxdemo.comp = { path = "../comp" } +TOML +printf 'import fw;\nimport comp;\nint main() { return fw_anchor() + comp_anchor() == 14 ? 0 : 1; }\n' > "$TMP/app/src/main.cpp" + +# ── A ────────────────────────────────────────────────────────────────────── +( cd "$TMP/app" && "$MCPP" build > build.log 2>&1 ) || fail "A: build failed" "$TMP/app/build.log" +( cd "$TMP/app" && "$MCPP" run > run.log 2>&1 ) || fail "A: the program did not exit 0" "$TMP/app/run.log" +n=$(grep -c "that identity is used" "$TMP/app/build.log" || true) +[ "$n" = "1" ] || fail "A: expected one identity warning, saw $n" "$TMP/app/build.log" +grep "that identity is used" "$TMP/app/build.log" | grep -q "'fw'" \ + && grep "that identity is used" "$TMP/app/build.log" | grep -q "mcpplibs.fw" \ + && grep "that identity is used" "$TMP/app/build.log" | grep -q "huxdemo.fw" \ + || fail "A: the warning does not name the key, its normalisation and the declaration" "$TMP/app/build.log" +units=$(find "$TMP/app/target" -path '*/obj/*' -name 'fw.m.o' | wc -l | tr -d ' ') +[ "$units" = "1" ] || fail "A: fw.cppm compiled $units times" +json=$(find "$TMP/app/target" -name resolution.json | head -1) +keys=$(python3 - "$json" <<'PY' +import json, sys +doc = json.load(open(sys.argv[1])) +fw = [p for p in doc["graph"]["packages"] if p["package"]["name"] == "fw"] +print(len(fw), " ".join(sorted(r["key"] for r in fw[0]["requested_by"])) if fw else "") +PY +) +[ "$keys" = "1 fw huxdemo.fw" ] || fail "A: graph record reads '$keys'" "$json" + +# ── B ────────────────────────────────────────────────────────────────────── +sed 's/^fw = { path/huxdemo.fw = { path/' "$TMP/comp/mcpp.toml" > "$TMP/comp/mcpp.toml.new" +mv "$TMP/comp/mcpp.toml.new" "$TMP/comp/mcpp.toml" +rm -rf "$TMP/app/target" +( cd "$TMP/app" && "$MCPP" build > build2.log 2>&1 ) || fail "B: build failed" "$TMP/app/build2.log" +if grep -q "that identity is used" "$TMP/app/build2.log"; then + fail "B: matching keys produced an identity warning" "$TMP/app/build2.log" +fi + +# ── C ────────────────────────────────────────────────────────────────────── +mkdir -p "$TMP/nons/fw/src" "$TMP/nons/app/src" +printf '[package]\nname = "fw"\nversion = "0.1.0"\n[targets.fw]\nkind = "lib"\n' > "$TMP/nons/fw/mcpp.toml" +printf 'export module fw;\nexport int fw_anchor() { return 7; }\n' > "$TMP/nons/fw/src/fw.cppm" +printf '[package]\nname = "app"\nversion = "0.1.0"\n[dependencies]\na.fw = { path = "../fw" }\nb.fw = { path = "../fw" }\n' > "$TMP/nons/app/mcpp.toml" +printf 'import fw;\nint main() { return 0; }\n' > "$TMP/nons/app/src/main.cpp" +if ( cd "$TMP/nons/app" && "$MCPP" build > build.log 2>&1 ); then + fail "C: two identities over one source built" "$TMP/nons/app/build.log" +fi +grep -q "one source is reached as two packages" "$TMP/nons/app/build.log" \ + || fail "C: the refusal does not name the two identities" "$TMP/nons/app/build.log" +if grep -q "already provided\|is provided by package" "$TMP/nons/app/build.log"; then + fail "C: the refusal came from the scanner, not before scanning" "$TMP/nons/app/build.log" +fi + +echo "OK" diff --git a/tests/e2e/680_a_dependency_floor_on_the_android_api_level.sh b/tests/e2e/680_a_dependency_floor_on_the_android_api_level.sh new file mode 100755 index 000000000..630334358 --- /dev/null +++ b/tests/e2e/680_a_dependency_floor_on_the_android_api_level.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# requires: elf gcc android-ndk +# 680 -- the engine states the target's platform floor as a fact in the +# platform's own words (`android.api-level`), and a dependency refuses a floor +# below what it needs through the ordinary `version-floor` requirement +# (#634, A9). The floor is not raised for the dependency: the application's +# floor decides which devices it installs on. +# +# Legs, on the `x86_64-linux-android` row: +# A. A root that states no level: refused, naming where 21 came from. +# B. A root stating 21: refused, naming `[target.] min_api_level`, +# and saying "this build targets". +# C. A root stating 24: builds. +# D. The host row states no such fact, so the requirement is silent there. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +TARGET=x86_64-linux-android +mkdir -p "$TMP/fw/src" "$TMP/app/src" +cat > "$TMP/fw/mcpp.toml" <<'TOML' +[package] +namespace = "demo" +name = "fw" +version = "0.1.0" +[targets.fw] +kind = "lib" +[[runtime.requirements]] +kind = "version-floor" +value = "android.api-level >= 23" +TOML +printf 'export module fw;\nexport int fw_anchor() { return 41; }\n' > "$TMP/fw/src/fw.cppm" +printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > "$TMP/app/src/main.cpp" +base='[package] +name = "app" +version = "0.1.0" +[dependencies] +demo.fw = { path = "../fw" } +' +cd "$TMP/app" + +# ── A ────────────────────────────────────────────────────────────────────── +printf '%s' "$base" > mcpp.toml +if "$MCPP" build --target "$TARGET" > a.log 2>&1; then fail "A: an unset level below the floor built" a.log; fi +grep -q "requires android.api-level >= 23, and this build targets" a.log || fail "A: refusal wording" a.log +grep -q "min_api_level is not set" a.log || fail "A: the refusal does not say where the level came from" a.log + +# ── B ────────────────────────────────────────────────────────────────────── +printf '%s[target.%s]\nmin_api_level = 21\n' "$base" "$TARGET" > mcpp.toml +if "$MCPP" build --target "$TARGET" > b.log 2>&1; then fail "B: level 21 built" b.log; fi +grep -q "requires android.api-level >= 23, and this build targets 21" b.log || fail "B: refusal wording" b.log +grep -qF "set by: [target.$TARGET] min_api_level" b.log || fail "B: the refusal does not name the key" b.log + +# ── C ────────────────────────────────────────────────────────────────────── +printf '%s[target.%s]\nmin_api_level = 24\n' "$base" "$TARGET" > mcpp.toml +"$MCPP" build --target "$TARGET" > c.log 2>&1 || fail "C: level 24 was refused" c.log + +# ── D ────────────────────────────────────────────────────────────────────── +printf '%s' "$base" > mcpp.toml +"$MCPP" build > d.log 2>&1 || fail "D: the host row was refused" d.log +if grep -q "android.api-level" d.log; then fail "D: the host row mentions the Android floor" d.log; fi + +echo "PASS: 680_a_dependency_floor_on_the_android_api_level" diff --git a/tests/e2e/681_index_repos_reach_an_existing_home.sh b/tests/e2e/681_index_repos_reach_an_existing_home.sh new file mode 100755 index 000000000..5c33eb0df --- /dev/null +++ b/tests/e2e/681_index_repos_reach_an_existing_home.sh @@ -0,0 +1,136 @@ +#!/usr/bin/env bash +# 681 -- `[index.repos.]` in config.toml reaches a registry that already +# exists (#634, C4). The table used to seed the registry's `.xlings.json` only +# when that file did not exist, so a table added to a home that had run once +# did nothing, without a word. xlings follows the file at its next index sync +# (measured separately: the index directory became the named checkout); this +# test holds the part mcpp owns, which is the file. +# +# Legs: +# A. A table added to an existing home is written into `index_repos`, with +# one line saying so, and the change is recorded. +# B. A second run changes nothing and prints nothing about the index. +# C. Removing the table restores the registry's previous state. +# D. An entry changed by someone else after mcpp wrote it is left alone when +# the table is removed. +# E. A config.toml without the `mcpplibs` table that mcpp seeds into it: the +# registry's `mcpplibs` entry is then mcpp's default rather than a table's, +# so a copy that differs from the default is neither rewritten nor +# reported as a table's change, and nothing is recorded for it. +# F. A `[index.repos.mcpplibs]` table added back replaces that entry, and +# removing the table restores the entry it replaced. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +export MCPP_HOME="$TMP/mcpp-home" +CFG="$MCPP_HOME/config.toml" +XJ="$MCPP_HOME/registry/.xlings.json" +REC="$MCPP_HOME/registry/.mcpp-index-overrides.json" +CHECKOUT="$TMP/xim-checkout" +mkdir -p "$CHECKOUT" +cd "$TMP" + +entry_url() { # entry_url -> url of that index_repos entry, or "absent" + python3 - "$XJ" "$1" <<'PY' +import json, sys +repos = json.load(open(sys.argv[1])).get("index_repos", []) +urls = [r.get("url") for r in repos if r.get("name") == sys.argv[2]] +print(urls[0] if urls else "absent") +PY +} + +same_json() { # same_json : equal as JSON values; mcpp writes its own formatting + python3 -c 'import json, sys; sys.exit(0 if json.load(open(sys.argv[1])) == json.load(open(sys.argv[2])) else 1)' "$1" "$2" +} + +"$MCPP" self env > /dev/null 2>&1 || true +[ -f "$XJ" ] || fail "the registry was not seeded" +[ "$(entry_url xim)" = "absent" ] || fail "a fresh home already names xim" "$XJ" + +# ── A ────────────────────────────────────────────────────────────────────── +printf '\n[index.repos.xim]\nurl = "%s"\n' "$CHECKOUT" >> "$CFG" +"$MCPP" self env > a.log 2>&1 || true +[ "$(entry_url xim)" = "$CHECKOUT" ] || fail "A: the table did not reach .xlings.json" "$XJ" a.log +grep -qF "xim -> $CHECKOUT ([index.repos.xim] in config.toml)" a.log || fail "A: no line names the change" a.log +[ -f "$REC" ] || fail "A: the change was not recorded" + +# ── B ────────────────────────────────────────────────────────────────────── +cp "$XJ" xj.before +"$MCPP" self env > b.log 2>&1 || true +cmp -s "$XJ" xj.before || fail "B: a second run changed .xlings.json" "$XJ" +if grep -q "xim -> " b.log; then fail "B: a second run printed the index line again" b.log; fi + +# ── C ────────────────────────────────────────────────────────────────────── +python3 - "$CFG" <<'PY' +import re, sys +p = sys.argv[1]; s = open(p).read() +open(p, "w").write(re.sub(r'\n\[index\.repos\.xim\]\nurl = "[^"]*"\n', '\n', s)) +PY +"$MCPP" self env > c.log 2>&1 || true +[ "$(entry_url xim)" = "absent" ] || fail "C: removing the table left the entry" "$XJ" c.log +grep -q "xim restored" c.log || fail "C: no line names the restoration" c.log +[ ! -f "$REC" ] || fail "C: the record outlived the table" "$REC" + +# ── D ────────────────────────────────────────────────────────────────────── +printf '\n[index.repos.xim]\nurl = "%s"\n' "$CHECKOUT" >> "$CFG" +"$MCPP" self env > /dev/null 2>&1 || true +python3 - "$XJ" <<'PY' +import json, sys +p = sys.argv[1]; d = json.load(open(p)) +for r in d["index_repos"]: + if r.get("name") == "xim": r["url"] = "/somewhere/else" +json.dump(d, open(p, "w"), indent=2) +PY +python3 - "$CFG" <<'PY' +import re, sys +p = sys.argv[1]; s = open(p).read() +open(p, "w").write(re.sub(r'\n\[index\.repos\.xim\]\nurl = "[^"]*"\n', '\n', s)) +PY +"$MCPP" self env > d.log 2>&1 || true +[ "$(entry_url xim)" = "/somewhere/else" ] || fail "D: an entry someone else changed was overwritten" "$XJ" + +# ── E ────────────────────────────────────────────────────────────────────── +# The seeded `[index.repos."mcpplibs"]` table leaves config.toml, and the +# registry's entry gains a key mcpp does not write. +grep -q '^\[index\.repos\."mcpplibs"\]' "$CFG" || fail "E: config.toml was not seeded with the mcpplibs table" "$CFG" +python3 - "$CFG" <<'PY' +import re, sys +p = sys.argv[1]; s = open(p).read() +open(p, "w").write(re.sub(r'\n\[index\.repos\."mcpplibs"\]\n(?:[^\[\n][^\n]*\n)*', '\n', s)) +PY +if grep -q 'index\.repos\."mcpplibs"' "$CFG"; then fail "E: the mcpplibs table is still in config.toml" "$CFG"; fi +python3 - "$XJ" <<'PY' +import json, sys +p = sys.argv[1]; d = json.load(open(p)) +for r in d["index_repos"]: + if r.get("name") == "mcpplibs": r["source"] = "git" +json.dump(d, open(p, "w"), indent=2) +PY +cp "$XJ" xj.e +rm -f "$REC" +"$MCPP" self env > e.log 2>&1 || true +cmp -s "$XJ" xj.e || fail "E: the default entry was rewritten without a table" "$XJ" e.log +if grep -q "mcpplibs -> " e.log; then fail "E: a table was named for the default entry" e.log; fi +[ ! -f "$REC" ] || fail "E: a change was recorded for the default entry" "$REC" + +# ── F ────────────────────────────────────────────────────────────────────── +MIRROR="$TMP/mcpplibs-checkout" +mkdir -p "$MIRROR" +printf '\n[index.repos.mcpplibs]\nurl = "%s"\n' "$MIRROR" >> "$CFG" +"$MCPP" self env > f1.log 2>&1 || true +[ "$(entry_url mcpplibs)" = "$MIRROR" ] || fail "F: the table did not replace the default entry" "$XJ" f1.log +grep -qF "mcpplibs -> $MIRROR ([index.repos.mcpplibs] in config.toml)" f1.log || fail "F: no line names the change" f1.log +python3 - "$CFG" <<'PY' +import re, sys +p = sys.argv[1]; s = open(p).read() +open(p, "w").write(re.sub(r'\n\[index\.repos\.mcpplibs\]\nurl = "[^"]*"\n', '\n', s)) +PY +"$MCPP" self env > f2.log 2>&1 || true +same_json "$XJ" xj.e || fail "F: removing the table did not restore the entry it replaced" "$XJ" f2.log +grep -q "mcpplibs restored" f2.log || fail "F: no line names the restoration" f2.log + +echo "OK" diff --git a/tests/e2e/682_why_deps_prints_the_resolved_graph.sh b/tests/e2e/682_why_deps_prints_the_resolved_graph.sh new file mode 100755 index 000000000..0ab895a51 --- /dev/null +++ b/tests/e2e/682_why_deps_prints_the_resolved_graph.sh @@ -0,0 +1,49 @@ +#!/usr/bin/env bash +# 682 -- `resolution.json` records the resolved dependency graph and +# `mcpp why deps` prints it (#634, X). Before, `why deps` printed only the +# lines of `mcpp.lock`, which does not record `path` dependencies, so a project +# of path dependencies showed "no mcpp.lock" after a successful build. +# +# Legs: +# A. `why deps` lists the path dependency with the key and table that +# requested it and its link form. +# B. The record keeps every field it had: `schema_version` 2, `toolchain`, +# `runtime`, and adds `graph` with the root first. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p "$TMP/fw/src" "$TMP/app/src" +printf '[package]\nnamespace = "demo"\nname = "fw"\nversion = "0.1.0"\n[targets.fw]\nkind = "lib"\n' > "$TMP/fw/mcpp.toml" +printf 'export module fw;\nexport int fw_anchor() { return 41; }\n' > "$TMP/fw/src/fw.cppm" +printf '[package]\nname = "app"\nversion = "0.1.0"\n[dependencies]\ndemo.fw = { path = "../fw" }\n' > "$TMP/app/mcpp.toml" +printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > "$TMP/app/src/main.cpp" +cd "$TMP/app" + +"$MCPP" build > build.log 2>&1 || fail "build failed" build.log + +# ── A ────────────────────────────────────────────────────────────────────── +"$MCPP" why deps > why.log 2>&1 || fail "A: why deps failed" why.log +grep -q "^dependency graph:" why.log || fail "A: no graph section" why.log +grep -q "demo.fw@0.1.0" why.log || fail "A: the path dependency is not listed" why.log +grep -qF "requested by mcpplibs.app@0.1.0 as 'demo.fw' in [dependencies]" why.log \ + || fail "A: the request is not listed" why.log +grep -q "linked static (default)" why.log || fail "A: the link form is not listed" why.log + +# ── B ────────────────────────────────────────────────────────────────────── +json=$(find target -name resolution.json | head -1) +python3 - "$json" <<'PY' || fail "B: record shape" "$json" +import json, sys +doc = json.load(open(sys.argv[1])) +assert doc["schema_version"] == 2, doc["schema_version"] +assert "toolchain" in doc and "runtime" in doc, sorted(doc) +pkgs = doc["graph"]["packages"] +assert pkgs[0]["root"] is True and pkgs[0]["requested_by"] == [], pkgs[0] +fw = [p for p in pkgs if p["package"]["canonical"] == "demo.fw@0.1.0"] +assert len(fw) == 1 and fw[0]["package"]["source"].startswith("path+"), fw +PY + +echo "OK" diff --git a/tests/e2e/683_a_build_program_reads_the_payload_pkg_config_view.sh b/tests/e2e/683_a_build_program_reads_the_payload_pkg_config_view.sh new file mode 100755 index 000000000..90df6055f --- /dev/null +++ b/tests/e2e/683_a_build_program_reads_the_payload_pkg_config_view.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash +# 683 -- `mcpp::pkg_config_libdir()` names the pkg-config directories of the +# registry SubOS, the view payload recipes declare their `.pc` files into +# (#634, A7). It is an accessor and not an environment default, so the host's +# pkg-config database is untouched for a package that means it. +# +# Legs: +# A. The value is the registry SubOS's `usr/lib/pkgconfig` and +# `usr/share/pkgconfig`, in that order. +# B. The build program's environment does not carry PKG_CONFIG_LIBDIR. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +printf '[package]\nname = "app"\nversion = "0.1.0"\n' > mcpp.toml +printf 'int main() { return 0; }\n' > src/main.cpp +cat > build.mcpp <<'CPP' +import std; +import mcpp; +int main() { + std::string line = std::string("libdir=") + mcpp::pkg_config_libdir(); + const char* inherited = std::getenv("PKG_CONFIG_LIBDIR"); + line += std::string(" inherited=") + (inherited ? inherited : "(unset)"); + mcpp::warning(line.c_str()); + return 0; +} +CPP +env -u PKG_CONFIG_LIBDIR "$MCPP" build > build.log 2>&1 || fail "build failed" build.log + +registry=$("$MCPP" self env 2>/dev/null | sed -n 's/^xlings home *= *//p' | head -1) +[ -n "$registry" ] || fail "cannot read the registry from mcpp self env" +# `self env` prints the native spelling (`C:\Users\...` on Windows); the +# accessor joins generic, `/`-separated paths with the platform's list +# separator, which the `?` below matches. +registry=${registry//\\//} +line=$(grep -o 'libdir=.*' build.log | head -1) +case "$line" in + "libdir=$registry/subos/default/usr/lib/pkgconfig"?"$registry/subos/default/usr/share/pkgconfig inherited=(unset)") ;; + *) fail "A/B: the build program read '$line'" build.log ;; +esac + +echo "OK" diff --git a/tests/e2e/684_a_runner_supplied_twice_is_refused.sh b/tests/e2e/684_a_runner_supplied_twice_is_refused.sh new file mode 100755 index 000000000..2bd651ab6 --- /dev/null +++ b/tests/e2e/684_a_runner_supplied_twice_is_refused.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 684 -- a runner that a dependency's build program supplies and the project's +# own build program also emits is refused naming both, and the manifest is the +# way to choose (#634, triage record §9 item 8). +# +# The tokens a build program emits are appended to the runner's argv, in +# emission order, which is the directive contract. Two dependencies that supply +# one name were already refused. A dependency and the root were not: measured +# on 2026.9.14.1, the two argvs became one, `run-A.sh run-B.sh `, and +# `mcpp run --list-runners` showed only the first. +# +# Criteria: +# 1. the build is refused, naming the dependency and the runner name; +# 2. with the name declared in `[target..runners]`, the build succeeds +# and `mcpp run --runner app` uses the manifest's runner; +# 3. negative direction: a runner only the dependency supplies is not refused. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +for r in A B C; do + printf '#!/bin/sh\nprintf "RUNNER-%s\\n"\nexec "$@"\n' "$r" > "$TMP/run-$r.sh" + chmod +x "$TMP/run-$r.sh" +done + +mkdir -p dep/src app/src +cat > dep/mcpp.toml <<'TOML' +[package] +namespace = "demo" +name = "dep" +version = "0.1.0" +[targets.dep] +kind = "lib" +TOML +printf 'export module dep;\nexport int dep_anchor() { return 0; }\n' > dep/src/dep.cppm +cat > dep/build.mcpp < app/mcpp.toml <<'TOML' +[package] +name = "app" +version = "0.1.0" +[dependencies] +demo.dep = { path = "../dep" } +TOML +printf 'import dep;\nint main() { return dep_anchor(); }\n' > app/src/main.cpp +cd app + +# ── 3 (the negative direction first): only the dependency supplies it ──── +"$MCPP" build > dep-only.log 2>&1 || fail "3: a runner only the dependency supplies was refused" dep-only.log + +# ── 1 ───────────────────────────────────────────────────────────────────── +cat > build.mcpp < both.log 2>&1; then fail "1: a runner supplied twice was accepted" both.log; fi +grep -q "the dependency 'dep' and this project's build program both supply a runner named 'app'" both.log \ + || fail "1: the refusal does not name the dependency and the runner" both.log + +# ── 2 ───────────────────────────────────────────────────────────────────── +# The host row's canonical name, which is what a native build resolves to. +arch=$(uname -m) +case "$arch" in arm64) arch=aarch64 ;; esac +case "$(uname -s)" in + Linux) row="$arch-linux-gnu" ;; + Darwin) row="$arch-macos" ;; + *) echo "SKIP: no canonical host row for $(uname -s)"; exit 0 ;; +esac +printf '\n[target.%s.runners]\napp = ["%s/run-C.sh"]\n' "$row" "$TMP" >> mcpp.toml +"$MCPP" build > manifest.log 2>&1 || fail "2: the manifest's runner did not settle it" manifest.log +"$MCPP" run --runner app > run.log 2>&1 || true +grep -q "RUNNER-C" run.log || fail "2: the manifest's runner was not used" run.log + +echo "PASS: 684_a_runner_supplied_twice_is_refused" diff --git a/tests/e2e/685_a_nested_test_loads_a_graph_shared_library.sh b/tests/e2e/685_a_nested_test_loads_a_graph_shared_library.sh new file mode 100755 index 000000000..7ea0ed4bf --- /dev/null +++ b/tests/e2e/685_a_nested_test_loads_a_graph_shared_library.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# requires: elf +# 685 -- a test program built from a subdirectory of `tests/` loads a shared +# library the dependency graph built. +# +# A consumer of a graph-built shared library searches its own directory +# (`$ORIGIN`), which holds the library for a program in `bin/`. A test from +# `tests/sub/` is linked to `bin/sub/`, and on 2026.9.14.1 the post-link +# closure check refused it ("libfw.so not found on the search path this +# artifact will actually use"), which failed the whole `mcpp test`. A consumer +# in another directory now also searches the relative path to the library's. +# +# Criteria: +# 1. `mcpp test` builds and passes a nested and a top-level test that both +# call into the shared library; +# 2. the nested test's search path carries `$ORIGIN/..`; +# 3. negative direction: the program in `bin/` carries no relative entry, +# so a consumer beside the library keeps its link line. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p fw/src app/src app/tests/sub +cat > fw/mcpp.toml <<'TOML' +[package] +namespace = "demo" +name = "fw" +version = "0.1.0" +[targets.fw] +kind = "shared" +TOML +printf 'export module fw;\nexport int fw_anchor() { return 41; }\n' > fw/src/fw.cppm +cat > app/mcpp.toml <<'TOML' +[package] +name = "app" +version = "0.1.0" +[dependencies] +demo.fw = { path = "../fw" } +TOML +printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > app/src/main.cpp +cp app/src/main.cpp app/tests/sub/deep.cpp +cp app/src/main.cpp app/tests/top.cpp +cd app + +# ── 1 ───────────────────────────────────────────────────────────────────── +"$MCPP" test > test.log 2>&1 || fail "1: mcpp test failed" test.log +grep -q "sub/deep ... ok" test.log || fail "1: the nested test did not pass" test.log +grep -q "top ... ok" test.log || fail "1: the top-level test did not pass" test.log + +# ── 2 ───────────────────────────────────────────────────────────────────── +deep=$(find target -type f -name deep -path '*/bin/sub/*' | head -1) +[ -n "$deep" ] || fail "2: no bin/sub/deep was built" test.log +readelf -d "$deep" | grep -E 'R(UN)?PATH' | grep -qF '$ORIGIN/..' \ + || fail "2: the nested test does not search \$ORIGIN/.." test.log + +# ── 3 ───────────────────────────────────────────────────────────────────── +"$MCPP" build > build.log 2>&1 || fail "3: mcpp build failed" build.log +prog=$(find target -type f -name app -path '*/bin/*' | head -1) +[ -n "$prog" ] || fail "3: no bin/app was built" build.log +if readelf -d "$prog" | grep -E 'R(UN)?PATH' | grep -qF '$ORIGIN/'; then + fail "3: the program beside the library carries a relative search entry" +fi + +echo "PASS: 685_a_nested_test_loads_a_graph_shared_library" diff --git a/tests/unit/test_build_flags.cpp b/tests/unit/test_build_flags.cpp index d67773dae..b9d1f6448 100644 --- a/tests/unit/test_build_flags.cpp +++ b/tests/unit/test_build_flags.cpp @@ -146,3 +146,22 @@ TEST(BuildFlags, NormalizeIncludeFlagsLeavesAbsolutePathsAlone) { } } // namespace + +// #634, item 11 of the triage record: a search-path entry that begins with a +// token the loader expands is relative to a loaded object, not to the package +// that wrote it. Both ldflag normalisers ask this one predicate. +TEST(BuildFlagsLoaderTokens, EveryLoaderTokenIsLeftAsWritten) { + for (std::string_view entry : {"$ORIGIN", "${ORIGIN}", "$ORIGIN/../lib", "$LIB", + "@executable_path", "@executable_path/../Frameworks", + "@loader_path", "@loader_path/", "@rpath", + "@rpath/sub"}) { + EXPECT_TRUE(mcpp::build::is_loader_relative_search_path(entry)) << entry; + } +} + +TEST(BuildFlagsLoaderTokens, OrdinaryPathsAreNotTokens) { + for (std::string_view entry : {"", "lib", "./lib", "/opt/lib", "rpath", + "@executable_pathology", "@rpathx", "x/@rpath"}) { + EXPECT_FALSE(mcpp::build::is_loader_relative_search_path(entry)) << entry; + } +} diff --git a/tests/unit/test_distribution.cpp b/tests/unit/test_distribution.cpp index a2b2f44d9..c4210eb04 100644 --- a/tests/unit/test_distribution.cpp +++ b/tests/unit/test_distribution.cpp @@ -747,6 +747,36 @@ TEST(Distribution, ASharedLibraryStillHidesTheArchivesWithoutASecondRuntime) { " /tc/libunwind.a -Wl,--exclude-libs,libunwind.a"); } +// THE NAMES `--exclude-libs` MATCHES ARE THE ARCHIVES THE LINKER OPENS +// (#634 A6). The Android NDK's per-API `libc++.a` is a linker script, +// `INPUT(-lc++_static -lc++abi)`: its members come from `libc++_static.a`, and +// hiding `libc++.a` alone left 161 dynamic symbols in a self-contained shared +// library where naming `libc++_static.a` left 4 (measured). The paths on the +// line are unchanged; only the hidden names grow, and only when given. +TEST(Distribution, ALinkerScriptsArchivesAreTheNamesHidden) { + dist::MechanismInput in; + in.format = dist::Format::Elf; + in.stdlibId = "libc++"; + in.role = dist::Role::SharedLibrary; + in.requested = dist::Contract::SelfContained; + in.libcxxArchive = "/ndk/sysroot/usr/lib/x86_64-linux-android/24/libc++.a"; + in.libcxxAbiArchive = "/ndk/sysroot/usr/lib/x86_64-linux-android/libc++abi.a"; + in.libunwindArchive = "/ndk/lib/clang/21/lib/linux/x86_64/libunwind.a"; + in.libcxxLinkedArchiveNames = {"libc++.a", "libc++abi.a", "libc++_static.a"}; + auto m = dist::resolve(in); + EXPECT_EQ(m.unitFlags, + " -nostdlib++ /ndk/sysroot/usr/lib/x86_64-linux-android/24/libc++.a" + " /ndk/sysroot/usr/lib/x86_64-linux-android/libc++abi.a" + " -Wl,--exclude-libs,libc++.a -Wl,--exclude-libs,libc++abi.a" + " -Wl,--exclude-libs,libc++_static.a" + " /ndk/lib/clang/21/lib/linux/x86_64/libunwind.a" + " -Wl,--exclude-libs,libunwind.a"); + + // An executable still carries no guard, whatever the names are. + in.role = dist::Role::Test; + EXPECT_EQ(dist::resolve(in).unitFlags.find("exclude-libs"), std::string::npos); +} + // ─── Format::Wasm, the member the module predicted and deferred ──────────── // // `format_for`'s own comment said a wasm triple "falls out of every branch" diff --git a/tests/unit/test_pack_closure.cpp b/tests/unit/test_pack_closure.cpp new file mode 100644 index 000000000..f00ab170e --- /dev/null +++ b/tests/unit/test_pack_closure.cpp @@ -0,0 +1,416 @@ +#include + +import std; +import mcpp.pack; + +// #634 A3: the closure `mcpp pack` reads from the files for the PE, Android and +// Mach-O rows. The rules are asserted here, over hand-built objects in a +// temporary tree, because two of the three rows cannot be produced on the +// machine that runs most of the suite: a Mach-O needs a macOS toolchain and a +// device-loadable Android tree needs an emulator to prove anything end to end. +// What the rule decides -- member, platform or unresolved, and where a member +// is staged -- is a pure function of the names, the rpaths and which files +// exist, and that is what these tests hold fixed. +// +// The fixtures follow `test_pack_binfmt.cpp`'s: written as plainly as possible, +// no spans and no lambdas that mutate captured state. + +namespace { + +namespace fs = std::filesystem; + +void put(std::string& b, std::size_t at, std::uint64_t v, std::size_t width) { + if (b.size() < at + width) b.resize(at + width, '\0'); + for (std::size_t i = 0; i < width; ++i) + b[at + i] = static_cast((v >> (8 * i)) & 0xFF); +} + +struct Tree { + fs::path root; + Tree() { + root = fs::temp_directory_path() + / std::format("mcpp_pack_closure_{}", std::random_device{}()); + fs::create_directories(root); + } + ~Tree() { + std::error_code ec; + fs::remove_all(root, ec); + } + Tree(const Tree&) = delete; + Tree& operator=(const Tree&) = delete; + + fs::path write(const fs::path& rel, const std::string& bytes) const { + const auto p = root / rel; + fs::create_directories(p.parent_path()); + std::ofstream os(p, std::ios::binary); + os.write(bytes.data(), static_cast(bytes.size())); + return p; + } +}; + +// A minimal ELF64 shared object whose PT_DYNAMIC carries `needed` as DT_NEEDED +// entries; the same layout `test_pack_binfmt.cpp` documents. +std::string elf_needing(const std::vector& needed) { + std::string b(64, '\0'); + b[0] = 0x7f; b[1] = 'E'; b[2] = 'L'; b[3] = 'F'; + b[4] = 2; b[5] = 1; b[6] = 1; + put(b, 16, 3, 2); + put(b, 18, 0x3E, 2); + put(b, 20, 1, 4); + put(b, 32, 64, 8); + put(b, 52, 64, 2); + put(b, 54, 56, 2); + put(b, 56, 2, 2); + + const std::size_t dynAt = 64 + 2 * 56; + const std::size_t dynSize = (needed.size() + 2) * 16; + const std::size_t strAt = dynAt + dynSize; + std::string strtab(1, '\0'); + std::vector offsets; + for (std::size_t k = 0; k < needed.size(); ++k) { + offsets.push_back(strtab.size()); + strtab += needed[k]; + strtab.push_back('\0'); + } + const std::size_t total = strAt + strtab.size(); + + put(b, 64 + 0, 1, 4); + put(b, 64 + 4, 5, 4); + put(b, 64 + 8, 0, 8); + put(b, 64 + 16, 0, 8); + put(b, 64 + 32, total, 8); + put(b, 64 + 40, total, 8); + put(b, 120 + 0, 2, 4); + put(b, 120 + 8, dynAt, 8); + put(b, 120 + 16, dynAt, 8); + put(b, 120 + 32, dynSize, 8); + + std::size_t at = dynAt; + for (std::size_t k = 0; k < offsets.size(); ++k) { + put(b, at, 1, 8); + put(b, at + 8, offsets[k], 8); + at += 16; + } + put(b, at, 5, 8); + put(b, at + 8, strAt, 8); + at += 16; + put(b, at, 0, 8); + put(b, at + 8, 0, 8); + + b.resize(total, '\0'); + for (std::size_t i = 0; i < strtab.size(); ++i) b[strAt + i] = strtab[i]; + return b; +} + +// A minimal PE32+ image importing `imports` through its import directory. +std::string pe_importing(const std::vector& imports) { + const std::size_t kNt = 0x40; + const std::size_t kOptSize = 0xF0; + const std::size_t kSecAt = kNt + 24 + kOptSize; + const std::size_t kRawAt = 0x400; + const std::uint32_t kSecVa = 0x1000; + const std::size_t dirsAt = kNt + 24 + 112; + + std::string b(kRawAt, '\0'); + b[0] = 'M'; b[1] = 'Z'; + put(b, 0x3C, kNt, 4); + b[kNt] = 'P'; b[kNt + 1] = 'E'; + put(b, kNt + 4, 0x8664, 2); + put(b, kNt + 6, 1, 2); + put(b, kNt + 20, kOptSize, 2); + put(b, kNt + 24, 0x20b, 2); + put(b, dirsAt - 4, 16, 4); + const std::string secName = ".rdata"; + for (std::size_t i = 0; i < secName.size(); ++i) b[kSecAt + i] = secName[i]; + put(b, kSecAt + 8, 0x1000, 4); + put(b, kSecAt + 12, kSecVa, 4); + put(b, kSecAt + 16, 0x1000, 4); + put(b, kSecAt + 20, kRawAt, 4); + + // `std::size_t` elements, not `std::uint32_t`: on macos-15 (clang 22, + // libc++ as a module) the first `push_back` into a `std::vector` + // here faulted in `memmove` at address 0 (lldb, run 34819771218), while + // `elf_needing`'s `std::vector` runs. The values fit either. + std::vector nameRvas; + std::size_t cursor = kRawAt; + for (std::size_t k = 0; k < imports.size(); ++k) { + nameRvas.push_back(kSecVa + (cursor - kRawAt)); + for (std::size_t i = 0; i < imports[k].size(); ++i) { + put(b, cursor, static_cast(imports[k][i]), 1); + ++cursor; + } + put(b, cursor, 0, 1); + ++cursor; + } + cursor = (cursor + 15) & ~static_cast(15); + const std::size_t importAt = cursor; + for (std::size_t k = 0; k < nameRvas.size(); ++k) { + put(b, cursor + 0, 0x9000, 4); + put(b, cursor + 12, nameRvas[k], 4); + put(b, cursor + 16, 0x9100, 4); + cursor += 20; + } + for (std::size_t i = 0; i < 20; ++i) { put(b, cursor, 0, 1); ++cursor; } + b.resize(std::max(b.size(), kRawAt + 0x1000), '\0'); + if (!imports.empty()) { + put(b, dirsAt + 1 * 8, static_cast(kSecVa + (importAt - kRawAt)), 4); + put(b, dirsAt + 1 * 8 + 4, 20 * (imports.size() + 1), 4); + } + return b; +} + +// A thin little-endian 64-bit arm64 Mach-O naming `dylibs` with LC_LOAD_DYLIB +// and carrying `rpaths` as LC_RPATH, in that order. +std::string macho_naming(const std::vector& dylibs, + const std::vector& rpaths) { + std::string b(32, '\0'); + b[0] = '\xcf'; b[1] = '\xfa'; b[2] = '\xed'; b[3] = '\xfe'; + put(b, 4, 0x0100000c, 4); + put(b, 12, 2, 4); + put(b, 16, dylibs.size() + rpaths.size(), 4); + std::size_t cursor = 32; + for (std::size_t k = 0; k < dylibs.size() + rpaths.size(); ++k) { + const bool isDylib = k < dylibs.size(); + const std::string& str = isDylib ? dylibs[k] : rpaths[k - dylibs.size()]; + const std::size_t structSize = isDylib ? 24 : 12; + const std::size_t cmdsize = (structSize + str.size() + 1 + 3) & ~std::size_t(3); + b.resize(cursor + cmdsize, '\0'); + put(b, cursor + 0, isDylib ? 0x0000000c : 0x8000001c, 4); + put(b, cursor + 4, cmdsize, 4); + put(b, cursor + 8, structSize, 4); + for (std::size_t i = 0; i < str.size(); ++i) b[cursor + structSize + i] = str[i]; + cursor += cmdsize; + } + put(b, 20, cursor - 32, 4); + return b; +} + +std::vector member_dests(const mcpp::pack::ClosureRead& r) { + std::vector out; + for (auto const& m : r.members) out.push_back(m.dest.generic_string()); + return out; +} + +std::vector unresolved_names(const mcpp::pack::ClosureRead& r) { + std::vector out; + for (auto const& u : r.unresolved) out.push_back(u.name); + return out; +} + +using Names = std::vector; + +} // namespace + +// ── Android ────────────────────────────────────────────────────────────── + +TEST(PackClosureAndroid, TheStubDirectoryIsTheDevicesAndEveryOtherNameTravels) { + Tree t; + t.write("platform/libc.so", "stub"); + t.write("platform/libm.so", "stub"); + auto app = t.write("build/libapp.so", + elf_needing({"libfw.so", "libc++_shared.so", "libc.so"})); + t.write("build/libfw.so", elf_needing({"libc++_shared.so", "libm.so"})); + t.write("ndk/libc++_shared.so", elf_needing({"libc.so"})); + + mcpp::pack::ClosureReadInput in; + in.object = app; + in.rule = mcpp::pack::ClosureRule::Android; + // The driver's search list holds the stub directory too; the platform + // check runs first, so a stub is never staged. + in.searchDirs = {t.root / "build", t.root / "platform", t.root / "ndk"}; + in.platformDirs = {t.root / "platform"}; + auto r = mcpp::pack::read_closure(in); + + EXPECT_EQ(member_dests(r), (Names{"libc++_shared.so", "libfw.so"})); + EXPECT_EQ(r.platform, (Names{"libc.so", "libm.so"})); + EXPECT_TRUE(r.unresolved.empty()); + ASSERT_EQ(r.members.size(), 2u); + EXPECT_EQ(r.members[0].source, t.root / "ndk" / "libc++_shared.so"); +} + +TEST(PackClosureAndroid, ANameFoundNowhereLeavesTheClosureIncomplete) { + Tree t; + t.write("platform/libc.so", "stub"); + auto app = t.write("build/libapp.so", elf_needing({"libgone.so", "libc.so"})); + + mcpp::pack::ClosureReadInput in; + in.object = app; + in.rule = mcpp::pack::ClosureRule::Android; + in.searchDirs = {t.root / "build"}; + in.platformDirs = {t.root / "platform"}; + auto r = mcpp::pack::read_closure(in); + + EXPECT_TRUE(r.members.empty()); + EXPECT_EQ(unresolved_names(r), (Names{"libgone.so"})); + ASSERT_EQ(r.unresolved.size(), 1u); + EXPECT_NE(r.unresolved[0].why.find((t.root / "build").string()), std::string::npos) + << "the reason names where it looked: " << r.unresolved[0].why; +} + +// ── PE, unchanged by the generalisation ────────────────────────────────── + +TEST(PackClosurePe, SystemNamesAndNamesFoundNowhereAreTheTargets) { + Tree t; + auto exe = t.write("bin/app.exe", + pe_importing({"KERNEL32.dll", "foo.dll", "missing.dll"})); + t.write("deps/foo.dll", pe_importing({"bar.dll"})); + t.write("deps/bar.dll", pe_importing({})); + + mcpp::pack::ClosureReadInput in; + in.object = exe; + in.rule = mcpp::pack::ClosureRule::Pe; + in.searchDirs = {t.root / "bin", t.root / "deps"}; + auto r = mcpp::pack::read_closure(in); + + EXPECT_EQ(member_dests(r), (Names{"bar.dll", "foo.dll"})); + EXPECT_EQ(r.platform, (Names{"KERNEL32.dll", "missing.dll"})); + EXPECT_TRUE(r.unresolved.empty()) << "PE's rule resolves every name"; +} + +TEST(PackClosurePe, ForceBundleReachesTheSystemList) { + Tree t; + auto exe = t.write("bin/app.exe", pe_importing({"dbghelp.dll"})); + t.write("deps/dbghelp.dll", pe_importing({})); + + mcpp::pack::ClosureReadInput in; + in.object = exe; + in.rule = mcpp::pack::ClosureRule::Pe; + in.searchDirs = {t.root / "deps"}; + auto without = mcpp::pack::read_closure(in); + EXPECT_EQ(without.platform, (Names{"dbghelp.dll"})); + + in.forceBundle = {"dbghelp.dll"}; + auto with = mcpp::pack::read_closure(in); + EXPECT_EQ(member_dests(with), (Names{"dbghelp.dll"})); + EXPECT_TRUE(with.platform.empty()); +} + +// ── Mach-O ─────────────────────────────────────────────────────────────── + +TEST(PackClosureMachO, AnRpathDylibBesideTheProgramIsAMemberTransitively) { + Tree t; + auto app = t.write("bin/app", + macho_naming({"@rpath/libfw.dylib", "/usr/lib/libSystem.B.dylib"}, + {"@loader_path"})); + // libfw carries no rpath of its own: it inherits the program's. + t.write("bin/libfw.dylib", + macho_naming({"@rpath/libdep.dylib", "/usr/lib/libc++.1.dylib"}, {})); + t.write("bin/libdep.dylib", macho_naming({"/usr/lib/libSystem.B.dylib"}, {})); + + mcpp::pack::ClosureReadInput in; + in.object = app; + in.rule = mcpp::pack::ClosureRule::MachO; + in.arch = "aarch64"; + auto r = mcpp::pack::read_closure(in); + + EXPECT_EQ(member_dests(r), (Names{"libdep.dylib", "libfw.dylib"})); + EXPECT_EQ(r.platform, (Names{"/usr/lib/libSystem.B.dylib", "/usr/lib/libc++.1.dylib"})); + EXPECT_TRUE(r.unresolved.empty()); +} + +TEST(PackClosureMachO, ATrailingSlashAndExecutablePathQualifyAsTheProgramsDirectory) { + Tree t; + auto app = t.write("bin/app", + macho_naming({"@rpath/liba.dylib", "@executable_path/libb.dylib"}, + {"@executable_path/"})); + t.write("bin/liba.dylib", macho_naming({}, {})); + t.write("bin/libb.dylib", macho_naming({}, {})); + + mcpp::pack::ClosureReadInput in; + in.object = app; + in.rule = mcpp::pack::ClosureRule::MachO; + auto r = mcpp::pack::read_closure(in); + EXPECT_EQ(member_dests(r), (Names{"liba.dylib", "libb.dylib"})); + EXPECT_TRUE(r.unresolved.empty()); +} + +TEST(PackClosureMachO, AnAbsoluteInstallNameOutsideTheOsRootsIsUnresolved) { + Tree t; + // The file exists on this machine; the loader on another one reads that + // path, not the tree, so a copy would not be what loads. An install name + // is a POSIX path: on a Windows host the temporary directory is not one, + // so the name is spelled as a macOS path there, and the rule is the same + // whether or not the file exists. + auto elsewhere = t.write("opt/libq.dylib", macho_naming({}, {})); + const std::string installName = elsewhere.string().starts_with('/') + ? elsewhere.string() : std::string("/opt/elsewhere/libq.dylib"); + auto app = t.write("bin/app", macho_naming({installName}, {"@loader_path"})); + + mcpp::pack::ClosureReadInput in; + in.object = app; + in.rule = mcpp::pack::ClosureRule::MachO; + auto r = mcpp::pack::read_closure(in); + EXPECT_TRUE(r.members.empty()); + EXPECT_EQ(unresolved_names(r), (Names{installName})); + ASSERT_EQ(r.unresolved.size(), 1u); + EXPECT_NE(r.unresolved[0].why.find("absolute install name"), std::string::npos) + << r.unresolved[0].why; +} + +TEST(PackClosureMachO, AnRpathOnlyAnAbsoluteEntryReachesIsUnresolved) { + Tree t; + t.write("store/libfw.dylib", macho_naming({}, {})); + auto app = t.write("bin/app", + macho_naming({"@rpath/libfw.dylib"}, {(t.root / "store").string()})); + + mcpp::pack::ClosureReadInput in; + in.object = app; + in.rule = mcpp::pack::ClosureRule::MachO; + auto r = mcpp::pack::read_closure(in); + EXPECT_TRUE(r.members.empty()); + EXPECT_EQ(unresolved_names(r), (Names{"@rpath/libfw.dylib"})); + ASSERT_EQ(r.unresolved.size(), 1u); + EXPECT_NE(r.unresolved[0].why.find("@loader_path"), std::string::npos) + << r.unresolved[0].why; + + // The negative direction: the same dylib staged through a qualifying + // rpath is a member, sourced from wherever the loader finds it here. + auto app2 = t.write("bin2/app", + macho_naming({"@rpath/libfw.dylib"}, + {(t.root / "store").string(), "@loader_path"})); + in.object = app2; + auto r2 = mcpp::pack::read_closure(in); + EXPECT_EQ(member_dests(r2), (Names{"libfw.dylib"})); + ASSERT_EQ(r2.members.size(), 1u); + EXPECT_EQ(r2.members[0].source, t.root / "store" / "libfw.dylib"); +} + +TEST(PackClosureMachO, ALoaderPathSubdirectoryIsStagedAtThatSubdirectory) { + Tree t; + auto app = t.write("bin/app", macho_naming({"@loader_path/plugins/libp.dylib"}, {})); + t.write("bin/plugins/libp.dylib", macho_naming({"@loader_path/libq.dylib"}, {})); + t.write("bin/plugins/libq.dylib", macho_naming({}, {})); + + mcpp::pack::ClosureReadInput in; + in.object = app; + in.rule = mcpp::pack::ClosureRule::MachO; + auto r = mcpp::pack::read_closure(in); + EXPECT_EQ(member_dests(r), (Names{"plugins/libp.dylib", "plugins/libq.dylib"})); + EXPECT_TRUE(r.unresolved.empty()); +} + +TEST(PackClosureMachO, ANameOutsideTheProgramsDirectoryIsUnresolved) { + Tree t; + t.write("Frameworks/libz.dylib", macho_naming({}, {})); + auto app = t.write("bin/app", + macho_naming({"@executable_path/../Frameworks/libz.dylib"}, {})); + + mcpp::pack::ClosureReadInput in; + in.object = app; + in.rule = mcpp::pack::ClosureRule::MachO; + auto r = mcpp::pack::read_closure(in); + EXPECT_TRUE(r.members.empty()); + EXPECT_EQ(unresolved_names(r), (Names{"@executable_path/../Frameworks/libz.dylib"})); +} + +TEST(PackClosureMachO, AMissingDylibIsUnresolvedAndNamed) { + Tree t; + auto app = t.write("bin/app", macho_naming({"@rpath/libgone.dylib"}, {"@loader_path"})); + + mcpp::pack::ClosureReadInput in; + in.object = app; + in.rule = mcpp::pack::ClosureRule::MachO; + auto r = mcpp::pack::read_closure(in); + EXPECT_TRUE(r.members.empty()); + EXPECT_EQ(unresolved_names(r), (Names{"@rpath/libgone.dylib"})); +} diff --git a/tests/unit/test_pack_stage_tree.cpp b/tests/unit/test_pack_stage_tree.cpp index 90c0e428f..f750d6c58 100644 --- a/tests/unit/test_pack_stage_tree.cpp +++ b/tests/unit/test_pack_stage_tree.cpp @@ -186,6 +186,55 @@ TEST(PackStageTree, WalkedAndNotWalkedProduceDifferentManifests) { EXPECT_NE(walked, notWalked); } +// ── #634 A3: the `needs` lines ────────────────────────────────────────── + +TEST(PackStageTree, NeedsLinesFollowTheHeaderSortedAndOnceEach) { + Tmp t; + auto stage = t.path / "app"; + write_file(stage / "lib" / "libfw.so", "fw"); + using K = mcpp::pack::ClosureNeed::Kind; + mcpp::pack::ClosureStatus closure; + // Out of order, and one platform name twice -- a several-ABI tree reads + // the same one once per leg. + closure.needs = { + {"libfw.so", K::Staged, "lib/libfw.so"}, + {"libc.so", K::Platform, {}}, + {"libc.so", K::Platform, {}}, + }; + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage, closure)); + auto text = read_file(mcpp::pack::stage_manifest_path(stage)); + EXPECT_EQ(text, + "closure = walked\n" + "needs\tlibc.so\tplatform\n" + "needs\tlibfw.so\tlib/libfw.so\n" + "2 lib/libfw.so\n"); +} + +TEST(PackStageTree, AnIncompleteClosureListsItsUnresolvedNamesAfterTheReason) { + Tmp t; + auto stage = t.path / "app"; + write_file(stage / "bin" / "app", "x"); + using K = mcpp::pack::ClosureNeed::Kind; + mcpp::pack::ClosureStatus closure; + closure.walked = false; + closure.reason = "the dependency closure of 'app' is incomplete"; + closure.needs = {{"@rpath/libgone.dylib", K::Unresolved, {}}}; + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage, closure)); + auto text = read_file(mcpp::pack::stage_manifest_path(stage)); + EXPECT_EQ(text, + "closure = not-walked\n" + "reason = the dependency closure of 'app' is incomplete\n" + "needs\t@rpath/libgone.dylib\tunresolved\n" + "1 bin/app\n"); +} + +TEST(PackStageTree, ANameWithASpaceStaysOneField) { + using K = mcpp::pack::ClosureNeed::Kind; + EXPECT_EQ(mcpp::pack::render_closure_need( + {"@rpath/My Kit.framework/My Kit", K::Staged, "bin/My Kit"}), + "needs\t@rpath/My Kit.framework/My Kit\tbin/My Kit"); +} + TEST(PackStageTree, AMissingTreeIsRefusedRatherThanDescribedAsEmpty) { Tmp t; // An empty manifest for a directory that does not exist would say "nothing diff --git a/tests/unit/test_target_scalar_keys.cpp b/tests/unit/test_target_scalar_keys.cpp index 519f7f01c..3f97f35f5 100644 --- a/tests/unit/test_target_scalar_keys.cpp +++ b/tests/unit/test_target_scalar_keys.cpp @@ -165,6 +165,73 @@ TEST(TargetScalarKeys, EveryParsedNonTableKeyIsKnownToTheSweep) { << "body.find(...) parse site"; } +// The whole `[target.]` loop body: the scalar parser, the sweep, and +// the conditional channel that follows it, up to the point the block is +// recorded. Sub-tables are parsed in both halves (`runners` before the sweep, +// `build`, `dependencies`, `targets` and the rest after it). +std::string target_loop_region(const std::string& source) { + auto begin = source.find("for (auto& [triple, val] : *tt) {"); + auto end = source.find("m.conditionalConfigs.push_back(std::move(cc));"); + if (begin == std::string::npos || end == std::string::npos || end < begin) return {}; + return source.substr(begin, end - begin); +} + +// Every sub-table the loop body reads: a `body.find("")` whose first type +// check is `is_table()`, and every `read_deps("", ...)`, the helper the +// three conditional dependency tables go through. +std::vector parsed_target_tables(const std::string& region) { + std::vector out; + const std::string needle = "body.find(\""; + std::vector> hits; + for (std::size_t pos = 0; (pos = region.find(needle, pos)) != std::string::npos;) { + std::size_t nameStart = pos + needle.size(); + std::size_t nameEnd = region.find('"', nameStart); + hits.emplace_back(pos, region.substr(nameStart, nameEnd - nameStart)); + pos = nameEnd; + } + for (std::size_t i = 0; i < hits.size(); ++i) { + std::size_t nextPos = (i + 1 < hits.size()) ? hits[i + 1].first : region.size(); + if (classify(region, hits[i].first, nextPos) == Kind::Table) + out.push_back(hits[i].second); + } + const std::string helper = "read_deps(\""; + for (std::size_t pos = 0; (pos = region.find(helper, pos)) != std::string::npos;) { + std::size_t nameStart = pos + helper.size(); + std::size_t nameEnd = region.find('"', nameStart); + out.push_back(region.substr(nameStart, nameEnd - nameStart)); + pos = nameEnd; + } + return out; +} + +TEST(TargetScalarKeys, EveryParsedSubTableIsKnownToTheSweep) { + // #634: the sweep used to skip every table-valued key, so a misspelled + // section, and a section an older engine does not read, did nothing + // without a word. It now checks tables against `kKnownTargetTables`, and + // this test holds that list to the parse sites in both directions, which + // is the drift the skip was introduced to avoid. + auto source = read_file(repo_root() / "modules" / "manifest" / "src" / "toml.cppm"); + ASSERT_FALSE(source.empty()) << "could not read toml.cppm"; + auto region = target_loop_region(source); + ASSERT_FALSE(region.empty()) << "could not locate the [target.] loop body"; + + auto parsed = parsed_target_tables(region); + auto known = known_list(region, "kKnownTargetTables[]"); + ASSERT_FALSE(parsed.empty()); + ASSERT_FALSE(known.empty()); + + for (auto const& key : parsed) + EXPECT_NE(std::ranges::find(known, key), known.end()) + << "'" << key << "' is parsed as a sub-table of [target.] " + << "but missing from kKnownTargetTables, so the sweep would report it"; + for (auto const& key : known) + EXPECT_NE(std::ranges::find(parsed, key), parsed.end()) + << "kKnownTargetTables names '" << key << "', which has no parse site " + << "in the [target.] loop body"; + // The table this list was introduced beside. + EXPECT_NE(std::ranges::find(parsed, std::string("targets")), parsed.end()); +} + TEST(TargetScalarKeys, MinApiLevelIsParsedAndKnown) { // A direct, non-derived check on the specific regression: `min_api_level` // must be both a parse site and a known scalar key. If this test passes diff --git a/tests/unit/test_test_targets.cpp b/tests/unit/test_test_targets.cpp index ea93fdda4..d03eff962 100644 --- a/tests/unit/test_test_targets.cpp +++ b/tests/unit/test_test_targets.cpp @@ -112,6 +112,106 @@ version = "0.1.0" std::filesystem::path("tests") / "vendor" / "external.cpp"); } +// `[test] discover` (#634 A5): names are relative to the fixed directory of +// the glob that found the file, `main` stays relative to the package root. +TEST(TestTargets, DiscoverNamesAreRelativeToTheGlobsDirectory) { + Tmp tmp; + write_manifest(tmp.path, R"([package] +name = "demo" +version = "0.1.0" + +[test] +discover = ["checks/**/*.cpp"] +)"); + write_file(tmp.path / "checks/deep/a.cpp", "int main() {}\n"); + write_file(tmp.path / "tests/ignored.cpp", "int main() {}\n"); + + auto result = mcpp::build::discover_test_targets(tmp.path, {}); + ASSERT_TRUE(result.has_value()) << result.error(); + EXPECT_TRUE(result->discoverDeclared); + ASSERT_EQ(result->targets.size(), 1u); + EXPECT_EQ(result->targets[0].name, "deep/a"); + EXPECT_EQ(std::filesystem::path(result->targets[0].main), + std::filesystem::path("checks") / "deep" / "a.cpp"); +} + +// An exclusion removes a file whichever positive glob found it, and a glob +// with no fixed directory names files relative to the package root. +TEST(TestTargets, DiscoverExclusionsApplyAcrossGlobs) { + Tmp tmp; + write_manifest(tmp.path, R"([package] +name = "demo" +version = "0.1.0" + +[test] +discover = ["tests/**/*.cpp", "*_check.cpp", "!tests/fixtures/**"] +)"); + write_file(tmp.path / "tests/kept.cpp", "int main() {}\n"); + write_file(tmp.path / "tests/fixtures/dropped.cpp", "int main() {}\n"); + write_file(tmp.path / "root_check.cpp", "int main() {}\n"); + + auto result = mcpp::build::discover_test_targets(tmp.path, {}); + ASSERT_TRUE(result.has_value()) << result.error(); + std::set names; + for (auto const& t : result->targets) names.insert(t.name); + EXPECT_EQ(names, (std::set{"kept", "root_check"})); +} + +TEST(TestTargets, AnEmptyDiscoverFindsNothing) { + Tmp tmp; + write_manifest(tmp.path, R"([package] +name = "demo" +version = "0.1.0" + +[test] +discover = [] +)"); + write_file(tmp.path / "tests/main.cpp", "int main() {}\n"); + + auto result = mcpp::build::discover_test_targets(tmp.path, {}); + ASSERT_TRUE(result.has_value()) << result.error(); + EXPECT_TRUE(result->discoverDeclared); + EXPECT_TRUE(result->discover.empty()); + EXPECT_TRUE(result->targets.empty()); +} + +TEST(TestTargets, TwoFilesWithOneNameAreRefusedNamingBoth) { + Tmp tmp; + write_manifest(tmp.path, R"([package] +name = "demo" +version = "0.1.0" + +[test] +discover = ["tests/*.cpp", "checks/*.cpp"] +)"); + write_file(tmp.path / "tests/same.cpp", "int main() {}\n"); + write_file(tmp.path / "checks/same.cpp", "int main() {}\n"); + + auto result = mcpp::build::discover_test_targets(tmp.path, {}); + ASSERT_FALSE(result.has_value()); + EXPECT_NE(result.error().find("duplicate test name 'same'"), std::string::npos) << result.error(); + EXPECT_NE(result.error().find("tests/same.cpp"), std::string::npos) << result.error(); + EXPECT_NE(result.error().find("checks/same.cpp"), std::string::npos) << result.error(); +} + +// Without the key the set, and every name, is the one earlier releases gave. +TEST(TestTargets, WithoutDiscoverTheDefaultIsTestsGlob) { + Tmp tmp; + write_manifest(tmp.path, R"([package] +name = "demo" +version = "0.1.0" +)"); + write_file(tmp.path / "tests/unit/a.cpp", "int main() {}\n"); + write_file(tmp.path / "checks/b.cpp", "int main() {}\n"); + + auto result = mcpp::build::discover_test_targets(tmp.path, {}); + ASSERT_TRUE(result.has_value()) << result.error(); + EXPECT_FALSE(result->discoverDeclared); + EXPECT_EQ(result->discover, std::vector{"tests/**/*.cpp"}); + ASSERT_EQ(result->targets.size(), 1u); + EXPECT_EQ(result->targets[0].name, "unit/a"); +} + TEST(TestTargets, BrokenManifestStillReturnsInventory) { Tmp tmp; write_file(tmp.path / "mcpp.toml", "this is not valid TOML\n");