Skip to content

Commit cd34a3b

Browse files
committed
examples: the module surface, and one header kept for contrast
The payload lane and the boundary lane are separate, and each example now shows one of them. `vulkan` reaches its SPIR-V through a MODULE. `src/vulkan/saxpy.cpp` opened with `#include "scale_comp.h"` -- a name no line in the project produced and no reader could derive without opening the rule -- and now imports `vulkan_saxpy.shaders`, whose name and whose `scale_comp()` accessor are both derived from names the project already wrote. The accessor answers with the address and the byte count together, which is what makes `sizeof` the wrong question rather than merely an awkward one: the rule may put the words in an object, and there is then no array to take the size of. `cuda` and `sycl` GENERATE their `extern "C"` boundary. Each entry point is marked with `MCPP_EXPORT_C` where it is defined, and `mcpp.tools.island` writes the header the island's compiler reads and the module the seam imports. `include/saxpy/saxpy.h` is gone from both. Both halves are handed to the scan, so a signature that drifted between the island and the CPU fallback is refused while the boundary is generated -- the one point at which both texts exist at once, and the only check available at a boundary where C language linkage does not mangle and the two halves are never in one link. The island names nothing and the host half writes one `#include`, and that asymmetry is measured rather than incidental. The island is compiled by a driver mcpp did not invoke, so its rule takes forced-include flags for that one command line. The host half is ordinary project C++, and the only project-wide channel would force the header into every C++ translation unit including the seam, where declarations ahead of `export module` are ill-formed. `hip` keeps its hand-written header, and its README says why. It computes the same thing through the same seam as `cuda`, so the difference between the two examples is exactly this one thing, and the hand-written form is what every C boundary looks like today. Neither is deprecated; the generated form is the default because the copy it removes is the one that goes wrong quietly. `multi-backend` reaches its SPIR-V as `opkit.shaders` too, and keeps `include/opkit/opkit.h` written by hand -- which is the distinction this example is for. That header is a LIBRARY boundary: consumers outside this tree compile against it, so it is not an intermediate and generating it would not remove a copy, it would move one. The shader payload is an intermediate, and it moves. `examples/10-graphics/offscreen` moves too, and it is the one example CI both builds AND RUNS on two platforms -- so the surface is exercised there rather than merely compiled. Two includes become one import, and the two stages are `offscreen::shaders::triangle_vert()` and `::triangle_frag()`. `cann` is unchanged. Its boundary has a different shape -- `saxpy_device` between the seam and a host glue file, `saxpy_launch` between that glue and the kernel -- and both halves of the first one are compiled by mcpp, so the forced-include asymmetry above does not apply. Its README already records that the example does not build yet. Verified on this host: cuda and sycl build and run on an RTX 4080 with the generated boundary on the island's own command line, vulkan runs on the same device with its shader reached as a module, multi-backend runs on lavapipe with `--accel "vulkan1.2"`, the offscreen triangle renders on lavapipe at centre pixel `(124, 70, 62, 255)` -- the same pixel its CPU leg produces, which is the assertion CI makes -- and each CPU leg builds and runs under `--no-accel`. `docs/20`'s two-lanes section said the seam is written by hand and the shader lane's equivalent is generated. Half of that still holds: the seam is a design decision. What sits under it is each signature written a second time, and that half is now generated too. The section says so, and gains the table that maps a file name to the call it arrives as -- which is the question a reader of that document actually has. Every example under `examples/09-heterogeneous` and `examples/10-graphics` pins 0.3.0 now, `hip` and `cann` included. Neither uses anything new, and leaving them behind would be defensible in isolation -- but `hip` exists to be read beside `cuda`, and a reader comparing the two would find a second difference the text never explains. The floor sentences move with them. The six plugin pins in `docs/05` and `docs/20` move too, in both languages. A version in a document is the line someone pastes, and 0.2.1 and 0.2.4 predate every declaration those sections now describe -- a reader who copied one would get a build in which the rules never route. The Windows graphics run step comes back, as a measurement. It was written once, failed, and came out with the wrong cause recorded; the cause has since been found in `xim:mesa-lavapipe`, which wrote its rewritten `library_path` into the ICD's JSON string unescaped -- `C:\Users\...` carries `\U`, which is not a JSON escape, so the loader's cJSON parser rejected the manifest and skipped the ICD silently. Fixed in openxlings/xim-pkgindex#781. The step removes the lavapipe store entry before rebuilding, and that is not tidiness: `install()` does not re-run over an existing payload and `~/.mcpp` is cached across runs, so a runner that installed 26.2.0 before the fix would keep the manifest it was given and the step would measure the cache. It also prints the manifest and asserts it PARSES, so a failure is read against what the loader was actually handed. If it is red the cause is somewhere else again, and it comes back out with what it showed recorded. Requires mcpp:plugins 0.3.0.
1 parent 4d3e969 commit cd34a3b

38 files changed

Lines changed: 562 additions & 201 deletions

File tree

.agents/docs/2026-09-07-module-first-heterogeneous-surface.md

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -846,6 +846,55 @@ and a newer one moves a project to the module surface with nothing declared.
846846
| Seamless upgrade | A project that writes nothing new keeps its behaviour | Header storage stays the default under `modules = false`; the default surface follows `[language] modules` |
847847
| Test coverage | Each surface and each storage has a consumer that runs | P7 |
848848
849+
### 13.3.1 What the self-review found after the surface was working
850+
851+
Four findings, three of them in names the surface generates. Every one was found
852+
by giving the generator an input nobody had tried, and none of them by reading
853+
the code -- which is the reason they are recorded together.
854+
855+
**The user-facing name came from the wrong question.** The module a consumer
856+
imports was derived from the leaf of `MCPP_MANIFEST_DIR` -- a directory name --
857+
because nothing in the build-program contract answered "what package am I
858+
building". `examples/09-heterogeneous/vulkan/app/` declares
859+
`name = "vulkan-saxpy"` and generated `app.shaders`. Worse than the defect is
860+
that no fixture could see it: all fourteen had a package name equal to their
861+
directory name, so both derivations produced the same string. Fixed by
862+
`MCPP_PKG_NAME` in the engine, and by making one fixture's two names differ on
863+
purpose.
864+
865+
**A generated name may not be a C++ keyword.** Six sites ran the same character
866+
filter and none checked for reserved words. `shaders/default/` produced
867+
`namespace default {` in a generated file. Fixed by one `surface::identifier`
868+
that owns all three transformations, applied at the point where the accessor is
869+
EMITTED rather than in each producer -- a rule that builds the name from a file
870+
stem cannot know it has produced a keyword until it reaches the line that writes
871+
the function's name.
872+
873+
**A seam's two halves are compared nowhere else.** The island and the host
874+
fallback define one `extern "C"` boundary and are never in one link, and C
875+
language linkage does not mangle, so two that declare a name differently build
876+
cleanly and the artifact reads its arguments by whichever it was compiled with.
877+
`mcpp.tools.island::scan` is handed both files and is therefore the only point
878+
at which both texts exist at once. It refuses there.
879+
880+
**One fix was written and withdrawn.** `host-module` is inferred from a rule
881+
feature the consumer REQUESTS. A rule package whose rule sits in its own
882+
`[features] default` activates without being named, so the rule modules are
883+
collected -- that reads the resolved set -- while the inference reads the
884+
requested one. Extending the inference to the dependency's defaults was measured
885+
against a probe package and made the outcome WORSE: the refusal went from
886+
887+
error: build.mcpp imports 'probe.rules.probe', and no dependency provides
888+
it as a host module.
889+
declared without `host-module = true`: probe.rules
890+
891+
to the same refusal plus `importable here: rules`, which says the dependency is
892+
wired up while the feature's own module still is not. The gap is further down,
893+
in when a default feature's sources are folded into the set `units()`
894+
enumerates. `mcpp:plugins` declares `default = []`, so nothing shipped reaches
895+
it, and the existing refusal already names exactly what to add. Recorded here
896+
rather than half-fixed.
897+
849898
### 13.4 Release and verification sequence
850899
851900
1. mcpp pull request: E1 through E5. CI green on the head, then merged, then the

.github/workflows/ci-windows.yml

Lines changed: 56 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -456,41 +456,65 @@ jobs:
456456
done
457457
echo "ok: both shader stages compiled and the Vulkan half linked"
458458
459-
# WINDOWS STAYS AT "BUILDS", AND WHAT REMAINS OPEN IS THE DRIVER.
459+
# THE RUN, WITH THE ICD MANIFEST FIXED UPSTREAM.
460460
#
461-
# A run step was written, pushed, and measured (run 34135108981). The
462-
# manifest declaration worked -- `Provisioning [xlings.workspace] entries
463-
# (xim:mesa-lavapipe@26.2.0)` -- the ICD manifest was found in the store,
464-
# and the program printed `render unavailable`, which is `src/main.cpp`
465-
# reporting that the render function returned nothing.
461+
# This step was written once, failed, and came out with the wrong cause
462+
# recorded. The program printed `render unavailable` -- `src/main.cpp`
463+
# reporting that the render function returned nothing -- and that was
464+
# attributed to a missing `vulkan-1.dll`. It cannot have been: the Vulkan
465+
# leg imports `vkCreateInstance` from that DLL, so a process that could not
466+
# find it would fail during image load and print nothing.
466467
#
467-
# THAT OUTPUT PLACES THE FAILURE AFTER THE LOADER, NOT AT IT. The Vulkan
468-
# leg imports `vkCreateInstance` from `vulkan-1.dll` through the import
469-
# library, so a process missing that DLL fails during image load and
470-
# prints nothing at all. It printed. The loader was there, it ran, and it
471-
# enumerated no device -- which is a statement about the ICD.
468+
# The cause was `xim:mesa-lavapipe` writing its rewritten `library_path`
469+
# into the ICD's JSON string unescaped, so the value carried `C:\Users\...`
470+
# and `\U` is not a JSON escape. The loader parses manifests with cJSON and
471+
# silently SKIPS one it cannot parse -- zero devices, no error, and
472+
# indistinguishable from having no GPU. openxlings/xim-pkgindex#781 writes
473+
# forward slashes instead.
472474
#
473-
# An earlier revision of this comment named a missing `vulkan-1.dll` as
474-
# the cause. That was an inference from the descriptor's note about static
475-
# linkage rather than a reading of the failure, and the log refutes it.
476-
# Two facts stand against it: this program ran, and mcpp-index's own
477-
# `vulkan-tests` member calls `vkEnumerateInstanceVersion` on the windows
478-
# shards and passes.
479-
#
480-
# A separate measurement, recorded here because it removes a second thing
481-
# from the list of suspects: the Khronos loader in `compat:vulkan` builds
482-
# into a working `vulkan-1.dll` from the source the index already carries
483-
# (mingw, 265 exports matching upstream's `vulkan-1.def` exactly, DllMain
484-
# present, importing only ADVAPI32/CFGMGR32/KERNEL32/msvcrt). The
485-
# descriptor's note argues that a Windows loader must be a DLL, not that
486-
# it cannot be built -- so a hermetic Windows loader is available whenever
487-
# it is wanted. It is not what this step is waiting on.
488-
#
489-
# What this step is waiting on is `xim:mesa-lavapipe`'s Windows payload
490-
# producing a device under an mcpp-launched process. Until that is
491-
# measured, the job builds the Vulkan half and runs the CPU fallback.
492-
# macOS is not in the same position: MoltenVK enumerates once the instance
493-
# asks for portability, which `src/vulkan/render.cpp` now does.
475+
# THE STORE ENTRY IS REMOVED FIRST, and that is not tidiness. `install()`
476+
# does not re-run over an existing payload and `~/.mcpp` is cached across
477+
# runs, so a runner that installed 26.2.0 before the fix would keep the
478+
# manifest it was given -- and this step would measure the cache.
479+
- name: "Graphics: the offscreen example RUNS on lavapipe"
480+
shell: bash
481+
run: |
482+
set -e
483+
export MCPP_VENDORED_XLINGS="$XLINGS_BIN"
484+
# Force install() to run again with the corrected hook.
485+
for store in "${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-mesa-lavapipe" \
486+
"$HOME/.xlings/data/xpkgs/xim-x-mesa-lavapipe"; do
487+
[ -d "$store" ] && { echo "removing stale $store"; rm -rf "$store"; }
488+
done
489+
cd "$GITHUB_WORKSPACE/examples/10-graphics/offscreen"
490+
# Rebuilt so the removal above is followed by a provision, and with the
491+
# SAME toolchain the build step named -- an unnamed one re-resolves to
492+
# whatever the default is and rebuilds everything for a reason that has
493+
# nothing to do with this step's subject.
494+
rm -rf target
495+
"$MCPP_SELF" build --toolchain "llvm@20.1.7"
496+
# NOT installed from here. The example declares `xim:mesa-lavapipe`
497+
# under `cfg(windows)`, so the build above provisioned it, and looking
498+
# for the ICD without installing anything is what asserts the
499+
# declaration works.
500+
icd=$(find "${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-mesa-lavapipe" \
501+
"$HOME/.xlings/data/xpkgs/xim-x-mesa-lavapipe" \
502+
-name 'lvp_icd.x86_64.json' -print -quit 2>/dev/null || true)
503+
[ -n "$icd" ] || { echo "no lavapipe ICD in either store"; exit 1; }
504+
echo "ICD: $icd"
505+
# The manifest has to PARSE, and that is the property this whole step
506+
# turns on. Printed before the run, so a failure below is read against
507+
# what the loader was actually handed.
508+
cat "$icd"
509+
python3 -c "import json,sys; json.load(open(sys.argv[1])); print('the ICD manifest parses as JSON')" "$icd"
510+
out=$(VK_DRIVER_FILES="$icd" "$MCPP_SELF" run --toolchain "llvm@20.1.7" 2>&1) \
511+
|| { echo "$out"; exit 1; }
512+
echo "$out"
513+
echo "$out" | grep -q 'llvmpipe' \
514+
|| { echo "the run did not reach the lavapipe device"; exit 1; }
515+
echo "$out" | grep -qE 'centre pixel: \([0-9]+, [0-9]+, [0-9]+, 255\)' \
516+
|| { echo "no centre pixel was reported"; exit 1; }
517+
echo "ok: lavapipe enumerated and the image was rendered on it"
494518
495519
- name: "Toolchain: LLVM — build mcpp (self-host)"
496520
shell: bash

docs/01-examples.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -32,10 +32,10 @@ examples.
3232
| 07 | [`examples/07-project-subos`](../examples/07-project-subos/) | A build program that finds its tools in the environment the project declared | `[xlings] subos`, `[xlings.workspace]`, a build program whose `PATH` is the environment the project named |
3333
| 08 | [`examples/08-build-rules`](../examples/08-build-rules/) | Two rule packages and a project that uses both | `host-module = true`, `[build-dependencies]`, `mcpp::action` with `role = "check"` |
3434
| 09 | [`examples/09-heterogeneous`](../examples/09-heterogeneous/) | One computation on a device, in several programming models, with a CPU fallback in each; plus one artifact carrying several backends at once | `accel`, constrained source globs, the seam module, rule packages from `mcpp:plugins`, `cfg(accelerator = …)` |
35-
| 09a | [`…/cuda`](../examples/09-heterogeneous/cuda/) | A CUDA kernel behind a seam module | `mcpp.rules.cuda`, `mcpp::action` with `role = "object"`, the driver stated as a fact and a floor |
36-
| 09b | [`…/vulkan`](../examples/09-heterogeneous/vulkan/) | The same computation as a Vulkan compute shader, on a GPU or on the CPU | `mcpp.rules.spirv`, `mcpp::action` with `role = "source"`, generated headers, a software driver as a payload |
37-
| 09c | [`…/sycl`](../examples/09-heterogeneous/sycl/) | The same computation as a SYCL kernel, compiled by a second compiler | `mcpp.rules.sycl`, the `.sycl` device extension, a chained `mcpp::action` for the device link, `compat:sycl-runtime` |
38-
| 09d | [`…/hip`](../examples/09-heterogeneous/hip/) | The same computation in HIP, reaching an NVIDIA device | `mcpp.rules.hip`, HIP as a header layer over the CUDA runtime, a two-chunk `accel` |
35+
| 09a | [`…/cuda`](../examples/09-heterogeneous/cuda/) | A CUDA kernel behind a seam module, with the `extern "C"` boundary GENERATED | `mcpp.rules.cuda`, `mcpp.tools.island`, `mcpp::action` with `role = "object"`, the driver stated as a fact and a floor |
36+
| 09b | [`…/vulkan`](../examples/09-heterogeneous/vulkan/) | The same computation as a Vulkan compute shader, with the SPIR-V payload reached as a MODULE | `mcpp.rules.spirv`, the module surface, `mcpp::action` with `role = "source"`, a software driver as a payload |
37+
| 09c | [`…/sycl`](../examples/09-heterogeneous/sycl/) | The same computation as a SYCL kernel, compiled by a second compiler | `mcpp.rules.sycl`, `mcpp.tools.island`, the `.sycl` device extension, a chained `mcpp::action` for the device link, `compat:sycl-runtime` |
38+
| 09d | [`…/hip`](../examples/09-heterogeneous/hip/) | The same computation in HIP, with the boundary WRITTEN BY HAND — the contrast against 09a | `mcpp.rules.hip`, HIP as a header layer over the CUDA runtime, a two-chunk `accel` |
3939
| 09e | [`…/multi-backend`](../examples/09-heterogeneous/multi-backend/) | Several backends in ONE artifact, chosen at run time — the library shape, not the program shape | `accel` as a set, `cfg(accelerator = "none")` and its negation, a dispatch chain, a module seam over a C island boundary |
4040
| 09f | [`…/cann`](../examples/09-heterogeneous/cann/) | An Ascend C kernel behind the same seam. **Does not build yet** — its README names the two missing pieces | the `.asc` device extension, `op_kernel`/`op_host` as an island CANN already has, `accelerator = "none"` for the fallback |
4141
| 10 | [`examples/10-graphics`](../examples/10-graphics/) | Graphics rather than compute: a rendering pipeline whose result is pixels | `mcpp.rules.spirv` for the vertex and fragment stages, offscreen rendering as the assertable form |

docs/05-mcpp-toml.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2308,7 +2308,7 @@ have none, and nothing is what they write:
23082308

23092309
```toml
23102310
[build-dependencies.mcpp]
2311-
plugins = { version = "0.2.4", features = ["rules-cuda"], host-module = true }
2311+
plugins = { version = "0.3.0", features = ["rules-cuda"], host-module = true }
23122312
```
23132313

23142314
That one edge is the whole declaration. The rule package names the packages its
@@ -2546,7 +2546,7 @@ rules-spirv = { sources = ["rules/spirv.cppm"] } # export module mcpp.rules.spir
25462546
```toml
25472547
# a consumer
25482548
[build-dependencies.mcpp]
2549-
plugins = { version = "0.2.1", features = ["rules-spirv"], host-module = true }
2549+
plugins = { version = "0.3.0", features = ["rules-spirv"], host-module = true }
25502550
```
25512551

25522552
**`[build-dependencies]`, not `[dependencies]`** — a rule package is the case

docs/20-heterogeneous-builds.md

Lines changed: 40 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -185,13 +185,20 @@ could read a BMI.
185185
The seam above is written by hand, and the shader lane's equivalent is
186186
generated. That is not an inconsistency; the two lanes carry different things.
187187
188-
**A device translation unit is code.** Its interface is a design decision --
189-
which functions, which types, what happens on failure -- and no generator can
190-
make that decision well. So CUDA, HIP, SYCL and Ascend C get a hand-written
191-
seam, and the `extern "C"` header exists for the ABI reason above. Both are
192-
already invisible to a consumer: only the seam includes the header, and
193-
everything downstream writes `import app.saxpy`. **These lanes are module-first
194-
today and always have been.**
188+
**A device translation unit is code, and the seam over it is a design
189+
decision** -- which functions, which types, what happens on failure. No
190+
generator makes that decision well, so CUDA, HIP, SYCL and Ascend C get a
191+
hand-written seam, and everything downstream writes `import app.saxpy`.
192+
193+
**The `extern "C"` boundary UNDER that seam is not a design decision.** It is
194+
each entry point's signature, stated a second time in a header, at the one place
195+
where a disagreement is invisible: C language linkage does not mangle, and a
196+
device island and its host fallback are never in one link. `mcpp.tools.island`
197+
reads the marked declarations out of both implementations and writes that header
198+
and a module over it, so the signatures exist once and two halves that disagree
199+
are refused where both texts are in front of the generator.
200+
`examples/09-heterogeneous/cuda` and `.../sycl` are that shape;
201+
`.../hip` keeps its header written by hand, so the two can be read side by side.
195202
196203
**A shader or an embedded file is data.** Its interface is an address and a
197204
size, which is mechanical, so a rule package generates it and a consumer writes
@@ -203,6 +210,31 @@ So the rule is not "generate the interface" or "write it by hand". It is: a
203210
mechanical interface is generated, a designed one is written, and in both cases
204211
the header is an intermediate that no consumer names.
205212
213+
#### The name a payload arrives under
214+
215+
The module and the namespace are one identifier path, derived from names the
216+
project already wrote.
217+
218+
| Written | Reached as |
219+
|---|---|
220+
| `[package] name = "myapp"` | module root `myapp` |
221+
| `shaders/scale.comp` | `myapp::shaders::scale_comp()` |
222+
| `shaders/a/scale.comp` | `myapp::shaders::a::scale_comp()` |
223+
| a `MCPP_EXPORT_C` entry point | `export using ::the_name;` in the boundary module |
224+
225+
The root is the **package's** name with non-identifier characters replaced, not
226+
its directory's -- the two differ whenever a package sits under a generic folder,
227+
and mcpp reports the package name to a build program from 2026.9.7.1 for exactly
228+
this. The stem and the stage make the accessor (`scale.comp` -> `scale_comp`),
229+
and the directory below the group's base becomes namespace segments, which is
230+
what makes two shaders sharing a stem two things rather than a collision. A
231+
project that wants another name passes one; `examples/09-heterogeneous/cuda`
232+
does, so its boundary is `app.kernels` beside its seam `app.saxpy`.
233+
234+
The accessor answers with the address and the byte count together. `sizeof` is
235+
not merely awkward at this boundary, it is unanswerable: the words may be in an
236+
object rather than in an array, and there is then nothing to take the size of.
237+
206238
## Compiling an island
207239
208240
The command that invokes a device compiler is not built into mcpp. It is
@@ -221,7 +253,7 @@ A project that wants a CUDA island writes one edge:
221253
222254
```toml
223255
[build-dependencies.mcpp]
224-
plugins = { version = "0.2.4", features = ["rules-cuda"], host-module = true }
256+
plugins = { version = "0.3.0", features = ["rules-cuda"], host-module = true }
225257
```
226258

227259
and nothing else. The vendor toolkit, its runtime and whatever else the rule

0 commit comments

Comments
 (0)