Skip to content

Commit bbecc48

Browse files
committed
examples: the module surface, and one header kept for contrast
The payload lane and the boundary lane are separate, and each example now shows one of them. `vulkan` reaches its SPIR-V through a MODULE. `src/vulkan/saxpy.cpp` opened with `#include "scale_comp.h"` -- a name no line in the project produced and no reader could derive without opening the rule -- and now imports `vulkan_saxpy.shaders`, whose name and whose `scale_comp()` accessor are both derived from names the project already wrote. The accessor answers with the address and the byte count together, which is what makes `sizeof` the wrong question rather than merely an awkward one: the rule may put the words in an object, and there is then no array to take the size of. `cuda` and `sycl` GENERATE their `extern "C"` boundary. Each entry point is marked with `MCPP_EXPORT_C` where it is defined, and `mcpp.tools.island` writes the header the island's compiler reads and the module the seam imports. `include/saxpy/saxpy.h` is gone from both. Both halves are handed to the scan, so a signature that drifted between the island and the CPU fallback is refused while the boundary is generated -- the one point at which both texts exist at once, and the only check available at a boundary where C language linkage does not mangle and the two halves are never in one link. The island names nothing and the host half writes one `#include`, and that asymmetry is measured rather than incidental. The island is compiled by a driver mcpp did not invoke, so its rule takes forced-include flags for that one command line. The host half is ordinary project C++, and the only project-wide channel would force the header into every C++ translation unit including the seam, where declarations ahead of `export module` are ill-formed. `hip` keeps its hand-written header, and its README says why. It computes the same thing through the same seam as `cuda`, so the difference between the two examples is exactly this one thing, and the hand-written form is what every C boundary looks like today. Neither is deprecated; the generated form is the default because the copy it removes is the one that goes wrong quietly. `multi-backend` reaches its SPIR-V as `opkit.shaders` too, and keeps `include/opkit/opkit.h` written by hand -- which is the distinction this example is for. That header is a LIBRARY boundary: consumers outside this tree compile against it, so it is not an intermediate and generating it would not remove a copy, it would move one. The shader payload is an intermediate, and it moves. `examples/10-graphics/offscreen` moves too, and it is the one example CI both builds AND RUNS on two platforms -- so the surface is exercised there rather than merely compiled. Two includes become one import, and the two stages are `offscreen::shaders::triangle_vert()` and `::triangle_frag()`. `cann` is unchanged. Its boundary has a different shape -- `saxpy_device` between the seam and a host glue file, `saxpy_launch` between that glue and the kernel -- and both halves of the first one are compiled by mcpp, so the forced-include asymmetry above does not apply. Its README already records that the example does not build yet. Verified on this host: cuda and sycl build and run on an RTX 4080 with the generated boundary on the island's own command line, vulkan runs on the same device with its shader reached as a module, multi-backend runs on lavapipe with `--accel "vulkan1.2"`, the offscreen triangle renders on lavapipe at centre pixel `(124, 70, 62, 255)` -- the same pixel its CPU leg produces, which is the assertion CI makes -- and each CPU leg builds and runs under `--no-accel`. `docs/20`'s two-lanes section said the seam is written by hand and the shader lane's equivalent is generated. Half of that still holds: the seam is a design decision. What sits under it is each signature written a second time, and that half is now generated too. The section says so, and gains the table that maps a file name to the call it arrives as -- which is the question a reader of that document actually has. Every example under `examples/09-heterogeneous` and `examples/10-graphics` pins 0.3.0 now, `hip` and `cann` included. Neither uses anything new, and leaving them behind would be defensible in isolation -- but `hip` exists to be read beside `cuda`, and a reader comparing the two would find a second difference the text never explains. The floor sentences move with them. The six plugin pins in `docs/05` and `docs/20` move too, in both languages. A version in a document is the line someone pastes, and 0.2.1 and 0.2.4 predate every declaration those sections now describe -- a reader who copied one would get a build in which the rules never route. The Windows graphics run step comes back, as a measurement. It was written once, failed, and came out with the wrong cause recorded; the cause has since been found in `xim:mesa-lavapipe`, which wrote its rewritten `library_path` into the ICD's JSON string unescaped -- `C:\Users\...` carries `\U`, which is not a JSON escape, so the loader's cJSON parser rejected the manifest and skipped the ICD silently. Fixed in openxlings/xim-pkgindex#781. The step removes the lavapipe store entry before rebuilding, and that is not tidiness: `install()` does not re-run over an existing payload and `~/.mcpp` is cached across runs, so a runner that installed 26.2.0 before the fix would keep the manifest it was given and the step would measure the cache. It also prints the manifest and asserts it PARSES, so a failure is read against what the loader was actually handed. Its first attempt failed on its own preparation, and the correction is the half that is easy to forget: removing the payload is not enough. mcpp records a provisioned package set under `<mcpp home>/provisioned/`, and with the marker present and the payload deleted the rebuild SKIPS provisioning -- it printed no `Provisioning` line at all and the step then reported `no lavapipe ICD in either store`, which reads as a missing package rather than as a skipped step. Both halves go, and the step now asserts that provisioning ran, so a later failure separates "the fix did not apply" from "it applied and the device still did not appear". The corrected run then showed the fix reaching the runner. Provisioning ran, and the manifest it produced is "library_path": "C:/Users/runneradmin/.mcpp/.../lib/vulkan_lvp.dll" forward slashes, and `the ICD manifest parses as JSON`. The step still exited 1, on `error: unknown option: --toolchain` -- `mcpp run` has no such option, and I had added it to that line. Removed; the job's earlier step sets the project default, so `run` replays what `build` selected. With that corrected the measurement is clean, and it says the fix was real and was not this failure's cause: Provisioning [xlings.workspace] entries (xim:mesa-lavapipe@26.2.0) "library_path": "C:/Users/runneradmin/.mcpp/.../lib/vulkan_lvp.dll" the ICD manifest parses as JSON render unavailable The loader parses the manifest and still enumerates nothing. `LoadLibrary` is ruled out by reading the payload: `vulkan_lvp.dll` imports only ADVAPI32, GDI32, KERNEL32, ntdll, ole32, SHELL32 and USER32, all of which any Windows has. What is left is the environment variable's vintage -- `VK_DRIVER_FILES` replaced `VK_ICD_FILENAMES` in loader 1.3.234, and an older loader ignores it and enumerates whatever the registry lists, which on a runner with no GPU is nothing -- or something the loader rejects on its own terms. So this iteration sets both spellings and, on failure, re-runs with `VK_LOADER_DEBUG=all`. `render unavailable` is the program's report that it got no device; it is not a reason, and the loader will give one. A red run now arrives with an account somebody can act on rather than with a new hypothesis. Requires mcpp:plugins 0.3.0.
1 parent 4d3e969 commit bbecc48

38 files changed

Lines changed: 610 additions & 200 deletions

File tree

.agents/docs/2026-09-07-module-first-heterogeneous-surface.md

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -846,6 +846,55 @@ and a newer one moves a project to the module surface with nothing declared.
846846
| Seamless upgrade | A project that writes nothing new keeps its behaviour | Header storage stays the default under `modules = false`; the default surface follows `[language] modules` |
847847
| Test coverage | Each surface and each storage has a consumer that runs | P7 |
848848
849+
### 13.3.1 What the self-review found after the surface was working
850+
851+
Four findings, three of them in names the surface generates. Every one was found
852+
by giving the generator an input nobody had tried, and none of them by reading
853+
the code -- which is the reason they are recorded together.
854+
855+
**The user-facing name came from the wrong question.** The module a consumer
856+
imports was derived from the leaf of `MCPP_MANIFEST_DIR` -- a directory name --
857+
because nothing in the build-program contract answered "what package am I
858+
building". `examples/09-heterogeneous/vulkan/app/` declares
859+
`name = "vulkan-saxpy"` and generated `app.shaders`. Worse than the defect is
860+
that no fixture could see it: all fourteen had a package name equal to their
861+
directory name, so both derivations produced the same string. Fixed by
862+
`MCPP_PKG_NAME` in the engine, and by making one fixture's two names differ on
863+
purpose.
864+
865+
**A generated name may not be a C++ keyword.** Six sites ran the same character
866+
filter and none checked for reserved words. `shaders/default/` produced
867+
`namespace default {` in a generated file. Fixed by one `surface::identifier`
868+
that owns all three transformations, applied at the point where the accessor is
869+
EMITTED rather than in each producer -- a rule that builds the name from a file
870+
stem cannot know it has produced a keyword until it reaches the line that writes
871+
the function's name.
872+
873+
**A seam's two halves are compared nowhere else.** The island and the host
874+
fallback define one `extern "C"` boundary and are never in one link, and C
875+
language linkage does not mangle, so two that declare a name differently build
876+
cleanly and the artifact reads its arguments by whichever it was compiled with.
877+
`mcpp.tools.island::scan` is handed both files and is therefore the only point
878+
at which both texts exist at once. It refuses there.
879+
880+
**One fix was written and withdrawn.** `host-module` is inferred from a rule
881+
feature the consumer REQUESTS. A rule package whose rule sits in its own
882+
`[features] default` activates without being named, so the rule modules are
883+
collected -- that reads the resolved set -- while the inference reads the
884+
requested one. Extending the inference to the dependency's defaults was measured
885+
against a probe package and made the outcome WORSE: the refusal went from
886+
887+
error: build.mcpp imports 'probe.rules.probe', and no dependency provides
888+
it as a host module.
889+
declared without `host-module = true`: probe.rules
890+
891+
to the same refusal plus `importable here: rules`, which says the dependency is
892+
wired up while the feature's own module still is not. The gap is further down,
893+
in when a default feature's sources are folded into the set `units()`
894+
enumerates. `mcpp:plugins` declares `default = []`, so nothing shipped reaches
895+
it, and the existing refusal already names exactly what to add. Recorded here
896+
rather than half-fixed.
897+
849898
### 13.4 Release and verification sequence
850899
851900
1. mcpp pull request: E1 through E5. CI green on the head, then merged, then the

.github/workflows/ci-windows.yml

Lines changed: 104 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -456,41 +456,114 @@ jobs:
456456
done
457457
echo "ok: both shader stages compiled and the Vulkan half linked"
458458
459-
# WINDOWS STAYS AT "BUILDS", AND WHAT REMAINS OPEN IS THE DRIVER.
459+
# THE RUN, WITH THE ICD MANIFEST FIXED UPSTREAM.
460460
#
461-
# A run step was written, pushed, and measured (run 34135108981). The
462-
# manifest declaration worked -- `Provisioning [xlings.workspace] entries
463-
# (xim:mesa-lavapipe@26.2.0)` -- the ICD manifest was found in the store,
464-
# and the program printed `render unavailable`, which is `src/main.cpp`
465-
# reporting that the render function returned nothing.
461+
# This step was written once, failed, and came out with the wrong cause
462+
# recorded. The program printed `render unavailable` -- `src/main.cpp`
463+
# reporting that the render function returned nothing -- and that was
464+
# attributed to a missing `vulkan-1.dll`. It cannot have been: the Vulkan
465+
# leg imports `vkCreateInstance` from that DLL, so a process that could not
466+
# find it would fail during image load and print nothing.
466467
#
467-
# THAT OUTPUT PLACES THE FAILURE AFTER THE LOADER, NOT AT IT. The Vulkan
468-
# leg imports `vkCreateInstance` from `vulkan-1.dll` through the import
469-
# library, so a process missing that DLL fails during image load and
470-
# prints nothing at all. It printed. The loader was there, it ran, and it
471-
# enumerated no device -- which is a statement about the ICD.
468+
# The cause was `xim:mesa-lavapipe` writing its rewritten `library_path`
469+
# into the ICD's JSON string unescaped, so the value carried `C:\Users\...`
470+
# and `\U` is not a JSON escape. The loader parses manifests with cJSON and
471+
# silently SKIPS one it cannot parse -- zero devices, no error, and
472+
# indistinguishable from having no GPU. openxlings/xim-pkgindex#781 writes
473+
# forward slashes instead.
472474
#
473-
# An earlier revision of this comment named a missing `vulkan-1.dll` as
474-
# the cause. That was an inference from the descriptor's note about static
475-
# linkage rather than a reading of the failure, and the log refutes it.
476-
# Two facts stand against it: this program ran, and mcpp-index's own
477-
# `vulkan-tests` member calls `vkEnumerateInstanceVersion` on the windows
478-
# shards and passes.
475+
# THE STORE ENTRY AND THE PROVISIONING MARKER BOTH GO FIRST, and that is
476+
# not tidiness. `install()` does not re-run over an existing payload and
477+
# `~/.mcpp` is cached across runs, so a runner that installed 26.2.0
478+
# before the fix would keep the manifest it was given and this step would
479+
# measure the cache.
479480
#
480-
# A separate measurement, recorded here because it removes a second thing
481-
# from the list of suspects: the Khronos loader in `compat:vulkan` builds
482-
# into a working `vulkan-1.dll` from the source the index already carries
483-
# (mingw, 265 exports matching upstream's `vulkan-1.def` exactly, DllMain
484-
# present, importing only ADVAPI32/CFGMGR32/KERNEL32/msvcrt). The
485-
# descriptor's note argues that a Windows loader must be a DLL, not that
486-
# it cannot be built -- so a hermetic Windows loader is available whenever
487-
# it is wanted. It is not what this step is waiting on.
488-
#
489-
# What this step is waiting on is `xim:mesa-lavapipe`'s Windows payload
490-
# producing a device under an mcpp-launched process. Until that is
491-
# measured, the job builds the Vulkan half and runs the CPU fallback.
492-
# macOS is not in the same position: MoltenVK enumerates once the instance
493-
# asks for portability, which `src/vulkan/render.cpp` now does.
481+
# THE MARKER IS THE HALF THAT IS EASY TO FORGET. mcpp records a
482+
# provisioned package set under `<mcpp home>/provisioned/`, as
483+
# `xlings-deps-<hash>`. With the marker present and the payload deleted,
484+
# the build SKIPS provisioning and the payload stays deleted -- measured
485+
# here, where the rebuild printed no `Provisioning` line at all and the
486+
# step failed on its own preparation with `no lavapipe ICD in either
487+
# store`. It is the same pair a developer machine needs when a payload
488+
# changed behaviour without changing version.
489+
- name: "Graphics: the offscreen example RUNS on lavapipe"
490+
shell: bash
491+
run: |
492+
set -e
493+
export MCPP_VENDORED_XLINGS="$XLINGS_BIN"
494+
# Force install() to run again with the corrected hook.
495+
# WRITTEN AS `if`, NOT `[ -d x ] && { ... }`. This step runs under
496+
# `bash -e`, where the exemption for a failing left-hand side of `&&`
497+
# is a rule worth not depending on: an absent directory is the normal
498+
# case here, and it must not end the step.
499+
for store in "${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-mesa-lavapipe" \
500+
"$HOME/.xlings/data/xpkgs/xim-x-mesa-lavapipe"; do
501+
if [ -d "$store" ]; then
502+
echo "removing stale payload $store"
503+
rm -rf "$store"
504+
fi
505+
done
506+
marker="${MCPP_HOME:-$HOME/.mcpp}/provisioned"
507+
if [ -d "$marker" ]; then
508+
echo "removing provisioning markers under $marker"
509+
rm -rf "$marker"
510+
fi
511+
cd "$GITHUB_WORKSPACE/examples/10-graphics/offscreen"
512+
# Rebuilt so the removal above is followed by a provision, and with the
513+
# SAME toolchain the build step named -- an unnamed one re-resolves to
514+
# whatever the default is and rebuilds everything for a reason that has
515+
# nothing to do with this step's subject.
516+
rm -rf target
517+
"$MCPP_SELF" build --toolchain "llvm@20.1.7" 2>&1 | tee build.log
518+
# PROVISIONING HAS TO HAVE RUN. Without this line a build that skipped
519+
# it fails four lines below on "no lavapipe ICD in either store",
520+
# which reads as a missing package rather than as a skipped step.
521+
grep -q "Provisioning \[xlings.workspace\] entries" build.log \
522+
|| { echo "the rebuild did not provision: the marker was not cleared"
523+
exit 1; }
524+
# NOT installed from here. The example declares `xim:mesa-lavapipe`
525+
# under `cfg(windows)`, so the build above provisioned it, and looking
526+
# for the ICD without installing anything is what asserts the
527+
# declaration works.
528+
icd=$(find "${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-mesa-lavapipe" \
529+
"$HOME/.xlings/data/xpkgs/xim-x-mesa-lavapipe" \
530+
-name 'lvp_icd.x86_64.json' -print -quit 2>/dev/null || true)
531+
[ -n "$icd" ] || { echo "no lavapipe ICD in either store"; exit 1; }
532+
echo "ICD: $icd"
533+
# The manifest has to PARSE, and that is the property this whole step
534+
# turns on. Printed before the run, so a failure below is read against
535+
# what the loader was actually handed.
536+
cat "$icd"
537+
python3 -c "import json,sys; json.load(open(sys.argv[1])); print('the ICD manifest parses as JSON')" "$icd"
538+
# BOTH SPELLINGS OF THE SAME ANSWER. `VK_DRIVER_FILES` replaced
539+
# `VK_ICD_FILENAMES` in loader 1.3.234, and a loader older than that
540+
# ignores it entirely and enumerates whatever the registry lists --
541+
# which on a runner with no GPU is nothing. Setting both costs nothing
542+
# and removes a whole class of explanation.
543+
#
544+
# NO `--toolchain`: `mcpp run` has no such option, and passing it exits
545+
# 1 on `unknown option` after every check above has passed, which reads
546+
# as "the device was not reached". The job's earlier step ran
547+
# `toolchain default llvm@20.1.7`, so `run` replays what `build`
548+
# selected.
549+
set +e
550+
out=$(VK_DRIVER_FILES="$icd" VK_ICD_FILENAMES="$icd" "$MCPP_SELF" run 2>&1)
551+
rc=$?
552+
set -e
553+
echo "$out"
554+
if [ $rc -ne 0 ] || ! echo "$out" | grep -q 'llvmpipe'; then
555+
# ASK THE LOADER. `render unavailable` is the program's report that
556+
# it got no device; it is not a reason. The loader will give one,
557+
# and a failure that arrives with it is a failure somebody can act
558+
# on rather than a new hypothesis.
559+
echo "=== the run reached no device; re-running with the loader talking ==="
560+
VK_LOADER_DEBUG=all VK_DRIVER_FILES="$icd" VK_ICD_FILENAMES="$icd" \
561+
"$MCPP_SELF" run 2>&1 | tail -80 || true
562+
exit 1
563+
fi
564+
echo "$out" | grep -qE 'centre pixel: \([0-9]+, [0-9]+, [0-9]+, 255\)' \
565+
|| { echo "no centre pixel was reported"; exit 1; }
566+
echo "ok: lavapipe enumerated and the image was rendered on it"
494567
495568
- name: "Toolchain: LLVM — build mcpp (self-host)"
496569
shell: bash

docs/01-examples.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -32,10 +32,10 @@ examples.
3232
| 07 | [`examples/07-project-subos`](../examples/07-project-subos/) | A build program that finds its tools in the environment the project declared | `[xlings] subos`, `[xlings.workspace]`, a build program whose `PATH` is the environment the project named |
3333
| 08 | [`examples/08-build-rules`](../examples/08-build-rules/) | Two rule packages and a project that uses both | `host-module = true`, `[build-dependencies]`, `mcpp::action` with `role = "check"` |
3434
| 09 | [`examples/09-heterogeneous`](../examples/09-heterogeneous/) | One computation on a device, in several programming models, with a CPU fallback in each; plus one artifact carrying several backends at once | `accel`, constrained source globs, the seam module, rule packages from `mcpp:plugins`, `cfg(accelerator = …)` |
35-
| 09a | [`…/cuda`](../examples/09-heterogeneous/cuda/) | A CUDA kernel behind a seam module | `mcpp.rules.cuda`, `mcpp::action` with `role = "object"`, the driver stated as a fact and a floor |
36-
| 09b | [`…/vulkan`](../examples/09-heterogeneous/vulkan/) | The same computation as a Vulkan compute shader, on a GPU or on the CPU | `mcpp.rules.spirv`, `mcpp::action` with `role = "source"`, generated headers, a software driver as a payload |
37-
| 09c | [`…/sycl`](../examples/09-heterogeneous/sycl/) | The same computation as a SYCL kernel, compiled by a second compiler | `mcpp.rules.sycl`, the `.sycl` device extension, a chained `mcpp::action` for the device link, `compat:sycl-runtime` |
38-
| 09d | [`…/hip`](../examples/09-heterogeneous/hip/) | The same computation in HIP, reaching an NVIDIA device | `mcpp.rules.hip`, HIP as a header layer over the CUDA runtime, a two-chunk `accel` |
35+
| 09a | [`…/cuda`](../examples/09-heterogeneous/cuda/) | A CUDA kernel behind a seam module, with the `extern "C"` boundary GENERATED | `mcpp.rules.cuda`, `mcpp.tools.island`, `mcpp::action` with `role = "object"`, the driver stated as a fact and a floor |
36+
| 09b | [`…/vulkan`](../examples/09-heterogeneous/vulkan/) | The same computation as a Vulkan compute shader, with the SPIR-V payload reached as a MODULE | `mcpp.rules.spirv`, the module surface, `mcpp::action` with `role = "source"`, a software driver as a payload |
37+
| 09c | [`…/sycl`](../examples/09-heterogeneous/sycl/) | The same computation as a SYCL kernel, compiled by a second compiler | `mcpp.rules.sycl`, `mcpp.tools.island`, the `.sycl` device extension, a chained `mcpp::action` for the device link, `compat:sycl-runtime` |
38+
| 09d | [`…/hip`](../examples/09-heterogeneous/hip/) | The same computation in HIP, with the boundary WRITTEN BY HAND — the contrast against 09a | `mcpp.rules.hip`, HIP as a header layer over the CUDA runtime, a two-chunk `accel` |
3939
| 09e | [`…/multi-backend`](../examples/09-heterogeneous/multi-backend/) | Several backends in ONE artifact, chosen at run time — the library shape, not the program shape | `accel` as a set, `cfg(accelerator = "none")` and its negation, a dispatch chain, a module seam over a C island boundary |
4040
| 09f | [`…/cann`](../examples/09-heterogeneous/cann/) | An Ascend C kernel behind the same seam. **Does not build yet** — its README names the two missing pieces | the `.asc` device extension, `op_kernel`/`op_host` as an island CANN already has, `accelerator = "none"` for the fallback |
4141
| 10 | [`examples/10-graphics`](../examples/10-graphics/) | Graphics rather than compute: a rendering pipeline whose result is pixels | `mcpp.rules.spirv` for the vertex and fragment stages, offscreen rendering as the assertable form |

docs/05-mcpp-toml.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2308,7 +2308,7 @@ have none, and nothing is what they write:
23082308

23092309
```toml
23102310
[build-dependencies.mcpp]
2311-
plugins = { version = "0.2.4", features = ["rules-cuda"], host-module = true }
2311+
plugins = { version = "0.3.0", features = ["rules-cuda"], host-module = true }
23122312
```
23132313

23142314
That one edge is the whole declaration. The rule package names the packages its
@@ -2546,7 +2546,7 @@ rules-spirv = { sources = ["rules/spirv.cppm"] } # export module mcpp.rules.spir
25462546
```toml
25472547
# a consumer
25482548
[build-dependencies.mcpp]
2549-
plugins = { version = "0.2.1", features = ["rules-spirv"], host-module = true }
2549+
plugins = { version = "0.3.0", features = ["rules-spirv"], host-module = true }
25502550
```
25512551

25522552
**`[build-dependencies]`, not `[dependencies]`** — a rule package is the case

0 commit comments

Comments
 (0)