Skip to content

Commit ebcbf78

Browse files
committed
build.mcpp: three channels a rule package had no way to reach the engine through
Each of these is small, and they share a shape: a rule package already knew something and nothing carried it to a decision the engine makes. `mcpp::action` gains `depfile`. An action's inputs are fixed when build.mcpp runs, before its command has executed, so a compiler that learns its include graph by parsing the source has no channel to report it: editing a file the command merely READ rebuilds nothing and `mcpp build` stays green over a stale artifact. With a depfile declared, ninja reads the file and folds what it names into the edge. Every device compiler this matters for already emits one -- `glslangValidator --depfile`, `glslc -MD -MF`, `slangc -depfile`, `nvcc` and `clang` `-MD -MF` -- so the missing half was always the receiving end. The depfile must not also be declared as an output: `deps = gcc` makes ninja consume and delete it, so an edge that promised it would be permanently dirty. The comment at the emission site says so. `.slang` joins the device-source extension table. A constrained glob's `accel` key does not make a file a device source; `kDeviceExtensions` does. Until it listed `.slang`, a rule package could not introduce a device language on its own: the payload installed, the rule compiled, and the file still fell through to the ordinary source scan and was refused with "mcpp has no role for the extension". Measured against 2026.9.6.6 with a working rule and a working `xim:slang`. Slang is a language rather than a second driver for GLSL -- its own module system, generics, and targets beyond SPIR-V -- so it has an extension here and a rule of its own outside. `[language] modules` is reported as `MCPP_LANGUAGE_MODULES`. A rule that GENERATES a consumer-facing declaration has to choose between a module interface and a header, and the project has already stated which it builds; deriving it any other way would be a second spelling of one decision. An engine that does not report it leaves the variable absent, which a rule reads as "header" -- the behaviour every consumer had before -- so the upgrade needs no project to declare anything. Tests. `tests/e2e/631` builds a project whose action writes a depfile naming a file it never declares as an input, and asserts the action reruns when that file is touched. It carries its own control: a no-op rebuild must not rerun the action, which rules out a backend that reruns unconditionally. A separate reverse control was run by hand against the same binary with `a.depfile` removed and the flag still passed: the action did not rerun, so the assertion measures the field rather than some other trigger.
1 parent 18ad9d9 commit ebcbf78

16 files changed

Lines changed: 1258 additions & 1 deletion

.agents/docs/2026-09-07-module-first-heterogeneous-surface.md

Lines changed: 806 additions & 0 deletions
Large diffs are not rendered by default.

CHANGELOG.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,33 @@
77

88
## [2026.9.7.1] - 2026-09-07
99

10+
### 三条通道,都是「规则包知道而引擎收不到」的形状
11+
12+
这一版加的三样东西各自很小,共同点是它们补的都是同一类缺口:规则包已经知道某件事,
13+
而没有任何通道把它送到引擎的决定上。
14+
15+
**`mcpp::action` 增加 `depfile`** action 的输入在 `build.mcpp` 运行时就定死了,那时
16+
命令还没执行。一个靠解析源码才知道自己 `#include` 图的编译器,因此没有办法把结果报回
17+
来:改动一个命令只是****过的文件不会触发任何重建,而 `mcpp build` 会在一个陈旧产物上
18+
保持绿色。声明了 depfile 之后,mcpp 为那条边写出 `depfile =``deps = gcc`,ninja
19+
读取并把它列出的文件并入这条边的依赖。相关的每个设备编译器本来就能输出它 ——
20+
`glslangValidator --depfile``glslc -MD -MF``slangc -depfile``nvcc`/`clang`
21+
`-MD -MF` —— 缺的一直是接收端。
22+
23+
**不要同时把 depfile 声明为 `output()`**:`deps = gcc` 会让 ninja 读完即删,一条承诺了
24+
该输出的边会永远是脏的。
25+
26+
**`.slang` 进入设备源扩展名表。** 受限 glob 的 `accel`****决定一个文件是不是设备
27+
源,`kDeviceExtensions` 才决定。所以在此之前,一个规则包无法自己引入一门设备语言:
28+
`xim:slang` 装得上、规则也编得过,而文件仍然掉进普通源集,报的是「mcpp has no role for
29+
the extension '.slang'」。Slang 是一门语言而不是 GLSL 的第二个驱动 —— 它有自己的模块
30+
系统、泛型,以及超出 SPIR-V 的目标集合 —— 所以它有自己的扩展名,外面也有自己的规则。
31+
32+
**`[language] modules``MCPP_LANGUAGE_MODULES` 报给构建程序。** **生成**面向消费者
33+
声明的规则要在「模块接口」与「头文件」之间做选择,而项目已经声明过它用哪一种;用别的
34+
方式推导就是同一个决定的第二种拼法。旧引擎不设这个变量,规则把缺席读作「头文件」——
35+
也就是这个变量存在之前每个消费者的行为,所以升级不需要任何项目改一个字。
36+
1037
### 两处只在 Linux 之外成立的缺陷,以及第一条图形管线
1138

1239
**引擎与规则层早就与平台无关,而生态只在 Linux 上完整。** 把规则包在另外两个平台上

docs/07-build-mcpp.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -470,6 +470,27 @@ package's compile edges to that package's action outputs.
470470
> package**, because `include_dir` colours only the declaring package's own
471471
> translation units.
472472
473+
**An action whose command discovers its own dependencies declares a depfile**
474+
(mcpp 2026.9.7.1+). `input()` fixes the edge's inputs when `build.mcpp` runs,
475+
before the command has executed, so a compiler that learns its `#include` graph
476+
by parsing the source has no channel to report it — and editing a file the
477+
command merely *read* reruns nothing, leaving `mcpp build` green over a stale
478+
artifact.
479+
480+
```cpp
481+
a.depfile = dep.c_str(); // a path the command writes
482+
a.arg("--depfile").arg(dep.c_str());
483+
```
484+
485+
mcpp emits `depfile =` and `deps = gcc` for that edge, so ninja reads the file
486+
and folds what it names into the edge's dependencies. Every device compiler this
487+
matters for already emits one: `glslangValidator --depfile`, `glslc -MD -MF`,
488+
`slangc -depfile`, `nvcc`/`clang` `-MD -MF`.
489+
490+
> **Do not also declare the depfile as an `output()`.** `deps = gcc` makes ninja
491+
> consume and delete it after reading, so an edge that promised it as an output
492+
> would be permanently dirty.
493+
473494
**A check's command does not have to write its stamp** (mcpp 2026.8.29.1+).
474495
The verdict is the exit code; the stamp is bookkeeping the graph needs, and
475496
mcpp creates it when the command succeeds. Before this, every check needed a
@@ -637,6 +658,7 @@ The running program receives the build context as `MCPP_*` variables
637658
| `MCPP_TOOLCHAIN_BINUTILS_DIR` *(2026.9.5.2+)* | `mcpp::toolchain_binutils_dir()` | the directory mcpp names with `-B`; empty when it names none (a musl or MinGW payload brings its own assembler and linker) |
638659
| `MCPP_CXX_STDLIB` *(2026.9.6.3+)* | `mcpp::cxx_stdlib()` | the C++ standard library the resolved toolchain uses — `libstdc++`, `libc++`, `msvc-stl`; empty when no toolchain resolved. A different question from `MCPP_TARGET_LIBC`, which is the C library |
639660
| `MCPP_ACCEL` *(2026.9.5.2+)* | `mcpp::accel()` | the device axis of this build, resolved — `--accel` / `--no-accel` over `[build] accel` — in the wire form `cuda12.9+{sm_89} ptx>=89`; empty when the build asks for no accelerator. A rule package derives its own flags (`-gencode`, `--offload-arch`) from it, so the architecture set is written once, in the manifest. The same value feeds the `cfg(accelerator = "…")` layer key |
661+
| `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | `1` when the declaring package sets `[language] modules`, `0` otherwise. A rule that GENERATES a consumer-facing declaration reads it to choose between a module interface and a header, so a project states that once and never again. An older engine leaves it absent, which a rule reads as `0` -- the behaviour every consumer had before the variable existed |
640662
| `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | the device-kind sources (`.cu`, `.hip`, …) the package's effective `sources` match, package-root-relative, one per line; empty when there are none. The engine compiles none of them — the rule package this program imports turns each into an `mcpp::action`. Already narrowed: a `{ glob, accel }` entry the build does not cover contributes nothing, so `--no-accel` yields an empty list |
641663
| `MCPP_OUT_DIR` | `mcpp::out_dir()` | a writable scratch/output dir owned by mcpp |
642664
| `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | the package root (= CWD) |

docs/20-heterogeneous-builds.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ does not accept C++20 modules.
9393
| CUDA, HIP | `.cu`, `.hip` |
9494
| SYCL | `.sycl` (2026.9.6.1+) |
9595
| Ascend C | `.asc`, `.cce` (2026.9.6.5+) |
96+
| Slang | `.slang` (2026.9.7.1+) |
9697
| GLSL, by stage | `.comp`, `.vert`, `.frag`, `.geom`, `.tesc`, `.tese`, `.mesh`, `.task`, `.rgen`, `.rint`, `.rahit`, `.rchit`, `.rmiss`, `.rcall` |
9798
| GLSL, stage-less | `.glsl` |
9899
| HLSL | `.hlsl` |
@@ -485,6 +486,7 @@ own denominator.
485486
| `rules-hip` | `mcpp.rules.hip` | the project's own clang (`-x cuda`) on the NVIDIA platform | the above plus `xim:hip-nvidia` | `hip, cuda12.9+{sm_89}` |
486487
| `rules-sycl` | `mcpp.rules.sycl` | the `xim:dpcpp` payload's clang (`-fsycl`) | `xim:dpcpp`; on Linux also `xim:gcc`, `xim:glibc`, `xim:linux-headers`; `xim:cuda-nvcc` for an NVIDIA target | `sycl` or `sycl, cuda12.9+{sm_89}` |
487488
| `rules-spirv` | `mcpp.rules.spirv` | `glslangValidator` or `glslc` | `xim:glslang` on Linux, `xim:shaderc` on macOS and Windows | `vulkan1.2` |
489+
| `rules-slang` | `mcpp.rules.slang` | `slangc` | `xim:slang` | `vulkan1.2` |
488490
| `rules-ascendc` | `mcpp.rules.ascendc` | `bisheng` (`-x asc`) from the CANN toolkit | `xim:cann-toolkit` | `ascend8.5+{dav-c220}` |
489491

490492
The payload column is what each rule declares for itself under

docs/zh/07-build-mcpp.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -401,6 +401,23 @@ int main() {
401401
> 任何边消费它 —— 于是它从不执行,而编译器读到的是 mcpp 为已声明产物写下的那个空占位
402402
> 文件。这条边**按包**划分,因为 `include_dir` 只染色声明它的那个包自己的 TU。
403403
404+
**命令自己发现依赖的 action 要声明 depfile**(mcpp 2026.9.7.1+)。`input()`
405+
`build.mcpp` 运行时就把边的输入定死了,而那时命令还没执行,所以一个靠解析源码才知道自己
406+
`#include` 图的编译器没有任何通道把结果报回来 —— 改动一个命令只是****过的文件不会触发
407+
任何重建,`mcpp build` 会在一个陈旧产物上保持绿色。
408+
409+
```cpp
410+
a.depfile = dep.c_str(); // 命令会写出的路径
411+
a.arg("--depfile").arg(dep.c_str());
412+
```
413+
414+
mcpp 为那条边写出 `depfile =``deps = gcc`,ninja 读取该文件并把它列出的文件并入这条边
415+
的依赖。与此相关的每个设备编译器都已经能输出它:`glslangValidator --depfile`
416+
`glslc -MD -MF``slangc -depfile``nvcc`/`clang``-MD -MF`
417+
418+
> **不要同时把 depfile 声明为 `output()`** `deps = gcc` 会让 ninja 读完即删,所以一条
419+
> 承诺了该输出的边会永远是脏的。
420+
404421
**check 的命令不必自己写 stamp**(mcpp 2026.8.29.1+)。判定是退出码,stamp 是**构建图**
405422
需要的记账;命令成功时由 mcpp 创建它。在此之前每个 check 都需要一个包装脚本去 touch
406423
那个文件 —— 而 command 是 argv、不假设有 shell,所以那个包装器**根本没法可移植地写出来**
@@ -545,6 +562,7 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存
545562
| `MCPP_TOOLCHAIN_BINUTILS_DIR` *(2026.9.5.2+)* | `mcpp::toolchain_binutils_dir()` | mcpp 用 `-B` 指的目录;不指时为空串(musl 与 MinGW 载荷自带汇编器与链接器) |
546563
| `MCPP_CXX_STDLIB` *(2026.9.6.3+)* | `mcpp::cxx_stdlib()` | 解析出的工具链使用的 C++ 标准库 —— `libstdc++``libc++``msvc-stl`;没有工具链解析时为空串。与 `MCPP_TARGET_LIBC` 不是同一个问题,后者是 C 库 |
547564
| `MCPP_ACCEL` *(2026.9.5.2+)* | `mcpp::accel()` | 本次构建的设备轴,已解析 —— `--accel` / `--no-accel` 优先于 `[build] accel` —— 线上形态 `cuda12.9+{sm_89} ptx>=89`;不要加速器时为空串。规则包从它推导自己的开关(`-gencode``--offload-arch`),架构集合因此只在 manifest 写一次。同一个值也喂给 `cfg(accelerator = "…")` 这个 layer 键 |
565+
| `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | 声明它的那个包设了 `[language] modules` 时为 `1`,否则 `0`**生成**面向消费者声明的规则读它来在模块接口与头文件之间选择,项目因此只需说一次。旧引擎不设这个变量,规则把缺席读作 `0` —— 也就是这个变量存在之前每个消费者的行为 |
548566
| `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | 本包有效 `sources` 匹配到的设备类源文件(`.cu``.hip`…),相对包根,一行一个;没有时为空串。引擎一个都不编译 —— 由本程序引入的规则包把每一个变成一条 `mcpp::action`。已经过收窄:构建未覆盖的 `{ glob, accel }` 条目贡献为空,因此 `--no-accel` 得到空列表 |
549567
| `MCPP_OUT_DIR` | `mcpp::out_dir()` | mcpp 提供的可写输出/暂存目录 |
550568
| `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | 包根(= CWD) |

docs/zh/20-heterogeneous-builds.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ C++20 modules 的编译器。
7575
| CUDA、HIP | `.cu``.hip` |
7676
| SYCL | `.sycl`(2026.9.6.1+) |
7777
| Ascend C | `.asc``.cce`(2026.9.6.5+) |
78+
| Slang | `.slang`(2026.9.7.1+) |
7879
| GLSL(按 stage) | `.comp``.vert``.frag``.geom``.tesc``.tese``.mesh``.task``.rgen``.rint``.rahit``.rchit``.rmiss``.rcall` |
7980
| GLSL(无 stage) | `.glsl` |
8081
| HLSL | `.hlsl` |
@@ -405,6 +406,7 @@ sources = ["src/cpu/*.cpp"]
405406
| `rules-hip` | `mcpp.rules.hip` | NVIDIA 平台上是工程自己的 clang(`-x cuda`) | 上面那些,再加 `xim:hip-nvidia` | `hip, cuda12.9+{sm_89}` |
406407
| `rules-sycl` | `mcpp.rules.sycl` | `xim:dpcpp` 载荷里的 clang(`-fsycl`) | `xim:dpcpp`;Linux 上另有 `xim:gcc``xim:glibc``xim:linux-headers`;NVIDIA 目标另加 `xim:cuda-nvcc` | `sycl``sycl, cuda12.9+{sm_89}` |
407408
| `rules-spirv` | `mcpp.rules.spirv` | `glslangValidator``glslc` | Linux 上 `xim:glslang`,macOS 与 Windows 上 `xim:shaderc` | `vulkan1.2` |
409+
| `rules-slang` | `mcpp.rules.slang` | `slangc` | `xim:slang` | `vulkan1.2` |
408410
| `rules-ascendc` | `mcpp.rules.ascendc` | CANN 工具包里的 `bisheng`(`-x asc`) | `xim:cann-toolkit` | `ascend8.5+{dav-c220}` |
409411

410412
载荷那一列是每条规则在 `cfg(accelerator = ...)` 之下**为自己**声明的东西,列出来是为了

modules/buildmcpp/src/directives.cppm

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -842,6 +842,7 @@ std::optional<mcpp::manifest::BuildAction> decode_action(std::string_view payloa
842842
arr("imports", a.imports);
843843
arr("targets", a.targets);
844844
a.blocking = j.value("blocking", false);
845+
a.depfile = j.value("depfile", std::string{});
845846
a.description = j.value("description", std::string{});
846847
if (a.command.empty() || a.outputs.empty()) return std::nullopt;
847848
if (a.id.empty()) a.id = a.outputs.front();
@@ -909,6 +910,14 @@ void prepare_actions(std::vector<mcpp::manifest::BuildAction>& actions,
909910
};
910911
absolutize(a.inputs);
911912
absolutize(a.outputs);
913+
// The depfile is an OUTPUT-SIDE path — the command WRITES it, ninja
914+
// reads it back — so it needs the identical anchoring `outputs` gets,
915+
// for every role, not just Source: left package-relative it would be
916+
// interpreted relative to the ninja build directory instead of the
917+
// package root, and the `depfile = ` line the backend emits would
918+
// simply never match the file the command actually wrote.
919+
if (!a.depfile.empty() && a.depfile.find("${mcpp.") == std::string::npos)
920+
a.depfile = abs_against(pkgRoot, a.depfile);
912921
if (a.role != mcpp::manifest::BuildAction::Role::Source) continue;
913922
for (auto const& o : a.outputs) {
914923
if (o.find("${mcpp.") != std::string::npos) continue;

modules/manifest/src/types.cppm

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -392,6 +392,30 @@ struct BuildAction {
392392
std::vector<std::string> imports;
393393
// Check only: make compilation wait for this to pass. Off by default.
394394
bool blocking = false;
395+
// A Make-style dependency file the action's COMMAND writes as a side
396+
// effect — ninja reads it once the command exits and folds it into its
397+
// own dependency log, the same `deps = gcc` mechanism a `cxx_object` edge
398+
// uses for a compiler's own `#include` graph (mcpp#235/#257).
399+
//
400+
// `inputs` cannot express what this covers, because it is FIXED AT
401+
// SUBMISSION — build.mcpp declares it before anything has been compiled.
402+
// A device-shader compiler discovers its own `#include` graph only by
403+
// parsing the shader (glslangValidator `--depfile`, glslc `-MD -MF`,
404+
// slangc `-depfile`, nvcc/clang `-MD -MF`), which is not knowable until
405+
// the action's command actually runs. Without this field a build stayed
406+
// green over a stale artifact: editing an included `.glsl`/`.cuh` changed
407+
// nothing the action had declared as an input, so nothing reran.
408+
//
409+
// Empty (the default) means the rule emits no depfile, and the action's
410+
// re-run set is exactly its declared `inputs` — unchanged from before this
411+
// field existed.
412+
//
413+
// MUST NOT also appear in `outputs`. `deps = gcc` makes ninja consume and
414+
// DELETE the depfile once it has read it (see the `rule mcpp_action_{i}`
415+
// emission in src/build/ninja_backend.cppm); a path that is simultaneously
416+
// a declared ninja OUTPUT of the same edge would be a file ninja expects
417+
// to still exist after a successful build and has itself just removed.
418+
std::string depfile;
395419
std::string description;
396420
};
397421

modules/source-kind/src/source_kind.cppm

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -271,8 +271,20 @@ constexpr std::string_view kHeaderExtensions[] = { ".h", ".hpp", ".hh", ".hxx" }
271271
// `op_kernel/` from `op_host/`, and CMake registers ASC as a LANGUAGE of its
272272
// own -- so the island is the shape Ascend already has, not one mcpp imposes.
273273
// `.cce` is the older spelling of the same thing and is accepted beside it.
274+
//
275+
// `.slang` is the Slang shading language, compiled by `slangc`. It is a
276+
// LANGUAGE rather than a second driver for GLSL -- its own module system,
277+
// generics, and a target set beyond SPIR-V -- which is why it has an extension
278+
// of its own here and a rule of its own outside.
279+
//
280+
// THIS TABLE IS WHAT DECIDES, NOT THE GLOB'S `accel` KEY. A constrained glob
281+
// carrying `accel = "vulkan1.2"` does not make a file a device source; this
282+
// list does, and a file whose extension is absent from it reaches the ordinary
283+
// source scan and is refused with "mcpp has no role for the extension". A rule
284+
// package therefore cannot introduce a device language on its own, and adding
285+
// one here is the engine half of doing so.
274286
constexpr std::string_view kDeviceExtensions[] = {
275-
".cu", ".hip", ".sycl", ".asc", ".cce",
287+
".cu", ".hip", ".sycl", ".asc", ".cce", ".slang",
276288
".comp", ".vert", ".frag", ".geom", ".tesc", ".tese", ".mesh", ".task",
277289
".rgen", ".rint", ".rahit", ".rchit", ".rmiss", ".rcall",
278290
".glsl", ".hlsl", ".cl", ".metal",

src/build/build_program.cppm

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,16 @@ struct BuildProgramEnv {
124124
// (`-gencode`, `--offload-arch`) from here and the architecture set is
125125
// written once, in the manifest, and never again in a build program.
126126
std::string accel;
127+
// Whether this package builds C++ modules (`[language] modules`).
128+
//
129+
// Reported because a rule package that GENERATES a consumer-facing
130+
// declaration has to choose between a module interface and a header, and
131+
// the project has already stated which it uses. Deriving it any other way
132+
// would be a second spelling of one decision. A rule that reads it can make
133+
// the module surface its default without any project declaring anything,
134+
// and an engine older than this one leaves the variable absent -- which a
135+
// rule reads as "header", the behaviour every consumer had before.
136+
bool languageModules = true;
127137
// The device-kind sources (`.cu`, `.hip`, ...) this package's effective
128138
// source set matches, package-root-relative with `/` separators, one per
129139
// line. The engine has no compile rule for them and hands the list to the
@@ -494,6 +504,7 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv
494504
e.emplace_back("MCPP_TARGET_LIBC", env.targetLibc);
495505
e.emplace_back("MCPP_PROFILE", env.profile);
496506
e.emplace_back("MCPP_ACCEL", env.accel);
507+
e.emplace_back("MCPP_LANGUAGE_MODULES", env.languageModules ? "1" : "0");
497508
{
498509
std::string joined;
499510
for (auto const& d : env.deviceSources) {

0 commit comments

Comments
 (0)