diff --git a/.agents/docs/2026-09-07-package-identity-and-doc-alignment.md b/.agents/docs/2026-09-07-package-identity-and-doc-alignment.md new file mode 100644 index 00000000..295cb8c2 --- /dev/null +++ b/.agents/docs/2026-09-07-package-identity-and-doc-alignment.md @@ -0,0 +1,368 @@ +# 一个包一个版本:xlings 地址的身份,以及 2026.9.6.5 之后的文档对齐 + +> 状态:**已实现**(mcpp 2026.9.6.6 / mcpp-plugins 0.2.4)。§10 是实现期的实测回填, +> 其中三条推翻了本文正文。原文保留,不回改——被推翻的判断本身是这份记录的一部分。本文的每一条「现状」都标注了它是**读**出来的还是**跑**出来的。 +> 伴随文档:`2026-09-07-general-build-infrastructure-gaps-design.md`(§15 记录了本轮 +> 五处缺口),本文处理它遗留的第六处,以及随之而来的文档与生态对齐。 + +## 0. 这份方案要解决的一句话 + +**版本在一个地方属于包的身份,在另一个地方不属于。** 于是同一个包被两处以不同版本声明 +时,系统认为那是两个包,把两份都装下来。 + +## 1. 现状:版本语义已经在了,缺的是身份的一致 + +### 1.1 版本语义已经可用(**实测**,两个方向) + +`[xlings.workspace]` 的版本位接受范围表达式,并且**真的被求解**: + +| manifest | 读数 | +|---|---| +| `"xim:shaderc" = ">=2026.1"` | `Provisioning [xlings.workspace] entries (xim:shaderc@>=2026.1)`,构建通过 | +| `"xim:shaderc" = ">=2099.1"` | 被拒绝:`package 'xim:shaderc@>=2099.1' not found in the synced index` | + +反向那条是判据的关键:只跑正向,一个把范围当字面量放过去的实现同样会「通过」。 + +索引侧也早在用同一套写法 —— `xim:libX11@>=1.8`、`xim:glibc@>=2.38` 出现在多个 recipe +的 `deps` 里。 + +**结论:不需要引入版本语义。** 「钉具体版本还是钉范围」已经是作者的选择。 + +### 1.2 但身份有两套定义(**读**代码) + +| 站点 | 身份是什么 | 冲突时 | +|---|---|---| +| `merge_conditional_config`(prepare.cppm:264) | `package_of()` → `(ns, name)`,**不含版本** | 更具体的赢,并发 `xlings/axis-override` 告警说明用了哪条 | +| `graph_xlings_split` | `find(rootSpecs, spec)` → **整个地址串** | 不同串 = 不同包,两份都进供给列表 | + +于是工程钉 `xim:cuda-nvcc@13.3.33`、规则包钉 `@12.9.86` 时,两份都装 —— 每份 2–3 GB。 +而 `fillXpkgDirs` 里「同名保留第一个值」的规则让 `xpkg_dir` 答工程那份,所以**装了两份、 +用了一份**,且没有任何一句话提到这件事。 + +正确的那套语义已经写好了,只是没有用在第二个站点上。 + +## 2. 修法:一条规则,两条配套,不需要求解器 + +### 2.1 规则 + +> **包的身份是 `(namespace, name)`。版本永远是**约束**,不是身份的一部分。** + +这一条同时给出两件被要求的性质: + +- 「所有地方都支持版本语义」—— 因为不再有第二种身份,就没有第二处需要单独支持; +- 「钉具体版本还是钉范围由开发者选」—— 因为两者都只是约束,系统不区分。 + +它**减少**机制:今天有两套身份定义,统一后剩一套。 + +### 2.2 裁决:离消费者更近的赢,并且说出来 + +同一身份上有多条声明时,**离消费者更近的那条赢**:工程 > 依赖。同一 manifest 内则是 +**更具体的赢**:`[target.]` > 顶层。 + +这是两条不同的序,不该用一个词盖住:「更具体」说的是谓词的窄窄,「更近」说的是声明者 +与最终产物的距离。后者是新的一条,前者 `merge_conditional_config` 已经在做,并且已经有 +`xlings/axis-override` 负责「说出来」。跨包这一趟复用同一条告警。 + +**两个依赖互相冲突**(规则 A 要 `>=8.5`、规则 B 要 `<8.5`,工程什么都没写)时没有「更近」 +可言,那种情形直接走 §2.3 的拒绝,消息点出两个包各自的约束。 + +### 2.3 校验:赢家必须满足其余每一条约束 + +这是**唯一新增**的一条,也是让「不需要求解器」成立的那条。 + +选版本不需要求交 —— 裁决说了算。求交只在「赢家违反了输家的约束」时才有意义,而那种情形 +正确的动作是**拒绝并点名两边**,不是悄悄选一个: + +``` +error: `xim:cann-toolkit` is pinned to 8.3.RC1 by this project, and + `mcpp:plugins` (feature `rules-ascendc`) requires >=8.5.0. + fix: raise the pin, or drop it and let the rule's constraint decide. +``` + +所以是「验证一个已选定的版本」,不是「在约束集合里搜索」。前者是一次比较,后者要知道 +索引里有哪些版本可选。 + +### 2.3.1 这条捷径的已知代价,不能不写 + +不求交意味着**系统会拒绝一些求解器本可以满足的组合**。工程写 `>=8.0`、规则写 `8.5.0`, +而索引里最新是 8.3:裁决让工程赢,`>=8.0` 解析到 8.3,校验发现它不满足规则的 `8.5.0`, +于是拒绝 —— 尽管 8.5.0 本身同时满足两边。 + +这是有意接受的代价,理由有三条:拒绝的消息里已经写着确切的出路(「抬 pin,或者去掉 pin +让规则的约束决定」);求解器要引入「可选版本集合」这个此处还不存在的输入;而这种组合 +(一边范围、一边精确、且交集不在范围的最大值上)在实践中是少数。 + +**如果它变成多数,那就是引入求交的信号** —— 而不是现在。 + +### 2.4 为什么这比「按包去重」更准确 + +我在讨论中先把出路列成三选一(不钉版本 / 引擎按包去重 / 维持现状)。**那个划分是错的**: +身份统一之后,「不钉版本」只是约束的一种取值,不需要单独成为一条路。三条塌缩成一条。 + +## 3. 影响面:三个站点必须一起改 + +单独改 `graph_xlings_split` 会造出本轮 §15.5 那种形状的缺陷 —— 装的是 A,`xpkg_dir` +答的是 B。三处共同定义了「哪个版本被安装、哪个版本被回答」: + +1. `graph_xlings_split` —— 去重与 root/graph 的划分 +2. `fillXpkgDirs` —— 「同名保留第一个值」,决定 `xpkg_dir` 答谁 +3. `provision_xlings_addresses` —— 实际收到的地址列表 + +判据必须同时观察「装了什么」和「答了什么」,只看其中一个会漏掉这类缺陷。 + +## 4. 判据 + +| # | 构造 | 断言 | 今天 | +|---|---|---|---| +| C1 | 工程与其规则包钉同一包的**不同版本** | store 里只出现**一个**版本目录 | 红 | +| C2 | 同上 | `xpkg_dir` 答的就是那个目录 | 绿(但与 C1 一起看才有意义) | +| C3 | 同上 | 输出里有一句说明谁赢了 | 红 | +| C4 | 工程钉的版本**不满足**规则的约束 | 拒绝,消息同时点出两边与各自的约束 | 红 | +| C5 | 只有规则声明(工程什么都不写) | 装规则要求的版本,构建通过 | 需实测 | +| C6 | 范围不可满足 | 拒绝 | 绿(§1.1 已测) | +| C7 | 工程只写 `[build-dependencies]` 那一行,不写任何 `[xlings.workspace]` | 规则声明的 toolkit 被装上,构建通过 | 需实测(§6.1) | +| C8 | 同上,再加工程自己的钉 | 只装工程钉的那一版,且构建通过 | 红(即 C1,带真实 payload) | + +C1 与 C4 是这份方案存在的理由;C6 是既有行为,列出来是为了防止改动把它弄坏。 + +## 5. 文档对齐:实测出来的清单 + +以下每一条都用 grep 在树上核过,不是回忆。 + +### 5.1 明确过期,必须改 + +| 文件 | 问题 | +|---|---| +| `docs/specs/manifest-semantics.md` §4.3.1 | 仍写「`` **禁止**命名五个目标侧层(`accelerator`、`c-abi`…)」。2026.9.6.5 起 `accelerator` **被接受**。且该句自身不自洽:说「五个」而括号里列了六个 | +| `docs/11-machine-output.md` | 本轮三处新拒绝(设备源无 action、未声明的后端、缺 host module)**都没有 refusal code** —— 实测 `refusal::record` 在三处附近均零次出现,而 `build_program.cppm` **已经 import 了 `mcpp.build.refusal`**,所以这是遗漏不是能力缺失。机器消费者分不开它们,而这份文档正是拒绝词表的规范 | +| `examples/09-heterogeneous/README.md` | 表格只列四个目录,`multi-backend/` 与 `cann/` 缺席;结尾「Start with `cuda/`. The other three assume it.」在六个目录下已不成立 | + +### 5.2 本轮已改,列出以便复查 + +`docs/05-mcpp-toml.md` / zh(`exports`、`accelerator` 谓词键、accel 门控 xlings 与 +dependencies、build-dependencies 的更正)、`docs/07-build-mcpp.md` / zh(`link-flag`、 +设备源必须到达 action、规则只取自己认领的扩展名、未提供的 import 被点名拒绝)、 +`docs/20-heterogeneous-builds.md` / zh(`accelerator = "none"` 进词表、CPU 回退改用 +`none`、`.asc`/`.cce` 进扩展名表、rules-ascendc 进 lane 表、四条 lane → 五条)。 + +### 5.3 不需要改 + +`docs/14-target-side.md` 的「五个层」指的就是被解析的那五个,从来不含 `accelerator` —— +与本轮的日程划分**恰好一致**。带 `(2026.9.6.x+)` 的行是历史标记,保持原样。 + +## 6. 生态侧:每个规则自带它的环境 + +### 6.1 规则拥有默认,工程拥有例外 + +这是本方案在用户侧的**主要**收益,比 §1.2 的「装两份」更常被碰到。 + +今天一个要编 CUDA 核的工程写两块:一条 `[build-dependencies]` 边选规则,再加四行 +`[xlings.workspace]` 把 toolkit、cudart、libcurand、cccl 各自钉一遍。第二块是**规则 +已经知道的事**——哪个包、最低哪一版,由写规则的人决定,而不是由用它的人重复。 + +改成: + +```toml +# rules/cuda.cppm 所在的包,mcpp-plugins +[target.'cfg(accelerator = "cuda")'.feature-xlings.rules-cuda] +"xim:cuda-nvcc" = ">=12.9.86" +"xim:cuda-cudart" = ">=12.9.79" +"xim:libcurand" = ">=10.3.10.19" +"xim:cuda-cccl" = ">=12.9.27" +``` + +两重门:feature 说「要不要这个规则」,selector 说「哪些构建真的下载」。CPU-only 构建 +两道门都不过,一个字节都不装。 + +于是工程侧只剩一行: + +```toml +[build-dependencies.mcpp] +plugins = { version = "0.2.4", features = ["rules-cuda"], host-module = true } +``` + +五个规则一起改,不留一个例外:`rules-cuda`、`rules-hip`、`rules-sycl`、`rules-spirv`、 +`rules-ascendc`。**前一稿写「CUDA 暂不动」是错的** —— 留一个例外就等于告诉用户「有时要 +自己写、有时不要」,而「什么时候要」没有任何地方说得清。理由那一稿写的是「12.9 与 13.x +耦合驱动下界,那个决定属于工程」;可决定属于工程**不等于**默认属于工程 —— 下界写成 +`>=12.9.86` 就把「至少要这么新」交给规则,把「具体哪一版」留给工程,两件事各归各位。 + +### 6.2 例外怎么写,以及为什么它现在才安全 + +工程仍然可以钉: + +```toml +[target.'cfg(accelerator = "cuda")'.xlings.workspace] +"xim:cuda-nvcc" = "13.3.33" +``` + +**这条覆盖之所以能用,正是 §2 的裁决与校验。** 没有身份统一,它不是覆盖而是第二份安装: +`xim:cuda-nvcc@13.3.33` 与 `xim:cuda-nvcc@>=12.9.86` 是两个串、两条记录、两份 2 GB 的 +payload,而 `xpkg_dir` 只答其中一个。所以 §6 不是 §2 的下游应用,它是 §2 **成立的判据** +——一个用户会走的、今天会坏的路径。 + +顺序因此是固定的:引擎先落地,插件再发。反过来做,插件的默认与工程的钉一起装两份。 + +### 6.3 文档要说的一句话 + +`docs/20` 与 `docs/05` 各加一段,说清三件事:默认由规则带,不需要写;要换版本就在工程里 +写同名条目,离消费者近的赢;赢家不满足规则的下界时构建被拒绝并点名两边。 + +### 6.4 examples + +- `cann/app`、`cuda/`、`sycl/`、`hip/`、`vulkan/`:删掉各自的 `[xlings.workspace]` + 加速器块。这是最直接的读数——**示例的行数就是用户要写的行数**。 +- `multi-backend`:**保留** `xim:cuda-nvcc = "12.9.86"` 一行,并把注释改成「这是覆盖 + 的示范」。整个生态需要恰好一个地方示范例外路径,多一个都是噪声。 +- `09-heterogeneous/README.md`:补 `multi-backend/` 与 `cann/` 两行,改结尾那句。 + +### 6.5 索引 + +无改动。`xim:cann-toolkit` 与 CUDA/SYCL 各包都已发布;规则包改用 feature-xlings 只需 +plugins 的一次小版本。 + +## 7. mcpp 依赖的 xlings pin + +`src/xlings/xlings.cppm` 的 `pinned::kXlingsVersion` 从 `2026.8.30.2` 抬到 +`2026.9.5.1`(当前最新)。这个常量是 `.github/tools/check_version_pins.sh` 的**唯一 +真源**:该脚本把 `.github/` 下每一处 xlings 版本与它比对,不等就红。所以这一处改动是 +一个常量加脚本枚举出来的那几处工作流,不是一次搜索替换 —— 判据是脚本本身通过。 + +它与本方案的关系是直接的:`[xlings.workspace]` 的范围求解由 xlings 侧执行,而 §2 的 +校验建立在「范围真的被求解」之上(§1.1 的两个方向就是在验这件事)。停在三个月前的 +pin 让这条依赖变成一个没人核过的假设。 + +## 8. 分期与依赖 + +| 期 | 内容 | 依赖 | +|---|---|---| +| 一 | §5.1 三处文档 + 三处新拒绝补 refusal code + §7 的 pin | 无 | +| 二 | §2 的身份统一(三个站点)+ C1/C3/C4/C5 | 一期的 refusal code(C4 要一个码) | +| 三 | §6.1 五个规则自带环境 + §6.4 示例瘦身 | 二期**必须先发布** | + +三期跨仓库且顺序不可交换:插件的默认要靠引擎的裁决才不会变成第二份安装。 + +## 9. 自我 review:讨论过程中被推翻的四处判断 + +| 我先说的 | 为什么错 | +|---|---| +| 「要不要引入版本语义」 | 语义已经在,实测两个方向。问题从头就不是这个 | +| 出路是三选一(不钉版本 / 按包去重 / 维持现状) | 身份统一之后三条塌缩成一条:「不钉版本」只是约束的一种取值 | +| 需要一个版本求解器 | 过重。选版本由裁决决定,求交只在**校验**时需要,而校验是一次比较不是搜索 | +| 「CUDA 暂不动,那个决定属于工程」 | 混淆了**决定权**与**默认值**。下界属于规则、具体版本属于工程,两者不冲突;留一个例外反而让「什么时候要自己写」无处可查(§6.1) | + +前三处有同一个形状:**在没有把已有机制查清楚之前就开始设计新机制。** §1.1 那两行实测 +是十几秒的事,而它把整份方案的规模从「引入约束求解」缩到「统一一处身份定义」。 + +第四处形状不同:**我把「这里有真实的复杂度」读成了「这里应该保持现状」。** 12.9 与 +13.x 的驱动下界差异是真的,但它论证的是「工程要能覆盖」,而我拿它论证了「规则不该有 +默认」——同一个事实支持的是相反的结论。 + +## 10. 实现期的实测回填 + +写下来的四条里,有三条改了正文的结论。 + +### 10.1 `xpkg_dir` 根本回答不了范围 —— 方案漏了这条 + +§1.1 只测了「范围被**供给**」,没测「范围被**回答**」。实现时才发现 +`xpkg_payload_at` 把整个版本位当目录名比对:`>=8.5.0` 装上了载荷,然后回答 +`nullopt`。 + +**这是让规则包无法声明下界的那道缝,而整份 §6 建立在它之上。** 少了这一处修复, +`[feature-xlings]` 里写 `>=` 的规则会「声明、装上、然后找不到」。 + +形状是熟悉的一种:**判据只走了一半的路**。供给与查询是同一条链的两端,而 §1.1 的两个 +方向都落在供给那一端 —— 反向腿(`>=2099.1` 被拒)证明了范围被求解,却完全不涉及求解 +之后谁去读它。 + +修法:版本位是常量就先按目录名精确匹配(保留「钉了就是钉了」,也保留 `8.0.RC1` 这类 +解析不了的拼法可寻址),是约束就在已安装的版本里挑满足它的最高一个。判据里有一条 +**双侧有界**的范围 —— 只用下界的话,一个忽略约束直接取最新的实现照样通过。 + +### 10.2 落败的精确钉不是「违反」,只有被陈述的**要求**才是 + +正文 §2.3 写「赢家必须满足其余每一条**约束**」。按字面实现,两条互不相同的精确钉会 +互相「违反」,于是**每一个「依赖钉了某工具、工程也钉了」的组合都在升级当天变成硬 +失败** —— 而那个分歧正是 §2.2 的裁决存在的理由。 + +改成:`>=` / `^` / `~` / 逗号组合是**要求**,可以被违反;不带运算符的裸版本是**选择**, +由裁决处理并被报告。§2.3 的例子本来就是「钉 vs 下界」,所以这是把正文的措辞收紧到它 +自己的例子上。 + +**这条不对称随后成了机制。** 规则包写精确版本 = 提供一个默认;写 `>=` = 陈述一条要求。 +同一个字段承载两种意图,而使用侧看到的行为恰好不同:覆盖默认是安静的,击穿要求是被 +拒绝的。 + +### 10.3 默认的形状是「耦合判断」,不是风格 —— §6.1 的通篇 `>=` 是错的 + +§6.1 写 `"xim:cuda-nvcc" = ">=12.9.86"`。实测:索引里有 12.9.86 与 13.3.33,`>=` 取 +最高,于是**默认变成 13.3.33** —— 而 13.x 把驱动下界抬到 r580,本轮实测的机器是 12.4。 +一条本意为「至少这么新」的下界,把整条 lane 换成了没验证过的那条。 + +规则:**版本耦合着规则看不见的东西时,默认写精确版本;不耦合时写下界。** CUDA / HIP +的运行时耦合驱动 → 钉 12.9 线;glslang / dpcpp / CANN 不耦合 → `>=`。 + +### 10.4 边界:规则声明它**编译时**用的,工程声明它**运行时**跑在上面的 + +`xim:mesa-lavapipe` 是一个软件 Vulkan 设备。把它挪进 `rules-spirv` 会给每一个消费者 +强加一个软件渲染器,包括有 GPU 的那些。它留在工程里。 + +同理 `compat:cuda-runtime` / `compat:sycl-runtime` / `compat:vulkan-runtime`:它们是 +产物的运行期适配器,而且有一条结构性理由 —— 插件包是一条 `[build-dependencies]` 边, +它自己的 `[dependencies]` **有意**不到达消费者的 target。 + +所以「插件自带环境」覆盖的是 xim 载荷那一半,不是全部。这条写进 plugins 的 README 与 +manifest 注释,因为它是唯一能回答「为什么这个还要我写」的地方。 + +### 10.5 修好一处判据,第一批红的是自己的夹具 + +把 plugins CI 的 `MCPP_VERSION` 从 2026.9.6.1 抬到 2026.9.6.6 之后,`hip-consumer` +当场变红:它的 glob 写 `accel = "hip, cuda12.9+{sm_89}"`,而 `[package] accelerators` +只有 `["hip"]`。`sycl-consumer` 与 mcpp 自己的 `examples/09-heterogeneous/{hip,sycl}` +同形。 + +**两边的 CI 都看不见它。** mcpp 侧那两个示例在 `build_examples.sh` 里是 SKIP;plugins +侧的引擎 pin 停在检查出现之前。这是一条只有**跨仓库、且两边都动**才会暴露的缺陷。 + +### 10.6 告警码:新开一个,不复用 + +正文 §2.2 说跨包这一趟复用 `xlings/axis-override`。没有采纳:那个码的含义写在它自己的 +注释里,是「同一份 manifest 的两条**轴**」。跨包的版本分歧是另一件事,而告警码是给机器 +读的分类。新码是 `xlings/version-override`。 + +### 10.8 生态验证顺手挖出一个引擎缺陷:`DT_RPATH` 的继承没被建模 + +为了让 `rules-ascendc` 的诊断真的被编译一次(五个夹具没有一个激活它),我拿 cann 示例 +跑了一遍。它被拒绝了,消息点名**八个**库找不到。 + +**而它刚刚链接出来的那个产物,把其中七个都解析掉了** —— 直接跑一次,加载器只在第八个 +上失败,那一个属于这台机器没有的驱动。模型与加载器给出相反的读数。 + +真因:`DT_RPATH` 被整条依赖链继承,`DT_RUNPATH` 不被继承;而闭包模型只查**发起请求 +的那个对象自己**的搜索路径。厂商工具包恰好是最坏的形状 —— 库与库之间按裸 SONAME 互相 +依赖、自己都不带搜索路径,而它们所在的目录只写在**可执行文件**的 `DT_RPATH` 里。 + +三条值得记下来的: + +1. **这与本轮的身份工作无关,是既有缺陷**。用已发布的 2026.9.6.5 加已发布的 plugins + 0.2.3 复现,读数一字不差 —— 先做这个对照,再动手。 +2. **它只在「跑一次」时暴露**。构建被拒绝,而拒绝消息本身是自洽的;唯一能分开「模型对」 + 和「加载器对」的动作,是执行那个产物。 +3. **抑制的那一半必须一起写**:带 `DT_RUNPATH` 的对象不使用任何 RPATH,自己的和继承的 + 都不用。只写正向那条,一个无条件继承的实现同样通过。 + +⭐ 更一般的形状:**为了让一段没人跑过的代码被编译一次而去跑它,挖到的往往不是那段代码 +的问题。** 五个夹具覆盖五条规则里的四条,而第五条一次都没被编译过。 + +### 10.7 落地顺序(不可交换) + +| # | 仓库 | 内容 | 前置 | +|---|---|---|---| +| 1 | mcpp | 引擎 + 文档 + 判据 + 2026.9.6.6 | 无 | +| 2 | — | 发布 2026.9.6.6 | 1 合入 | +| 3 | mcpp-plugins | 五条规则自带环境 + 0.2.4 | 2(CI 要下载这个引擎) | +| 4 | — | 0.2.4 进索引 | 3 合入 | +| 5 | mcpp | 示例瘦身到 0.2.4 | 4(示例从索引解析 plugins) | + +第 5 步是 mcpp 的**第二个** PR。它不是拆分,是两次跨仓库发布夹在中间的必然结果: +示例引用的是已发布的 plugins,而那一版依赖第 1 步的引擎。 diff --git a/.agents/docs/2026-09-07-round7-verify.sh b/.agents/docs/2026-09-07-round7-verify.sh new file mode 100755 index 00000000..b8e33ccc --- /dev/null +++ b/.agents/docs/2026-09-07-round7-verify.sh @@ -0,0 +1,307 @@ +#!/usr/bin/env bash +# Ecosystem verification for round 7: one package, one version -- and a rule +# package that brings its own environment. Against a PUBLISHED mcpp and a +# PUBLISHED mcpp:plugins. +# +# # The sandbox has an EMPTY $HOME and a fresh /tmp, so this file is not +# # visible from inside it. Pass the script itself in: +# B64=$(base64 -w0 ) +# xlings subos use verify-966 --sandbox --cmd \ +# "echo $B64 | base64 -d > /tmp/v.sh && MCPP_VERIFY_VERSION=2026.9.6.6 bash /tmp/v.sh" +# +# mcpp is addressed by its STORE path, which is the one thing the sandbox does +# share: the xlings data directory. A bare `mcpp` is not on PATH in there. +# +# WHY A SANDBOX IS THE RIGHT PLACE FOR THIS ONE. Every criterion below is about +# what gets INSTALLED. On a machine that has built any of this before, the +# payload is in the registry already and "one version" reads the same whether +# the engine unified anything or not. A sandbox has an empty registry, so the +# question can be asked at all. +# +# EVERY CRITERION NAMES THE OBJECT IT SELECTED, and every section that did not +# run is listed again in the summary. "0 assertions failed" printed by a script +# that skipped four sections is the failure mode this shape exists to prevent. +set -u + +VER="${MCPP_VERIFY_VERSION:?set MCPP_VERIFY_VERSION}" +STORE="${MCPP_VERIFY_BIN:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/$VER/bin/mcpp}" +PLUGINS="${MCPP_VERIFY_PLUGINS:-0.2.4}" + +fails=0 +skipped="" +fail() { printf 'ASSERT-FAIL: %s\n' "$1"; fails=$((fails + 1)); } +ok() { printf 'ok: %s\n' "$1"; } +section() { printf '\n== %s ==\n' "$1"; } +skip() { printf 'NOT RUN: %s\n' "$1"; skipped="$skipped + - $1"; } + +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT + +# The store this run installs into, so "how many versions of X are here" is a +# question about THIS run and not about the machine. +# `MCPP_HOME` first: the sandbox does not set it, so `$HOME/.mcpp` is right +# there -- but a rehearsal on the host does set it, and reading the real home +# instead answers "one version installed" from a registry this run never +# touched. Measured: section D passed on the host for exactly that reason. +XPKGS="${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs" +count_versions() { ls -1 "$XPKGS/xim-x-$1" 2>/dev/null | wc -l | tr -d ' '; } +list_versions() { ls -1 "$XPKGS/xim-x-$1" 2>/dev/null | sort | tr '\n' ' '; } + +# ── A. identity ───────────────────────────────────────────────────────────── +section "A. identity" +if [ ! -x "$STORE" ]; then + fail "no released binary at $STORE" + printf 'FAIL: nothing further can run\n'; exit 1 +fi +got=$("$STORE" --version 2>&1 | head -1) +if [ "$got" = "mcpp $VER" ]; then ok "$got from $STORE" +else fail "version is '$got' at $STORE"; fi + +"$STORE" self config --mirror "${MCPP_VERIFY_MIRROR:-CN}" >/dev/null 2>&1 || true + +# ── B. a range is installed AND answered ──────────────────────────────────── +# +# Engine-only: no rule package, no index entry beyond the tool itself. This is +# the gap that made a floor unusable -- the payload arrived and `xpkg_dir` +# answered "". +section "B. a range is installed and answered (engine only)" +mkdir -p "$work/b/src" +printf 'int main(){return 0;}\n' > "$work/b/src/main.cpp" +cat > "$work/b/build.mcpp" <<'EOF' +#include +#include +import mcpp; +int main() { + const char* d = mcpp::xpkg_dir("xim", "shaderc"); + std::string out = std::string(mcpp::manifest_dir()) + "/answered.txt"; + std::FILE* f = std::fopen(out.c_str(), "w"); + if (f == nullptr) return 3; + std::fprintf(f, "%s\n", d == nullptr ? "" : d); + std::fclose(f); + return 0; +} +EOF +cat > "$work/b/mcpp.toml" <<'EOF' +[package] +name = "rangecheck" +version = "0.1.0" +[xlings.workspace] +"xim:shaderc" = ">=2026.1" +[build] +sources = ["src/main.cpp"] +[targets.rangecheck] +kind = "bin" +main = "src/main.cpp" +EOF +if ( cd "$work/b" && "$STORE" build >build.log 2>&1 ); then + if [ -s "$work/b/answered.txt" ] && grep -q 'xim-x-shaderc' "$work/b/answered.txt"; then + ok "a range was answered: $(cat "$work/b/answered.txt")" + else + fail "the range installed but xpkg_dir answered '$(cat "$work/b/answered.txt" 2>/dev/null)'" + fi +else + fail "a project declaring a range did not build" + tail -5 "$work/b/build.log" +fi + +# …and the other direction, so a build that treats the range as a literal is +# not read as a pass. +mkdir -p "$work/b2/src" +printf 'int main(){return 0;}\n' > "$work/b2/src/main.cpp" +sed 's/>=2026.1/>=2099.1/' "$work/b/mcpp.toml" > "$work/b2/mcpp.toml" +if ( cd "$work/b2" && "$STORE" build >build.log 2>&1 ); then + fail "an unsatisfiable range was accepted" +else + if grep -q 'not found in the synced index' "$work/b2/build.log"; then + ok "an unsatisfiable range is refused, naming it" + else + fail "refused, but not for the stated reason: $(grep -m1 error "$work/b2/build.log")" + fi +fi + +# ── C. one edge, and the rule brings its environment ──────────────────────── +section "C. one edge, no [xlings.workspace] anywhere in the project" +mk_spirv_project() { # $1 = dir, $2 = extra manifest text + mkdir -p "$1/src" "$1/shaders" + cat > "$1/shaders/scale.comp" <<'EOF' +#version 450 +layout(local_size_x = 64) in; +layout(std430, binding = 0) buffer Data { float v[]; }; +layout(push_constant) uniform Push { float a; uint n; } push; +void main() { + const uint i = gl_GlobalInvocationID.x; + if (i >= push.n) return; + v[2u * push.n + i] = push.a * v[i] + v[push.n + i]; +} +EOF + cat > "$1/src/main.cpp" <<'EOF' +#include +#include +#include "scale_comp.h" +int main() { + const std::uint32_t magic = scale_comp_spv[0]; + std::printf("magic=%08x\n", magic); + return magic == 0x07230203u ? 0 : 1; +} +EOF + cat > "$1/build.mcpp" <<'EOF' +import std; +import mcpp; +import mcpp.rules.spirv; +int main() { + mcpp::rerun_if_changed_glob("shaders/**/*.comp"); + mcpp::rules::spirv::options opt; + opt.includes = { "shaders" }; + return mcpp::rules::spirv::compile(opt) ? 0 : 1; +} +EOF + cat > "$1/mcpp.toml" <build.log 2>&1 ); then + if grep -q 'entries declared by dependencies' "$work/c/build.log"; then + ok "the payload came from the rule: $(grep -m1 'entries declared by dependencies' "$work/c/build.log")" + else + fail "it built, but nothing was provisioned from the graph -- the rule declared no payload" + fi + out=$( cd "$work/c" && "$STORE" run 2>&1 | tail -1 ) + if [ "$out" = "magic=07230203" ]; then ok "and it runs: $out" + else fail "ran and printed '$out'"; fi +else + # THE SKIP IS RECOGNISED BY THE RESOLUTION ERROR, NOT BY "it failed and the + # log mentions plugins". `E_NOT_FOUND` with the wire address is a sentence + # only the resolver writes; a compile or link failure cannot produce it, so + # a genuine defect is never filed as "not published yet". + if grep -q "E_NOT_FOUND: package 'mcpp:plugins@$PLUGINS' not found" "$work/c/build.log"; then + skip "C: mcpp:plugins@$PLUGINS is not in the index yet" + skip "D: needs C" + skip "E: needs C" + else + fail "a project writing only the plugins edge did not build" + tail -15 "$work/c/build.log" + fi +fi + +# ── D. the override: one version, and the nearer declaration wins ─────────── +if [ -d "$work/c/target" ]; then + section "D. the project's own pin overrides the rule's, and only one is installed" + mk_spirv_project "$work/d" ' +[target.'"'"'cfg(accelerator = "vulkan")'"'"'.xlings.workspace] +"xim:glslang" = "15.1.0" +' + if ( cd "$work/d" && "$STORE" build >build.log 2>&1 ); then + n=$(count_versions glslang) + if [ "$n" = "1" ]; then ok "one glslang installed: $(list_versions glslang)" + else fail "two declarations of one package installed $n versions: $(list_versions glslang)"; fi + else + fail "the override did not build" + tail -10 "$work/d/build.log" + fi + +fi + +# ── E. a pin below a stated floor is refused, naming both sides ───────────── +# +# NOT THROUGH A RULE PACKAGE, and the reason is worth recording: no published +# rule states a floor that a published version can sit below. `rules-spirv` +# requires `>=15.1.0` and 15.1.0 is the only glslang in the index; the same +# holds for dpcpp and for the CANN toolkit. A fixture pinning `1.0.0` to get +# under the floor is refused one step earlier, by provisioning, with +# `not found in the synced index` -- the root's own pins are resolved before +# the graph's constraints are known, so that refusal always wins. Measured: +# the first draft of this section asserted the conflict message and read back +# the provisioning one. +# +# So the construction is a local dependency and `xim:zoxide`, which publishes +# three versions. What it verifies here rather than in the e2e suite is the +# RELEASED binary. +section "E. a pin below a stated floor is refused" +mkdir -p "$work/e/dep/src" "$work/e/app/src" +printf 'int dep_touch() { return 1; }\n' > "$work/e/dep/src/lib.cpp" +cat > "$work/e/dep/mcpp.toml" <<'EOF' +[package] +name = "toolowner" +version = "0.1.0" +[modules] +sources = ["src/**/*.cpp"] +[xlings.workspace] +"xim:zoxide" = ">=0.9.9" +[targets.toolowner] +kind = "lib" +EOF +printf 'int main(){return 0;}\n' > "$work/e/app/src/main.cpp" +cat > "$work/e/app/mcpp.toml" <<'EOF' +[package] +name = "consumer" +version = "0.1.0" +[dependencies] +toolowner = { path = "../dep" } +[xlings.workspace] +"xim:zoxide" = "0.9.7" +[targets.consumer] +kind = "bin" +main = "src/main.cpp" +EOF +if ( cd "$work/e/app" && "$STORE" build >build.log 2>&1 ); then + fail "a pin below a dependency's stated floor was accepted" +else + miss="" + for needle in 'xim:zoxide' '0.9.7' '>=0.9.9' 'toolowner' 'drop the pin'; do + grep -q -- "$needle" "$work/e/app/build.log" || miss="$miss $needle" + done + if [ -z "$miss" ]; then ok "refused, naming both sides and the way out" + else fail "refused, but the message omits:$miss"; fi +fi + +# …and raising it clears the refusal. Without this leg the section also passes +# on an engine that refuses every project naming a tool its dependency names. +sed 's/"0.9.7"/"0.9.9"/' "$work/e/app/mcpp.toml" > "$work/e/app/mcpp.toml.new" +mv "$work/e/app/mcpp.toml.new" "$work/e/app/mcpp.toml" +rm -rf "$work/e/app/target" +# +# NO VERSION COUNT HERE. The refused leg above already installed 0.9.7: the +# root's own pins are provisioned before the graph's constraints are known, so +# a build that is about to be refused has already paid for what the project +# asked for. Counting across both legs therefore reads two, and it is not the +# defect this section is about -- section D owns "one package, one version", +# on a store that only one build has touched. +if ( cd "$work/e/app" && "$STORE" build >build2.log 2>&1 ); then + ok "raising the pin to satisfy the floor builds" +else + fail "a pin that satisfies the floor was still refused" + tail -10 "$work/e/app/build2.log" +fi + +# ── summary ──────────────────────────────────────────────────────────────── +section "summary" +printf '%s assertion(s) failed\n' "$fails" +if [ -n "$skipped" ]; then + printf 'sections that did NOT run:%s\n' "$skipped" +fi +[ "$fails" -eq 0 ] || exit 1 diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index 6c6ec132..c549b0ad 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -25,7 +25,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.8.30.2' + default: '2026.9.5.1' cache-target: description: also restore/save target/ (build artifacts + BMIs) required: false diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index a27f868a..c1e67b14 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.8.30.2' + default: '2026.9.5.1' runs: using: composite diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index af574f32..9e177681 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.8.30.2' + XLINGS_VERSION: '2026.9.5.1' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index a7aca72e..abb0256d 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -152,7 +152,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.30.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.5.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -293,7 +293,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.30.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.5.1 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -364,7 +364,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.30.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.5.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 2cb131b5..0c017088 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -282,7 +282,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.30.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.5.1 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index ac524491..9175d63a 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -122,7 +122,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.8.30.2' + XLINGS_VERSION: '2026.9.5.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -263,7 +263,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.30.2' + XLINGS_VERSION: '2026.9.5.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e0cf5fba..f3f3f0fe 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -96,7 +96,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.8.30.2' + XLINGS_VERSION: '2026.9.5.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" @@ -289,7 +289,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.30.2' + XLINGS_VERSION: '2026.9.5.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -360,7 +360,7 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.8.30.2-linux-aarch64.tar.gz" + XLA="xlings-2026.9.5.1-linux-aarch64.tar.gz" # NOT fetch_release.sh: this asset is OPTIONAL and the `if` is the # point — an arch with no prebuilt xlings must fall through quietly, # while the helper retries a 404 five times before giving up. The one @@ -369,9 +369,9 @@ jobs: # cover it. if curl -fsSL --retry 3 --retry-delay 2 --retry-all-errors \ --connect-timeout 20 --max-time 600 -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.8.30.2/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.9.5.1/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.8.30.2-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.9.5.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -449,7 +449,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.30.2' + XLINGS_VERSION: '2026.9.5.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then WORK=$(mktemp -d) @@ -632,7 +632,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.30.2' + XLINGS_VERSION: '2026.9.5.1' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash diff --git a/CHANGELOG.md b/CHANGELOG.md index f868fa8f..930574dd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,82 @@ ## [Unreleased] +## [2026.9.6.6] - 2026-09-07 + +### 一个包一个版本:规则自带环境,工程只写例外 + +**版本在一个地方属于包的身份,在另一个地方不属于。** 条件合并按包去重(`xim:glibc` 与 +`xim:glibc@2.40` 是一个条目),而跨包的划分比对整个地址串(是两个包)。于是工程钉 +`xim:cuda-nvcc@13.3.33`、规则包钉 `@>=12.9.86` 时,两份都装 —— 每份数 GB —— 而 +`fillXpkgDirs` 里「同名保留第一个值」让 `xpkg_dir` 只答其中一个。**装了两份、用了一 +份,且没有任何一句话提到它。** + +现在身份统一为 `(namespace, name)`,版本永远是这个包上的**约束**。装哪一版分两步: +**裁决**——离产物更近的声明赢(工程 > 依赖),不带版本的声明弃权,结果被报出来; +**校验**——赢家必须满足每一条落败的**要求**(`>=` / `^` / `~` / 逗号组合),不满足就 +拒绝并同时点出两侧与出路。裸版本是**选择**不是要求,两条不同的精确钉走裁决而不被拒绝 +—— 否则每一个「依赖钉了工具、工程也钉了」的组合会在升级当天变成硬失败。 + +校验是**一次比较,不是一次搜索**:版本由裁决选定再被检查,所以引擎从不需要问索引「有 +哪些版本」,也就不带约束求解器。代价写在文档里而不是藏起来 —— 一些求解器本可满足的 +组合会被拒绝,而拒绝消息里写着怎么走。 + +**`mcpp::xpkg_dir` 现在回答范围。** 版本位一直接受范围表达式且 xlings 真的求解它,而 +这个查询把整个位置当目录名比对 —— 于是 `>=8.5.0` 装上了载荷,然后回答「没装」。这正是 +让规则包无法声明下界的那道缝:它声明、装上、然后找不到。 + +由此,**规则包自带它的环境**: + +```toml +# mcpp:plugins 里 +[target.'cfg(accelerator = "cuda")'.feature-xlings.rules-cuda] +"xim:cuda-nvcc" = ">=12.9.86" +``` + +工程侧只剩 `[build-dependencies]` 那一行。两重门:feature 说「要不要」,selector 说 +「哪些构建真的下载」,CPU-only 构建一个字节都不装。要换版本就在工程里写同名条目 —— +`examples/09-heterogeneous/multi-backend` 是仓库里唯一走这条路径的示例,其余每一个都 +只写那条边。 + +配套:e2e 627(装了什么与答了什么必须同时断言)、628(拒绝 + 抬钉后通过的反向腿 + +`--format json` 的 `tool-version-conflict`)、629(范围被安装并被回答,而工程没声明任何 +工具);单测 `test_xlings_address_set.cpp` 与 `xpkg_payload_at` 的范围/不可解析版本两组。 +规范:SPEC-001 §10、SPEC-004 §4.5。 + +### 运行期闭包检查漏掉了 `DT_RPATH` 的继承,于是拒绝了一个能跑起来的产物 + +`DT_RPATH` 被整条依赖链继承,`DT_RUNPATH` 不被继承。而闭包模型只查**发起请求的那个 +对象自己**的搜索路径列表 —— 于是一个厂商工具包的形状(库与库之间按裸 SONAME 互相 +依赖、都不带搜索路径,而它们所在的目录只写在**可执行文件**的 `DT_RPATH` 里)整批读成 +「找不到」。 + +`examples/09-heterogeneous/cann` 上实测:mcpp 拒绝构建并点名**八个**库,而它刚刚链接 +出来的那个产物解析掉了其中七个,只在第八个上失败 —— 而那一个属于这台机器没有的驱动。 +**模型与加载器给的是相反的读数,而模型赢了报告、加载器赢了现实。** + +抑制的那一半同样是规则的一部分:带 `DT_RUNPATH` 的对象不使用任何 RPATH,自己的和 +继承的都不用。两条腿各有单测,而正向那条在修复被拿掉时当场变红。 + +### 四处新拒绝有了名字 + +本轮之前新增的三处拒绝(设备源没到达 action、glob 命名了本包未声明的后端、`build.mcpp` +导入了没人以 host module 提供的模块)与本轮新增的工具版本冲突,都写进了 +`refusal::Code` 与 `docs/11` 的词表:`device-source-unconsumed`、 +`accel-backend-undeclared`、`host-module-missing`、`tool-version-conflict`。机器消费者 +此前只能靠匹配散文把它们与别的拒绝分开,而 `docs/11` 正是承诺不必如此的那份规范。 + +### xlings pin 抬到 2026.9.5.1 + +`pinned::kXlingsVersion` 从 2026.8.30.2 抬到当前最新。这个常量是 +`.github/tools/check_version_pins.sh` 的唯一真源,`.github/` 下每一处都按它比对。范围的 +求解由 xlings 侧执行,停在三个月前的 pin 让这条依赖成为一个没人核过的假设。 + + +## [2026.9.6.3 - 2026.9.6.5] + +> 这三个版本发布时没有各自分节,条目按轮次而非按版本聚在一起。保留原样而不是 +> 事后拆分:拆分要靠回忆去分配,而这些条目里没有一条记录着它进的是哪一版。 + ### 写第二个后端的时候,才发现接口只对第一个成立 多后端示例本身没建起来,而它暴露的四件事都不是示例的问题。 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index 9800c529..0be954e1 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -2245,6 +2245,88 @@ A feature name no `[features]` table declares is reported as a schema warning: it activates for nobody and installs nothing, and a tool whose absence is only visible as *"the device is never reachable"* is the hardest kind to diagnose. +#### A rule package brings its own environment (2026.9.6.6+) + +The table above is what a project writes when it has an opinion. Most projects +have none, and nothing is what they write: + +```toml +[build-dependencies.mcpp] +plugins = { version = "0.2.4", features = ["rules-cuda"], host-module = true } +``` + +That one edge is the whole declaration. The rule package names the packages its +rule needs and the version it needs them from, under the feature that selects +it and the accelerator it is for: + +```toml +# in the rule package, not in your project +[target.'cfg(accelerator = "cuda")'.feature-xlings.rules-cuda] +"xim:cuda-nvcc" = ">=12.9.86" +"xim:cuda-cudart" = ">=12.9.79" +``` + +Two gates, and both must open. The feature says *whether this rule is wanted*; +the selector says *which builds actually download it*. A CPU-only build of the +same project opens neither and installs nothing. + +Which package, and how old it may be, is the rule author's knowledge. Repeating +it in every project that uses the rule is a copy that goes stale silently — the +rule moves and the projects do not. + +#### One package, one version (2026.9.6.6+) + +A tool address is `[:][@]`, and its **identity is the +`(namespace, name)` pair**. The version is a constraint on that package, never +part of its name, so `xim:glibc`, `xim:glibc@2.40` and `xim:glibc@>=2.38` all +name one package. One version of it is installed per build. + +Which one is decided in two steps. + +**Adjudication — the declaration nearer the artifact wins.** Your project +outranks a package it depends on, so a pin overrides a rule's requirement: + +```toml +# the project, when it does have an opinion +[target.'cfg(accelerator = "cuda")'.xlings.workspace] +"xim:cuda-nvcc" = "13.3.33" +``` + +A declaration that names no version abstains: it says the package is wanted and +nothing about which version, so it cannot outrank a floor merely by being +nearer. When two declarations disagree and both name a version, mcpp reports +which was used — an override visible only as *"two versions were declared and +one directory exists"* is a fact the reader has to reconstruct from the +filesystem. + +**Validation — the winner must satisfy every requirement that lost.** `>=`, +`^`, `~` and comma-combined forms are requirements. A pin that fails one is +refused, naming both sides: + +``` +error: `xim:cuda-nvcc` is pinned to 12.0.0 by this project, and mcpp:plugins + requires >=12.9.86. + One version of a package is installed, so the two cannot both hold. + fix: pin a version satisfying >=12.9.86, or drop the pin and let the + requirement decide. +``` + +A bare version is a *choice*, not a requirement: two exact pins that differ are +adjudicated and reported, not refused. Only a stated requirement can be +violated. + +**This is a comparison, not a search.** The version is chosen by adjudication +and then checked, so mcpp never has to ask the index which versions exist and +carries no constraint solver. The cost is stated rather than hidden: a +combination a solver could satisfy — a project's `>=8.0`, a rule's `8.5.0`, and +8.3 as the newest in the index — is refused instead, and the refusal says how to +proceed. + +Ranges are resolved in both directions. `>=2026.1` installs the highest +published version satisfying it, `>=2099.1` is refused as unsatisfiable, and +`mcpp::xpkg_dir` answers with the highest **installed** version satisfying the +range — a rule that declares a floor can find what the floor brought in. + #### Which version a tool the project did not name resolves to | The project declares | The version comes from | diff --git a/docs/07-build-mcpp.md b/docs/07-build-mcpp.md index 335fc2f0..ea4a5971 100644 --- a/docs/07-build-mcpp.md +++ b/docs/07-build-mcpp.md @@ -121,7 +121,7 @@ int main() { | `mcpp::link_flag(s)` *(2026.9.6.5+)* | `mcpp:link-flag=` | | `mcpp::link_script(p)` *(2026.8.19+)* | `mcpp:link-script=` | | `mcpp::runner(tok)` *(2026.8.19.2+)* | `mcpp:runner=` — see below | -| `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | the payload directory of a package this manifest declared in `[xlings.workspace]`; `""` when it was not declared or is not installed (see below) | +| `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | the payload directory of a package declared in `[xlings.workspace]` — by this manifest, or by a dependency compiled into this build program *(2026.9.6.6+)*; `""` when it was not declared or is not installed (see below) | | `mcpp::warning(text)` *(2026.8.21.2+)* | `mcpp:warning=` — see below | | `mcpp::action{…}.submit()` *(2026.8.5.1+)* | `mcpp:action=` — declares a **build-graph node** instead of doing the work here (see below) | @@ -335,6 +335,18 @@ A **pinned** reference resolves to exactly that version or to nothing. A build that asked for `1.8.12` and silently got `1.9.0` is an answer only discovered later, in the artifact. +A **constrained** one (`>=8.5.0`, `^1.2`) resolves to the highest installed +version satisfying it *(2026.9.6.6+)*. Before that release the whole version +position was compared against a directory name, so a range installed a payload +and then answered that nothing was installed — which is why a rule package +could not state a floor and every project repeated its rule's package list. + +**A package a DEPENDENCY declared is answered too** *(2026.9.6.6+)*, at the +version this build actually installed rather than the one the local manifest +wrote. One package means one version: where a project and a rule both name it, +the declaration nearer the artifact wins and both sides are told the same +answer. See *One package, one version* in [05 — mcpp.toml](05-mcpp-toml.md). + **`[feature-xlings.]` is answered too, while `` is active** *(2026.9.6.2+)*. That table has provisioned its packages since it existed -- naming one downloads and installs it -- but the build program's environment was diff --git a/docs/11-machine-output.md b/docs/11-machine-output.md index ff444af3..6116c789 100644 --- a/docs/11-machine-output.md +++ b/docs/11-machine-output.md @@ -380,6 +380,10 @@ a program classifying the outcome reads `reason`: | `exclusive-capability` | two packages provide one capability and at least one declared it exclusive | | `version-floor-unmet` | a package requires more of the machine than the machine was declared to have | | `accel-mismatch` | a `[build] sources` entry is constrained to a device set this build does not cover | +| `accel-backend-undeclared` | a `[build] sources` entry names a backend its own package does not list in `[package] accelerators` | +| `device-source-unconsumed` | a device-kind source reached no action, so nothing compiles it | +| `host-module-missing` | `build.mcpp` imports a module no dependency supplies as a host module | +| `tool-version-conflict` | two declarations name one xlings package at versions that cannot both hold | | `other` | a refusal whose branch has not been given a token yet | **Exit 0 whenever the question was answered, including "refused".** "Would diff --git a/docs/17-the-project-environment.md b/docs/17-the-project-environment.md index c3196e94..414d8eda 100644 --- a/docs/17-the-project-environment.md +++ b/docs/17-the-project-environment.md @@ -68,8 +68,9 @@ behind it. ### 2.1 Which version pins apply (2026.9.3+) Naming an environment also changes where a tool's version comes from. A -project's own `[xlings.workspace]` entries always win; what differs is what -they are laid over: +project's own `[xlings.workspace]` entries always win — over the environment +here, and over a dependency's declaration by the rule in section 3 — and what +differs is what they are laid over: | The project declares | The version of a tool it did not name comes from | |---|---| @@ -93,9 +94,23 @@ different question from `PATH` and stays a different answer: a build program that needs a package's data files (protoc's well-known `.proto` files, say) asks for the directory, and one that needs to *run* a program asks `PATH`. -A dependency's own `[xlings]` declaration is never consulted or propagated. In -a workspace build the workspace root owns the selection; a member's declaration -applies only when that member is built as an independent root. +**A workspace member's declaration is not the workspace's.** In a workspace +build the workspace root owns the selection; a member's `[xlings]` applies only +when that member is built as an independent root. + +**A dependency's declaration is a different matter, and it is honoured** +(2026.9.5.4+ for `[xlings] deps`, 2026.9.6.6 for the version rule below). A +board-support package knows which emulator reaches its machine, and a rule +package knows which toolkit its rule drives; a consumer that had to repeat +either is the duplication such packages exist to remove. What the dependency +declares is installed, and `MCPP_XPKG__DIR` answers for it in that +dependency's own build program. + +Where a project and a dependency name **one package**, one version of it is +installed: identity is `(namespace, name)` and the version is a constraint on +it. The declaration nearer the artifact wins and the override is reported; +a pin that fails a requirement the other side stated is refused naming both. +See *One package, one version* in [05 — mcpp.toml](05-mcpp-toml.md). ## 4. Reading an environment, never creating one diff --git a/docs/20-heterogeneous-builds.md b/docs/20-heterogeneous-builds.md index db4061aa..6165a4ac 100644 --- a/docs/20-heterogeneous-builds.md +++ b/docs/20-heterogeneous-builds.md @@ -21,10 +21,15 @@ entry (`cuda12.9+{sm_89} ptx>=89` is one backend with an architecture set and a PTX floor). See "Several backends in one build" below for how the source sets are then written. -Four programming models have rule packages today -- CUDA, HIP, SYCL and -Vulkan/SPIR-V -- and the table under "The lanes" says which compiler and which -payloads each drives. Nothing in the engine holds a vendor name, so a fifth is -a package rather than an engine change. +`accelerator = "none"` is the empty set -- true exactly when this build named +no accelerator at all. It is the one value that is not a backend name, and it +exists because the rest of the vocabulary is **open**: a set that can grow +cannot be negated by listing it. See "The CPU fallback" below. + +Five programming models have rule packages today -- CUDA, HIP, SYCL, +Vulkan/SPIR-V and Ascend C -- and the table under "The lanes" says which +compiler and which payloads each drives. Nothing in the engine holds a vendor +name, so a sixth is a package rather than an engine change. ## Two shapes, and why one mechanism reaches both @@ -144,6 +149,56 @@ The division is deliberate. mcpp owns the graph, the artifact's identity and the set of architectures; a vendor's flag spelling, its architecture syntax and its host-compiler requirements belong to the rule. +## The toolkit comes with the rule (2026.9.6.6+) + +A project that wants a CUDA island writes one edge: + +```toml +[build-dependencies.mcpp] +plugins = { version = "0.2.4", features = ["rules-cuda"], host-module = true } +``` + +and nothing else. The vendor toolkit, its runtime and whatever else the rule +needs are declared **by the rule package**, under the feature that selects it +and the accelerator it serves: + +```toml +# mcpp:plugins, not your project +[target.'cfg(accelerator = "cuda")'.feature-xlings.rules-cuda] +"xim:cuda-nvcc" = ">=12.9.86" +"xim:cuda-cudart" = ">=12.9.79" +"xim:libcurand" = ">=10.3.10.19" +"xim:cuda-cccl" = ">=12.9.27" +``` + +Two gates, and both must open before a byte is downloaded. The feature says +*whether this rule is wanted*; the `cfg(accelerator = ...)` selector says +*which builds actually need the device toolkit*. `mcpp build` with no +accelerator opens neither, which is what keeps the cheapest build cheap -- and +that is the build CI runs. + +Every lane works this way: `rules-cuda`, `rules-hip`, `rules-sycl`, +`rules-spirv` and `rules-ascendc` each carry their own. No lane expects the +project to supply the environment instead, because "sometimes it has to" is a +rule nothing can state precisely enough to be useful. + +**Overriding is one line, in the project.** Which package and how old it may be +is the rule author's knowledge; *exactly which version* sometimes belongs to the +project, because a CUDA line is coupled to a driver floor that is a property of +the machines it will run on: + +```toml +[target.'cfg(accelerator = "cuda")'.xlings.workspace] +"xim:cuda-nvcc" = "13.3.33" +``` + +The nearer declaration wins, one version is installed, and mcpp says which. A +pin that does not satisfy the rule's floor is refused naming both sides rather +than installed alongside it. See *One package, one version* in +[05 — mcpp.toml](05-mcpp-toml.md) for the full rule; +`examples/09-heterogeneous/multi-backend` is the one example in this repository +that takes the override path, and every other one writes only the edge. + ## What the rule package reports before the first compile Three things go wrong late with a device toolkit, and none of them is a fact @@ -348,17 +403,35 @@ ld: obj/src/cpu/impl.o: multiple definition of `impl'; ``` Loud, and at the link rather than at run time -- but it is a failure the -manifest can avoid. Negate the whole set: +manifest can avoid. Say "this build named no backend" directly: ```toml -[target.'cfg(not(any(accelerator = "cuda", accelerator = "vulkan")))'.build] +[target.'cfg(accelerator = "none")'.build] sources = ["src/cpu/*.cpp"] ``` -`any`, `all` and `not` compose over the membership test as ordinary boolean -combinators. The cost is that the list is a second place the backend set is -written: a third backend added without extending this line produces the link -error above. +`none` (mcpp 2026.9.6.5) is the empty set: it holds when this build named no +accelerator at all, and it goes on holding after the ecosystem grows. + +**The enumeration it replaces was correct and did not stay correct.** Before +`none` the only spelling was to negate the whole set, + +```toml +[target.'cfg(not(any(accelerator = "cuda", accelerator = "vulkan")))'.build] +``` + +which made that line a **second place the backend set is written**. `any`, +`all` and `not` still compose over the membership test as ordinary boolean +combinators, and there are predicates that need them -- but this is not one. +`accelerator` is an **open** vocabulary: a third backend is a package rather +than an engine change, so the day one arrives, an enumeration that nobody +remembered to extend stops meaning what it says. It does not fail loudly when +that happens; it silently selects the CPU implementation for a build that has a +device, or the link error above, depending on which half was forgotten. + +`cfg(not(accelerator = "none"))` is the same statement inverted -- "this build +named at least one backend" -- and is what a dispatcher shared by several +backends should be conditioned on. ### The other shape: no link-time selection @@ -371,7 +444,7 @@ on whatever the machine has should not select at link time: * give each one a distinct name, and have the seam ask at run time which devices are present. -There is then no `not(any(...))` to maintain, and the artifact answers the +There is then no exclusion to maintain at all, and the artifact answers the question the user of the binary actually has. This is the shape ggml uses: each backend registers itself, and `ggml_backend_reg_by_name` picks one when the program runs. `ggml-org:llamacpp` is built that way, and its `backend-vulkan` @@ -406,12 +479,18 @@ names: `tests/unit/test_core_vendor_probes.cpp` asserts that no vendor tool name appears in `src/` once comments are stripped, with the file count as its own denominator. -| feature of `mcpp:plugins` | module | compiler it drives | payloads it needs | `[build] accel` | +| feature of `mcpp:plugins` | module | compiler it drives | payloads it declares | `[build] accel` | |---|---|---|---|---| | `rules-cuda` | `mcpp.rules.cuda` | the project's own clang (`-x cuda`), or nvcc with a GCC toolchain | `xim:cuda-nvcc`, `xim:cuda-cudart`, `xim:libcurand`, `xim:cuda-cccl` | `cuda12.9+{sm_89} ptx>=89` | | `rules-hip` | `mcpp.rules.hip` | the project's own clang (`-x cuda`) on the NVIDIA platform | the above plus `xim:hip-nvidia` | `hip, cuda12.9+{sm_89}` | | `rules-sycl` | `mcpp.rules.sycl` | the `xim:dpcpp` payload's clang (`-fsycl`) | `xim:dpcpp`, `xim:gcc`, `xim:cuda-nvcc` for an NVIDIA target | `sycl` or `sycl, cuda12.9+{sm_89}` | | `rules-spirv` | `mcpp.rules.spirv` | `glslangValidator` or `glslc` | `xim:glslang` or `xim:shaderc` | `vulkan1.2` | +| `rules-ascendc` | `mcpp.rules.ascendc` | `bisheng` (`-x asc`) from the CANN toolkit | `xim:cann-toolkit` | `ascend8.5+{dav-c220}` | + +The payload column is what each rule declares for itself under +`cfg(accelerator = ...)`, listed so the cost of a lane is legible before it is +taken. A project writes none of it -- see *The toolkit comes with the rule* +above. Two chunks, not one, in the `accel` value of the HIP and SYCL rows. The first names the programming model and the second names the device, so a device is @@ -435,7 +514,8 @@ runtime that threw it is not the one the caller would unwind with. ## What a framework looks like on top of this -The four lanes prove a rule package can drive four compilers. A framework is +The five lanes prove a rule package can drive five compilers, the newest of +them a vendor outside the NVIDIA and Khronos lineages. A framework is the next question -- whether the mechanism carries something a person would deploy -- and the answer has a shape of its own, measured on llama.cpp's Vulkan backend. diff --git a/docs/specs/manifest-semantics.md b/docs/specs/manifest-semantics.md index 68b706ed..3b208143 100644 --- a/docs/specs/manifest-semantics.md +++ b/docs/specs/manifest-semantics.md @@ -170,20 +170,26 @@ feature-deps feature-xlings ← 限定词是门 **状态:已实现**(2026.9.6.4)。 -### 4.3.1 工具的 selector 禁止命名目标侧层 +### 4.3.1 工具的 selector 禁止命名**被解析的**层 -`[target..xlings…]` 的 `` **禁止**命名 docs/14 的五个目标侧层 -(`accelerator`、`c-abi`、`c++-abi`、`compiler`、`compiler-runtime`、`kernel-abi`)。 +`[target..xlings…]` 的 `` **禁止**命名由依赖解析回答的五个层 +(`c-abi`、`c++-abi`、`compiler`、`compiler-runtime`、`kernel-abi`)。 -理由是**时序**而非风格:层由依赖解析回答,因此命名层的谓词被推迟到第二遍合并,而那 -一遍在工具供给之后、每个包的 build.mcpp 之后。在那里被接受的条目会被**声明却永远装 -不上**,产生的失败形态是最坏的一种:构建成功,工具不在。 +理由是**时序**而非风格:这五个层由依赖解析回答,因此命名它们的谓词被推迟到第二遍合并, +而那一遍在工具供给之后、每个包的 build.mcpp 之后。在那里被接受的条目会被**声明却永远 +装不上**,产生的失败形态是最坏的一种:构建成功,工具不在。 -实现**必须**在第一个载荷被取回之前拒绝,消息**必须**同时点出工具与谓词,并指出两条 -出路:按目标条件化,或用 `[feature-xlings.]` 做门——feature 在任何东西被供给之前 -就已知。 +`accelerator` **被接受**,且划分依据正是上面那条时序而不是这个键的主题。它不由任何东西 +解析:它是 `--accel` 或 `[build] accel`,在第一个包被查找之前就已读入,因此以它为谓词的 +条目在**第一**遍合并里就已折叠,与任何其他条目一样被安装。曾经把它一并拒绝的代价落在 +每一个带设备孤岛的工程的每一次构建上——厂商工具包只能无条件声明或完全不声明,于是 +一次 CPU-only 构建为它并不编译的设备下载数 GB。 -**状态:已实现**(2026.9.6.4)。 +实现**必须**在第一个载荷被取回之前拒绝被禁止的谓词,消息**必须**同时点出工具与谓词, +并指出两条出路:按目标条件化,或用 `[feature-xlings.]` 做门——feature 在任何东西被 +供给之前就已知。 + +**状态:已实现**(2026.9.6.4;`accelerator` 的接纳为 2026.9.6.5)。 ### 4.3.2 目标轴不进描述符 @@ -205,6 +211,32 @@ feature-deps feature-xlings ← 限定词是门 **状态:已实现**(2026.9.6.4)。 +### 4.5 一个包一个版本 + +`[xlings.workspace]` / `[xlings] deps` / `[feature-xlings.]` 里一条地址的身份是 +`(namespace, name)`。**版本永远是这个包上的约束,不是它名字的一部分。** 命名空间缺省 +为 `xim`,与 `[:][@]` 的解析一致。 + +一次构建里同一身份**只安装一个版本**,由两步决定: + +1. **裁决**——离产物更近的声明赢:工程 > 它依赖的包。同一份 manifest 内的两条轴仍按 + §4.3 的「更具体的赢」。**不带版本的声明弃权**:它陈述了「要这个包」而没有陈述「要 + 哪一版」,因此不参与这个问题。全都不带版本时,结果就是那个裸地址。 +2. **校验**——赢家**必须**满足每一条落败的**要求**。`>=` / `^` / `~` / 逗号组合是 + 要求;不带运算符的裸版本是**选择**,由裁决处理而不是校验。两条精确钉写得不同,是 + 两个选择,较近的那条赢并**必须**被报告;精确钉不满足某条要求时,实现**必须**拒绝 + 并同时点出两侧各自的声明与出路。 + +校验是**一次比较**,不是在索引里搜索。选哪一版由裁决决定,因此实现不需要「有哪些版本 +可选」这个输入,也就不需要约束求解器。代价是明确的:一些求解器本可满足的组合会被拒绝 +(工程写 `>=8.0`、规则写 `8.5.0`、索引最新 8.3),而拒绝消息里写着出路。 + +版本位接受范围表达式,并且**必须**在两个方向上都被求解:`>=2026.1` 装到满足它的最高 +版本,`>=2099.1` 被拒绝。实现**必须**让 `mcpp::xpkg_dir` 回答范围——安装了却答「不 +存在」,是让规则包无法声明下界的那个缺口。 + +**状态:已实现**(2026.9.6.6)。 + ## 5. 命名规约 ### 5.1 两种 case,按面划分 @@ -250,7 +282,15 @@ feature-deps feature-xlings ← 限定词是门 manifest 被拒绝。 4. §4.3 的"按包去重"判据:两条轴各写一次同一个包,`xlings.deps` 里该包**只出现 一次**,且是 `[target.]` 那条。 -5. §4.3.1 的判据:一份用层谓词声明工具的 manifest 被拒绝,且拒绝发生在任何下载之前。 +5. §4.3.1 的判据:一份用**被解析的**层谓词声明工具的 manifest 被拒绝,且拒绝发生 + 在任何下载之前;而同一份 manifest 把谓词换成 `accelerator` 时构建通过并装上工具。 + 两个方向都要跑:只跑拒绝那半,一个把所有层谓词都拒掉的实现同样通过。 +6. §4.5 的判据**必须**同时观察「装了什么」和「答了什么」。只断言 store 里有一个版本 + 目录,会在一个装 A 而 `xpkg_dir` 答 B 的实现上通过;只断言答案,会在一个装两份的 + 实现上通过(`tests/e2e/627_one_package_one_version.sh`)。 +7. §4.5 的拒绝判据**必须**带反向腿:把钉抬到满足要求后同一份工程构建通过。否则一个 + 「凡工程与依赖同时声明同一个包就拒绝」的实现也会通过 + (`tests/e2e/628_a_pin_below_a_stated_floor_is_refused.sh`)。 ## 变更记录 @@ -258,3 +298,4 @@ feature-deps feature-xlings ← 限定词是门 |---|---|---| | 1.0 | 2026-09-07 | 首版。平面(§2)、条件化唯一形状(§3)、两条解析轴(§4)、命名规约(§5)、条件化准入(§6)。目标轴列为未实现。 | | 1.1 | 2026-09-07 | 目标轴落地(mcpp 2026.9.6.4):§4.3.1 工具 selector 禁止命名目标侧层;`[target..xlings…]` 与 `[target..feature-xlings.]` 转为已实现;§4.3 补两条轴同时命名一个包时的取舍与按包去重;§4.4 转为已实现;§7 补第 4 条判据。 | +| 1.2 | 2026-09-07 | 一个包一个版本(mcpp 2026.9.6.6):新增 §4.5(身份=`(namespace, name)`,版本是约束;裁决与校验两步;范围必须双向可解且可被 `xpkg_dir` 回答);§4.3.1 改为「禁止命名**被解析的**层」,`accelerator` 明确被接受(2026.9.6.5);§7 补第 5 条的反向腿与第 6、7 条判据。 | diff --git a/docs/specs/package-identity.md b/docs/specs/package-identity.md index 8f67f986..28f83a52 100644 --- a/docs/specs/package-identity.md +++ b/docs/specs/package-identity.md @@ -428,10 +428,35 @@ lua = "0.0.3" --- -## 10. 变更记录 +## 10. 工具地址:同一条身份规则(2026.9.6.6+) + +`[xlings.workspace]` / `[xlings] deps` / `[feature-xlings.]` 的条目寻址的是 xim 侧的 +包,写法是 `[:][@]`。**它们适用同一条身份规则:身份是 +`(namespace, name)`,版本不是身份的一部分。** 命名空间缺省为 `xim`。 + +这一条此前只在 mcpp 的 `[dependencies]` 侧成立。工具侧有过两套定义:条件合并按包去重, +而跨包的划分比对整个地址串。于是工程钉 `xim:cuda-nvcc@13.3.33`、规则包钉 +`@>=12.9.86` 时,系统认为那是两个包,把两份都装下来——每份数 GB——而 `xpkg_dir` 只 +回答其中一个。**装了两份、用了一份,没有任何一句话提到它。** + +统一之后: + +- 版本是这个包上的**约束**,「钉具体版本」与「钉范围」是同一机制的两种取值; +- 同一身份在一次构建里**只安装一个版本**,由裁决与校验两步决定(SPEC-004 §4.5); +- `mcpp::xpkg_dir` 必须能回答范围——安装了却答「不存在」,是让规则包无法声明下界的 + 那个缺口。 + +**边界不变(§1.2)。** 范围的**求解**仍由 xlings 执行:mcpp 把地址原样交出去,并按 +已安装的版本目录回答查询。mcpp 这边新增的只有「哪一条声明胜出」与「胜出的那条是否 +满足其余要求」,两者都不需要知道索引里有哪些版本。 + +--- + +## 11. 变更记录 | 版本 | 日期 | 变更 | |---|---|---| +| 1.3 | 2026-09-07 | 新增 §10:同一条身份规则扩展到 xlings 工具地址(mcpp 2026.9.6.6)。此前工具侧有两套定义,同一个包被两处以不同版本声明时两份都装、只用一份 | | 1.2 | 2026-08-09 | selector 收敛为唯一精确 PackageId:裸名只表示默认 mcpplibs,dotted 以最后一段为 name;移除 compat/空 namespace 隐式候选,加入 lock 保持与一个 release train 的双 selector 迁移 warning | | 1.1 | 2026-08-03 | 按当前实现复核:澄清文件名发现是快路径加身份回退扫描,修正 legacy `package.name` 的 wire key 示例,并将 0.0.106 明确为最低实现版本 | | 0.1 | 2026-07-25 | 首版草案。整合 #278 的双向验证结论:确立「身份 = `(namespace, name)`、层级归 `namespace`、`name` 为原子段」为规范形态,并如实标注 0.0.105 的过渡形态(强制 FQN)与全部待实现项 | diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index 4e603498..f7a5892a 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -1911,6 +1911,75 @@ hardware = {} `[features]` 里没有声明过的 feature 名会作为 schema 警告报出:它对谁都不激活、什么 都不装,而这种工具的缺席只表现为「设备就是连不上」,是最难诊断的一种。 +#### 规则包自带它的环境(2026.9.6.6+) + +上面那张表是工程**有主张**时写的。多数工程没有主张,写下的也就是空无一物: + +```toml +[build-dependencies.mcpp] +plugins = { version = "0.2.4", features = ["rules-cuda"], host-module = true } +``` + +这一条边就是全部声明。规则包在选中它的那个 feature、它所服务的加速器之下,声明自己 +需要哪些包、最低到哪一版: + +```toml +# 写在规则包里,不写在你的工程里 +[target.'cfg(accelerator = "cuda")'.feature-xlings.rules-cuda] +"xim:cuda-nvcc" = ">=12.9.86" +"xim:cuda-cudart" = ">=12.9.79" +``` + +**两重门,都要开。** feature 说「要不要这个规则」,selector 说「哪些构建真的下载」。 +同一个工程的 CPU-only 构建两道门都不过,一个字节都不装。 + +「需要哪些包、最低到哪一版」是规则作者的知识。在每个用它的工程里重写一遍,是一份会 +悄悄过期的副本——规则动了,而那些工程不会跟着动。 + +#### 一个包一个版本(2026.9.6.6+) + +工具地址是 `[:][@<版本>]`,它的**身份是 `(namespace, name)` 二元组**。版本是 +这个包上的约束,不是它名字的一部分,所以 `xim:glibc`、`xim:glibc@2.40` 与 +`xim:glibc@>=2.38` 指的是同一个包。**一次构建只装它的一个版本。** + +装哪一个,分两步决定。 + +**裁决——离产物更近的声明赢。** 工程压过它依赖的包,于是一个钉覆盖规则的要求: + +```toml +# 工程侧,当它确实有主张时 +[target.'cfg(accelerator = "cuda")'.xlings.workspace] +"xim:cuda-nvcc" = "13.3.33" +``` + +**不带版本的声明弃权:** 它陈述了「要这个包」而没有陈述「要哪一版」,因此不会仅仅 +因为更近就压过一条下界。两条声明都带版本且不一致时,mcpp 会报出用了哪一条——一个 +只能表现为「声明了两个版本而目录里有一个」的覆盖,是要读者自己去文件系统里重建的 +事实。 + +**校验——赢家必须满足每一条落败的要求。** `>=`、`^`、`~` 以及逗号组合是**要求**。 +不满足的钉被拒绝,并同时点出两侧: + +``` +error: `xim:cuda-nvcc` is pinned to 12.0.0 by this project, and mcpp:plugins + requires >=12.9.86. + One version of a package is installed, so the two cannot both hold. + fix: pin a version satisfying >=12.9.86, or drop the pin and let the + requirement decide. +``` + +裸版本是**选择**而不是要求:两条互不相同的精确钉走裁决并被报告,不被拒绝。只有被 +陈述出来的要求才谈得上违反。 + +**这是一次比较,不是一次搜索。** 版本由裁决选定、再被检查,所以 mcpp 从不需要问索引 +「有哪些版本」,也就不带约束求解器。代价被写出来而不是藏起来:一个求解器本可满足的 +组合——工程写 `>=8.0`、规则写 `8.5.0`、而索引里最新是 8.3——会被拒绝,而拒绝消息里 +写着怎么往下走。 + +范围在两个方向上都被求解。`>=2026.1` 装到满足它的最高已发布版本,`>=2099.1` 作为 +不可满足被拒绝,而 `mcpp::xpkg_dir` 回答满足该范围的最高**已安装**版本——声明了下界 +的规则找得到下界带进来的东西。 + #### 工程没点名的工具,其版本的来源 | 工程声明了 | 版本来自 | diff --git a/docs/zh/07-build-mcpp.md b/docs/zh/07-build-mcpp.md index ef2aaf4c..59e5d8d4 100644 --- a/docs/zh/07-build-mcpp.md +++ b/docs/zh/07-build-mcpp.md @@ -110,7 +110,7 @@ int main() { | `mcpp::link_flag(s)` *(2026.9.6.5+)* | `mcpp:link-flag=` | | `mcpp::link_script(p)` *(2026.8.19+)* | `mcpp:link-script=` | | `mcpp::runner(tok)` *(2026.8.19.2+)* | `mcpp:runner=` —— 见下 | -| `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | 本 manifest 在 `[xlings.workspace]` 里声明的包的载荷目录;没声明或没安装时返回 `""`(见下) | +| `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | `[xlings.workspace]` 里声明的包的载荷目录 —— 本 manifest 声明的,或编进本构建程序的某个依赖声明的(2026.9.6.6+);没声明或没安装时返回 `""`(见下) | | `mcpp::warning(text)` *(2026.8.21.2+)* | `mcpp:warning=` —— 见下 | | `mcpp::action{…}.submit()` *(2026.8.5.1+)* | `mcpp:action=` —— **声明一个构建图节点**,而不是在这里把活干了(见下) | @@ -289,6 +289,14 @@ store 内部结构 —— 与 `dep_dir` 存在的理由相同。 **带版本固定**的引用只解析到那个版本,否则什么都不返回。请求 `1.8.12` 却静默拿 到 `1.9.0`,是那种要到产物里才被发现的答案。 +**带约束**的引用(`>=8.5.0`、`^1.2`)解析到满足它的**最高已安装版本** +(2026.9.6.6+)。在那之前整个版本位是拿去与目录名比对的,于是一条范围装上了载荷,然后 +回答「没装」—— 这正是规则包无法声明下界、而每个工程都要把规则的包列表重写一遍的原因。 + +**依赖声明的包同样被作答**(2026.9.6.6+),而且答的是这次构建**真正装上**的版本,不是 +本地 manifest 写下的那个。一个包只有一个版本:工程与规则都命名它时,离产物更近的声明赢, +而两侧被告知同一个答案。见 [05 — mcpp.toml](05-mcpp-toml.md) 的「一个包一个版本」。 + **`[feature-xlings.]` 在 `` 生效时同样被作答**(2026.9.6.2+)。这张表从诞生 起就参与供给 —— 在那里写下一个包,它就会被下载并安装 —— 但构建程序的环境只由 `[xlings.workspace]` 填充,于是载荷明明在盘上,`xpkg_dir` 却返回 `""`。这时构建程序 diff --git a/docs/zh/11-machine-output.md b/docs/zh/11-machine-output.md index 39d834bb..00920ab8 100644 --- a/docs/zh/11-machine-output.md +++ b/docs/zh/11-machine-output.md @@ -334,6 +334,10 @@ mcpp why toolchain [--target ] [--toolchain ] --format json | `exclusive-capability` | 一个能力有多个提供者,而其中至少一个声明了独占 | | `version-floor-unmet` | 一个包对机器的要求高于机器被声明拥有的 | | `accel-mismatch` | 一条 `[build] sources` 条目被约束到本次构建未覆盖的设备集合 | +| `accel-backend-undeclared` | 一条 `[build] sources` 条目命名了本包 `[package] accelerators` 未列出的后端 | +| `device-source-unconsumed` | 一个设备类源文件没有到达任何 action,因此没有东西编译它 | +| `host-module-missing` | `build.mcpp` 导入了没有任何依赖以 host module 形式提供的模块 | +| `tool-version-conflict` | 两处声明把同一个 xlings 包定在不能同时成立的版本上 | | `other` | 一处还没有被命名的拒绝分支 | **只要问题被回答了就退 0,包括答案是「拒绝」。** 「它能不能构建,不能的话 diff --git a/docs/zh/17-the-project-environment.md b/docs/zh/17-the-project-environment.md index 66359452..f85458f8 100644 --- a/docs/zh/17-the-project-environment.md +++ b/docs/zh/17-the-project-environment.md @@ -79,8 +79,17 @@ SubOS 里。前置让被声明的环境成为默认答案;其余的仍在它后 的数据文件(比如 protoc 自带的 well-known `.proto`)的构建程序问目录,需要 **运行**某个程序的构建程序问 `PATH`。 -依赖自己的 `[xlings]` 声明从不被读取也不被传播。工作区构建中由工作区根持有这个 -选择;成员的声明只在该成员作为独立根被构建时生效。 +**工作区成员的声明不是工作区的声明。** 工作区构建中由工作区根持有这个选择;成员的 +`[xlings]` 只在该成员作为独立根被构建时生效。 + +**依赖的声明是另一回事,而且它被采纳**(`[xlings] deps` 自 2026.9.5.4,下面那条版本 +规则自 2026.9.6.6)。板级支持包知道哪个模拟器够得到它那台机器,规则包知道它驱动哪个 +工具包;要消费者把这些再写一遍,正是这类包存在的意义所反对的重复。依赖声明的东西会被 +装上,而 `MCPP_XPKG__DIR` 在那个依赖自己的构建程序里为它作答。 + +工程与依赖命名**同一个包**时,只装它的一个版本:身份是 `(namespace, name)`,版本是这个 +包上的约束。离产物更近的声明赢,并且覆盖会被报出来;不满足对方所陈述之要求的钉会被拒绝 +并点出两侧。见 [05 — mcpp.toml](05-mcpp-toml.md) 的「一个包一个版本」。 ## 4. 只读取环境,从不创建环境 diff --git a/docs/zh/20-heterogeneous-builds.md b/docs/zh/20-heterogeneous-builds.md index 0c9b358a..3bbbae6a 100644 --- a/docs/zh/20-heterogeneous-builds.md +++ b/docs/zh/20-heterogeneous-builds.md @@ -18,9 +18,13 @@ accel = "cuda12.9+{sm_89}, vulkan1.2" (`cuda12.9+{sm_89} ptx>=89` 是一个后端加一组架构再加一条 PTX 下界)。源码集合怎么写 见下面「一个构建里的多个后端」。 -今天有规则包的编程模型是四个 —— CUDA、HIP、SYCL、Vulkan/SPIR-V —— 「各条 lane」那张表 -写明每个驱动哪个编译器、需要哪些载荷。引擎里不持有任何厂商名字,所以第五个是**一个包** -而不是一次引擎改动。 +`accelerator = "none"` 是**空集** —— 恰好在本次构建完全没有命名加速器时为真。它是 +词表里唯一一个不是后端名字的取值,而它之所以存在,是因为词表的其余部分是**开放**的: +一个会增长的集合不能靠列举来否定。见下面「CPU 回退」。 + +今天有规则包的编程模型是五个 —— CUDA、HIP、SYCL、Vulkan/SPIR-V、Ascend C —— +「各条 lane」那张表写明每个驱动哪个编译器、需要哪些载荷。引擎里不持有任何厂商名字, +所以第六个是**一个包**而不是一次引擎改动。 ## 两种形态,以及为什么一套机制够到两者 @@ -115,6 +119,48 @@ tarball 已经发出去了。设备源文件必须被显式点名。 这个划分是刻意的。mcpp 拥有构建图、产物身份与架构集合;厂商的 flag 拼法、 架构语法与宿主编译器要求属于规则包。 +## 工具包跟着规则来(2026.9.6.6+) + +一个要 CUDA 岛的工程只写一条边: + +```toml +[build-dependencies.mcpp] +plugins = { version = "0.2.4", features = ["rules-cuda"], host-module = true } +``` + +**别的什么都不写。** 厂商工具包、它的运行时,以及规则需要的其余东西,都由**规则包** +在选中它的那个 feature、它所服务的加速器之下声明: + +```toml +# 写在 mcpp:plugins 里,不写在你的工程里 +[target.'cfg(accelerator = "cuda")'.feature-xlings.rules-cuda] +"xim:cuda-nvcc" = ">=12.9.86" +"xim:cuda-cudart" = ">=12.9.79" +"xim:libcurand" = ">=10.3.10.19" +"xim:cuda-cccl" = ">=12.9.27" +``` + +**两重门,在下载第一个字节之前都要开。** feature 说「要不要这个规则」, +`cfg(accelerator = ...)` 说「哪些构建真的需要设备工具包」。不带加速器的 `mcpp build` +两道门都不过——这正是让最便宜的那次构建保持便宜的原因,而那是 CI 跑的那一次。 + +每条 lane 都是这样:`rules-cuda`、`rules-hip`、`rules-sycl`、`rules-spirv`、 +`rules-ascendc` 各自带着自己的环境。**不存在需要工程自己补环境的 lane**,因为「有时候 +需要」是一条没有任何地方能说清楚到可用的规则。 + +**覆盖是工程里的一行。** 哪个包、最低到哪一版,是规则作者的知识;**具体哪一版**有时 +属于工程——一条 CUDA 线耦合着驱动下界,而那是产物将要运行的那批机器的属性: + +```toml +[target.'cfg(accelerator = "cuda")'.xlings.workspace] +"xim:cuda-nvcc" = "13.3.33" +``` + +更近的声明赢,装一个版本,并且 mcpp 说出用了哪条。不满足规则下界的钉会被拒绝并点出 +两侧,而不是与它并排装下来。完整规则见 [05 — mcpp.toml](05-mcpp-toml.md) 的 +「一个包一个版本」;`examples/09-heterogeneous/multi-backend` 是本仓库里唯一走覆盖 +路径的示例,其余每一个都只写那条边。 + ## 规则包在第一次编译之前报告的事 设备工具包有三件事出错得很晚,而没有一件是关于构建图的事实。它们由驱动这些工具的 @@ -289,15 +335,31 @@ ld: obj/src/cpu/impl.o: multiple definition of `impl'; obj/src/cuda/impl.o: first defined here ``` -响,而且发生在链接期而不是运行期 —— 但这是 manifest 本可以避免的失败。否定整个集合: +响,而且发生在链接期而不是运行期 —— 但这是 manifest 本可以避免的失败。直接说 +「本次构建没有命名任何后端」: ```toml -[target.'cfg(not(any(accelerator = "cuda", accelerator = "vulkan")))'.build] +[target.'cfg(accelerator = "none")'.build] sources = ["src/cpu/*.cpp"] ``` -`any` / `all` / `not` 作为普通布尔组合子在成员判定之上组合。代价是这份名单成了后端集合 -被写下的**第二个地方**:加第三个后端时忘了改这一行,拿到的就是上面那条链接错误。 +`none`(mcpp 2026.9.6.5)是**空集**:它在本次构建完全没有命名加速器时成立,并且在生态 +长出新后端之后**继续**成立。 + +**它取代的那种枚举写法当时是对的,但不会一直对。** `none` 之前唯一的拼法是否定整个集合, + +```toml +[target.'cfg(not(any(accelerator = "cuda", accelerator = "vulkan")))'.build] +``` + +于是这一行成了**后端集合被写下的第二个地方**。`any` / `all` / `not` 仍然作为普通布尔 +组合子在成员判定之上组合,也确实有需要它们的谓词 —— 但这里不是。`accelerator` 是一个 +**开放**词表:第三个后端是一个**包**而不是一次引擎改动,所以它出现的那天,没人记得去 +扩的那份枚举就不再表示它写着的意思。而且它**不会响**:视乎忘掉的是哪一半,得到的要么 +是给有设备的构建静默选上了 CPU 实现,要么是上面那条链接错误。 + +`cfg(not(accelerator = "none"))` 是同一句话的反面 —— 「本次构建至少命名了一个后端」 +—— 这正是多个后端共用的分发器应当条件化的东西。 ### 另一种形态:不在链接期做选择 @@ -308,7 +370,7 @@ sources = ["src/cpu/*.cpp"] 源码,CPU 实现无条件在内; * 给每个实现一个不同的名字,让接缝在**运行期**去问哪些设备在场。 -这样就没有 `not(any(...))` 要维护,而产物回答的是二进制的使用者真正的问题。ggml 就是 +这样就完全没有排除关系要维护,而产物回答的是二进制的使用者真正的问题。ggml 就是 这个形态:每个后端自己注册,程序运行时由 `ggml_backend_reg_by_name` 选一个。 `ggml-org:llamacpp` 就是这样构建的,它的 `backend-vulkan` 相对 `backend-cpu` 是**可加的** 而不是互斥的。 @@ -337,12 +399,16 @@ sources = ["src/cpu/*.cpp"] `tests/unit/test_core_vendor_probes.cpp` 断言剥掉注释后 `src/` 里不出现任何厂商工具名, 并自带分母(枚举到的文件数)。 -| `mcpp:plugins` 的 feature | 模块 | 它驱动的编译器 | 它需要的载荷 | `[build] accel` | +| `mcpp:plugins` 的 feature | 模块 | 它驱动的编译器 | 它声明的载荷 | `[build] accel` | |---|---|---|---|---| | `rules-cuda` | `mcpp.rules.cuda` | 工程自己的 clang(`-x cuda`),或 GCC 工具链下的 nvcc | `xim:cuda-nvcc`、`xim:cuda-cudart`、`xim:libcurand`、`xim:cuda-cccl` | `cuda12.9+{sm_89} ptx>=89` | | `rules-hip` | `mcpp.rules.hip` | NVIDIA 平台上是工程自己的 clang(`-x cuda`) | 上面那些,再加 `xim:hip-nvidia` | `hip, cuda12.9+{sm_89}` | | `rules-sycl` | `mcpp.rules.sycl` | `xim:dpcpp` 载荷里的 clang(`-fsycl`) | `xim:dpcpp`、`xim:gcc`,NVIDIA 目标另加 `xim:cuda-nvcc` | `sycl` 或 `sycl, cuda12.9+{sm_89}` | | `rules-spirv` | `mcpp.rules.spirv` | `glslangValidator` 或 `glslc` | `xim:glslang` 或 `xim:shaderc` | `vulkan1.2` | +| `rules-ascendc` | `mcpp.rules.ascendc` | CANN 工具包里的 `bisheng`(`-x asc`) | `xim:cann-toolkit` | `ascend8.5+{dav-c220}` | + +载荷那一列是每条规则在 `cfg(accelerator = ...)` 之下**为自己**声明的东西,列出来是为了 +让一条 lane 的代价在选它之前就可读。工程一个字都不用写——见上文「工具包跟着规则来」。 HIP 与 SYCL 两行的 `accel` 值是**两段**而不是一段。第一段命名编程模型,第二段命名设备, 于是一个设备在本生态里只有一种拼法,无论有多少个模型去够它:`sm_89` 无论被哪条规则读到 @@ -360,7 +426,8 @@ CUDA 入口点的内联包装,所以目标文件链接的是 CUDA 运行时,机 ## 在此之上,一个框架是什么形状 -四条 lane 证明了规则包能驱动四个编译器。框架是下一个问题 —— 这套机制能不能扛起一个 +五条 lane 证明了规则包能驱动五个编译器,其中最新的一个来自 NVIDIA 与 Khronos 两个 +谱系之外的厂商。框架是下一个问题 —— 这套机制能不能扛起一个 真会被部署的东西 —— 而它自己有一个形状,是在 llama.cpp 的 Vulkan 后端上量出来的。 **框架自带生成器,生态应当驱动它而不是替换它。** llama.cpp 的着色器由一个与后端放在 diff --git a/examples/09-heterogeneous/README.md b/examples/09-heterogeneous/README.md index 1f1898c2..df8c2104 100644 --- a/examples/09-heterogeneous/README.md +++ b/examples/09-heterogeneous/README.md @@ -1,10 +1,10 @@ -# 09 — Heterogeneous builds: one lesson, four programming models +# 09 — Heterogeneous builds: one lesson, several programming models -Four subdirectories compute the same thing — `2.0 * [1,2,3,4] + [10,20,30,40]`, +Each subdirectory computes the same thing — `2.0 * [1,2,3,4] + [10,20,30,40]`, printed as `12 24 36 48` — on a device, and the same thing on the CPU when -there is no device. They are not four lessons. They are one structure -instantiated four times, and reading any two of them shows which parts belong -to the structure and which to a vendor. +there is no device. They are not several lessons. They are one structure +instantiated several times, and reading any two of them shows which parts +belong to the structure and which to a vendor. | directory | model | device compiler | what it adds to the structure | |---|---|---|---| @@ -12,8 +12,17 @@ to the structure and which to a vendor. | [`vulkan/`](vulkan/) | Vulkan compute | glslang or shaderc, to SPIR-V | a device output that is a header rather than an object, and one artifact that runs on three devices | | [`sycl/`](sycl/) | SYCL | the dpcpp payload's clang | a second compiler with its own standard library, and a chained action for the device link | | [`hip/`](hip/) | HIP | the project's own clang, NVIDIA platform | a programming model that is a header layer over another model's runtime | +| [`cann/`](cann/) | Ascend C | the toolkit's BiSheng (`-x asc`) | a device object for hardware nobody in this repository has, and a host half that declines cleanly | +| [`multi-backend/`](multi-backend/) | CUDA **and** Vulkan | both of the above | the other shape: backends that are additive rather than a seam, chosen at run time | -Start with `cuda/`. The other three assume it. +Start with `cuda/`. The four beside it assume it, and `multi-backend/` assumes +two of them. + +The first five are one shape — a **seam**: exactly one implementation exists in +the artifact and the choice was made at build time. `multi-backend/` is the +other — several implementations in one artifact, chosen when the program runs. +A program can take either; a library that is compiled once and consumed by +people whose machines differ can only take the second. ## The structure @@ -62,11 +71,26 @@ anything is compiled. The two variants land in different artifact directories because the device axis is part of the build's identity, so alternating between them does not rebuild from scratch. -**A rule package.** Every vendor spelling — `--cuda-gpu-arch`, `-gencode`, -`--target-env`, `-fsycl-targets`, `-fsycl-link` — lives in `mcpp:plugins`, a -package the project depends on and selects features from. The engine owns the -graph, the artifact's identity and the accelerator axis; it holds no vendor -name. A unit test refuses vendor probes in `src/`. +**A rule package, which brings its own environment.** Every vendor spelling — +`--cuda-gpu-arch`, `-gencode`, `--target-env`, `-fsycl-targets`, `-fsycl-link`, +`--cce-aicore-arch` — lives in `mcpp:plugins`, a package the project depends on +and selects features from. The engine owns the graph, the artifact's identity +and the accelerator axis; it holds no vendor name. A unit test refuses vendor +probes in `src/`. + +The rule also declares the **payloads** it needs, under the feature that +selects it and the accelerator it serves, so a project writes one edge and no +`[xlings.workspace]` block: + +```toml +[build-dependencies.mcpp] +plugins = { version = "0.2.4", features = ["rules-cuda"], host-module = true } +``` + +`multi-backend/` is the one example here that also pins a version, and it does +so to demonstrate the override: the rule owns "which package, and no older than +what", the project owns "and exactly this one". One version is installed either +way — see *One package, one version* in `docs/05-mcpp-toml.md`. ## The layers underneath diff --git a/examples/09-heterogeneous/cann/app/README.md b/examples/09-heterogeneous/cann/app/README.md index 0447fd15..16814fd4 100644 --- a/examples/09-heterogeneous/cann/app/README.md +++ b/examples/09-heterogeneous/cann/app/README.md @@ -27,9 +27,18 @@ Measured on an x86_64 machine with **no Ascend hardware and no Ascend driver**: | the kernel compiles | `bisheng -x asc --cce-aicore-arch=dav-c220` | | the object joins the ordinary link | mixed mode: an x86-64 object carrying the device binary | | the host half links | ACL, plus the six-library closure the rule names | +| the runtime-closure check | names `libascend_hal.so` and nothing else (2026.9.6.6+) | | the artifact starts | **no** -- `libascend_hal.so` is missing | | `--no-accel` | builds and runs: `12 24 36 48`, `device: cpu` | +The closure row is a measurement that once disagreed with the loader. Before +2026.9.6.6 mcpp named **eight** libraries, and the artifact it had just linked +resolved seven of them: the toolkit's shared libraries depend on each other by +bare SONAME and carry no search path, while the directory holding them is named +once, in the executable's `DT_RPATH`. The model searched only the requesting +object's own list, and `DT_RPATH` is inherited down the whole chain. The engine +now models that, so the refusal names exactly what the loader will fail on. + `libascend_hal.so` belongs to the **driver**, not the toolkit, and is the role `libcuda.so.1` plays for CUDA: in ABI lockstep with the kernel module, not redistributable, and absent on a machine with no NPU. A device build of this diff --git a/examples/09-heterogeneous/hip/app/mcpp.toml b/examples/09-heterogeneous/hip/app/mcpp.toml index 015ac711..842c9cdf 100644 --- a/examples/09-heterogeneous/hip/app/mcpp.toml +++ b/examples/09-heterogeneous/hip/app/mcpp.toml @@ -3,7 +3,10 @@ name = "hip-saxpy" namespace = "example" version = "0.1.0" description = "A HIP kernel behind a seam module, reaching an NVIDIA device, with a CPU fallback" -accelerators = ["hip"] +# BOTH, because the constrained glob below names both. On the NVIDIA platform +# HIP compiles THROUGH the CUDA back end, so `cuda` is a backend this package's +# sources are written for even though nothing in them says the word. +accelerators = ["hip", "cuda"] [language] standard = "c++23" diff --git a/examples/09-heterogeneous/sycl/app/mcpp.toml b/examples/09-heterogeneous/sycl/app/mcpp.toml index e9af3ad9..c41b3204 100644 --- a/examples/09-heterogeneous/sycl/app/mcpp.toml +++ b/examples/09-heterogeneous/sycl/app/mcpp.toml @@ -3,7 +3,10 @@ name = "sycl-saxpy" namespace = "example" version = "0.1.0" description = "A SYCL kernel behind a seam module, reaching an NVIDIA device, with a CPU fallback" -accelerators = ["sycl"] +# BOTH, for the reason the HIP example records: the glob below names the NVIDIA +# target the SYCL unit is compiled for, so `cuda` is a backend this package's +# sources are written for. +accelerators = ["sycl", "cuda"] [language] standard = "c++23" diff --git a/mcpp.toml b/mcpp.toml index d2d091b8..6655a311 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.6.5" +version = "2026.9.6.6" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index fc835a05..f41fd6e1 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.6.5"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.6.6"; } // namespace mcpp diff --git a/modules/versioning/src/version_req.cppm b/modules/versioning/src/version_req.cppm index b72976a9..fac07d94 100644 --- a/modules/versioning/src/version_req.cppm +++ b/modules/versioning/src/version_req.cppm @@ -128,6 +128,21 @@ struct Requirement { std::expected parse_req(std::string_view s); +// Does this string ASK for a version, or NAME one? +// +// The distinction is not decidable from `parse_req` alone: `parse_req("1.2.3")` +// succeeds, so "does it parse" cannot separate a constraint from a literal. The +// separation matters wherever a version position doubles as an address — +// mcpp's package resolution treats a bare version as the exact wire address and +// only a constraint as something to solve, and an installed-payload lookup must +// not answer a pinned request with a different version that happens to be +// present. +// +// Spelled here rather than beside either caller because it is a fact about THIS +// grammar: the set of leading operators is the one `parse_req` accepts, and a +// second copy would drift the first time an operator is added. +bool is_constraint(std::string_view s); + bool matches(const Requirement& r, const Version& v); // Indices of ALL versions in `available` that match `req` and tie for highest @@ -319,6 +334,14 @@ bool prerelease_visible(const Requirement& r, const Version& v) { } // namespace +bool is_constraint(std::string_view s) { + if (s.empty()) return true; // "any version" + if (s == "*") return true; + const char c = s.front(); + if (c == '^' || c == '~' || c == '>' || c == '<' || c == '=') return true; + return s.find(',') != std::string_view::npos; // AND-combined parts +} + std::expected parse_req(std::string_view s) { s = strip_ws(s); Requirement r; diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index f0706959..ce9d3ec5 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -874,6 +874,8 @@ std::expected run_build_program( if (!importable.empty()) importable += ", "; importable += hm.logical; } + mcpp::build::refusal::record( + mcpp::build::refusal::Code::HostModuleMissing); return std::unexpected(std::format( "build.mcpp imports '{}', and no dependency provides it as a " "host module.\n" diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index db1097fe..2974f6e2 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -19,6 +19,7 @@ import std; import mcpp.targetside; import mcpp.diag; import mcpp.build.refusal; +import mcpp.xlings.address_set; import mcpp.build.version_floor; import mcpp.home; import mcpp.platform.axis; @@ -261,8 +262,13 @@ materialize_generated_files(const std::filesystem::path& root, export void merge_conditional_xlings(mcpp::manifest::Manifest& m, const mcpp::manifest::ConditionalConfig& cc) { + // ONE DEFINITION OF IDENTITY, and it is not local to this merge. It used + // to be `parse_address(a).target` — the bare name, so `xim:cuda` and a + // hypothetical `scode:cuda` collided, and the graph split a few thousand + // lines below compared whole address strings instead. See + // mcpp.xlings.address_set for what the two definitions cost. auto package_of = [](std::string_view address) { - return mcpp::manifest::parse_address(address).target; + return mcpp::xlings::addrset::package_key(address); }; for (auto const& a : cc.xlings.deps) { const auto pkg = package_of(a); @@ -3783,11 +3789,28 @@ prepare_build(bool print_fingerprint, // declared. The graph's own declarations are provisioned after // resolution, which is the first moment they are known — see the // second pass near `xlingsDepBinDirs`. - const auto declaredDeps = applicable_xlings_addresses( - runtimeOwnerManifest, - feature_closure(runtimeOwnerManifest, - parse_feature_request(overrides.features)), - toolPurpose, /*isRoot=*/true); + // ONE PACKAGE, ONE VERSION, INSIDE ONE MANIFEST TOO. The + // conditional merge already unified the two tool AXES by package; + // what it cannot see is `[xlings.workspace]` and + // `[feature-xlings.]` naming one package at two versions, which + // reaches here as two addresses and used to install both. + std::vector rootClaims; + for (auto const& spec : applicable_xlings_addresses( + runtimeOwnerManifest, + feature_closure(runtimeOwnerManifest, + parse_feature_request(overrides.features)), + toolPurpose, /*isRoot=*/true)) + rootClaims.push_back({spec, "this project", 0}); + auto rootUnified = mcpp::xlings::addrset::unify(rootClaims); + if (!rootUnified) { + refusal::record(refusal::Code::ToolVersionConflict); + return std::unexpected(rootUnified.error()); + } + for (auto const& note : rootUnified->overrides) + mcpp::diag::warning("xlings/version-override", note); + std::vector declaredDeps; + for (auto const& w : rootUnified->winners) + declaredDeps.push_back(w.address); if (materializeRootRuntime && !declaredDeps.empty()) { if (auto pv = provision_xlings_addresses( **cfg2, declaredDeps, runtimeSelection.ownerRoot, @@ -3837,26 +3860,71 @@ prepare_build(bool print_fingerprint, // rule. std::vector> activeFeaturesByPackage; + // WHICH VERSION OF EACH TOOL PACKAGE THIS BUILD USES, decided once. + // + // Keyed by `(namespace, name)` — the identity, with the version treated as + // a constraint on it. Filled by the first `graph_xlings_split()` below and + // read by `fillXpkgDirs`, because those two answer the same question from + // different ends: one decides what is installed, the other tells a build + // program where it landed. They used to derive it separately, and the + // failure that produced is the quiet one — installed A, answered B. + std::map xlingsWinner; + // The split is computed HERE and reused by the late pass, so the two // cannot disagree about what "the graph declared" means. - auto graph_xlings_split = [&] { - std::vector rootSpecs = applicable_xlings_addresses( - runtimeOwnerManifest, activeFeaturesByPackage.empty() - ? std::vector{} : activeFeaturesByPackage[0], - toolPurpose, /*isRoot=*/true); - std::vector fromGraph; + // + // ONE PACKAGE, ONE VERSION. This used to compare whole address strings, so + // `xim:cuda-nvcc@13.3.33` from the project and `xim:cuda-nvcc@>=12.9.86` + // from a rule package were two packages: both installed, gigabytes each, + // and `xpkg_dir` answered one of them. The unification below adjudicates + // (the declaration nearer the artifact wins) and validates (the winner must + // satisfy every requirement that lost) — see mcpp.xlings.address_set. + auto graph_xlings_split = [&]() -> std::expected< + std::pair, std::vector>, + std::string> { + namespace addrset = mcpp::xlings::addrset; + auto describe = [&](std::size_t i) { + auto const& pkg = packages[i].manifest.package; + return pkg.namespace_.empty() ? pkg.name + : pkg.namespace_ + ":" + pkg.name; + }; + std::vector claims; + for (auto const& spec : applicable_xlings_addresses( + runtimeOwnerManifest, activeFeaturesByPackage.empty() + ? std::vector{} : activeFeaturesByPackage[0], + toolPurpose, /*isRoot=*/true)) + claims.push_back({spec, "this project", 0}); + // THE BUCKET IS DECIDED BY WHERE THE WINNING CLAIM SITS IN THIS LIST, + // not by its distance. The root's own pass provisions exactly the + // addresses collected above; anything else has to reach the graph pass + // or nothing installs it. Those two lists are the same one whenever the + // project is its own runtime owner, and differ under a workspace. + const std::size_t rootClaims = claims.size(); for (std::size_t i = 0; i < packages.size(); ++i) { const auto& man = packages[i].manifest; const auto feats = i < activeFeaturesByPackage.size() ? activeFeaturesByPackage[i] : std::vector{}; for (auto const& spec : applicable_xlings_addresses( - man, feats, toolPurpose, /*isRoot=*/i == 0)) { - if (std::ranges::find(rootSpecs, spec) != rootSpecs.end()) - continue; - if (std::ranges::find(fromGraph, spec) == fromGraph.end()) - fromGraph.push_back(spec); - } - } + man, feats, toolPurpose, /*isRoot=*/i == 0)) + claims.push_back({spec, describe(i), i == 0 ? 0 : 1}); + } + auto unified = addrset::unify(claims); + if (!unified) return std::unexpected(unified.error()); + std::vector rootSpecs, fromGraph; + for (auto const& w : unified->winners) { + (w.claim < rootClaims ? rootSpecs : fromGraph).push_back(w.address); + xlingsWinner[addrset::package_key(w.address)] = w.address; + } + // REPORTED, NOT INFERRED. An override that is only visible as "two + // versions were declared and one directory exists" is a fact the reader + // has to reconstruct from the filesystem. + // + // This lambda runs twice per build (the early pass and the late one), + // and the reader sees each note ONCE: `mcpp::diag` deduplicates by the + // whole payload, which is a designed property rather than an accident + // of where these two calls sit. + for (auto const& note : unified->overrides) + mcpp::diag::warning("xlings/version-override", note); return std::pair{std::move(rootSpecs), std::move(fromGraph)}; }; packages.push_back({*root, *m}); @@ -4965,7 +5033,20 @@ prepare_build(bool print_fingerprint, auto cfg = get_cfg(); if (!cfg) return; auto xlEnv = mcpp::config::make_xlings_env(**cfg); - for (auto const& spec : declared) { + std::set answered; + for (auto const& raw : declared) { + // THE VERSION THIS BUILD INSTALLED, NOT THE ONE THIS MANIFEST + // WROTE. Both statements are about one package, and only one + // version of it exists on disk; answering from the local spelling + // is how a rule package could declare `>=8.5.0`, have the project's + // exact pin installed instead, and then be told nothing is there. + // `xlingsWinner` is empty only before the split has run, and every + // caller of this lambda runs after it — the fallback keeps that a + // fact about ordering rather than a crash. + const auto key = mcpp::xlings::addrset::package_key(raw); + if (!answered.insert(key).second) continue; + auto wit = xlingsWinner.find(key); + const std::string spec = wit == xlingsWinner.end() ? raw : wit->second; auto ref = mcpp::xlings::paths::parse_xpkg_ref(spec); auto dir = mcpp::xlings::paths::xpkg_payload(xlEnv, ref); if (!dir) continue; // declared but not installed: "" is the answer @@ -7201,6 +7282,7 @@ prepare_build(bool print_fingerprint, std::string declared; for (auto const& a : pkg.manifest.package.accelerators) declared += (declared.empty() ? "" : ", ") + a; + refusal::record(refusal::Code::AccelBackendUndeclared); return std::unexpected(std::format( "`{}`: [build] sources entry '{}' names accelerator " "backend \"{}\", which this package does not declare.\n" @@ -7320,7 +7402,12 @@ prepare_build(bool print_fingerprint, for (auto const& pkg : packages) if (auto why = layer_predicated_xlings_refusal(pkg.manifest)) return std::unexpected(*why); - auto [_rootSpecs, fromGraph] = graph_xlings_split(); + auto split = graph_xlings_split(); + if (!split) { + refusal::record(refusal::Code::ToolVersionConflict); + return std::unexpected(split.error()); + } + auto const& fromGraph = split->second; if (!fromGraph.empty()) { if (auto cfg = get_cfg()) { if (auto pv = provision_xlings_addresses( @@ -8994,6 +9081,7 @@ prepare_build(bool print_fingerprint, if (orphans.empty()) continue; std::error_code hasEc; const bool hasProgram = std::filesystem::exists(pkg.root / "build.mcpp", hasEc); + refusal::record(refusal::Code::DeviceSourceUnconsumed); return std::unexpected(std::format( "`{}`: device sources that no action compiles:\n{}" " A device-kind source is compiled by this package's build program\n" @@ -9671,7 +9759,13 @@ prepare_build(bool print_fingerprint, // provisioning that has to happen before build.mcpp; this site reads it // for the records below. Two copies of "what did the graph declare" // would be two definitions of the same word. - auto [xlingsSpecs, fromGraph] = graph_xlings_split(); + auto split = graph_xlings_split(); + if (!split) { + refusal::record(refusal::Code::ToolVersionConflict); + return std::unexpected(split.error()); + } + auto& xlingsSpecs = split->first; + auto& fromGraph = split->second; // THE ROOT'S OWN PASS RAN LONG AGO, AND THIS ONE MUST NOT REPEAT IT. // The stamp is keyed by the LIST, so provisioning root+graph together // would key a different list than the early pass wrote and re-run an diff --git a/src/build/refusal.cppm b/src/build/refusal.cppm index 519b4518..2da47aba 100644 --- a/src/build/refusal.cppm +++ b/src/build/refusal.cppm @@ -74,6 +74,25 @@ enum class Code { // the project's own sources ask for a device the build was not told to // target. AccelMismatch, + // A source glob names an accelerator backend its own package does not list + // in `[package] accelerators`. Distinct from AccelMismatch, which is about + // THIS build's device set: this one is a property of the manifest alone and + // is refused whether or not the build names any accelerator at all. + AccelBackendUndeclared, + // A device-kind source reached no action. The engine has no compile rule + // for those extensions, so a file the package's build program did not claim + // is a file nothing compiles -- reported here rather than as the undefined + // reference it used to become at the link. + DeviceSourceUnconsumed, + // `build.mcpp` imports a module no dependency supplies as a host module. + // Distinct from an ordinary missing dependency: the package may well be in + // the graph and reaching the target, and still not be compiled for the + // build program, which is what `host-module = true` asks for. + HostModuleMissing, + // Two declarations name one xlings package at versions that cannot both + // hold. Distinct from VersionFloorUnmet, which is about the machine: this + // one is about two manifests disagreeing over a tool. + ToolVersionConflict, Other, // a refusal that has not been given a code yet }; @@ -97,6 +116,12 @@ constexpr std::string_view name(Code c) { case Code::ExclusiveCapability: return "exclusive-capability"; case Code::VersionFloorUnmet: return "version-floor-unmet"; case Code::AccelMismatch: return "accel-mismatch"; + case Code::AccelBackendUndeclared: + return "accel-backend-undeclared"; + case Code::DeviceSourceUnconsumed: + return "device-source-unconsumed"; + case Code::HostModuleMissing: return "host-module-missing"; + case Code::ToolVersionConflict: return "tool-version-conflict"; case Code::Other: return "other"; } return "other"; diff --git a/src/pm/resolver.cppm b/src/pm/resolver.cppm index e59b0e54..b7df0b24 100644 --- a/src/pm/resolver.cppm +++ b/src/pm/resolver.cppm @@ -92,12 +92,9 @@ try_merge_semver(std::string_view name, namespace mcpp::pm { bool is_version_constraint(std::string_view v) { - if (v.empty()) return true; - if (v == "*") return true; - char c = v.front(); - if (c == '^' || c == '~' || c == '>' || c == '<' || c == '=') return true; - if (v.find(',') != std::string_view::npos) return true; - return false; + // One definition, in the module that owns the requirement grammar. This + // name is kept because it is what the package-manager side reads as. + return mcpp::version_req::is_constraint(v); } // ─── Namespace-aware resolve_semver (canonical, 0.0.10+) ───────────── diff --git a/src/runtime/elf.cppm b/src/runtime/elf.cppm index 5c24ad1d..5c13b2e8 100644 --- a/src/runtime/elf.cppm +++ b/src/runtime/elf.cppm @@ -395,7 +395,8 @@ std::optional resolve_needed( std::string_view soname, const ElfRuntimeFacts& requester, const mcpp::platform::runtime::RuntimeBinding& binding, - std::span additionalSearchDirs) { + std::span additionalSearchDirs, + std::span inheritedRpaths) { std::filesystem::path named(soname); std::error_code ec; if (named.has_parent_path()) { @@ -409,6 +410,26 @@ std::optional resolve_needed( std::vector dirs; for (auto const& raw : requester.runpaths) append_unique_path(dirs, expand_origin(raw, requester.artifact)); + // DT_RPATH IS INHERITED DOWN THE DEPENDENCY CHAIN; DT_RUNPATH IS NOT. + // + // The loader searches the DT_RPATH of every object on the chain that + // loaded this one, not just this object's own. Modelling only the + // requester's list reported a library as unfindable whenever a payload's + // shared libraries depend on each other and the RPATH naming their + // directory sits on the EXECUTABLE -- which is the ordinary shape for a + // vendor toolkit. Measured on examples/09-heterogeneous/cann: mcpp refused + // the build naming eight libraries, and the artifact it had just linked + // resolved seven of them and failed on the eighth, which belongs to a + // driver that machine does not have. + // + // The suppression is a property of the SEARCHING object: an object that + // carries DT_RUNPATH uses no RPATH at all, its own or inherited. `Both` is + // that case too, because glibc ignores DT_RPATH whenever DT_RUNPATH is + // present. + if (requester.searchPathTag != SearchPathTag::Runpath + && requester.searchPathTag != SearchPathTag::Both) + for (auto const& raw : inheritedRpaths) + append_unique_path(dirs, expand_origin(raw, requester.artifact)); for (auto const& dir : additionalSearchDirs) append_unique_path(dirs, dir); for (auto const& dir : binding.libraryDirs) append_unique_path(dirs, dir); // NOTE: the SubOS farm is NOT read from the binding here. @@ -910,8 +931,16 @@ RuntimeResolution resolve_runtime_closure( resolution.artifact = std::move(*root); resolution.artifactIsElf = true; - std::deque queue; - queue.push_back(resolution.artifact); + // Each queued object carries the DT_RPATHs it inherited from the chain + // that loaded it. Kept beside the facts rather than inside them: it is a + // property of HOW this object was reached, and the same file reached twice + // is one loaded object with the first arrival's chain. + struct Pending { + ElfRuntimeFacts facts; + std::vector inheritedRpaths; + }; + std::deque queue; + queue.push_back({resolution.artifact, {}}); std::set visited; visited.insert(detail::comparable_path(artifact)); // The ELF loader maintains one process-global loaded-object namespace. @@ -924,8 +953,18 @@ RuntimeResolution resolve_runtime_closure( } constexpr std::size_t kMaxClosureObjects = 512; while (!queue.empty() && resolution.objects.size() < kMaxClosureObjects) { - auto requester = std::move(queue.front()); + auto pending = std::move(queue.front()); queue.pop_front(); + const auto& requester = pending.facts; + // What this object hands to the objects it loads: its own DT_RPATH + // when that is the tag it carries, on top of whatever it inherited. + // DT_RUNPATH never propagates, and a `Both` object's DT_RPATH is dead + // to the loader, so neither contributes. + std::vector childRpaths = pending.inheritedRpaths; + if (requester.searchPathTag == SearchPathTag::Rpath) + for (auto const& raw : requester.runpaths) + if (std::ranges::find(childRpaths, raw) == childRpaths.end()) + childRpaths.push_back(raw); for (auto const& soname : requester.needed) { std::optional path; if (auto loaded = loadedBySoname.find(soname); @@ -933,7 +972,8 @@ RuntimeResolution resolve_runtime_closure( path = loaded->second; } else { path = detail::resolve_needed( - soname, requester, binding, additionalSearchDirs); + soname, requester, binding, additionalSearchDirs, + pending.inheritedRpaths); if (!path) { resolution.unresolved.push_back(soname); resolution.unresolvedSonames.push_back(soname); @@ -968,7 +1008,7 @@ RuntimeResolution resolve_runtime_closure( } loadedBySoname.emplace(parsed->soname, *path); } - queue.push_back(*parsed); + queue.push_back({*parsed, childRpaths}); resolution.objects.push_back(std::move(*parsed)); } } diff --git a/src/xlings/address_set.cppm b/src/xlings/address_set.cppm new file mode 100644 index 00000000..b2fb05be --- /dev/null +++ b/src/xlings/address_set.cppm @@ -0,0 +1,182 @@ +// mcpp.xlings.address_set — one package, one version. +// +// THE PROBLEM THIS EXISTS FOR. An xlings address is written +// `[:][@]`, and mcpp had two definitions of when two such +// strings name the same thing. The conditional merge compared the PACKAGE, so +// `xim:glibc` and `xim:glibc@2.40` were one entry with the more specific +// spelling winning. The graph split compared the whole STRING, so the same two +// were two packages: both were installed, at two versions, several gigabytes +// each for a vendor toolkit — and the answer `xpkg_dir` gave to a build program +// was whichever one a later "keep the first value for a name" rule happened to +// reach. Installed twice, answered once, and nothing said so. +// +// THE RULE. A package's identity is `(namespace, name)`. A version is always a +// CONSTRAINT on that package, never part of its name. Everything else follows: +// "pin an exact version" and "pin a range" stop being two mechanisms and become +// two values of one, and there is no second place that has to learn about +// ranges because there is no second identity. +// +// ADJUDICATION AND VALIDATION ARE DIFFERENT QUESTIONS, and conflating them is +// what makes a version solver look necessary here. +// +// - WHICH version — adjudicated by distance. The declaration nearer the +// artifact wins: a project outranks a package it depends on. A rule package +// owns "which packages, and no older than what"; the project owns "and +// exactly this one". Both are stated, one is used, and the override is +// reported rather than inferred from an install log. +// - WHETHER it holds — validated by comparison. The winner is checked against +// every REQUIREMENT that lost. That is one comparison against a known +// version, not a search through an index for a version satisfying a set, so +// the input a solver needs (which versions exist) is never consulted. +// +// A LOST EXACT PIN IS NOT A VIOLATION. Two exact pins that differ are two +// CHOICES, and choosing is what adjudication is for; the nearer one wins and +// the other is reported. A `>=`/`^`/`~` statement is a REQUIREMENT, and a +// winner that fails it is refused. The asymmetry is the point: an upgrade must +// not turn every dependency that pinned a tool into a hard failure, while a +// floor a rule package depends on must not be silently lowered. +export module mcpp.xlings.address_set; + +import std; +import mcpp.manifest; +import mcpp.version_req; + +export namespace mcpp::xlings::addrset { + +// One declaration of one address, and where it came from. +struct Claim { + std::string address; // `[:][@]`, exactly as authored + std::string declaredBy; // "this project", "mcpp:plugins" — named in messages + // 0 = the consumer's own manifest, 1 = a package the consumer depends on. + // Two levels, because that is the distinction the rule draws; a deeper + // dependency is not nearer to the artifact than a shallower one in any + // sense a manifest author can act on. + int distance = 0; +}; + +struct Winner { + std::string address; // the winning claim's address, verbatim + std::size_t claim; // index into the input, so callers can ask who won +}; + +struct Resolution { + // One winner per package, in the order the packages were first claimed. + std::vector winners; + // One sentence per disagreement that was resolved rather than refused. + // Returned rather than printed: this module has no output channel, which is + // also what makes it testable without one. + std::vector overrides; +}; + +// `[:]` with the namespace defaulted, which is the identity. +// +// The default is `xim`, matching `parse_xpkg_ref`: a manifest writing `ninja` +// and one writing `xim:ninja` name one package, and an identity that says +// otherwise would reintroduce the split this module removes. +std::string package_key(std::string_view address); + +// The version position of `address`, or empty when it names none. +std::string version_of(std::string_view address); + +// Refuses when a winner fails a requirement that lost. Never refuses for a +// spelling it cannot evaluate: an unparseable version is reported as an +// override, because a refusal manufactured from ignorance is worse than the +// duplicate install it would be preventing. +std::expected unify(std::span claims); + +} // namespace mcpp::xlings::addrset + +// ── implementation ────────────────────────────────────────────────────────── + +namespace mcpp::xlings::addrset { + +std::string package_key(std::string_view address) { + const auto e = mcpp::manifest::parse_address(address); + return (e.ns.empty() ? std::string("xim") : e.ns) + ":" + e.target; +} + +std::string version_of(std::string_view address) { + return mcpp::manifest::parse_address(address).version; +} + +namespace { + +enum class Verdict { + Holds, // the winner satisfies this statement, or the statement is silent + Differs, // two choices; the nearer one is used and the other reported + Violated, // the winner fails a stated requirement +}; + +Verdict check(std::string_view chosen, std::string_view stated) { + if (stated.empty() || stated == chosen) return Verdict::Holds; + if (!mcpp::version_req::is_constraint(stated)) return Verdict::Differs; + // A requirement. It can only be tested against a version, so a winner that + // is itself a range is not yet an answer to it. + if (mcpp::version_req::is_constraint(chosen)) return Verdict::Differs; + auto v = mcpp::version_req::parse_version(chosen); + if (!v) return Verdict::Differs; + auto r = mcpp::version_req::parse_req(stated); + if (!r) return Verdict::Differs; + return mcpp::version_req::matches(*r, *v) ? Verdict::Holds : Verdict::Violated; +} + +} // namespace + +std::expected unify(std::span claims) { + Resolution out; + std::vector order; + std::map> byPackage; + for (std::size_t i = 0; i < claims.size(); ++i) { + const auto key = package_key(claims[i].address); + auto [it, fresh] = byPackage.try_emplace(key); + if (fresh) order.push_back(key); + it->second.push_back(i); + } + + for (auto const& key : order) { + auto const& idx = byPackage[key]; + // THE WINNER IS THE NEAREST CLAIM THAT NAMES A VERSION. A claim with no + // version states that the package is wanted and nothing about which + // version, so it abstains from a question it did not answer — without + // this, a project naming a tool bare would silently drop a floor its + // dependency depends on. + std::optional win; + for (auto i : idx) { + if (version_of(claims[i].address).empty()) continue; + if (!win || claims[i].distance < claims[*win].distance) win = i; + } + if (!win) { out.winners.push_back({claims[idx.front()].address, idx.front()}); continue; } + + const auto chosen = version_of(claims[*win].address); + for (auto i : idx) { + if (i == *win) continue; + const auto stated = version_of(claims[i].address); + switch (check(chosen, stated)) { + case Verdict::Holds: break; + case Verdict::Differs: + out.overrides.push_back(std::format( + "'{}' is declared at two versions: '{}' by {}, and '{}' by " + "{}. One version is installed, and the declaration nearer " + "the artifact is the one used -- here '{}'. Drop the " + "nearer declaration to take the other.", + key, chosen, claims[*win].declaredBy, stated, + claims[i].declaredBy, chosen)); + break; + case Verdict::Violated: + return std::unexpected(std::format( + "`{}` is pinned to {} by {}, and {} requires {}.\n" + " One version of a package is installed, so the two " + "cannot both hold.\n" + " fix: pin a version satisfying {}, or drop the pin " + "and let the\n" + " requirement decide.", + key, chosen, claims[*win].declaredBy, + claims[i].declaredBy, stated, stated)); + } + } + out.winners.push_back({claims[*win].address, *win}); + } + return out; +} + +} // namespace mcpp::xlings::addrset diff --git a/src/xlings/xlings.cppm b/src/xlings/xlings.cppm index 150d1696..aa7dfc97 100644 --- a/src/xlings/xlings.cppm +++ b/src/xlings/xlings.cppm @@ -20,6 +20,7 @@ import mcpp.pm.index_snapshot; import mcpp.platform; import mcpp.log; import mcpp.home; +import mcpp.version_req; export namespace mcpp::xlings { @@ -82,7 +83,7 @@ namespace pinned { // correctly — which is why this is a floor and not a preference, and why // an index must still not publish a deliberately colliding // `@` (see .agents/docs/2026-08-30-cross-repo-fix-plan §1). - inline constexpr std::string_view kXlingsVersion = "2026.8.30.2"; + inline constexpr std::string_view kXlingsVersion = "2026.9.5.1"; inline constexpr std::string_view kNasmVersion = "3.02"; } @@ -134,6 +135,15 @@ namespace paths { // that is only discovered later, in the artifact. An unpinned ref takes // the highest version present — compared by numeric segments, because a // plain string sort puts "0.4.11" before "0.4.9". + // + // A CONSTRAINED ref (`>=8.5.0`, `^1.2`) takes the highest INSTALLED version + // satisfying it. The version position of an xlings address has accepted + // range expressions all along — xlings resolves one when it installs — but + // this lookup treated the whole position as a directory name, so a range + // installed a payload and then answered that nothing was installed. That + // asymmetry is what made `[feature-xlings]` floors unusable: a rule package + // could declare `>=8.5.0`, get it installed, and still see `xpkg_dir` + // return "". std::optional xpkg_payload(const Env& env, const XpkgRef& ref); @@ -826,9 +836,32 @@ xpkg_payload_at(const std::filesystem::path& xpkgsBase, const XpkgRef& ref) { const auto root = xpkgsBase / std::format("{}-x-{}", ref.ns, ref.name); std::error_code ec; if (!ref.version.empty()) { + // THE LITERAL DIRECTORY IS TRIED FIRST, AND IT IS TRIED FOR EVERY + // SPELLING. An installed version whose name does not parse as a SemVer + // — `8.0.RC1` is a real one — is addressable only this way, and a + // version that both names a directory and reads as a constraint (`=` + // is not part of any directory name, but a future operator might be) + // must resolve to the directory it names. auto p = root / ref.version; if (std::filesystem::is_directory(p, ec)) return p; - return std::nullopt; // pinned and absent: NOT "some other version" + // A pinned version that is absent is NOT "some other version". + if (!mcpp::version_req::is_constraint(ref.version)) return std::nullopt; + auto req = mcpp::version_req::parse_req(ref.version); + if (!req) return std::nullopt; + if (!std::filesystem::is_directory(root, ec)) return std::nullopt; + std::optional pick; + std::optional pickV; + for (auto const& e : std::filesystem::directory_iterator(root, ec)) { + if (!e.is_directory(ec)) continue; + // A directory whose name does not parse cannot be TESTED against a + // requirement, so it is not a candidate for one. It remains + // addressable by the exact spelling above. + auto v = mcpp::version_req::parse_version(e.path().filename().string()); + if (!v) continue; + if (!mcpp::version_req::matches(*req, *v)) continue; + if (!pickV || *pickV < *v) { pick = e.path(); pickV = *v; } + } + return pick; } if (!std::filesystem::is_directory(root, ec)) return std::nullopt; auto key_of = [](const std::string& s) { diff --git a/tests/e2e/627_one_package_one_version.sh b/tests/e2e/627_one_package_one_version.sh new file mode 100755 index 00000000..65dbe5a8 --- /dev/null +++ b/tests/e2e/627_one_package_one_version.sh @@ -0,0 +1,127 @@ +#!/usr/bin/env bash +# requires: gcc +# ONE PACKAGE, ONE VERSION. +# +# A project and one of its dependencies both name `xim:zoxide`, at different +# versions. mcpp had two definitions of when two addresses name one package: the +# conditional merge compared the PACKAGE, the graph split compared the whole +# STRING. Under the second, these were two packages -- both installed, and the +# `xpkg_dir` answer decided by whichever "keep the first value for a name" rule +# a later pass happened to reach. Installed twice, answered once, said nothing. +# +# THE CRITERION IS BOTH HALVES AT ONCE. Asserting only "one directory exists" +# would pass on an engine that installs one and answers the other; asserting +# only the answer would pass on one that installs two. The pair is what +# identifies the defect. +# +# WHY zoxide: three versions in the index for all three host platforms, about a +# megabyte each. The tool is never run -- only its payload directory is asked +# about -- so the test measures resolution, not the tool. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +TOOL=zoxide +PROJECT_PIN=0.9.9 +DEP_PIN=0.9.7 + +mkdir -p dep/src +cat > dep/src/lib.cpp <<'EOF' +int dep_touch() { return 1; } +EOF +# THE ANSWER IS RECORDED BY THE DEPENDENCY'S OWN BUILD PROGRAM, which is the +# reader that has to agree with the installer. To a file, because mcpp prints a +# build program's output only when it fails. +cat > dep/build.mcpp < +#include +import mcpp; +int main() { + const char* d = mcpp::xpkg_dir("xim", "$TOOL"); + std::string out = std::string(mcpp::manifest_dir()) + "/answered.txt"; + std::FILE* f = std::fopen(out.c_str(), "w"); + if (f == nullptr) return 3; + std::fprintf(f, "%s\n", d == nullptr ? "" : d); + std::fclose(f); + return 0; +} +EOF +cat > dep/mcpp.toml < app/src/main.cpp <<'EOF' +int main() { return 0; } +EOF +cat > app/mcpp.toml <build.log 2>&1; then + echo "FAIL: the consumer did not build" + tail -20 build.log + exit 1 +fi + +STORE="$MCPP_HOME/registry/data/xpkgs/xim-x-$TOOL" +[ -d "$STORE" ] || { echo "FAIL: $TOOL was never installed ($STORE)"; tail -20 build.log; exit 1; } +installed=$(ls -1 "$STORE" | sort | tr '\n' ' ') +count=$(ls -1 "$STORE" | wc -l | tr -d ' ') +echo "installed versions: $installed" +[ "$count" -eq 1 ] || { + echo "FAIL: two declarations of one package installed $count versions: $installed" + exit 1 +} +[ -d "$STORE/$PROJECT_PIN" ] || { + echo "FAIL: the declaration nearer the artifact did not win: $installed" + exit 1 +} + +answered="$TMP/dep/answered.txt" +[ -s "$answered" ] || { echo "FAIL: the dependency's build program recorded nothing"; exit 1; } +echo "xpkg_dir answered: $(cat "$answered")" +grep -q "$PROJECT_PIN" "$answered" || { + echo "FAIL: installed $PROJECT_PIN and answered $(cat "$answered")" + exit 1 +} + +# AND IT IS SAID OUT LOUD. An override visible only as "two versions were +# declared and one directory exists" is a fact the reader has to reconstruct +# from the filesystem. +grep -q "declared at two versions" build.log || { + echo "FAIL: the override was not reported" + grep -i 'zoxide' build.log | head -5 + exit 1 +} +grep -q "toolowner" build.log || { + echo "FAIL: the report does not name the dependency that lost" + exit 1 +} + +echo "PASS: one package, one version -- installed, answered and reported" diff --git a/tests/e2e/628_a_pin_below_a_stated_floor_is_refused.sh b/tests/e2e/628_a_pin_below_a_stated_floor_is_refused.sh new file mode 100755 index 00000000..d33f2d98 --- /dev/null +++ b/tests/e2e/628_a_pin_below_a_stated_floor_is_refused.sh @@ -0,0 +1,117 @@ +#!/usr/bin/env bash +# requires: +# NO CAPABILITY GATE, unlike its two neighbours. They need a dependency's +# build program to run, and `gcc` is how this runner spells "a toolchain that +# compiles one"; this test's first leg refuses before anything is compiled and +# its second only builds an ordinary library. The rule it checks is +# platform-independent, so it should be checked on every platform -- a Windows +# runner is where a path or a quoting assumption would surface. +# A PIN THAT DOES NOT SATISFY A STATED FLOOR IS A REFUSAL, NOT A SECOND INSTALL. +# +# Once one package means one version, the two declarations have to be +# compatible. A rule package writes `>=0.9.9` because that is the version its +# rule needs; a project that pins 0.9.7 has stated something that cannot hold at +# the same time. Installing both -- what the engine used to do -- postponed the +# contradiction to whatever the rule failed to find, or to nothing at all. +# +# THIS IS A COMPARISON, NOT A SEARCH. The version is chosen by adjudication (the +# declaration nearer the artifact wins) and then checked against the +# requirements that lost. Nothing here asks the index which versions exist, so +# there is no constraint solver in the engine and none is needed for this. +# +# THE MESSAGE MUST NAME BOTH SIDES. "cannot be satisfied" without saying who +# asked for what leaves the author to find the second declaration by grep, and +# it is usually in a package they do not own. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +TOOL=zoxide + +mkdir -p dep/src +cat > dep/src/lib.cpp <<'EOF' +int dep_touch() { return 1; } +EOF +cat > dep/mcpp.toml < app/src/main.cpp <<'EOF' +int main() { return 0; } +EOF +cat > app/mcpp.toml <build.log 2>&1; then + echo "FAIL: a pin below the dependency's floor was accepted" + grep -iE "$TOOL" build.log | head -5 + exit 1 +fi +out=$(cat build.log) + +for needle in "xim:$TOOL" "0.9.7" ">=0.9.9" "toolowner" "this project"; do + echo "$out" | grep -q -- "$needle" || { + echo "FAIL: the refusal does not mention '$needle'" + echo "$out" | tail -10 + exit 1 + } +done +echo "$out" | grep -q "drop the pin" || { + echo "FAIL: the refusal does not state the way out" + echo "$out" | tail -10 + exit 1 +} + +# AND THE REFUSAL HAS A TOKEN. A machine consumer classifying this outcome must +# not be reduced to matching the prose -- which is what docs/11 exists to +# promise, and what a reworded sentence silently breaks. +if command -v jq >/dev/null 2>&1; then + "$MCPP" why toolchain --format json >why.json 2>/dev/null || true + reason=$(jq -r '.data.reason // empty' why.json 2>/dev/null || true) + [ "$reason" = "tool-version-conflict" ] || { + echo "FAIL: reason was '$reason', expected tool-version-conflict" + head -c 400 why.json + exit 1 + } + echo "machine-readable reason: $reason" +fi + +# …AND RAISING THE PIN CLEARS IT. Without this leg the test would also pass on +# an engine that refuses every project declaring a tool its dependency also +# declares. +cd "$TMP" +sed -i.bak "s/\"0.9.7\"/\"0.9.9\"/" app/mcpp.toml +rm -f app/mcpp.toml.bak +cd app +"$MCPP" build >build2.log 2>&1 || { + echo "FAIL: a pin that DOES satisfy the floor was still refused" + tail -20 build2.log + exit 1 +} + +echo "PASS: a pin below a stated floor is refused, and one that satisfies it is not" diff --git a/tests/e2e/629_a_version_range_is_answered_by_xpkg_dir.sh b/tests/e2e/629_a_version_range_is_answered_by_xpkg_dir.sh new file mode 100755 index 00000000..28d70961 --- /dev/null +++ b/tests/e2e/629_a_version_range_is_answered_by_xpkg_dir.sh @@ -0,0 +1,140 @@ +#!/usr/bin/env bash +# requires: gcc +# A RANGE IN THE VERSION POSITION IS A CONSTRAINT, AND IT MUST BE ANSWERABLE. +# +# The version position of an xlings address has accepted range expressions all +# along and xlings resolves one when it installs. `xpkg_dir` did not: it treated +# the whole position as a directory name, so `>=0.9.9` installed a payload and +# then reported that nothing was installed. A rule package could therefore state +# a floor and could not then find what the floor had brought in -- which is why +# every project using a rule repeated the rule's own package list with exact +# pins. +# +# THIS IS THE ARRANGEMENT THE RANGE EXISTS FOR: the rule owns "which package, +# and no older than what", the project owns "and exactly this one" -- and writes +# nothing at all when it has no opinion, which is the common case. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +TOOL=zoxide + +mkdir -p dep/src +cat > dep/src/lib.cpp <<'EOF' +int dep_touch() { return 1; } +EOF +cat > dep/build.mcpp < +#include +import mcpp; +int main() { + const char* d = mcpp::xpkg_dir("xim", "$TOOL"); + if (d == nullptr || d[0] == '\0') { + std::fprintf(stderr, + "a [feature-xlings] entry stated as a range was installed and then " + "answered as absent\n"); + return 2; + } + std::string out = std::string(mcpp::manifest_dir()) + "/answered.txt"; + std::FILE* f = std::fopen(out.c_str(), "w"); + if (f == nullptr) return 3; + std::fprintf(f, "%s\n", d); + std::fclose(f); + return 0; +} +EOF +# UNDER A FEATURE, which is the shape a rule package uses: the tool arrives only +# for the consumers that asked for the rule. +cat > dep/mcpp.toml < app/src/main.cpp <<'EOF' +int main() { return 0; } +EOF +# THE PROJECT DECLARES NO TOOL AT ALL. That is the whole claim: one edge, and +# the environment the rule needs comes with it. +cat > app/mcpp.toml <<'EOF' +[package] +name = "consumer" +version = "0.1.0" +[dependencies] +toolowner = { path = "../dep", features = ["usestool"] } +[targets.consumer] +kind = "bin" +main = "src/main.cpp" +EOF + +export MCPP_HOME="$TMP/home" +mkdir -p "$MCPP_HOME" + +cd app +if ! "$MCPP" build >build.log 2>&1; then + echo "FAIL: the consumer did not build" + grep -iE 'answered as absent|error' build.log | head -5 + tail -10 build.log + exit 1 +fi + +answered="$TMP/dep/answered.txt" +[ -s "$answered" ] || { + echo "FAIL: the build program recorded nothing" + tail -10 build.log + exit 1 +} +echo "xpkg_dir answered: $(cat "$answered")" +grep -q "xim-x-$TOOL" "$answered" || { + echo "FAIL: the recorded path does not name $TOOL: $(cat "$answered")" + exit 1 +} +# THE HIGHEST INSTALLED VERSION SATISFYING THE RANGE, not merely some version. +# 0.9.7 is in the index and does not satisfy `>=0.9.9`; answering it would be a +# floor silently lowered. +grep -qE '/(0\.9\.9|0\.10\.[0-9]+)$' "$answered" || { + echo "FAIL: the answer does not satisfy >=0.9.9: $(cat "$answered")" + exit 1 +} + +# ── the other direction ───────────────────────────────────────────────────── +# +# AN IMPLEMENTATION THAT PASSED THE RANGE THROUGH AS A LITERAL WOULD ALSO PASS +# EVERYTHING ABOVE. What separates "the range was solved" from "the string +# happened to name something" is a range that nothing can satisfy: it has to be +# refused, and refused for that reason. +cd "$TMP" +sed 's/>=0.9.9/>=99.0.0/' dep/mcpp.toml > dep/mcpp.toml.new +mv dep/mcpp.toml.new dep/mcpp.toml +rm -rf app/target dep/answered.txt +cd app +if "$MCPP" build >unsat.log 2>&1; then + echo "FAIL: an unsatisfiable range was accepted" + grep -i "$TOOL" unsat.log | head -3 + exit 1 +fi +grep -q 'not found in the synced index' unsat.log || { + echo "FAIL: refused, but not because the range is unsatisfiable" + grep -i error unsat.log | head -3 + exit 1 +} +grep -q ">=99.0.0" unsat.log || { + echo "FAIL: the refusal does not quote the range that could not be satisfied" + grep -i error unsat.log | head -3 + exit 1 +} +echo "an unsatisfiable range is refused, naming it" + +echo "PASS: a range is installed and answered in one direction, and refused in the other" diff --git a/tests/e2e/630_a_rule_brings_its_own_environment.sh b/tests/e2e/630_a_rule_brings_its_own_environment.sh new file mode 100755 index 00000000..727d363d --- /dev/null +++ b/tests/e2e/630_a_rule_brings_its_own_environment.sh @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +# requires: gcc +# A RULE PACKAGE DECLARES THE PAYLOADS ITS RULE NEEDS, AND THE PROJECT WRITES +# ONE EDGE. +# +# This is the arrangement the two preceding tests exist to make safe. A project +# that wants a device island used to write the rule edge AND repeat the rule's +# own package list with exact pins -- a copy that goes stale silently, because +# the rule moves and the projects do not. +# +# TWO GATES, AND BOTH MUST OPEN. The feature says whether the rule is wanted; +# the `cfg(accelerator = ...)` selector says which builds actually need the +# device toolkit. Both legs are here: without the second, an implementation +# that ignores the selector and downloads the toolkit for every build passes -- +# and that build is the cheapest one, the one CI runs, and the one a vendor +# toolkit is gigabytes too expensive for. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +TOOL=zoxide + +mkdir -p dep/src +cat > dep/src/lib.cpp <<'EOF' +int dep_touch() { return 1; } +EOF +cat > dep/build.mcpp < +#include +import mcpp; +int main() { + const char* d = mcpp::xpkg_dir("xim", "$TOOL"); + std::string out = std::string(mcpp::manifest_dir()) + "/answered.txt"; + std::FILE* f = std::fopen(out.c_str(), "w"); + if (f == nullptr) return 3; + std::fprintf(f, "%s\n", d == nullptr ? "" : d); + std::fclose(f); + return 0; +} +EOF +# THE SHAPE mcpp:plugins USES: the payload is named by the rule, under the +# feature that selects the rule and the accelerator the rule serves. +cat > dep/mcpp.toml < app/src/main.cpp <<'EOF' +int main() { return 0; } +EOF +# ONE EDGE, AND NOTHING ELSE. No [xlings.workspace] anywhere in the project. +cat > app/mcpp.toml <<'EOF' +[package] +name = "consumer" +version = "0.1.0" +accelerators = ["cuda"] +[dependencies] +ruleowner = { path = "../dep", features = ["rules-toy"] } +[targets.consumer] +kind = "bin" +main = "src/main.cpp" +EOF + +export MCPP_HOME="$TMP/home" +mkdir -p "$MCPP_HOME" +STORE="$MCPP_HOME/registry/data/xpkgs/xim-x-$TOOL" + +cd app + +# ── leg 1: no accelerator, so neither gate opens ──────────────────────────── +"$MCPP" build >cpu.log 2>&1 || { echo "FAIL: the CPU-only build did not build"; tail -15 cpu.log; exit 1; } +[ -d "$STORE" ] && { + echo "FAIL: a build that named no accelerator downloaded the device payload" + ls "$STORE" + exit 1 +} +echo "no accelerator: nothing installed" + +# ── leg 2: the accelerator is named, so both gates open ───────────────────── +"$MCPP" build --accel "cuda12.9+{sm_89}" >dev.log 2>&1 || { + echo "FAIL: the device build did not build" + tail -15 dev.log + exit 1 +} +[ -d "$STORE" ] || { + echo "FAIL: the rule's payload was never installed" + tail -15 dev.log + exit 1 +} +answered="$TMP/dep/answered.txt" +[ -s "$answered" ] || { echo "FAIL: the rule's build program recorded nothing"; exit 1; } +echo "the rule's build program saw: $(cat "$answered")" +grep -q "xim-x-$TOOL" "$answered" || { + echo "FAIL: the rule cannot find the payload it declared: $(cat "$answered")" + exit 1 +} + +echo "PASS: the rule brings its own environment, and only when both gates open" diff --git a/tests/unit/test_elf_runtime.cpp b/tests/unit/test_elf_runtime.cpp index c05db8dd..f3e3e328 100644 --- a/tests/unit/test_elf_runtime.cpp +++ b/tests/unit/test_elf_runtime.cpp @@ -55,6 +55,13 @@ struct ElfFixtureSpec { std::string interp = "/store/glibc/2.44/lib64/ld-linux-x86-64.so.2"; std::vector needed = {"libc.so.6"}; std::string runpath = "/host/z:/host/a"; + // Emit the search path as DT_RPATH and nothing else. The default image + // carries both tags, which is the common shape and the one glibc reads as + // DT_RUNPATH -- so a test about DT_RPATH's reach cannot use it. + bool rpathOnly = false; + // No search path at all: the object that has to reach its dependencies + // through someone else's DT_RPATH. + bool noSearchPath = false; }; // One deliberately tiny ELF64-LE image. It has no executable code; the test @@ -108,7 +115,8 @@ std::filesystem::path write_elf_fixture( auto versionOwner = needed.empty() ? append_string(b, kDynstr, cursor, "libc.so.6") : needed.front(); - auto rpath = append_string(b, kDynstr, cursor, "/legacy/ignored"); + auto rpath = append_string(b, kDynstr, cursor, + spec.rpathOnly ? spec.runpath : "/legacy/ignored"); auto runpath = append_string(b, kDynstr, cursor, spec.runpath); auto needVersion = append_string(b, kDynstr, cursor, "GLIBC_2.40"); auto defVersion = append_string(b, kDynstr, cursor, "GLIBC_2.44"); @@ -123,8 +131,10 @@ std::filesystem::path write_elf_fixture( dyn(5, kVaddr + kDynstr); // DT_STRTAB dyn(10, dynstrSize); // DT_STRSZ for (auto offset : needed) dyn(1, offset); // DT_NEEDED - dyn(15, rpath); // DT_RPATH (ignored when RUNPATH exists) - dyn(29, runpath); // DT_RUNPATH + if (!spec.noSearchPath) { + dyn(15, rpath); // DT_RPATH (ignored when RUNPATH exists) + if (!spec.rpathOnly) dyn(29, runpath); // DT_RUNPATH + } dyn(0x6ffffffe, kVaddr + kVerneed); // DT_VERNEED dyn(0x6fffffff, 1); // DT_VERNEEDNUM dyn(0x6ffffffc, kVaddr + kVerdef); // DT_VERDEF @@ -283,6 +293,80 @@ TEST(ElfRuntime, ReusesAnAlreadyLoadedSonameAcrossDependencyRunpaths) { std::filesystem::weakly_canonical(glibc44 / "libc.so.6")); } +// DT_RPATH REACHES THE WHOLE CHAIN; DT_RUNPATH REACHES ONE OBJECT. +// +// A vendor toolkit's shared libraries depend on each other by bare SONAME and +// carry no search path of their own; the directory holding them is named once, +// in the EXECUTABLE's DT_RPATH. The model searched only the requesting +// object's own list, so every such library read as unfindable. +// +// Measured on examples/09-heterogeneous/cann before this: mcpp refused the +// build naming eight libraries, and the artifact it had just linked resolved +// seven of them -- failing only on the one that belongs to a driver the +// machine does not have. +TEST(ElfRuntime, ADependencyInheritsTheExecutablesRpath) { + if constexpr (!mcpp::platform::is_linux) + GTEST_SKIP() << "ELF/glibc runtime physics only apply on Linux"; + Tmp t; + auto payload = t.path / "store"; + auto glibc = payload / "2.44" / "lib64"; + auto toolkit = t.path / "toolkit" / "lib64"; + std::filesystem::create_directories(glibc); + std::filesystem::create_directories(toolkit); + + write_elf_fixture(glibc / "libc.so.6", { .needed = {}, .runpath = glibc.string() }); + // The leaf, reachable only through the executable's DT_RPATH. + write_elf_fixture(toolkit / "libdeep.so", { .needed = {}, .noSearchPath = true }); + // The middle object: names its dependency and says nothing about where it + // lives, which is what a vendor library does. + write_elf_fixture(toolkit / "libtop.so", + { .needed = {"libdeep.so"}, .noSearchPath = true }); + auto app = write_elf_fixture(t.path / "app", { + .needed = {"libtop.so", "libc.so.6"}, + .runpath = toolkit.string() + ":" + glibc.string(), + .rpathOnly = true, + }); + + auto resolution = elf::resolve_runtime_closure(app, binding_for(payload)); + EXPECT_TRUE(resolution.unresolvedSonames.empty()) + << "unresolved: " << (resolution.unresolvedSonames.empty() + ? std::string{} : resolution.unresolvedSonames.front()); +} + +// …and the suppression, which is the half that makes the rule a rule. An +// object carrying DT_RUNPATH uses no RPATH at all -- its own or inherited -- +// so a test with only the leg above would also pass on an implementation that +// inherited unconditionally. +TEST(ElfRuntime, ADependencyWithItsOwnRunpathDoesNotInheritOne) { + if constexpr (!mcpp::platform::is_linux) + GTEST_SKIP() << "ELF/glibc runtime physics only apply on Linux"; + Tmp t; + auto payload = t.path / "store"; + auto glibc = payload / "2.44" / "lib64"; + auto toolkit = t.path / "toolkit" / "lib64"; + auto elsewhere = t.path / "elsewhere"; + std::filesystem::create_directories(glibc); + std::filesystem::create_directories(toolkit); + std::filesystem::create_directories(elsewhere); + + write_elf_fixture(glibc / "libc.so.6", { .needed = {}, .runpath = glibc.string() }); + write_elf_fixture(toolkit / "libdeep.so", { .needed = {}, .noSearchPath = true }); + // Same graph as above, except this middle object carries DT_RUNPATH. It + // names a directory that does not hold `libdeep.so`, and glibc will not + // fall back to the executable's DT_RPATH for it. + write_elf_fixture(toolkit / "libtop.so", + { .needed = {"libdeep.so"}, .runpath = elsewhere.string() }); + auto app = write_elf_fixture(t.path / "app", { + .needed = {"libtop.so", "libc.so.6"}, + .runpath = toolkit.string() + ":" + glibc.string(), + .rpathOnly = true, + }); + + auto resolution = elf::resolve_runtime_closure(app, binding_for(payload)); + ASSERT_EQ(resolution.unresolvedSonames.size(), 1u); + EXPECT_EQ(resolution.unresolvedSonames.front(), "libdeep.so"); +} + TEST(RuntimePhysics, RuleBRejectsInterpreterAndLibcFromDifferentPayloads) { if constexpr (!mcpp::platform::is_linux) GTEST_SKIP() << "ELF/glibc runtime physics only apply on Linux"; diff --git a/tests/unit/test_freestanding.cpp b/tests/unit/test_freestanding.cpp index c73f7e2f..fc2d80f6 100644 --- a/tests/unit/test_freestanding.cpp +++ b/tests/unit/test_freestanding.cpp @@ -218,6 +218,65 @@ TEST(XpkgPayload, AnUnpinnedRefTakesTheHighestVersionNumerically) { std::filesystem::remove_all(base); } +// A CONSTRAINT IN THE VERSION POSITION IS A CONSTRAINT, NOT A DIRECTORY NAME. +// +// The address grammar has accepted `>=8.5.0` all along and xlings resolves one +// when it installs, but this lookup compared the whole position against a +// directory name -- so a range installed a payload and then reported that +// nothing was installed. That is what made a rule package unable to state a +// floor: it declared `>=8.5.0`, got it, and `xpkg_dir` answered "". +TEST(XpkgPayload, AConstrainedRefTakesTheHighestInstalledVersionSatisfyingIt) { + namespace xp = mcpp::xlings::paths; + auto base = std::filesystem::temp_directory_path() + / std::format("mcpp-xpkg-test3-{}", ::getpid()); + std::filesystem::remove_all(base); + for (auto v : { "8.0.0", "8.5.0", "8.7.1" }) + std::filesystem::create_directories(base / "xim-x-demo" / v); + + auto floor = xp::xpkg_payload_at(base, xp::parse_xpkg_ref("xim:demo@>=8.5.0")); + ASSERT_TRUE(floor.has_value()); + EXPECT_EQ(floor->filename(), "8.7.1"); + + // Bounded on both sides, so the highest is NOT the answer -- a test that + // only used a lower bound would pass on an implementation that ignored the + // requirement and took the newest. + auto ranged = xp::xpkg_payload_at(base, xp::parse_xpkg_ref("xim:demo@>=8.0.0, <8.7.0")); + ASSERT_TRUE(ranged.has_value()); + EXPECT_EQ(ranged->filename(), "8.5.0"); + + // Unsatisfiable is absent, not "the closest one". + EXPECT_FALSE(xp::xpkg_payload_at(base, xp::parse_xpkg_ref("xim:demo@>=9.0.0")) + .has_value()); + + // AND AN EXACT PIN STILL MEANS EXACTLY THAT. The constrained path must not + // capture a pinned-and-absent request: 8.6.0 is missing and 8.7.1 is right + // there, and answering with it is the failure the pin exists to prevent. + EXPECT_FALSE(xp::xpkg_payload_at(base, xp::parse_xpkg_ref("xim:demo@8.6.0")) + .has_value()); + + std::filesystem::remove_all(base); +} + +// An installed version whose directory name is not a SemVer is addressable by +// its exact spelling and by nothing else. `8.0.RC1` is a real CANN version. +TEST(XpkgPayload, AnUnparseableVersionIsStillAddressableExactly) { + namespace xp = mcpp::xlings::paths; + auto base = std::filesystem::temp_directory_path() + / std::format("mcpp-xpkg-test4-{}", ::getpid()); + std::filesystem::remove_all(base); + std::filesystem::create_directories(base / "xim-x-demo" / "8.0.RC1"); + + auto exact = xp::xpkg_payload_at(base, xp::parse_xpkg_ref("xim:demo@8.0.RC1")); + ASSERT_TRUE(exact.has_value()); + EXPECT_EQ(exact->filename(), "8.0.RC1"); + + // It cannot be TESTED against a requirement, so it does not answer one. + EXPECT_FALSE(xp::xpkg_payload_at(base, xp::parse_xpkg_ref("xim:demo@>=1.0")) + .has_value()); + + std::filesystem::remove_all(base); +} + TEST(XpkgEnvVar, BothSpellingsAreDerivedFromOneSanitizer) { using mcpp::build::xpkg_env_var; // The two sides of the channel must agree; drifting apart would make the diff --git a/tests/unit/test_xlings_address_set.cpp b/tests/unit/test_xlings_address_set.cpp new file mode 100644 index 00000000..b756c242 --- /dev/null +++ b/tests/unit/test_xlings_address_set.cpp @@ -0,0 +1,208 @@ +#include + +import std; +import mcpp.xlings.address_set; + +namespace addrset = mcpp::xlings::addrset; + +namespace { + +addrset::Claim project(std::string address) { + return { std::move(address), "this project", 0 }; +} +addrset::Claim from(std::string who, std::string address) { + return { std::move(address), std::move(who), 1 }; +} + +std::vector addresses(const addrset::Resolution& r) { + std::vector out; + for (auto const& w : r.winners) out.push_back(w.address); + return out; +} + +} // namespace + +// ─── Identity ────────────────────────────────────────────────────────────── + +// The version is a constraint on a package, never part of its name. Two +// definitions of this used to coexist: the conditional merge compared the bare +// name, the graph split compared the whole address string. +TEST(XlingsIdentity, IsTheNamespaceAndNameAndNeverTheVersion) { + EXPECT_EQ(addrset::package_key("xim:glibc@2.40"), "xim:glibc"); + EXPECT_EQ(addrset::package_key("xim:glibc"), "xim:glibc"); + EXPECT_EQ(addrset::package_key("xim:glibc@>=2.38"), "xim:glibc"); + + // The manifest's default namespace, spelled the same way `parse_xpkg_ref` + // spells it: `ninja` and `xim:ninja` name one package. + EXPECT_EQ(addrset::package_key("ninja"), "xim:ninja"); + + // …and a different index is a different package, which the bare-name + // definition could not express. + EXPECT_NE(addrset::package_key("scode:cuda"), addrset::package_key("xim:cuda")); +} + +// ─── Adjudication ────────────────────────────────────────────────────────── + +// The arrangement section 6 of the design describes: the rule owns the floor, +// the project owns the exact version. Two statements, one install, and no +// warning -- a note here would fire on every build of every project that pins a +// tool its rule also requires, which is the intended arrangement. +TEST(XlingsUnify, OnePackageYieldsOneAddress) { + std::vector claims{ + project("xim:cuda-nvcc@13.3.33"), + from("mcpp:plugins", "xim:cuda-nvcc@>=12.9.86"), + }; + auto r = addrset::unify(claims); + ASSERT_TRUE(r.has_value()) << r.error(); + EXPECT_EQ(addresses(*r), (std::vector{"xim:cuda-nvcc@13.3.33"})); + EXPECT_TRUE(r->overrides.empty()); +} + +// …and when the two really do disagree, the note names both sides and says +// which was used. Reported, not inferred from an install log. +TEST(XlingsUnify, AResolvedDisagreementNamesBothSidesAndTheWinner) { + std::vector claims{ + project("xim:cuda-nvcc@13.3.33"), + from("mcpp:plugins", "xim:cuda-nvcc@12.9.86"), + }; + auto r = addrset::unify(claims); + ASSERT_TRUE(r.has_value()) << r.error(); + ASSERT_EQ(r->overrides.size(), 1u); + EXPECT_NE(r->overrides[0].find("mcpp:plugins"), std::string::npos); + EXPECT_NE(r->overrides[0].find("this project"), std::string::npos); + EXPECT_NE(r->overrides[0].find("12.9.86"), std::string::npos); + EXPECT_NE(r->overrides[0].find("13.3.33"), std::string::npos); +} + +// A claim naming no version states that the package is wanted and nothing about +// which version, so it must not out-rank a floor merely by being nearer. +TEST(XlingsUnify, AnUnversionedClaimAbstainsFromTheVersionQuestion) { + std::vector claims{ + project("xim:cann-toolkit"), + from("mcpp:plugins", "xim:cann-toolkit@>=8.5.0"), + }; + auto r = addrset::unify(claims); + ASSERT_TRUE(r.has_value()) << r.error(); + EXPECT_EQ(addresses(*r), + (std::vector{"xim:cann-toolkit@>=8.5.0"})); + EXPECT_TRUE(r->overrides.empty()); +} + +TEST(XlingsUnify, WithNoVersionAnywhereTheBareAddressSurvives) { + std::vector claims{ + project("xim:ninja"), + from("board:virt", "ninja"), + }; + auto r = addrset::unify(claims); + ASSERT_TRUE(r.has_value()) << r.error(); + EXPECT_EQ(addresses(*r), (std::vector{"xim:ninja"})); +} + +// Order is the order packages were first claimed, so a build's install list +// does not permute when an unrelated dependency is added. +TEST(XlingsUnify, KeepsFirstClaimedOrder) { + std::vector claims{ + project("xim:b@1.0"), project("xim:a@1.0"), + from("dep", "xim:a@>=1.0"), from("dep", "xim:c@1.0"), + }; + auto r = addrset::unify(claims); + ASSERT_TRUE(r.has_value()) << r.error(); + EXPECT_EQ(addresses(*r), + (std::vector{"xim:b@1.0", "xim:a@1.0", "xim:c@1.0"})); +} + +// ─── Validation ──────────────────────────────────────────────────────────── + +// The one refusal this module adds. It is a comparison against a chosen +// version, not a search for one: nothing here consults which versions exist. +TEST(XlingsUnify, RefusesAPinBelowAStatedFloorAndNamesBothSides) { + std::vector claims{ + project("xim:cann-toolkit@8.0.0"), + from("mcpp:plugins", "xim:cann-toolkit@>=8.5.0"), + }; + auto r = addrset::unify(claims); + ASSERT_FALSE(r.has_value()); + EXPECT_NE(r.error().find("xim:cann-toolkit"), std::string::npos); + EXPECT_NE(r.error().find("8.0.0"), std::string::npos); + EXPECT_NE(r.error().find(">=8.5.0"), std::string::npos); + EXPECT_NE(r.error().find("mcpp:plugins"), std::string::npos); + // The way out is in the message, which is what makes the shortcut of not + // intersecting constraints an acceptable one. + EXPECT_NE(r.error().find("drop the pin"), std::string::npos); +} + +TEST(XlingsUnify, AcceptsAPinThatSatisfiesTheFloor) { + std::vector claims{ + project("xim:cann-toolkit@8.5.0"), + from("mcpp:plugins", "xim:cann-toolkit@>=8.5.0"), + }; + auto r = addrset::unify(claims); + ASSERT_TRUE(r.has_value()) << r.error(); + EXPECT_EQ(addresses(*r), (std::vector{"xim:cann-toolkit@8.5.0"})); + // Satisfied is silent: an override note here would fire on every build of + // every project that pins a tool its rule also requires, which is the + // intended arrangement rather than a problem. + EXPECT_TRUE(r->overrides.empty()); +} + +// TWO EXACT PINS ARE TWO CHOICES, NOT A VIOLATED REQUIREMENT. Refusing them +// would turn every dependency that pinned a tool the project also pins into a +// hard failure on upgrade, for a disagreement adjudication already settles. +TEST(XlingsUnify, TwoExactPinsAreAdjudicatedRatherThanRefused) { + std::vector claims{ + project("xim:cuda-nvcc@13.3.33"), + from("mcpp:plugins", "xim:cuda-nvcc@12.9.86"), + }; + auto r = addrset::unify(claims); + ASSERT_TRUE(r.has_value()) << r.error(); + EXPECT_EQ(addresses(*r), (std::vector{"xim:cuda-nvcc@13.3.33"})); + EXPECT_EQ(r->overrides.size(), 1u); +} + +// Two dependencies, no project statement: there is no "nearer", so the tie is +// broken by first appearance -- and the validation still runs, which is what +// keeps an arbitrary tie-break from silently lowering a floor. +TEST(XlingsUnify, TwoDependenciesInConflictAreStillValidated) { + std::vector ok{ + from("dep-a", "xim:tool@2.0.0"), + from("dep-b", "xim:tool@>=1.0.0"), + }; + ASSERT_TRUE(addrset::unify(ok).has_value()); + + std::vector bad{ + from("dep-a", "xim:tool@1.0.0"), + from("dep-b", "xim:tool@>=2.0.0"), + }; + auto r = addrset::unify(bad); + ASSERT_FALSE(r.has_value()); + EXPECT_NE(r.error().find("dep-a"), std::string::npos); + EXPECT_NE(r.error().find("dep-b"), std::string::npos); +} + +// A version this grammar cannot parse is not evidence of a conflict. Refusing +// on a spelling nobody can evaluate would be a refusal manufactured from +// ignorance -- `8.0.RC1` is a real published version. +TEST(XlingsUnify, AnUnevaluableSpellingIsReportedNotRefused) { + std::vector claims{ + project("xim:cann-toolkit@8.0.RC1"), + from("mcpp:plugins", "xim:cann-toolkit@>=8.5.0"), + }; + auto r = addrset::unify(claims); + ASSERT_TRUE(r.has_value()) << r.error(); + EXPECT_EQ(addresses(*r), (std::vector{"xim:cann-toolkit@8.0.RC1"})); + EXPECT_EQ(r->overrides.size(), 1u); +} + +// The same spelling twice is one statement, not a disagreement. Without this a +// project that is its own runtime owner would warn about itself on every build. +TEST(XlingsUnify, RepeatingOneSpellingIsSilent) { + std::vector claims{ + project("xim:shaderc@2026.3"), + project("xim:shaderc@2026.3"), + from("dep", "xim:shaderc@2026.3"), + }; + auto r = addrset::unify(claims); + ASSERT_TRUE(r.has_value()) << r.error(); + EXPECT_EQ(addresses(*r), (std::vector{"xim:shaderc@2026.3"})); + EXPECT_TRUE(r->overrides.empty()); +}