Skip to content

mcpp-language-server 0.0.12: the maintained mcppls-clangd engine, mcpp formatting, crash recovery, linux-arm64 again - #46

Merged
Sunrisepeak merged 44 commits into
mainfrom
release/0.0.12-joint
Oct 11, 2026
Merged

Sunrisepeak merged 44 commits into
mainfrom
release/0.0.12-joint

Conversation

@Sunrisepeak

@Sunrisepeak Sunrisepeak commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

mcpp-language-server 0.0.12 bundles the maintained engine, mcppls-clangd 0.0.12 (LLVM 23.1.0 + 25 patches for C++ modules), on linux-x64, linux-arm64, darwin-arm64 and win32-x64.

What changed

Engine and payload

  • Every platform's lock entry names mcppls-clangd 23.1.0-mcppls.0. The payload binds the engine's immutable identity (payload version 4, engine.json); assembly and the server verify it before trusting the engine's declared capabilities, and mcppls.pack.clangd takes the part byte for byte.
  • Workarounds retire per verified engine feature, never per version: WA-CLANGD-001 (module-directive-recovery), 006, 010 and the new WA-CLANGD-013 (unresolved-import-recovery: no stand-ins for a file's own unresolved imports, no file set aside for one on disk). A module unit's own stand-ins (WA-CLANGD-002) stay.
  • Files mixing heavy headers (Qt, <windows.h>) with module imports keep a verified preamble on every platform, and typing no longer waits on module rebuilds (engine PR mcppls-clangd 0.0.12: preambles kept for files mixing headers with imports; typing, edits and lost BMIs do not wait on module rebuilds mcppls-clangd#4). Qt importer completion 0.58 s → 0.05 s (stock clangd 0.58 s); typing in it p95 0.17 s (stock 1.00 s).

mcpp's build programs (#51)

  • mcpp 2026.10.11.1 describes build.mcpp, the libraries of [build-dependencies] and mcpp's engine interface in the build database. These units now get navigation, completion and diagnostics.
    • .mcpp is C++ in every editor integration.
    • Target sets are read first. Build programs with a toolchain of their own are read in <package>:build.
    • An import across the two configurations is an error at the import.
    • The engine interface is read-only.
  • An older mcpp says which version describes build programs. New fixtures mcpp-build-programs (real mcpp 2026.10.11.1 on Linux, macOS and Windows) and mcpp-build-programs-cross.

Module cache (engine and product)

  • A BMI that is lost, evicted or damaged is rebuilt without rebuilding its importers: ux-mcpp U11 recovery 6.6 s, down from 25–36 s.
  • Reusing a cached module no longer scans every cache slot, and a file's last prerequisites are kept between requests. ux-xlings completion while typing: p95 0.23 s (U5) and 0.46 s (U6, autosave), down from 1.00 s.
  • On Linux, mcppls-clangd's owned module cache gets three quarters of mcppls.cache.maxBytes (at least 1 GiB), not a fixed 1 GiB. A profile change no longer waits for room.

Fixes

  • preparation-stalled no longer fires the moment preparation starts again after idling ([preparation-stalled] module preparation made no progress for a minute (34/39 done) #49).
  • A doomed file that clangd closed is not taken for a spin when it goes back; a syntax error in a file opened outside the workspace is no project issue; a planned unit clangd cannot scan gets its stand-in on the first report.
  • Body-only saves the plan already covers do not trigger a whole-project plan; test_save_plan used an uncanonicalized temporary root.

Editing, VS Code, platforms (unchanged from earlier in this PR)

  • mcppls.format.fallbackStyle (mcpp style for mcpp projects); crash-during-initialize recovery (WA-VSCODE-003/004).
  • linux-arm64 is bundled again (glibc 2.31); linux-x64's clangd needs glibc 2.31 and macOS 12; Intel macOS scaffolding is removed.

CI

  • New fixtures: ux-modules-qt, ux-modules-windows (answers, latency while typing, crashes, stalls), preparation-after-idle ([preparation-stalled] module preparation made no progress for a minute (34/39 done) #49). Typing edits keep a file's line endings, as an editor does. ux-modules-windows' typing budgets are what the Windows runner measures: about 0.5 s per completion even when idle. A failed fixture can print the server log lines it names (failure-log-lines).
  • A PR can test with an engine PR's build before the engine is released (packaging/engine-candidate.json; docs/92-release.md, "A new engine"); CI and Release refuse it outside a PR.

Evidence

Raw evidence from the joint work is in mcpp-language-server-0.0.12-evidence.tar.zst on the evidence-0.0.12 release; the plan and its record are in .agents/docs/2026-10-09-0.0.12-part3-convergence-plan.md (§8.4).

Companion engine PRs: Sunrisepeak/mcppls-clangd#2, #3, #4, released as mcppls-clangd v0.0.12 and pinned in packaging/payload.lock.json.

Closes #49, #51.

…h recovery, save planning

The product side of the 0.0.12 joint work with mcppls-clangd, without the raw
evidence it produced (that is in the evidence-0.0.12 archive):

- a payload binds its engine's immutable identity (payload version 4,
  engine.json: version, LLVM base and commit, fork commit, patch-series digest,
  platform, binary SHA-256, features), verified at assembly and at startup; a
  maintained engine selects the kit of its LLVM base, and workarounds retire
  only for a capability the verified identity declares
- format.fallbackStyle: without a project .clang-format, an mcpp project gets
  the pinned mcpp style when the bundled engine declares it
- VS Code: a server exiting during initialize, or a failed protocol write,
  settles the client before a fresh one starts (WA-VSCODE-003/004); recovery
  shares the manual lifecycle queue and the crash budget; the log path is
  learned from stderr before a report can be interrupted
- a body save whose disk and draft module structure the plan already covers
  does not schedule another whole-project plan
- the model gateway's initialize has its own bounded deadline
- Darwin pipes to children are protected from SIGPIPE per descriptor
- CI keeps bounded timing and stability diagnostics for native runs
…taken as built, save planning holds on aliased paths

- linux-arm64 returns to the lock, the release manifest, the extension and
  every CI matrix: its removal was a scope decision with no technical failure
  behind it, and the maintained engine is built for it at the Ubuntu 20.04
  floor (glibc 2.31 instead of LLVM's own build's 2.34).
- mcppls.pack.clangd takes a maintained engine part (mcppls-clangd's
  clangd-<version>-<platform>.tar.gz, whose clangd/ holds engine.json) byte
  for byte: no strip, no thinning, no re-signing, so the identity the payload
  verifies names the bytes it ships.
- test_save_plan failed on macOS and Windows only: its hand-built model named
  files under the raw temporary directory while the workspace resolves a
  document's URI canonically, as a loaded model does (/var is /private/var,
  Windows may hand out a short name). Reproduced on Linux with a symlinked
  TMPDIR (the same four assertions), fixed by giving the fixture the
  canonical root a loaded model has.
- the default background-index priority applies on macOS only, the one host
  with evidence for it (Linux maps both priorities to SCHED_IDLE; Windows'
  background mode also lowers I/O priority, unmeasured for indexing).
- format.fallbackStyle compares `auto` case-insensitively, like `mcpp`.
- payload version 4 is a named constant shared by assembly and verification.
- Intel macOS build scaffolding goes (its cross target, the __bzero stub, the
  Neovim bootstrap, the x86_64 SIGPIPE path); 0.0.11 never shipped it.
- mcppls-devtools check tree: no tracked file above 1 MiB, none generated.
- the xlings descriptors depend on the LLVM base of a maintained engine.
- CHANGELOG 0.0.12, install floors (glibc 2.31, macOS 12), the part 3 plan
  and its record; the earlier status documents point at the evidence archive.
@Sunrisepeak
Sunrisepeak force-pushed the release/0.0.12-joint branch from 0592ccd to 39a2b23 Compare October 9, 2026 07:03
@Sunrisepeak Sunrisepeak changed the title release: 0.0.12 joint maintained clangd implementation mcpp-language-server 0.0.12: the maintained mcppls-clangd engine, mcpp formatting, crash recovery, linux-arm64 again Oct 9, 2026
… not won in a polling race

The server answers initialize within milliseconds, so finding it between its
spawn and the client's start by polling the process table was a race; under
PRoot on linux-x64 the poll lost it for a full minute. MCPPLS_SERVER, read at
every start, now names a stand-in that waits three seconds before becoming the
real server, so the test kills a process whose initialize is certainly
unanswered, and the recovery it checks is unchanged. The format fallback line
of the initialization options moves below the engine.workers comment it had
split.
…fixes retire their workarounds

Running the conformance fixtures against the maintained engine, which product
CI had never done, showed where it speaks differently from stock clangd:

- A failed prerequisite is reported per importing file, "Failed to build
  module prerequisites for <file>; due to module worker failed: <source>:...",
  naming the unit that failed rather than the module. The parser took
  "prerequisites for <file>" for a module name, so no module-failed
  diagnostic, closure containment or std kit fallback happened
  (module-faults). It now reads the importer and the failed unit, and the
  module is the one the plan gave that unit.
- An import with no unit in the project is kept textual: "Keeping import of
  third-party module <name> textual; no module unit for it in this project".
  It is an unresolved module, as stock clangd's "Don't get the module unit"
  is, so it gets its stand-in (partial-scan-standins).
- WA-CLANGD-001 and WA-CLANGD-010 retire for an engine whose verified identity
  declares module-directive-recovery and const-correctness-views, as WA-006
  already did for module-directive-diagnostic-ranges. Conformance checks take
  "retired-by": a check watching a defect the payload's engine is proved free
  of holds and says why (workaround-canaries' 001 and 010, typing-import-spin's
  T2); against any other clangd it runs.

Both log formats keep working for an external stock clangd.
packaging/payload.lock.json names mcppls-clangd v0.0.12's
clangd-23.1.0-mcppls.0-<platform>.tar.gz for linux-x64, linux-arm64,
darwin-arm64 and win32-x64 (each sha256 from two downloads and the digest the
release publishes beside the asset); the stock clangd archives no platform
names go, the kit sources stay. clangd-version and CLANGD_VERSION are
23.1.0-mcppls.0; the kit keeps the LLVM base, 23.1.0.

The engine's identity declares module-directive-recovery, so the checks that
watch WA-CLANGD-001 itself being on (typing-import T6) or its disk-side
set-aside (typing-autosave T3) are retired-by it, as the canaries are.
…nd the scenario tests read its owned cache

The first product CI run with mcppls-clangd bundled showed three more places
where the product assumed stock clangd:

- Under PRoot every process is traced call by call, and mcppls-clangd's
  supervised compiler worker is a process per module; with PROOT_NO_SECCOMP
  clangd was still preparing after 20 s. For a verified maintained engine
  under a detected sandbox the server passes
  --modules-builder-worker-policy=in-process (with the owned cache's payload
  bounds off, which need a worker), the upstream way.
- mcppls-clangd's owned cache publishes every module file as
  .owned-payload-v1/generation-<slot>-<n>/payload.pcm. The scenario tests took
  a module file's unit from its directory and its module from its file name,
  so a warm start that reused all 111 xlings BMIs counted 135 rebuilds, SC4
  found no std file and U11 nothing to truncate. An owned payload's unit is
  now the source its control block names, its module the one that source
  declares, and a new generation holding the bytes of a payload there was is
  a read copy, not a build.
- In a module importer that was just opened, mcppls-clangd waits for the
  file's first preamble before it answers a completion (issue #50), where
  stock clangd answers at once without imported declarations. Typing checks
  with "firstOpenSeconds" now measure that wait on its own, with its own
  budget, and hold the typing after it to the unchanged budgets: ux-xlings U5
  locally, first answer 6.1 s after opening, then p95 0.14 s (stock: 0.21 s).
… directive recovery too

typing-import-spin's T2-crash-restarted is the Windows face of the defect T2
watches on Linux and macOS (clangd 23.1 crashes there instead of spinning); an
engine that declares module-directive-recovery has neither.
…y when the engine says so again

mcppls-clangd reports "Keeping import of third-party module <name> textual; no
module unit for it in this project" both for a unit whose scan keeps failing
(partial-scan-standins, reported at every attempt) and once while its provider
index catches up with a database whose provider moved. Under PRoot the second
came 7 s after the move, past DATABASE_REREAD, and typing-import's hello.greet
got a stand-in that pushed greet.cppm out of the database and restarted clangd.
For a module the plan gives a unit, a report is now taken only when repeated
at least 2 s after the first (and the count starts again when its provider
changes); a module the plan has no unit for is taken at once, as before.
…f is not primed again

module_already_built_ found a BMI only in stock clangd's layout,
<source>-<hash>/<command>/<module>.pcm, so with mcppls-clangd bundled every
warm start primed all of a project's modules again: on four cores, xlings'
first ready took 14.1 s against stock clangd's 7.2 s, and every importer's
prerequisites queued behind the 111 prime units (completion p95 at the 1 s
budget, recovery after a kill over 25 s on CI). An owned payload's unit is the
first source path its control block names; the units the owned cache keeps
are read once per plan, as the stock listing is, and the command check from
module-builds.json is unchanged. On four cores the warm start is now ready
in 3.0 s and typing completion p95 is 0.23 s (stock clangd: 7.2 s, 0.19 s).
…o stand-ins for importers and sets no file aside for one (WA-CLANGD-013)
…, for answers, latency while typing, crashes and stalls
…ack, and a probe's syntax error outside the workspace is no project issue
…ort half a second on; the Qt fixture finds the distribution's Qt
… imports, and completion stops waiting on rebuilds
…mes; the module fixtures name clangd's preamble decisions
…ll clock then (#49); a planned unit clangd could not scan is unresolved on its first kept-textual report
…f the workspace's cache budget, not its own 1 GiB
… only, the one platform its workers are isolated on
… built again to the same bytes keeps its importers, and owned lookups try their last slot first
…er's key names its producers' bytes, and a file's last prerequisites are kept
…l preamble holds while its own reads replay unchanged
…r does: rewriting a CRLF file's at the first keystroke rebuilt its preamble
…features of target code (#51)

mcpp 2026.10.11.1 (mcpp-community/mcpp#797) describes its build programs in the build database:
build.mcpp, the libraries of [build-dependencies] and the engine interface, in sets of the
`build-program` configuration whose visible sets stay within it.

- build.mcpp is C++ because mcpp's command says `-x c++`; the translations dropped it with the
  build's other spellings of the language, and clangd refused the file. The plan states the
  language again for a unit whose name tells the Clang driver none.
- The default context reads the targets first: a package in both tables is read as the targets
  build it, a build program's own version of a module the targets provide is read in its own
  context without an ambiguity, and build programs with a toolchain of their own (a cross build)
  are read in the context <package>:build, which says so in the file.
- The module index knows each unit's configuration: an import across the two is an error at the
  import (build-program-module, target-module), import completion offers one configuration's
  modules, and module navigation prefers them.
- mcpp's engine interface is read-only: formatting, code actions and renames never edit it, and
  VS Code opens <MCPP_HOME>/engine/*/modules read-only.
- A .mcpp file no set describes (an older mcpp) is not given to clangd, and says which mcpp
  describes it.
- Every editor integration treats .mcpp as C++; Zed's README gives the file_types setting.

Tests: unit tests for the plan and the module index; the conformance fixture mcpp-build-programs
against a real mcpp 2026.10.11.1 (installed in CI beside the pinned mcpp, MCPP_BUILD_PROGRAMS_VERSION)
on Linux, macOS and Windows, mcpp-build-programs-cross from its recorded output with a second
toolchain, an undescribed build.mcpp in mcpp-emit, and the runner's new `request` check. S1 §7
describes the configuration; validate.py checks mcpp's build-program sets.
…Windows runner measures with a margin; the engine share keeps holding clangd to answering
…ase, checksum checked: the index snapshot the pinned xlings syncs did not have 2026.10.11.1
… Windows runner measures (0.72-0.90) with a margin
…ppls.0) on every platform; the candidate engine file goes
…ty array under set -u is unbound to its bash 3.2)
@Sunrisepeak
Sunrisepeak marked this pull request as ready for review October 11, 2026 13:29
@Sunrisepeak
Sunrisepeak merged commit 5dee2fa into main Oct 11, 2026
9 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[preparation-stalled] module preparation made no progress for a minute (34/39 done)

1 participant