Skip to content

Benchmark project builds and output sizes in CI - #9272

Merged
cknitt merged 1 commit into
worker-allocations-reportfrom
project-build-benchmarks
Nov 9, 2026
Merged

cknitt merged 1 commit into
worker-allocations-reportfrom
project-build-benchmarks

Conversation

@cknitt

@cknitt cknitt commented Nov 9, 2026

Copy link
Copy Markdown
Member

The benchmarks job replays single bsc calls only, so the build system's own work (scanning, dependency graph, freshness checks, in-process .cmi loading, scheduling) never showed up. This adds:

  • Project builds: tests/tests (~680 modules) built with rescript build on one worker domain (REWATCH_COMPILER_DOMAINS=1): clean, no-op, after editing a leaf test, and after editing test_utils.res (~200 dependents). Allocated words for all four, including the workers' allocations that the build reports with REWATCH_REPORT_WORKER_ALLOCATIONS (Report the build's worker allocations when asked #9271, below this PR). Instructions under cachegrind for the no-op and leaf-edit builds only: the bigger builds would take minutes, the replayed builds cover their compiler work, and with two domains collecting their instruction counts depend on interleaving (the shared edit differed by 9.5% between base and head of an earlier run). Both sides build the base's tests/tests from a fixed directory, with -warn-error dropped, so new tests or warnings in a PR can't break either build.
  • Output sizes: total .cmi / .cmti / .cmt / .cmj bytes for runtime, Belt and tests/tests, plus the playground's compiler.js. A new "Output size" section, alerting like the others at 1%.

Local runs (linux-arm64): clean build 1.4 s, the other three 0.1–0.6 s; repeated runs agree within 0.0004% (allocations), sizes are identical.

Job time: 266 s in an earlier run of this change (was 234 s on master; runner speed varies by more than that). The two playground builds are 75 s of it; the critical path (Windows build) stays ~550 s.

Replaces #9270, split so that this PR's base already reports the workers' allocations.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Kbf3gGPsF3fHZn9rBxA7Hm

The benchmarks only replayed single bsc calls, so the build system's own
work never showed. They now also build the base's tests/tests on one worker
domain (clean, no-op, leaf edit, shared edit), adding the workers'
allocations the build reports, and measure the .cmi/.cmti/.cmt/.cmj sizes
and the playground bundle. Only the no-op and leaf-edit builds run under
cachegrind: the larger ones take minutes, and with two domains collecting
their instructions depend on how the domains interleave.

Review: Why build the base's tests/tests on both sides? The base compiler
builds them already, so a pull request adding tests with new syntax can't
break the base measurement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gGPsF3fHZn9rBxA7Hm
Signed-off-by: Christoph Knittel <ck@cca.io>
@cknitt
cknitt added this pull request to stack #9273 November 9, 2026 06:47
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

Changelog

No changelog entry.

Semantic Diff

1 commit · Full page

Benchmark project builds and output sizes in CI

1673f76 · cknitt + Claude · Nov 09, 2026, 06:46 UTC

The benchmarks only replayed single bsc calls, so the build system's own work never showed. They now also build the base's tests/tests on one worker domain (clean, no-op, leaf edit, shared edit), adding the workers' allocations the build reports, and measure the .cmi/.cmti/.cmt/.cmj sizes and the playground bundle. Only the no-op and leaf-edit builds run under cachegrind: the larger ones take minutes, and with two domains collecting their instructions depend on how the domains interleave.

Review

  • Why build the base's tests/tests on both sides? The base compiler builds them already, so a pull request adding tests with new syntax can't break the base measurement.

@cknitt cknitt added the changelog:skip No release notes needed label Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

✅ Compiler benchmarks

Allocated words

In total: 806,109,534 → 806,109,309 (±0.00%).

None of the 261 benchmarks changed by 0.1% or more.

Instructions

In total: 11,321,687,111 → 11,321,641,809 (±0.00%).

None of the 35 benchmarks changed by 0.1% or more.

Output size

In total: 29,806,856 → 29,806,856 (±0.00%).

None of the 13 benchmarks changed by 0.1% or more.

Pull request 1673f76 compared with base a5e4a65 in this run.

@pkg-pr-new

pkg-pr-new Bot commented Nov 9, 2026

Copy link
Copy Markdown

Open in StackBlitz

rescript

npm i https://pkg.pr.new/rescript@9272

@rescript/belt

npm i https://pkg.pr.new/@rescript/belt@9272

@rescript/darwin-arm64

npm i https://pkg.pr.new/@rescript/darwin-arm64@9272

@rescript/linux-arm64

npm i https://pkg.pr.new/@rescript/linux-arm64@9272

@rescript/linux-x64

npm i https://pkg.pr.new/@rescript/linux-x64@9272

@rescript/runtime

npm i https://pkg.pr.new/@rescript/runtime@9272

@rescript/win32-x64

npm i https://pkg.pr.new/@rescript/win32-x64@9272

commit: 1673f76

@cknitt
cknitt marked this pull request as ready for review November 9, 2026 07:49
@cknitt
cknitt requested a review from fhammerschmidt November 9, 2026 07:49
@cknitt
cknitt merged commit a6ae63c into master Nov 9, 2026
27 checks passed
@cknitt
cknitt deleted the project-build-benchmarks branch November 9, 2026 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog:skip No release notes needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants