Skip to content

test/goroot: retire stale expectations and classify full-shard outcomes - #2419

Open
cpunion wants to merge 2 commits into
xgo-dev:mainfrom
cpunion:codex/xfail-stability-main-20260826
Open

test/goroot: retire stale expectations and classify full-shard outcomes#2419
cpunion wants to merge 2 commits into
xgo-dev:mainfrom
cpunion:codex/xfail-stability-main-20260826

Conversation

@cpunion

@cpunion cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR retires the fully audited stale hosted expectations and classifies the outcomes found by the complete GOROOT matrix.

  • keeps all 175 previously validated expectation removals;
  • makes notapplicable.yaml global by construction: all 121 version and 9 platform selectors are removed, duplicate rangegen.go/maymorestack.go entries are collapsed, and stack.go is added once globally;
  • accepts either success or an ordinary failure for a not-applicable case while keeping resource-guard failures fatal;
  • globally host-skips fixedbugs/issue16016.go, chan/goroutines.go, and chanlinear.go, which can exhaust LLGo's one-thread-per-goroutine host model;
  • adds ordinary xfail coverage for index0.go, fixedbugs/issue34123.go, fixedbugs/issue52612.go, and Linux Go 1.26 convert5.go, and consolidates heapsampling.go globally;
  • makes the observed typeparam/chans.go flake global.

The repository now has 133 unique global not-applicable entries, with no version/platform selectors. stack.go is no longer present in flakes.

Full-shard findings

The unmodified 16-shard Darwin/Linux × Go 1.25.0/1.26.5 run on the prior head received all 16 reports and observed all 4,170 selected cases:

  • 4,170 selected / 4,170 observed;
  • 4,125 classified passes, 41 failures, 4 skips;
  • 16 unique failing paths;
  • zero failure tuple intersections with the 175 validated removals.

Those 16 paths are now handled as follows:

The older full run also exposed chanlinear.go; it is now a global host skip. typeparam/chans.go was independently reproduced as flaky and is globally classified.

Relationship to other PRs

Validation

The current integration head cb9de75e5 is exact PR head 9501a7ba6 plus #2416 head 78a9f63bc; .github/workflows/goroot.yml is byte-identical to main. Go 1.24 is not a compatibility target; supported runtime validation is Go 1.25/1.26.

@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Full GOROOT validation run (16 configured OS/Go-version/shard jobs):

https://github.com/cpunion/llgo/actions/runs/32942085570

This run is pinned to head a94c25f7f25a0127046022ae447380db477f12b6.

@codecov

codecov Bot commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

This is a clean, well-scoped change. The bulk of the diff is data (retiring 187 validated stability expectations from xfail.yaml), and the code changes are small and targeted:

  • The procPin relocation from runtime/internal/lib/runtime/sync_runtime_llgo.go into the new core file runtime/internal/runtime/procpin.go is a pure relocation — the mutex/Once logic is byte-for-byte identical to the pre-existing code, and the darwin || linux build tag is preserved. The complementary !darwin && !linux no-op in runtime/_patch/sync/atomic/value_other.go keeps the linkname coverage exhaustive with no overlap. No behavior change.
  • runtime.NumGoroutine now reports live LLGo runtime contexts; calling getg() first to register a lazily-initialized main/foreign thread before reading the count is correct (getg -> initRuntimeContext -> retainG).
  • TestRuntimeNumGoroutineTracksWorkers is correctly designed around the worker-release race, using a deadline-bounded Gosched poll rather than an immediate assertion.
  • Forcing GOTOOLCHAIN=local and running stdlib discovery from an isolated temp dir is a sensible hardening/isolation improvement, and the stack.go global not-applicable reason ("recursively grows through 8,000 large frames") matches upstream test/stack.go (recur(8000)).

No blocking issues found. A few low-severity maintainability/documentation notes are inline.

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review notes (follow-up)

The two low-severity maintainability/documentation notes from my summary review are attached inline below. No blocking issues.

Comment thread runtime/internal/runtime/proc.go Outdated
Comment thread test/goroot/runner_test.go Outdated
@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

The first dispatched run was cancelled after all started Linux shards exposed one deterministic runner setup error: the temporary GOPATH still linked the checkout under the pre-migration github.com/goplus/llgo path, while the module/runtime imports use github.com/xgo-dev/llgo. The failures happened before individual expectation validity could be evaluated.

Fixed in f9eeed831 with a path-level regression test and a bounded real compile/run of 235.go on Go 1.26.5.

Replacement full 16-shard validation run (exact updated head):
https://github.com/cpunion/llgo/actions/runs/32942797439

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown

LLGo baseline benchmarks

9501a7ba60aa | workflow run | long-term charts

Program measurements

Platform Workload File size vs base Text size vs base Build vs base Run vs base
Linux cprintf 19256 B 0 B / +0.0% 387 B 0 B / +0.0% 315.243 ms +71.65 ms / +29.4% (worse) 925.272 us +78.26 us / +9.2% (worse)
Linux cprintf-lto 19088 B 0 B / +0.0% 368 B 0 B / +0.0% 270.670 ms +26.2 ms / +10.7% (worse) 895.976 us +117.1 us / +15.0% (worse)
Linux fmtprintf 1653232 B 0 B / +0.0% 501083 B 0 B / +0.0% 2.305 s +7.763 ms / +0.3% (worse) 2.256 ms -13.6 us / -0.6% (better)
Linux fmtprintf-lto 1525824 B 0 B / +0.0% 459700 B 0 B / +0.0% 6.852 s +11.09 ms / +0.2% (worse) 2.249 ms -75.59 us / -3.3% (better)
Linux println 62032 B 0 B / +0.0% 15261 B 0 B / +0.0% 285.753 ms +38.52 ms / +15.6% (worse) 1.102 ms +75.17 us / +7.3% (worse)
Linux println-lto 53840 B 0 B / +0.0% 12882 B 0 B / +0.0% 383.959 ms +3.505 ms / +0.9% (worse) 1.070 ms +34.26 us / +3.3% (worse)
macOS cprintf 84480 B 0 B / +0.0% 16493 B 0 B / +0.0% 615.907 ms +167.2 ms / +37.3% (worse) 3.717 ms +808.2 us / +27.8% (worse)
macOS cprintf-lto 100704 B 0 B / +0.0% 16473 B 0 B / +0.0% 776.654 ms +370.7 ms / +91.3% (worse) 10.016 ms +6.938 ms / +225.4% (worse)
macOS fmtprintf 1498256 B 0 B / +0.0% 882133 B 0 B / +0.0% 4.613 s +1.692 s / +57.9% (worse) 7.460 ms +939.3 us / +14.4% (worse)
macOS fmtprintf-lto 1208976 B 0 B / +0.0% 875129 B 0 B / +0.0% 7.517 s +264.2 ms / +3.6% (worse) 7.971 ms +1.596 ms / +25.0% (worse)
macOS println 114832 B 0 B / +0.0% 34849 B 0 B / +0.0% 712.881 ms +344.2 ms / +93.4% (worse) 4.348 ms +738.2 us / +20.5% (worse)
macOS println-lto 118656 B 0 B / +0.0% 32489 B 0 B / +0.0% 788.978 ms +232.5 ms / +41.8% (worse) 5.115 ms +2.236 ms / +77.7% (worse)
Core language and compiler benchmarks
Platform Benchmark ns/op vs base
Linux BenchmarkLookupPCRandom 8.291 ns/op +0.027 ns/op / +0.3% (worse)
Linux BenchmarkMergeCompilerFlags 103.100 ns/op +1.8 ns/op / +1.8% (worse)
Linux BenchmarkMergeLinkerFlags 65.880 ns/op +0.8 ns/op / +1.2% (worse)
Linux BenchmarkChannelBuffered 41.090 ns/op +0.97 ns/op / +2.4% (worse)
Linux BenchmarkChannelHandoff 23954 ns/op +967 ns/op / +4.2% (worse)
Linux BenchmarkDefer 43.690 ns/op +2.56 ns/op / +6.2% (worse)
Linux BenchmarkDirectCall 0.259 ns/op +0.0025 ns/op / +1.0% (worse)
Linux BenchmarkGlobalRead 0.373 ns/op +0.0114 ns/op / +3.2% (worse)
Linux BenchmarkGlobalWrite 7.154 ns/op +0.142 ns/op / +2.0% (worse)
Linux BenchmarkGoroutine 40242 ns/op +6343 ns/op / +18.7% (worse)
Linux BenchmarkInterfaceCall 4.336 ns/op -0.021 ns/op / -0.5% (better)
Linux BenchmarkRuntimeGetG 0.754 ns/op +0.0265 ns/op / +3.6% (worse)
macOS BenchmarkLookupPCRandom 15.500 ns/op +0.83 ns/op / +5.7% (worse)
macOS BenchmarkMergeCompilerFlags 147.900 ns/op +22.6 ns/op / +18.0% (worse)
macOS BenchmarkMergeLinkerFlags 99.880 ns/op +17.64 ns/op / +21.4% (worse)
macOS BenchmarkChannelBuffered 29.440 ns/op -2.54 ns/op / -7.9% (better)
macOS BenchmarkChannelHandoff 8842 ns/op +1459 ns/op / +19.8% (worse)
macOS BenchmarkDefer 42.160 ns/op -8.69 ns/op / -17.1% (better)
macOS BenchmarkDirectCall 1.111 ns/op -0.312 ns/op / -21.9% (better)
macOS BenchmarkGlobalRead 1.051 ns/op -0.274 ns/op / -20.7% (better)
macOS BenchmarkGlobalWrite 1.054 ns/op -0.51 ns/op / -32.6% (better)
macOS BenchmarkGoroutine 51952 ns/op +14836 ns/op / +40.0% (worse)
macOS BenchmarkInterfaceCall 5.218 ns/op -1.948 ns/op / -27.2% (better)
macOS BenchmarkRuntimeGetG 2.488 ns/op -0.152 ns/op / -5.8% (better)

Compared with 6fe8babd468c measured in the same runner job.

@cpunion
cpunion force-pushed the codex/xfail-stability-main-20260826 branch 2 times, most recently from cf428fa to 6f1a070 Compare August 26, 2026 09:19
@cpunion cpunion changed the title test/goroot: retire validated hosted stability expectations test/goroot: remove validated hosted expectations Aug 26, 2026
@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Fresh deletion-only GOROOT validation run:

https://github.com/cpunion/llgo/actions/runs/32954622318

The integration ref is fa08f98c8: exact #2419 head 6f1a070f6 plus only #2416's workspace-path commit 8ba212556, which is required for the fork/module rename. It does not include #2421 or #2416's compiler fixes. The workflow runs all 16 Darwin/Linux × Go 1.25.0/1.26.5 shards with the existing build/run timeouts and 3 GiB RSS guard.

@cpunion
cpunion force-pushed the codex/xfail-stability-main-20260826 branch from 6f1a070 to 4e5c75d Compare August 26, 2026 09:53
@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Regular CI retry note: the only failed check on 6f1a070f6 was the macOS Intel release-artifact job failing to download a 272 MB artifact after five retries, before testing it. The contribution commit was amended without content changes to trigger a fresh CI run; 6f1a070f6 and current head 4e5c75d90 have the identical tree 5d56480e3735d7d718a62b56ee068e16c974f554.

The full GOROOT run on the temporary integration ref remains an exact tree validation of the current PR content plus the separately owned workspace-path commit.

@cpunion
cpunion force-pushed the codex/xfail-stability-main-20260826 branch from 4e5c75d to 408e13e Compare August 26, 2026 10:28
@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

CI retry: the macOS Go job on 4e5c75d9 failed because GitHub returned HTTP 502 while downloading goplus/compiler-rt; a later attempt in the same job downloaded it successfully. I refreshed the fork contribution commit to retrigger checks without changing source content. New head: 408e13e1444e8da5673a24681b204153a32f1204; tree: 5d56480e3735d7d718a62b56ee068e16c974f554 (identical to 4e5c75d9).

@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Final deletion-only GOROOT result: https://github.com/cpunion/llgo/actions/runs/32954622318

  • all 16 Darwin/Linux × Go 1.25.0/1.26.5 shard artifacts were produced;
  • selected = 4,170 and observed = 4,170, so no selected case was omitted;
  • the workflow is red because main currently has 41 unclassified failures across 17 case paths;
  • conservative path-only comparison against all 176 removed entries (100 unique paths) found zero overlap; exact platform/version/directive matching therefore also has zero overlap;
  • all 4 skips are selectors deliberately retained in this deletion-only PR.

The integration commit fa08f98c8 contains the PR tree 5d56480e3735d7d718a62b56ee068e16c974f554 plus only #2416 workspace-path commit 8ba212556; it contains none of #2421 and none of #2416 compiler fixes. No deleted expectation needs to be restored.

@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Focused GOROOT pass run for the deletion set only: https://github.com/cpunion/llgo/actions/runs/32961976670

The temporary fork ref keeps the exact #2419 tree, plus #2416 workspace-path setup and a workflow-only filter commit. It runs every removed selector rather than the unrelated full GOROOT corpus:

  • Darwin/Linux × Go 1.24.0/1.26.5 × 2 shards (8 jobs);
  • 99/61/10/9 removed entries in the four platform/version lanes, covering 97/61/10/9 unique directive/path executions;
  • versionless selectors run in both Go lanes;
  • GOMAXPROCS=1, 3-minute build timeout, 1-minute run timeout, 3 GiB RSS cap, and 90-minute job cap.

Other PR checks are intentionally not being used as the gate until this focused GOROOT run passes.

@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Focused GOROOT replacement run: https://github.com/cpunion/llgo/actions/runs/32962217541

The first focused run was cancelled after the Go 1.24 lanes failed before case selection: the repository go.mod requires Go 1.25+, so Go 1.24 cannot compile the test runner. The replacement keeps Go 1.24 as the target GOROOT/baseline but uses Go 1.26.5 to compile the repository runner. The candidate set and all resource bounds are unchanged.

@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Supported-toolchain focused GOROOT run: https://github.com/cpunion/llgo/actions/runs/32962609618

Per maintainer direction, Go 1.24 is not a compatibility target. This final matrix runs only Go 1.25.0 and 1.26.5 on Darwin/Linux. It covers every removed selector applicable to those supported toolchains (2/61/1/9 selectors by platform/version lane); the 106 Go1.24-only entries are deletion of unsupported compatibility metadata, not runtime claims. Resource bounds remain GOMAXPROCS=1, 3-minute build, 1-minute run, 3 GiB RSS, and 90-minute job cap.

@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Focused run finding: https://github.com/cpunion/llgo/actions/runs/32962609618 completed 72/73 applicable selectors successfully. Darwin Go 1.26 rangegen.go crossed the strict RSS guard (3076.7 MiB > 3072 MiB) while baseline Go compiled the generated program. Because it failed a bounded validation run, its Go1.26 20-minute timeout entry will be retained; the unsupported Go1.24 entry remains removed.

Replacement validation for the resulting 175-entry deletion set (60 Darwin Go1.26 selectors instead of 61): https://github.com/cpunion/llgo/actions/runs/32963761626. The PR branch itself is intentionally not updated until this GOROOT run passes, so other checks are not retriggered early.

@cpunion
cpunion force-pushed the codex/xfail-stability-main-20260826 branch from 408e13e to 8f990d5 Compare August 26, 2026 11:46
@cpunion
cpunion force-pushed the codex/xfail-stability-main-20260826 branch from 8f990d5 to 47647ff Compare August 26, 2026 11:47
@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Final focused GOROOT gate passed: https://github.com/cpunion/llgo/actions/runs/32963761626

All four supported platform/toolchain jobs are green and their artifacts report:

  • Darwin Go 1.25: 2 selected / 2 observed / 2 passed;
  • Darwin Go 1.26: 60 / 60 / 60;
  • Linux Go 1.25: 1 / 1 / 1;
  • Linux Go 1.26: 9 / 9 / 9;
  • total: 72 selected / 72 observed / 72 passed, 0 failed, 0 skipped.

Final PR head 47647ffc8 retains the Darwin Go1.26 rangegen.go timeout found necessary by the strict RSS run. The final semantic diff is 175 deletions and zero additions. Other PR checks have now been retriggered from this final head.

@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Final head 47647ffc8 is fully green: 42 checks passed, 1 release job was expectedly skipped, 0 failed, and 0 remain pending. The PR is mergeable with a clean merge state. Both review conversations are resolved. The diff remains one file (test/goroot/xfail.yaml) with 175 semantic deletions and zero additions.

@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Correction to the focused-run evidence: run 32963761626 used a temporary modified goroot.yml, so it is diagnostic evidence only and is not the final GOROOT gate. That temporary remote branch has been deleted.

The final full-shard run uses the repository workflow unchanged: https://github.com/cpunion/llgo/actions/runs/32972501873

Integration head c5668df9d is exact #2419 head 47647ffc8 plus only workspace-path commit 8ba212556. .github/workflows/goroot.yml is byte-identical to the PR/base version. This run executes the original 16 Darwin/Linux × Go1.25/1.26 × 4-shard jobs.

@cpunion cpunion changed the title test/goroot: remove validated hosted expectations test/goroot: retire stale expectations and classify full-shard outcomes Aug 26, 2026
@cpunion

cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator Author

Scope updated from deletion-only based on the complete 16-shard result.

  • notapplicable.yaml is now genuinely global: no version/platform selectors, duplicates collapsed, and stack.go added once.
  • not-applicable success and ordinary failure are both classified; resource-guard failures remain fatal.
  • the remaining observed semantic/runtime errors have ordinary xfail coverage; unbounded thread-creation cases use host skips instead of being executed as xfails.
  • fixedbugs/bug449.go is still left entirely to ssa: lower large array comparisons to runtime equality algorithms #2416.

New head: 9501a7ba6. The replacement unmodified 16-shard run is https://github.com/cpunion/llgo/actions/runs/32980102573.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant