Skip to content

windows: integrate Windows into shared CI matrices (R8, depends on #2422) - #2425

Open
cpunion wants to merge 194 commits into
xgo-dev:mainfrom
cpunion:codex/windows-r8-on-r7-integration-20260826
Open

windows: integrate Windows into shared CI matrices (R8, depends on #2422)#2425
cpunion wants to merge 194 commits into
xgo-dev:mainfrom
cpunion:codex/windows-r8-on-r7-integration-20260826

Conversation

@cpunion

@cpunion cpunion commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Depends on #2422. Part of the Windows support proposal in #2325.

Until #2422 merges, GitHub's full diff also contains the stacked R6/R7 work. The R8-only review range is 09a23ef5aa341f6d3c196ccca2785cc34fbed9cc..f29367759ec3fa39e7637121f187e0e4c8af4593.

What this changes

  • Removes the standalone windows.yml workflow and adds windows-2022 to the existing Go, LLGo, GOROOT, standard-library coverage, docs-local, build-cache, and benchmark matrices.
  • Keeps every compiler/runtime/FFI/import audit from the dedicated workflow while making the ordinary shared jobs the source of truth for Windows coverage.
  • Makes the existing integration helpers and C/C++/socket/thread/sync/export demos portable to native Windows instead of skipping them.
  • Adds native PE section metrics and Windows benchmark execution, using Go 1.26.7 and the R7 LLVM 19/VS2022 bootstrap toolchain.
  • Covers official CPython 3.12 embedding on Windows and keeps generated Windows programs on the MSVC ABI.
  • Makes the LLDB smoke/integration suite portable to Windows, including target-aware formatter activation and Windows-compatible Go debug types and pointer parameters.
  • Keeps the low-level once API ABI-clean on every host: Once.Do accepts the bare void (*)(void) callback used by pthread_once, while the explicit Once.DoFunc adapter retains a capturing Go closure above the C boundary and passes only a fixed callback plus opaque context to native code.

Remote-install coverage is intentionally unchanged, as agreed in #2325. Removing the MSYS2 host-toolchain dependency is R9; clean release artifacts and broader debugger qualification remain R12.

The temporary github.com/cpunion/lib replacements provide the Windows-capable bindings already exercised by the integration branch. Their upstream contribution is staged in goplus/lib#26; the replacements will be removed after an upstream-owned revision is available.

Validation

  • Windows 11 ARM64 VM (8 GiB), running the x64 Go 1.26.7/LLVM 19.1.7/VS2022 toolchain under Windows x64 emulation for windows/amd64 qualification, cold LLGo cache:
    • all 287 root-module packages passed with the Windows external-link configuration;
    • ./test/go passed in 641 seconds;
    • cl passed with 96.6% statement coverage, ssa with 95.1%, internal/crosscompile with 82.2%, and internal/cabi with 86.3%;
    • the LLDB integration suite passed all 224 checks, including the ordinary-C, unsupported-marker, and ambiguous-marker isolation cases;
    • cold-cache Thumb, AVR, RISC-V, WASI, and ESP runtime/toolchain downloads and builds passed;
    • CPython/NumPy normal and DWARF execution matrices passed and imported the official python312.dll.
  • The corrected once boundary passed actual LLGo execution on both macOS ARM64 and native Windows 11 ARM64, covering a raw callback, a captured closure, four concurrent callers with distinct contexts, and nested DoFunc calls.
  • The once IR/golden fixture and the runtime synchronization package tests pass locally.
  • macOS targeted build, Python fixture, output-path, Darwin staging, LLDB launcher, and SSA debug-info tests passed.
  • Same-host R7-vs-R8 macOS measurements were identical for all 24 checked non-Windows binary-size metrics.
  • The full 444 MiB Windows ESP archive passed the restored GNU tar+xz path in 71.8 seconds locally; the hosted Build Cache job completed in 6m33s after that path was restored.
  • The previous full CI head passed every required check, including the 96.31% Codecov patch result; CI for the two follow-up once commits is running.
  • git diff --check, YAML parsing, Bash syntax, PowerShell parsing, Python source compilation, and actionlint all pass.

cpunion added 30 commits August 25, 2026 21:47
@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown

LLGo baseline benchmarks

4b8549e709bb | workflow run | long-term charts

Program measurements

Platform Workload File size vs base Text size vs base Build vs base Run vs base
Linux cprintf 19432 B +288 B / +1.5% (worse) 387 B 0 B / +0.0% 341.566 ms -6.977 ms / -2.0% (better) 1.241 ms -85.78 us / -6.5% (better)
Linux cprintf-lto 19264 B +288 B / +1.5% (worse) 368 B 0 B / +0.0% 328.361 ms +1.345 ms / +0.4% (worse) 1.224 ms -58.61 us / -4.6% (better)
Linux fmtprintf 1652216 B +2880 B / +0.2% (worse) 501874 B +842 B / +0.2% (worse) 2.757 s +9.797 ms / +0.4% (worse) 3.548 ms +132.5 us / +3.9% (worse)
Linux fmtprintf-lto 1524168 B +2304 B / +0.2% (worse) 460334 B +760 B / +0.2% (worse) 8.523 s -247.6 ms / -2.8% (better) 3.394 ms +176.1 us / +5.5% (worse)
Linux println 62264 B +320 B / +0.5% (worse) 15286 B +30 B / +0.2% (worse) 330.133 ms +3.367 ms / +1.0% (worse) 1.657 ms +33.73 us / +2.1% (worse)
Linux println-lto 54096 B +288 B / +0.5% (worse) 12914 B +32 B / +0.2% (worse) 503.432 ms +4.722 ms / +0.9% (worse) 1.817 ms +148.8 us / +8.9% (worse)
macOS cprintf 84480 B 0 B / +0.0% 16669 B +288 B / +1.8% (worse) 383.876 ms +9.087 ms / +2.4% (worse) 2.273 ms +80.42 us / +3.7% (worse)
macOS cprintf-lto 100704 B 0 B / +0.0% 16649 B +288 B / +1.8% (worse) 383.912 ms +17.61 ms / +4.8% (worse) 3.028 ms +807.5 us / +36.4% (worse)
macOS fmtprintf 1512800 B +16416 B / +1.1% (worse) 882816 B +988 B / +0.1% (worse) 2.417 s -48.69 ms / -2.0% (better) 4.798 ms -809 us / -14.4% (better)
macOS fmtprintf-lto 1208464 B 0 B / +0.0% 875856 B +1004 B / +0.1% (worse) 5.031 s -1.089 s / -17.8% (better) 4.862 ms -39.88 us / -0.8% (better)
macOS println 114784 B 0 B / +0.0% 35101 B +304 B / +0.9% (worse) 330.634 ms -9.494 ms / -2.8% (better) 2.862 ms -14.58 us / -0.5% (better)
macOS println-lto 118656 B 0 B / +0.0% 32753 B +304 B / +0.9% (worse) 448.421 ms -20.4 ms / -4.4% (better) 2.837 ms -787.8 us / -21.7% (better)
Windows cprintf 120320 B new 65787 B new 782.552 ms new 3.358 ms new
Windows cprintf-lto 119808 B new 65723 B new 804.659 ms new 3.150 ms new
Windows fmtprintf 1608192 B new 687414 B new 5.018 s new 7.916 ms new
Windows fmtprintf-lto 1650176 B new 686022 B new 10.757 s new 8.201 ms new
Windows println 194560 B new 120534 B new 780.672 ms new 6.521 ms new
Windows println-lto 190976 B new 117606 B new 987.159 ms new 6.312 ms new
Core language and compiler benchmarks
Platform Benchmark ns/op vs base
Linux BenchmarkLookupPCRandom 11.460 ns/op -0.41 ns/op / -3.5% (better)
Linux BenchmarkMergeCompilerFlags 138.500 ns/op -1.3 ns/op / -0.9% (better)
Linux BenchmarkMergeLinkerFlags 89.550 ns/op -0.34 ns/op / -0.4% (better)
Linux BenchmarkChannelBuffered 36.530 ns/op +0.89 ns/op / +2.5% (worse)
Linux BenchmarkChannelHandoff 24319 ns/op +500 ns/op / +2.1% (worse)
Linux BenchmarkDefer 48.340 ns/op +0.19 ns/op / +0.4% (worse)
Linux BenchmarkDirectCall 1.914 ns/op +0.255 ns/op / +15.4% (worse)
Linux BenchmarkGlobalRead 1.977 ns/op +0.024 ns/op / +1.2% (worse)
Linux BenchmarkGlobalWrite 2.622 ns/op -0.044 ns/op / -1.7% (better)
Linux BenchmarkGoroutine 28630 ns/op -3494 ns/op / -10.9% (better)
Linux BenchmarkInterfaceCall 8.449 ns/op +0.25 ns/op / +3.0% (worse)
Linux BenchmarkRuntimeGetG 2.315 ns/op +0.631 ns/op / +37.5% (worse)
macOS BenchmarkLookupPCRandom 10.460 ns/op -1.26 ns/op / -10.8% (better)
macOS BenchmarkMergeCompilerFlags 98.700 ns/op -32.2 ns/op / -24.6% (better)
macOS BenchmarkMergeLinkerFlags 66.190 ns/op -40.51 ns/op / -38.0% (better)
macOS BenchmarkChannelBuffered 21.410 ns/op -3.84 ns/op / -15.2% (better)
macOS BenchmarkChannelHandoff 6453 ns/op -2470 ns/op / -27.7% (better)
macOS BenchmarkDefer 28.820 ns/op -6.75 ns/op / -19.0% (better)
macOS BenchmarkDirectCall 0.943 ns/op -0.1131 ns/op / -10.7% (better)
macOS BenchmarkGlobalRead 0.943 ns/op -0.1804 ns/op / -16.1% (better)
macOS BenchmarkGlobalWrite 0.953 ns/op -0.1026 ns/op / -9.7% (better)
macOS BenchmarkGoroutine 27756 ns/op +1841 ns/op / +7.1% (worse)
macOS BenchmarkInterfaceCall 4.086 ns/op -1.155 ns/op / -22.0% (better)
macOS BenchmarkRuntimeGetG 1.884 ns/op -0.226 ns/op / -10.7% (better)
Windows BenchmarkLookupPCRandom 15.930 ns/op new
Windows BenchmarkMergeCompilerFlags 575.800 ns/op new
Windows BenchmarkMergeLinkerFlags 515.500 ns/op new
Windows BenchmarkChannelBuffered 34.780 ns/op new
Windows BenchmarkChannelHandoff 911 ns/op new
Windows BenchmarkDefer 54.570 ns/op new
Windows BenchmarkDirectCall 1.547 ns/op new
Windows BenchmarkGlobalRead 1.859 ns/op new
Windows BenchmarkGlobalWrite 2.457 ns/op new
Windows BenchmarkGoroutine 97440 ns/op new
Windows BenchmarkInterfaceCall 8.999 ns/op new
Windows BenchmarkRuntimeGetG 2.479 ns/op new

Compared with e1791699056b measured in the same runner job. Platforms without a paired baseline are marked new.

@codecov

codecov Bot commented Aug 26, 2026

Copy link
Copy Markdown

@cpunion
cpunion marked this pull request as ready for review August 27, 2026 01:43
@cpunion

cpunion commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

@fennoai please review the completed R8 shared-CI integration. All Linux, macOS, and Windows jobs and the Codecov patch gate are passing.

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Windows R8-on-R7 integration

Reviewed the R8-only range (09a23ef..f293677, 84 files) — the actual scope of this PR per the description; the full base-diff also carries the stacked #2422 work, which is out of scope here.

This is a large, well-executed portability change: CI matrix consolidation (removing windows.yml), platform-split demos, portable LLDB tooling, and Windows PE metrics. Test coverage is strong — nearly every production change is paired with a test (TestAddPESections, TestWindowsTarXzTools, TestWindowsDebugPointerParameter, TestRunQuietIncludesCommandOutput, etc.). Comments explain the Windows-specific rationale well. No high-severity or blocking issues found; the notes below are refinements.

Cross-cutting notes (no reliable single inline location):

  • Supply-chain — github.com/cpunion/lib replacement. The root go.mod:30 and all demo/tool go.mod files replace github.com/goplus/lib with a personal fork pinned to v0.0.0-20260822071805-ebe22363a225. The PR body already flags this as temporary, which is right — the important one is the root go.mod (the compiler build itself), not just the demos. go.sum hashes guard against tampered fetches, but this should revert to an upstream-owned tag before release. Tracking this is worthwhile.
  • internal/crosscompile/fetch.go.tar.xz extraction trust path. The Go-native extractTarGz/extractZip paths enforce a path-traversal (illegal file path) guard; the .tar.xz path shells out to external tar/bsdtar/MSYS2 tar with --force-local and relies on the tool's own protections. Combined with downloads still having no checksum/signature verification (downloadFile), a moved/altered release archive could write outside dest. This is largely pre-existing (the PR only changed tool selection), but pinning checksums for the ESP/WASI archives would close the gap.
  • _demo/c/syncdebug/once_default.go vs once_windows.go. The two files implement the same llsync.Once test with gratuitously different structure (package-level vars + named funcs vs. locals + closures). If the closure form works on Windows it almost certainly works everywhere; a shared implementation (or a comment explaining why they must differ) would reduce drift risk.

Verified and dismissed: an internal reviewer flagged test_windows_runtime.ps1 referencing a missing check_windows_imports.ps1, but that file exists at .github/workflows/check_windows_imports.ps1 — not an issue.

Comment thread cmd/internal/lldb/llgo_plugin.py Outdated
Comment thread benchmark/baseline/main.go
@fennoai

fennoai Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

FennoAI is processing your request, please wait a moment...

@cpunion

cpunion commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review follow-up:

  • Addressed both inline findings in c71c807 and resolved their threads.
  • Unified the duplicated syncdebug Once fixture across hosts in 2910a58; Windows still exercises the closure form.
  • The cpunion/lib replacement remains an explicitly staged dependency and must be removed through an upstream-owned revision before R12 release qualification.
  • Archive checksum/traversal hardening is pre-existing download infrastructure, not part of the shared-matrix change; it should remain a separate scoped follow-up rather than expanding R8.

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: R8 shared-CI Windows integration

Reviewed the locked R8-only range e1791699...f2936775 (357 files, ~13.7k/-2.2k). Focus was the production runtime/codegen surface — Windows fault handling, SEH unwind, semaphores, threading, RNG, print, setjmp, and the CI matrix reshaping. Four review passes ran (quality, performance, security, docs).

Overall: high-quality, carefully engineered work. The _default/_windows/_windows_<arch> build-tag partitioning is, in nearly every family I checked (setjmp, fault, cpuprof, sema, foreign-thread, ffi-abi, gc-threads, llvm-config), provably complete and non-overlapping, and the native C/asm bridges (rand, fault handler, print, syscall, setjmp, thread/FLS) are bounds-checked with no memory-safety, injection, or weak-randomness issues found. CI trust boundaries are sound: pull_request (not pull_request_target), contents: read, secrets confined to appropriate steps, toolchain pins consistent (Go 1.26.7 / LLVM 19 / VS2022).

A few items worth a look, left as inline comments. None are blocking.

Notes (not inline):

  • Supply-chain (Medium): replace github.com/goplus/lib => github.com/cpunion/lib is added to the root and every sub-module go.mod (_xtool, doc/_readme, _demo/{c,go,py,embed}). Content is pinned + go.sum-verified, and the PR body flags it as temporary — recommend gating release on moving these commits to an org-owned/tagged upstream so a released version never depends on a personal fork. (Inline note on root go.mod.)
  • CI hardening (Low, pre-existing): .github/workflows/benchmark.yml interpolates ${{ github.event.pull_request.base.ref }} directly into a run: block. The line itself is pre-existing (this PR only adds shell: bash to that step), exploitability is low (base ref is an existing branch name, pull_request trigger, contents: read, no secrets), but normalizing to the env:-indirection pattern already used elsewhere in the same file would be good hygiene.

Verification of the described validation matrix (287 packages, ./test/go, coverage %, LLDB suite, cross-target runtime builds) relies on the CI results reported as green; I reviewed the code and configuration, not the live job logs.

Additional findings

  • runtime/internal/lib/runtime/runtime_windows.go:1: [P2] Windows build-tag overlaps baremetal/wasm (duplicate LLGoFiles): runtime_windows.go is tagged //go:build windows, while sibling runtime_baremetal.go is //go:build baremetal || wasm. A windows && (baremetal || wasm) build satisfies both, and both declare LLGoPackage/LLGoFiles constants — that configuration would fail with duplicate declarations. runtime_default.go in the same package correctly excludes windows (!baremetal && !wasm && !windows), so this is the one file in the family whose tag isn't provably disjoint. If windows && baremetal/windows && wasm is not a real target the impact is latent, but tightening to //go:build windows && !baremetal && !wasm removes the ambiguity at zero cost.

Comment thread go.mod Outdated

replace github.com/xgo-dev/llgo/runtime => ./runtime

replace github.com/goplus/lib => github.com/cpunion/lib v0.0.0-20260822071805-ebe22363a225

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Temporary personal-fork replace should not ship in a release

github.com/goplus/lib (compiled into every LLGo-built binary) is redirected to the personal fork github.com/cpunion/lib, and this replace is duplicated across _xtool/go.mod, doc/_readme/go.mod, and _demo/{c,go,py,embed}/go.mod. The pseudo-version is pinned and go.sum-verified, so content tampering is guarded, but a personal fork is a weaker trust/longevity anchor for a shared project. The PR body notes this is temporary — recommend gating merge/release on moving these commits into an org-owned repo (or a tagged upstream release of xgo-dev/lib) so no released version depends on a personal fork.

}
}

func semaRelease(addr *uint32) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] semaRelease issues a wake syscall on every uncontended release

semaRelease calls psync.WakeUint32 (→ WakeByAddressSingle, a kernel transition) unconditionally on every release. This Windows path keeps no waiter count — semaAcquire is a bare CAS loop plus WaitUint32, with no waiters field — so the release side can never elide the wake when nobody is waiting. The unix design (sema_cond_llgo.go) only signals when waiters != 0. Since this backs sync_runtime_Semrelease/poll_runtime_Semrelease, high-frequency uncontended releases (waitgroup counter reaching zero, channel/select semaphores) pay a syscall per op that unix avoids. Not a correctness bug — spurious wakeups are handled by the acquire loop — but a per-op syscall on the fast path. A per-address waiter count maintained in the acquire loop would let release skip the wake.

@cpunion

cpunion commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

CI follow-up: the attempted syncdebug deduplication in 2910a58 was not valid. POSIX pthread_once accepts a bare C function pointer, while an LLGo capturing closure is represented as {function, environment}; the Windows wrapper deliberately supplies the closure bridge. This caused the same LLVM verifier failure in the macOS primary job and both Go development jobs.

Reverted that change in b7fce75, documented the required platform distinction in 4b8549e, and locally verified the Unix fixture through LLGo. The two inline review fixes remain intact.

@cpunion

cpunion commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

Once ABI follow-up:

  • Once.Do now accepts the same bare void (*)(void) callback as POSIX pthread_once; Windows adapts only the native INIT_ONCE callback signature and no longer sees LLGo's closure representation.
  • Capturing closures use the explicit Once.DoFunc upper-layer adapter. Go retains the closure, while C receives a fixed callback and opaque context pointer. Concurrent distinct contexts and nested calls are covered.
  • The corresponding dependency is now available for review as c: support Windows MSVC interop goplus/lib#26; its CI is passing.
  • Actual LLGo execution passed on macOS ARM64 and Windows 11 ARM64, and the once IR fixture plus runtime synchronization package tests pass locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant