runtime,internal/poll,loader: wasip2 pollable poll-integration + TCP I/O#5390
Open
achille-roussel wants to merge 1 commit into
Open
runtime,internal/poll,loader: wasip2 pollable poll-integration + TCP I/O#5390achille-roussel wants to merge 1 commit into
achille-roussel wants to merge 1 commit into
Conversation
3baa42c to
39b8954
Compare
Member
|
@achille-roussel can you please rebase this PR against the latest Also, can you please remove Claude as your co-author? Thank you! |
Mirrors PR tinygo-org#5386's wasip1 work for wasip2. The cooperative scheduler's idle path now calls wasi:io/poll.Poll over a combined list of (clock pollable, registered pollables) instead of blocking the wasm module on a single monotonic-clock subscription, so goroutines doing TCP I/O can park while the scheduler runs other goroutines. Plumbing components: - runtime/netpoll_wasip2.go: pollable-keyed pollDesc registry; pollIO builds one combined wasi:io/poll.Poll call (clock pollable + active pollables). Linkname-exposed runtime_netpoll_addpollable_wasip2 / done / pdfired / wake for internal/poll and future net. - runtime/scheduler_idle_wasip2.go + scheduler_idle_wasip2_none.go: cooperative-variant sleepTicks / waitForEvents that route through pollIO; non-coop fallback uses monotonicclock.Block. Mirrors the wasip1 structure introduced in 7000e7b. - runtime/runtime_wasip2.go: sleepTicks moved out to the scheduler_idle_wasip2*.go files. - runtime/wait_other.go: build tag tightened to exclude wasip2. internal/poll surface: - internal/poll/fd_wasip2.go: WasipNFD wraps a (TcpSocket, InputStream, OutputStream) triple. DialTCPWasip2, ListenTCPWasip2, Accept, Read, Write, Close, SetDeadline*. Each blocking op tries the wasi call, on would-block subscribes, parks, retries — same pattern as the wasip1 internal/poll.FD but pollable-keyed. Linkname-friendly Wasip2TCP{Listen,Dial,Accept,Read,Write,Close,SetDeadline} wrappers for test / future net callers. - internal/poll/errors_wasip.go: ErrFileClosing / ErrNetClosing / ErrDeadlineExceeded / ErrNoDeadline extracted from fd_wasip1.go to a wasip1||wasip2 shared file. Loader change: - loader/goroot.go: listGorootMergeLinks now filters TinyGo files by //go:build constraints (via go/build.Context.MatchFile) before deciding "TinyGo owns this directory". Files that don't match the current target no longer cause upstream Go files at the same level to be dropped. Unblocks per-target overrides in directories like src/net/ for future net.wasip2 work without disturbing wasip1. End-to-end verification: $ wasmtime run -Sinherit-network -Stcp ./tcpecho_wasip2.wasm & listening on 127.0.0.1:9999 tick 1 tick 2 tick 3 $ echo hello | nc 127.0.0.1 9999 hello # echoed by the wasm $ # two concurrent clients echo cleanly while ticker keeps ticking The test program (not shipped) uses //go:linkname to drive the internal/poll TCP helpers directly, since TinyGo doesn't yet have a net.Listen / net.Dial path on wasip2 (upstream Go's net doesn't build for wasip2 due to cgo_linux.go reaching for Linux headers). The src/net/ wasip2 wrappers are out of scope for this PR and tracked as follow-up — once they land, callers will use net.Listen / Dial directly and the linkname wrappers can drop. Wasip1 regression sweep: tcpecho.wasm still passes; time.Sleep / parkfile / parksynth unchanged.
39b8954 to
d9e0859
Compare
Contributor
Author
|
Done. I'm exploring how to add a few tests to verify the new behavior as well. |
b14e4c3 to
d9e0859
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #5386. Mirrors the wasip1 work for wasip2. The cooperative scheduler's idle path now calls
wasi:io/poll.Pollover a combined list of (clock pollable, registered pollables) instead of blocking the wasm module on a singlemonotonic-clocksubscription, so goroutines doing TCP I/O can park while the scheduler runs other goroutines.End to end:
Two concurrent
ncclients both echo while the ticker keeps progressing — the cooperative scheduler keeps running goroutines parked onsock_recvvia the new pollable registry.How it works
Wasip2's polling primitive is
wasi:io/poll.Poll(list<own<pollable>>) -> list<u32>taking a list of pollable resource handles. Each blocking wasi operation has asubscribe()that yields a pollable.The wasip2 path mirrors the wasip1 PR's structure file-for-file:
runtime/netpoll_wasip1.go(FD-keyedpoll_oneoffregistry)runtime/netpoll_wasip2.go(pollable-keyedwasi:io/poll.Pollregistry)runtime/scheduler_idle_wasip1.go(cooperativesleepTicksroutes viapollIO)runtime/scheduler_idle_wasip2.go(cooperativesleepTicksroutes viapollIO)runtime/scheduler_idle_wasip1_none.go(non-coop fallback)runtime/scheduler_idle_wasip2_none.go(non-coop fallback)internal/poll/fd_wasip1.go(FD type)internal/poll/fd_wasip2.go(WasipNFD over (TcpSocket, InputStream, OutputStream))syscall/syscall_libc_wasip1.gopark-on-EAGAINwould-blockdirectly intointernal/poll)What's added
src/runtime/netpoll_wasip2.go— pollable-keyedpollDescregistry,pollIO(timeoutNs)building one combinedwasi:io/poll.Pollcall (clock + active pollables), linkname-exposed wake helpers (runtime_netpoll_addpollable_wasip2/done/pdfired/wake). Three timing cases handled like wasip1'spollIO.src/runtime/scheduler_idle_wasip2.go+scheduler_idle_wasip2_none.go— cooperative-variantsleepTicks/waitForEventsthat route throughpollIOwhen pollables are registered; non-coop fallback usesmonotonicclock.Block. Matches the wasip1 file split.src/runtime/runtime_wasip2.go—sleepTicksmoved out (now per-config inscheduler_idle_wasip2*.go).src/runtime/wait_other.go— build tag tightened to exclude wasip2.src/internal/poll/fd_wasip2.go—WasipNFDwraps(TcpSocket, InputStream, OutputStream).DialTCPWasip2,ListenTCPWasip2,Accept,Read,Write,Close,SetDeadline*. Each blocking op tries the wasi call, onwould-blocksubscribes, parks viaruntime_netpoll_addpollable_wasip2 + task.Pause, on resume drops pollable + retries. Deadline-aware variants useparkUntil + time.AfterFunc + runtime_netpoll_wake_wasip2. Linkname-friendlyWasip2TCP{Listen,Dial,Accept,Read,Write,Close,SetDeadline}wrappers for test / future net callers.src/internal/poll/errors_wasip.go— error sentinels (ErrFileClosing,ErrNetClosing,ErrDeadlineExceeded,ErrNoDeadline) extracted fromfd_wasip1.goto awasip1 || wasip2shared file.src/os/poll_link_wasip2.go— pullsinternal/pollinto the wasip2 build (no-op blank import with justifying comment for the lint check).loader/goroot.go—listGorootMergeLinksnow filters TinyGo files by//go:buildconstraints (viago/build.Context.MatchFile) before deciding "TinyGo owns this directory". Files that don't match the current target no longer cause upstream Go files at the same level to be dropped. Unblocks per-target overrides for follow-up work without disturbing wasip1.Non-goals (deferred)
net.Listen("tcp", ...)/net.Dial("tcp", ...)on wasip2 via the standardnetpackage. Upstream Go'snetdoesn't currently build for wasip2 because itscgo_linux.goreaches fornetdb.heven though TinyGo doesn't enable cgo. Bringing up upstream net on wasip2 (either by filtering itscgofiles from the merge or providing a TinyGo-native net override) is its own piece of work — the linkname-friendly TCP helpers in this PR are the foundation. The synthetic test program calls them directly.PacketConnand DNS resolution. Same shape will work (wasi:sockets/udp,wasi:sockets/ip-name-lookup) but not in scope here.Wasip1 regression sweep:
tcpecho.wasmfrom #5386 still passes (echo hi | nc 127.0.0.1 9998round-trip + concurrent clients);time.Sleep/parkfile/parksynthunchanged.