Skip to content

runtime,internal/poll,loader: wasip2 pollable poll-integration + TCP I/O#5390

Open
achille-roussel wants to merge 1 commit into
tinygo-org:devfrom
achille-roussel:wasip2-net
Open

runtime,internal/poll,loader: wasip2 pollable poll-integration + TCP I/O#5390
achille-roussel wants to merge 1 commit into
tinygo-org:devfrom
achille-roussel:wasip2-net

Conversation

@achille-roussel
Copy link
Copy Markdown
Contributor

Stacked on #5386. Mirrors the wasip1 work for wasip2. The cooperative scheduler's idle path now calls wasi:io/poll.Poll over a combined list of (clock pollable, registered pollables) instead of blocking the wasm module on a single monotonic-clock subscription, so goroutines doing TCP I/O can park while the scheduler runs other goroutines.

End to end:

$ tinygo build -target=wasip2 -o tcpecho_wasip2.wasm ./tcpecho_wasip2.go
$ wasmtime run -Sinherit-network -Stcp ./tcpecho_wasip2.wasm &
listening on 127.0.0.1:9999
tick 1
tick 2
tick 3

$ echo hello | nc 127.0.0.1 9999
hello                                    # echoed by the wasm

Two concurrent nc clients both echo while the ticker keeps progressing — the cooperative scheduler keeps running goroutines parked on sock_recv via the new pollable registry.

How it works

Wasip2's polling primitive is wasi:io/poll.Poll(list<own<pollable>>) -> list<u32> taking a list of pollable resource handles. Each blocking wasi operation has a subscribe() that yields a pollable.

TCPConn.Read / Write   ─would-block─►  internal/poll registry  ─►  task.Pause()
                                                │
                                                ▼
                       scheduler idle  ──►  pollIO(timeoutNs)
                                                │
                                                ├─ build pollables: [clock?, p1, p2, …]
                                                ├─ wasi:io/poll.Poll(...)
                                                └─ wake matched tasks → run queue

The wasip2 path mirrors the wasip1 PR's structure file-for-file:

wasip1 wasip2
runtime/netpoll_wasip1.go (FD-keyed poll_oneoff registry) runtime/netpoll_wasip2.go (pollable-keyed wasi:io/poll.Poll registry)
runtime/scheduler_idle_wasip1.go (cooperative sleepTicks routes via pollIO) runtime/scheduler_idle_wasip2.go (cooperative sleepTicks routes via pollIO)
runtime/scheduler_idle_wasip1_none.go (non-coop fallback) runtime/scheduler_idle_wasip2_none.go (non-coop fallback)
internal/poll/fd_wasip1.go (FD type) internal/poll/fd_wasip2.go (WasipNFD over (TcpSocket, InputStream, OutputStream))
syscall/syscall_libc_wasip1.go park-on-EAGAIN (not needed — wasip2 wasi calls return would-block directly into internal/poll)

What's added

  • src/runtime/netpoll_wasip2.go — pollable-keyed pollDesc registry, pollIO(timeoutNs) building one combined wasi:io/poll.Poll call (clock + active pollables), linkname-exposed wake helpers (runtime_netpoll_addpollable_wasip2 / done / pdfired / wake). Three timing cases handled like wasip1's pollIO.
  • src/runtime/scheduler_idle_wasip2.go + scheduler_idle_wasip2_none.go — cooperative-variant sleepTicks / waitForEvents that route through pollIO when pollables are registered; non-coop fallback uses monotonicclock.Block. Matches the wasip1 file split.
  • src/runtime/runtime_wasip2.gosleepTicks moved out (now per-config in scheduler_idle_wasip2*.go).
  • src/runtime/wait_other.go — build tag tightened to exclude wasip2.
  • src/internal/poll/fd_wasip2.goWasipNFD wraps (TcpSocket, InputStream, OutputStream). DialTCPWasip2, ListenTCPWasip2, Accept, Read, Write, Close, SetDeadline*. Each blocking op tries the wasi call, on would-block subscribes, parks via runtime_netpoll_addpollable_wasip2 + task.Pause, on resume drops pollable + retries. Deadline-aware variants use parkUntil + time.AfterFunc + runtime_netpoll_wake_wasip2. Linkname-friendly Wasip2TCP{Listen,Dial,Accept,Read,Write,Close,SetDeadline} wrappers for test / future net callers.
  • src/internal/poll/errors_wasip.go — error sentinels (ErrFileClosing, ErrNetClosing, ErrDeadlineExceeded, ErrNoDeadline) extracted from fd_wasip1.go to a wasip1 || wasip2 shared file.
  • src/os/poll_link_wasip2.go — pulls internal/poll into the wasip2 build (no-op blank import with justifying comment for the lint check).
  • loader/goroot.golistGorootMergeLinks now filters TinyGo files by //go:build constraints (via go/build.Context.MatchFile) before deciding "TinyGo owns this directory". Files that don't match the current target no longer cause upstream Go files at the same level to be dropped. Unblocks per-target overrides for follow-up work without disturbing wasip1.

Non-goals (deferred)

  • net.Listen("tcp", ...) / net.Dial("tcp", ...) on wasip2 via the standard net package. Upstream Go's net doesn't currently build for wasip2 because its cgo_linux.go reaches for netdb.h even though TinyGo doesn't enable cgo. Bringing up upstream net on wasip2 (either by filtering its cgo files from the merge or providing a TinyGo-native net override) is its own piece of work — the linkname-friendly TCP helpers in this PR are the foundation. The synthetic test program calls them directly.
  • UDP / PacketConn and DNS resolution. Same shape will work (wasi:sockets/udp, wasi:sockets/ip-name-lookup) but not in scope here.

Wasip1 regression sweep: tcpecho.wasm from #5386 still passes (echo hi | nc 127.0.0.1 9998 round-trip + concurrent clients); time.Sleep / parkfile / parksynth unchanged.

@deadprogram
Copy link
Copy Markdown
Member

@achille-roussel can you please rebase this PR against the latest dev.

Also, can you please remove Claude as your co-author?

Thank you!

Mirrors PR tinygo-org#5386's wasip1 work for wasip2. The cooperative scheduler's
idle path now calls wasi:io/poll.Poll over a combined list of (clock
pollable, registered pollables) instead of blocking the wasm module on
a single monotonic-clock subscription, so goroutines doing TCP I/O can
park while the scheduler runs other goroutines.

Plumbing components:

- runtime/netpoll_wasip2.go: pollable-keyed pollDesc registry; pollIO
  builds one combined wasi:io/poll.Poll call (clock pollable + active
  pollables). Linkname-exposed runtime_netpoll_addpollable_wasip2 /
  done / pdfired / wake for internal/poll and future net.
- runtime/scheduler_idle_wasip2.go + scheduler_idle_wasip2_none.go:
  cooperative-variant sleepTicks / waitForEvents that route through
  pollIO; non-coop fallback uses monotonicclock.Block. Mirrors the
  wasip1 structure introduced in 7000e7b.
- runtime/runtime_wasip2.go: sleepTicks moved out to the
  scheduler_idle_wasip2*.go files.
- runtime/wait_other.go: build tag tightened to exclude wasip2.

internal/poll surface:

- internal/poll/fd_wasip2.go: WasipNFD wraps a (TcpSocket, InputStream,
  OutputStream) triple. DialTCPWasip2, ListenTCPWasip2, Accept, Read,
  Write, Close, SetDeadline*. Each blocking op tries the wasi call,
  on would-block subscribes, parks, retries — same pattern as the
  wasip1 internal/poll.FD but pollable-keyed. Linkname-friendly
  Wasip2TCP{Listen,Dial,Accept,Read,Write,Close,SetDeadline} wrappers
  for test / future net callers.
- internal/poll/errors_wasip.go: ErrFileClosing / ErrNetClosing /
  ErrDeadlineExceeded / ErrNoDeadline extracted from fd_wasip1.go to
  a wasip1||wasip2 shared file.

Loader change:

- loader/goroot.go: listGorootMergeLinks now filters TinyGo files by
  //go:build constraints (via go/build.Context.MatchFile) before
  deciding "TinyGo owns this directory". Files that don't match the
  current target no longer cause upstream Go files at the same level
  to be dropped. Unblocks per-target overrides in directories like
  src/net/ for future net.wasip2 work without disturbing wasip1.

End-to-end verification:

  $ wasmtime run -Sinherit-network -Stcp ./tcpecho_wasip2.wasm &
  listening on 127.0.0.1:9999
  tick 1
  tick 2
  tick 3
  $ echo hello | nc 127.0.0.1 9999
  hello                                    # echoed by the wasm
  $ # two concurrent clients echo cleanly while ticker keeps ticking

The test program (not shipped) uses //go:linkname to drive the
internal/poll TCP helpers directly, since TinyGo doesn't yet have a
net.Listen / net.Dial path on wasip2 (upstream Go's net doesn't build
for wasip2 due to cgo_linux.go reaching for Linux headers). The
src/net/ wasip2 wrappers are out of scope for this PR and tracked as
follow-up — once they land, callers will use net.Listen / Dial
directly and the linkname wrappers can drop.

Wasip1 regression sweep: tcpecho.wasm still passes; time.Sleep / parkfile
/ parksynth unchanged.
@achille-roussel
Copy link
Copy Markdown
Contributor Author

Done.

I'm exploring how to add a few tests to verify the new behavior as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants