Skip to content

usb/cdc: fix RP2 USB CDC TX race with cores scheduler#5391

Open
rdon-key wants to merge 3 commits into
tinygo-org:devfrom
rdon-key:fix-rp2-usb-cdc-tx-race
Open

usb/cdc: fix RP2 USB CDC TX race with cores scheduler#5391
rdon-key wants to merge 3 commits into
tinygo-org:devfrom
rdon-key:fix-rp2-usb-cdc-tx-race

Conversation

@rdon-key
Copy link
Copy Markdown
Contributor

Fixes #5377.

This PR fixes a USB CDC TX race on RP2 targets when using -scheduler=cores.

The issue was reproduced on Raspberry Pi Pico / RP2040 with TinyGo 0.41.1.
RP2350 may be affected as well because it shares the RP2 USB backend and the
same USB CDC TX state machine.

Cause

USB CDC TX is driven from two places:

  • Write() via kickTx()
  • the USB TX completion handler via txhandler()

The previous code used inflight both as:

  • the number of bytes currently submitted to the USB IN endpoint
  • an implicit guard for whether the TX pump is active

That is not sufficient on multicore targets.

With -scheduler=cores, Write() may run on one core while the USB TX
completion handler runs on another core. In that case, kickTx() can observe the
TX path as idle while txhandler() is still processing a completed packet and
chaining the next one.

That can allow concurrent or re-entrant calls into sendFromRing() /
machine.SendUSBInPacket() for the same USB IN endpoint, which can corrupt USB
CDC output.

Fix

This PR adds a separate atomic txActive flag.

txActive represents ownership of the USB CDC TX pump. inflight remains only
the number of bytes currently submitted to the endpoint.

kickTx() now starts the TX pump only if it can acquire txActive with CAS.
The TX completion handler keeps TX pump ownership while chaining packets, and
ownership is released only when the TX ring is empty.

A final ring re-check is used when releasing ownership to avoid a missed wakeup
if Write() appends data while txActive is still set.

Manual test

Test program:

package main

import "time"

func main() {
	time.Sleep(2 * time.Second)

	println("start usb cdc tx stress")

	for i := 0; i < 1000; i++ {
		println("usb cdc tx test:", i, "abcdefghijklmnopqrstuvwxyz0123456789")
		time.Sleep(1 * time.Millisecond)
	}

	println("test finished")
}

Commands:

~/work/tinygo/build/tinygo flash -target=pico -scheduler=cores -monitor usbcdc_tx_stress.go
~/work/tinygo/build/tinygo flash -target=pico -scheduler=tasks -monitor usbcdc_tx_stress.go

Results:

Before this fix:
- TinyGo 0.41.1 + -scheduler=cores: USB CDC output was corrupted
- TinyGo 0.41.1 + -scheduler=tasks: OK

After this fix:
- local dev build + -scheduler=cores: OK
- local dev build + -scheduler=tasks: OK

The -scheduler=cores test was repeated multiple times without observing
dropped or corrupted USB CDC output.

@rdon-key rdon-key force-pushed the fix-rp2-usb-cdc-tx-race branch 2 times, most recently from ce5c04a to c059707 Compare May 11, 2026 14:29
Comment thread src/machine/usb/cdc/usbcdc.go Outdated
@deadprogram
Copy link
Copy Markdown
Member

@rdon-key thank you for looking into this. Your solution makes sense to me, I just suggested a small correction in form.

@rdon-key
Copy link
Copy Markdown
Contributor Author

@deadprogram
Thank you for the review! I applied the suggested change.

@deadprogram
Copy link
Copy Markdown
Member

Anyone else have any feedback or comments before merge?

@deadprogram
Copy link
Copy Markdown
Member

@soypat ?

@soypat soypat added the rp2 RP2350/RP2040 label May 28, 2026
Comment on lines +139 to +152
for {
d1, _ := usbcdc.tx.Peek()
if len(d1) == 0 {
usbcdc.txActive.Store(0)

// Avoid a missed wakeup: Write may append data while txActive is
// still set, causing kickTx to return without starting a transfer.
switch {
case usbcdc.tx.Used() == 0:
return
case !usbcdc.txActive.CompareAndSwap(0, 1):
return
}
continue
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This for loop is confusing to me. Could we make it more explicit?

d1, _ := usbcdc.tx.Peek()
for len(d1)==0 {
     // store...
    d1, _ = usbcdc.tx.Peek() 
}

Also, it seems weird to me that we are looping while len of data is zero. If length of data is zero does that not mean data is sent already and you can return?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking a look.

The outer for is not intended to loop while the ring is empty. It is the main TX pump loop: each iteration peeks the ring and, if data is available, sends one packet.
The len(d1) == 0 branch is only the idle/exit path. It handles the missed-wakeup case: after the peek sees an empty ring, Write may still append data while txActive is set, causing kickTx to return without starting a new transfer. So we clear txActive, then re-check whether data appeared. If it did, we try to re-acquire txActive with CAS and continue the main TX loop so it can re-peek and send normally.
Rewriting this as for len(d1) == 0 would make it look like the code is waiting for data while the ring is empty, which is not the intent.

I've pushed a comment above the loop to make the pump/idle-exit structure explicit — let me know if it's clearer now.

@rdon-key rdon-key force-pushed the fix-rp2-usb-cdc-tx-race branch from fb52e10 to c690c2c Compare May 29, 2026 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rp2 RP2350/RP2040

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RP2040: USB CDC output drops characters with -scheduler=cores on TinyGo 0.41.1

3 participants