USB Compliance Verification Test Suite (USBCV)

[hathach]

What is this issue for

The test suite include many corner cases which require both update of usbd and dcd to pass. This issue is used to keep track of current status of compliance test of tinyusb device stack on each DCD ports. Once it is all passed, we could close this.

What is USB Compliance Verification (USBCV)

USBCV is test suite tool used by usb-if to verify all usb hardware/software/controller which is required to passed in order to have USB logo on the product. https://www.usb.org/compliancetools . It is a good stress test with unusual case such as switching configuration 0 -> 1 -> 0, suspend/resume/remote-wakeup. Enumeration 150 times in a row as well as weird case like MSC READ10 with direction bit set to output etc ...

How to run USBCV

usbcv tool only runs on windows , depending on usb controller type on your PC you will need to either download the USB3CV (xHCI) or USB2CV (EHCI) from https://www.usb.org/compliancetools . Note: even your mcu is only full speed, but testing on xHCI host, you would need to run USB3CV.

Which tests suite to run

Within the scope of tinyusb device stack, we only need to run follow test suite

• Chapter 9 Test [USB 2 devices]
• HID Tests
• MSC Tests

Compliance Status

Example/Application can have large impact on the test result, the follow result is tested mostly with cdc_msc and hid_composite

MCU \ Test Suite	Chapter9	HID	MSC
nRF52840	✔️	✔️	✔️
SAMD 21/51	✔️	✔️	✔️
RP2040	✔️	✔️	✔️
STM32F4	✔️	✔️	✔️
Renesas RX	✔️	✔️	✔️
iMXRT	✔️	✔️	✔️
ESP32-S2/3	✔️	✔️	✔️
Kinetic KL25/32	✔️	✔️	✔️
DA146x	✔️	✔️	✔️
NUC 12x	✔️	❓	❓

[HiFiPhile]

I'm trying to test my project based on STM32L071.

SetConfiguration passed even without #1058

TD 9.13 - SetConfiguration Test (Configuration Index 0)Passed
INFO
Start time: Aug 30, 2021 - 14:23:20
INFO
SetConfiguration with configuration value : 0x1
INFO
Unconfigured the device
INFO
SetConfiguration with configuration value : 0x1
INFO

Stop time: Aug 30, 2021 - 14:23:21
INFO
Duration:  1 second.
INFO
Stopping Test [ TD 9.13 - SetConfiguration Test (Configuration Index 0):
     Number of: Fails (0); Aborts (0); Warnings (0) ]

TD 9.30 Configuration Summary Descriptor Test (Configuration Index 0x00) - Device State ConfiguredPassed
INFO
Start time: Aug 30, 2021 - 14:23:21
INFO
Configuration Index 0x00 has a Configuration Value of 1.
INFO
Initialized the device to the Configured state with Configuration Value 1.
INFO
Retrieved Device Descriptor.
INFO
Device Under Test supports USB Version 0x2.0x10.
INFO
Retrieved BOS Descriptors.
INFO
No Configuration Summary Descriptors found.
INFO

Stop time: Aug 30, 2021 - 14:23:22
INFO
Duration:  1 second.
INFO
Stopping Test [ TD 9.30 Configuration Summary Descriptor Test (Configuration Index 0x00) - Device State Configured:
     Number of: Fails (0); Aborts (0); Warnings (0) ]

TD 9.14 - Suspend/Resume Test (Configuration Index 0)Passed
INFO
Start time: Aug 30, 2021 - 14:23:22
INFO
Suspended the parent port of the Device Under Test.
INFO
Resumed the parent port of the Device Under Test.
INFO
Got device descriptor of Device Under Test
INFO

Stop time: Aug 30, 2021 - 14:23:23
INFO
Duration:  1 second.
INFO
Stopping Test [ TD 9.14 - Suspend/Resume Test (Configuration Index 0):
     Number of: Fails (0); Aborts (0); Warnings (0) ]

Then it fails at L1Suspend/Resume Test (Configuration Index 0)

INFO
Start time: Aug 30, 2021 - 14:23:25
INFO
Checking Device Under Test for LPM L1 Compatibility...
INFO
USB version of device is 2.10.
INFO
DUT IS compatible with LPM.
INFO
LPM IS required for DUT
FAIL
(USB: 9.6.2.1.9) A USB device with a USB version greater than or equal to 0x210 must include a USB 2.0 Extension descriptor.
INFO

Stop time: Aug 30, 2021 - 14:23:26
INFO
Duration:  1 second.
INFO
Stopping Test [ L1Suspend/Resume Test (Configuration Index 0):
     Number of: Fails (1); Aborts (0); Warnings (0) ]

I set bcdUSB to 0x0210 to be driver-free like WebUSB. However I looked USB 2.0 and 3.2 specifications but can't find this 9.6.2.1.9 chapter. Do you have idea where it is ?

---

TUD_HID_REPORT_DESC_GENERIC_INOUT fails HID Class Report Descriptor Test :

Report descriptor length: 32
Item count in report descriptor: 16
External report descriptor parser encountered 2 errors

(HID: 3.2.61) The report descriptor returned in response to a GetDescriptor(Report) must be compliant with the HID specification.
Byte Number:   11h (  17d)
  Data Field: 81 02
  Mnemonic:  Input
  Value: (Variable)
  Errors: Error:   Logical Minimum MUST be less than Logical Maximum

Byte Number:   1Dh (  29d)
  Data Field: 91 02
  Mnemonic:  Output
  Value: (Variable)
  Errors: Error:   Logical Minimum MUST be less than Logical Maximum

[hathach]

@HiFiPhile USB 2.0 extension section 9.6.2.1 is part of USB 3.2 revision 1.0 specs

SetConfiguration passed even without #1058

it passes but fake switching configuration to 0, and your device didn't implement the multiple configuration. Along the MSC test, it does assume switching configuration from 1 to 0 (unconfig) then back to 1 will reset all the driver state including stalled endpoint etc ... which could cause an issue. dcd_edpt_close_all() is added to resolve this correctly.

[HiFiPhile]

NUC126 passed chapter 9 tests with custom vendor class.
Chapter 9 Tests - USB 2 - Passed - 2021-09-09 20-35-38.zip

[hathach]

NUC126 passed chapter 9 tests with custom vendor class.
Chapter 9 Tests - USB 2 - Passed - 2021-09-09 20-35-38.zip

Thank for the testing, though, did you test with the hid composite, one of the chapter9 is remote wakeup which is only tested with hid example due to TUSB_DESC_CONFIG_ATT_REMOTE_WAKEUP

[HiFiPhile]

Thank for the testing, though, did you test with the hid composite, one of the chapter9 is remote wakeup which is only tested with hid example due to TUSB_DESC_CONFIG_ATT_REMOTE_WAKEUP

It didn't work.
There is a typo of |= but still something more.

tinyusb/src/portable/nuvoton/nuc121/dcd_nuc121.c

Line 258 in d66d817

USBD->ATTR = USBD_ATTR_RWAKEUP_Msk;

HID test passed.
USB3CV.zip

---

Something is wrong with suspend detection.
_usbd_dev.remote_wakeup_en is 1 but _usbd_dev.suspended is 0

USBD Setup Received 00 03 01 00 00 00 00 00
  Set Feature
    Enable Remote Wakeup
  Queue EP 80 with 0 bytes ...
USBD Xfer Complete on EP 80 with 0 bytes

USBD Setup Received 80 00 00 00 00 00 02 00
  Get Status
  Queue EP 80 with 2 bytes ...
USBD Xfer Complete on EP 80 with 2 bytes
  Queue EP 00 with 0 bytes ...
USBD Xfer Complete on EP 00 with 0 bytes

USBD Setup Received 80 00 00 00 00 00 02 00
  Get Status
  Queue EP 80 with 2 bytes ...
USBD Xfer Complete on EP 80 with 2 bytes
  Queue EP 00 with 0 bytes ...
USBD Xfer Complete on EP 00 with 0 bytes
USBD Suspend : Remote Wakeup = 1
USBD Resume
USBD Resume