Add esptool --port-filter SN filter. (ESPTOOL-950) #1027

cederom · 2024-10-27T12:53:47Z

Is your feature request related to a problem?

If there are several boards attached to a single build machine of the same VID:PID and Name it is impossible to distinguish them when no SN can be provided to port filter.
The perfect unique identifier of a board is its USB SN string.

Describe the solution you'd like

Please add Serial Number (SN) field to --port-filter functionality.

Describe alternatives you've considered

Complex scripting must be done so far to know what exact device is at each /dev/cuaU* device.
Some develkits (i.e. ESP32-DevKitC with CP2102 chip) can have Name and SN fields customized that helps a lot with identification, but still there is no SN filter yet in esptool.

Additional context

#1026

The text was updated successfully, but these errors were encountered:

Dzarda7 · 2024-10-31T10:24:41Z

Hello @cederom, thanks for the suggestion, this sounds reasonable. I will take a look at it and try to implement the feature as soon as possible.

Not sure if you are aware of it, but on linux machines you can make alias for each USB port which will stay the same. It is not the solution how to distinguish each esp chip, but as long as you connect chip into the same port, this works.

Nevertheless I will take a look at it.

cederom · 2024-10-31T18:05:13Z

Thank you @Dzarda7 :-)

I am working on FreeBSD :-) Sure various OS have options to manage USB devices on attach (i.e. devd/devfs). But having option straight in the esptool is the most universal solution and requires no system modification (i.e. this can be dynamically created VM). You can use underlying pyserial functionalities :-)

cederom · 2024-11-03T19:12:33Z

BIG THANK YOU @Dzarda7 !! :-)

cederom · 2024-11-03T19:13:20Z

Do you know when it will land in a release and the time of release? We will update the system package :-)

Dzarda7 · 2024-11-03T19:43:35Z

Will let you know tomorrow.

Dzarda7 · 2024-11-04T10:18:02Z

We are not sure when the next release will happen, but we can make dev release if this is important for you, just let me know.

Dzarda7 · 2024-11-04T12:56:46Z

Esptool v4.9.dev1 will be released tomorrow, but won't be part of the esp-idf yet, if you are interested, you can update the version in esp-idf by yourself as stated in the documentation.

cederom · 2024-11-04T22:08:55Z

Thank you @Dzarda7 this is perfectly fine we only need esptool package on FreeBSD :-) BIG THANK YOU!! :-)

Dzarda7 · 2024-11-05T13:35:41Z

Esptool v4.9.dev1 was released.

cederom · 2024-11-05T15:01:22Z

@Dzarda7: Esptool v4.9.dev1 was released.

Thanks.. but why not 4.8.2 / 4.9.0 on GitHub? We use GH releases for packages. Or is it only test preview? Will test and report back in a moment :-)

Dzarda7 · 2024-11-05T15:29:07Z

We do not have enough features for the release and we do not do dev releases on github. If you are looking for the built binaries, you can find them for example here.

cederom · 2024-11-05T15:41:59Z

Okay, good idea to test before a release, that port filter does not work for me, and I need to build most of the pip packages as binary packages are usally not provided for FreeBSD :-)

(venv3.9embedded) uname -a
FreeBSD octagon 13.3-RELEASE-p7 FreeBSD 13.3-RELEASE-p7 GENERIC amd64

I have updated esptool to version 4.9.dev1.

(venv3.9embedded) esptool.py version
esptool.py v4.9.dev1
4.9.dev1

I have connected ESP32-DevKitC only to USB having it at /dev/cuaU0.

(venv3.9embedded) esptool.py chip_id
esptool.py v4.9.dev1
Serial port /dev/cuaU0
Connecting...............
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting.........
Detecting chip type... ESP32
Chip is ESP32-D0WD-V3 (revision v3.0)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 8c:4b:14:81:0b:ec
Uploading stub...
Running stub...
Stub running...
Warning: ESP32 has no Chip ID. Reading MAC instead.
MAC: 8c:4b:14:81:0b:ec
Hard resetting via RTS pin...

Checked the serial string (custom / modified ):

(venv3.9embedded) usbconfig -d 0.6 dump_device_desc
ugen0.6: <Silicon Labs ESP32DEVKITC-CeDeROM-76f24aad88225fedf7260bb4f54c525d> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)

  bLength = 0x0012
  bDescriptorType = 0x0001
  bcdUSB = 0x0110
  bDeviceClass = 0x0000  <Probed by interface class>
  bDeviceSubClass = 0x0000
  bDeviceProtocol = 0x0000
  bMaxPacketSize0 = 0x0040
  idVendor = 0x10c4
  idProduct = 0xea60
  bcdDevice = 0x0100
  iManufacturer = 0x0001  <Silicon Labs>
  iProduct = 0x0002  <ESP32DEVKITC-CeDeROM-76f24aad88225fedf7260bb4f54c525d>
  iSerialNumber = 0x0003  <ESP32DEVKITC-CeDeROM-76f24aad88225fedf7260bb4f54c525d>
  bNumConfigurations = 0x0001

Then tested the serial port filter with only one device:

(venv3.9embedded) esptool.py --port-filter serial="ESP32DEVKITC-CeDeROM-76f24aad88225fedf7260bb4f54c525d" chip_id
esptool.py v4.9.dev1
Serial port /dev/cuaU0
Connecting...........
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting....
Detecting chip type... ESP32
Chip is ESP32-D0WD-V3 (revision v3.0)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 8c:4b:14:81:0b:ec
Uploading stub...
Running stub...
Stub running...
Warning: ESP32 has no Chip ID. Reading MAC instead.
MAC: 8c:4b:14:81:0b:ec
Hard resetting via RTS pin...

Then detached ESP32-DevkitC, attached ESP32C3-Devkit first (at /dev/cuaU0), then attached ESP32-DevKitC (at /dev/cuaU1). Providing filter for ESP32 at /dev/cuaU1 esptool connected to ESP32C3 at /dev/cuaU0.

(venv3.9embedded) ls /dev/cuaU*
/dev/cuaU0      /dev/cuaU0.init /dev/cuaU0.lock /dev/cuaU1      /dev/cuaU1.init /dev/cuaU1.lock

(venv3.9embedded) esptool.py --port-filter serial="ESP32DEVKITC-CeDeROM-76f24aad88225fedf7260bb4f54c525d" chip_id
esptool.py v4.9.dev1
Serial port /dev/cuaU0
Connecting...
Detecting chip type... ESP32-C3
Chip is ESP32-C3 (QFN32) (revision v0.3)
Features: WiFi, BLE, Embedded Flash 4MB (XMC)
Crystal is 40MHz
MAC: 60:55:f9:cc:e2:b8
Uploading stub...
Running stub...
Stub running...
Warning: ESP32-C3 has no Chip ID. Reading MAC instead.
MAC: 60:55:f9:cc:e2:b8
Hard resetting with RTC WDT...

Here is the usb device descriptor for ESP32C3 and we can see it has kinda funny serial:

(venv3.9embedded) usbconfig -d 0.6 dump_device_desc
ugen0.6: <Espressif USB JTAG/serial debug unit> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (500mA)

  bLength = 0x0012
  bDescriptorType = 0x0001
  bcdUSB = 0x0200
  bDeviceClass = 0x00ef  <Miscellaneous device>
  bDeviceSubClass = 0x0002
  bDeviceProtocol = 0x0001
  bMaxPacketSize0 = 0x0040
  idVendor = 0x303a
  idProduct = 0x1001
  bcdDevice = 0x0101
  iManufacturer = 0x0001  <Espressif>
  iProduct = 0x0002  <USB JTAG/serial debug unit>
  iSerialNumber = 0x0003  <60:55:F9:CC:E2:B8>
  bNumConfigurations = 0x0001

I have disconnected both devkits, connected ESP32 first (/dev/cuaU0), then ESP32C3 (/dev/cuaU1), and the esptool still connects to /dev/cuaU0

(venv3.9embedded) esptool.py --port-filter serial='60:55:F9:CC:E2:B8' chip_id
esptool.py v4.9.dev1
Serial port /dev/cuaU0
Connecting.........
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting.....
Detecting chip type... ESP32
Chip is ESP32-D0WD-V3 (revision v3.0)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 8c:4b:14:81:0b:ec
Uploading stub...
Running stub...
Stub running...
Warning: ESP32 has no Chip ID. Reading MAC instead.
MAC: 8c:4b:14:81:0b:ec
Hard resetting via RTS pin...

Okay lets try the name port filter:

(venv3.9embedded) esptool.py --port-filter name='USB JTAG/serial debug unit' chip_id
esptool.py v4.9.dev1
Serial port /dev/cuaU0
Connecting............
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting....
Detecting chip type... ESP32
Chip is ESP32-D0WD-V3 (revision v3.0)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 8c:4b:14:81:0b:ec
Uploading stub...
Running stub...
Stub running...
Warning: ESP32 has no Chip ID. Reading MAC instead.
MAC: 8c:4b:14:81:0b:ec
Hard resetting via RTS pin...

The same trick after swap to ESP32 to /dev/cuaU1:

(venv3.9embedded) esptool.py --port-filter name=ESP32DEVKITC-CeDeROM-76f24aad88225fedf7260bb4f54c525d chip_id
esptool.py v4.9.dev1
Serial port /dev/cuaU0
Connecting...
Detecting chip type... ESP32-C3
Chip is ESP32-C3 (QFN32) (revision v0.3)
Features: WiFi, BLE, Embedded Flash 4MB (XMC)
Crystal is 40MHz
MAC: 60:55:f9:cc:e2:b8
Uploading stub...
Running stub...
Stub running...
Warning: ESP32-C3 has no Chip ID. Reading MAC instead.
MAC: 60:55:f9:cc:e2:b8
Hard resetting with RTC WDT...

(venv3.9embedded) esptool.py --port-filter name='ESP32DEVKITC-CeDeROM-76f24aad88225fedf7260bb4f54c525d' chip_id
esptool.py v4.9.dev1
Serial port /dev/cuaU0
Connecting...
Detecting chip type... ESP32-C3
Chip is ESP32-C3 (QFN32) (revision v0.3)
Features: WiFi, BLE, Embedded Flash 4MB (XMC)
Crystal is 40MHz
MAC: 60:55:f9:cc:e2:b8
Uploading stub...
Running stub...
Stub running...
Warning: ESP32-C3 has no Chip ID. Reading MAC instead.
MAC: 60:55:f9:cc:e2:b8
Hard resetting with RTC WDT...

(venv3.9embedded) esptool.py --port-filter name="ESP32DEVKITC-CeDeROM-76f24aad88225fedf7260bb4f54c525d" chip_id
esptool.py v4.9.dev1
Serial port /dev/cuaU0
Connecting...
Detecting chip type... ESP32-C3
Chip is ESP32-C3 (QFN32) (revision v0.3)
Features: WiFi, BLE, Embedded Flash 4MB (XMC)
Crystal is 40MHz
MAC: 60:55:f9:cc:e2:b8
Uploading stub...
Running stub...
Stub running...
Warning: ESP32-C3 has no Chip ID. Reading MAC instead.
MAC: 60:55:f9:cc:e2:b8
Hard resetting with RTC WDT...

Just to verify how the pyserial see the ports, maybe this is the problem? :-)

(venv3.9embedded) pip show pyserial
Name: pyserial
Version: 3.5
Summary: Python Serial Port Extension
Home-page: https://github.com/pyserial/pyserial
Author: Chris Liechti
Author-email: [email protected]
License: BSD
Location: /usr/home/cederom/.local/venv3.9embedded/lib/python3.9/site-packages
Requires:
Required-by: adafruit-ampy, bl60x-flash, esptool, mpremote, nrfutil, pyspinel, yoctools

(venv3.9embedded) pyserial-ports --verbose
/dev/cuaU0
    desc: n/a
    hwid: n/a
/dev/cuaU1
    desc: n/a
    hwid: n/a
/dev/cuau0
    desc: n/a
    hwid: n/a
3 ports found

cederom · 2024-11-05T15:49:22Z

If the port filter depends on pyserial I saw recenlty an update that extends the pyserial-ports verbose output. If you can provide your output or what is expected we will update pyserial so it works on FreeBSD too and provide necessary information for esptool port filter :-)

Dzarda7 · 2024-11-06T07:30:55Z

Thanks for noticing this and great report. I tried it on my system and it works great as you can see below:

(venv) ➜  esptool_test_freebsd pyserial-ports --verbose
/dev/ttyUSB0        
    desc: CP2102N USB to UART Bridge Controller
    hwid: USB VID:PID=10C4:EA60 SER=7c98d1065267ee11bcc4c8ab93cd958c LOCATION=3-6.1.7


(venv) ➜  esptool_test_freebsd esptool.py --port-filter serial=7c98d1065267ee11bcc4c8ab93cd958c flash_id
esptool.py v4.9.dev1
Found 1 serial ports
Serial port /dev/ttyUSB0
Connecting.....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting....
Detecting chip type... ESP32
Chip is ESP32-D0WD-V3 (revision v3.1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 10:06:1c:98:2e:e0
Uploading stub...
Running stub...
Stub running...
Manufacturer: c8
Device: 4017
Detected flash size: 8MB
Flash voltage set by a strapping pin to 3.3V
Hard resetting via RTS pin...


(venv) ➜  esptool_test_freebsd esptool.py --port-filter serial=7c98d1065267ee11bcc4c8ab93cd958d flash_id
esptool.py v4.9.dev1
Found 0 serial ports

A fatal error occurred: Could not connect to an Espressif device on any of the 0 available serial ports.

I also tried it in docker with your version of python (3.9) using docker run --privileged --rm -it --device=/dev/ttyUSB0 python:3.9 bash which also works as expected:

root@e8ff0c799bc6:/# pyserial-ports --verbose
/dev/ttyUSB0        
    desc: CP2102N USB to UART Bridge Controller
    hwid: USB VID:PID=10C4:EA60 SER=7c98d1065267ee11bcc4c8ab93cd958c LOCATION=3-6.1.7


root@e8ff0c799bc6:/# esptool.py --port-filter serial=7c98d1065267ee11bcc4c8ab93cd958c flash_id
esptool.py v4.9.dev1
Found 1 serial ports
Serial port /dev/ttyUSB0
Connecting.....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting....
Detecting chip type... ESP32
Chip is ESP32-D0WD-V3 (revision v3.1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 10:06:1c:98:2e:e0
Uploading stub...
Running stub...
Stub running...
Manufacturer: c8
Device: 4017
Detected flash size: 8MB
Flash voltage set by a strapping pin to 3.3V
Hard resetting via RTS pin...


root@e8ff0c799bc6:/# esptool.py --port-filter serial=7c98d1065267ee11bcc4c8ab93cd958d flash_id
esptool.py v4.9.dev1
Found 0 serial ports

A fatal error occurred: Could not connect to an Espressif device on any of the 0 available serial ports.

Yes, port filter depends on pyserial. As you already noticed, problem seems to be the pyserial that does not see port information. I am not really sure, what you mean by updating pyserial as version 3.5 is required by esptool:

esptool_test_freebsd pip show pyserial            
Name: pyserial
Version: 3.5
Summary: Python Serial Port Extension
Home-page: https://github.com/pyserial/pyserial
Author: Chris Liechti
Author-email: [email protected]
License: BSD
Location: /home/jarda/Documents/Espressif/esptool_test_freebsd/venv/lib64/python3.13/site-packages
Requires: 
Required-by: esptool

cederom · 2024-11-06T09:55:25Z

Thank you @Dzarda7 :-)

Can you please double check on your Linux with two different devkits attached in different order and use port filter to one connected to /dev/ttyUSB1?

If that works for you on Linux that means we need to patch pyserial to work properly on FreeBSD :-)

Dzarda7 · 2024-11-06T10:30:00Z

ESP32H2 - /dev/ttyUSB0, ESP32 - /dev/ttyUSB1:

(venv) ➜  esptool_test_freebsd pyserial-ports --verbose
/dev/ttyUSB0        
    desc: CP2102N USB to UART Bridge Controller
    hwid: USB VID:PID=10C4:EA60 SER=14fefc9fe665ee119c09926293cd958c LOCATION=3-6.1.5
/dev/ttyUSB1        
    desc: CP2102N USB to UART Bridge Controller
    hwid: USB VID:PID=10C4:EA60 SER=7c98d1065267ee11bcc4c8ab93cd958c LOCATION=3-6.1.7


(venv) ➜  esptool_test_freebsd esptool.py --port-filter serial=14fefc9fe665ee119c09926293cd958c flash_id
esptool.py v4.9.dev1
Found 1 serial ports
Serial port /dev/ttyUSB0
Connecting.....
Detecting chip type... ESP32-H2
Chip is ESP32-H2 (revision v0.1)
Features: BLE, IEEE802.15.4
Crystal is 32MHz
MAC: 74:4d:bd:ff:fe:63:37:06
BASE MAC: 74:4d:bd:63:37:06
MAC_EXT: ff:fe
Uploading stub...
Running stub...
Stub running...
Manufacturer: c8
Device: 4016
Detected flash size: 4MB
Hard resetting via RTS pin...


(venv) ➜  esptool_test_freebsd esptool.py --port-filter serial=7c98d1065267ee11bcc4c8ab93cd958c flash_id
esptool.py v4.9.dev1
Found 1 serial ports
Serial port /dev/ttyUSB1
Connecting.......
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting.....
Detecting chip type... ESP32
Chip is ESP32-D0WD-V3 (revision v3.1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 10:06:1c:98:2e:e0
Uploading stub...
Running stub...
Stub running...
Manufacturer: c8
Device: 4017
Detected flash size: 8MB
Flash voltage set by a strapping pin to 3.3V
Hard resetting via RTS pin...

ESP32H2 - /dev/ttyUSB0, ESP32 - /dev/ttyUSB1:

(venv) ➜  esptool_test_freebsd pyserial-ports --verbose
/dev/ttyUSB0        
    desc: CP2102N USB to UART Bridge Controller
    hwid: USB VID:PID=10C4:EA60 SER=7c98d1065267ee11bcc4c8ab93cd958c LOCATION=3-6.1.7
/dev/ttyUSB1        
    desc: CP2102N USB to UART Bridge Controller
    hwid: USB VID:PID=10C4:EA60 SER=14fefc9fe665ee119c09926293cd958c LOCATION=3-6.1.5


(venv) ➜  esptool_test_freebsd esptool.py --port-filter serial=14fefc9fe665ee119c09926293cd958c flash_id
esptool.py v4.9.dev1
Found 1 serial ports
Serial port /dev/ttyUSB1
Connecting....
Detecting chip type... ESP32-H2
Chip is ESP32-H2 (revision v0.1)
Features: BLE, IEEE802.15.4
Crystal is 32MHz
MAC: 74:4d:bd:ff:fe:63:37:06
BASE MAC: 74:4d:bd:63:37:06
MAC_EXT: ff:fe
Uploading stub...
Running stub...
Stub running...
Manufacturer: c8
Device: 4016
Detected flash size: 4MB
Hard resetting via RTS pin...


(venv) ➜  esptool_test_freebsd esptool.py --port-filter serial=7c98d1065267ee11bcc4c8ab93cd958c flash_id
esptool.py v4.9.dev1
Found 1 serial ports
Serial port /dev/ttyUSB0
Connecting....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting......
Detecting chip type... ESP32
Chip is ESP32-D0WD-V3 (revision v3.1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 10:06:1c:98:2e:e0
Uploading stub...
Running stub...
Stub running...
Manufacturer: c8
Device: 4017
Detected flash size: 8MB
Flash voltage set by a strapping pin to 3.3V
Hard resetting via RTS pin...

Hope this helps, you can see that it works for me without the problem. Please beware that ends of both serial numbers are the same.

cederom · 2024-11-06T10:58:11Z

Thank you @Dzarda7 looks like a problem on our end will try to align pyserial on FreeBSD side :-)

cederom added the Type: Feature Request label Oct 27, 2024

github-actions bot changed the title ~~Add esptool --port-filter SN filter.~~ Add esptool --port-filter SN filter. (ESPTOOL-950) Oct 27, 2024

espressif-bot closed this as completed in 88319db Nov 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add esptool --port-filter SN filter. (ESPTOOL-950) #1027

Add esptool --port-filter SN filter. (ESPTOOL-950) #1027

cederom commented Oct 27, 2024

Dzarda7 commented Oct 31, 2024

cederom commented Oct 31, 2024

cederom commented Nov 3, 2024

cederom commented Nov 3, 2024

Dzarda7 commented Nov 3, 2024

Dzarda7 commented Nov 4, 2024

Dzarda7 commented Nov 4, 2024 •

edited by radimkarnis

Loading

cederom commented Nov 4, 2024

Dzarda7 commented Nov 5, 2024 •

edited by radimkarnis

Loading

cederom commented Nov 5, 2024

Dzarda7 commented Nov 5, 2024

cederom commented Nov 5, 2024 •

edited

Loading

cederom commented Nov 5, 2024

Dzarda7 commented Nov 6, 2024

cederom commented Nov 6, 2024 •

edited

Loading

Dzarda7 commented Nov 6, 2024

cederom commented Nov 6, 2024

Add esptool --port-filter SN filter. (ESPTOOL-950) #1027

Add esptool --port-filter SN filter. (ESPTOOL-950) #1027

Comments

cederom commented Oct 27, 2024

Is your feature request related to a problem?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Dzarda7 commented Oct 31, 2024

cederom commented Oct 31, 2024

cederom commented Nov 3, 2024

cederom commented Nov 3, 2024

Dzarda7 commented Nov 3, 2024

Dzarda7 commented Nov 4, 2024

Dzarda7 commented Nov 4, 2024 • edited by radimkarnis Loading

cederom commented Nov 4, 2024

Dzarda7 commented Nov 5, 2024 • edited by radimkarnis Loading

cederom commented Nov 5, 2024

Dzarda7 commented Nov 5, 2024

cederom commented Nov 5, 2024 • edited Loading

cederom commented Nov 5, 2024

Dzarda7 commented Nov 6, 2024

cederom commented Nov 6, 2024 • edited Loading

Dzarda7 commented Nov 6, 2024

cederom commented Nov 6, 2024

Dzarda7 commented Nov 4, 2024 •

edited by radimkarnis

Loading

Dzarda7 commented Nov 5, 2024 •

edited by radimkarnis

Loading

cederom commented Nov 5, 2024 •

edited

Loading

cederom commented Nov 6, 2024 •

edited

Loading