esptool via python code - Failed to enter Flash download mode. Only got 2 byte status response. (ESPTOOL-945)

[brentpicasso]

Operating System

Linux

Esptool Version

4.8.1

Python Version

3.11.9

Full Esptool Command Line that Was Run

N/A

Esptool Output

I'm trying to flash an ESP32-S3 within my python program. using esptool (4.8.1) I can successfully flash using the command line version of esptool.py:

esptool.py -p /dev/ttyACM0 write_flash 0x10000 firmware.bin

However, my sample code (using the same esptool version) generates an error when calling esp.flash_begin():

Failed to enter Flash download mode. Only got 2 byte status response.

failing sample code

from esptool.cmds import detect_chip

try_port = "/dev/ttyACM0"
filename = "./firmware.bin"
def progress_callback(current, total):
    percent = current / total * 100
    print(f"percent {percent}")
    
with detect_chip(try_port) as esp:
    print(str(type(esp)))
    esp.connect()
    esp.change_baud(115200)
    chip_desc = esp.get_chip_description()
    features = esp.get_chip_features()
    print(f"Detected bootloader on port {try_port} : {chip_desc}")
    print(f"Features {features}")
    port = try_port
    esp.run_stub()

    with open(filename, 'rb') as firmware:
        firmware_data = firmware.read()
        print(f"{len(firmware_data)}")
        esp.flash_begin(len(firmware_data), 0x10000)
        esp.flash_data(firmware_data, progress_callback=progress_callback)
        esp.flash_finish()

It works up until the flash_begin command:

python f.py 
Connecting...
Detecting chip type... ESP32-S3
<class 'esptool.targets.esp32s3.ESP32S3ROM'>
Connecting...
Changing baud rate to 115200
Changed.
Detected bootloader on port /dev/ttyACM0 : ESP32-S3 (QFN56) (revision v0.2)
Features ['WiFi', 'BLE', 'Embedded PSRAM 2MB (AP_3v3)']
Uploading stub...
Running stub...
Stub running...
1562064
Traceback (most recent call last):
  File "/home/brent/git-projects/RC_app_private/f.py", line 19, in <module>
    esp.flash_begin(len(firmware_data), 0x10000)
  File "/home/brent/.pyenv/versions/r_c/lib/python3.11/site-packages/esptool/loader.py", line 916, in flash_begin
    self.check_command(
  File "/home/brent/.pyenv/versions/r_c/lib/python3.11/site-packages/esptool/loader.py", line 516, in check_command
    raise FatalError(
esptool.util.FatalError: Failed to enter Flash download mode. Only got 2 byte status response.

I'm assuming there might be additional setup / configuration within the code, or there is a bug in esptool.

I have tested it successfully with this approach. However, I need to use a progress callback, so calling esptool.main() with args is not an option.

from esptool.cmds import detect_chip, write_flash
import argparse
import esptool

try_port = "/dev/ttyACM0"
filename = "./firmware.bin"
#esp = detect_chip(try_port)
args = ['--port', try_port, 'write_flash', '0x10000', filename]
esptool.main(args)

### What is the Expected Behaviour?

Flashing should happen via the python code.

### More Information

_No response_

### Other Steps to Reproduce

_No response_

[dobairoland]

Duplicate here: https://esp32.com/viewtopic.php?f=13&t=42469

[radimkarnis]

When you initialize the esp object, you get a ROM class - because esptool first expects to talk to the ROM bootloader.

ESP32-S3 ROM sends 4 status bytes in a response (read here), but the stub flasher sends only 2 bytes.

Only got 2 byte status response means that esptool is expecting 4 bytes (talking to the ROM), but only gets 2 (because you've activated the stub flasher).

If you inspect how esp.run_stub() is used in esptool, you'll see it replaces the original esp instance as esp = esp.run_stub(). It uploads and runs the stub and then returns an appropriate stub class (e.g. ESP32S3StubLoader) to replace the ROM class (ESP32S3ROM). The stub class is almost the same but contains little adjustments (e.g. the number of status bytes) to accommodate the stub running on the chip.

[radimkarnis]

Search for STATUS_BYTES_LENGTH in esptool to see the implementation details.

Btw, in the next release, we plan to make esptool easier to use as a Python module. Feel free to share more of your experience with writing a custom script using esptool. We'll take this as a feedback of the current state.

[brentpicasso]

@radimkarnis thank you for the quick reply. Happy to help provide feedback on flashing via code, if at least providing a concise example of how to flash through this effort here 👍

I'm getting further; I took you advise and I'm executing the commands via the stub.

from esptool.cmds import detect_chip

try_port = "/dev/ttyACM0"
filename = "./firmware.bin"
def progress_callback(current, total):
    percent = current / total * 100
    print(f"percent {percent}")
    
with detect_chip(try_port) as esp:
    print(str(type(esp)))
    esp.connect()
    esp.change_baud(115200)
    chip_desc = esp.get_chip_description()
    features = esp.get_chip_features()
    print(f"Detected bootloader on port {try_port} : {chip_desc}")
    print(f"Features {features}")
    port = try_port
    stub = esp.run_stub()

    with open(filename, 'rb') as firmware:
        firmware_data = firmware.read()
        print(f"{len(firmware_data)}")
        stub.flash_begin(len(firmware_data), 0x10000)
        stub.flash_data(firmware_data, progress_callback=progress_callback)
        stub.flash_finish()

I'm seeing the stub loader raising this error

raceback (most recent call last):
  File "/home/brent/git-projects/RC_app_private/f.py", line 24, in <module>
    stub.flash_data(firmware_data, progress_callback=progress_callback)
    ^^^^^^^^^^^^^^^
AttributeError: 'ESP32S3StubLoader' object has no attribute 'flash_data'

I think I was getting this from an old example, possibly for an older version of esptool. Where would I look for an example of how to flash a region?

[brentpicasso]

Update, I think I have it working: Please advise if I'm doing it correctly, especially around if last block is handled correctly, since it will not be a whole block size:

from esptool.cmds import detect_chip

try_port = "/dev/ttyACM0"
filename = "./firmware.bin"
BLOCK_SIZE = 0x4000 # Typical block size (16 KB)
FLASH_BEGIN = 0x10000

def progress_callback(current, total):
    percent = int(current / total * 100)
    print(f"percent {percent}")
    
with detect_chip(try_port) as esp:
    print(str(type(esp)))
    esp.connect()
    esp.change_baud(115200)
    chip_desc = esp.get_chip_description()
    features = esp.get_chip_features()
    print(f"Detected bootloader on port {try_port} : {chip_desc}")
    print(f"Features {features}")
    port = try_port
    stub = esp.run_stub()

    with open(filename, 'rb') as firmware:
        firmware_data = firmware.read()
        print(f"firmware length {len(firmware_data)}")
        total_size = len(firmware_data)
        stub.flash_begin(total_size, FLASH_BEGIN)

        # Flash in blocks using flash_block
        block_size = BLOCK_SIZE
        for i in range(0, total_size, block_size):
            chunk = firmware_data[i:i + block_size]
            stub.flash_block(chunk, i + FLASH_BEGIN)
            progress_callback(i + len(chunk), total_size)        
        stub.flash_finish()
        
        stub.hard_reset()

[radimkarnis]

In theory, this looks good, except for the last block handling. Esptool pads the last block with 0xFF bytes, see here.

I would advise you to study the write_flash function in esptool. It contains many checks and scenarios (e.g. compressed flashing, encrypted flashing, ...), but if you strip it all down, it basically boils down to what you are trying to achieve here. The functional description of writing data is here.

[brentpicasso]

@radimkarnis Thank you for the feedback, I had a sense that last block might be an issue!

In the spirit of making the library easier to use from code, I have a new, related question.

Our workflow is this:

• User goes to flash firmware in our app
• our app triggers a soft reset-into-bootloader using this technique in the firmware
• our app flashes the firmware
• when done, the device resets back into our firmware

How we are triggering the bootloader via firmware:

void cpu_device_reset(int bootloader)
{
        /* TODO perform a soft reset of the CPU */
        if(bootloader>0)
                REG_WRITE(RTC_CNTL_OPTION1_REG, RTC_CNTL_FORCE_DOWNLOAD_BOOT);

        //trigger the reset
        esp_restart(); 
}

Problem that happens:

• esptool.py command line works correctly with the soft-reboot-into-bootloader
• our script (example above) fails with the soft-reboot-into-bootloader
• However our script works correctly when hard-reset (power cycle) into bootloader mode by holding button

demo

esptool_flashing_after_soft_reboot-2024-10-29_07.39.34-ezgif.com-video-speed.1.mp4

[radimkarnis]

I love the video and detailed issue report!

I do not know the exact reason why this is happening, but I have a suspicion. Hopefully, you'll be able to find the culprit with my pointers:

Do you really need the esp.connect() line?

with detect_chip(try_port) as esp:
    print(str(type(esp)))
    esp.connect()

detect_chip() calls connect() internally, I think you don't need to call it again. Since connect() triggers the bootloader reset, I think there might be unnecessary resets happening.

In your video, you can even see that the script works until you call esp.connect() again - that's where the chip resets out of bootloader.

Please see the logic for hard-resetting ESP32-S3. We found out, that it is impossible to exit the bootloader mode if a hard reset is performed (unplugging and plugging again while holding the BOOT button) in USB-Serial/JTAG mode. For this reason, the RTC WDT reset was introduced recently.

There is logic to decide if an RTC WDT reset or a classic reset via RTS pin happens. In your case, maybe a different reset gets triggered in the two cases - one resulting in a reset back to the bootloader and the other resulting in running your firmware. I suggest adding a few print statements to see which one is being triggered.

[brentpicasso]

Thank you for the quick reply. Just confirmed, removing the extra .connect() does solve the problem!

Our test script does seem to trigger the WDT reboot, so I trust that once the next version of esptool is released it will start working as expected.

I will follow up with corrected example code that includes the 0xFF padding.

Is there a place in your documentation this sample could be added to help others? I'll have to check to see how your documentation is designed, and if it is github based. I'll be happy to contribute it there.

[brentpicasso]

Here's the final example code. I can see where esptool could be updated to encapsulate this logic, but it's still pretty concise for people to use as-is.

#############################################################
# Demo code functionally equivalent to
# esptool.py -p /dev/ttyACM0 write_flash 0x10000 firmware.bin
#############################################################
from esptool.cmds import detect_chip

# the port of the connected ESP32
port = "/dev/ttyACM0"

# The firmware file
filename = "./firmware.bin"

# Typical block size (16 KB)
BLOCK_SIZE = 0x4000

#beginning of flash address
FLASH_BEGIN = 0x10000

def progress_callback(percent):
    print(f"percent {int(percent)}")
    
with detect_chip(port) as esp:
    chip_desc = esp.get_chip_description()
    features = esp.get_chip_features()
    print(f"Detected bootloader on port {port} : {chip_desc}")
    print(f"Features {features}")
    
    stub = esp.run_stub()
    with open(filename, 'rb') as firmware:
        firmware_data = firmware.read()
        print(f"firmware length {len(firmware_data)}")
        total_size = len(firmware_data)
        stub.flash_begin(total_size, FLASH_BEGIN)

        # Flash in blocks using flash_block
        block_size = BLOCK_SIZE
        for i in range(0, total_size, block_size):
            block = firmware_data[i:i + block_size]
            # pad the last block
            block = block + bytes([0xFF]) * (BLOCK_SIZE - len(block))
            stub.flash_block(block, i + FLASH_BEGIN)
            progress_callback(float(i + len(block)) / total_size * 100)
        stub.flash_finish()
        
        stub.hard_reset()

[brentpicasso]

Follow up: #1029