Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ESP32-LyraTD-MSC board can't run ESP-ADF wwe example. (AUD-5393) #1203

Closed
hyuan-kamuda opened this issue May 8, 2024 · 3 comments
Closed

ESP32-LyraTD-MSC board can't run ESP-ADF wwe example. (AUD-5393) #1203

hyuan-kamuda opened this issue May 8, 2024 · 3 comments

Comments

@hyuan-kamuda
Copy link

hyuan-kamuda commented May 8, 2024 •
edited
Loading

## Environment

  • Audio development kit: ESP32-LyraTD-MSC
  • Audio kit version: v2.2
  • [Required] Module or chip used: ESP32-WROVER-E
  • [Required] IDF version: v5.2.1
  • [Required] ADF version: v2.6-108-g1f275b50
  • Build system: visual code/idf.py
  • [Required] Running log: All logs from power-on to problem recurrence
  • Compiler version : xtensa-esp-elf-gcc.exe (crosstool-NG esp-13.2.0_20230928) 13.2.0
  • Operating system: Windows
  • (Windows only) Environment type: PowerShell
  • Using an IDE?: Yes, Visual Code+ IDF extension
  • Power supply: USB

## Problem Description

ESP32-LyraTD-MSC board can't run ESP-ADF wwe example, and results in CPU halt/crash.
With ‘I (12132) main_task: Returned from app_main()“ log printed, any speech input the system has no response, after xx seconds, the crash will happen.

### Expected Behavior
Should work normally.

### Actual Behavior
Crash.

### Steps to Reproduce

  1. From https://github.com/espressif/esp-adf/tree/master/examples readme table, wwe example(examples/speech_recognition/wwe) should be supported by ESP32-LyraTD-MSC board. And tried another speech_recognition example(examples/speech_recognition/vad) with same board which can work normally.
  2. In VSCode, create a project from example of wwe.
  3. Adjusted some memory/system settings following https://docs.espressif.com/projects/esp-adf/en/latest/design-guide/design-considerations.html:
    a. Enabled SPI PSRAM. The option CONFIG_SPIRAM_CACHE_WORKAROUND, set by default in the same menu, kept enabled.
    b. Not enable Bluetooth.
    c. CONFIG_WIFI_LWIP_ALLOCATION_FROM_SPIRAM_FIRST set as “yes”
    d. Set CONFIG_WL_SECTOR_SIZE as 512 in Component config > Wear Levelling
    e. Flash SPI mode as QIO
    f. Flash SPI speed as 80MHz
    g. CPU frequency as 240MHz
    h. Set Default receive window size as 5 times(7200) greater than Maximum Segment Size(1440) in Component config > LWIP > TCP
    i. Set Flash Size as 4MB
  4. Follow section of "Troubleshooting" of readme of https://github.com/espressif/esp-adf/tree/master/examples/speech_recognition/wwe, changed esp\esp-adf\components\audio_hal\driver\zl38063.c ( tw_upload_dsp_firmware(0) ===> tw_upload_dsp_firmware(-1); )
  5. Build the application and flash/run it, system will hang, found there's warning: "W (10615) RECORDER_SR: Multinet is not enabled in SDKCONFIG" , then:
    . enable "use multinet"
    . Chinese Speech Commands Model: Chinese Single Recognition(mn2_cn)
    . Select wake words: nihaoxiaozhi(wn5_nihaoxiaozi)
  6. Rebuild application, and flash, run it.

rst:0x1 (POWERON_RESET),boot:0x1f (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:7524
load:0x40078000,len:16528
ho 0 tail 12 room 4
load:0x40080400,len:4
0x40080400: _init at ??:?

load:0x40080404,len:4312
entry 0x4008065c
I (31) boot: ESP-IDF v5.2.1-dirty 2nd stage bootloader
I (31) boot: compile time May 7 2024 23:55:14
I (31) boot: Multicore bootloader
I (35) boot: chip revision: v1.0
I (39) qio_mode: Enabling default flash chip QIO
I (44) boot.esp32: SPI Speed : 80MHz
I (49) boot.esp32: SPI Mode : QIO
I (54) boot.esp32: SPI Flash Size : 4MB
I (58) boot: Enabling RNG early entropy source...
I (64) boot: Partition Table:
I (67) boot: ## Label Usage Type ST Offset Length
I (74) boot: 0 nvs WiFi data 01 02 00009000 00004000
I (82) boot: 1 otadata OTA data 01 00 0000d000 00002000
I (89) boot: 2 phy_init RF data 01 01 0000f000 00001000
I (97) boot: 3 ota_0 OTA app 00 10 00010000 00300000
I (104) boot: 4 flash_tone Unknown data 01 27 00310000 00032000
I (112) boot: End of partition table
I (116) boot: No factory image, trying OTA 0
I (121) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=193ea0h (1654432) map
I (554) esp_image: segment 1: paddr=001a3ec8 vaddr=3ffb0000 size=0285ch ( 10332) load
I (557) esp_image: segment 2: paddr=001a672c vaddr=40080000 size=098ech ( 39148) load
I (571) esp_image: segment 3: paddr=001b0020 vaddr=400d0020 size=50828h (329768) map
I (656) esp_image: segment 4: paddr=00200850 vaddr=400898ec size=0a0a4h ( 41124) load
I (679) boot: Loaded app from partition at offset 0x10000
I (717) boot: Set actual ota_seq=1 in otadata[0]
I (718) boot: Disabling RNG early entropy source...
I (728) cpu_start: Multicore app
I (728) quad_psram: This chip is ESP32-D0WD
I (729) esp_psram: Found 8MB PSRAM device
I (731) esp_psram: Speed: 80MHz
I (734) esp_psram: PSRAM initialized, cache is in low/high (2-core) mode.
W (742) esp_psram: Virtual address not enough for PSRAM, map as much as we can. 4MB is mapped
I (1214) esp_psram: SPI SRAM memory test OK
I (1222) cpu_start: Pro cpu start user code
I (1222) cpu_start: cpu freq: 240000000 Hz
I (1222) cpu_start: Application information:
I (1226) cpu_start: Project name: example_wwe
I (1231) cpu_start: App version: 1
I (1236) cpu_start: Compile time: May 8 2024 10:09:58
I (1242) cpu_start: ELF file SHA256: b05478608...
I (1247) cpu_start: ESP-IDF: v5.2.1-dirty
I (1253) cpu_start: Min chip rev: v0.0
I (1257) cpu_start: Max chip rev: v3.99
I (1262) cpu_start: Chip rev: v1.0
I (1267) heap_init: Initializing. RAM available for dynamic allocation:
I (1274) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (1281) heap_init: At 3FFB3938 len 0002C6C8 (177 KiB): DRAM
I (1287) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (1293) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (1300) heap_init: At 40093990 len 0000C670 (49 KiB): IRAM
I (1306) esp_psram: Adding pool of 4096K of PSRAM memory to heap allocator
I (1314) spi_flash: detected chip: generic
I (1318) spi_flash: flash io: qio
W (1323) i2c: This driver is an old driver, please migrate your application code to adapt driver/i2c_master.h
W (1333) ADC: legacy driver is deprecated, please migrate to esp_adc/adc_oneshot.h
I (1342) main_task: Started on CPU0
I (1352) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (1352) main_task: Calling app_main()
I (2352) SPI: ** Loading DSP firmware ret:0,Status:0 **
I (2352) SPI: b=1
I (2352) SPI: 1- Firmware boot loading started ....
I (5392) SPI: prgmBase 0x00080200

I (5392) SPI: execAddr 0x00084040

I (5392) SPI: DONE

I (5402) SPI: 2- Loading the image to RAM....done
I (5402) SPI: -- Saving firmware to flash....
I (11002) SPI: -- Saving firmware to flash....done
I (11022) SPI: 3- Loading the config file into the device RAM....
I (11192) SPI: -- Saving config to flash....
I (11432) SPI: -- Saving config to flash....done
I (11632) SPI: Device boot loading completed successfully...
I (11632) gpio: GPIO[22]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (11642) gpio: GPIO[21]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
W (11652) AUDIO_BOARD: The board has already been initialized!

----------------------------- ESP Audio Platform -----------------------------
| |
| ESP_AUDIO-v1.7.3-40-g00ffda8-dbab925-9815982 |
| Compile date: Apr 12 2024-02:32:01 |

I (11692) wwe_example: Func:setup_player, Line:140, MEM Total:4434152 Bytes, Inter:332075 Bytes, Dram:282103 Bytes

I (11702) wwe_example: esp_audio instance is:0x3f801d10

I (11822) wwe_example: Recorder has been created
I (11822) RECORDER_SR: The first wakenet model: wn5_nihaoxiaozhi

I (11822) AFE_SR: afe interface for speech recognition

I (11832) AFE_SR: AFE version: SR_V220727

I (11832) AFE_SR: Initial auido front-end, total channel: 2, mic num: 1, ref num: 1

I (11842) AFE_SR: aec_init: 0, se_init: 1, vad_init: 1

I (11842) AFE_SR: wakenet_init: 1

Deprecated Quantized wakenet5: wakeNet5_v1_nihaoxiaozhi_6_0.983_0.95, mode:0 (Nov 21 2023 19:15:51)
Please switch to ESP32-S3 and wakenet9 ! // any problem? ESP32/ESP32-LyraTD-MSC should be supported?
I (11962) AFE_SR: wake num: 1, mode: 1, (Nov 21 2023 19:15:51)

SINGLE_RECOGNITION: v2; (Nov 21 2023 19:40:21)
W (11962) MN: MultiNet may cause WatchDog on ESP32 sometimes! //Why???
esp_mn_commands_update_from_sdkconfig
I (11972) MN_COMMAND: ---------------------SPEECH COMMANDS---------------------
I (11972) MN_COMMAND: Command ID0, phrase ID0: da kai kong tiao
I (11982) MN_COMMAND: Command ID1, phrase ID1: guan bi kong tiao
I (11992) MN_COMMAND: Command ID2, phrase ID2: zeng da feng su
I (11992) MN_COMMAND: Command ID3, phrase ID3: jian xiao feng su
I (12002) MN_COMMAND: Command ID4, phrase ID4: sheng gao yi du
I (12012) MN_COMMAND: Command ID5, phrase ID5: jiang di yi du
I (12012) MN_COMMAND: Command ID6, phrase ID6: zhi re mo shi
I (12022) MN_COMMAND: Command ID7, phrase ID7: zhi leng mo shi
I (12032) MN_COMMAND: Command ID8, phrase ID8: song feng mo shi
I (12032) MN_COMMAND: Command ID9, phrase ID9: jie neng mo shi
I (12042) MN_COMMAND: Command ID10, phrase ID10: chu shi mo shi
I (12042) MN_COMMAND: Command ID11, phrase ID11: jian kang mo shi
I (12052) MN_COMMAND: Command ID12, phrase ID12: shui mian mo shi
I (12062) MN_COMMAND: Command ID13, phrase ID13: da kai lan ya
I (12062) MN_COMMAND: Command ID14, phrase ID14: guan bi lan ya
I (12072) MN_COMMAND: Command ID15, phrase ID15: kai shi bo fang
I (12082) MN_COMMAND: Command ID16, phrase ID16: zan ting bo fang
I (12092) MN_COMMAND: Command ID17, phrase ID17: ding shi yi xiao shi
I (12092) MN_COMMAND: Command ID18, phrase ID18: da kai dian deng
I (12102) MN_COMMAND: Command ID19, phrase ID19: guan bi dian deng
I (12112) MN_COMMAND: ---------------------------------------------------------

I (12132) main_task: Returned from app_main() // system stuck here for xx seconds then crash, see below log, any speech input will not have output.
Guru Meditation Error: Core 0 panic'ed (LoadStoreAlignment). Exception was unhandled.

Core 0 register dump:
PC : 0x4011e70f PS : 0x00060730 A0 : 0x800f69c5 A1 : 0x3f826950
0x4011e70f: bitrv2 at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_processor/noise_suppression/webrtc_ns.c:234

A2 : 0x00000100 A3 : 0x00000000 A4 : 0x3f826df0 A5 : 0x4015c205
A6 : 0x00000000 A7 : 0x00000000 A8 : 0x00000000 A9 : 0x7f7fffff
A10 : 0xb8bd1c6d A11 : 0xbffe49e8 A12 : 0xbb360b61 A13 : 0x3f826df0
A14 : 0x003fffff A15 : 0x3ffb8470 SAR : 0x00000004 EXCCAUSE: 0x00000009
EXCVADDR: 0x4015c205 LBEG : 0x400f8841 LEND : 0x400f8866 LCOUNT : 0x00000000
0x400f8841: WindowingEnergy at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_processor/noise_suppression/webrtc_ns.c:1645
(inlined by) WebRtcNs_ProcessCore at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_processor/noise_suppression/webrtc_ns.c:1916
0x400f8866: WebRtcNs_ProcessCore at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_processor/noise_suppression/webrtc_ns.c:1917

Backtrace: 0x4011e70c:0x3f826950 0x400f69c2:0x3f8269a0 0x400f8a39:0x3f8269f0 0x400f95be:0x3f827b10 0x400f4b89:0x3f827b30 0x400f36da:0x3f827b60 0x400f3898:0x3f827b80 0x400dd015:0x3f827ba0
0x4011e70c: bitrv2 at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_processor/noise_suppression/webrtc_ns.c:230
0x400f69c2: WebRtc_rdft at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_processor/noise_suppression/webrtc_ns.c:155
0x400f8a39: FFT at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_processor/noise_suppression/webrtc_ns.c:1564
(inlined by) WebRtcNs_ProcessCore at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_processor/noise_suppression/webrtc_ns.c:1944
0x400f95be: WebRtcNs_Process at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_processor/noise_suppression/webrtc_ns.c:2126
0x400f4b89: ns_process at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_processor/esp_ns.c:16
0x400f36da: afe_feed_aec_init_false at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_front_end/esp_afe_sr.c:546
0x400f3898: afe_feed at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_front_end/esp_afe_sr.c:623
0x400dd015: feed_task at C:/Users/hyuan/esp/esp-adf/components/audio_recorder/recorder_sr.c:248

ELF file SHA256: b05478608

CPU halted.

Other Items If Possible

  • [ X] sdkconfig file (Attach the sdkconfig file from your project folder), please see attache sdkconfig.

sdkconfig.txt
@github-actions github-actions bot changed the title ESP32-LyraTD-MSC board can't run ESP-ADF wwe example. ESP32-LyraTD-MSC board can't run ESP-ADF wwe example. (AUD-5393) May 8, 2024
@xkkjiayou
Copy link

我也是这个问题，解决了吗
bitrv2 at /home/sunxiangyu/workspace/esp_sr_lib/components/esp_audio_processor/noise_suppression/webrtc_ns.c:234

@hbler99
Copy link

hbler99 commented Jul 15, 2024 •
edited
Loading

Have you execute tw_upload_dsp_firmware(-1) ===> tw_upload_dsp_firmware(0); after first flash? This change is to update the firmware for zl38063.c, and after updating, it needs to be reverted back.
Additionally， for esp32, Multinet is no long supported, and it's disabled by default.

@jason-mao
Copy link
Collaborator

The ESP32-LyraTD-MSC is EOL, please test WWE with other boards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@hyuan-kamuda @jason-mao @xkkjiayou @hbler99