PDM Microphone samples are not recognized at the input, Amplitude sits at 0

[markusdd]

What happened?

I have (new) boards sitting here with a TDK T3902 PDM MEMS Microphone and I am trying to get this to work.
ESP is an ESP32S3-N16R8.
I have tried WLED 0.15 as well my own build from latest/greatest master branch.

Issue I am observing: In PDM mode I am using GPIO4 as Clock out and GPIO5 as data in.
With an oscilloscope I can see the clock leaving the ESP and the data returning (yellow clock, blue data):

This gives me pretty high confidence that the hardware is fine.
Also, to exclude channel left/right woes, I reworked one board to have a left channel output and the other right channel output. The datasheet of the mic is here: https://www.lcsc.com/datasheet/lcsc_datasheet_2411220511_TDK-InvenSense-MMICT390200012_C3171752.pdf
To me, this looks like a very 'run of the mill' PDM mic with hardly any specialties, the protocol is super simple. Rising edge right channel, falling edge left channel, that's it.

My pin config is as follows:

I also tried using WS as clk, same picture. I get proper output/input on the scope, but my console does this: (I have compiled with -D MIC_LOGGER -D SR_DEBUG):

Absolutely dead. I also tried forcing the data pin high from 3.3V, as this should result in a pegged high maximum volume. Also nothing.

When I configure it as a normal I2S Microphone, I do see this:

Level and Real sit around 30k constantly and volume is fluctuating. But there is zero 'real' reaction to music or clapping, which is logical, because this isn't a PCM signal but a PDM signal on the input, so essentially these numbers are random garbage, they will never be interpreted correctly in non-PDM mode.

I've tried everything I could think of, but I do not understand why there is zero reading of PDM data although it is there.

I tried skimming the C-Code of the audioreactive mod and there are numerous comments in the code regading PDM that I cannot make sense of, like this:

I do not assume we have an actual clock issue because it comes out.
I rather suspect these 2 (I have not touched any of the code, everything like it is on master except the debug defines):

So for some reason the data is not being sampled. I am at a loss for why.

Any guidance?

To Reproduce Bug

See above, but I guess you'd need such a mic or at least a PDM mic combined with an ESP32S3

Expected Behavior

PDM data is being sampled and audioreactive works.

Install Method

Self-Compiled

What version of WLED?

0.15 and 0.16alpha@master

Which microcontroller/board are you seeing the problem on?

ESP32-S3

Relevant log/trace output

Anything else?

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[markusdd]

When checking the log, the setup indeed looks as expected. I am at a loss as to why in proper PDM mode the data pin is seemingly not read.

Can anyone give guidance how to further debug?

Usermods setup
AR: I2S PDM Microphone - left channel only.
I2SSource:: initialize().
PIN ALLOC: Pin 4 successfully allocated by 0x20.
PIN ALLOC: 0x0000000000000lX.
PIN ALLOC: Pin 5 successfully allocated by 0x20.
PIN ALLOC: 0x0000000000000lX.
AR: I2S#0 driver without aPLL; fixed_mclk=0.
AR: 32 bits, Sample scaling factor = 0.2500
AR: I2S#0 driver installed in PDM MASTER mode.
FFT started on core: Total # of palettes: 0
0
Adding audio palettes.
1
2
3

[blazoncek]

Perhaps a bug in Audioreactive usermod.
I know @softhack007 and @troyhacks were working on PDM microphone for MM fork.

[troyhacks]

I've had success with PDM mics in WLED-MM with no major issues.

Sometimes I've had to set -D I2S_USE_16BIT_SAMPLES in my builds or otherwise the audio is too low to be recognized. Quindor was doing some PDM testing the other day and he didn't have to add that tho.

@markusdd could you cross-test with WLED-MM mdev and see if that helps?

WLED-MM has forked pretty far from mainstream WLED, but the AudioReactive fixes should be easy to backport if needed.

[markusdd]

after the hint of @blazoncek I tried the MM fork yesterday.

The behavior is identical as far as I could make out.
To get a proper build config I stripped the Hub75 stuff from the s3 moon module config as otherwise my board is similar, which uses an ESP32S3 N16R8. I also added WLED_DEBUG and SR_DEBUG flags. and adapted the pins.

[env:esp32S3_8MB_PSRAM_M]
extends = esp32_4MB_V4_M_base

board = esp32-s3-devkitc-1 ;; generic S3 dev board; the next line adds PSRAM support
board_build.arduino.memory_type = qio_opi     ;; use with PSRAM: 8MB or 16MB
;; board_build.arduino.memory_type = qio_qspi ;; use with PSRAM: 2MB or  4MB
board_build.flash_mode = qio ;; use "dio" if your board gets unstable with "qio"; also change then previous lines accordingly (memory_type)

build_unflags = ${env:esp32S3_8MB_M.build_unflags}   ;; use the same as "normal" S3 buildenv
build_flags = ${common.build_flags} ${esp32s3.build_flags} -Wno-misleading-indentation -Wno-format-truncation
  ${common_mm.build_flags_S} ${common_mm.build_flags_M}
  # ${common_mm.HUB75_build_flags}
  -DBOARD_HAS_PSRAM ;; -D WLED_USE_PSRAM ;; your board supports PSRAM
  -D WLED_USE_PSRAM_JSON -DALL_JSON_TO_PSRAM ; WLEDMM --> force all JSON stuff into PSRAM; gives more free heap
  -D WLED_RELEASE_NAME=esp32S3_8MB_PSRAM_M
  -DARDUINO_USB_MODE=1 -DARDUINO_USB_CDC_ON_BOOT=0 -DARDUINO_USB_MSC_ON_BOOT=0 -DARDUINO_USB_DFU_ON_BOOT=0     ;; for Serial-to-USB chip
  ;;-DARDUINO_USB_MODE=1 -DARDUINO_USB_CDC_ON_BOOT=1 -DARDUINO_USB_MSC_ON_BOOT=0 -DARDUINO_USB_DFU_ON_BOOT=1   ;; for Hardware-CDC USB mode
  ;;-D WLED_DISABLE_ADALIGHT     ;; disables serial protocols - recommended for Hardware-CDC USB (Serial RX will receive junk commands when RX pin is unconnected, unless its pulled down by resistor)
  -D WLED_WATCHDOG_TIMEOUT=0 -D CONFIG_ASYNC_TCP_USE_WDT=0
  ;;-D WLEDMM_FASTPATH ; WLEDMM experimental option. Reduces audio lag (latency), and allows for faster LED framerates. May break compatibility with previous versions.
  -D WLED_DISABLE_LOXONE ; FLASH 1272 bytes
  -D WLED_DISABLE_ALEXA ;  RAM 116 bytes; FLASH 13524 bytes
  ; -D WLED_DISABLE_MQTT ;   RAM 216 bytes; FLASH 16496 bytes
  ; -D WLED_DISABLE_HUESYNC ;RAM 122 bytes; FLASH 6308 bytes
  ; -D WLED_DISABLE_INFRARED ;RAM 136 bytes; FLASH 24492 bytes
  -D LEDPIN=38
  ;;-D DATA_PINS=21,48,3 -D PIXEL_COUNTS=30,1,144  ;; just an example: my board has a builtin neopixel on gpio48
  ; -D STATUSLED=2
  -D BTNPIN=-1 -D RLYPIN=-1 -D IRPIN=-1
  -D HW_PIN_SDA=-1 -D HW_PIN_SCL=-1
  -D AUDIOPIN=-1
  -D SR_DMTYPE=5 -D I2S_SDPIN=5 -D I2S_CKPIN=-1 -D I2S_WSPIN=4 -D MCLK_PIN=-1
  ;;-D ES7243_SDAPIN=8 -D ES7243_SCLPIN=18
  -D WLED_DEBUG
  -D SR_DEBUG
  ; -D MIC_LOGGER
lib_deps = ${esp32s3.lib_deps} ${common_mm.lib_deps_S}  ${common_mm.lib_deps_V4_M}
;${common_mm.HUB75_lib_deps}
;lib_ignore = IRremoteESP8266 ; use with WLED_DISABLE_INFRARED for faster compilation
board_build.partitions = tools/WLED_ESP32_8MB.csv
; RAM:   [==        ]  21.1% (used 69156 bytes from 327680 bytes)
; Flash: [========  ]  75.9% (used 1591817 bytes from 2097152 bytes)

According to the logs the config is fine and PDM master driver is starting up.
But behavior is exact same as with normal WLED, sound processing is running but signal is dead 0.

Usermods setup ...
AR: I2S PDM Microphone - left channel only.
I2SSource:: initialize().
PIN ALLOC: Pin 4 successfully allocated by AudioReactive (UM) = 0x20 (32)
PIN ALLOC: Pin 5 successfully allocated by AudioReactive (UM) = 0x20 (32)
AR: I2S#0 driver without aPLL; fixed_mclk=0.
AR: 32 bits, Sample scaling factor = 0.2500
AR: I2S#0 driver installed in PDM MASTER mode.
AR: FFT task started on core 0 [prio=4, min free stack=|| loopTask  min free stack 3880
2640]
AR: sound input driver initialized successfully.

Free heap 273072

alocateFFTBuffers() completed successfully.

EDIT: pasted correct config I actually ended up using.

[markusdd]

Ok, so after some more trying I am at this stage now:

This uses these settings as well as the -D I2S_USE_16BIT_SAMPLES define.

It only work with this (I have to take the legacy mode, normal PDM mode is dead):

So with these settings there seems to be something there, but very faint:

Also it seems not really reactive to any music, so probably more noise floor than anything else, but at least not totally dead.

I had linked the datasheet of the Microphone I use above, any idea what might be different?
Maybe right instead of left channel? Any other weird settings?

EDIT: Also, it must actually be capturing something from the input, it's not just internal noise. When I cut VDD from the microphone the LEDs go completely dark. and we drop to these constant values:

EDIT2:
When I remove the 16bit define, I get these values:

What's notable is that MicMax sometimes shows 0.01, so whatever we are getting seems extremely faint.

So I am more and more leaning towards a processing issue or some littel nitpick in the i2s config oder pdm processing.

EDIT3:

When switching away from legacy PDM to Generic PDM it goes completely dead, flat 0 all the time (same behavior as in normal non-MM WLED):

EDIT4:

In legacy PDM mode, without 16-bit define, when I force the data input high with a 3.3V line, we get here:

So that is essentially the maximum possible volume, wwhich is way too low.

[markusdd]

Any thoughts from the experts on what to try next?

I think we have found modes where there certainly is an input being recognized but in any case whatever is there is way too low, even when holding the input high. So I assume it must be some magic setting in the software.

[netmindz]

See recent discussion on PDM - https://discord.com/channels/700041398778331156/1076072513102168084/1339765830371639408

[markusdd]

It says I do not have access to this.

[github-actions]

Hey! This issue has been open for quite some time without any new comments now. It will be closed automatically in a week if no further activity occurs.
Thank you for using WLED! ✨