DMIC pop / click sounds between each memory block

RE: DMIC pop / click sounds between each memory block

ovrebekk — Fri, 12 Jan 2024 09:00:24 GMT

Hi Timon

Very good to hear that this fixed the issue 🙂

If you look into the exact timings of the printk calls maybe you would find when they start to lag too much behind the DMIC operation to cause issues. I understand the desire to fully understand what is happening, but it is probably not worth the investment in time in this case.

The best of luck with your project!

Best regards
Torbjørn

RE: DMIC pop / click sounds between each memory block

timonsku — Thu, 11 Jan 2024 15:09:34 GMT

Hi Torbjørn,

that is really interesting and yes it does fix the issue!
I'm a bit puzzled as that means something works very differently than I thought with the dmic driver but I'm glad to know that I can ensure things stream correctly if there is definitely no bandwidth bottleneck.
There is that "starting pop" still but that is fine and seems to be what the Thingy:53 firmware mentioned.
Also good to know that UART is not async by default

Thank you for looking at this so thoroughly, it's much appreciated!

RE: DMIC pop / click sounds between each memory block

ovrebekk — Thu, 11 Jan 2024 12:39:22 GMT

Hi Timon

I did some tests myself in order to evaluate the performance of the UART communication.

Trying out the code you sent me I connected the logic analyzer to the TX line in order to see if there could be any issues with the UART communication, and the results were interesting:

After every byte I see a gap of around 3.5us, which means the overall throughput will be reduced by a significant factor.

Exactly why this happens I am not sure, but I wouldn't be surprised if the console driver (printk) sends data to the underlying transport layer (UART in this case) byte by byte, which means the UART driver is not able to buffer multiple bytes in one transaction.

I did a quick rewrite of the code integrating a more efficient UART layer, utilizing the UART async driver to send larger buffers in one go, and now communication seems more efficient:

Most bytes are sent back to back, except for a ~35us delay when switching from one buffer to the next.

There is a larger delay every 100ms which I assume shows that the bandwidth is now more than sufficient to send the data in time.

Can you try out my code and see if it solves the glitching issue?

devzone.nordicsemi.com/.../319611_5F00_dmic_5F00_issue.zip

Please note I use a wrapper for the UART async driver called 'app_uart' in order to simplify the UART interface, but everything is included in the repo. In order to track my changes you can check the git log.

Best regards
Torbjørn

RE: DMIC pop / click sounds between each memory block

timonsku — Tue, 09 Jan 2024 16:59:16 GMT

Yea I can hook up a Saleae but need to check if there are usable test points. the uart is read out via the onboard debugger.
Ultimately I want to send the audio samples via BLE GATT (BLE audio is not well enough supported by other devices and not necessary for this application, the samples will be processed by software), while it would be really helpful for development to have some other way to send out sample data to debug I would mostly care about ensuring that you can actually stream DMIC data "easily", the current issue worries me a bit that this is going to very finicky to avoid such artifacts.

I'm sceptical that the UART is the problem here because of the following observation and assumptions.
The dmic peripheral is provided 4 memory blocks to use for sampling. Sampling is done via DMA so user code should not be blocking the dmic peripheral from continuing to fill up the next free block with sample data. Only when all 4 blocks are exhausted and not freed by the user should the user code become a bottleneck but the clicks appear after the very first block.
If I would miss sample data after the 4th block it would be fair to assume that we can't keep up with freeing memory blocks fast enough to provide free blocks to the dmic peripheral.

Which leads me to assume that there is either something that needs to be done to correctly join blocks of sample data and my naive way of just appending it all is not correct or that there is something wrong in the way I setup the dmic peripheral that it maybe resets something with each block read.
I saw a reference in the firmware for the Thingy:53 of avoiding a click sound when starting with the sampling but not specifics as to what the reason for that artifact is. I do observe that the samples need to settle for a few ms, there is a consistent observable "curve" whenever I make a recording.

E: See here for the mentioned comment: https://github.com/edgeimpulse/firmware-nordic-thingy53/blob/main/src/sensors/ei_microphone.cpp#L314-L331

Also here a screenshot of the attached audio data visualized to maybe give you better context. In this case little bit of the first sample got missed in my log, usually the click occurs after the first sample, here it looks like it starts shortly within the first sample. The spikes after are from the 2-4th sample. You can see the regularity of a click after exactly 100ms, which is how much sampling data fits into one block.

RE: DMIC pop / click sounds between each memory block

ovrebekk — Tue, 09 Jan 2024 10:47:43 GMT

Hi Timon

I will set aside some time tomorrow to test this on my end. I am relatively sure that the UART link is somehow part of the problem, since the DMIC code itself is very straight forward.

Do you have a scope or logic analyzer available in order to check the UART activity?
The UART would have to run close to 100% of the time for the bandwidth to be sufficient, any gaps between transactions or slowdowns would cause issues.

Best regards
Torbjørn

RE: DMIC pop / click sounds between each memory block

timonsku — Fri, 05 Jan 2024 21:34:21 GMT

I had tested fewer samples before, even just doing a single block results in having at least one click. The only times I saw a click sometimes not appear was with increasing block size significantly but it was inconsistent with some blocks having it and some not.

I dump the samples by connecting via USB to a PC and saving the serial output to a file/logging it and use the python script in the project zip file to turn it from ASCII into a binary file. That binary file I then open in Adobe Audition by doing a raw PCM import. Audacity would probably also work.
For the serial logging I use "tio" on an Ubuntu machine but any serial terminal should do it including the VSCode one that Nordic ships.

Thanks for the link, I will see if there is anything significantly different that I can try.

RE: DMIC pop / click sounds between each memory block

ovrebekk — Fri, 05 Jan 2024 12:49:20 GMT

Hi Timon

[quote user="DiodesDelight"]the UART is set close to 1M Baud, that should be more than enough in theoery.[/quote]

Hm, I would say it is just about enough.

16000 samples/sec * 5 ascii characters pr sample * 10 bits pr character (including stop and start bit) = 800kBaud

So it is enough, yes, but not by a large factor. The question is how much overhead there is in the driver, and how much CPU time is required to process all those samples and convert them to a string, before passing them to the UART driver.

I would suggest running a quick test just changing num_samples in the loop to a small number like 10 and see if it makes a difference.

A colleague of mine recently did some work integrating a PDM mic on the nRF52832, and the code looks very similar:
https://github.com/Noy0908/smart_remote/blob/desktop_remote/src/smart_remote/drv_mic.c#L191-L218

By the way, how do you transfer the audio data to the PC for analysis?
If you still can't make it work I will try to replicate your test here, but I need to figure out the easiest way of transferring the audio data somewhere else for playback. Maybe storing to SD card on the audio DK is the best bet...

Best regards
Torbjørn

RE: DMIC pop / click sounds between each memory block

timonsku — Thu, 04 Jan 2024 14:12:27 GMT

Hi Torbjørn,

the UART is set close to 1M Baud, that should be more than enough in theoery.

The pop occurs even at lower baud rates without a noticable change and given the dmic reads via DMA I don't think that could cause a hickup? If its due to being too slow to free the memory blocks then I should get 4 click free blocks before seeing clicks but this is not the case.

I only have the UART to check the validity of the samples. Printk via logging is disabled so the logging infrastructure should not interfere or be an overhead.

RE: DMIC pop / click sounds between each memory block

ovrebekk — Thu, 04 Jan 2024 10:15:13 GMT

Hi Timon

Thanks, now I could download all the files 🙂

My main concern looking at your code is the fact that you are printing all the data to the console using printk(), this might introduce quite a lot of delay in the main loop. Could you try to comment this out and see if the pops go away?

Best regards
Torbjørn

RE: DMIC pop / click sounds between each memory block

timonsku — Wed, 03 Jan 2024 17:04:31 GMT

Oh weird, I edited the post with a new upload. Can you try again?

RE: DMIC pop / click sounds between each memory block

ovrebekk — Wed, 03 Jan 2024 13:26:18 GMT

Hi Timon

For some reason the dmic.zip attachment is not able to load (the other attachment works). Would you be able to attach this file one more time?

Best regards
Torbjørn

RE: DMIC pop / click sounds between each memory block

timonsku — Wed, 27 Dec 2023 17:39:50 GMT

Hi ovrebekk totally understand!

RE: DMIC pop / click sounds between each memory block

ovrebekk — Wed, 27 Dec 2023 12:33:01 GMT

Hi Timon

I ended up with this case also. Unfortunately we are severely understaffed during the Christmas period, and I will have to investigate further next week. I apologize for the inconvenience.

Best regards
Torbjørn