Feasibility: continuous audio capture + FFT processing + LTE-M clip upload on nRF9151

[Thijmen de Vries]

Hi,

We’re Hydro-Guard, a volunteer student team (TU Delft / University of Milan) building solar-powered acoustic recorders that detect illegal boats in nature parks (short video). We currently use an ESP32 and are considering the nRF9151 for our next generation.

What the device needs to do:

1. Continuously sample a digital MEMS mic (PDM or I²S) at 16 kHz.
2. Run an FFT-based detector on the application core: ~75 FFTs of 4096 points per 5 s clip, ~20–40 kB working memory.
3. Compress selected 5 s clips (~10–40 kB each) and upload them over LTE-M, up to ~100 clips/day.
4. Run on a small solar panel + battery.

Questions:

• Is there enough RAM left for audio buffers and this processing while the modem library and TLS are active?
• Can audio capture and processing keep running while the modem is transmitting?
• Are there samples or reference designs for audio capture + LTE-M upload on the nRF91 series?

We’re new to the Nordic platform, so any guidance on where to start is very welcome.

Thanks!
Thijmen, Hydro-Guard

[Øyvind]

Hello Thijmen,

First of all, welcome to the Nordic universe! Before starting I hope you are familiar with our academy.nordicsemi.com and docs.nordicsemi.com. In regards to the hardware design guidelines for the nRF9151, have a look at this link to ensure proper design around the solar panel + battery. 

In regards to what you are trying to achieve, there are no specific audio samples for the nRF9151. There is support for nRF9151 i2s and nRF9151 PDM using e.g. inter ic sound (i2s) API, which you could try on your side. Have a look at the Asset Tracker Template which might be a good starting point, documentation found here. You could also use the HTTP Client Sample as a starting point. 

• nRF9151 application core has 256 KB RAM total
• The Modem library requires shared memory that must reside in the lower 128 KB of RAM
  • Have a look at Cellular iot fundamentals lesson 3 nrf-modem-library-api

If there is not enough memory on the nRF9151 in order to reach your goals, then I suggest adding e.g. nRF54LM20B which can support FFT (Fast Fourier Transform) through the Axon NPU.

Or wait for the upcoming nRF92 Series which also adds the Axon NPU, our dedicated AI accelerator core, designed to increase the speed and efficiency of TensorFlow Lite models at the edge. It delivers up to 15× higher energy efficiency and up to 15× faster inference compared to running LiteRT on the CPU, and adds new capabilities such as audio classification. You can for example integrate voice control capabilities directly into your application running on the nRF92 Series cellular module, saving power and reducing the need for a host processor. (Edge AI for cellular)

Kind regards,
Øyvind

[Thijmen de Vries]

Hi Øyvind,

Thank you so much for the warm welcome and for taking the time to answer in such detail. This really helps us get started on the right foot.

Our nRF9151 DK is on its way, and we will start this weekend. Following your advice, we will begin with the HTTPS Client sample to get LTE and uploads working, then add our existing I²S microphone through the Zephyr I²S API. The hardware design guidelines for the solar panel and battery are exactly what we needed for the next step, so thank you for pointing us to those as well.

We would be happy to share our measurements and results with you once we have them. It would be great to show what the nRF9151 can do for protecting nature areas.

Thanks again, and kind regards,
Thijmen