Simultaneous DSP and LE Audio transmission on nrf5340

[npwalker]

My desired work flow is the following.

Input: Two electret microphones -> Audio ADC -> I2S Stereo -> nrf5340

Processing: Buffers (10-50 bytes) channel 2 to create delay, subtracts delayed channel 2 from channel 1, and outputs single-channel signal. This is a simple differential beamformer and needs to run at the ADC sampling rate of 44.1 kHz.

Output (two user options):

1. nrf5340 -> LE Audio -> a second nrf5340 -> USB audio class -> USB-C
2. nrf5340 -> I2S mono -> Audio DAC -> 3.5 mm audio jack

Can the nrf5340 achieve this workflow without additional MPU or ICs (except the ADC and DAC already mentioned)? Specifically, can I implement this signal processing algorithm and run it simultaneously to LE Audio transmission? If so, can you recommend resources for utilizing the dual-core chip to achieve this? 

Thank you in advance!

[Samruddhi Jadhav]

Hello,

I have discussed this internally with the team and have some feedback here from them. From your description here, it doesn't sound too demanding CPU usage wise so we think you could manage without a DSP hardware codec or anything like that. So you should use the nRF5340. However, please note that, once you decide to implement an audio feature, you end up wanting to do a lot of other things too and these functions can be quite CPU-demanding. The Cirrus Logic chip on our audio dk is extremely powerful and it's a good starting point. We do not have any specific recommendations but I would suggest you to contact the regional sales manager of your area for further information about this. 

To build further on the first reply in this case, "output 1" problem is valid that it needs a USB host but phones can operate as that. When it comes to "output 2", we think it should work fine as long as the same sample rate is used just like with a walkie-talkie. 

Please let us know if you have any further questions. 

Best Regards,

Samruddhi

[npwalker]

Let me make sure I understand. I could just have my microphones input to nRF5340 and do the limited amount of processing I mentioned. However, you are saying that I will likely want the CS47L63 as an interface since it is much more feature rich for audio processing (EQ and filtering for example). 

Also, I've updated my architecture slightly. I'm now using two digital MEMS microphones with PDM output like Figure 7. 

So to comply with your advice, I should have the following:

Input: 2 DMICs -> CS47L63 (PDM pins Figure 2-1) -> nRF5340

Any heavy processing happens at the CS47L63 and my beamforming algorithm happens at the nRF5340?

Then my outputs 2 returns nRF5340 -> CS47L63 -> headphone jack

In this way, wouldn't a single I2S interface between CS47L63 and nRF5340 would need to handle input and output? Is that feasible?

I have purchased a nRF5340 Audio DK. I'll be using/testing the nRF Connect SDK for VS Code and the nrf_audio application. I assume that the walkie-talkie demo will help me get going? Any advice on where to start modifying this to implement my beamforming algo? 

Thanks!

[Maria Gilje]

Hello,

npwalker said:

Any heavy processing happens at the CS47L63 and my beamforming algorithm happens at the nRF5340?

Correct. 

npwalker said:

In this way, wouldn't a single I2S interface between CS47L63 and nRF5340 would need to handle input and output? Is that feasible?

The I2S peripheral on the nRF5340 can handle simultaneous input and output. This seems to also be the case for the CS47L63. 

npwalker said:

I have purchased a nRF5340 Audio DK. I'll be using/testing the nRF Connect SDK for VS Code and the nrf_audio application. I assume that the walkie-talkie demo will help me get going? Any advice on where to start modifying this to implement my beamforming algo? 

Do you have one or several Audio DKs? The full functionality of the walkie-talkie demo is enabled with two DKs running compatible walkie-talkie firmware (one client and one server). Functionality-wise, the demo has one PDM microphone input on each DK and has a bidirectional CIS connection between the two DKs. 

Regarding modifications, you would at the very least need to catch the microphone data to do the processing. All audio related source code is found in nrf_audio\src\audio. If you add a DMIC to a walkie-talkie device you would also have to make sure that the data from both microphones are picked up accordingly. At this point I am not sure if this is only on the CS47L63 side or in the appliation firmware somewhere, though. 

I hope you'll be able to get something up and running from this. Please let us know if you have further questions or if you run into issues, and we'll help out. 

Best regards,

Maria