nRF5340 LC3 decoder question

[gfgf2400a]

Hi,

I am evaluating nRF5340 audio dk with CS47L63 DSP. My goal is to find out the MCU processing consumption for LC3 codec what is MCU processing power left for the application.

So far I found

1. https://devzone.nordicsemi.com/f/nordic-q-a/87822/ble-audio---many-to-one-power-consumptions-of-dk/367897 The rule of thumb is that the LC3 codec produces approximately ~15% CPU load when decoding and ~30% CPU load when encoding a 96 kbps stream, so you may indeed decode multiple incoming streams on your nRF5340 Audio DK.

2. My actual benchmark from the nRF5340 audio software for gateway and headset are 13% for decoding and ~48% for encoding.

3. In developer.nordicsemi.com/.../README.html, "udio_datapath.c module sends the compressed audio data to the LC3 audio decoder for decoding".

My question is why only use CS47L63 DSP for decoding but not encoding? Do you have any plan to use CS47L63 DSP for encoding in the near future?

Thanks

[gfgf2400a]

For the LC3 encoding, would LC3EncodeSessionData() execution is on CS47L63 DSP? If that is the case, why the encoding take so much more execution cycles? thanks 

[Karl Ylvisaker]

Hello Dennis,

I think there has been a misunderstanding here about the purpose of the CS47L63 hardware codec and the LC3 software codec. I concur that using the term 'codec' for both is not ideal, so I fully understand this confusion since there is overlap between the terminology here.
All codec, encode/decode, mentions are usually in relation to the LC3 Software codec, unless we specify that it is in relation to the hardware codec or CS47L63.
The CS47L63 is actually not used for any signal processing as it is right now - it is really only being used for the DAC, as it receives the I2S digital output from the nRF5340, and converts it to analogue output for the HEADPHONE jack.
The CS47L63 is also able to do DSP on the received audio data, but this feature is not being used on the nRF5340 Audio DK.

My question is why only use CS47L63 DSP for decoding but not encoding? Do you have any plan to use CS47L63 DSP for encoding in the near future?

All encoding and decoding (apart from the final DAC part) happens in the LC3 software codec. The processing required to do this is too intensive to be run on the CS47L63 alone, and so there is no plans to move any part of it to the CS47L63 in the future. The only changes to the usage of the CS47L63 that could be considered right now is to let it do some DSP on the signal before outputting it, but that would be the extent of it - and I have not heard any plans for this specifically, or heard that it is on the roadmap as of right now, at least.
If you have specific questions about the roadmap for the nRF5340 Audio application, I would have to ask that you direct these to your Regional Sales Director (RSD), since we do not discuss future releases here in DevZone.

2. My actual benchmark from the nRF5340 audio software for gateway and headset are 13% for decoding and ~48% for encoding.

Could you elaborate on your test setup here - which application were you running, and in what configuration, when you measured the ~48% CPU usage for encoding?
This is much higher than our benchmarking of the encoding, and so I suspect that something might have gone wrong.
Please note that the 30% CPU usage rule of thumb only applies to the encoding of a stream, so anything else you are doing in your application will add on to the overall CPU utilization.

Best regards,
Karl

[gfgf2400a]

Hi Karl,

Karl Ylvisaker said:

The CS47L63 is actually not used for any signal processing as it is right now - it is really only being used for the DAC, as it receives the I2S digital output from the nRF5340, and converts it to analogue output for the HEADPHONE jack.
The CS47L63 is also able to do DSP on the received audio data, but this feature is not being used on the nRF5340 Audio DK

That's good to know. I would thought the DSP can do more of the heavy lifting for the signal processing. We can look into this further.

Karl Ylvisaker said:

Could you elaborate on your test setup here - which application were you running, and in what configuration, when you measured the ~48% CPU usage for encoding?

First, I used nrf5340 audio DK template to create the gateway code. Gateway source is I2S, LC3 10ms frame. No other configuration changed. No code change. Laptop audio to gateway line-in connector. I have nrf5340 audio DK template to create the headset code. I was able to hear audio from the headphone jack. On the gateway, I used Zephyr "kernel threads" command to display the total execution cycles. The encoder thread actual shows 52%.

0x200016a0 ENCODER
options: 0x0, priority: 3 timeout: 319045
state: pending, entry: 0x3839
Total execution cycles: 13680764 (52 %)
stack size 6752, unused 588, usage 6164 / 6752 (91 %)

Second, I added k_uptime_delta() for sw_codec_encode() in sw_codec_select.c. For 10ms frame data, k_uptime_delta() reported ~4.8ms. This is where I said ~48% CPU usage for encoding. This number aligned with the 52% total execution cycles from ENCODER thread in Zephyr.

Thanks

[Karl Ylvisaker]

Hello again Dennis,

Thank you for your patience with this.

gfgf2400a said:

That's good to know. I would thought the DSP can do more of the heavy lifting for the signal processing. We can look into this further.

Yes - I have not done much work the CS47L63 myself, but from the datasheet it seems that it can do quite some audio enhancement, if enable.

gfgf2400a said:

This number aligned with the 52% total execution cycles from ENCODER thread in Zephyr.

The default application is the stereo CIS mode, which means that it will be encoding two streams, which should come out at around ~60% CPU usage, which is close to what you are seeing.
If you could like to check the LC3 encoder CPU usage for 1 stream you can either disable one of the CIS streams, or set up a BIS instead - the BIS will require the same amount of work from the LC3, but it default to a single stream being broadcasted.

Would you give this another try for the BIS, and see if it results in roughly ~30% usage for the encoder thread? You can change the gateway into BIS by adding the CONFIG_TRANSPORT_BIS=y Kconfig into your prj.conf file before doing a pristine build.

Best regards,
Karl