[nRF5340 Audio DK] Inquiry About CPU Load for CIS Bidirectional Streaming with USB Audio Source

[EvanKim]

Hello,

We are currently evaluating BLE Gateway (CIS, BIS) development using the nRF5340 Audio DK.

We would like to ask whether bidirectional CIS streaming is officially supported when using the USB audio source.

In the documentation below, we found a statement mentioning CPU load concerns for bidirectional streaming when USB audio source is used:
Nordic Semiconductor Documentation

"The audio source selection affects the firmware architecture and available features. USB audio source is limited to unidirectional streams due to CPU load considerations, while I2S supports bidirectional communication."

Could you please explain in more detail what kind of CPU load limitations or concerns are being referred to?

Also, are there any plans to improve or address these limitations in future software and/or hardware updates?

Currently, we have tested a bidirectional CIS stream configuration using two nRF5340 Audio DK boards configured as:

• CIS Gateway (client)
• CIS Headset (server)

The test results showed normal operation:

• Gateway: outputs a 1 kHz tone
• Headset: speaker-to-microphone loopback
• Gateway: records and confirms the received 1 kHz tone

We are currently using nRF Connect SDK v3.2.4.

We have additionally ordered one more nRF5340 Audio DK and plan to test a configuration with:

• 1 CIS Gateway (client)
• 2 CIS Headsets (servers)

Additionally, the Headset board currently uses the microphone as the audio input source.
Is it possible to configure the audio input source as a 3.5 mm line input instead?
We would like to perform a loopback test using 3.5 mm output and 3.5 mm input.

Thank you.

[Maria Gilje]

Hello, 

We would like to ask whether bidirectional CIS streaming is officially supported when using the USB audio source.

In the documentation below, we found a statement mentioning CPU load concerns for bidirectional streaming when USB audio source is used:
Nordic Semiconductor Documentation

"The audio source selection affects the firmware architecture and available features. USB audio source is limited to unidirectional streams due to CPU load considerations, while I2S supports bidirectional communication."

Could you please explain in more detail what kind of CPU load limitations or concerns are being referred to?

Bidirectional CIS with USB source is supported with some adjustments. USB supports 48kHz sample rate, which needs -- as a rule-of-thumb -- around 30% of the CPU to encode. Decoding takes around half of the encoding load (normally a bit more than half). If you are encoding at another sample rate than 48kHz, the sample rate converter is active, which takes up more of the CPU. Stereo encoding/decoding will also increase the CPU load. So, for example when using sample rate and encoding rate of 48kHz the CPU load for encoding and decoding is ~46% CPU load. For stereo this will exceed 90% CPU load (if both directions are stereo). 

Also, are there any plans to improve or address these limitations in future software and/or hardware updates?

These limitations are connected to the nRF5340 SoC capabilities. Future devices supporting LE Audio may have improvements here. If you have questions about future products, please contact your Regional Sales Director. 

Additionally, the Headset board currently uses the microphone as the audio input source.
Is it possible to configure the audio input source as a 3.5 mm line input instead?
We would like to perform a loopback test using 3.5 mm output and 3.5 mm input.

Yes, it is possible to modify the application such that the line-in is used as the input source for the headset. The application has code in place which sets up I2S as the source, so while there could be a bit of work to modify this, it should be possible to do so with the code base you have available. 

Thank you for your patience while I took some time to reply to this. My apologies for the long wait. 

Best regards,

Maria

[EvanKim]

Thank you for your reply.

However, there is one point that I do not fully understand, and I would like to ask a follow-up question.

1. Should we understand that there are currently no plans to officially support bidirectional stereo streaming with a USB audio source on the nRF5340 platform?

2. You explained that in the case of bidirectional stereo streaming, the CPU load exceeds 90% due to both encoding and decoding processes. You also mentioned that this limitation is related to the performance capabilities of the nRF5340 SoC (presumably the application processor).

However, it seems that the CPU load caused by encoding and decoding should apply to both USB audio sources and I2S audio sources, as illustrated in the figure below. If that is the case, does it mean that bidirectional stereo streaming over I2S would also be unstable due to the same CPU limitations?

As mentioned earlier, the official documentation states that I2S supports bidirectional communication.

Could you please clarify why bidirectional communication is considered reliable when using I2S, while it is considered limited or unstable when using a USB audio source?

Is the difference caused by additional CPU overhead introduced by the USB audio stack, buffering, or sample rate conversion? If so, could you provide more details on where the extra processing load comes from and how significant it is compared to the I2S case?

Thank you for your support.

[Maria Gilje]

Hello, 

EvanKim said:

1. Should we understand that there are currently no plans to officially support bidirectional stereo streaming with a USB audio source on the nRF5340 platform?

It is supported by HW, given needed optimization in SW, but not included as a configuration in the default version of the Audio application. I am unfortunately not able to discuss future plans for what the Audio application includes. 

EvanKim said:

2. You explained that in the case of bidirectional stereo streaming, the CPU load exceeds 90% due to both encoding and decoding processes. You also mentioned that this limitation is related to the performance capabilities of the nRF5340 SoC (presumably the application processor).

However, it seems that the CPU load caused by encoding and decoding should apply to both USB audio sources and I2S audio sources, as illustrated in the figure below. If that is the case, does it mean that bidirectional stereo streaming over I2S would also be unstable due to the same CPU limitations?

I2S as source is a bit more flexible due to supporting lower sampling frequencies. Using a sample rate of 24kHz or 16kHz reduces the encoding time and therefore the CPU load as well. In the Audio application, 24kHz is the default sample rate when bidirectional streams are enabled. So this is why bidirectional streams with I2S source is stable.

I don't have estimated CPU load values for 24kHz and 16kHz at this point. 

EvanKim said:

Is the difference caused by additional CPU overhead introduced by the USB audio stack, buffering, or sample rate conversion? If so, could you provide more details on where the extra processing load comes from and how significant it is compared to the I2S case?

You can reduce the sample rate for the USB source mode, but note that this will activate the sample rate converter which needs some processing power. See i.e. the following commit in a forked sdk-nrf repo: https://github.com/rick1082/sdk-nrf/commit/aa5c816f842bed449641a9fbf169b2651378ee36. There, the sample rate for bidirectional stream is reduced to 16kHz, which solved an issue where the CPU load was too high to process streams with higher sample rate in a modified Audio application. Note that the symbol CONFIG_BT_AUDIO_PREF_SAMPLE_RATE_16KHZ is equivalent to CONFIG_BT_AUDIO_PREF_SOURCE_SAMPLE_RATE_16KHZ in nRF Connect SDK v3.x.x. 

Best regards,

Maria