nRFready audio compression/transmission resource profiling

[Tom]

Hi there,

I have an nRFready Smart Remote 3 in front of me to evaluate for development of an upcoming product. I've had a look around the forums and documentation, but haven't found too much detail as yet. Is there any information available on resource usage on the nRF52 when using the audio streaming features? I imagine there's a bunch of DSP and compression that happens before transmission, and then the BLE payload is probably significant.

Is there any benchmarking for CPU/battery/etc when implementing the audio streaming sample project? If not can you point me to some info on a good way to do this benchmarking?

Thanks,

Tom

[Filip Kubicz]

Hi Tom,

First of all, thank you for interest in Smart Remote.

In Smart Remote 3 v1.1, after each finished audio transmission over BLE you will see a report:
- min/average/max CPU used by audio module
- min/average/max CPU used by codec
- min/average/max audio bitrate
- similar benchmark for Acoustic Noise Reduction (ANR) if it is enabled

The benchmarking is enabled by default, so you can see it if you flash precompiled firmware. You can see how the benchmarking is done in Source/Debug/m_audio_gauges.c. It is also easy to adapt the code to measure CPU usage of your custom audio algorithm. For reference, please have a look at m_audio_process() function in Source/Modules/m_audio.c.

EDIT: you can display total CPU load measured at 1 second interval if you enable CONFIG_PWR_MGMT_CPU_USAGE_MONITOR_ENABLED.

Good news for you is that in the new Smart Remote release, which is coming in a couple of weeks, firmware is going to be even more helpful in this matter. It has command-line interface with commands for checking audio subsystem settings and changing them in runtime. The benchmark framework counts lost frames (in case BLE bandwidth is not enough for audio) and lost buffers from microphone (in case the CPU is too busy to process audio). The framework also informs you what's the current and maximal usage of queues with audio data, so you can tune memory settings.

Testing of audio algorithms should be easier, too. New release comes with audio instrumentation framework which lets you capture or inject sound at any stage of DSP processing in Smart Remote.

I hope this helps.

Kind regards,
Filip

[Tom]

Great - thanks for the info - it looks like there's some good tools to look into. Just to make sure though - before I go and do the bringup to get the system running - there's no documentation already available that outlines the benchmarking under some standard use cases?

Thanks,

Tom

[Filip Kubicz]

It is best to benchmark in your specific use case.

I can give you some rough estimates in the default firmware configuration:
- ADPCM encoder CPU usage is ~3%
- SBC encoder in mSBC mode ~ 5% CPU
- Opus CELT VBR with 20ms audio frame:
- complexity 0: ~28% CPU
- complexity 4:  ~40% CPU
- complexity 10 ~75% CPU
- Opus SILK VBR with 20ms audio frame:
- complexity 0 ~49% CPU
- complexity 4 ~75% CPU

Opus has a higher CPU consumption than other codecs, however it gives you great sound quality with high compression ratio - which means you need less bandwidth on Bluetooth Low Energy and thus the overall power usage (CPU + radio) is lower. Opus CELT VBR is a good choice for a very power efficient application.

If you want to minimize CPU consumption for Opus, it's best to use GCC with LTO and O2 optimization level. Also, to use less CPU do not use --specs=nano.specs.

[Tom]

Ah great, just the kind of info I was looking for, thanks Filip!

Any info on RAM/ROM usage? Also if you have the information it'd be great to know more about the compressed data size (e.g. Kb/s of compressed audio), to get an idea how that compares with the BLE link transfer rate.

T.

[Filip Kubicz]

Memory consumption depends on codec configuration, frame length etc.

Some information about RAM usage:
Base stack usage for Smart Remote 3 is 2kB. It is enough for ADPCM.
BroadVoice needs 6kB additional stack.
Opus CELT uses 7kB for encoder state and 19.5kB of additional stack.
Opus SILK uses 11kB for encoder state and 25kB of additional stack.

For ROM usage I don't have the values at hand. You can look at the size of compiled firmware or study the map to check it.

As for bitrates that need to be transmitted:
- ADPCM 67kb/s
- BroadVoice 32 kb/s
- Opus - it all depends on configuration, but generally significantly less than the codecs mentioned above. VBR mode can be constrained not to surpass a certain value.

[Tom]

This is all really useful information. Thanks Filip. Is this documented somewhere? I haven't found too much info in the form of whitepapers/implementation considerations. The only thing I've dug up is the User Guide and the Firmware Documentation, but didn't see it in there.

Thanks again for having all the details,

Tom

[Filip Kubicz]

I think these details are not described in documentation simply because there too many factors that have impact. I suggest you to run the firmware, connect to logger and observe the benchmark values.

Also you will find a lot of information in the code. It's well structured and modular. First, have a look at configuration files (sr3_config_pca20023.h and sr3_config_pca63519.h - open in Keil to have a convenient feature option tree).