OPUS decoder

[Michal]

Hi, I was recently asking about OPUS - https://devzone.nordicsemi.com/f/nordic-q-a/32875/how-compile-opus/128181#128181 . Now I am able to run encode on audio data and decode it back successfully. The problem is when I take .opus file encoded on PC. I have 2 issues:

1. OPUS is using 2 codecs - SILK and CELT and when encoding. And with opustools I used on PC there is no switch to use either SILK or CELT - it uses both of them. But in opus config file in SR the hybrid mode is described as not supported. I wonder what is the reason for that and If that reason is also a case for decoding?

2. OPUS files uses specific header to specify encode parameters. I saw that in SR there is no header created and raw OPUS data is send -  is that right? Also can you please share some information , human readable desription of header format? I started to read OPUS format specifciation but there are few hundreds of pages and I need to know if we can use OPUS for yesteraday ..

Thank you in advance.

[Stian Røed Hafskjold]

Hello,

1. Hybrid mode is not supported because then you have to compile in both SILK and CELT which would make the code too big. Also, SILK is using quite a lot of CPU, so we decided to support only either SILK or CELT. So you would need to find a PC tool which supports selecting only one of them.
2. SR3 is sending only raw compressed data, you don't need any header to decode the data. http://opus-codec.org/docs/opus_api-1.2/group__opus__decoder.html#ga7d1111f64c36027ddcb81799df9b3fc9

[Stian Røed Hafskjold]

Just to make things clear:

1. You can encode A on nRF
2. You can decode A on nRF
3. You are not able to decode A on a PC
4. You can encode B on a PC
5. You are not able to decode B on nRF

Is this correct?

[Stian Røed Hafskjold]

The smart remote team is going to look into the possibility of adding downstream audio on the SR3 (decoding). So we can provide you with some more details within a couple of days.

[Filip Kubicz]

Hi,

1. As Stian said - Opus has 2 algorithms, CELT and SILK. Both of them simultaneously will not fit into the nRF52832 memory, so in Smart Remote you can choose at compile time which one you want to use. The hybrid mode is available to use if only you have enough flash and RAM memory.

2. The Opus decoder does not need additional information to know which algorithm was used to encode the current frame. In the original Opus code (https://opus-codec.org/) there is a function opus_packet_get_mode() which checks if the frame was encoded with CELT or SILK. 

You can decode the data encoded on PC - if you open the .opus file and send it frame by frame to nRF52 it is possible to use Opus decoder as provided in Smart Remote code (there is not support for this scenario yet).

Host decoding:
There is the NVS (Nordic Voice Solution) package which is responsible for decoding sound on the host side (Linux). It can be found in downloads section:
https://www.nordicsemi.com/eng/Products/Bluetooth-low-energy/nRFready-Smart-Remote-3-for-nRF52-Series
When you download "Host software", you can see in the source code how the Opus decoder is used to decode sound received from nRF52 / Smart Remote.

Embedded decoding on nRF52:
There is work ongoing to provide downstream audio support.

EDIT: There was some confusion on which header we are talking about:
- Opus file header with encode parameters
- the optional header added to each frame by Smart Remote to be able to easily reassembly sound on host side (in NVS) when streaming. Irrelevant to this question.

[Filip Kubicz]

Please see updated answer.