Any ADPCM / Opus example available?

RE: Any ADPCM / Opus example available?

Einar Thorsrud — Fri, 15 Feb 2019 08:10:42 GMT

1. The linker is complaining that the binary is too large for your Keil license. The SR3 firmware is quite large, so there is no way it can be built with the free Keil license (32 kB limitation).

2. The SR3 firmware use a modified SDK, so there are some differences. The latest is based on SDK 14.2.

RE: Any ADPCM / Opus example available?

Matthew K — Fri, 15 Feb 2019 07:12:31 GMT

Thanks, Einar. Before closing this thread, may I add 2 questions, please?

I just installed the EXE file you've mentioned.

1. Error Message

63519-SR3_nRF52832_Shield.axf: error: L6050U

has appeared.I never saw something like this; can you help me to solve this, please?

2. Is this Firmware example use the same nRF52 SDK 15.2 as well or it is a different SDK?

-Thanks!

RE: Any ADPCM / Opus example available?

Einar Thorsrud — Thu, 14 Feb 2019 09:10:46 GMT

Hi,

You need to download the "Desktop Software" file. Then extract it on a windows computer, and run the installer (nRFready Smart Remote 3 nRF52 v121alpha.exe). Then you will get the firmware installed under C:\Nordic Semiconductor\nRFready Smart Remote 3 nRF52 v1.2.1alpha unless you select a different location. You can always copy that folder and move the source code to the mac of another machine without running the installer again.

You will find the Smart remote projects under nRF5x SDK-SR3\examples\ble_peripheral\smart_remote_3_nrf52\Projects\. Note that there is no SES project, but there are projects for keil. In any case this is the full SR3 reference design, so you will probably want to use it for inspiration and extract what you need rather than using the provided projects. Unfortunately, that is up to you, as we do not provide any other examples.

RE: Any ADPCM / Opus example available?

Matthew K — Wed, 13 Feb 2019 17:52:40 GMT

Hello, Einar.

[quote userid="7377" url="~/f/nordic-q-a/42875/any-adpcm-opus-example-available/170800"]The firmware implementation is available here (just download the zip, unzip it and run the installer on a windows computer. [/quote]

Are you mentioning about the "nRF6939-FW"? I'm using a MacBook (OS version 10.14.3) and a Windows 10 laptop, by the way.

I was expecting typical SEGGER ES projects or Keil Projects and C source files, I noticed 2 files; binaries & sources.

[quote userid="7377" url="~/f/nordic-q-a/42875/any-adpcm-opus-example-available/170800"]To look at the audio implementation you can start in the m_audio.c file and see how the audio input system is implemented, using I2S and Opus (optionally).[/quote]

Then, do I need to type, from the amd64 directory,

$ sudo dpkg -i *.deb

on my MacBook to see the "m_audio.c" file?

How can I install this on Windows 10, then?

-Thanks for your help

RE: Any ADPCM / Opus example available?

Einar Thorsrud — Wed, 13 Feb 2019 09:33:06 GMT

Hi,

The Smart Remote reference design has most of what you ask for: A example which use Opus and I2S microphones. It is for the nRF52832, but it is equally valid for the nRF52840. Unfortunately we do not have any example in the SDK itself, nor an example which is intended for the nRF52840 DK, but you should be able to use the Smart Remote implementation as a reference.

You can see the documentation for the SR3 her. The firmware implementation is available here (just download the zip, unzip it and run the installer on a windows computer. To look at the audio implementation you can start in the m_audio.c file and see how the audio input system is implemented, using I2S and Opus (optionally). Unfortunately it is quite complex as it is part of a full smart remote control reference design, but it is the only such example we provide.

RE: Any ADPCM / Opus example available?

Matthew K — Tue, 12 Feb 2019 22:54:40 GMT

Hello, Einar. I want to see an OPUS example which uses I2S and the nRF52840 DK.

I’m currently using Keil MDK 5 and SEGGER ES 3. Can I get an example C code of this?

The Smart Remote 3 firmware (nRF6939-FW) has multiple TGZ files and I cannot find anything related to OPUS.

As a result, I wish to get an example C code which uses OPUS with I2S.

It will be great if that example can be compiled using Keil or SEGGER ES.

Will this be available?

-Thanks!

RE: Any ADPCM / Opus example available?

Einar Thorsrud — Tue, 12 Feb 2019 10:34:33 GMT

Hi Matthew K,

You have gotten excellent answers from wpaul, so there is not much to add. I did not fully understand what you would like a comment on from Nordic, but what I can say is that we have some reference designs using both ADPCM and OPUS, which you can refer to. The nRFready Smart Remote 3 for nRF52 Series support both, and the Thingy:52 firmware use ADPCM. In both cases it is used to compress microphone data.

Please let me know if you have more specific questions you want me to comment on.

RE: Any ADPCM / Opus example available?

wpaul — Sat, 02 Feb 2019 23:57:46 GMT

When you say "external stereo" you mean what amplifier, right? I think we settled on the Texas Instruments LM4880. We're making a small board, so we're trying to use small speakers (Dayton Audio CM20-14M-8).

Unfortunately we're probably going to have to get a prototype board fabricated for testing rather than breadboard the amplifier since most parts are SMT only.

Right now for my test setup, I have the CS4344 breakout board stuffed into one of the headers on the nRF52840 DK board, and I have my computer speakers plugged into the jack so I can hear properly. It's messy, but at least I can tell that I've got my IS2 code working right.

RE: Any ADPCM / Opus example available?

Matthew K — Sat, 02 Feb 2019 23:44:32 GMT

...

Now I remember. I saw CS4344 on STM32 Disco boards.

If it is fine to reveal it, can I ask what external stereo you will use?

[quote userid="75667" url="~/f/nordic-q-a/42875/any-adpcm-opus-example-available/167268"] while the MAX98357A is monaural, it has a built-in amplifier. [/quote]

For my case, I will only use one 3W speaker for the prototype. In this case, I won't need a stereo codec like you, right?

Or do you have any recommendation for alternative I2S amps to play songs which can be mounted on breadboards? MAX98357A was the only I could find from online.

I'm waiting for the Nordic team's reply about ADPCM / Opus. Thanks for helping me while waiting for their answer.

-Thanks!

RE: Any ADPCM / Opus example available?

wpaul — Wed, 23 Jan 2019 07:12:00 GMT

Look at the I2S section of the nRF52840 product specification document. You need to configure two things: the MCLK frequency, and the left/right clock (LRCLK) divisor, which yields the LRCLK frequency. Typically both the MCLK and LRCLK must be supplied to the I2S codec chip. (It looks like the MAX98357A doesn't need MCLK, but the CS4344 does.) Nordic does not let you choose arbitrary values for these things: you get to choose from a specific set of MCLK and LRCLK settings. There's some tables which show you some possible values. I chose values that yielded 15.625KHz for the LRCLK:

MCLK == 32MHz divided by 8 == 4MHz

LRCLK divisor == 256

4000000 / 256 == 15625

This effectively also sets the audio sample rate, so when you encode your audio samples, you have to do it at this same rate in order for them to play correctly.

You can choose different values for a higher sample rate -- the higher the sample rate, the higher the maximum audio frequency you can sample. The max audio frequency is usually half the sample rate, so with 15.625KHz sample rate, the audio frequency range is 7812.5Hz. This is more than good enough for voice and 'ok' for music, though for music content you usually want 20KHz as the top end frequency. This is why 44.1KHz is used for sampling high quality audio (44.1KHz is basically two times 20KHz).

A higher sample rate means you end up with more samples per second though so you end up with bigger files. I chose 15.625KHz as a good compromise between audio quality and file size for my application.

The reason I used the CS4344 is because it's a stereo codec (it has two audio outputs, for left and right channel). However while the MAX98357A is monaural, it has a built-in amplifier. (My friend and I are planning to use an external stereo amplifier with the CS4344.) The I2S controller in the nRF52 supports both mono and stereo modes. With stereo, you have twice the audio samples, interleaved. The first sample is for the left channel and the second is for the right. The left and right channels are automatically routed to the right output of the codec chip by the hardware.

You can see the sample driver code that I wrote here:

https://github.com/netik/dc27_badge/blob/master/software/firmware/badge_840/i2s_lld.c

I used a couple of dirty tricks to make it work the way I wanted for my application, but it basically follows the same setup steps as documented in the nRF52840 manual.

The nRF52840 doesn't have any hardware tricks in it for handling ADPCM (as far as I know) so any work Nordic would do here would likely involve porting an ADPCM decoder library so that it builds with the SDK. ADPCM is probably simpler to implement than Opus meaning it would require less processing overhead, but I think Opus gives you better compression.

RE: Any ADPCM / Opus example available?

Matthew K — Wed, 23 Jan 2019 06:46:08 GMT

Thanks for your kind reply!

0) I apologize for my ignorance in audio;

2^16 = 65,536, 2^15 = 32,768, but where did the15625 came from?

1) You’ve mentioned bout CS4344 to hear the sound.

Does this mean when I use MAX98357A, this cannot make a sound like CS4344?

What is the key feature that CS4344 has compared to other products?

Thanks a million.

I did hear that the Nordic team is focusing on ADPCM on the other post so I was curious what will the Nordic team will answer.

RE: Any ADPCM / Opus example available?

wpaul — Tue, 22 Jan 2019 22:24:40 GMT

0) I didn't see any.

1) I didn't see any.

2) ADPCM, Opus and MP3 are all different audio encoding/compression algorithms. So no, if you have software to decode ADPCM and/or Opus, it will will not also decode MP3.

Note that MP3 was considered proprietary but I think the patents all expired in 2017 so there may no longer be any strings attached.

Regardless of which codec or format you use, the idea is that you'll need software to decode the data back into raw audio samples and then use the I2S block to play them. (You also will need to connect an I2S chip like the Cirrus Logic CS4344 in order to hear the sound.) The trick will be a) getting the codec library code to compile into your nRF52 project, and b) getting it to decode the audio data and feed it to the I2S controller fast enough so there's no gaps in the playback. You'll basically be doing:

- read a chunk (from SD card)

- decode a chunk

- play a chunk

If the "decode a chunk" step is too slow, your playback will be choppy.

I've experimented with the nRF52 I2S block and gotten it to work, but I decided to just convert all my audio files to raw I2S samples rather than try to put any decompression or decoding software on the nRF52 itself. I used the SOX audio tool for this, using a command like the following:

% sox --encoding signed-integer originalfile.mp3 outputfile.raw channels 2 rate 15625

This is obviously for stereo mode. I chose a sample rate of 15.625KHz because it yielded acceptable audio quality and was a rate that was easily selectable using the MCLK frequencies and LRCLK divisors that Nordic saw fit to provide in their I2S implementation. Basically, this produces 16-bit sample data, with 15625 samples per second. Since this is stereo that means there's 2 values per sample interval. You can estimate the file size from this. For example, say you have 20 seconds of audio:

15625 x 2 bytes per sample x 2 channels = 62500 bytes per second

62500 bytes per second x 20 seconds = 1250000 bytes

So your 20-second speech clip would be 1.25MB in uncompressed form. If you only want mono audio, it would be half that (625KB).

You say you plan to have about 200 files. Worst case, at 1.25MB per file, that's about 250MB. SD cards these days are often 8 or 16GB, so space should not be a problem. That's probably a lot for QSPI though (last time I looked, the largest chip I could find was 256MB).

In any case, using an uncompressed file might be handy at least for testing, so that you can figure out how to get audio from the SD card to the I2S controller. After that you can figure out how to incorporate a compression/decompression library.