Nrf5340 audio with sensor data collection.

[Adarsh_1]

Hallo Team,

We are planning to use nrf5340 for one of our project which requires low energy audio and sensor data transfer. Can you please evaluate if our use case is possible with existing / upcoming revision of the nrf connect SDK?

I have already tested nrf5340 audio DK. Right now I am trying to connect an external hardware audio codec to reproduce stereo audio. We need a common headphone solution. (Microphone input is also required, but however I understand that it takes some time and cannot be implemented with the existing nrf connect release ncs v 2.0.2). 

We are designing a headphone which should be able to communicate with multiple peripheral sensors apart from the smart phone (These are standalone ble sensors which transfer sensor data at 64 Hz. I am aware that we have to wait for android 13 release or higher for LE audio streaming)  So apart from audio streaming, the headphone should also be able to collect sensor data from four peripheral ble sensors. There should be separate service and characteristics available to transfer the collected sensor data to mobile phone along with audio streaming.

So we need a solution where our headphone should acts as both central (where it collects sensor data from 4 peripheral ble sensors) and peripheral (connected to mobile phone for audio streaming). The sensor data should also be transferred to mobile device along with audio streaming. Is this supported at the moment? Can this be accomplished?  

Would it be possible for an android device to act as a central device where it can collect data from four ble sensors and stream audio to the headset at the same time? Then there is no need to have extra complication (central and peripheral role) at the headset end. Would this be possible?

I would highly appreciate your suggestions and solutions.

Thank you for your time.

Best regards,

Adarsh

[Karl Ylvisaker]

Hello again, Adarsh

Adarsh_1 said:

However, here I am facing difficulties. How can i modify the "le_audio_send" function to to send stereo data?  Should I create a buffer here to load the audio samples? Or can I use the iso_stream_send function to send stereo audio without splitting the data?

Can you show an example / code snippets on how this can be achieved? 

Are you still facing difficulties with this? You should here modify the le_audio_send function so that it sends stereo data (by changing its current behavior, which is to split the data into two streams).

Adarsh_1 said:

While working with the steps you provided I found out that the sdk no more supports SBC codec. 

This is only the case for the main branch - it is not yet effective for the v2.0.2 nRF Connect SDK that you are working with.

Adarsh_1 said:

I do not intend to use LC3 at the moment. And need to test with SBC alone due to internal reasons. 

I understand. If you stick to using the v2.0.2 you will still be able to use the SBC, no problem.
If you are to upgrade to future releases you will need to add the non-LC3 codec yourself, in the case that you still will not be working with the LC3.
Please know that we will soon be making changes to the LC3 agreement, so that the evaluation agreement no longer is necessary in order to be able to evaluate the LC3 (I mention this in case this is relevant for your reasoning to keep with the SBC codec).

Adarsh_1 said:

So is it possible to work with stereo audio using just the SBC codec? Can I perform changes in the ncsV2.0.0 Tag to let the gateway send stereo in a single CIS using SBC codec?

The CIS does not check the contents of the stream - it does not know if the stream is encoded using LC3, SBC, or any other codec, so long as it fits into the Isochronous channel, with size being the main parameter.

Best regards,
Karl

[Adarsh_1]

Hai Karl,

Karl Ylvisaker said:

Are you still facing difficulties with this? You should here modify the le_audio_send function so that it sends stereo data (by changing its current behavior, which is to split the data into two streams).

Yes. I have not yet made it to work. I should have mentioned about the use of SBC before. Anyway now you know that I am working with NCS V2.0.0 and testing with SBC at the moment. I can see that the master branch has added and changed few files. 

I was not able to find an equivalent function for le_audio_send in the NCS V2.0.0. How do I proceed from here? I am not able to get my head arround this properly.

Karl Ylvisaker said:

2.You will also have to modify the gateway behavior, so that it does not split up the data into two separate streams in the le_audio_send function.

Where are the equivalent codes / files situvated in NCS V2.0.0. I am sorry I was not able to find it.

Can you provide a bit more information here?

Thank you for your time.

Best regards,

Adarsh

[Adarsh_1]

Hai Karl, 

Karl Ylvisaker said:

1.In audio_system.c you will need to change the decode type from SW_CODEC_MONO to SW_CODEC_STEREO, which will make the headset device expect to receive a stereo stream.

Just like the steps you mentioned for the main branch, can I have the steps for V2.0.0 / V2.0.2 modify the gateway code to send stereo audio so that it does not split data into two different stream? 

This is where I am finding it dificult. Where exactly is the streams been split to left and right in the gateway code. For the past 1 week, I am looking in to the "bluetooth" folder inside the "src", but I cannot figure it out. 

Can you please explain me how to/ where to modify the code in NCS V2.0.0/V2.0.2 so that the gateway sends stereo stream rather than two separate streams for left and right?

Karl Ylvisaker said:

2.You will also have to modify the gateway behavior, so that it does not split up the data into two separate streams in the le_audio_send function.

This is not helping, since I am using NCS V2.0.0. 

Thank you for your time. Any suggestions will be highly appreciated.

Best regards,

Adarsh  

[Adarsh_1]

Hy Karl,

Patiently waiting for your reply.

best regards,

Adarsh

[Karl Ylvisaker]

Hello Adarsh,

Thank you for your extreme patience with this. I have been out of office for some time, but now I am back.

Do you still require technical support with this issue?

Best regards,
Karl