Connecting nRF5340 LE Audio to other LE Audio equipment

[hakved_vea]

Hi,

I'm trying to connect the nRF5340 LE Audio example to another LE Audio equipment: VOCE by Nexum (https://www.nexum-design.com/) 

I asked Nexum if this was possible and got this reply:

Hi, This is totally ok if you set VOCE as Auracast TX mode. but please note, the standard sample rate of Auracast is 24K sample rate. And VOCE now is set by 48Kh. We will upgrade our VOCE to support 16 / 24Khz by next upgrade. now, you need to make sure nRF5340 as Auracast RX device can also support 16bit/48K sample rate. BR, NEXUM

I also opened a ticket on DevZone earlier but thought this got a little off topic, 

 Building nRF5340_audio example for nRF5340-DK (not NRF5340-AUDIO-DK) 

I was able to build the example and connect two nRF5340-Audio-DK kits. 

Do you have any advice for how to attack this?

I'm currently trying out different config parameters in addition to adding LOG_INF(...) to see if nRF5340 sees the VOCE device. But i have't got any closer..

Thanks in advance for any reply on this,

Best,

Håkon 

[hakved_vea]

Hi Dejan,
thanks for your reply,

I would like VOCE to be connected to a microphone and transmit on a broadcast stream, and then setting up multiple nRF5340 as receivers and connect them to speakers.

Initially I want to connect the two devices in any way so I can see that they are compatible, then I can use that as a baseline for further development. So if you think switching roles would be easier I can try that first.  

BR,
Håkon

[dejans]

Hi Håkon,

Our nRF5340 audio in BIS mode supports 16 KHz/24 KHz/48 KHz and it is currently set in compile time. You could consider even replacing VOCE with nRF5340 audio device. This can be done since nRF5340 audio can be set as BIS gateway using microphone as audio source.

Best regards,
Dejan

[dejans]

Hi Håkon,

hakved_vea said:

Could you provide me with the configurations that might affect the connection of the two devices?

You could start by looking at nrf5340_audio/src/bluetooth/Kconfig and Kconfig.defaults.

hakved_vea said:

Is it somewhere in the code that i can print out BLE devices that the nRF5340 sees?

When using nRF5340 as a receiver of audio broadcast, it is le_audio_bis_headset.c that is going to be used. Printing of devices is possible from scan_recv_cb().

hakved_vea said:

Have you at Nordic connected a nRF5340 in BIS mode to any other equipment before?

Your local regional sales manager might be able to provide you this information.

Best regards,
Dejan

[hakved_vea]

Hi Dejan,

Thanks for your reply, I have been testing and reading about LE Audio, so sorry for late replay.

I managed to connect to VOCE with the ISO Receive example from zephyr, as I see it, it does not implement any audio output.

The two examples are quite different ISO Receive and nRF5340_Audio example, can you help me in any way to merge the two examples?

Best regards,

Håkon

[hakved_vea]

Hi again,

I have investigated the nRF5340_Audio example code and found:

"C:\ncs\v2.2.0\zephyr\subsys\bluetooth\audio\broadcast_sink.c" contains most of the code present in the ISO Receive example. It seems that the example is adding a broadcast id in the extended advertisement data 

I comment out the validation check for this ID and the advertisement data was passed forward to le_audio_bis_headset.c scan_recv_cb callback. This checks for device name peer, that needs to be equal to CONFIG_BT_DEVICE_NAME. So the example seems to be proprietary to nRF5340_Audio_DK boards.

I comment out the device name check and was able to establish a connection with the VOCE device. However, the application got an ASSERTION FAIL

Below is the dump from the debug terminal:

*** Booting Zephyr OS build v3.2.99-ncs1 ***
HL [00:00:00.258,300] [1B][0m<inf> fw_info: [1B][0;32m
	 nRF5340 Audio nRF5340 Audio DK cpuapp 			    
	 NCS base version: 2.2.0 			    
	 Cmake run : Thu Jan 26 14:18:52 2023[1B][0m[1B][0m
HL [00:00:00.258,300] [1B][0m<inf> fw_info: ------- DEBUG BUILD -------[1B][0m
HL [00:00:00.258,331] [1B][0m<inf> fw_info: [1B][0;36mHEADSET left device[1B][0m[1B][0m
HL [00:00:00.268,920] [1B][0m<inf> board_version: [1B][0;32mCompatible board/HW version found: 1.0.0[1B][0m[1B][0m
HL [00:00:02.313,781] [1B][1;33m<wrn> bt_hci_core: Controller to host flow control not supported[1B][0m
HL [00:00:02.442,535] [1B][0m<inf> ble: MAC: EB:37:46:99:6E:86 (random)[1B][0m
HL [00:00:02.442,749] [1B][0m<inf> ble: Controller version: 3310[1B][0m
HL [00:00:02.537,475] [1B][0m<inf> bis_headset: Scanning for broadcast sources[1B][0m
HL [00:00:02.554,809] [1B][0m<inf> bis_headset: Broadcast source ªªªªªªªªªªªª[1D] found[1B][0m
HL [00:00:02.702,362] [1B][0m<inf> bis_headset: Codec config for LC3:[1B][0m
HL [00:00:02.702,362] [1B][0m<inf> bis_headset: 	Frequency: 48000 Hz[1B][0m
HL [00:00:02.702,392] [1B][0m<inf> bis_headset: 	Frame Duration: 10000 us[1B][0m
HL [00:00:02.702,392] [1B][0m<inf> bis_headset: 	Octets per frame: 100 (80000 kbps)[1B][0m
HL [00:00:02.702,392] [1B][0m<inf> bis_headset: 	Frames per SDU: 1[1B][0m
HL [00:00:02.702,392] [1B][0m<inf> bis_headset: 	Channel allocation: 0x0[1B][0m
HL [00:00:02.702,423] [1B][0m<inf> bis_headset: Codec config for LC3:[1B][0m
HL [00:00:02.702,423] [1B][0m<inf> bis_headset: 	Frequency: 48000 Hz[1B][0m
HL [00:00:02.702,453] [1B][0m<inf> bis_headset: 	Frame Duration: 10000 us[1B][0m
HL [00:00:02.702,453] [1B][0m<inf> bis_headset: 	Octets per frame: 100 (80000 kbps)[1B][0m
HL [00:00:02.702,453] [1B][0m<inf> bis_headset: 	Frames per SDU: 1[1B][0m
HL [00:00:02.702,453] [1B][0m<inf> bis_headset: 	Channel allocation: 0x0[1B][0m
HL [00:00:02.702,545] [1B][0m<inf> bis_headset: Syncing to broadcast stream index 0[1B][0m
HL [00:00:02.762,725] [1B][0m<inf> bis_headset: Stream index 0 started[1B][0m
HL [00:00:02.780,181] [1B][1;33m<wrn> audio_datapath: I2S RX overrun. Single msg[1B][0m
HL [00:00:07.772,247] [1B][1;33m<wrn> audio_datapath: In I2S TX underrun condition, total: 5000[1B][0m
ASSERTION FAIL [err == 0] @ WEST_TOPDIR/zephyr/subsys/bluetooth/host/hci_core.c:329
	k_sem_take failed with err -11
HL [00:00:12.772,338] [1B][1;33m<wrn> audio_datapath: In I2S TX underrun condition, total: 10000[1B][0m
HL [00:00:13.318,664] [1B][1;31m<err> os: r0/a1:  0x00000003  r1/a2:  0x00000000  r2/a3:  0x00000000[1B][0m
HL [00:00:13.318,664] [1B][1;31m<err> os: r3/a4:  0x200143f5 r12/ip:  0x00000000 r14/lr:  0x0001ca87[1B][0m
HL [00:00:13.318,664] [1B][1;31m<err> os:  xpsr:  0x61100000[1B][0m
HL [00:00:13.318,695] [1B][1;31m<err> os: s[ 0]:  0x00000000  s[ 1]:  0x00000000  s[ 2]:  0x00000000  s[ 3]:  0x00000000[1B][0m
HL [00:00:13.318,695] [1B][1;31m<err> os: s[ 4]:  0x00000000  s[ 5]:  0x00000000  s[ 6]:  0x00000000  s[ 7]:  0x00000000[1B][0m
HL [00:00:13.318,695] [1B][1;31m<err> os: s[ 8]:  0x00000000  s[ 9]:  0x00000000  s[10]:  0x00000000  s[11]:  0x00000000[1B][0m
HL [00:00:13.318,725] [1B][1;31m<err> os: s[12]:  0x00000000  s[13]:  0x00000000  s[14]:  0x00000000  s[15]:  0x00000000[1B][0m
HL [00:00:13.318,725] [1B][1;31m<err> os: fpscr:  0x00000000[1B][0m
HL [00:00:13.318,725] [1B][1;31m<err> os: Faulting instruction address (r15/pc): 0x0001ca92[1B][0m
HL [00:00:13.318,756] [1B][1;31m<err> os: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0[1B][0m
HL [00:00:13.318,756] [1B][1;31m<err> os: Current thread: 0x200032f0 (sysworkq)[1B][0m
HL [00:00:13.318,786] [1B][1;31m<err> error_handler: Caught system error -- reason 3. Entering infinite loop[1B][0m

 Can you from the terminal dump see if I'm on the right trail?

Best regards,
Håkon

[dejans]

Hi Håkon,

hakved_vea said:

I managed to connect to VOCE with the ISO Receive example from zephyr, as I see it, it does not implement any audio output.

On which device do you run ISO Receive sample? What is running on the other device?

hakved_vea said:

The two examples are quite different ISO Receive and nRF5340_Audio example, can you help me in any way to merge the two examples?

What do you expect to get from this merge? Could you elaborate on what you try to achieve?

Best regards,
Dejan

[hakved_vea]

Hi Dejan,

I programmed nRF5340-Audio-DK with ISO Receive example, the VOCE is set up as tx broadcast (BIS), this is programmed by NEXUM. 

from the image above, I want to keep all I2S and audio functionality, then stream audio from the VOCE device to the headphone output jack port.  

Best regards,
Håkon

[dejans]

Hi Håkon,

hakved_vea said:

I programmed nRF5340-Audio-DK with ISO Receive example

Where did you make all previously mentioned changes? Did you manually copy some files from nRF5340 Audio sample to ISO Receive sample and if so, which files? Could you show the current folder structure of your project which is programmed on nRF5340 audio DK?

hakved_vea said:

so that is selected both indexes BIT(1) and BIT(2). This is what ISO Receive had in the sync parameters.

Could you specify these sync parameters in ISO Receive?

hakved_vea said:

Then I was able to get the audio stream from VOCE to play on nRF5340, however the sound is disturbed.

Could you provide more information about the sound? How is it disturbed?

Best regards,
Dejan

[hakved_vea]

Hi Dejan,

I have been working with the nRF5340 Audio example, and made the changes described above in the files regarding that example. I only tested ISO receive to compare the two source codes. 

The nRF5340 Audio DK is only receiving a mono signal, while VOCE is broadcasting a stereo, when syncing to both channels I think the audio is mixed into the one mono channel and half of the packets gets duplicated reject. When listening to songs I can recognize the song playing, however it sounds broken.

I see that from the terminal printout that:
nRF5340 have define Channel allocation: 0x1
while VOCE sets Channel allocation: 0x0
What implications does this have?

Best regards,
Håkon

[hakved_vea]

Hi again,

I have tried to compare the two nRF5340 and VOCE in details.

below is comparison of sync adv packet and BIG info packet:

Difference between nRF5340 and VOCE

VOCE:
  BIG INFO[0]:
    [DEVICE]: 54:B7:E5:B3:75:7A (public)
    sid 0x08                  *!* Advertiser SID
    num_bis 2                     Number of BISes in the BIG
    nse 8                     *!* Maximum number of subevents in each isochronous event
    interval 0x0010 (20 ms)   *!* Interval between two BIG anchor point (N * 1.25 ms)
    bn 2                      *!* The number of new payloads in each BIS event
    pto 0                         Offset used for pre-transmissions
    irc 4                     *!* The number of times a payload is transmitted in a BIS event
    max_pdu 100                   Maximum size, in octets, of the payload
    sdu_interval 10000 us         The interval, in microseconds, of periodic SDUs.
    max_sdu 100                   Maximum size of an SDU, in octets.
    phy LE 2M                     Channel PHY
    without framing               Channel framing mode
    not encrypted                 Whether or not the BIG is encrypted

  PER_ADV_SYNC[0]:
    [DEVICE]: 54:B7:E5:B3:75:40 (public)
    tx_power 0
    RSSI -41
    CTE 0
    data length 42
    data: 29 16 51 18 00 00 00 01 02 06 00 00 00 00 0a 02 01 08 02 02 01 03 04 64 00 00 01 06 05 03 01 00 00 00 02 06 05 03 02 00 00 00

    16          Service data, 16-bit UUID
    51 18       BT_UUID_BASIC_AUDIO
    00 00 00    QoS Presentation Delay
    01          Number of subgroups in the BASE
    02          Number of BIS in the subgroup
    06          Codec ID
    00 00       Codec Company ID
    00 00       Codec Company Vendor ID
    0a          Codec configuration data length
    02          LTV: length
    01          LTV: type: Sampling_Frequency
    08          LTV: Value: 48000
    02          LTV: length
    02          LTV: type: Frame_Duration
    01          LTV: Value:  Use 10 ms codec frames
    03          LTV: length
    04          LTV: type: Octets_Per_Codec_Frame
    64 00       LTV: Value: 100
    00          codec metadata length
    01          Unique index of the BIS
    06          codec config data length
    05          LTV: length
    03          LTV: type: Audio_Channel_Allocation
    01 00 00 00 LTV: Value: Front Left
    02          Unique index of the BIS
    06          codec config data length
    05          LTV: length
    03          LTV: type: Audio_Channel_Allocation
    02 00 00 00 LTV: Value: Front Right

nRF5340:
  BIG INFO[0]:
    [DEVICE]: 34:F6:FB:BC:69:8C (random)
    sid 0x00                  *!* Advertiser SID
    num_bis 2                     Number of BISes in the BIG
    nse 5                     *!* Maximum number of subevents in each isochronous event
    interval 0x0008 (10 ms)   *!* Interval between two BIG anchor point (N * 1.25 ms)
    bn 1                      *!* The number of new payloads in each BIS event
    pto 0                         Offset used for pre-transmissions
    irc 5                     *!* The number of times a payload is transmitted in a BIS event
    max_pdu 100                   Maximum size, in octets, of the payload
    sdu_interval 10000 us         The interval, in microseconds, of periodic SDUs.
    max_sdu 100                   Maximum size of an SDU, in octets.
    phy LE 2M                     Channel PHY
    without framing               Channel framing mode
    not encrypted                 Whether or not the BIG is encrypted

  PER_ADV_SYNC[0]:
    [DEVICE]: 1F:0E:61:A3:79:A3 (random)
    tx_power 0
    RSSI -40
    CTE 0
    data length 40
    data: 27 16 51 18 10 27 00 01 02 06 00 00 00 00 10 02 01 08 02 02 01 05 03 01 00 00 00 03 04 64 00 04 03 02 04 00 01 00 02 00

    16          Service data, 16-bit UUID
    51 18       BT_UUID_BASIC_AUDIO
    10 27 00    QoS Presentation Delay
    01          Number of subgroups in the BASE
    02          Number of BIS in the subgroup
    06          Codec ID
    00 00       Codec Company ID
    00 00       Codec Company Vendor ID
    10          Codec configuration data length
    02          LTV: length
    01          LTV: type: Sampling_Frequency
    08          LTV: Value: 48000 Hz
    02          LTV: length
    02          LTV: type: Frame_Duration
    01          LTV: Value: Use 10 ms codec frames
    05          LTV: length
    03          LTV: type: Audio_Channel_Allocation
    01 00 00 00 LTV: Value: Front Left
    03          LTV: length
    04          LTV: type: Octets_Per_Codec_Frame
    64 00       LTV: Value: 100
    04          codec metadata length
    03          LTV: length
    02          LTV: type: Streaming_Audio_Contexts
    04 00       LTV: Value: Bitfield of Context Type values
    01          Unique index of the BIS
    00          codec config data length
    02          Unique index of the BIS
    00          codec config data length

I assume that some of these differences is handled by the LE Audio implementation, can you help me point out the differences that is not supported by the example code?

I see that VOCE specify channel allocation per bis, while nRF5340 sets it globally, I think my main challenge is with the synchronization of the channels  

I see this comment on line 339 in broadcast_source.c

	/* NOTE: It is also possible to have the codec configuration data per
	 * BIS index. As our API does not support such specialized BISes we
	 * currently don't do that.
	 */

alos at line 264 in broadcast_source.c

	/* TODO: The following encoding should be done for each subgroup once
	 * supported
	 */

is this something that you are currently working on?

Best regards,
Håkon

[dejans]

Hi Håkon, 

hakved_vea said:

When listening to songs I can recognize the song playing, however it sounds broken.

How did you verify that the sound is broken? How does it sound broken?

hakved_vea said:

I see that from the terminal printout that:
nRF5340 have define Channel allocation: 0x1
while VOCE sets Channel allocation: 0x0
What implications does this have?

Could you show the terminal output?

Best regards,
Dejan

[dejans]

Hi Håkon,

hakved_vea said:

I assume that some of these differences is handled by the LE Audio implementation, can you help me point out the differences that is not supported by the example code?

Could you please rephrase your question?

hakved_vea said:

alos at line 264 in broadcast_source.c

hakved_vea said:

I see this comment on line 339 in broadcast_source.c

hakved_vea said:

is this something that you are currently working on?

Currently supported features can be seen in the feature support matrix.

Best regards,
Dejan