Connecting nRF5340 LE Audio to other LE Audio equipment

[hakved_vea]

Hi,

I'm trying to connect the nRF5340 LE Audio example to another LE Audio equipment: VOCE by Nexum (https://www.nexum-design.com/) 

I asked Nexum if this was possible and got this reply:

Hi, This is totally ok if you set VOCE as Auracast TX mode. but please note, the standard sample rate of Auracast is 24K sample rate. And VOCE now is set by 48Kh. We will upgrade our VOCE to support 16 / 24Khz by next upgrade. now, you need to make sure nRF5340 as Auracast RX device can also support 16bit/48K sample rate. BR, NEXUM

I also opened a ticket on DevZone earlier but thought this got a little off topic, 

 Building nRF5340_audio example for nRF5340-DK (not NRF5340-AUDIO-DK) 

I was able to build the example and connect two nRF5340-Audio-DK kits. 

Do you have any advice for how to attack this?

I'm currently trying out different config parameters in addition to adding LOG_INF(...) to see if nRF5340 sees the VOCE device. But i have't got any closer..

Thanks in advance for any reply on this,

Best,

Håkon 

[hakved_vea]

Hi Dejan,
thanks for your reply,

I would like VOCE to be connected to a microphone and transmit on a broadcast stream, and then setting up multiple nRF5340 as receivers and connect them to speakers.

Initially I want to connect the two devices in any way so I can see that they are compatible, then I can use that as a baseline for further development. So if you think switching roles would be easier I can try that first.  

BR,
Håkon

[dejans]

Hi Håkon,

Our nRF5340 audio in BIS mode supports 16 KHz/24 KHz/48 KHz and it is currently set in compile time. You could consider even replacing VOCE with nRF5340 audio device. This can be done since nRF5340 audio can be set as BIS gateway using microphone as audio source.

Best regards,
Dejan

[hakved_vea]

Hi again,

I edited line 320 in le_audio_bis_headset.c:

	ret = bt_audio_broadcast_sink_sync(broadcast_sink,6/*bis_index_bitfields[active_stream_index]*/ ,
					   audio_streams_p, NULL);

so that is selected both indexes BIT(1) and BIT(2). This is what ISO Receive had in the sync parameters.

Then I was able to get the audio stream from VOCE to play on nRF5340, however the sound is disturbed.

*** Booting Zephyr OS build v3.2.99-ncs1 ***
HL [00:00:00.351,654] [1B][0m<inf> fw_info: [1B][0;32m
	 nRF5340 Audio nRF5340 Audio DK cpuapp 			    
	 NCS base version: 2.2.0 			    
	 Cmake run : Fri Jan 27 13:46:00 2023[1B][0m[1B][0m
HL [00:00:00.351,654] [1B][0m<inf> fw_info: ------- DEBUG BUILD -------[1B][0m
HL [00:00:00.351,684] [1B][0m<inf> fw_info: [1B][0;36mHEADSET left device[1B][0m[1B][0m
HL [00:00:00.362,304] [1B][0m<inf> board_version: [1B][0;32mCompatible board/HW version found: 1.0.0[1B][0m[1B][0m
HL [00:00:02.407,196] [1B][1;33m<wrn> bt_hci_core: Controller to host flow control not supported[1B][0m
HL [00:00:02.535,919] [1B][0m<inf> ble: MAC: EB:37:46:99:6E:86 (random)[1B][0m
HL [00:00:02.536,132] [1B][0m<inf> ble: Controller version: 3310[1B][0m
HL [00:00:02.630,828] [1B][0m<inf> bis_headset: Scanning for broadcast sources[1B][0m
HL [00:00:02.640,167] [1B][0m<inf> bis_headset: Broadcast source ªªªªªªªªªªªª[1D] found[1B][0m
HL [00:00:02.847,717] [1B][0m<inf> bis_headset: Codec config for LC3:[1B][0m
HL [00:00:02.847,747] [1B][0m<inf> bis_headset: 	Frequency: 48000 Hz[1B][0m
HL [00:00:02.847,747] [1B][0m<inf> bis_headset: 	Frame Duration: 10000 us[1B][0m
HL [00:00:02.847,747] [1B][0m<inf> bis_headset: 	Octets per frame: 100 (80000 kbps)[1B][0m
HL [00:00:02.847,778] [1B][0m<inf> bis_headset: 	Frames per SDU: 1[1B][0m
HL [00:00:02.847,778] [1B][0m<inf> bis_headset: 	Channel allocation: 0x0[1B][0m
HL [00:00:02.847,778] [1B][0m<inf> bis_headset: Codec config for LC3:[1B][0m
HL [00:00:02.847,808] [1B][0m<inf> bis_headset: 	Frequency: 48000 Hz[1B][0m
HL [00:00:02.847,808] [1B][0m<inf> bis_headset: 	Frame Duration: 10000 us[1B][0m
HL [00:00:02.847,808] [1B][0m<inf> bis_headset: 	Octets per frame: 100 (80000 kbps)[1B][0m
HL [00:00:02.847,808] [1B][0m<inf> bis_headset: 	Frames per SDU: 1[1B][0m
HL [00:00:02.847,839] [1B][0m<inf> bis_headset: 	Channel allocation: 0x0[1B][0m
HL [00:00:02.847,930] [1B][0m<inf> bis_headset: Syncing to broadcast stream index 0[1B][0m
HL [00:00:02.908,233] [1B][0m<inf> bis_headset: Stream index 0 started[1B][0m
HL [00:00:02.908,477] [1B][0m<inf> bis_headset: Stream index 0 started[1B][0m
HL [00:00:02.926,269] [1B][1;33m<wrn> audio_datapath: I2S RX overrun. Single msg[1B][0m
HL [00:00:02.932,281] [1B][0m<inf> audio_datapath: Drft comp state: CALIB[1B][0m
HL [00:00:02.933,197] [1B][1;33m<wrn> audio_datapath: Duplicate sdu_ref_us (2574616) - Dropping audio frame[1B][0m
HL [00:00:02.933,197] [1B][1;33m<wrn> audio_datapath: Duplicate sdu_ref_us (2574616) - Dropping audio frame[1B][0m
HL [00:00:02.933,227] [1B][1;33m<wrn> audio_datapath: Duplicate sdu_ref_us (2574616) - Dropping audio frame[1B][0m
HL [00:00:02.933,258] [1B][1;33m<wrn> audio_datapath: Data received, total underruns: 14[1B][0m
HL [00:00:02.951,599] [1B][0m<inf> audio_datapath: sdu_ref_us not from consecutive frames[1B][0m
HL [00:00:02.953,155] [1B][1;33m<wrn> audio_datapath: Duplicate sdu_ref_us (2594617) - Dropping audio frame[1B][0m
[1B][1;31m--- 3 messages dropped ---
[1B][0mHL [00:00:02.971,588] [1B][0m<inf> audio_datapath: sdu_ref_us not from consecutive frames[1B][0m
HL [00:00:02.973,175] [1B][1;33m<wrn> audio_datapath: Duplicate sdu_ref_us (2614617) - Dropping audio frame[1B][0m
[1B][1;31m--- 5 messages dropped ---
[1B][0mHL [00:00:02.993,194] [1B][1;33m<wrn> audio_datapath: Duplicate sdu_ref_us (2634617) - Dropping audio frame[1B][0m
[1B][1;31m--- 5 messages dropped ---
[1B][0mHL [00:00:03.013,244] [1B][1;33m<wrn> audio_datapath: Data received, total underruns: 54[1B][0m
[1B][1;31m--- 1 messages dropped ---
[1B][0mHL [00:00:03.031,585] [1B][0m<inf> audio_datapath: sdu_ref_us not from consecutive frames[1B][0m
[1B][1;31m--- 4 messages dropped ---
[1B][0mHL [00:00:03.033,264] [1B][1;33m<wrn> audio_datapath: Data received, total underruns: 64[1B][0m
HL [00:00:03.051,605] [1B][0m<inf> audio_datapath: sdu_ref_us not from consecutive frames[1B][0m
[1B][1;31m--- 4 messages dropped ---
[1B][0mHL [00:00:03.071,594] [1B][0m<inf> audio_datapath: sdu_ref_us not from consecutive frames[1B][0m
HL [00:00:03.073,150] [1B][1;33m<wrn> audio_datapath: Duplicate sdu_ref_us (2714617) - Dropping audio frame[1B][0m

Above is a dump from the terminal. It seems it connects both ISO channels to the same audio channel.

I'm a bit lost in why this work and why setting bis_bitfields to the correct value causes the nRF5340 to crash?

Best regards,
Håkon

[dejans]

Hi Håkon,

hakved_vea said:

I programmed nRF5340-Audio-DK with ISO Receive example

Where did you make all previously mentioned changes? Did you manually copy some files from nRF5340 Audio sample to ISO Receive sample and if so, which files? Could you show the current folder structure of your project which is programmed on nRF5340 audio DK?

hakved_vea said:

so that is selected both indexes BIT(1) and BIT(2). This is what ISO Receive had in the sync parameters.

Could you specify these sync parameters in ISO Receive?

hakved_vea said:

Then I was able to get the audio stream from VOCE to play on nRF5340, however the sound is disturbed.

Could you provide more information about the sound? How is it disturbed?

Best regards,
Dejan

[hakved_vea]

Hi Dejan,

I have been working with the nRF5340 Audio example, and made the changes described above in the files regarding that example. I only tested ISO receive to compare the two source codes. 

The nRF5340 Audio DK is only receiving a mono signal, while VOCE is broadcasting a stereo, when syncing to both channels I think the audio is mixed into the one mono channel and half of the packets gets duplicated reject. When listening to songs I can recognize the song playing, however it sounds broken.

I see that from the terminal printout that:
nRF5340 have define Channel allocation: 0x1
while VOCE sets Channel allocation: 0x0
What implications does this have?

Best regards,
Håkon

[hakved_vea]

Hi again,

I have tried to compare the two nRF5340 and VOCE in details.

below is comparison of sync adv packet and BIG info packet:

Difference between nRF5340 and VOCE

VOCE:
  BIG INFO[0]:
    [DEVICE]: 54:B7:E5:B3:75:7A (public)
    sid 0x08                  *!* Advertiser SID
    num_bis 2                     Number of BISes in the BIG
    nse 8                     *!* Maximum number of subevents in each isochronous event
    interval 0x0010 (20 ms)   *!* Interval between two BIG anchor point (N * 1.25 ms)
    bn 2                      *!* The number of new payloads in each BIS event
    pto 0                         Offset used for pre-transmissions
    irc 4                     *!* The number of times a payload is transmitted in a BIS event
    max_pdu 100                   Maximum size, in octets, of the payload
    sdu_interval 10000 us         The interval, in microseconds, of periodic SDUs.
    max_sdu 100                   Maximum size of an SDU, in octets.
    phy LE 2M                     Channel PHY
    without framing               Channel framing mode
    not encrypted                 Whether or not the BIG is encrypted

  PER_ADV_SYNC[0]:
    [DEVICE]: 54:B7:E5:B3:75:40 (public)
    tx_power 0
    RSSI -41
    CTE 0
    data length 42
    data: 29 16 51 18 00 00 00 01 02 06 00 00 00 00 0a 02 01 08 02 02 01 03 04 64 00 00 01 06 05 03 01 00 00 00 02 06 05 03 02 00 00 00

    16          Service data, 16-bit UUID
    51 18       BT_UUID_BASIC_AUDIO
    00 00 00    QoS Presentation Delay
    01          Number of subgroups in the BASE
    02          Number of BIS in the subgroup
    06          Codec ID
    00 00       Codec Company ID
    00 00       Codec Company Vendor ID
    0a          Codec configuration data length
    02          LTV: length
    01          LTV: type: Sampling_Frequency
    08          LTV: Value: 48000
    02          LTV: length
    02          LTV: type: Frame_Duration
    01          LTV: Value:  Use 10 ms codec frames
    03          LTV: length
    04          LTV: type: Octets_Per_Codec_Frame
    64 00       LTV: Value: 100
    00          codec metadata length
    01          Unique index of the BIS
    06          codec config data length
    05          LTV: length
    03          LTV: type: Audio_Channel_Allocation
    01 00 00 00 LTV: Value: Front Left
    02          Unique index of the BIS
    06          codec config data length
    05          LTV: length
    03          LTV: type: Audio_Channel_Allocation
    02 00 00 00 LTV: Value: Front Right

nRF5340:
  BIG INFO[0]:
    [DEVICE]: 34:F6:FB:BC:69:8C (random)
    sid 0x00                  *!* Advertiser SID
    num_bis 2                     Number of BISes in the BIG
    nse 5                     *!* Maximum number of subevents in each isochronous event
    interval 0x0008 (10 ms)   *!* Interval between two BIG anchor point (N * 1.25 ms)
    bn 1                      *!* The number of new payloads in each BIS event
    pto 0                         Offset used for pre-transmissions
    irc 5                     *!* The number of times a payload is transmitted in a BIS event
    max_pdu 100                   Maximum size, in octets, of the payload
    sdu_interval 10000 us         The interval, in microseconds, of periodic SDUs.
    max_sdu 100                   Maximum size of an SDU, in octets.
    phy LE 2M                     Channel PHY
    without framing               Channel framing mode
    not encrypted                 Whether or not the BIG is encrypted

  PER_ADV_SYNC[0]:
    [DEVICE]: 1F:0E:61:A3:79:A3 (random)
    tx_power 0
    RSSI -40
    CTE 0
    data length 40
    data: 27 16 51 18 10 27 00 01 02 06 00 00 00 00 10 02 01 08 02 02 01 05 03 01 00 00 00 03 04 64 00 04 03 02 04 00 01 00 02 00

    16          Service data, 16-bit UUID
    51 18       BT_UUID_BASIC_AUDIO
    10 27 00    QoS Presentation Delay
    01          Number of subgroups in the BASE
    02          Number of BIS in the subgroup
    06          Codec ID
    00 00       Codec Company ID
    00 00       Codec Company Vendor ID
    10          Codec configuration data length
    02          LTV: length
    01          LTV: type: Sampling_Frequency
    08          LTV: Value: 48000 Hz
    02          LTV: length
    02          LTV: type: Frame_Duration
    01          LTV: Value: Use 10 ms codec frames
    05          LTV: length
    03          LTV: type: Audio_Channel_Allocation
    01 00 00 00 LTV: Value: Front Left
    03          LTV: length
    04          LTV: type: Octets_Per_Codec_Frame
    64 00       LTV: Value: 100
    04          codec metadata length
    03          LTV: length
    02          LTV: type: Streaming_Audio_Contexts
    04 00       LTV: Value: Bitfield of Context Type values
    01          Unique index of the BIS
    00          codec config data length
    02          Unique index of the BIS
    00          codec config data length

I assume that some of these differences is handled by the LE Audio implementation, can you help me point out the differences that is not supported by the example code?

I see that VOCE specify channel allocation per bis, while nRF5340 sets it globally, I think my main challenge is with the synchronization of the channels  

I see this comment on line 339 in broadcast_source.c

	/* NOTE: It is also possible to have the codec configuration data per
	 * BIS index. As our API does not support such specialized BISes we
	 * currently don't do that.
	 */

alos at line 264 in broadcast_source.c

	/* TODO: The following encoding should be done for each subgroup once
	 * supported
	 */

is this something that you are currently working on?

Best regards,
Håkon

[dejans]

Hi Håkon,

hakved_vea said:

I edited line 320 in le_audio_bis_headset.c:

hakved_vea said:

so that is selected both indexes BIT(1) and BIT(2). This is what ISO Receive had in the sync parameters.

Could you show these sync parameters in ISO Receive?

hakved_vea said:

I'm a bit lost in why this work and why setting bis_bitfields to the correct value causes the nRF5340 to crash?

What is the correct value? Which value works and which doesn't work? Do you have a log showing the crash?

Best regards,
Dejan