Audio streaming between dk and audio dk over BLE is not happening on nRF5340

[PassionPioneer]

Hi Nordic Team,

My project goal is to send sensor data and audio stream from PDM mic over BLE from nRF5340 to another nRF5340 board.

For this I have split the project into two phases. In first phase I am able to send sensor data from nRF5340 dk board to nodeMCU(which in turn connects over USB-COM to PC for data analyses). 

In second phase, I used nRF5340 dk and audio dk boards to stream the PDM mic audio data. Initially I flashed unicast server (IDE: nRF connect for VScode) sample code on dk board and audio dk board is configured in headset mode. Then both boards are connected over BLE and the sample code (a beep sound is heard on headset connected to audio dk board) is tested OK. I interfaced PDM mic to dk board on same example project and did some modifications to replace sample audio buffer with PDM mic's PCM data buffer. Unfortunately, I couldn't hear sound played across mic.

Currently I planning to use 2 audio dk boards for PDM mic data stream. Since the audio dk application is a customized code, I need guidance and continuous support to modify the code in achieving the requirement.

Please give your support.

regards,

Naveen

[Amanda Hsieh]

Hi, 

For nRF5340 audio DK, the PDM mic is not connected to nRF5340 directly but connects to CS47L63 through PDM, then CS47L63 using I2S to interface with nRF5340. You can use PDM directly, but the sampling rate from nRF5340 is 16KHz. Also, the audio datapath module is designed for the I2S interface, so you would need to handle the data fetch from PDM and the clock drift issues yourself.

Regards,
Amanda H.

[PassionPioneer]

Hi,

Thank you for your feedback.

Amanda Hsieh said:

but connects to CS47L63 through PDM, then CS47L63 using I2S to interface with nRF5340.

This would be fine; I will use this interface as it is. In walkie talkie mode, the two audio dk boards can communicate mic audio over BLE please confirm. 

As I mentioned that the sensor data(such as Time of Flight) need to send over BLE, Is it possible to initialize and use ../bluetooth/services/nus.c library along with LE audio library?

Regards,

Naveen

[Amanda Hsieh]

Hi,

PassionPioneer said:

In walkie talkie mode, the two audio dk boards can communicate mic audio over BLE please confirm. 

 The nRF5340 Audio DK boards can communicate mic audio over BLE in walkie-talkie mode. See Enabling the walkie-talkie demo.

PassionPioneer said:

As I mentioned that the sensor data(such as Time of Flight) need to send over BLE, Is it possible to initialize and use ../bluetooth/services/nus.c library along with LE audio library?

That's possible.  There is a hard limitation on the total number of concurrent connections: 2 ISO + 2 ACL, and so you could for instance use this for 2 CIS (1 CIS requires 1 ISO + 1 ACL), or a combination of 1 or 2 BIS + 1 or 2 ACLs. In case of the latter, you can see an example of how to do this here, but please keep in mind that this is an old example that we do not maintain, but it highlights the general approach you should take to achieve this. This post might help. 

-Amanda H.

[PassionPioneer]

HI Amanda,

Thanks for your detailed explanation and suggestions.

Amanda Hsieh said:

but please keep in mind that this is an old example that we do not maintain, but it highlights the general approach you should take to achieve this. This post might help. 

I have gone through the link nRF5340 Audio - configure gateway unit also as ACL peripheral provided in the solution reply of above link. I applied modified the files and tested, the board is able to get detect and connect to smartphone and also the data sent from nRF toolbox app is received by the board but the board is getting disconnected after few seconds from connection please see the log given below

*** Booting nRF Connect SDK d96769faceca ***

GW [00:00:00.270,996] <inf> board_version: Compatible board/HW version found: 1. 1.0

GW [00:00:00.295,471] <inf> audio_usb: Ready for USB host to send/receive.

GW [00:00:00.295,501] <inf> fw_info:

nRF5340 Audio nRF5340 Audio DK cpuapp

NCS base version: 2.6.0

Cmake run : Thu Oct 03 15:38:49 2024

GW [00:00:00.295,532] <inf> fw_info: ------- DEBUG BUILD -------

GW [00:00:00.295,532] <inf> fw_info: Compiled for GATEWAY device

GW [00:00:00.309,478] <dbg> bt_mgmt: bt_enabled_cb: Bluetooth initialized

GW [00:00:00.309,967] <inf> bt_mgmt_ctlr_cfg: Controller: SoftDevice: Version 5. 4 (0x0d), Revision 8591

GW [00:00:00.310,150] <inf> bt_mgmt: Local identity addr: C4:96:F7:03:D5:3B (ran dom)

GW [00:00:00.310,455] <inf> broadcast_source: Enabling broadcast_source NRF5340_ BROADCASTER

GW [00:00:00.314,300] <inf> bt_mgmt_adv: Local addr: 61:22:B1:B7:29:BB (random)

GW [00:00:00.315,643] <inf> main: Ext adv ready

GW [00:00:00.316,436] <inf> bt_mgmt_adv: Advertising successfully started

GW [00:00:00.326,660] <inf> broadcast_source: Broadcast source 0x2000d16c starte d

GW [00:00:00.327,117] <inf> broadcast_source: Broadcast source 0x2000d194 starte d

GW [00:00:02.601,440] <inf> bt_mgmt: Connected: 7F:8B:B6:F4:9E:3F (random)

GW [00:00:02.603,820] <inf> bt_mgmt_scan: Local addr: 65:B1:39:82:6E:9F (random) . May time out. Updates not printed

GW [00:00:02.603,820] <inf> bt_mgmt_scan: Scanning successfully started

GW [00:00:02.603,851] <inf> main: BT_MGMT_CONNECTED

GW [00:00:10.003,509] <inf> main: NUS

68 69 |hi

GW [00:00:13.264,465] <inf> audio_usb: USB RX first data received.

A

GW [00:00:35.744,262] <wrn> bt_mgmt: Security failed: level 1 err 9

GW [00:00:35.744,293] <wrn> bt_mgmt: Failed to disconnect -128

GW [00:00:35.744,445] <inf> bt_mgmt: Disconnected: 7F:8B:B6:F4:9E:3F (random) (r eason 0x13)

GW [00:00:35.744,445] <inf> main: BT_MGMT_DISCONNECTED

GW [00:00:35.745,727] <inf> bt_mgmt_scan: Local addr: 65:B1:39:82:6E:9F (random) . May time out. Updates not printed

GW [00:00:35.745,758] <inf> bt_mgmt_scan: Scanning successfully started

ASSERTION FAIL [old > 0] @ WEST_TOPDIR/zephyr/subsys/bluetooth/host/conn.c:1318

Conn reference counter is 0

GW [00:00:35.745,788] <err> error_handler: Assert post action: file: WEST_TOPDIR /zephyr/subsys/bluetooth/host/conn.c, line 1318

GW [00:00:35.745,788] <err> error_handler: Caught system error -- reason 0. Ente ring infinite loop

GW [00:00:35.778,015] <err> usb_nrfx: USBD event allocation failed!

GW [00:00:35.810,028] <err> usb_nrfx: USBD event allocation failed!

Thanks,

Naveen

[Amanda Hsieh]

Hi, 

PassionPioneer said:

GW [00:00:35.744,445] <inf> bt_mgmt: Disconnected: 7F:8B:B6:F4:9E:3F (random) (r eason 0x13)

Disconnect reason 0x13 indicates that the remote user terminated the connection (as defined in ble_hci.h). It means that since you're seeing it on the peripheral side, the central side is triggering the disconnect for some reason. If you only see this after the devices are paired, that's likely because the bonding information is erased in one device while the other tries reconnecting with the "old" bonding information. If that's the case, you need to erase bonding information on the phone side before reconnecting and pairing again. 

-Amanda H.

[PassionPioneer]

Hi,

I I repeated the tests and found two things as follows,

1. In the file bt_mgmt.c, the security check is getting failed as you can the failure message i.e., "GW [00:00:35.744,262] <wrn> bt_mgmt: Security failed: level 1 err 9" in the log.

2. On nRFtoolbox app, when I select the device to connect there is popup message as shown below

In both the case, I think the pairing is not happening because of security pin and also when I click on pair&connect in notification on smartphone still behavior is same. That's why the central(smartphone) is disconnecting from the board as you said.

Regards,

Naveen