Creating multiple BIGs with one gateway (nRF5340 Audio DK)

[dark73]

My project goal is to broadcast four different languages simultaneously using Auracast. Each language will be transmitted on its own BIG so that Auracast-compatible devices, such as Galaxy Buds, can select and listen to the desired language.

As a first step, I am currently trying to receive two language streams and create two BIGs.

Before starting this work, I successfully built and ran the broadcast_source sample from the nrf_audio application in nRF Connect SDK.

I have two questions:

1. I would like to receive audio for multiple languages over a single USB connection by using multiple USB audio channels (or another recommended USB audio architecture). Is this the recommended approach? If so, are there any examples or documentation that I should refer to?
2. Before creating multiple BIGs, my understanding is that each audio stream needs to be encoded separately in encoder_thread() before being sent to the corresponding BIG. Is my understanding correct? If so, could you point me to the relevant source files, documentation, or examples that explain how this part of the audio pipeline is implemented?

I have tried to find documentation and examples related to these two topics, but I have not been able to find any useful references.

I would greatly appreciate it if you could provide some guidance on where I should start looking or which parts of the source code and documentation I should study.

Thank you very much.

[Øyvind]

Hello,

Thanks for registering your question with us. Please note that Nordic Semiconductor is currently in a summer break period and many of our resources are out of office. We will provide an answer back in a timely manner, and update you as soon as possible. In advance, thank you for you patience.

---

I've forwarded your question internally and hope to have an answer within 3 business days.

Kind regards,
Øyvind

[dark73]

I have a few additional questions regarding a mono audio broadcasting configuration.

Assuming that I use mono audio, I would like to create 4 BIGs, where each BIG contains a single BIS (for a total of 4 BIGs and 4 BIS streams).

My questions are:

1. Is it sufficient to run four LC3 encoders (one encoder per BIS) in this configuration?
2. Is it possible to implement this on a single nRF5340 Audio DK?

In addition, I would like to ask about the following:

1. How can I configure the nRF5340 Audio DK to broadcast mono audio using 4 BIGs, with each BIG containing a single BIS (4 BIGs / 4 BIS in total)?
2. If I receive a single USB Audio stream from a host (e.g., Jetson), what is the recommended way to separate it into four channels (KOR, ENG, JAP, CHA) and feed them to four LC3 encoders? Is this architecture supported or recommended? (As I understand it, the audio_system.c receives audio as two channels (L/ R stereo))

Thank you for your help.

[dark73]

*** Booting nRF Connect SDK v3.4.0-99553055607b ***
*** Using Zephyr OS v4.4.0-bf801e4e3d19 ***
-- [00:00:00.258,697] <wrn> location: Invalid location in UICR, using default location
-- [00:00:00.284,820] <inf> bt_mgmt: Controller: SoftDevice: Version 6.3 (0x11), Revision 8249
-- [00:00:00.284,881] <inf> bt_mgmt: Local identity addr: DE:DA:E7:0B:7A:69 (random)
-- [00:00:00.284,942] <inf> audio_usb: USB initialized as unidirectional (headphones only).
-- [00:00:00.285,156] <inf> audio_usb: Ready for USB host to send/receive: No Sync (multi-clock)
-- [00:00:00.306,304] <wrn> sd_card_lc3_file: Failed to initialize SD card: -19
-- [00:00:00.306,304] <wrn> lc3_streamer: Failed to initialize LC3 file module -19
-- [00:00:00.306,335] <wrn> main: Failed to initialize LC3 streamer. Transmitting fixed buffers

**

-- [00:00:00.531,280] <err> udc: Failed to allocate net_buf 4095, ep 0x80

**
==============================================================================



uart:~$ nac start 0
-- [00:00:16.754,150] <inf> broadcast_source: Created: 0x20027bc4 TV-audio
-- [00:00:16.756,958] <inf> bt_mgmt_adv: Local addr: DE:DA:E7:0B:7A:69 (random)
-- [00:00:16.759,735] <inf> bt_mgmt_adv: Advertising successfully started
-- [00:00:16.768,127] <inf> broadcast_source: Broadcast stream 0x200090cc started
-- [00:00:16.768,157] <inf> le_audio: LC3 codec config for source:
-- [00:00:16.768,157] <inf> le_audio: Frequency: 24000 Hz
-- [00:00:16.768,157] <inf> le_audio: Duration: 10000 us
-- [00:00:16.768,157] <inf> le_audio: Chan alloc: 0x1
-- [00:00:16.768,157] <inf> le_audio: Oct pr frame: 60 (48000 bps)
-- [00:00:16.768,157] <inf> le_audio: Frames pr SDU: 1
-- [00:00:16.768,310] <inf> audio_system: Gateway configured for 2 encoder channels
-- [00:00:16.768,798] <inf> broadcast_source: Broadcast stream 0x200090f8 started
-- [00:00:16.768,798] <inf> le_audio: LC3 codec config for source:
-- [00:00:16.768,829] <inf> le_audio: Frequency: 24000 Hz
-- [00:00:16.768,829] <inf> le_audio: Duration: 10000 us
-- [00:00:16.768,829] <inf> le_audio: Chan alloc: 0x2
-- [00:00:16.768,829] <inf> le_audio: Oct pr frame: 60 (48000 bps)
-- [00:00:16.768,829] <inf> le_audio: Frames pr SDU: 1
-- [00:00:16.768,829] <inf> broadcast_source: Source 0x20027bc4 started
-- [00:00:16.768,829] <inf> broadcast_source: Index: 0
-- [00:00:16.768,859] <inf> broadcast_source: ID: 2950679
-- [00:00:16.768,859] <inf> broadcast_source: Num subgroups: 1
-- [00:00:16.768,859] <inf> broadcast_source: Encrypted with key: Auratest

**
-- [00:00:19.813,171] <err> sw_codec_select: Encode failed
-- [00:00:19.813,171] <err> sw_codec_select: ERR_CHK Err_code: [-5006] @ line: 197
-- [00:00:19.813,171] <err> os: r0/a1: 0x00000003 r1/a2: 0x00000002 r2/a3: 0x00000001
-- [00:00:19.813,171] <err> os: r3/a4: 0x00000003 r12/ip: 0x00000000 r14/lr: 0x0000bd0f
-- [00:00:19.813,171] <err> os: xpsr: 0x01100000
-- [00:00:19.813,201] <err> os: s[ 0]: 0x00000000 s[ 1]: 0x00000000 s[ 2]: 0x00000001 s[ 3]: 0x00000001
-- [00:00:19.813,201] <err> os: s[ 4]: 0x20002af8 s[ 5]: 0x000000c5 s[ 6]: 0xffffec72 s[ 7]: 0x000542f0
-- [00:00:19.813,201] <err> os: s[ 8]: 0xffffec72 s[ 9]: 0x20002994 s[10]: 0x20006d00 s[11]: 0x00000001
-- [00:00:19.813,201] <err> os: s[12]: 0x00000001 s[13]: 0x2002d9c8 s[14]: 0x200483fc s[15]: 0x0001142f
-- [00:00:19.813,201] <err> os: fpscr: 0x00000001
-- [00:00:19.813,232] <err> os: Faulting instruction address (r15/pc): 0x0000bbd6
-- [00:00:19.813,232] <err> os: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0
-- [00:00:19.813,262] <err> os: Current thread: 0x200061d8 (Encoder)
-- [00:00:19.959,472] <err> os: Halting system

**

I successfully detected the USB device, but I'm now encountering a buffer allocation issue and an encoding failure.

[dark73]

I've attached the prj.conf file used for the Auraconfig sample.


#

# Copyright (c) 2024 Nordic Semiconductor ASA

#

# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause

#

# General

CONFIG_REBOOT=y

CONFIG_DEBUG=y

CONFIG_ASSERT=y

CONFIG_STACK_USAGE=y

CONFIG_THREAD_RUNTIME_STATS=y

CONFIG_MAIN_THREAD_PRIORITY=10

CONFIG_STACK_SENTINEL=y

CONFIG_INIT_STACKS=y

CONFIG_MAIN_STACK_SIZE=12000

CONFIG_THREAD_NAME=y

# 2000

CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=4096

CONFIG_NET_BUF_POOL_USAGE=y

# Uart driver

CONFIG_SERIAL=y

# Logging

CONFIG_LOG=y

CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y

CONFIG_LOG_TAG_MAX_LEN=2

CONFIG_LOG_TAG_DEFAULT="--"

CONFIG_LOG_BUFFER_SIZE=4096

CONFIG_USE_SEGGER_RTT=n

CONFIG_LOG_BACKEND_RTT=n

CONFIG_SOC_NRF53_CPUNET_ENABLE=y

CONFIG_ZBUS=y

CONFIG_ZBUS_RUNTIME_OBSERVERS=y

CONFIG_ZBUS_MSG_SUBSCRIBER=y

CONFIG_ZBUS_MSG_SUBSCRIBER_NET_BUF_POOL_SIZE=20

CONFIG_REGULATOR=y

CONFIG_CONTIN_ARRAY=y

CONFIG_DATA_FIFO=y

# Enable NRFX_CLOCK for ACLK control

CONFIG_NRFX_CLOCK=y

CONFIG_NEWLIB_LIBC=y

# Audio codec LC3 related defines

# FPU_SHARING enables preservation of the hardware floating point registers

# across context switches to allow multiple threads to perform concurrent

# floating point operations.

CONFIG_FPU=y

CONFIG_FPU_SHARING=y

# Enable SDHC interface

CONFIG_DISK_DRIVERS=y

CONFIG_DISK_DRIVER_SDMMC=y

# Allocate buffer on RAM for transferring chunck of data

# from Flash to SPI

CONFIG_SPI_NRFX_RAM_BUFFER_SIZE=8

# CONFIG_the file system

CONFIG_FILE_SYSTEM=y

CONFIG_FAT_FILESYSTEM_ELM=y

CONFIG_FS_FATFS_LFN=y

CONFIG_FS_FATFS_LFN_MODE_STACK=y

# exFAT enabled to support longer file names and higher transfer speed

CONFIG_FS_FATFS_EXFAT=y

CONFIG_FS_FATFS_MAX_LFN=255

# Enable SPI interface

CONFIG_SPI=y

# Enable ADC for board version readback

CONFIG_ADC=y

CONFIG_WATCHDOG=y

CONFIG_TASK_WDT=y

# Use this for debugging thread usage

#CONFIG_LOG_THREAD_ID_PREFIX=y

# Console related defines

CONFIG_CONSOLE=y

CONFIG_UART_CONSOLE=y

# Shell related defines

CONFIG_SHELL=y

CONFIG_KERNEL_SHELL=y

CONFIG_SHELL_BACKEND_SERIAL=y

CONFIG_SHELL_VT100_COMMANDS=y

CONFIG_SHELL_VT100_COLORS=y

CONFIG_SHELL_STACK_SIZE=8096

CONFIG_SHELL_CMD_BUFF_SIZE=128

CONFIG_SHELL_DEFAULT_TERMINAL_WIDTH=120

## Reduce shell memory usage

CONFIG_SHELL_WILDCARD=n

CONFIG_SHELL_HELP_ON_WRONG_ARGUMENT_COUNT=n

CONFIG_SHELL_STATS=n

CONFIG_SHELL_CMDS=n

CONFIG_SHELL_HISTORY=y

# Turn off default shell commands

CONFIG_I2C_SHELL=n

CONFIG_HWINFO_SHELL=n

CONFIG_CLOCK_CONTROL_NRF_SHELL=n

CONFIG_FLASH_SHELL=n

CONFIG_DEVICE_SHELL=n

# Suppress LOG_ERR messages from sd_check_card_type. Because SPI_SDHC has no card presence method,

# assume card is in slot. Thus error message is always shown if card is not inserted

CONFIG_SD_LOG_LEVEL_OFF=y

# Suppress LOG_INF messages from hci_core

CONFIG_BT_HCI_CORE_LOG_LEVEL_WRN=y

## ISO related configs ##

CONFIG_BT=y

CONFIG_UTF8=y

CONFIG_BT_DEVICE_NAME_DYNAMIC=y

CONFIG_BT_CAP_INITIATOR=y

CONFIG_BT_ISO_BROADCASTER=y

CONFIG_BT_ISO_MAX_BIG=2

CONFIG_BT_ISO_MAX_CHAN=8

CONFIG_BT_ISO_TX_BUF_COUNT=3

CONFIG_BT_BAP_BROADCAST_SOURCE=y

CONFIG_BT_BAP_BROADCAST_SRC_COUNT=2

CONFIG_BT_EXT_ADV_MAX_ADV_SET=2

CONFIG_BT_BAP_BROADCAST_SRC_SUBGROUP_COUNT=4

CONFIG_BT_BAP_BROADCAST_SRC_STREAM_COUNT=8

# Broadcasting Device - 0x0885

CONFIG_BT_DEVICE_APPEARANCE=2181

CONFIG_TRANSPORT_BIS=y

CONFIG_BT_AUDIO_BROADCAST_CONFIGURABLE=y

CONFIG_BT_AUDIO_BROADCAST_ZBUS_EVT_STREAM_SENT=y

CONFIG_LE_AUDIO_MSG_SUB_THREAD_PRIO=3

# CONFIG_SW_CODEC_NONE=y

CONFIG_NRF_AUDIO_SD_CARD_MODULE=y

CONFIG_NRF_AUDIO_SD_CARD_LC3_FILE=y

CONFIG_NRF_AUDIO_SD_CARD_LC3_STREAMER=y

CONFIG_SD_CARD_LC3_STREAMER_STACK_SIZE=8000

CONFIG_MODULE_SD_CARD_LOG_LEVEL_WRN=y

CONFIG_AUDIO_SOURCE_I2S=n

# USB 추가

CONFIG_POLL=y

CONFIG_APP_NRF_AUDIO=y

CONFIG_SAMPLE_RATE_CONVERTER=y

CONFIG_SAMPLE_RATE_CONVERTER_FILTER_SIMPLE=y

CONFIG_SPEED_OPTIMIZATIONS=y

CONFIG_CPU_LOAD=y

CONFIG_CPU_LOAD_LOG_PERIODICALLY=5000

# General

CONFIG_BT_RX_STACK_SIZE=2500

CONFIG_UDC_BUF_COUNT=32

[dark73]

Thank you very much for your continuous support and for taking the time to answer my questions. All previous issues have been resolved.

I separated the broadcast into two subgroups and implemented the code so that each encoded audio stream (KOR and ENG) can be transmitted through different subgroups. The KOR audio is assigned to SUB0, and the ENG audio is assigned to SUB1.

However, only the KOR audio is being broadcast. The audio from SUB0 also affects SUB1, so the KOR audio can be heard from SUB1 as well. I verified that KOR and ENG are different audio streams by broadcasting them separately through only SUB0.

My question is: I called subgroup_send() with different audio frames according to each subgroup, but why is the audio assigned to SUB0 also being transmitted through SUB1?

I modified subgroup_send() so that it receives the encoded audio frame as an argument and sends it using broadcast_source_send(audio_frame, stream_idx.lvl1, stream_idx.lvl2). I expected the behavior to be:

• SUB0 → KOR audio → BIG0
• SUB1 → ENG audio → BIG1

However, the actual behavior is:

• SUB0 → KOR audio → BIG0
• SUB1 → KOR audio → BIG1

It seems that the frame buffer or stream mapping is still shared between the subgroups.

I would like to know if there is any additional configuration or buffer separation required when using multiple subgroups. Also, does broadcast_source_send() use the subgroup index (lvl1, lvl2) only for routing, or is there another internal mechanism that determines which LC3 frame is transmitted for each subgroup?

Below is my modified subgroup_send() code.

** The stream_frame_send() function is called from encoder_thread() **

void stream_frame_send(struct net_buf *audio_kor, struct net_buf *audio_eng)

{

    struct stream_index kor_idx = {

        .lvl1 = 0,

        .lvl2 = 0,

        .lvl3 = 0

    };

    subgroup_send(kor_idx, audio_kor);

   

    struct stream_index eng_idx = {

        .lvl1 = 0,

        .lvl2 = 1,

        .lvl3 = 0

    };

    subgroup_send(eng_idx, audio_eng);

   

}

static void subgroup_send(struct stream_index stream_idx, struct net_buf *audio_frame_out)
{
	int ret;
	static int prev_ret;

	uint8_t num_bis = subgroups[stream_idx.lvl1][stream_idx.lvl2].num_bises;
	size_t frame_size = lc3_stream_infos[stream_idx.lvl1][stream_idx.lvl2].frame_size;
	struct net_buf *audio_frame = net_buf_alloc(&ble_tx_pool, K_NO_WAIT);
	struct audio_metadata *meta = net_buf_user_data(audio_frame);

	memset(meta, 0, sizeof(*meta));

	if (audio_frame == NULL) {
		LOG_ERR("Out of RX buffers");
		return;
	}
	
	for (int i = 0; i < num_bis; i++) {
		uint8_t *frame_ptr =
			lc3_stream_infos[stream_idx.lvl1][stream_idx.lvl2].frame_ptrs[i];

		// net_buf_add_mem(audio_frame, frame_ptr, frame_size);
		
		meta->locations |=
			(uint32_t)subgroups[stream_idx.lvl1][stream_idx.lvl2].location[i];
	}

	// Modified code
	net_buf_add_mem(audio_frame,
                audio_frame_out->data,
                audio_frame_out->len);

	meta->data_coding = LC3;

	struct bt_audio_codec_cfg *codec_cfg =
		&subgroups[stream_idx.lvl1][stream_idx.lvl2].group_lc3_preset.codec_cfg;

	ret = le_audio_freq_hz_get(codec_cfg, &meta->sample_rate_hz);
	if (ret) {
		LOG_ERR("Failed to get frequency: %d", ret);
	}

	ret = le_audio_duration_us_get(codec_cfg, &meta->data_len_us);
	if (ret) {
		LOG_ERR("Failed to get frame duration: %d", ret);
	}

	ret = broadcast_source_send(audio_frame, stream_idx.lvl1, stream_idx.lvl2);

	if (ret != 0 && ret != prev_ret) {
		if (ret == -ECANCELED) {
			LOG_WRN("Sending cancelled");
		} else {
			LOG_WRN("broadcast_source_send returned: %d", ret);
		}
	}

	net_buf_unref(audio_frame);

	prev_ret = ret;
}

[dark73]

I would like to correct my previous statement.

and SUB0 and SUB1 are displayed on the phone.

behavior to be:

• SUB0 → KOR audio 
  SUB1 → ENG audio 

However, the actual behavior is:

• SUB0 → KOR audio 
• SUB1 → KOR audio 

[Maria Gilje]

Hello, 

I just want to let you know that Øyvind is out of office and I will support you from now on. I have started to look into your code so far and I will share some initial feedback before Thursday this week. 

Just to make sure that I'm not omitting anything important: You note that all previous issues have been resolved in one of your most recent replies. Should I only focus on the content of your reply from July 18th/19th (depending on time zone)?

Best regards,

Maria

[dark73]

Yes, I have resolved all of the previous issues. Therefore, I believe you only need to focus on the content of the last two replies, specifically the discussion related to subgroup_send().

Thank you in advance for your time and feedback.