nRF5340 Audio - 6 streams broadcasting

[catalinio]

Dear Nordic team,

I'm evaluating nRF5340 Audio DK for the use-case of broadcasting multiple streams.

Until now I've succeeded in broadcasting 4 streams (LC4 bitrate set up to 48kbps) and I would like to to increase to 6. I've increased all the configuration parameters to support 6 streams, but I receive an error in initializing the broadcast source.

I'm attaching the whole debug log, but the main error is here:

GW [00:00:02.445,953] <wrn> bt_hci_core: opcode 0x2068 status 0x0d

GW [00:00:02.445,953] <dbg> bt_iso: bt_iso_big_create: Could not create BIG -5

 

Opcode 0x68 is BT_HCI_OP_LE_CREATE_BIG.

The status code 0x0D is BT_HCI_ERR_INSUFFICIENT_RESOURCES, and I've added warn log messages to all the C-code returning this value, but it's not printed. So I suspect this could be in the LE Audio controller from the Network Core (binary release). Most of the code is indeed in the BT Controller (C:\ncs\v2.2.0-rc1\zephyr\subsys\bluetooth\controller).

I'm attaching the prj.conf with the modified configuration values, of the nRF5340 Audio application.

I'm using the nRF Connect SDK v2.2.0-rc1.

I understand the LC3 encoding processing requirements are quite high, but as it takes 6 milisec for 4 streams (10milisec PCM slice, 32kbps), I suppose there is time for 6 streams @32kbps.

Please let me know if there's any more configuration values that needs to be tuned, in order for the 6 (or maybe 8) streams to be broadcasted, or maybe some timeouts increased.

Thank you!

6036.terminal_for_support.txt

[catalinio]

Here is the prj.conf

Fullscreen 5305.prj_conf.txt Download

#
# Copyright (c) 2022 Nordic Semiconductor ASA
#
# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
#

# nRF5340 Audio
CONFIG_NRF5340_AUDIO=y

# Custom config
CONFIG_AUDIO_SOURCE_I2S=y
CONFIG_TRANSPORT_BIS=y

CONFIG_FIFO_FRAME_SPLIT_NUM=10
CONFIG_FIFO_TX_FRAME_COUNT=6
CONFIG_FIFO_RX_FRAME_COUNT=1

# minimum bitrate
CONFIG_LC3_BITRATE=32000
CONFIG_BT_AUDIO_BROADCAST_16_2_1=y

# configuration ok, for 4 streams
#CONFIG_LC3_BITRATE=48000
#CONFIG_BT_AUDIO_BROADCAST_24_2_1=y

#doesn't work for 4 channels
#CONFIG_LC3_BITRATE=96000

CONFIG_AUDIO_SAMPLE_RATE_16000_HZ=y
CONFIG_AUDIO_BIT_DEPTH_16=y

CONFIG_BT_AUDIO_BROADCAST_SRC_STREAM_COUNT=6
CONFIG_BT_AUDIO_BROADCAST_SNK_STREAM_COUNT=6

CONFIG_BT_ISO_MAX_CHAN=6
CONFIG_BT_ISO_TX_BUF_COUNT=6
CONFIG_BT_ISO_TX_FRAG_COUNT=6

CONFIG_BT_EXT_ADV_MAX_ADV_SET=6
CONFIG_BT_ISO_TX_MTU=500
CONFIG_BT_ISO_MAX_BIG=2

# Bluetooth buffer configuration
# default 27
CONFIG_BT_BUF_ACL_TX_SIZE=83
#default 3
CONFIG_BT_BUF_ACL_TX_COUNT=6
#default 68
CONFIG_BT_BUF_EVT_RX_SIZE=255
#default 58
CONFIG_BT_BUF_EVT_DISCARDABLE_SIZE=255
#default 3
CONFIG_BT_BUF_EVT_DISCARDABLE_COUNT=10
#default 255
CONFIG_BT_BUF_CMD_TX_SIZE=255
#default 10
CONFIG_BT_BUF_CMD_TX_COUNT=20

CONFIG_BT_CTLR_ADV_ISO_PDU_LEN_MAX=???

#default n
CONFIG_BT_HCI_TX_STACK_SIZE_WITH_PROMPT=y
#default 1024
CONFIG_BT_HCI_TX_STACK_SIZE=2048

CONFIG_BT_AUDIO_BROADCAST_SRC_COUNT=2
#CONFIG_BT_BUF_ACL_
#CONFIG_BT_BUF_ACL_

CONFIG_BT_LIM_ADV_TIMEOUT=120

CONFIG_LC3_ENC_CHAN_MAX=6
CONFIG_ENCODER_STACK_SIZE=16384
CONFIG_AUDIO_DATAPATH_STACK_SIZE=16384

CONFIG_MAIN_LOG_LEVEL_DBG=y
CONFIG_AUDIO_SYSTEM_LOG_LEVEL_DBG=y
CONFIG_SW_CODEC_SELECT_LOG_LEVEL_DBG=y
CONFIG_BLE_LOG_LEVEL_DBG=y
CONFIG_BT_LOG_LEVEL_DBG=y
CONFIG_BT_AUDIO_DEBUG_STREAM=y
CONFIG_BT_AUDIO_DEBUG_STREAM_DATA=y
CONFIG_BT_AUDIO_DEBUG_BROADCAST_SOURCE=y

CONFIG_STREAMCTRL_LOG_LEVEL_DBG=y
CONFIG_BT_AUDIO_DEBUG_STREAM=y

CONFIG_LOG_DEFAULT_LEVEL=4
CONFIG_THREAD_ANALYZER=y

# General
CONFIG_DEBUG=y
CONFIG_DEBUG_INFO=y
CONFIG_ASSERT=y
CONFIG_STACK_USAGE=y
CONFIG_THREAD_RUNTIME_STATS=y
CONFIG_STACK_SENTINEL=y
CONFIG_INIT_STACKS=y

# Uart driver
CONFIG_SERIAL=y

# Logging
CONFIG_LOG=y
CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y
CONFIG_LOG_TAG_MAX_LEN=2
CONFIG_LOG_TAG_DEFAULT="--"
CONFIG_LOG_PROCESS_THREAD_SLEEP_MS=500
CONFIG_LOG_PROCESS_THREAD_STACK_SIZE=8192
CONFIG_LOG_BUFFER_SIZE=65536
CONFIG_LOG_SPEED=y

## Avoid redundant warnings for endpoint setting in USB stack
CONFIG_USB_DRIVER_LOG_LEVEL_ERR=y
CONFIG_USB_DEVICE_LOG_LEVEL_ERR=y

# Console related defines
CONFIG_CONSOLE=y
CONFIG_RTT_CONSOLE=y
CONFIG_UART_CONSOLE=y

# Shell related defines
CONFIG_SHELL=y
CONFIG_KERNEL_SHELL=y
CONFIG_USE_SEGGER_RTT=y
## Disable logs on RTT
CONFIG_SHELL_RTT_INIT_LOG_LEVEL_NONE=y
CONFIG_SHELL_BACKEND_RTT=y
CONFIG_SHELL_BACKEND_SERIAL=n
CONFIG_SHELL_VT100_COLORS=y
CONFIG_SHELL_STACK_SIZE=1024
CONFIG_SHELL_CMD_BUFF_SIZE=128
## Reduce shell memory usage
CONFIG_SHELL_WILDCARD=n
CONFIG_SHELL_HELP_ON_WRONG_ARGUMENT_COUNT=n
CONFIG_SHELL_STATS=n
CONFIG_SHELL_CMDS=n
CONFIG_SHELL_HISTORY=y

# Turn off default shell commands
CONFIG_I2C_SHELL=n
CONFIG_HWINFO_SHELL=n
CONFIG_CLOCK_CONTROL_NRF_SHELL=n
CONFIG_FLASH_SHELL=n
CONFIG_DEVICE_SHELL=n

# Suppress err msg from sd_check_card_type. Because SPI_SDHC has no card presence method,
# assume card is in slot. Thus error message is always shown if card is not inserted
CONFIG_SD_LOG_LEVEL_OFF=y
CONFIG_NVS_LOG_LEVEL_WRN=y

CONFIG_CPU_LOAD=y
CONFIG_CPU_LOAD_LOG_PERIODIC=y
CONFIG_CPU_LOAD_LOG_INTERVAL=2000

[dejans]

Hi,

There is a limitation on the controller side to maximum 4 BIS streams.

Best regards,
Dejan

[dejans]

Hi,

It is not supported at the moment. For any other information related to future plans and releases please contact your regional sales manager.

Best regards,
Dejan

[uzer123]

I am also interested in a similar configuration, so i find it beneficial to write here. From this thread i see that 4 BIS streams is the max supported by the BLE audio controller.

My question is: to enable the 4 BIS streams (in my use case audio quality is not important) you just use the aforementioned flags in prj.conf ? (CONFIG_BT_AUDIO_BROADCAST_24_2_1=y, CONFIG_ENCODER_STACK_SIZE=16384, CONFIG_LC3_ENC_CHAN_MAX=4)

And if this was your only change to the default source code, how do you send the 4 streams to the board?

Can i send 4 mono streams from USB or 2 mono streams through USB and 2 through LINE IN?

[catalinio]

hi  uzer123 ,

I'm glad that more people are trying to stream multiple channels, so Nordic could hear us and release a 6-8 streams BLE controller.

I have not input more than 2 audio channels in the Gateway. I've just generated the additional 2 channels (3rd and 4th), with sine-waves. Afik, I2S and USB are each limited to 2 channels, and using both would mean redesigning the application.

Basically, my experiment was to test how many channels can be LC3 encoded and next broadcasted on BLE Audio. In my test, I see that 6 channels is a doable (85% CPU usage, 48kbps bitrate), while 8 streams is a strech-goal.

My next plan is to input on UART/SPI already LC3 encoded audio streams on the Gateway, and next broadcast them, maybe trying the new multi-gateway feature (available in github, not released yet).

As for code modifications, make sure in encoder_thread() there is enough space to handle tmp_pcm_raw_data.  

Additionally, in the LC3 wrapper (file: sw_codec_selec.c), make sure LC3 is initialized with 4 channels and in the sw_codec_encode() you design a way to split/concatenate all the 4 channels.

br, catalinio

[Tommy_Li]

Cool, Looking forward to your progress on lc3 encoed audio streams transferred by UART/SPI.

[dejans]

Hi uzer123 ,

uzer123 said:

My question is: to enable the 4 BIS streams (in my use case audio quality is not important) you just use the aforementioned flags in prj.conf ? (CONFIG_BT_AUDIO_BROADCAST_24_2_1=y, CONFIG_ENCODER_STACK_SIZE=16384, CONFIG_LC3_ENC_CHAN_MAX=4)

And if this was your only change to the default source code, how do you send the 4 streams to the board?

Can i send 4 mono streams from USB or 2 mono streams through USB and 2 through LINE IN?

Please find below some suggestions from our developers.
Encoding each stream takes around 30% of the CPU time. Therefore, there might not be enough CPU time to encode 4 streams.
Current USB implementation only has 2 (stereo) endpoints. As USB endpoint is synchronous, it is controlled by PC clock. For avoiding the packet loss. synchronization of USB to I2S is required but you would be on your own in potentially achieving this.
With regard to config options, you would need to set CONFIG_BT_CTLR_CONN_ISO_STREAMS=4 and CONFIG_BT_CTLR_SYNC_ISO_STREAM_COUNT=4 in the controller's prj.conf.
In general, in trying to achieve 4 streams, issues might arise due to CPU, USB and clock sources. One could consider trying to find ways to transfer pre-encoded LC3 to the nrf5340 using for example SPI.

Best regards,
Dejan

[uzer123]

Thank you for the detailed and clear answer