nrf5340_audio_dk audio application: I2S buffer over and underruns on both gateway and headset when using I2S microphone connected to the p10 pins, no sync between gateway and headset

Hi,

We have been trying to get the audio application to work using an external I2S microphone, but have recently run into the following problem when trying to send audio data from the gateway to the headset (CIS): 

1) We don't hear audio when plugging headphones into the 'headphone' jack on the headset dk

2) When checking if the I2S microphone writes to pcm_raw_data in audio_system.c, we do in fact see that pcm_raw_data contains the correct data regarding the audio we give the I2S microphone. (This check was done before sw_codec_encode()). So we know the microphone is working properly and sending its data in the correct format.)

3) When looking at the LOGs for the gateway device, we get this:

Gateway:
*** Booting nRF Connect SDK v3.5.99-ncs1 ***
-- [00:00:00.261,474] <inf> main: Testing the lowpass filter - response
-- [00:00:00.515,625] <inf> main: signal amplitude at Hz: 100.000000 999
-- [00:00:00.769,836] <inf> main: signal amplitude at Hz: 1100.000000 955
-- [00:00:01.023,956] <inf> main: signal amplitude at Hz: 2100.000000 532
-- [00:00:01.278,167] <inf> main: signal amplitude at Hz: 3100.000000 258
-- [00:00:01.532,318] <inf> main: signal amplitude at Hz: 4100.000000 157
-- [00:00:01.786,468] <inf> main: signal amplitude at Hz: 5100.000000 114
-- [00:00:02.040,679] <inf> main: signal amplitude at Hz: 6100.000000 85
-- [00:00:02.294,799] <inf> main: signal amplitude at Hz: 7100.000000 64
-- [00:00:02.549,011] <inf> main: signal amplitude at Hz: 8100.000000 48
GW [00:00:02.803,131] <inf> main: signal amplitude at Hz: 9100.000000 34
GW [00:00:02.814,331] <inf> board_version: Compatible board/HW version found: 1.0.0
GW [00:00:02.841,033] <inf> fw_info: 
         nRF5340 Audio nRF5340 Audio DK cpuapp                      
         NCS base version: 2.6.0                            
         Cmake run : Sun Apr 28 21:37:39 2024
GW [00:00:02.841,033] <inf> fw_info: ------- DEBUG BUILD -------
GW [00:00:02.841,033] <inf> fw_info: Compiled for GATEWAY device
GW [00:00:02.860,137] <inf> bt_mgmt_ctlr_cfg: Controller: SoftDevice: Version 5.4 (0x0d), Revision 8591
GW [00:00:02.860,321] <inf> bt_mgmt: Local identity addr: D6:15:82:71:32:16 (random)
GW [00:00:02.866,027] <inf> bt_mgmt_scan: Local addr: 65:80:1D:F5:82:A4 (random). May time out. Updates not printed
GW [00:00:02.866,027] <inf> bt_mgmt_scan: Scanning successfully started
GW [00:00:02.927,215] <inf> bt_mgmt_scan: Creating connection to device: C4:1E:CF:8C:97:90 (random)
GW [00:00:03.030,548] <inf> bt_mgmt: Connected: C4:1E:CF:8C:97:90 (random)
GW [00:00:03.032,043] <inf> bt_mgmt_scan: Local addr: 65:80:1D:F5:82:A4 (random). May time out. Updates not printed
GW [00:00:03.032,043] <inf> bt_mgmt_scan: Scanning successfully started
GW [00:00:03.032,073] <inf> main: Device connected
GW [00:00:03.311,981] <inf> main: Security changed
GW [00:00:04.071,807] <inf> bt_rend_vol: VCS discover finished
GW [00:00:05.953,216] <inf> unicast_client: LEFT sink stream configured
GW [00:00:06.032,684] <inf> unicast_client: Enable stream 0x20002ab0
GW [00:00:06.352,111] <inf> unicast_client: Stream 0x20002ab0 started
GW [00:00:06.367,736] <wrn> audio_datapath: I2S RX overrun. Single msg
--- 1 messages dropped ---
GW [00:00:06.401,245] <wrn> audio_datapath: I2S RX continuing stream
--- 22 messages dropped ---
GW [00:00:06.485,260] <wrn> audio_datapath: I2S RX overrun. Single msg
--- 20 messages dropped ---
GW [00:00:06.546,600] <wrn> bt_le_audio_tx: HCI ISO TX overrun on stream 0x20002ab0 - Single print
--- 21 messages dropped ---
GW [00:00:06.603,240] <wrn> audio_datapath: I2S RX overrun. Single msg
--- 16 messages dropped ---
GW [00:00:06.656,250] <wrn> audio_datapath: I2S RX continuing stream
--- 14 messages dropped ---
GW [00:00:06.712,249] <wrn> audio_datapath: I2S RX continuing stream
--- 15 messages dropped ---
GW [00:00:06.768,249] <wrn> audio_datapath: I2S RX continuing stream
--- 15 messages dropped ---
GW [00:00:06.821,258] <wrn> audio_datapath: I2S RX overrun. Single msg
--- 13 messages dropped ---
GW [00:00:06.887,756] <wrn> bt_le_audio_tx: HCI ISO TX overrun on stream 0x20002ab0 - Single print
--- 20 messages dropped ---
GW [00:00:06.966,247] <wrn> audio_datapath: I2S RX overrun. Single msg
--- 18 messages dropped ---
GW [00:00:07.029,266] <wrn> audio_datapath: I2S RX overrun. Single msg
--- 18 messages dropped ---
GW [00:00:07.092,254] <wrn> audio_datapath: I2S RX overrun. Single msg
--- 16 messages dropped ---
GW [00:00:07.156,768] <wrn> bt_le_audio_tx: HCI ISO TX overrun on stream 0x20002ab0 - Single print
--- 21 messages dropped ---
GW [00:00:07.228,271] <wrn> audio_datapath: I2S RX continuing stream
--- 16 messages dropped ---
GW [00:00:07.290,771] <wrn> audio_datapath: I2S RX continuing stream
--- 15 messages dropped ---
GW [00:00:07.337,249] <wrn> audio_datapath: I2S RX continuing stream
--- 17 messages dropped ---
GW [00:00:07.407,745] <wrn> audio_datapath: I2S RX continuing stream
--- 14 messages dropped ---
GW [00:00:07.472,259] <wrn> audio_datapath: I2S RX continuing stream

4) When looking at the LOGs for the headset device, we get this:

Headset:
HL [00:00:00.300,109] <inf> fw_info: ------- DEBUG BUILD -------
HL [00:00:00.300,140] <inf> fw_info: HEADSET left device
HL [00:00:00.320,129] <inf> bt_mgmt_ctlr_cfg: Controller: SoftDevice: Version 5.4 (0x0d), Revision 8591
HL [00:00:00.320,343] <inf> bt_mgmt: Local identity addr: C4:1E:CF:8C:97:90 (random)
HL [00:00:00.327,911] <inf> bt_mgmt_adv: Local addr: 57:3E:83:9C:E5:04 (random)
HL [00:00:00.328,002] <inf> bt_mgmt_adv: Adv directed to: D6:15:82:71:32:16 (random).
HL [00:00:00.328,613] <inf> bt_mgmt_adv: Advertising successfully started
HL [00:00:00.604,309] <inf> bt_mgmt: Connected: D6:15:82:71:32:16 (random)
HL [00:00:00.604,339] <inf> main: Connected
HL [00:00:00.604,919] <inf> bt_mgmt_adv: RPA (Resolvable Private Address) expired.
HL [00:00:00.605,010] <inf> bt_mgmt_adv: Local addr: 57:3E:83:9C:E5:04 (random)
HL [00:00:00.844,970] <inf> main: Security changed
HL [00:00:00.845,245] <wrn> bt_gatt: Device is not subscribed to characteristic
HL [00:00:00.845,245] <wrn> bt_pacs: PACS notify failed: -22
HL [00:00:02.845,581] <inf> bt_content_ctrl_media: Discovery of MCS finished
HL [00:00:03.325,622] <inf> unicast_server: LC3 codec config for sink:
HL [00:00:03.325,622] <inf> unicast_server:     Frequency: 24000 Hz
HL [00:00:03.325,622] <inf> unicast_server:     Duration: 10000 us
HL [00:00:03.325,622] <inf> unicast_server:     Channel allocation: 0x1
HL [00:00:03.325,622] <inf> unicast_server:     Octets per frame: 60 (48000 bps)
HL [00:00:03.325,622] <inf> unicast_server:     Frames per SDU: 1
HL [00:00:03.405,517] <inf> main: Presentation delay 10000 us is set by initiator
HL [00:00:03.732,055] <inf> unicast_server: Stream 0x20014430 started
HL [00:00:03.743,621] <wrn> audio_datapath: Data received, total under-runs: 13
HL [00:00:03.753,601] <wrn> audio_datapath: Data received, total under-runs: 23
HL [00:00:03.763,610] <wrn> audio_datapath: Data received, total under-runs: 33
HL [00:00:03.773,620] <wrn> audio_datapath: Data received, total under-runs: 43
HL [00:00:03.782,104] <wrn> audio_datapath: Data received, total under-runs: 50
HL [00:00:03.793,121] <wrn> audio_datapath: Data received, total under-runs: 52
HL [00:00:03.803,100] <wrn> audio_datapath: Data received, total under-runs: 62
HL [00:00:03.813,110] <wrn> audio_datapath: Data received, total under-runs: 72
HL [00:00:03.822,601] <wrn> audio_datapath: Data received, total under-runs: 81
HL [00:00:03.832,611] <wrn> audio_datapath: Data received, total under-runs: 91
HL [00:00:03.842,620] <wrn> audio_datapath: Data received, total under-runs: 101
HL [00:00:03.852,600] <wrn> audio_datapath: Data received, total under-runs: 111
HL [00:00:03.862,609] <wrn> audio_datapath: Data received, total under-runs: 121
HL [00:00:03.872,619] <wrn> audio_datapath: Data received, total under-runs: 131
HL [00:00:03.882,598] <wrn> audio_datapath: Data received, total under-runs: 141
HL [00:00:03.892,608] <wrn> audio_datapath: Data received, total under-runs: 151
HL [00:00:03.902,618] <wrn> audio_datapath: Data received, total under-runs: 161
HL [00:00:03.912,597] <wrn> audio_datapath: Data received, total under-runs: 171
HL [00:00:03.922,607] <wrn> audio_datapath: Data received, total under-runs: 181
HL [00:00:03.932,617] <wrn> audio_datapath: Data received, total under-runs: 191
HL [00:00:03.942,596] <wrn> audio_datapath: Data received, total under-runs: 201
HL [00:00:03.952,606] <wrn> audio_datapath: Data received, total under-runs: 211
HL [00:00:03.962,615] <wrn> audio_datapath: Data received, total under-runs: 221
HL [00:00:03.972,595] <wrn> audio_datapath: Data received, total under-runs: 231
HL [00:00:03.982,604] <wrn> audio_datapath: Data received, total under-runs: 241
HL [00:00:03.992,614] <wrn> audio_datapath: Data received, total under-runs: 251
HL [00:00:04.002,624] <wrn> audio_datapath: Data received, total under-runs: 261
HL [00:00:04.012,603] <wrn> audio_datapath: Data received, total under-runs: 271
HL [00:00:04.022,613] <wrn> audio_datapath: Data received, total under-runs: 281
HL [00:00:04.032,623] <wrn> audio_datapath: Data received, total under-runs: 291
HL [00:00:04.042,602] <wrn> audio_datapath: Data received, total under-runs: 301
HL [00:00:04.052,612] <wrn> audio_datapath: Data received, total under-runs: 311
HL [00:00:04.062,622] <wrn> audio_datapath: Data received, total under-runs: 321
HL [00:00:04.072,601] <wrn> audio_datapath: Data received, total under-runs: 331
HL [00:00:04.082,611] <wrn> audio_datapath: Data received, total under-runs: 341
HL [00:00:04.092,620] <wrn> audio_datapath: Data received, total under-runs: 351
HL [00:00:04.103,118] <wrn> audio_datapath: Data received, total under-runs: 362
HL [00:00:04.113,098] <wrn> audio_datapath: Data received, total under-runs: 372
HL [00:00:04.123,107] <wrn> audio_datapath: Data received, total under-runs: 382
HL [00:00:04.133,117] <wrn> audio_datapath: Data received, total under-runs: 392

The following LEDs (apart from the big one in the center that is green on gateway and blue on headset) are on:

on the gateway: OB/EXT is solid orange (because we use the p10 pins), LED3 is blinking green (app core is running), LED1 is blinking blue.

on the headset: LED3 is blinking green, LED1 is blinking blue.

LED2 is turning on on neither of the boards, so there appears to be a sync issue.

We have made the following modifications to the code:

in unicast client main.c, to use the p10 pins:

nrf_gpio_cfg_output(21);
nrf_gpio_pin_set(21);


in audio_i2s.c, because we are using the adafruit mems microphone which uses 24 in 32 bit: https://cdn-shop.adafruit.com/product-files/3421/i2S+Datasheet.PDF, and because our audio sample rate is 24 kHz:

#if CONFIG_AUDIO_SAMPLE_RATE_16000_HZ
#define CONFIG_AUDIO_RATIO NRF_I2S_RATIO_384X
#elif CONFIG_AUDIO_SAMPLE_RATE_24000_HZ
#define CONFIG_AUDIO_RATIO NRF_I2S_RATIO_256X
#elif CONFIG_AUDIO_SAMPLE_RATE_48000_HZ
#define CONFIG_AUDIO_RATIO NRF_I2S_RATIO_128X
#else
#error "Current AUDIO_SAMPLE_RATE_HZ setting not supported"
#endif

static nrfx_i2s_t i2s_inst = NRFX_I2S_INSTANCE(0);

static nrfx_i2s_config_t cfg = {
	/* Pins are configured by pinctrl. */
	.skip_gpio_cfg = true,
	.skip_psel_cfg = true,
	.irq_priority = DT_IRQ(I2S_NL, priority),
	.mode = NRF_I2S_MODE_MASTER,
	.format = NRF_I2S_FORMAT_I2S,
	.alignment = NRF_I2S_ALIGN_LEFT,
	.ratio = NRF_I2S_RATIO_64X,
	.mck_setup = 954433536,
#if (CONFIG_AUDIO_BIT_DEPTH_16)
	.sample_width = NRF_I2S_SWIDTH_16BIT,
#elif (CONFIG_AUDIO_BIT_DEPTH_32)
//  .sample_width = NRF_I2S_SWIDTH_32BIT,
	.sample_width = NRF_I2S_SWIDTH_24BIT_IN32BIT,
#else
#error Invalid bit depth selected
#endif /* (CONFIG_AUDIO_BIT_DEPTH_16) */
	.channels = NRF_I2S_CHANNELS_STEREO,
	.clksrc = NRF_I2S_CLKSRC_ACLK,
	.enable_bypass = false,
};

We also made every function in hardware_codec.c return 0 immediately so we could use our external microphone:

/*
 * Copyright (c) 2018 Nordic Semiconductor ASA
 *
 * SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
 */

#include "hw_codec.h"

#include <zephyr/kernel.h>
#include <stdlib.h>
#include <stdint.h>
#include <ctype.h>
#include <zephyr/drivers/gpio.h>
#include <zephyr/device.h>
#include <zephyr/shell/shell.h>
#include <zephyr/zbus/zbus.h>

#include "macros_common.h"
#include "nrf5340_audio_common.h"
#include "cs47l63.h"
#include "cs47l63_spec.h"
#include "cs47l63_reg_conf.h"
#include "cs47l63_comm.h"

#include <zephyr/logging/log.h>
LOG_MODULE_REGISTER(hw_codec, CONFIG_MODULE_HW_CODEC_LOG_LEVEL);

#define VOLUME_ADJUST_STEP_DB 3
#define BASE_10		      10

ZBUS_SUBSCRIBER_DEFINE(volume_evt_sub, CONFIG_VOLUME_MSG_SUB_QUEUE_SIZE);

static uint32_t prev_volume_reg_val = OUT_VOLUME_DEFAULT;

static cs47l63_t cs47l63_driver;

static k_tid_t volume_msg_sub_thread_id;
static struct k_thread volume_msg_sub_thread_data;

K_THREAD_STACK_DEFINE(volume_msg_sub_thread_stack, CONFIG_VOLUME_MSG_SUB_STACK_SIZE);

/**
 * @brief	Convert the zbus volume to the actual volume setting for the HW codec.
 *
 * @note	The range for zbus volume is from 0 to 255 and the
 *		range for HW codec volume is from 0 to 128.
 */
static uint16_t zbus_vol_conversion(uint8_t volume)
{
	return 0;

	return (((uint16_t)volume + 1) / 2);
}

/**
 * @brief	Handle volume events from zbus.
 */
static void volume_msg_sub_thread(void)
{
	return 0;

	int ret;

	const struct zbus_channel *chan;

	while (1) {
		ret = zbus_sub_wait(&volume_evt_sub, &chan, K_FOREVER);
		ERR_CHK(ret);

		struct volume_msg msg;

		ret = zbus_chan_read(chan, &msg, ZBUS_READ_TIMEOUT_MS);
		if (ret) {
			LOG_ERR("Failed to read from zbus: %d", ret);
		}

		uint8_t event = msg.event;
		uint8_t volume = msg.volume;

		LOG_DBG("Received event = %d, volume = %d", event, volume);

		switch (event) {

/*
dit is waar de button presses verwerkt worden --> we zitten niet in de interrupt/tijdscritische code
dus kunnen hier de leds wa late knipperen enzo :)

*/

		case VOLUME_UP:
			LOG_DBG("Volume up received");
			ret = hw_codec_volume_increase();
			if (ret) {
				LOG_ERR("Failed to increase volume, ret: %d", ret);
			}
			break;
		case VOLUME_DOWN:
			LOG_DBG("Volume down received");
			ret = hw_codec_volume_decrease();
			if (ret) {
				LOG_ERR("Failed to decrease volume, ret: %d", ret);
			}
			break;
		case VOLUME_SET:
			LOG_DBG("Volume set received");
			ret = hw_codec_volume_set(zbus_vol_conversion(volume));
			if (ret) {
				LOG_ERR("Failed to set the volume to %d, ret: %d", volume, ret);
			}
			break;
		case VOLUME_MUTE:
			LOG_DBG("Volume mute received");
			ret = hw_codec_volume_mute();
			if (ret) {
				LOG_ERR("Failed to mute volume, ret: %d", ret);
			}
			break;
		case VOLUME_UNMUTE:
			LOG_DBG("Volume unmute received");
			ret = hw_codec_volume_unmute();
			if (ret) {
				LOG_ERR("Failed to unmute volume, ret: %d", ret);
			}
			break;
		default:
			LOG_WRN("Unexpected/unhandled volume event: %d", event);
			break;
		}

		STACK_USAGE_PRINT("volume_msg_thread", &volume_msg_sub_thread_data);
	}
}

/**
 * @brief Write to multiple registers in CS47L63.
 */
static int cs47l63_comm_reg_conf_write(const uint32_t config[][2], uint32_t num_of_regs)
{
	return 0;

	int ret;
	uint32_t reg;
	uint32_t value;

	for (int i = 0; i < num_of_regs; i++) {
		reg = config[i][0];
		value = config[i][1];

		if (reg == SPI_BUSY_WAIT) {
			LOG_DBG("Busy waiting instead of writing to CS47L63");
			/* Wait for us defined in value */
			k_busy_wait(value);
		} else {
			ret = cs47l63_write_reg(&cs47l63_driver, reg, value);
			if (ret) {
				return ret;
			}
		}
	}

	return 0;
}

int hw_codec_volume_set(uint8_t set_val)
{
	return 0;

	int ret;
	uint32_t volume_reg_val;

	volume_reg_val = set_val;
	if (volume_reg_val == 0) {
		LOG_WRN("Volume at MIN (-64dB)");
	} else if (volume_reg_val >= MAX_VOLUME_REG_VAL) {
		LOG_WRN("Volume at MAX (0dB)");
		volume_reg_val = MAX_VOLUME_REG_VAL;
	}

	ret = cs47l63_write_reg(&cs47l63_driver, CS47L63_OUT1L_VOLUME_1,
				volume_reg_val | CS47L63_OUT_VU);
	if (ret) {
		return ret;
	}

	prev_volume_reg_val = volume_reg_val;

	/* This is rounded down to nearest integer */
	LOG_DBG("Volume: %" PRId32 " dB", (volume_reg_val / 2) - MAX_VOLUME_DB);

	return 0;
}

int hw_codec_volume_adjust(int8_t adjustment_db)
{
	return 0;

	int ret;
	int32_t new_volume_reg_val;

	LOG_DBG("Adj dB in: %d", adjustment_db);

	if (adjustment_db == 0) {
		new_volume_reg_val = prev_volume_reg_val;
	} else {
		uint32_t volume_reg_val;

		ret = cs47l63_read_reg(&cs47l63_driver, CS47L63_OUT1L_VOLUME_1, &volume_reg_val);
		if (ret) {
			LOG_ERR("Failed to get volume from CS47L63");
			return ret;
		}

		volume_reg_val &= CS47L63_OUT1L_VOL_MASK;

		/* The adjustment is in dB, 1 bit equals 0.5 dB,
		 * so multiply by 2 to get increments of 1 dB
		 */
		new_volume_reg_val = volume_reg_val + (adjustment_db * 2);
		if (new_volume_reg_val <= 0) {
			LOG_WRN("Volume at MIN (-64dB)");
			new_volume_reg_val = 0;
		} else if (new_volume_reg_val >= MAX_VOLUME_REG_VAL) {
			LOG_WRN("Volume at MAX (0dB)");
			new_volume_reg_val = MAX_VOLUME_REG_VAL;
		}
	}

	ret = hw_codec_volume_set(new_volume_reg_val);
	if (ret) {
		return ret;
	}

	return 0;
}

int hw_codec_volume_decrease(void)
{
	return 0;

	int ret;

	ret = hw_codec_volume_adjust(-VOLUME_ADJUST_STEP_DB);
	if (ret) {
		return ret;
	}

	return 0;
}

int hw_codec_volume_increase(void)
{
	return 0;

	int ret;

	ret = hw_codec_volume_adjust(VOLUME_ADJUST_STEP_DB);
	if (ret) {
		return ret;
	}

	return 0;
}

int hw_codec_volume_mute(void)
{
	return 0;

	int ret;
	uint32_t volume_reg_val;

	ret = cs47l63_read_reg(&cs47l63_driver, CS47L63_OUT1L_VOLUME_1, &volume_reg_val);
	if (ret) {
		return ret;
	}

	BIT_SET(volume_reg_val, CS47L63_OUT1L_MUTE_MASK);

	ret = cs47l63_write_reg(&cs47l63_driver, CS47L63_OUT1L_VOLUME_1,
				volume_reg_val | CS47L63_OUT_VU);
	if (ret) {
		return ret;
	}

	return 0;
}

int hw_codec_volume_unmute(void)
{
	return 0;

	int ret;
	uint32_t volume_reg_val;

	ret = cs47l63_read_reg(&cs47l63_driver, CS47L63_OUT1L_VOLUME_1, &volume_reg_val);
	if (ret) {
		return ret;
	}

	BIT_CLEAR(volume_reg_val, CS47L63_OUT1L_MUTE_MASK);

	ret = cs47l63_write_reg(&cs47l63_driver, CS47L63_OUT1L_VOLUME_1,
				volume_reg_val | CS47L63_OUT_VU);
	if (ret) {
		return ret;
	}

	return 0;
}

int hw_codec_default_conf_enable(void)
{
	return 0;

	int ret;

	ret = cs47l63_comm_reg_conf_write(clock_configuration, ARRAY_SIZE(clock_configuration));
	if (ret) {
		return ret;
	}

	ret = cs47l63_comm_reg_conf_write(GPIO_configuration, ARRAY_SIZE(GPIO_configuration));
	if (ret) {
		return ret;
	}

	ret = cs47l63_comm_reg_conf_write(asp1_enable, ARRAY_SIZE(asp1_enable));
	if (ret) {
		return ret;
	}

	ret = cs47l63_comm_reg_conf_write(output_enable, ARRAY_SIZE(output_enable));
	if (ret) {
		return ret;
	}

	ret = hw_codec_volume_adjust(0);
	if (ret) {
		return ret;
	}

#if ((CONFIG_AUDIO_DEV == GATEWAY) && (CONFIG_AUDIO_SOURCE_I2S))
	if (IS_ENABLED(CONFIG_WALKIE_TALKIE_DEMO)) {
		ret = cs47l63_comm_reg_conf_write(pdm_mic_enable_configure,
						  ARRAY_SIZE(pdm_mic_enable_configure));
		if (ret) {
			return ret;
		}
	} else {
		ret = cs47l63_comm_reg_conf_write(line_in_enable, ARRAY_SIZE(line_in_enable));
		if (ret) {
			return ret;
		}
	}
#endif /* ((CONFIG_AUDIO_DEV == GATEWAY) && (CONFIG_AUDIO_SOURCE_I2S)) */

#if ((CONFIG_AUDIO_DEV == HEADSET) && CONFIG_STREAM_BIDIRECTIONAL)
	ret = cs47l63_comm_reg_conf_write(pdm_mic_enable_configure,
					  ARRAY_SIZE(pdm_mic_enable_configure));
	if (ret) {
		return ret;
	}
#endif /* ((CONFIG_AUDIO_DEV == HEADSET) && CONFIG_STREAM_BIDIRECTIONAL) */

	/* Toggle FLL to start up CS47L63 */
	ret = cs47l63_comm_reg_conf_write(FLL_toggle, ARRAY_SIZE(FLL_toggle));
	if (ret) {
		return ret;
	}

	return 0;
}

int hw_codec_soft_reset(void)
{
	return 0;

	int ret;

	ret = cs47l63_comm_reg_conf_write(output_disable, ARRAY_SIZE(output_disable));
	if (ret) {
		return ret;
	}

	ret = cs47l63_comm_reg_conf_write(soft_reset, ARRAY_SIZE(soft_reset));
	if (ret) {
		return ret;
	}

	return 0;
}

int hw_codec_init(void)
{
	return 0;

	int ret;

	ret = cs47l63_comm_init(&cs47l63_driver);
	if (ret) {
		return ret;
	}

	/* Run a soft reset on start to make sure all registers are default values */
	ret = cs47l63_comm_reg_conf_write(soft_reset, ARRAY_SIZE(soft_reset));
	if (ret) {
		return ret;
	}
	cs47l63_driver.state = CS47L63_STATE_STANDBY;

	volume_msg_sub_thread_id = k_thread_create(
		&volume_msg_sub_thread_data, volume_msg_sub_thread_stack,
		CONFIG_VOLUME_MSG_SUB_STACK_SIZE, (k_thread_entry_t)volume_msg_sub_thread, NULL,
		NULL, NULL, K_PRIO_PREEMPT(CONFIG_VOLUME_MSG_SUB_THREAD_PRIO), 0, K_NO_WAIT);
	ret = k_thread_name_set(volume_msg_sub_thread_id, "VOLUME_MSG_SUB");
	ERR_CHK(ret);

	return 0;
}

static int cmd_input(const struct shell *shell, size_t argc, char **argv)
{
	return 0;

	int ret;
	uint8_t idx;

	enum hw_codec_input {
		LINE_IN,
		PDM_MIC,
		NUM_INPUTS,
	};

	if (argc != 2) {
		shell_error(shell, "Only one argument required, provided: %d", argc);
		return -EINVAL;
	}

	if ((CONFIG_AUDIO_DEV == GATEWAY) && IS_ENABLED(CONFIG_AUDIO_SOURCE_USB)) {
		shell_error(shell, "Can't select PDM mic if audio source is USB");
		return -EINVAL;
	}

	if ((CONFIG_AUDIO_DEV == HEADSET) && !IS_ENABLED(CONFIG_STREAM_BIDIRECTIONAL)) {
		shell_error(shell, "Can't select input if headset is not in bidirectional stream");
		return -EINVAL;
	}

	if (!isdigit((int)argv[1][0])) {
		shell_error(shell, "Supplied argument is not numeric");
		return -EINVAL;
	}

	idx = strtoul(argv[1], NULL, BASE_10);

	switch (idx) {
	case LINE_IN: {
		if (CONFIG_AUDIO_DEV == HEADSET) {
			ret = cs47l63_comm_reg_conf_write(line_in_enable,
							  ARRAY_SIZE(line_in_enable));
			if (ret) {
				shell_error(shell, "Failed to enable LINE-IN");
				return ret;
			}
		}

		ret = cs47l63_write_reg(&cs47l63_driver, CS47L63_ASP1TX1_INPUT1, 0x800012);
		if (ret) {
			shell_error(shell, "Failed to route LINE-IN to I2S");
			return ret;
		}

		ret = cs47l63_write_reg(&cs47l63_driver, CS47L63_ASP1TX2_INPUT1, 0x800013);
		if (ret) {
			shell_error(shell, "Failed to route LINE-IN to I2S");
			return ret;
		}

		shell_print(shell, "Selected LINE-IN as input");
		break;
	}
	case PDM_MIC: {
		if (CONFIG_AUDIO_DEV == GATEWAY) {
			ret = cs47l63_comm_reg_conf_write(pdm_mic_enable_configure,
							  ARRAY_SIZE(pdm_mic_enable_configure));
			if (ret) {
				shell_error(shell, "Failed to enable PDM mic");
				return ret;
			}
		}

		ret = cs47l63_write_reg(&cs47l63_driver, CS47L63_ASP1TX1_INPUT1, 0x800010);
		if (ret) {
			shell_error(shell, "Failed to route PDM mic to I2S");
			return ret;
		}

		ret = cs47l63_write_reg(&cs47l63_driver, CS47L63_ASP1TX2_INPUT1, 0x800011);
		if (ret) {
			shell_error(shell, "Failed to route PDM mic to I2S");
			return ret;
		}

		shell_print(shell, "Selected PDM mic as input");
		break;
	}
	default:
		shell_error(shell, "Invalid input");
		return -EINVAL;
	}

	return 0;
}

SHELL_STATIC_SUBCMD_SET_CREATE(hw_codec_cmd,
			       SHELL_COND_CMD(CONFIG_SHELL, input, NULL,
					      " Select input\n\t0: LINE_IN\n\t\t1: PDM_MIC",
					      cmd_input),
			       SHELL_SUBCMD_SET_END);

SHELL_CMD_REGISTER(hw_codec, &hw_codec_cmd, "Change settings on HW codec", NULL);

and finally our prj.conf:

#
# Copyright (c) 2022 Nordic Semiconductor ASA
#
# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
#

# nRF5340 Audio
CONFIG_NRF5340_AUDIO=y

CONFIG_SAMPLE_RATE_CONVERTER=y
CONFIG_SAMPLE_RATE_CONVERTER_FILTER_SIMPLE=y

# General
CONFIG_DEBUG=y
CONFIG_DEBUG_INFO=y
CONFIG_ASSERT=y
CONFIG_STACK_USAGE=y
CONFIG_THREAD_RUNTIME_STATS=y
CONFIG_STACK_SENTINEL=y
CONFIG_INIT_STACKS=y

# Uart driver
CONFIG_SERIAL=y

# Logging
CONFIG_LOG=y
CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y
CONFIG_LOG_TAG_MAX_LEN=2
CONFIG_LOG_TAG_DEFAULT="--"
CONFIG_LOG_BACKEND_UART=y

# Use this for debugging thread usage
#CONFIG_LOG_THREAD_ID_PREFIX=y

# Console related defines
CONFIG_CONSOLE=y
CONFIG_RTT_CONSOLE=y
CONFIG_UART_CONSOLE=y

# Shell related defines
CONFIG_SHELL=y
CONFIG_KERNEL_SHELL=y
CONFIG_USE_SEGGER_RTT=y
## Disable logs on RTT
CONFIG_SHELL_RTT_INIT_LOG_LEVEL_NONE=y
CONFIG_SHELL_BACKEND_RTT=y
CONFIG_SHELL_BACKEND_SERIAL=n
CONFIG_SHELL_VT100_COMMANDS=y
CONFIG_SHELL_VT100_COLORS=y
CONFIG_SHELL_STACK_SIZE=4096
CONFIG_SHELL_CMD_BUFF_SIZE=128
## Reduce shell memory usage
CONFIG_SHELL_WILDCARD=n
CONFIG_SHELL_HELP_ON_WRONG_ARGUMENT_COUNT=n
CONFIG_SHELL_STATS=n
CONFIG_SHELL_CMDS=n
CONFIG_SHELL_HISTORY=y

# Turn off default shell commands
CONFIG_I2C_SHELL=n
CONFIG_HWINFO_SHELL=n
CONFIG_CLOCK_CONTROL_NRF_SHELL=n
CONFIG_FLASH_SHELL=n
CONFIG_DEVICE_SHELL=n

# Suppress LOG_ERR messages from sd_check_card_type. Because SPI_SDHC has no card presence method,
# assume card is in slot. Thus error message is always shown if card is not inserted
CONFIG_SD_LOG_LEVEL_OFF=y

# Suppress LOG_INF messages from hci_core
CONFIG_BT_HCI_CORE_LOG_LEVEL_WRN=y

# Walkie talkie demo enable
# CONFIG_WALKIE_TALKIE_DEMO=y

# Build as headset = 1, Build as gateway = 2
# CONFIG_AUDIO_DEV=2


# configs for dsp
CONFIG_CMSIS_DSP=y
CONFIG_FPU=y
CONFIG_NEWLIB_LIBC=y
CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y
CONFIG_FPU_SHARING=y

# bit depth op 32 zetten
CONFIG_AUDIO_BIT_DEPTH_32=y

# Nodige configs om de sample rate naar 24khz te zetten + I2S enable, necessary for microphones
CONFIG_AUDIO_SAMPLE_RATE_24000_HZ=y
CONFIG_BT_AUDIO_PREF_SAMPLE_RATE_24KHZ=y
CONFIG_AUDIO_SOURCE_I2S=y


# stack overflow errors ==> stack bigger
CONFIG_MAIN_STACK_SIZE=16384
CONFIG_ENCODER_STACK_SIZE=48000
CONFIG_AUDIO_DATAPATH_STACK_SIZE=48000

# error: sample rate convertor : GW [00:03:29.940,399] <err> sw_codec_select: Failed to convert sample rate: -22
# GW [00:03:29.960,418] <err> bt_le_audio_tx: The encoded data size does not match the SDU size
# GW [00:03:30.030,578] <err> sample_rate_converter: Conversion process will produce more bytes than the out
CONFIG_AUDIO_MAX_PRES_DLY_US=120000
CONFIG_SAMPLE_RATE_CONVERTER_BLOCK_SIZE_MAX=1920
CONFIG_SAMPLE_RATE_CONVERTER_BIT_DEPTH_32=y

What is wrong with our configurations and what should we be changing to get our setup to work? The end goal is to transmit the audio data via BIS, however changing to BIS in prj.conf did not solve our problem, so for now we are still using CIS. 

  • Hi, 

    There are some suggestions from the team

    I2S RX overrun means the gateway got too much data from the MIC, and too much data needs to be sent over the ISO channel, which would cause an ISO TX overrun. 

    Please make sure the default project without using custom MIC can work in 24KHz and use a logic analyzer to check if the MIC is working as expected. Also, check the I2S bus from the gateway side, to see if the configuration can meet the description in the datasheet

       

    Regards,
    Amanda H.

  • Hi Amanda, thanks for the fast response.

    We checked the I2S bus from the gateway side with a logic analyzer and the MIC is working as expected. The samples are read correctly.

    We will check if the default project without using the custom MIC can work in 24KHz and get back to you.

  • Hi again Amanda,

    thanks to your suggestion of checking if the default project worked in 24kHz we found the issue and updated our files accordingly. It now works using BIS as well. We thought we had to change the .ratio and .mck_setup in audio._2s.c,  but this was an oversight on our part and it turns out the default settings work fine with our microphone. We also had to change a couple things in prj.conf (lines 111 to 120):

    #
    # Copyright (c) 2022 Nordic Semiconductor ASA
    #
    # SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
    #
    
    # nRF5340 Audio
    CONFIG_NRF5340_AUDIO=y
    
    CONFIG_SAMPLE_RATE_CONVERTER=y
    CONFIG_SAMPLE_RATE_CONVERTER_FILTER_SIMPLE=y
    
    # General
    CONFIG_DEBUG=y
    CONFIG_DEBUG_INFO=y
    CONFIG_ASSERT=y
    CONFIG_STACK_USAGE=y
    CONFIG_THREAD_RUNTIME_STATS=y
    CONFIG_STACK_SENTINEL=y
    CONFIG_INIT_STACKS=y
    
    # Uart driver
    CONFIG_SERIAL=y
    
    # Logging
    CONFIG_LOG=y
    CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y
    CONFIG_LOG_TAG_MAX_LEN=2
    CONFIG_LOG_TAG_DEFAULT="--"
    CONFIG_LOG_BACKEND_UART=y
    
    # Use this for debugging thread usage
    #CONFIG_LOG_THREAD_ID_PREFIX=y
    
    # Console related defines
    CONFIG_CONSOLE=y
    CONFIG_RTT_CONSOLE=y
    CONFIG_UART_CONSOLE=y
    
    # Shell related defines
    CONFIG_SHELL=y
    CONFIG_KERNEL_SHELL=y
    CONFIG_USE_SEGGER_RTT=y
    ## Disable logs on RTT
    CONFIG_SHELL_RTT_INIT_LOG_LEVEL_NONE=y
    CONFIG_SHELL_BACKEND_RTT=y
    CONFIG_SHELL_BACKEND_SERIAL=n
    CONFIG_SHELL_VT100_COMMANDS=y
    CONFIG_SHELL_VT100_COLORS=y
    CONFIG_SHELL_STACK_SIZE=4096
    CONFIG_SHELL_CMD_BUFF_SIZE=128
    ## Reduce shell memory usage
    CONFIG_SHELL_WILDCARD=n
    CONFIG_SHELL_HELP_ON_WRONG_ARGUMENT_COUNT=n
    CONFIG_SHELL_STATS=n
    CONFIG_SHELL_CMDS=n
    CONFIG_SHELL_HISTORY=y
    
    # Turn off default shell commands
    CONFIG_I2C_SHELL=n
    CONFIG_HWINFO_SHELL=n
    CONFIG_CLOCK_CONTROL_NRF_SHELL=n
    CONFIG_FLASH_SHELL=n
    CONFIG_DEVICE_SHELL=n
    
    # Suppress LOG_ERR messages from sd_check_card_type. Because SPI_SDHC has no card presence method,
    # assume card is in slot. Thus error message is always shown if card is not inserted
    CONFIG_SD_LOG_LEVEL_OFF=y
    
    # Suppress LOG_INF messages from hci_core
    CONFIG_BT_HCI_CORE_LOG_LEVEL_WRN=y
    
    # Walkie talkie demo enable
    # CONFIG_WALKIE_TALKIE_DEMO=y
    
    # Build as headset = 1, Build as gateway = 2
    # CONFIG_AUDIO_DEV=2
    
    
    # configs for dsp
    CONFIG_CMSIS_DSP=y
    CONFIG_FPU=y
    CONFIG_NEWLIB_LIBC=y
    CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y
    CONFIG_FPU_SHARING=y
    
    # bit depth op 32 zetten
    CONFIG_AUDIO_BIT_DEPTH_32=y
    
    # Nodige configs om de sample rate naar 24khz te zetten + I2S enable, necessary for microphones
    CONFIG_AUDIO_SAMPLE_RATE_24000_HZ=y
    CONFIG_BT_AUDIO_PREF_SAMPLE_RATE_24KHZ=y
    CONFIG_AUDIO_SOURCE_I2S=y
    
    # stack overflow errors ==> stack bigger
    CONFIG_MAIN_STACK_SIZE=16384
    CONFIG_ENCODER_STACK_SIZE=48000
    CONFIG_AUDIO_DATAPATH_STACK_SIZE=48000
    
    # making the sample rate convertor work with new audio sample rate
    CONFIG_AUDIO_MAX_PRES_DLY_US=120000
    CONFIG_SAMPLE_RATE_CONVERTER_BLOCK_SIZE_MAX=1920
    CONFIG_SAMPLE_RATE_CONVERTER_BIT_DEPTH_32=y
    
    
    # voor BIS
    CONFIG_TRANSPORT_BIS=y
    
    
    # voor LC3 errors weg te krijgen
    CONFIG_LC3_ENC_SAMPLE_RATE_8KHZ_SUPPORT=n
    CONFIG_LC3_ENC_SAMPLE_RATE_16KHZ_SUPPORT=n
    CONFIG_LC3_ENC_SAMPLE_RATE_32KHZ_SUPPORT=n
    CONFIG_LC3_ENC_SAMPLE_RATE_441KHZ_SUPPORT=n
    CONFIG_LC3_ENC_SAMPLE_RATE_48KHZ_SUPPORT=n
    CONFIG_LC3_DEC_SAMPLE_RATE_8KHZ_SUPPORT=n
    CONFIG_LC3_DEC_SAMPLE_RATE_16KHZ_SUPPORT=n
    CONFIG_LC3_DEC_SAMPLE_RATE_32KHZ_SUPPORT=n
    CONFIG_LC3_DEC_SAMPLE_RATE_441KHZ_SUPPORT=n
    CONFIG_LC3_DEC_SAMPLE_RATE_48KHZ_SUPPORT=n
    
    
    

    To get BIS to work with I2S, we added these two lines in main.c (broadcast_source):

    nrf_gpio_cfg_output(21);
    nrf_gpio_pin_set(21);

    this we had only added to the main.c of unicast_client, but it was of course necessary to add this to the main.c of broadcast_source as well.

    Thanks again for the help.

Related