Output of my Audio DK is noisy/clipping when coming via i2s

[daubsi]

I am quite new to the nRF5340 Audio DK and used ace.johnny 's project on https://github.com/ace-johnny/nrfadk-hello_codec to get me started.

The overall project goal is to process audio data coming via line-in on the nRF5340 to add filters, effects, etc. and output again via the headphone.

In order to achieve this I transmit the data from the codec via i2s to the MCU, process it there and send it back. However, something in this chain is introducting a lot of noise/clipping and I can't find the reason for this. Right now my processing of the received data is just copying it 1:1 to the transmit buffer - no change whatsoever.

When I directly route the line-in input to the output and not going via i2s to the MCU and back everything is crystal clear.

This is the code I use:

/**
 * @file        main.c
 * 
 * @brief       Audio DK HW_CODEC test using I2S loop and tone/noise generators.
 */

#include <zephyr/kernel.h>
#include <nrf.h>
#include <nrfx_clock.h>

#include "cs47l63_comm.h"

////////////////////////////////////////////////////////////////////////////////
// NRFX_CLOCKS

#define HFCLKAUDIO_12_288_MHZ 0x9BA6
#define ENABLE_LINEIN
#undef ENABLE_MIC


/**
 * @brief       Initialize the high-frequency clocks and wait for each to start.
 * 
 * @details     HFCLK =         128,000,000 Hz
 *              HFCLKAUDIO =     12,288,000 Hz
 */
static int nrfadk_hfclocks_init(void)
{
	nrfx_err_t err;


	// HFCLK
	err = nrfx_clock_divider_set(NRF_CLOCK_DOMAIN_HFCLK, NRF_CLOCK_HFCLK_DIV_1);
	if (err != NRFX_SUCCESS) return (err - NRFX_ERROR_BASE_NUM);

	nrfx_clock_start(NRF_CLOCK_DOMAIN_HFCLK);
	while (!nrfx_clock_is_running(NRF_CLOCK_DOMAIN_HFCLK, NULL)) k_msleep(1);


	// HFCLKAUDIO
	nrfx_clock_hfclkaudio_config_set(HFCLKAUDIO_12_288_MHZ);

	nrfx_clock_start(NRF_CLOCK_DOMAIN_HFCLKAUDIO);
	while (!nrfx_clock_is_running(NRF_CLOCK_DOMAIN_HFCLKAUDIO, NULL)) k_msleep(1);


	return 0;
}



////////////////////////////////////////////////////////////////////////////////
// NRF_I2S

#define MCKFREQ_6_144_MHZ 0x66666000


#define I2S_BUFF_SIZE 256  // Define an appropriate buffer size

static int16_t rx_buffer[I2S_BUFF_SIZE];  // Buffer for received data
static int16_t tx_buffer[I2S_BUFF_SIZE]; 

void process_audio(int16_t *rx, int16_t *tx, size_t size)
{
    for (size_t i = 0; i < size; i++) {
		tx[i] = rx[i];  // Simple passthrough (modify as needed)
        	//tx[i] = (int16_t)(rx[i]*0.1);  // Volume control
		//tx[size-1-i] = rx[i];  // Reverse - doesn't work... -> only outputs silence...?
    }
}

/**
 * @brief       Initialize and start the I2S peripheral using NRF registers.
 * 
 * @details     I2S master, 48kHz 16bit, Left mono, TX only.
 */
static int nrfadk_i2s_reg_init(void)
{
	// Configure and enable
	NRF_I2S0->CONFIG.CLKCONFIG =    I2S_CONFIG_CLKCONFIG_CLKSRC_ACLK;
	NRF_I2S0->CONFIG.MCKFREQ =      MCKFREQ_6_144_MHZ;
	NRF_I2S0->CONFIG.RATIO =        I2S_CONFIG_RATIO_RATIO_128X;
	NRF_I2S0->CONFIG.CHANNELS =     I2S_CONFIG_CHANNELS_CHANNELS_Left;	
	NRF_I2S0->CONFIG.TXEN =         I2S_CONFIG_TXEN_TXEN_Enabled;  // Send audio samples to the nRF5340
	NRF_I2S0->CONFIG.RXEN = 	I2S_CONFIG_RXEN_RXEN_Enabled;  // Enable RX to receive the data from the nRF5340

	NRF_I2S0->ENABLE =              I2S_ENABLE_ENABLE_Enabled;

	// Start TX buffer
	NRF_I2S0->RXD.PTR = 		(uint32_t)rx_buffer;
	NRF_I2S0->TXD.PTR = 		(uint32_t)tx_buffer;
	NRF_I2S0->RXTXD.MAXCNT =        I2S_BUFF_SIZE / sizeof(uint32_t);	

        // Clear pending events
    	NRF_I2S0->EVENTS_RXPTRUPD = 0;
	    NRF_I2S0->EVENTS_TXPTRUPD = 0;

	NRF_I2S0->TASKS_START =         I2S_TASKS_START_TASKS_START_Trigger;
	return 0;
}

void i2s_polling_loop(void)
{
    while (1)
    {
        // Wait for new RX data
        while (NRF_I2S0->EVENTS_RXPTRUPD == 0);
		
        // Clear RX event
        NRF_I2S0->EVENTS_RXPTRUPD = 0;

        // Process audio data
        process_audio(rx_buffer, tx_buffer, I2S_BUFF_SIZE); // For now, just set tx_buffer == rx_buffer

        // Ensure we restart I2S for continuous operation
        NRF_I2S0->TASKS_START = I2S_TASKS_START_TASKS_START_Trigger;
	while (NRF_I2S0->EVENTS_TXPTRUPD == 0);
    }
}

////////////////////////////////////////////////////////////////////////////////
// HW_CODEC

/** CS47L63 driver state handle. */
static cs47l63_t cs47l63_driver;


/** CS47L63 subsystems configuration. */
static const uint32_t cs47l63_cfg[][2] =
{

	// Audio Serial Port 1 (I2S slave, 48kHz 16bit, Left mono, RX and TX)
	{ CS47L63_ASP1_CONTROL2,
		(0x10  << CS47L63_ASP1_RX_WIDTH_SHIFT) |        // 16bit
		(0x10  << CS47L63_ASP1_TX_WIDTH_SHIFT) |        // 16bit
		(0b010 << CS47L63_ASP1_FMT_SHIFT)               // I2S
	},
	{ CS47L63_ASP1_CONTROL3,
		(0b00 << CS47L63_ASP1_DOUT_HIZ_CTRL_SHIFT)      // Always 0
	},
	// Enable the various channels for RX and TX
	{ CS47L63_ASP1_ENABLES1,
		(0 << CS47L63_ASP1_RX2_EN_SHIFT) |              // Disabled
		(1 << CS47L63_ASP1_RX1_EN_SHIFT) |              // Enabled
		(0 << CS47L63_ASP1_TX2_EN_SHIFT) |              // Disabled
		//(0 << CS47L63_ASP1_TX1_EN_SHIFT)                // Disabled
		(1 << CS47L63_ASP1_TX1_EN_SHIFT)                // Enabled - we want to send something to the nRF5340
	},

#ifdef ENABLE_LINEIN
	// Enable line-in
	{ CS47L63_INPUT2_CONTROL1, 0x00050020 },/* MODE=analog */ 
	{ CS47L63_IN2L_CONTROL1, 0x10000000 },  /* SRC=IN2LP */
	{ CS47L63_IN2R_CONTROL1, 0x10000000 },  /* SRC=IN2RP */
	{ CS47L63_INPUT_CONTROL, 0x0000000C },  /* IN2_EN=1 */
	// Set volume for line-in
	{ CS47L63_IN2L_CONTROL2, 0x00800080 },  /* VOL=0dB, MUTE=0 */
	{ CS47L63_IN2R_CONTROL2, 0x00800080 },  /* VOL=0dB, MUTE=0 */
	{ CS47L63_INPUT_CONTROL3, 0x20000000 }, /* VU=1 */
	// Important
	/* Route IN2L and IN2R to I2S */
	{ CS47L63_ASP1TX1_INPUT1, 0x800012 },
	{ CS47L63_ASP1TX2_INPUT1, 0x800013 },
#endif

	// Output 1 Left (reduced MIX_VOLs to prevent clipping summed signals)
	// this here is only there so we hear also that i2s data is sent 
	
	{ CS47L63_OUT1L_INPUT1,
		(0x2B  << CS47L63_OUT1LMIX_VOL1_SHIFT) |        // quite weak
		(0x020 << CS47L63_OUT1L_SRC1_SHIFT)             // ASP1_RX1 // from MCU (currently a sine wave only)
	},

	{ CS47L63_OUT1L_INPUT2,
		(0x2B  << CS47L63_OUT1LMIX_VOL2_SHIFT) |        // 
		(0x021 << CS47L63_OUT1L_SRC2_SHIFT)             // ASP1_RX2
	},
	
#undef ENABLE_LINEIN
#ifdef ENABLE_LINEIN
	// We need both channels here, even if we only have one output channel
	// If we uncomment the next two {} we won´t get any line in pass-through, only i2s
	{
		CS47L63_OUT1L_INPUT3,
		(0x2B  << CS47L63_OUT1LMIX_VOL3_SHIFT) |
		(0x012 << CS47L63_OUT1L_SRC3_SHIFT)    // 0x12=IN2L
	},
	{
		CS47L63_OUT1L_INPUT4,
		(0x2B  << CS47L63_OUT1LMIX_VOL4_SHIFT) |
		(0x013 << CS47L63_OUT1L_SRC4_SHIFT)    // 0x13=IN2R
	},
#endif

	{ CS47L63_OUTPUT_ENABLE_1,
		(1 << CS47L63_OUT1L_EN_SHIFT)                   // Enabled
	},
};


/**
 * @brief       Write a configuration array to multiple CS47L63 registers.
 * 
 * @param[in]   config: Array of address/data pairs.
 * @param[in]   length: Number of registers to write.
 * 
 * @retval      `CS47L63_STATUS_OK`     The operation was successful.
 * @retval      `CS47L63_STATUS_FAIL`   Writing to the control port failed.
 */
static int nrfadk_hwcodec_config(const uint32_t config[][2], uint32_t length)
{
	int ret;
	uint32_t addr;
	uint32_t data;

	for (int i = 0; i < length; i++)
	{
		addr = config[i][0];
		data = config[i][1];

		ret = cs47l63_write_reg(&cs47l63_driver, addr, data);
		if (ret) return ret;
	}

	return CS47L63_STATUS_OK;
}


/**
 * @brief       Initialize the CS47L63, start clocks, and configure subsystems.
 * 
 * @details     MCLK1 =   6,144,000 Hz  (I2S MCK = CONFIG.MCKFREQ)
 *              FSYNC =      48,000 Hz  (I2S LRCK = MCK / CONFIG.RATIO)
 *              BCLK =    1,536,000 Hz  (I2S SCK = LRCK * CONFIG.SWIDTH * 2)
 *              FLL1 =   49,152,000 Hz  (MCLK1 * 8)
 *              SYSCLK = 98,304,000 Hz  (FLL1 * 2)
 * 
 * @retval      `CS47L63_STATUS_OK`     The operation was successful.
 * @retval      `CS47L63_STATUS_FAIL`   Initializing the CS47L63 failed.
 * 
 * @note        I2S MCK must already be running before calling this function.
 */
static int nrfadk_hwcodec_init(void)
{
	int ret = CS47L63_STATUS_OK;


	// Initialize driver
	ret += cs47l63_comm_init(&cs47l63_driver);


	// Start FLL1 and SYSCLK
	ret += cs47l63_fll_config(&cs47l63_driver, CS47L63_FLL1,
	                          CS47L63_FLL_SRC_MCLK1, 6144000, 49152000);

	ret += cs47l63_fll_enable(&cs47l63_driver, CS47L63_FLL1);

	ret += cs47l63_fll_wait_for_lock(&cs47l63_driver, CS47L63_FLL1);

	ret += cs47l63_update_reg(&cs47l63_driver, CS47L63_SYSTEM_CLOCK1,
	                          CS47L63_SYSCLK_EN_MASK, CS47L63_SYSCLK_EN);


	// Configure subsystems
	ret += nrfadk_hwcodec_config(cs47l63_cfg, ARRAY_SIZE(cs47l63_cfg));


	return ret;
}



////////////////////////////////////////////////////////////////////////////////
// MAIN

int main(void)
{
	// Initialize Audio DK

	if (nrfadk_hfclocks_init() ||
	    nrfadk_i2s_reg_init()  ||
	    nrfadk_hwcodec_init())
	{
		printk("\nError initializing Audio DK\n");
		return -1;
	}

	printk("\nAudio DK initialized\n");
	k_msleep(1250);



	// Unmute OUT1L I2S playback and enable NOISE/TONE1 generators

	cs47l63_update_reg(&cs47l63_driver, CS47L63_OUT1L_VOLUME_1,
	                   CS47L63_OUT_VU_MASK | CS47L63_OUT1L_MUTE_MASK,
	                   CS47L63_OUT_VU | 0);

	printk("\nOUT1L unmuted for 5000ms\n");

	k_msleep(5000);
	i2s_polling_loop(); // We actually never return from here (testing)

	while(1);
	cs47l63_update_reg(&cs47l63_driver, CS47L63_OUT1L_VOLUME_1,
	                   CS47L63_OUT_VU_MASK | CS47L63_OUT1L_MUTE_MASK,
	                   CS47L63_OUT_VU | CS47L63_OUT1L_MUTE);

	printk("OUT1L muted\n");
	k_msleep(1250);



	// Shutdown Audio DK (reverse order initialized)

	cs47l63_update_reg(&cs47l63_driver, CS47L63_OUTPUT_ENABLE_1,
	                   CS47L63_OUT1L_EN_MASK, 0);
	printk("\nOUT1L disabled\n");
	k_msleep(250);

	cs47l63_update_reg(&cs47l63_driver, CS47L63_SYSTEM_CLOCK1,
	                   CS47L63_SYSCLK_EN_MASK, 0);
	printk("SYSCLK disabled\n");
	k_msleep(250);

	cs47l63_fll_disable(&cs47l63_driver, CS47L63_FLL1);
	printk("FLL1 disabled\n");
	k_msleep(250);

	NRF_I2S0->TASKS_STOP =  I2S_TASKS_STOP_TASKS_STOP_Trigger;
	NRF_I2S0->ENABLE =      I2S_ENABLE_ENABLE_Disabled;
	printk("I2S disabled\n");
	k_msleep(250);

	nrfx_clock_stop(NRF_CLOCK_DOMAIN_HFCLKAUDIO);
	while (nrfx_clock_is_running(NRF_CLOCK_DOMAIN_HFCLKAUDIO, NULL)) k_msleep(1);
	printk("HFCLKAUDIO stopped\n");
	k_msleep(250);

	nrfx_clock_stop(NRF_CLOCK_DOMAIN_HFCLK);
	while (nrfx_clock_is_running(NRF_CLOCK_DOMAIN_HFCLK, NULL)) k_msleep(1);
	printk("HFCLK stopped\n");
	k_msleep(250);


	printk("\nAudio DK shutdown\n\n");
	k_msleep(100);


	return 0;
}

process_audio() is the simple function which just sets the TX buffer to the RX buffer's values. 

(Here I want eventually to add my processing)

nrfadk_i2s_reg_init() initializes the i2s on the nRF5340 side. I've added here also the reception of the data from the codec

i2s_polling_loop() constantly checks if i2s data is present, then copies it over and starts the back transmission

cs47l63_cfg() configures the line-in input, the i2s in the ASP1 and the output channel coming from i2s 

My assumption is that the error is in one of those regions:

i2s_polling_loop() - maybe done completely wrong at all?

cs47l63_cfg() - I am confused - do I have a stereo input with line-in or only mono? Will I need ASP1RX1,2 and ASP1TX1,2 then or just ASP1RX1 and ASP1TX1?

Is it OK to use int16_t as the values in the buffer but uint32_t to address the elements? Does the i2s config in the codec match?

Is the noise maybe just a result of the MCU not being able to keep up? 

The audio is properly played back in genereal but there is noise/clipping present all the time - not only during loud parts of the music.

[daubsi]

Could you kindly reiterate the following statement:

IN1(LR) is dedicated exclusively to the onboard digital PDM1 mic, however its second channel(R) remains unused, so is essentially a mono signal. This is important to remember when routing signals inside the codec.

IN2(LR) is shared between analog stereo LINE-IN and an auxiliary digital stereo PDM2 mic input on connector P15, and only one of these can be selected at a time.

Where do you see this?

In the datasheet on p25 it appears to me the digital mic's data at IN1_PDMCLK and IN1_PDMDATA gets to IN1L and IN1R likewise? Or is it something you see on https://docs-be.nordicsemi.com/bundle/ug_nrf5340_audio/page/UG/nrf5340_audio/images/audio_codec.svg ? If so, where please? I fully agree to the 2nd statement about IN2 (LR) from what I see on page 25, but I don´t see what you  mentioned in the first statement. 

Thank you!

Also, did you try turning on ANC (p36, 4.2.9) or the equalizer (p41, 4.3.4). It is not clear to me how to connect the EQ into the signalchain. I understand it has to happen somewhere in the digital core and associated with one of the channels of e.g. the ASPx or the line-in interface, but it don´t understand right away how. Any opinion on this? 

[daubsi]

Thank you this makes it clear now. I have to admit I didn't even look at the datasheet of the VM3011 yet at all. Your explanation was very helpful!
Regarding the misconfiguration, I understand in order to reconfigure the VM3011, if one wants to do so, the C47L63 also would need to be configured via i2c and not SPI? (which is not possible right now, due to the fact that Nordic configures it as SPI in the first connection)

[daubsi]

So basically if I have my PDM I just need to set this input as EQ1_SRC1 and enable the EQ in order to integrate it's filtering in the chain? (Fig 4.16 in the datasheet). Ofc one would have to visually define the filter curve using WISCE and then configure the retrieved constants beforehand to configure the EQ accordingly. 

[daubsi]

I've now added successfully the PDM to the input chain by adding the relevant config bits and pieces. No idea why it didn´t work before but now it seemed like the simplest thing to do.

Next I tried to add a simple echo effect, i.e adding a delayed by D samples and weakened RX sample to our current RX sample and copying it to our TX buffer.   

What should work in theory just doesn´t work out in code. I don´t hear any echo at all, no matter how I play with the DELAY constant. size is at 8192 which is what I defined as a suitable buffer size by experimenting. The audio is clear like in the original tx[i] = rx[i] version, I just don´t hear any echo.

My calculation was that at 48k samples * 2 channels per 1000ms with a delay of 2000 samples I would get an echo of approx 20ms?   

Do you have any idea what might be wrong? 

#define DELAY 2048

void process_audio(int16_t *rx, int16_t *tx, size_t size)
{
	static int16_t delay_buffer[DELAY];
	static int delay_index = 0;

	for (int i=0; i<size; ++i) {
		int16_t delayed_sample = delay_buffer[delay_index];
		tx[i] = rx[i] + delayed_sample * 0.5;
		delay_buffer[delay_index] = rx[i];
		delay_index = (delay_index + 1) % DELAY;
	}
}

[daubsi]

You're right, sorry. 
I've pushed my current working state to https://github.com/OevreFlataeker/nrf5340_ak_fx

It's actually still pretty much your code 1:1, I just added the PMIC enable and removed noise and test tone stuff and all my 100s comments and defines, etc. ;-)

If you, in process_audio() just revert to the plain:

for (size_t i = 0; i < size; i++) {
		tx[i] = rx[i];  // Simple passthrough (modify as needed)
}

in there you should hear the input taken from the PDM 1:1 in the headphone. 

I"ve cleaned up quite a bit now without begin able to live test ist. If it fails kindly try also a commit from last evening, e.g c10b0aba3c9289a2692bcc42ef18992a89caaeeb