nRFX based PDM to PCM implementation (I2S example)

[Nikhil Komalan]

Hello,

I am currently interfacing my nRF52840 DK with i2S based MP34DT05 sensor. I am using nrfx i2s driver's for interfacing with sensor. I am using master clk of 2MHz, LCLK 16KHZ. 

My sensor output PDM data. In my project I am recording audio samples on left channel, so I am currently getting 2*16bit of left data per 32bits as shown in nrf docs. I have got 16bit recorded samples in buffer, but to how to convert this pdm data in pcm format. 

Normally we need to divide the PDM data with a decimation factor, but in I2S we already are dividing the bit sampling rate with ratio value. So is the output from the I2S itself is decimated PCM ?  

Or can anyone give a example of how to convert the PDM to PCM data. Any sort of help is dually appreciated. 

[Audun]

Btw, I realized I linked to and old version of the documentation, but these function names have not changed as far as I know. Also, note that in the PDM ISR/handler, you must used timeout value K_NO_WAIT 

[Audun]

Hello,

I suggest changing the data_item_t struct slightly to make it easier to get the block size right. Currently you have to manually make sure that the slab definition has block size that matches sizeof struct + *buffer:

struct data_item_t {
    void *fifo_reserved;   /* 1st word reserved for use by FIFO */
    int16_t buffer[16000]; /* 1 second of mono audio @ 16 kHz */
};

//Memory slab
struct k_mem_slab audio_slab;
K_MEM_SLAB_DEFINE(audio_slab, sizeof(struct data_item_t), 2, 4);

//FIFO to hold pointers to memory slab
struct k_fifo my_fifo;
K_FIFO_DEFINE(my_fifo);

Now you can request a buffer in the pdm handler, and put filled buffers into the fifo:

void nrfx_pdm_event_handler(nrfx_pdm_evt_t const * p_evt)
{
	if(p_evt->buffer_requested)
	{
		struct data_item_t *new_block;
		int err;

		err = k_mem_slab_alloc(&audio_slab, (void**) &new_block, K_NO_WAIT);
		if (err) {
			printk("alloc fail\n");
		} else {
			nrfx_err_t nrfx_err = nrfx_pdm_buffer_set(new_block->buffer, ARRAY_SIZE(new_block->buffer));
			if (nrfx_err != NRFX_SUCCESS) {
				printk("nrfx_pdm_buffer_set: %d\n", nrfx_err);
				// Free block if set buffer fails
				k_mem_slab_free(&audio_slab, (void**) &new_block);
			}
		}
		
	}
	if(p_evt->buffer_released != 0)
	{
		struct data_item_t *filled_block;

		// Get pointer to data_item_t which the released buffer is part of
		filled_block = CONTAINER_OF(p_evt->buffer_released, struct data_item_t, buffer);
		k_fifo_put(&my_fifo,&filled_block);
	}
}

Your main loop (or somewhere else that is not in the ISR) can then fetch data from the fifo and process it:

struct data_item_t *audio_block;

audio_block = k_fifo_get(&my_fifo, K_FOREVER);
if (audio_block == NULL) {
	printk("fifo error\n");
} else {
	// TODO: Process audio_block->buffer

	k_mem_slab_free(&audio_slab, (void**) &audio_block);
}

Nikhil Komalan said:

In the python script you gave me why do we require extra processing after creating a PCM list of integers in first couple of lines.  Is it just to generate .wav files or it have anything to do with PCM data directly. Also in PCM list we have converted negative int to positive int.

The extra processing is indeed to make python wave library happy. The human-readable format would be signed 16-bit integers. That is, numbers in the range [-32767, 32767]. Your text output was as unsigned hex, which needed some massaging for python wave.

Nikhil Komalan said:

So is it required to transform the data received from PDM interface in little endian format to train the model ? From the Nordic docs "If OPERATION=Mono, RAM will contain a succession of mono samples." i am assuming i can directly use the raw PCM data available in my int16_t buffer for training and testing.

You should be able to use the raw PCM data (int16_t) directly. The questions is what format you use to transfer it from the nRF52840 to your training system. You can print it as text (like you did before), or in some binary format. Text is easier to read, but has more overhead than a binary format. The easiest is probably to match the expected input format in your training system, assuming the UART is fast enough to transfer this format.

If you transfer the data over UART as a binary format, you need to take the endianness into account in the training system. For example, lets say on the host side you read the audio as a 32000 byte buffer. To convert 32000 bytes into 16000 signed 16-bit integers, the proper endianness must be known.

[Audun]

Hello,

I see a couple of problems here, some from my code reference in earlier reply:

1) You don't need both "K_MEM_SLAB_DEFINE" and "struct k_mem_slab audio_slab;". Same for FIFO

2) The mem slab needs to be at least 3 blocks long. The PDM driver will request 2 blocks right away, and request a third before you will have had time to process the first released one:

//Memory slab
K_MEM_SLAB_DEFINE(audio_slab, sizeof(struct data_item_t), 3, 4);

//FIFO to hold pointers to memory slab
K_FIFO_DEFINE(my_fifo);

3) No double-pointer to k_fifo_put. It should be like this:

k_fifo_put(&my_fifo, filled_block);

4) k_sleep() must be removed from Microphone_Start_Record(). Otherwise slab will be emptied while the main thread is sleeping. If you want to skip the first X seconds of audio, you need to free the first Y blocks in main() without printing them out first (k_mem_slab_free)

[Audun]

You could try making the printouts more efficient. For example:

		static char str[4000 * 4 + 1];
		size_t len = 0;

		for(int i=0; i<4000;i++)
		{
			int err = snprintf(&str[len], sizeof(str) - len, "%04X", (unsigned short) audio_block->buffer[i]);
			if (err < 0 || err > 4)
			{
				printk("Format error: %d\n", err);
				return;
			}
			len += err;
		}
		printk("%s\n", str);