nRFX based PDM to PCM implementation (I2S example)

[Nikhil Komalan]

Hello,

I am currently interfacing my nRF52840 DK with i2S based MP34DT05 sensor. I am using nrfx i2s driver's for interfacing with sensor. I am using master clk of 2MHz, LCLK 16KHZ. 

My sensor output PDM data. In my project I am recording audio samples on left channel, so I am currently getting 2*16bit of left data per 32bits as shown in nrf docs. I have got 16bit recorded samples in buffer, but to how to convert this pdm data in pcm format. 

Normally we need to divide the PDM data with a decimation factor, but in I2S we already are dividing the bit sampling rate with ratio value. So is the output from the I2S itself is decimated PCM ?  

Or can anyone give a example of how to convert the PDM to PCM data. Any sort of help is dually appreciated. 

[Nikhil Komalan]

Audun said:

You should be able to use the raw PCM data (int16_t) directly.

Yes i will be transmitting training data in int16_t format only.

I have two examples now.

1. Were i am testing my mic audio output by listening to .wav file generated from your given python script. This is just for testing to see, whether the audio is getting recorded correctly or not. And in this example i am using uint16_t buffer to print data in hex format and using python to do all other processing. This example will let me know whether i am able to record audio continuously or not, If yes than i will implement the same for edge impulse training and testing.

2. Second is what i will be using for training and testing my model. Here i have created a int16_t buffer and printing data in decimal format. And uploading data to edge impulse without any other processing.

Now i have changed my example code using your above code snippets to continuously receive audio samples.

main.c file

/*
 * Copyright (c) 2020 BayLibre, SAS
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include <zephyr.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <stdint.h>
#include <sys/printk.h>
#include <sys/util.h>
#include <string.h>
#include <usb/usb_device.h>
#include <drivers/uart.h>
#include <drivers/gpio.h>
#include <nrfx_pdm.h>

#define PDM_BUF_SIZE 32000  //length of buffer in 16 bit words
#define PDM_CLK_PIN  46   // P1.14 (32+14)
#define PDM_DIN_PIN  45   // P1.13 (32+13)



void Microphone_Init(void);
void Microphone_Start_Record(void);
void Microphone_Stop(void);





/* Audio sampling config */
#define AUDIO_SAMPLING_FREQUENCY            32000
#define AUDIO_SAMPLES_PER_MS                (AUDIO_SAMPLING_FREQUENCY / 1000)
#define AUDIO_DSP_SAMPLE_LENGTH_MS          16
#define AUDIO_DSP_SAMPLE_RESOLUTION         (sizeof(short))
//#define AUDIO_DSP_SAMPLE_BUFFER_SIZE        (AUDIO_SAMPLES_PER_MS * AUDIO_DSP_SAMPLE_LENGTH_MS * AUDIO_DSP_SAMPLE_RESOLUTION) //4096
#define AUDIO_DSP_SAMPLE_BUFFER_SIZE        32000

/* Buffers for receiving PDM mic data */

static uint16_t pdm_buffer_temp[2][AUDIO_DSP_SAMPLE_BUFFER_SIZE] = {0};
uint16_t *current_buff;
bool write_data = false;
uint16_t pdm_buf[AUDIO_DSP_SAMPLE_BUFFER_SIZE];


static bool data_ready_flag = false;
int8_t counter=0;

static uint8_t buf_toggle = 0;
bool flag = false;



//Memory Slab and FIFO Init
struct data_item_t {
    void *fifo_reserved;   /* 1st word reserved for use by FIFO */
    int16_t buffer[4000]; /* just for testing */
};

//Memory slab
struct k_mem_slab audio_slab;
K_MEM_SLAB_DEFINE(audio_slab, sizeof(struct data_item_t), 2, 4);

//FIFO to hold pointers to memory slab
struct k_fifo my_fifo;
K_FIFO_DEFINE(my_fifo);



void nrfx_pdm_event_handler(nrfx_pdm_evt_t const * p_evt)
{

  if(p_evt->buffer_requested)
  {
      struct data_item_t *new_block;
      int err;

      err = k_mem_slab_alloc(&audio_slab, (void**) &new_block, K_NO_WAIT);
      if (err) 
      {
          printk("alloc fail\n");
      } 
      else 
      {
          nrfx_err_t nrfx_err = nrfx_pdm_buffer_set(new_block->buffer, ARRAY_SIZE(new_block->buffer));
          if (nrfx_err != NRFX_SUCCESS) 
          {
              printk("nrfx_pdm_buffer_set: %d\n", nrfx_err);
              // Free block if set buffer fails
              k_mem_slab_free(&audio_slab, (void**) &new_block);
          }
      }       
  }


  if(p_evt->buffer_released != 0)
  {
      struct data_item_t *filled_block;

      // Get pointer to data_item_t which the released buffer is part of
      filled_block = CONTAINER_OF(p_evt->buffer_released, struct data_item_t, buffer);
      k_fifo_put(&my_fifo,&filled_block);
  }

}


/*ISR workaround given by Nordic*/
ISR_DIRECT_DECLARE(pdm_isr_handler)
{
  nrfx_pdm_irq_handler();
  ISR_DIRECT_PM(); /* PM done after servicing interrupt for best latency
                    */
  return 1; /* We should check if scheduling decision should be made */
}


void main(void)
{
    nrfx_err_t ret = NRFX_SUCCESS;


    //USB interface
    const struct device *dev = device_get_binding(CONFIG_UART_CONSOLE_ON_DEV_NAME);
    uint32_t dtr = 0;


    if (usb_enable(NULL)) {
            return;
    }

    ////// Poll if the DTR flag was set, optional 
    while (!dtr) {
            uart_line_ctrl_get(dev, UART_LINE_CTRL_DTR, &dtr);
    }

    if (strlen(CONFIG_UART_CONSOLE_ON_DEV_NAME) !=  strlen("CDC_ACM_0") ||
        strncmp(CONFIG_UART_CONSOLE_ON_DEV_NAME, "CDC_ACM_0",
                strlen(CONFIG_UART_CONSOLE_ON_DEV_NAME))) {
           printk("Error: Console device name is not USB ACM\n");

            return;
    }


  printk("Starting Init\r\n");
  Microphone_Init();

  printk("Starting recording samples\r\n");
  Microphone_Start_Record();


 
  while(1)
  {
    struct data_item_t *audio_block;

    audio_block = k_fifo_get(&my_fifo, K_FOREVER);
     if (audio_block == NULL) 
     {
	printk("fifo error\n");
     } 
     else 
     {
	//Process audio_block->buffer
        printk("Got Slab\r\n");
        
          for(int i=0; i<4000;i++)
        {
            printk("%d, ", audio_block->buffer[i] );
        }

        k_mem_slab_free(&audio_slab, (void**) &audio_block);
        printk("Released slab\r\n");
     }

      printk("\r\n");
     


  }

}




void Microphone_Init(void)
{
    nrfx_err_t ret = NRFX_SUCCESS;

  /* ISR for PDM */
    IRQ_DIRECT_CONNECT(PDM_IRQn, 0, pdm_isr_handler, 0);  

  /* Microphone MP34DT05 Configuration 
     PDM Aduio Sampling Frequency = CLK Freq / Decimation factor (ratio)
     Configuration for 16KHz (1280K/80) audio sampling frequency */

    nrfx_pdm_config_t pdm_config = NRFX_PDM_DEFAULT_CONFIG(PDM_CLK_PIN, PDM_DIN_PIN);
    pdm_config.clock_freq = PDM_PDMCLKCTRL_FREQ_1280K; //MP34DT05 requirement (1.2MHz-3MHz)
    pdm_config.ratio = NRF_PDM_RATIO_80X;       
    pdm_config.edge = NRF_PDM_EDGE_LEFTRISING;    //MP34DT05 requirement
    pdm_config.gain_l = NRF_PDM_GAIN_MAXIMUM;
    pdm_config.gain_r = NRF_PDM_GAIN_MAXIMUM;

    ret = nrfx_pdm_init(&pdm_config, nrfx_pdm_event_handler);
    if(ret != NRFX_SUCCESS){
      printk("PDM Init failed.\r\n");
    }
  
    k_sleep(K_MSEC(10));

}


void Microphone_Start_Record(void)
{
    nrfx_err_t ret = NRFX_SUCCESS;

    /* Start Recoridng samples */
    ret = nrfx_pdm_start();
    if(ret != NRFX_SUCCESS)
    {
      printk("PDM recording failed\r\n");
    }
    
    //Turn-on time for microphone to get valid output data is 10ms.
    k_sleep(K_MSEC(1000));
}


void Microphone_Stop(void)
{
    nrfx_err_t ret = NRFX_SUCCESS;

    /* Stop Recoridng samples */
    ret = nrfx_pdm_stop();
    if(ret != NRFX_SUCCESS){
     printk("Error Init\r\n");
    }
}

It doesn't display the "Got slab", "Released slab" and also the data. 

Than i started debugging the code. In PDM handler, the memory slab gets created and nrfx_pdm_buff_set is successful as well. Data is received in buffer. In buffer release, also the buffer address is received by filled_buffer pointer variable. 

In buffer released, after putting data in the queue the head and tail value of queue changes to buffer address.

After returning 1 from ISR_DIRECT_DECLARE, the cursor goes directly to fault_s.S file. 

After pressing the continue execution, it again goes back to the request buffer and never goes into while loop to process the buffer data, but the slab gets freed. How i don't know ? as i have added a breakpoint in the while loop so, it never reaches there to free the memory slab.  So how to solve this issue ?

I have attached the IDE Images hopefully it will help t understand my prob.

The audio slab state in debug look like this. With same head and tail.

I have given main.c above to generate the same issue ?

Although it's a different topic, but i get issues in compiling nrfx_pdm_driver while i convert main.c file to main.cpp. If you can have a look here will be helpful too:

https://devzone.nordicsemi.com/f/nordic-q-a/76237/invalid-conversion-from-long-unsigned-int-to-nrf_pdm_freq_t--fpermissive-while-using-in-c

[Audun]

Hello,

I see a couple of problems here, some from my code reference in earlier reply:

1) You don't need both "K_MEM_SLAB_DEFINE" and "struct k_mem_slab audio_slab;". Same for FIFO

2) The mem slab needs to be at least 3 blocks long. The PDM driver will request 2 blocks right away, and request a third before you will have had time to process the first released one:

//Memory slab
K_MEM_SLAB_DEFINE(audio_slab, sizeof(struct data_item_t), 3, 4);

//FIFO to hold pointers to memory slab
K_FIFO_DEFINE(my_fifo);

3) No double-pointer to k_fifo_put. It should be like this:

k_fifo_put(&my_fifo, filled_block);

4) k_sleep() must be removed from Microphone_Start_Record(). Otherwise slab will be emptied while the main thread is sleeping. If you want to skip the first X seconds of audio, you need to free the first Y blocks in main() without printing them out first (k_mem_slab_free)