PDM data to speaker

[Kashyap23]

Hi,I am working with pdm mic to read data .I can able to get the audio data using dmic_read api in pcm format.I got clock of 1280000 and sampling rate of 16000 of each 16 bit sample.Now I want to play that data in tas2562 speaker amplifier.I configured pins for i2s tx i.e,clock,fclk,data out pin and I got clock frequency of 507936 of sampling rate 15873.And I got following output.I am making buffer slab free for every loop.

00> [00:00:00.001,215] <inf> dmic_sample: DMIC sample
00> [00:00:00.001,229] <inf> dmic_nrfx_pdm: PDM clock frequency: 1280000, actual PCM rate: 16000
00> [00:00:00.001,268] <inf> i2s_nrfx: I2S MCK frequency: 507936, actual PCM rate: 15873
00> Configure Stream Successful.
00> Trigger Command Successful.
00> [00:00:10.911,696] <err> dmic_nrfx_pdm: No room in RX queue
00> [00:00:11.012,512] <err> i2s_nrfx: Next buffers not supplied on time
00> [00:00:11.952,097] <err> dmic_sample: read failed: -11

[Edvin]

Hello,

It is difficult to say exact without seeing the entire application, and being able to reproduce, but you should try replacing i2s_buf_write() with i2s_write(). The issue is that i2s_buf_write() is blocking waiting for a free buffer, but this is not being provided. Also, you should only free the mem_slab if there is an error (because i2s_write() frees it upon success).

ret = i2s_write(i2s_dev_tx, buffer, size);
if (ret) {
    LOG_ERR("Failed to write I2S buffer: %d", ret);
    k_mem_slab_free(&mem_slab, buffer);  // only free on error
    return 0;
}

Try increasing the block count from 8 to e.g. 16.

#define BLOCK_COUNT 16

Try moving dmic_trigger() to after the I2S has been configured:

	config.word_size = 16;
	config.channels = NUMBER_OF_CHANNELS;
	config.format = I2S_FMT_DATA_FORMAT_I2S;
	config.options = I2S_OPT_BIT_CLK_MASTER | I2S_OPT_FRAME_CLK_MASTER;
	config.frame_clk_freq = MAX_SAMPLE_RATE;
	config.mem_slab = &mem_slab;
	config.block_size = BLOCK_SIZE;
	config.timeout = READ_TIMEOUT;
	if (!configure_streams(i2s_dev_tx, &config))
	{
		return 0;
	}
	
	ret = dmic_trigger(dmic_dev, DMIC_TRIGGER_START);
	if (ret < 0)
	{
		LOG_ERR("START trigger failed: %d", ret);
		return ret;
	}

Let me know if that changes anything. If not, it would be helpful if you could zip and post the application that you are using to replicate this, so that I can try to do it on a DK.

Best regards,

Edvin

[Kashyap23]

I tried with the changes you mentioned but the errors its throwing ,I am attaching logs and zip file .00> [00:00:00.001,184] <inf> dmic_sample: DMIC sample
00> [00:00:00.001,197] <inf> dmic_nrfx_pdm: PDM clock frequency: 1280000, actual PCM rate: 16000
00> [00:00:00.001,222] <inf> i2s_nrfx: I2S MCK frequency: 507936, actual PCM rate: 15873
00> Configure Stream Successful.
00> Trigger Command Successful.
00> [00:00:10.911,696] <err> dmic_nrfx_pdm: No room in RX queue
00> [00:00:11.012,482] <err> i2s_nrfx: Next buffers not supplied on time
00> [00:00:11.952,065] <err> dmic_sample: read failed: -11

8358.dmic.zip

[Edvin]

You cut away parts of the log here, right?

When I run your application, there are a lot of logs saying "got buffer of 320 bytes" for about 10 seconds before I see the error messages. Do you see the same?

However, it ends with:

...
00> [00:00:00.001,184] <inf> dmic_sample: DMIC sample
00> [00:00:00.001,197] <inf> dmic_nrfx_pdm: PDM clock frequency: 1280000, actual PCM rate: 16000
00> [00:00:00.001,222] <inf> i2s_nrfx: I2S MCK frequency: 507936, actual PCM rate: 15873
00> Configure Stream Successful.
00> Trigger Command Successful.
00> [00:00:10.911,696] <err> dmic_nrfx_pdm: No room in RX queue
00> [00:00:11.012,482] <err> i2s_nrfx: Next buffers not supplied on time
00> [00:00:11.952,065] <err> dmic_sample: read failed: -11

Assuming this is what you see as well:

The main issue is that the mic and speaker are working at two slightly different speeds (16000 in via mic and 15873 out via speaker). What happens is that the DMIC's internal queue is slowly filling up, and after about 10 seconds, the buffers are full, and it is stuck trying to read more, while the buffer is full. However, the buffers will never be freed, because the I2S is being prevented from executing (it is still waiting for the DMIC trying to read).

One way to work around this is to split this up into two threads. One constantly reading the mic, storing the data in buffers, and another one emptying it, outputting to the speaker. 

Since there is a difference in sample rate, there will still be some data that can't be output to the speaker, but you will be able to detect it, and skip one block.

Alternatively, you need to look at your sample rates, and try to find something that is a better match, if the resulting outcome is not sufficient.

Attached is a simple application showing how it can be done.

43325.dmic.zip

Best regards,

Edvin

[Kashyap23]

Hi Edvin,

Edvin said:

Assuming this is what you see as well:

Yes,I am seeing same what you have mentioned

Edvin said:

One way to work around this is to split this up into two threads. One constantly reading the mic, storing the data in buffers, and another one emptying it, outputting to the speaker. 

I checked with the code you shared where it skips the block of data ,it worked.But I didn't understand why we have used mem_slab of 64 blocks and  #AUDIO_Q_LEN  of 32,as we're doing in real time it will automatically clear buffer when we sent to speaker then why can't we use limited buffers of 16 and make it work.Why data is getting hold in that many buffers.When I tried I got following

00> [00:00:00.002,148] <inf> dmic_sample: DMIC sample
00> [00:00:00.002,162] <inf> dmic_nrfx_pdm: PDM clock frequency: 1280000, actual PCM rate: 16000
00> [00:00:00.002,187] <inf> i2s_nrfx: I2S MCK frequency: 507936, actual PCM rate: 15873
00> Configure Stream Successful.
00> Trigger Command Successful.
00> [00:00:14.976,144] <err> dmic_nrfx_pdm: Failed to allocate buffer: -12
00> [00:00:15.115,783] <err> i2s_nrfx: Next buffers not supplied on time
00> [00:00:15.976,247] <wrn> dmic_sample: dmic_read backpressure/retry: 1

Edvin said:

Since there is a difference in sample rate, there will still be some data that can't be output to the speaker, but you will be able to detect it, and skip one block.

As I checked Nrf54 datasheet it says the following sampling rates will occur in I2S configuration with some error.I am not sure how I can tune samples .And also make me sure is this correct process for reading pdm data and send to speaker?

With Regards,

Kashyap

[Edvin]

Hello Kashyap,

It is correct that you probably don't need that many buffers. I just experimented with your sample to see why it was failing (and forgot to turn the buffer count back down after it didn't seem to affect much to increase it). 

Having too many buffers will increase the time it takes before the first buffer is dropped, but it also introduces a delay (don't know whether that is an issue in your case). 

So you should be able to turn these down a bit.

Kashyap23 said:

When I tried I got following

What exactly did you try in this case? What changes did you do?

Kashyap23 said:

As I checked Nrf54 datasheet it says the following sampling rates will occur in I2S configuration with some error.I am not sure how I can tune samples .And also make me sure is this correct process for reading pdm data and send to speaker?

I assume this comes from this table:

https://docs.nordicsemi.com/bundle/ps_nrf9151/page/i2s.html#ariaid-title6

As you can see, it is not possible to make it exactly 16 000 Hz. You will have some error. So you need to pick a sample rate for the I2S that is either a bit too fast or a bit too slow. Perhaps, in your case, you can pick one that is a bit too fast. This means that you will still get some cracks in the sound, but it will not loose any data. You can experiment and see what works best in your case. I still recommend that you do this in two separate threads. Keep track of how many of the buffers that are filled. If no buffers are ready, wait for the DMIC to finish another sampling before you try to queue up the buffer in the speaker. 

Best regards,

Edvin

[Kashyap23]

Edvin said:

What exactly did you try in this case? What changes did you do?

I tried to reduced block count to 16 and also AUDIO_Q_LEN to 16 and 8 but after few seconds I got same issue even if Iused thread method also
00> *** Using Zephyr OS v4.1.99-ff8f0c579eeb ***
00> [00:00:00.002,148] <inf> dmic_sample: DMIC sample
00> [00:00:00.002,162] <inf> dmic_nrfx_pdm: PDM clock frequency: 1280000, actual PCM rate: 16000
00> [00:00:00.002,187] <inf> i2s_nrfx: I2S MCK frequency: 507936, actual PCM rate: 15873
00> Configure Stream Successful.
00> Trigger Command Successful.
00> [00:00:14.990,039] <err> dmic_nrfx_pdm: Failed to allocate buffer: -12
00> [00:00:15.129,861] <err> i2s_nrfx: Next buffers not supplied on time
00> [00:00:15.990,132] <wrn> dmic_sample: dmic_read backpressure/retry: 1

Edvin said:

Keep track of how many of the buffers that are filled. If no buffers are ready, wait for the DMIC to finish another sampling before you try to queue up the buffer in the speaker. 

Here can we send buffer of data to speaker or we're waiting all the data i getting qued then we sending to speaker?And I want to do some processing before data is sending to speaker.I want to do filtering and also ANC in that case i need to use 2 channel data.

[Edvin]

I think the BLOCK_COUNT needs to be at least AUDIO_Q_LEN + 10. The AUDIO_Q_LEN uses blocks from the mem_slab (size = BLOCK_COUNT), and then you need some more for the dmic. I think the limit is AUDIO_Q_LEN + 2 for DMIC DMA + 4 for I2S tx queue and 2 for I2S HW. In addition, you need 2 for handover. Perhaps use AUDIO_Q_LEN + 12 to be sure. 

Best regards,

Edvin

[Kashyap23]

Yes,when I follow this buffers I am just loosing one block of audio per second but audio flow is working fine.But when I used I2S_RX and TX there it worked even for 4 blocks in mem_slab without issue .Is it because here  PDM to I2s it's taking many blocks in recent approach(threads).
And in final I want to know reading data from PDM Mic and getting PCM data will cause more processing in time in compare with using I2S MIC  with i2s_read?.I want to confirm what mic should I use as I will be doing more processing before sending to speaker.Will both the processes will require same processing or different?

With Regards,

Kashyap

[Kashyap23]

And also to mention I can see noticeable delay in the output while listening audio in speaker.Why we are getting delay ? we are doing using threads and process is straight we read and just send in speaker.I kept AUDIO_Q_LEN + 12 memory. And if I do processing in between will delay will be increased?

[Edvin]

The reason for all this is that the pdm is sampling a bit faster than the I2S is outputting. This means (as an example, not actual numbers) that if the mic records for 1 second, that results in the speaker playing 1.1 seconds (a bit of slow motion). The difference is listed in the table that you pasted earlier:

The reason the nRF54L15 is not capable to do Audio over Bluetooth LE (like the nRF5340 is) is because it lacks a HW component, the Audio PLL, which is responsible for adjusting the clock speed to avoid these mismatches. 

There may be better ways to do this than what I suggested. It was just an attempt to make the error that you saw go away. Perhaps you can make it work with just 2 buffers, if you handle the logic yourself. Just make sure that wherever you are trying to store your dmic buffers to are free/available.

Best regards,

Edvin