Data Logging

RE: Data Logging

Håkon Alseth — Thu, 19 Dec 2024 16:43:55 GMT

Are you certain that it is the logging function that is corrupting/missing data, and not the I2S DMA process that is overwriting the buffers before you're copying them or in the middle of that sequence? Or creating a minimalistic example with little / no other functions, just sampling of a microphone?

DMA can be a bit of a struggle to debug, especially if there is a timing skew (caused by ISRs or other threads) in the copy sequence.

Instead of processing each sample individually, then RTT print, could you try to copy it into a larger buffer, then print the whole buffer?

static void copyBuffer(int32_t *Buff_In,int16_t *Buff_Out,uint16_t number_of_samples)
{
	int16_t print_buf[sizeof(MAX_SIZE)];
	for(int i = 0; .....) { /*copy int and shift -> int16_t */ }
	my_print_func(print_buf, number_of_samples*sizeof(int16_t));
}

Kind regards,

Håkon

RE: Data Logging

Kashyap23 — Thu, 19 Dec 2024 10:30:23 GMT

yah thanks Alseth,now I able to convert raw data to wav file but converting to integers and logging leads to data loss.I was not able to solve that.

RE: Data Logging

Håkon Alseth — Wed, 18 Dec 2024 15:24:07 GMT

What I would recommend is that you write known values to RTT, and see what is present in the file afterwards.

For instance sending several values like, val=0x11223344 and val=0x12345678, and see what is present in the stored file on the PC afterwards.

Kind regards,

Håkon

RE: Data Logging

Kashyap23 — Wed, 18 Dec 2024 13:33:27 GMT

yes,I am trying with raw as I am missing data.By using raw data converting to wav file giving noise.I followed your step and logged data.

devzone.nordicsemi.com/.../rwData.txt

RE: Data Logging

Håkon Alseth — Wed, 18 Dec 2024 13:16:12 GMT

Hi,

[quote user="Kashyap23"]what does it refer?[/quote]

To use USB with an audio class to send/receive audio.

I think you're sending it as raw data now:

[quote user="Kashyap23"]

am using same jlink base compact at max speed even though i am getting same audio mismatch.Audio is not missing at end it's missing between the data.If it's sampling at 16000 ,how data logging is missing in between,it should go one by one?I am not sure how data log works.I am using SEGGER_RTT_PRINTF.I thought to use SEGGER_RTT_WRITE ,I am getting data like(

ûù

Æù

wù

mù

^ù

?ù

eù

ºù

¾ù

cù

7ù

Tù

Pù

[/quote]

echo -n "ùûùÆùwùmù^ù?ùeùºù¾ùcù7ùTùPù" | xxd
00000000: c3b9 c3bb c3b9 c386 c3b9 77c3 b96d c3b9  ..........w..m..
00000010: 5ec3 b93f c3b9 65c3 b9c2 bac3 b9c2 bec3  ^..?..e.........
00000020: b963 c3b9 37c3 b954 c3b9 50c3 b9         .c..7..T..P..

You need to remove the delimiter \n to listen back to the data.

Kind regards,

Håkon

RE: Data Logging

Kashyap23 — Wed, 18 Dec 2024 11:47:58 GMT

[quote userid="2115" url="~/f/nordic-q-a/116983/data-logging/515166"]Have you considered using usb as the loopback instance instead?[/quote]

what does it refer?

[quote userid="2115" url="~/f/nordic-q-a/116983/data-logging/515166"]Also, if you use an external debugger (JLink Base compact for instance), you can increase the SWD speed, for more throughput on the actual reads from the JLink debugger itself[/quote]

I am using same jlink base compact at max speed even though i am getting same audio mismatch.Audio is not missing at end it's missing between the data.If it's sampling at 16000 ,how data logging is missing in between,it should go one by one?I am not sure how data log works.I am using SEGGER_RTT_PRINTF.I thought to use SEGGER_RTT_WRITE ,I am getting data like(

ûù

Æù

wù

mù

^ù

?ù

eù

ºù

¾ù

cù

7ù

Tù

Pù

4ù) but converting it to wave getting only noise.

static void copyBuffer(int32_t *Buff_In,int16_t *Buff_Out,uint16_t number_of_samples)
{
	int i=0;
	
	for( i = 0; i < number_of_samples; i++) 
	{
		Buff_Out[i]= (Buff_In[i]>>16);
		
		// printk("%hx %hx\n",bellMicData,ancMicData);
		 SEGGER_RTT_printf(0,"%d\n",Buff_Out[i]);
 		// SEGGER_RTT_Write(0,(const void*) &Buff_Out[i], 2);
	}
	// SEGGER_RTT_Write(0,&newline,1); 
 }

while rearding one mic i didnt got issue,on logging two only i am getting issue

RE: Data Logging

Håkon Alseth — Mon, 16 Dec 2024 15:20:34 GMT

Hi,

It does sound like you are missing data, especially on '14' and possibly on '7'.

Have you considered using usb as the loopback instance instead?

Also, if you use an external debugger (JLink Base compact for instance), you can increase the SWD speed, for more throughput on the actual reads from the JLink debugger itself.

Kind regards,

Håkon

RE: Data Logging

Kashyap23 — Mon, 16 Dec 2024 05:30:55 GMT

Yes,I set to 128 MHZ ,I cant see any messages dropped error .But when I am logging data and converting to audio file ,i can see some data is missing.I have played numbers which says at each second in mic ,I can see 11 numbers played in 9 seconds itself.Will this can be solved any issue in logging speed clocks?can we increase that clocks?

devzone.nordicsemi.com/.../output_5F00_audio.wav

RE: Data Logging

Håkon Alseth — Tue, 10 Dec 2024 13:47:38 GMT

Hi,

Did you try to run the CPU at 128 MHz to see if this is a cpu constraint?

Kind regards,

Håkon

RE: Data Logging

Kashyap23 — Mon, 09 Dec 2024 13:56:02 GMT

[quote userid="2115" url="~/f/nordic-q-a/116983/data-logging/514039"]Does this issue happen "after a while" ? Meaning that from boot, you're able to read out for some period of time before it starts to print "dropped messages" in the log?[/quote]

Yes,if I took buffer of 244, 16 messages are dropping and it repeats as same for next buffer of 244 again 16 messages are dropping.I attatched logging file where in each lineit print left and right mic data.

devzone.nordicsemi.com/.../data.log

RE: Data Logging

Håkon Alseth — Mon, 09 Dec 2024 13:41:09 GMT

Hi,

Each 244*1/16kHz, equals around 15.25 ms interval for each time this function is called.

I fear that the high CPU utilization is the root cause of the print issue, causing the logger thread to be blocked for larger periods of time.

Does this issue happen "after a while" ? Meaning that from boot, you're able to read out for some period of time before it starts to print "dropped messages" in the log?

You can setup blocking printing, but this will then block other functions from executing.

You could try running the CPU at 128 MHz for debugging purposes:

https://github.com/nrfconnect/sdk-nrf/blob/main/samples/wifi/shell/src/main.c#L54-L55

If that does not work, then the only option is to optimize functions around, to ensure that the logger thread has the time to execute the prints.

Kind regards,

Håkon

RE: Data Logging

Kashyap23 — Mon, 09 Dec 2024 04:53:57 GMT

static void copyBuffer(int32_t *Buff_In,int16_t *Buff_Out,uint16_t number_of_samples)

{

for(int i = 0; i < number_of_samples; ++i)

{

Buff_Out[i]= Buff_In[i]>>16;

printk("%d\n",Buff_Out[i]);

}

}.I am using this logic each data of 32 bit is shifted to 16 bit and it prints,I am logging continuous data coming from i2s_buf_read function of buffer 244.

RE: Data Logging

Håkon Alseth — Wed, 04 Dec 2024 13:27:31 GMT

Hi,

How often are you logging? Ie. what is the throughput?

If you log each sample at 16 KHz, and also convert it to ASCII string, it is likely that this will take some time. And if your firmware is blocking the logger thread from running in longer intervals, it will overflow after a while

Kind regards,

Håkon