Nor memory QSPI efficient data logging

[vincent.golle]

Dear Nordic,

For my application I am now using a NOR memory with my nRF52840.

This memory needs the FAT file system as my device implements USB MSC and uses regular files.

My app needs to log data every second (about 20 bytes per write), and this logged data can amount to several MB in total before it is unloaded.

I have noticed that using a file in append mode in the NOR memory takes (very) long processing times... and can also crash the system !

My question is the following:

How an I efficiently store this data in my NOR flash memory using the nrf_qspi library ?

[Hung Bui]

Dear customer, 

Due to limited staff during the holiday period, we unfortunately can't handle your case before we get back to the normal staff by the first week of January 2019. We are sorry indeed for the delay. We wish you a great holiday and a Happy New Year 2019. 

Best Regards, 

Nordic Tech Support Team  

[haakonsh]

"I have noticed that using a file in append mode in the NOR memory takes (very) long processing times... and can also crash the system !"

Do you have any more information to share on this issue?

Are you buffering any of the data before writing it to the memory chip?

[vincent.golle]

Yes I do !

I have a fixed size struct array that I use to store data (in a floating point format).

One struct contains 6 floating points values.

This buffer is printed to the file as soon as it is full (using snprintf followed by a f_write).

With my current QSPI flash parameters I measured:

73ms for the write of a buffer of size 1 (i.e. 6 floats)

121ms for the write of a buffer of size 20 (i.e. 120 floats)

I currently append the data as a newline at the end of the file. I also noticed that this operation can crash my system...

I currently have not implemented anything to check the available space on the memory before writing.

for (size_t i=0; i< nb_pos; i++) {
		// print histo
		int to_wr = snprintf(g_bufferWrite, sizeof(g_bufferWrite), "%f;%f;%f;%lu;%d\r\n",
				att[i].loc.lat, att[i].loc.lon,
				att[i].loc.alt, att[i].date.secj,
				att[i].pwr);

		if (to_wr > 0 && to_wr < (int)sizeof(g_bufferWrite)) {
			f_write(&g_fileObject, g_bufferWrite, to_wr, NULL);
		} else {
			APP_ERROR_CHECK(0x2);
		}

		perform_system_tasks();
		//yield();
}

[haakonsh]

One source of QSPI write errors is not using lengths of 4 bytes (32-bit word alignment).

121ms to write 120 floats seems a bit too long.

1. What's your QSPI frequency? 
2. Are you running single, dual, or quad data lines?
3. What is the specified write speed of the NOR memory? 
4. Are you running any floating point operations on your data? 

[vincent.golle]

The NOR chip is running at 4MHz (MT25QL128 from Micron) in a classic SPI mode (single data lines).

The data that is saved is a copy of the data used by the program, which indeed is used in A LOT of floating point operations. But what would be the relation to the issue ?