Writing to external flash - too slow?

[nWre]

Hi again Nordic.

I'm trying to write sensor data to the external flash continuously as there is not enough RAM to hold all the data required. However, the writing operation is too slow and data gets lost.
I'm getting 4000 64-bits int every second from an SPI device and need to write this to Ext flash without losing a beat.

My code as of now is like this:

if (dataAvailable){
    readData(oneEntry);
    fs_write(oneEntry);
}

but this takes too much time and my sensor overflows. Is there a way to speed this up? Asynchronous writing to the external flash while filling a list? Has anyone come across this problem?

According to this post:https://devzone.nordicsemi.com/f/nordic-q-a/61735/external-flash-qspi-vs-internal-flash-write-performance the writespeed is 0,06 MB/s.

The recording takes up (64 * 4000)/8 = 0,032 MB/s so it should write fast enough. Maybe I should store the recording to the internal flash and then move it to the external once that the recording is complete? The entire recording takes up 0,512 MB. When trying to time the fs_write function there is times when it is fast enough and times it stalls. Maybe this is some sort of garbage collection and there is a more efficient way of writing data quickly than fs_write?

Would an SD-card connected via SPI be a better choice for this? Maybe the frequent writes to the flash memory will wear it out.

Thanks beforehand and happy holidays!

[Simon]

Hello, sorry for the delay on this. A huge part of the support team has been on vacation, but we will look into it the next week

Best regards,

Simon

[nWre]

No worries, the amount of questions I'm throwing your way would warrant vacation for the whole team :) I'm trying out an sd-card in the meantime and will update the thread accordingly.

[Simon]

I did a test with the nRF9160DK (write speed of littlefs-->spi_nor-->spi-->mx25r6435f ) and measured a speed of 0.0255MB/s. I have asked a developer internally if there is any ways to improve this or if you need to use an alternative solution. I will get back to you if I get any replies.

[Simon]

Hello, sorry for the delay on this. I did not get any answer internally unfortunately. Be aware that SPI is slower than QSPI (which was used in the ticket you linked to)

So I will take a closer look at this myself. It seems like you're going for an SD-card: https://devzone.nordicsemi.com/f/nordic-q-a/83552/sd-card-on-nrf9160dk 

Have you got any progress?

[nWre]

Hi! Yes, I got an sd-card reader working and the write speed is very good. However, there is a delay when the data being written exceeds the cluster size(in my case 4096 bits). I'm currently working on this as the delay is so small it doesn't occur everytime. My databuffer overflows after 23ms so the resizing after exceeding cluster must be less than that and right now it's between 20-24 ms but not consistent. Maybe that deserves its own thread and it might be a more generic problem with fat32 and zephyr than nordic. I guess this can be closed with the conclusion: SD-cards are faster than the external flash when writing data.