No regular time to write data on sd card

[swip]

Hello all,

I have noticed that a delay appear periodically when writing SD card for example, I have something that the following code

       

int iNumbers=10; //Size of the data I want to save 
int iTime=5; //iTime the size of the time corresponding to the value 
int8_t *size=malloc(sizeof(*size));
*size=iNumbers+iTime;

for(int i=0; i<1000; i++){
    char * buffer = malloc((iNumbers+iTime)*sizeof(*buffer));
    get_value_and_write_into_buffer(buffer);
    f_write(&file, buffer, *size, (UINT *) &bytes_written); //function to write on SD card
}

Unfortunately, I have a jump into the values when looking at the time written on the SD card at the end of the program. The longer the data is, the closer the discontinuous time will be. If iNumbers=5; I have a jump every 70 loop increments over i and if iNumbers=15; it will be every 20 loop increments.

Obviously I don't have any problem, if I save the 1000 data into a big array and write it after on the SD card. In this second case the clock is continuous. Therefore the only possible reason is something happen when writing the data.

Do you have any suggestion with what I am struggling with

Thank you for your help !

[MartinBL]

Hi,

How do you write to the SD card? Could it be that you are writing too fast so that each write operation doesn't have time to complete before your for loop starts another write operation?

Can you share a log?

Can you share your code?

[swip]

Hello Martin,

Below you can see a log file. The "error sensor time data" means the the buffer has not been over-read, the fifo is not empty, but it is not crucial. But the "error skipped frame count" is very problematic. It occurs periodically as you can see in the following log file.

Initializing disk 0 (SDC)... Capacity: 3796 MB 
Mounting volume... 
Successfully deleted file: file.csv.
flushing data from fifo before starting 1003 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 15 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 45 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 44 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 45 -- ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 33 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 44 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 43 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 44 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 45 -- ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 16 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 45 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 43 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 45 -- ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 31 -- ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 15 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 44 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 44 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 45 -- ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 31 -- ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 17 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 44 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 45 -- ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 47 -- ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SKIPPED FRAME COUNT : 31 -- ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 
ERROR SENSOR TIME DATA : 0 

The main.c file is

for(;;)
        {
            gyro_frames_req = 32;
            accel_frames_req = 32;
            BMI160->fifo->length=220;
            nrf_delay_ms(1);
            get_fifo_data(BMI160,gyro_data,&gyro_frames_req,accel_data,&accel_frames_req);
            if(accel_frames_req!=gyro_frames_req)
                SEGGER_RTT_printf(0,"bad gyro diff from accel \n");
            SEGGER_RTT_printf(0,"accel_frames_req %d \n",accel_frames_req);
            for(int frame=0; frame<accel_frames_req; frame++){
                ff_result=f_write(&file,&accel_data[frame].x, 2, (UINT *) &bytes_written);
                ff_result+=f_write(&file,&accel_data[frame].y, 2, (UINT *) &bytes_written);
                ff_result+=f_write(&file,&accel_data[frame].z, 2, (UINT *) &bytes_written);
                ff_result+=f_write(&file,&BMI160->fifo->sensor_time, 4, (UINT *) &bytes_written);
                ff_result+=f_write(&file,&gyro_data[frame].x, 2, (UINT *) &bytes_written);
                ff_result+=f_write(&file,&gyro_data[frame].y, 2, (UINT *) &bytes_written);
                ff_result+=f_write(&file,&gyro_data[frame].z, 2, (UINT *) &bytes_written);
                if (ff_result != FR_OK)
                {
                    SEGGER_RTT_printf(0,"Write failed\r\n.");
                    break;
                }
            }
        }

Even if you don't have the detail of each function, the name are quiet self-understandable. Three steps in the infinite loop :

 - wait to be sure the fifo has a value

 - get the value from the fifo

 - write the value on the sd card

Most of the time there is absolutely no problem and the average fifo length (the loop on the number of frame) is 3 frames. But for time to time, unexpectedly the f_write function on the SD card take a lot of time and the fifo overflows.

May be, when using the sd card the memory is not continuous  and it takes some time to find another sector on the card ? I can use an empty card, do you think it is possible to write the data more continuously ?

Thank you for your answer,

Swip

[MartinBL]

Hi,

Sorry for the slow response. I have been out of office for a while.

Have you found a workaround?

do you think it is possible to write the data more continuously?
If I understand you correctly, you write individual samples with time stamps to your SD cards, causing a lot of write operations and overhead. Maybe it could be an idea to temporarily store a number samples in a software buffer and write them all at once to reduce the write frequency. Sort of like what user rct42 suggests here: SD Write Problem.

[swip]

Hi,

I haven't found a completely workaround for this issue.

Yes, you get it right. I have a lot of write operations with time stamps on my SD card. I am using the time written on the sd card to see the lack of data. I have tried to use a software buffer to store my data and an gpiote interrupt to get the data from the sensor to the software buffer. But I am facing trouble working with gpiote and ble now.

For some reasons, the program does not jump into the interrupt handler function when running the program with ble. I don't know if you have an idea why ? 

And finally yes, my problem is a lot like the link given in your previous rely. The code gets trapped in the write function for a while from time to time and it is not absurd to see an increase in current consumption.

[MartinBL]

For some reasons, the program does not jump into the interrupt handler function when running the program with ble. I don't know if you have an idea why ?

1. Does your application otherwise work as intended? No errors or asserts?
2. Does your interrupt handler work when you don't use BLE?
3. Can you upload your code?