BLE+FDS Problem to read/write flash data err

RE: BLE+FDS Problem to read/write flash data err

Hung Bui — Wed, 25 Sep 2024 10:53:00 GMT

Hi Zicongkuang,
I don't think there is a workaround to avoid assert&restart except for that you make sure there shouldn't be a problem.

One thing you can do though, is that if the err_code returns FDS_ERR_BUSY then you can wait and try again after a while instead of reset. But make sure you don't end up in a dead loop. You should trigger a reset if the same error returned for a while. This make sure the product can recover with a reset.

RE: BLE+FDS Problem to read/write flash data err

kenyon — Tue, 24 Sep 2024 04:27:01 GMT

Hi Hung Bui
Thank you very much for your advice. After the one-month test, our products have no major problems.
But I have one last question, to avoid APP_ERROR_CHECK(err_code); Cause the device to restart.
So I would like to know if there are any errors in the following functions that need to happen easily and need to be handled in time？

err_code = fds_init();
err_code = fds_stat(&stat);
err_code = fds_record_find(CONFIG_FILE, CONFIG_REC_KEY, &desc, &tok);
err_code = fds_record_open(&desc, &config);
err_code = fds_record_close(&desc);
err_code = fds_record_delete(&desc);
err_code = fds_gc();
err_code = fds_record_update(&desc, &gtPara_record);

Thanks！
Best Regard!
Kenyon

RE: BLE+FDS Problem to read/write flash data err

Hung Bui — Tue, 20 Aug 2024 08:24:40 GMT

Hi Zicongkuang,

Yes it's the case. You can take a look at this similar ticket here:
Flash Data Storage (FDS) fds_record_update() -> Reset -> Two valid records possible?

Please follow the suggestion from tesc to use the record_id to check which one is the latest one. If you find duplicated records, just delete the ones with smaller record_id.

RE: BLE+FDS Problem to read/write flash data err

kenyon — Tue, 20 Aug 2024 02:51:16 GMT

[quote userid="2121" url="~/f/nordic-q-a/113792/ble-fds-problem-to-read-write-flash-data-err/498761"]Could you give a brief info on how to test it with your code ?
Could you elaborate more on " call collision of fds_update " ? [/quote]

The code is very simple,flash_store is invoked by fleetly pressing keys repeatedly to create fds_update conflicts with the scheduled flash_store.

[quote userid="2121" url="~/f/nordic-q-a/113792/ble-fds-problem-to-read-write-flash-data-err/498761"]The way it works is that fds_update will write the content of the new record to a new location in flash and mark the old one as invalid. There could be a chance that if the reset occurs right after the new data written in flash but the old record has not been marked as dirty then it will cause duplicated records. [/quote]

I think that might be the case.

I look forward to your verification.

RE: BLE+FDS Problem to read/write flash data err

Hung Bui — Mon, 19 Aug 2024 08:07:53 GMT

Hi,
Thanks for the code. I will take a look.
Could you give a brief info on how to test it with your code ?
Could you elaborate more on " call collision of fds_update " ?
The way it works is that fds_update will write the content of the new record to a new location in flash and mark the old one as invalid. There could be a chance that if the reset occurs right after the new data written in flash but the old record has not been marked as dirty then it will cause duplicated records.
I will check internally to see if it's the case and if we have a solution.

RE: BLE+FDS Problem to read/write flash data err

kenyon — Thu, 15 Aug 2024 16:23:32 GMT

Hi Hung Bui

I made a DK and was able to reproduce the problem on the development board (52832).
I found that the call collision of fds_update caused two valid data to appear instead of fds_write. If there are any errors in the code, please correct, thank you very much.

If fds_update does cause two valid pieces of data, how can we avoid it?

devzone.nordicsemi.com/.../TestFlash_5F00_sdk17.7z

RE: BLE+FDS Problem to read/write flash data err

Hung Bui — Wed, 14 Aug 2024 14:31:06 GMT

Hi again,
How long did it take to reproduce the issue ?
Can you reproduce it on a DK ? If you can send a minimal sample that I can test here I can try to investigate.

RE: BLE+FDS Problem to read/write flash data err

kenyon — Wed, 14 Aug 2024 11:21:06 GMT

Hi
In the test I did just now, I didn't find that they called fds_write the second time, but I did find two more valid data in flash.

RE: BLE+FDS Problem to read/write flash data err

Hung Bui — Wed, 14 Aug 2024 11:00:19 GMT

Hi,

I don't think turning that on would prevent the duplicate write issue. The setting is to check if the data writen has been modified when writing. If you have a reset in the middle of the writing process, in theory the data should just be marked as non-valid (not marked as valid).
When you turn on FDS_CRC_CHECK_ON_READ a corrupted record will still can be found but not open. Have you make sure you can open the duplicated records ?
As I mentioned, please try to check and make sure you are not calling write fds_record_write() twice for the same record.

RE: BLE+FDS Problem to read/write flash data err

kenyon — Wed, 14 Aug 2024 09:20:21 GMT

Hi Hung Bui

[quote userid="2121" url="~/f/nordic-q-a/113792/ble-fds-problem-to-read-write-flash-data-err/498063"]To make sure you are not calling write twice, you should have a semaphore to wait for the event FDS_EVT_WRITE in fds_evt_handler() to be success before you call write again. [/quote]

This, in my opinion, only prevents two consecutive calls to fds_write caused by faulty software logic. Because the act of charging RESET is uncontrollable and random.Is it right？

#ifndef FDS_CRC_CHECK_ON_READ
#define FDS_CRC_CHECK_ON_READ 1
#endif

#ifndef FDS_CRC_CHECK_ON_WRITE
#define FDS_CRC_CHECK_ON_WRITE 0
#endif

I only turned on the read CRC，I may need to turn on write crc as well.
But I have a question: if there is no CRC write check, will fds_update also probably result in two sets of valid data in the same KEY &&FILE ID?

RE: BLE+FDS Problem to read/write flash data err

Hung Bui — Tue, 13 Aug 2024 13:06:48 GMT

Hi Kenyon,

To make sure you are not calling write twice, you should have a semaphore to wait for the event FDS_EVT_WRITE in fds_evt_handler() to be success before you call write again.

I don't think doing reset multiple times when writing can cause the problem. If the record is written correctly before the reset, then the next fds_record_find() should be able to find it. If the writing is not finished, the CRC will not match.

RE: BLE+FDS Problem to read/write flash data err

kenyon — Tue, 13 Aug 2024 12:11:22 GMT

Hi Hung Bui
I can reproduce the issue，maybe need a long time to reproduce.
Because the charging port of our device is equipped with hardware reset function，I reproduce this issue by continuously charging reset(write flash) + key_longPress() (write flash).

#include <string.h>
#include "fds.h"

unsigned long gulFlashStoreNeeded = 0;////flash_store();                            
unsigned long gulFlashStoreCnt = 0x80000000;

#define CONFIG_FILE     (0x8010)
#define CONFIG_REC_KEY  (0x7010)
T_Para gtPara;

extern T_LL_HardwareID gtHwID;
///* A dummy structure to save in flash. */
typedef struct
{
    unsigned long p_v;
    T_DeviceLumen temote;
    signed char scBleAddr[6];
    E_Mode eMOWL; //
    unsigned long ulB;
    unsigned long eBmo;
    unsigned long ulNeedCharge;
    signed short sxyz[3];
    signed short ssThd;
    unsigned short usAdcValue;
    unsigned char aucSN[20];
    unsigned char ucBLight;
    E_B_M_O_P_O_O b_m_f;
    unsigned long flash_erase_flag;
    #define GET_BAR_STATE(states, bar_index)    ((states>>(bar_index<<1)) & 0x3) // 2 bit for each bar
    unsigned long c_f_t_s[6][7];   
    unsigned char brightness[6];    
    unsigned long ulChecksum;
    unsigned char publicNet;
    unsigned char netID[4];
    #define getCustomizedSetting(xxx)  gtPara.xxx
	unsigned char Profiles[4];
	unsigned char mode;
	unsigned char profile_nn;
    unsigned char netID_e;
    unsigned char PowerOff;
    unsigned char enabledHallSensor;    
}T_Para; //284 bytes

static T_Para gtStorePara = 
{
	.p_v = 0x01010102,
	.temote.ulRoundRobin = 0,
	.temote.ulIng        = 0,
	.temote.ID[0] = 0x00,
	.temote.ID[1] = 0x00,
	.temote.bitsc = 0x00,
	.eMOWL = 0,
	.ulB = 0,
	.eBmo = 0x02,
	.ulNeedCharge = 0,
	.sxyz[0] = 100,
	.sxyz[1] = 0,
	.sxyz[2] = 0,
	.ssThd = -3500,
	.ucBLight = 70,
	.publicNet = 0,
	.netID_e = 0,
	.PowerOff = 0,
	.b_m_f = 0x06,
};


/* A record containing dummy configuration data. */
static fds_record_t const gtPara_record =
{
    .file_id           = CONFIG_FILE,
    .key               = CONFIG_REC_KEY,
    .data.p_data       = &gtStorePara,
    /* The length of a record is always expressed in 4-byte units (words). */
    .data.length_words = (sizeof(gtStorePara) + sizeof(uint32_t) - 1) / sizeof(uint32_t),
};


static bool volatile m_fds_initialized;
static bool volatile m_fds_writeflag;
static bool volatile m_fds_gc_done = true;

static void fds_evt_handler(fds_evt_t const * p_evt)
{

    switch (p_evt->id)
    {
        case FDS_EVT_INIT:
            if (p_evt->result == NRF_SUCCESS)
            {
                m_fds_initialized = true;
            }
            break;

        case FDS_EVT_WRITE:
        case FDS_EVT_UPDATE:
        {
            if (p_evt->result == NRF_SUCCESS)
            {
				m_fds_writeflag = true;
            }                
        }break;
        case FDS_EVT_DEL_RECORD:
        {
            if (p_evt->result == NRF_SUCCESS)
            {

            }
            //m_delete_all.pending = false;
        } break;
        case FDS_EVT_GC:
            if (p_evt->result == NRF_SUCCESS)
            {
                LL_LED_OFF(E_LL_LED_PCB_G);
                m_fds_gc_done = true;
            }
        default:
            break;
    }
}


void flash_init(void)
{
   ret_code_t err_code;
    
    //write_flag init
    m_fds_writeflag = true;
    
   (void) fds_register(fds_evt_handler);//FDS register
    err_code = fds_init();//fds init
	
    APP_ERROR_CHECK(err_code);
	
	while (!m_fds_initialized)//wait init
    {
        sd_app_evt_wait();//standby
    }		
		
	fds_stat_t stat = {0};
    err_code = fds_stat(&stat);//Set statistics
    APP_ERROR_CHECK(err_code);
	
	//record_delete_next();//clear all records
			
	fds_record_desc_t desc = {0};//The descriptor structure used to manipulate the record is zeroed out
    fds_find_token_t  tok  = {0};//The token that saves the key is cleared
		
    err_code = fds_record_find(CONFIG_FILE, CONFIG_REC_KEY, &desc, &tok);//Find data, file ID, record KEY
		
		
 if (err_code == NRF_SUCCESS)//if find successful
    {
        /* A config file is in flash. Let's update it. */
        fds_flash_record_t config = {0};//res

        /* Open the record and read its contents. */
        err_code = fds_record_open(&desc, &config);//open record to read data

        APP_ERROR_CHECK(err_code); 
        /* Copy the configuration from flash into m_dummy_cfg*/
        memcpy(&gtPara, config.p_data, sizeof(gtPara));

        //data = (uint32_t *)config.p_data;
                
        /* Close the record when done reading. */
        err_code = fds_record_close(&desc);//close record
        APP_ERROR_CHECK(err_code);
    }else{
    	set_default; //set all data default
    	gulFlashStoreNeeded = 1; //store data
	}			
}

bool flash_gc_finish(void)
{
    return m_fds_gc_done;
}

bool flash_store_finish(void)
{
    return m_fds_writeflag;
}

void flash_store(void)
{
		ret_code_t err_code;
		fds_record_desc_t desc = {0};//The descriptor structure used to manipulate the record is zeroed out
        fds_find_token_t  tok  = {0};//The token that saves the key is cleared
   
        memcpy(&gtStorePara, &gtPara, sizeof(gtPara));
        
        //write_flag set
        m_fds_writeflag = false;
		// Check the record exists
		err_code = fds_record_find( CONFIG_FILE, CONFIG_REC_KEY, &desc, &tok );
		
		/* Write the updated record to flash.*/
		if( err_code == NRF_SUCCESS){
            err_code = fds_record_update(&desc, &gtPara_record);
            if(err_code == FDS_ERR_NO_SPACE_IN_FLASH){
                m_fds_gc_done = false;
                LL_LED_ON(E_LL_LED_PCB_G);
                err_code = fds_gc();
                APP_ERROR_CHECK(err_code);
                
                gulFlashStoreNeeded = 1; //store data
            }               
    
            fds_stat_t stat = {0};
            err_code = fds_stat(&stat);
            APP_ERROR_CHECK(err_code);				
		}else{
            err_code = fds_record_write(&desc, &gtPara_record);//Write records and data. Descriptor, used later to access the record ID
            APP_ERROR_CHECK(err_code);					
		}			
}

#define LL_FLASH_WAITING_TIME_BEFORE_SLEEP  (98 + 338*1024/1000 + 1280) + 1000    // 98ms page-erase, 338us 4-byte-write, 1024 4-byte per page, 1280ms top-prior-BLE-adv. (refer to "chapter 9 Flash memory API" of "S132_SDS_v2.0.pdf")
void LL_Power_BeforeSleep(void)
{
    //Store data and write crc checks again before sleep if charge or some data has not written
    if(gulFlashStoreNeeded){
        flash_store();
    }	
    
    //wait flash write success 
    if(!flash_store_finish()){ 
        for(uint16_t i=0; i<LL_FLASH_WAITING_TIME_BEFORE_SLEEP; i++){
            if(flash_store_finish()){break;}
            nrf_delay_ms(1);
        }
    }
}

int main(void)
{
    bool erase_bonds;
    ret_code_t err_code;
		
    // Initialize the async SVCI interface to bootloader before any interrupts are enabled.
    err_code = ble_dfu_buttonless_async_svci_init();
    APP_ERROR_CHECK(err_code);
   
    // Initialize.
    timers_init();

    ble_stack_init();
    gap_params_init();
    gatt_init();
    services_init();

    advertising_init();
    conn_params_init();
		//
#ifdef  _LL_TIMESLOT
    err_code = esb_timeslot_init();
    APP_ERROR_CHECK(err_code);
	
    err_code = esb_timeslot_sd_start();
    APP_ERROR_CHECK(err_code);
#endif

	LL_Key_Init();
		
    LL_Battery_Charging__Init();
        
	flash_init();

    // system off if not from key trigger or charging.
	if(LL_Battery_Charge__isCharging()) {  //if charging
        gtPara.ulNeedCharge = 0; gulFlashStoreNeeded = 1;  //store gtPara.ulNeedCharge
        LL_Power_BeforeSleep(); 
        LL_Power_Sleep(); 
    }else if( key_shortPress() ) {		// if key pressed
       	// continue			       
    }else{
        LL_Power_BeforeSleep(); 
        sd_power_system_off();            
    }
    
	// Start execution.
    advertising_start();//	
	
	while(1)
	{
		if(gulFlashStoreNeeded == 1){
			gulFlashStoreNeeded = 2;
			TimeStart(0);//Start the clock here 
		}else if(gulFlashStoreNeeded == 2){
			if(TimeStart(2000)) {//If you time it for 2000ms, start storing data
				flash_store();	
				gulFlashStoreNeeded = 0; //Wait for the next data store
			}
		}
		
		if( key_longPress()) { //if key longPress
			gtPara.ulNeedCharge = 1; gulFlashStoreNeeded = 1;
        	LL_Power_BeforeSleep(); 
        	sd_power_system_off();
		}
	}
}

When I checked thoroughly today, I found that it was because I did not know when I called err_code = fds_record_write(&desc, &gtPara_record) twice; The result is that two pieces of data are stored under the same KEY and File ID.
In my code, the condition for calling fds_record_write is err_code = fds_record_find(CONFIG_FILE, CONFIG_REC_KEY, &desc, &tok); fails to call fds_record_write again.

I wonder if the constant charging reset will cause this situation in flash？

RE: BLE+FDS Problem to read/write flash data err

Hung Bui — Mon, 12 Aug 2024 12:41:52 GMT

Hi Kenyon,

Have you managed to reproduce the issue ? If not please double check the code to see if there is any chance that you created two record with the same key ?
How did you find out the problem ? Did it has any relation to a DFU update ?

RE: BLE+FDS Problem to read/write flash data err

kenyon — Mon, 12 Aug 2024 11:26:31 GMT

Hi Hung Bui

#define CONFIG_FILE (0x8010)
#define CONFIG_REC_KEY (0x7010)
static fds_record_t const gtPara_record =
{
.file_id = CONFIG_FILE,
.key = CONFIG_REC_KEY,
.data.p_data = &gtStorePara,
/* The length of a record is always expressed in 4-byte units (words). */
.data.length_words = (sizeof(gtStorePara) + sizeof(uint32_t) - 1) / sizeof(uint32_t),
};

We are constantly updating data under the same File ID and Record Key. So I think in flash we should only have one valid data. But strangely enough, we found two valid bits of data in flash using the hex file. Why is that?
Because of the existence of two valid data, we have to read the data two times each time to read out the latest update data.How can I avoid this?

Best regard
Kenyon

RE: BLE+FDS Problem to read/write flash data err

Hung Bui — Mon, 12 Aug 2024 07:16:34 GMT

Hi Kenyon,
Could you please explain a little bit more on " there were two valid flash data in the flash area of the hex file" ?
Please describe what do you expect and what was wrong with that. Did it have anything to do with FDS_ERR_NO_SPACE_IN_FLASH issue ?

You mentioned that you used secure bootloader, was the problem related to the DFU update ?

RE: BLE+FDS Problem to read/write flash data err

kenyon — Fri, 09 Aug 2024 17:21:06 GMT

Hi Hung Bui

After I used the FDS module, there was a problem found that there were two valid flash data in the flash area of the hex file, why is this, is there a solution? Or how can we avoid it？

Best Regard
Kenyon

RE: BLE+FDS Problem to read/write flash data err

Hung Bui — Fri, 09 Aug 2024 12:39:25 GMT

Hi Zicongkuang,

1. It depends. The gc process basically erase all the records that's marked as "dirty" . It does it by copy the "valid" records to the swap page, erase the page and copy the valid record to the current page. So the process may take longer time if you have a lot of pages to copy&erase.
As mentioned in the documentation, you should wait for the FDS_EVT_GC event with NRF_SUCCESS status before continue. But I don't think it would take very long time (usually less than a second). You can find the timing specification here: https://docs.nordicsemi.com/bundle/ps_nrf52810/page/nvmc.html#unique_728973123

2. There is a checksum at each step in gc process. A page is backed up before it's erased. So there shouldn't be a situation when you lose your data when interrupted when doing gc. I don't think the GC process automatically resume after a reset, so you may need to call it again if you run out of flash again.

It's a good practice to use fds_stat to monitor the amount of flash available instead of waiting for FDS_ERR_NO_SPACE_IN_FLASH error.