FDS CRC READ CHECK ERR

[kenyon]

sdk：17.1
example: ble_app_uart+ secure_bootloader.

// <h> CRC - CRC functionality

//==========================================================
// <e> FDS_CRC_CHECK_ON_READ - Enable CRC checks.

// <i> Save a record's CRC when it is written to flash and check it when the record is opened.
// <i> Records with an incorrect CRC can still be 'seen' by the user using FDS functions, but they cannot be opened.
// <i> Additionally, they will not be garbage collected until they are deleted.
//==========================================================
#ifndef FDS_CRC_CHECK_ON_READ
#define FDS_CRC_CHECK_ON_READ 1
#endif
// <o> FDS_CRC_CHECK_ON_WRITE  - Perform a CRC check on newly written records.
 

// <i> Perform a CRC check on newly written records.
// <i> This setting can be used to make sure that the record data was not altered while being written to flash.
// <1=> Enabled 
// <0=> Disabled 

#ifndef FDS_CRC_CHECK_ON_WRITE
#define FDS_CRC_CHECK_ON_WRITE 0
#endif

Recently my device sometimes returns FDS_ERR_CRC_CHECK_FAILED in this code 'err_code = fds_record_open(&desc, &config);APP_ERROR_CHECK(err_code);'

when it switches on and off.Eventually it died in the bootloader and couldn't get out.

This is not an inevitable situation, but it does happen. He can't fix it by DFU another firmware, unless it's a DFU upgrade package that FDS_CRC_CHECK_ON_READ set 0.

So I want to know, if I set FDS_CRC_CHECK_ON_READ to 0, will it have any bad effect on my data storage? Or is there any other way to solve the read crc error?

[Vidar Berg]

Hello,

The CRC error indicates that the data record was not stored correctly and I recommend investigating what is causing it. Disabling FDS_CRC_CHECK_ON_READ might only mask the problem. Could you share some code snippets that show how you write or update the data record that has the CRC errors? Maybe the record data is altered before the write is completed?

Best regards,

Vidar

[kenyon]

///*******************************************************************************
// Copyright (c) 2016 Lumen Labs (HK) Limited. All Rights Reserved.
//*******************************************************************************/

#include <string.h>
#include "fds.h"

#include "LL_Flash.h"

// <e> FDS_ENABLED - fds - Flash data storage module
//==========================================================
#ifndef FDS_ENABLED
#define FDS_ENABLED 1
#endif
// <h> Pages - Virtual page settings

// <i> Configure the number of virtual pages to use and their size.
//==========================================================
// <o> FDS_VIRTUAL_PAGES - Number of virtual flash pages to use. 
// <i> One of the virtual pages is reserved by the system for garbage collection.
// <i> Therefore, the minimum is two virtual pages: one page to store data and one page to be used by the system for garbage collection.
// <i> The total amount of flash memory that is used by FDS amounts to @ref FDS_VIRTUAL_PAGES * @ref FDS_VIRTUAL_PAGE_SIZE * 4 bytes.

#ifndef FDS_VIRTUAL_PAGES
#define FDS_VIRTUAL_PAGES 2
#endif

// <o> FDS_VIRTUAL_PAGE_SIZE  - The size of a virtual flash page.
 

// <i> Expressed in number of 4-byte words.
// <i> By default, a virtual page is the same size as a physical page.
// <i> The size of a virtual page must be a multiple of the size of a physical page.
// <1024=> 1024 
// <2048=> 2048 

#ifndef FDS_VIRTUAL_PAGE_SIZE
#define FDS_VIRTUAL_PAGE_SIZE 1024
#endif

// <o> FDS_VIRTUAL_PAGES_RESERVED - The number of virtual flash pages that are used by other modules. 
// <i> FDS module stores its data in the last pages of the flash memory.
// <i> By setting this value, you can move flash end address used by the FDS.
// <i> As a result the reserved space can be used by other modules.

#ifndef FDS_VIRTUAL_PAGES_RESERVED
#define FDS_VIRTUAL_PAGES_RESERVED 0
#endif

// </h> 
//==========================================================

// <h> Backend - Backend configuration

// <i> Configure which nrf_fstorage backend is used by FDS to write to flash.
//==========================================================
// <o> FDS_BACKEND  - FDS flash backend.
 

// <i> NRF_FSTORAGE_SD uses the nrf_fstorage_sd backend implementation using the SoftDevice API. Use this if you have a SoftDevice present.
// <i> NRF_FSTORAGE_NVMC uses the nrf_fstorage_nvmc implementation. Use this setting if you don't use the SoftDevice.
// <1=> NRF_FSTORAGE_NVMC 
// <2=> NRF_FSTORAGE_SD 

#ifndef FDS_BACKEND
#define FDS_BACKEND 2
#endif

// </h> 
//==========================================================

// <h> Queue - Queue settings

//==========================================================
// <o> FDS_OP_QUEUE_SIZE - Size of the internal queue. 
// <i> Increase this value if you frequently get synchronous FDS_ERR_NO_SPACE_IN_QUEUES errors.

#ifndef FDS_OP_QUEUE_SIZE
#define FDS_OP_QUEUE_SIZE 4
#endif

// </h> 
//==========================================================

// <h> CRC - CRC functionality

//==========================================================
// <e> FDS_CRC_CHECK_ON_READ - Enable CRC checks.

// <i> Save a record's CRC when it is written to flash and check it when the record is opened.
// <i> Records with an incorrect CRC can still be 'seen' by the user using FDS functions, but they cannot be opened.
// <i> Additionally, they will not be garbage collected until they are deleted.
//==========================================================
#ifndef FDS_CRC_CHECK_ON_READ
#define FDS_CRC_CHECK_ON_READ 0
#endif
// <o> FDS_CRC_CHECK_ON_WRITE  - Perform a CRC check on newly written records.
 

// <i> Perform a CRC check on newly written records.
// <i> This setting can be used to make sure that the record data was not altered while being written to flash.
// <1=> Enabled 
// <0=> Disabled 

#ifndef FDS_CRC_CHECK_ON_WRITE
#define FDS_CRC_CHECK_ON_WRITE 0
#endif

unsigned long gulFlashStoreNeeded = 0;////flash_store();                            
unsigned long gulFlashStoreCnt = 0x80000000;

#define CONFIG_FILE     (0x8010)
#define CONFIG_REC_KEY  (0x7010)

typedef struct
{
    unsigned char a;                   
    unsigned char b[8];  
}T_LL_HardwareID;

typedef struct
{
    T_LL_HardwareID HardwareID[2]; 
	unsigned char a1;
    unsigned char b1;
   	unsigned char c1;
    unsigned char d1;
    unsigned long e1;
}T_DeviceLumen;
///* A dummy structure to save in flash. */
typedef struct
{
    unsigned long a;
    T_DeviceLumen b;
    signed char c[6];
    unsigned long d;
    unsigned long e;
    unsigned long f;
    signed short g[3];
    signed short h;
    unsigned short i;
    unsigned char sn[20];
    unsigned char bright;
    unsigned long flash_flag;

    #define GET_BAR_STATE(states, bar_index)    ((states>>(bar_index<<1)) & 0x3) // 2 bit for each bar
    unsigned long customer_flashing_timeslot_states[6][5];   
    // Brightness of front light and rear panel
    unsigned char brightness[6];    
    // 校验码:
    unsigned long ulChecksum;
    
    unsigned char pnet;
    unsigned char netID[8];
    //
    #define getCustomizedSetting(xxx)  gtPara.xxx
		unsigned char Profiles[4];
		unsigned char mode;
		unsigned char profile_nn;
    //
    unsigned char netIDex;
    //
    unsigned char syncPOff;
    //
    unsigned char enabledhs;    
}T_Para;

T_Para gtPara = 
{
	.para_version = 0x01010102,
	.b.a1 = 0,
	.b.b1        = 0,
	.b.HardwareID[0] = 0x00,
	.b.HardwareID[1] = 0x00,
	.b.c1 = 0x00,
	.d = 0,
	.e = 2,
	.f = 0,
	.g[0] = 100,
	.g[1] = 0,
	.g[2] = 0,
	.h = -3500,
	.i = 70,
	.pnet = 0,
	.netIDex = 0,
	.syncPOff = 0,
	.profile_nn = 5,
};


/* A record containing dummy configuration data. */
static fds_record_t const gtPara_record =
{
    .file_id           = CONFIG_FILE,
    .key               = CONFIG_REC_KEY,
    .data.p_data       = &gtPara,
    /* The length of a record is always expressed in 4-byte units (words). */
    .data.length_words = (sizeof(gtPara) + 2) / sizeof(uint32_t),
};

/* Keep track of the progress of a delete_all operation. 
static struct
{
    bool delete_next;   //!< Delete next record.
    bool pending;       //!< Waiting for an fds FDS_EVT_DEL_RECORD event, to delete the next record.
} m_delete_all;
*/
/* Flag to check fds initialization. */
static bool volatile m_fds_initialized;
static bool volatile m_fds_writeflag;

static void fds_evt_handler(fds_evt_t const * p_evt)
{
    if (p_evt->result == NRF_SUCCESS)
    {      
    }
    else
    {
    }

    switch (p_evt->id)
    {
        case FDS_EVT_INIT:
            if (p_evt->result == NRF_SUCCESS)
            {
                m_fds_initialized = true;
            }
            break;

        case FDS_EVT_WRITE:
        {
            if (p_evt->result == NRF_SUCCESS)
            {
								m_fds_writeflag = true;
            }
        } break;

        case FDS_EVT_DEL_RECORD:
        {
            if (p_evt->result == NRF_SUCCESS)
            {

            }
            //m_delete_all.pending = false;
        } break;

        default:
            break;
    }
}

void flash_init(void)
{
   ret_code_t err_code;

   (void) fds_register(fds_evt_handler);//FDS register
	  err_code = fds_init();//fds init
	
    APP_ERROR_CHECK(err_code);
	
	  while (!m_fds_initialized)//wait init
    {
        sd_app_evt_wait();//standby
    }		
		
	  fds_stat_t stat = {0};
    err_code = fds_stat(&stat);//Set statistics
    APP_ERROR_CHECK(err_code);
	
		//record_delete_next();//clear all records
			
		fds_record_desc_t desc = {0};//The descriptor structure used to manipulate the record is zeroed out
    fds_find_token_t  tok  = {0};//The token that saves the key is cleared
		
    err_code = fds_record_find(CONFIG_FILE, CONFIG_REC_KEY, &desc, &tok);//Find data, file ID, record KEY
		
		
 if (err_code == NRF_SUCCESS)//if find successful
    {
			//set_default();
        /* A config file is in flash. Let's update it. */
        fds_flash_record_t config = {0};//res

        /* Open the record and read its contents. */
        err_code = fds_record_open(&desc, &config);//open record to read data
        APP_ERROR_CHECK(err_code);
    
				/* Copy the configuration from flash into m_dummy_cfg*/
        memcpy(&gtPara, config.p_data, sizeof(gtPara));

				//data = (uint32_t *)config.p_data;
				
        /* Close the record when done reading. */
        err_code = fds_record_close(&desc);//close record
        APP_ERROR_CHECK(err_code);
    }
		else //not found record, first write
		{
				//record_delete_next();//clear all records
				set_default();
				err_code = fds_record_write(&desc, &gtPara_record);//Write records and data. Descriptor, used later to access the record ID
				APP_ERROR_CHECK(err_code);	
				//while(!m_fds_writeflag);
		}			
}

void flash_store(void)
{
	ret_code_t err_code;
	fds_record_desc_t desc = {0};//The descriptor structure used to manipulate the record is zeroed out
    fds_find_token_t  tok  = {0};//The token that saves the key is cleared
	//fds_record_t record = {0};

	// Check the record exists
	err_code = fds_record_find( CONFIG_FILE, CONFIG_REC_KEY, &desc, &tok );
		
	/* Write the updated record to flash.*/
	if( err_code == NRF_SUCCESS){
		err_code = fds_record_update(&desc, &gtPara_record);
			if(err_code == FDS_ERR_NO_SPACE_IN_FLASH){
				err_code = fds_gc();
				gulFlashStoreNeeded = 1; //store data
			}
				
				APP_ERROR_CHECK(err_code);	
		
				fds_stat_t stat = {0};

				err_code = fds_stat(&stat);
				APP_ERROR_CHECK(err_code);
				
		}else{
				err_code = fds_record_write(&desc, &gtPara_record);//Write records and data. Descriptor, used later to access the record ID
				APP_ERROR_CHECK(err_code);					
		}			

}

int main(void)
{
    bool erase_bonds;
    ret_code_t err_code;
		
    // Initialize the async SVCI interface to bootloader before any interrupts are enabled.
    err_code = ble_dfu_buttonless_async_svci_init();
    APP_ERROR_CHECK(err_code);

	LL_HardwareID_Get(&gtHwID);    
    // Initialize.
    timers_init();
    buttons_leds_init(&erase_bonds);
    //power_management_init();

    ble_stack_init();
    gap_params_init();
    gatt_init();
    services_init();

    advertising_init();
    conn_params_init();
	//
    err_code = esb_timeslot_init();
    APP_ERROR_CHECK(err_code);
	
    err_code = esb_timeslot_sd_start();
    APP_ERROR_CHECK(err_code);

	LL_Key_Init();	
	flash_init();
	
	if(key_no_press){
		//enter sleep
	    sd_power_system_off();	
	}else{  //if key short press,power on
		//enter work
	}
	
	while(1){
		if(key_short_press){
			//wait 2s 
			timer_count_2s();
			//then storage data
			flash_store();
		}else if(key_long_press_2s){
			//enter sleep
		    sd_power_system_off();			
		} 
	}	
}

Hi Vidar 
I made a simple code, including flash driver initialization and how to store it.
Can you help to see if the flash storage data and write data will have problems?

Thanks very much!
Best regard!
Kenyon

[Vidar Berg]

Hi Kenyon,

Thanks. Just to confirm, are you able to reproduce the CRC errors with the code you posted above?

Best regards,

Vidar

[kenyon]

Hi Vidar
No, he's not a must. He may suddenly report a CRC error when you turn on the device one day. This is one of my more vexed points..

Best regards 
Kenyon

[Vidar Berg]

Hi Kenyon,

OK, I see the data record is stored in static memory, which is good. However, is there a chance that the 'gtPara' variable could be updated by the application before the flash write is complete?

The CRC of the 'gtPara' data is calculated when you call fds_record_write/fds_record_update(), so the CRC will become invalid if the data is altered between the time the flash operation is scheduled and when it is complete.

Best regards,

Vidar

[kenyon]

Hi Vidar

Vidar Berg said:

so the CRC will become invalid if the data is altered between the time the flash operation is scheduled and when it is complete.

I don't understand this because, as you can see from the above code, every time I decide to write flash area data, I wait for 2s before writing the data. This 2s time is sufficient for the last update of flash data.

Am I not waiting for the flash update data operation to complete before going to sleep, resulting in a CRC error?

Best Regard 
Kenyon

[Vidar Berg]

Hi Kenyon,

kenyon said:

I don't understand this because, as you can see from the above code, every time I decide to write flash area data, I wait for 2s before writing the data. This 2s time is sufficient for the last update of flash data.

I was wondering if there were other places in the application that may update the 'gtPara' variable while the FDS update task is pending. I would recommend you inspect the FDS pages in flash and see if you can find the data record and see if the data is stored correctly or not, or if it contain corrupted values. The storage format is documented here: Storage format.

kenyon said:

Am I not waiting for the flash update data operation to complete before going to sleep, resulting in a CRC error?

The File ID should also have been missing in that case, resulting in the data record not being found.

Best regards,

Vidar