BLE+FDS Problem to read/write flash data err

[kenyon]

sdk：17.1
example: ble_app_uart+ secure_bootloader.
Our device is charged with power-off reset function.

When I call fds_record_update(&desc, &gtPara_record),if returns FDS_ERR_NO_SPACE_IN_FLASH, which means I need to call to clean up the garbage.So i need to call err_code = fds_gc();Right？

// <e> FDS_ENABLED - fds - Flash data storage module
//==========================================================
#ifndef FDS_ENABLED
#define FDS_ENABLED 1
#endif
// <h> Pages - Virtual page settings

// <i> Configure the number of virtual pages to use and their size.
//==========================================================
// <o> FDS_VIRTUAL_PAGES - Number of virtual flash pages to use. 
// <i> One of the virtual pages is reserved by the system for garbage collection.
// <i> Therefore, the minimum is two virtual pages: one page to store data and one page to be used by the system for garbage collection.
// <i> The total amount of flash memory that is used by FDS amounts to @ref FDS_VIRTUAL_PAGES * @ref FDS_VIRTUAL_PAGE_SIZE * 4 bytes.

#ifndef FDS_VIRTUAL_PAGES
#define FDS_VIRTUAL_PAGES 2
#endif

// <o> FDS_VIRTUAL_PAGE_SIZE  - The size of a virtual flash page.
 

// <i> Expressed in number of 4-byte words.
// <i> By default, a virtual page is the same size as a physical page.
// <i> The size of a virtual page must be a multiple of the size of a physical page.
// <1024=> 1024 
// <2048=> 2048 

#ifndef FDS_VIRTUAL_PAGE_SIZE
#define FDS_VIRTUAL_PAGE_SIZE 1024
#endif

// <o> FDS_VIRTUAL_PAGES_RESERVED - The number of virtual flash pages that are used by other modules. 
// <i> FDS module stores its data in the last pages of the flash memory.
// <i> By setting this value, you can move flash end address used by the FDS.
// <i> As a result the reserved space can be used by other modules.

#ifndef FDS_VIRTUAL_PAGES_RESERVED
#define FDS_VIRTUAL_PAGES_RESERVED 0
#endif

// </h> 
//==========================================================

// <h> Backend - Backend configuration

// <i> Configure which nrf_fstorage backend is used by FDS to write to flash.
//==========================================================
// <o> FDS_BACKEND  - FDS flash backend.
 

// <i> NRF_FSTORAGE_SD uses the nrf_fstorage_sd backend implementation using the SoftDevice API. Use this if you have a SoftDevice present.
// <i> NRF_FSTORAGE_NVMC uses the nrf_fstorage_nvmc implementation. Use this setting if you don't use the SoftDevice.
// <1=> NRF_FSTORAGE_NVMC 
// <2=> NRF_FSTORAGE_SD 

#ifndef FDS_BACKEND
#define FDS_BACKEND 2
#endif

// </h> 
//==========================================================

// <h> Queue - Queue settings

//==========================================================
// <o> FDS_OP_QUEUE_SIZE - Size of the internal queue. 
// <i> Increase this value if you frequently get synchronous FDS_ERR_NO_SPACE_IN_QUEUES errors.

#ifndef FDS_OP_QUEUE_SIZE
#define FDS_OP_QUEUE_SIZE 4
#endif

// </h> 
//==========================================================

// <h> CRC - CRC functionality

//==========================================================
// <e> FDS_CRC_CHECK_ON_READ - Enable CRC checks.

// <i> Save a record's CRC when it is written to flash and check it when the record is opened.
// <i> Records with an incorrect CRC can still be 'seen' by the user using FDS functions, but they cannot be opened.
// <i> Additionally, they will not be garbage collected until they are deleted.
//==========================================================
#ifndef FDS_CRC_CHECK_ON_READ
#define FDS_CRC_CHECK_ON_READ 1
#endif
// <o> FDS_CRC_CHECK_ON_WRITE  - Perform a CRC check on newly written records.
 

// <i> Perform a CRC check on newly written records.
// <i> This setting can be used to make sure that the record data was not altered while being written to flash.
// <1=> Enabled 
// <0=> Disabled 

#ifndef FDS_CRC_CHECK_ON_WRITE
#define FDS_CRC_CHECK_ON_WRITE 0
#endif

I have two questions about that.
1. How long does it take to clean fds gc.
2. What happens if the fds_gc is interrupted by a charging reset during cleanup?

Best regard
Kenyon

[Hung Bui]

Hi Zicongkuang, 

1. It depends. The gc process basically erase all the records that's marked as "dirty" . It does it by copy the "valid" records to the swap page, erase the page and copy the valid record to the current page. So the process may take longer time if you have a lot of pages to copy&erase. 
As mentioned in the documentation, you should wait for the FDS_EVT_GC event with NRF_SUCCESS status before continue. But I don't think it would take very long time (usually less than a second). You can find the timing specification here: https://docs.nordicsemi.com/bundle/ps_nrf52810/page/nvmc.html#unique_728973123

2. There is a checksum at each step in gc process. A page is backed up before it's erased. So there shouldn't be a situation when you lose your data when interrupted when doing gc. I don't think the GC process automatically resume after a reset, so you may need to call it again if you run out of flash again. 

It's a good practice to use  fds_stat to monitor the amount of flash available instead of waiting for FDS_ERR_NO_SPACE_IN_FLASH error. 

[kenyon]

Hi Hung Bui


After I used the FDS module, there was a problem found that there were two valid flash data in the flash area of the hex file, why is this, is there a solution? Or how can we avoid it？

Best Regard
Kenyon

[Hung Bui]

Hi, 
Thanks for the code. I will take a look.
Could you give a brief info on how to test it with your code ? 
Could you elaborate more on " call collision of fds_update " ? 
The way it works is that fds_update will write the content of the new record to a new location in flash and mark the old one as invalid. There could be a chance that if the reset occurs right after the new data written in flash but the old record has not been marked as dirty then it will cause duplicated records. 
I will check internally to see if it's the case and if we have a solution. 

[kenyon]

Hi

Hung Bui said:

Could you give a brief info on how to test it with your code ? 
Could you elaborate more on " call collision of fds_update " ? 

The code is very simple,flash_store is invoked by fleetly pressing keys repeatedly to create fds_update conflicts with the scheduled flash_store.

Hung Bui said:

The way it works is that fds_update will write the content of the new record to a new location in flash and mark the old one as invalid. There could be a chance that if the reset occurs right after the new data written in flash but the old record has not been marked as dirty then it will cause duplicated records. 

I think that might be the case.

I look forward to your verification.

[Hung Bui]

Hi Zicongkuang, 

Yes it's the case. You can take a look at this similar ticket here: 
Flash Data Storage (FDS) fds_record_update() -> Reset -> Two valid records possible?

Please follow the suggestion from tesc to use the record_id to check which one is the latest one. If you find duplicated records, just delete the ones with smaller record_id. 

[kenyon]

Hi Hung Bui
Thank you very much for your advice. After the one-month test, our products have no major problems.
But I have one last question, to avoid APP_ERROR_CHECK(err_code); Cause the device to restart.
So I would like to know if there are any errors in the following functions that need to happen easily and need to be handled in time？

• err_code = fds_init();
• err_code = fds_stat(&stat);
• err_code = fds_record_find(CONFIG_FILE, CONFIG_REC_KEY, &desc, &tok);
• err_code = fds_record_open(&desc, &config);
• err_code = fds_record_close(&desc);
• err_code = fds_record_delete(&desc);
• err_code = fds_gc();
• err_code = fds_record_update(&desc, &gtPara_record);
  
  Thanks！
  Best Regard!
  Kenyon

[Hung Bui]

Hi Zicongkuang, 
I don't think there is a workaround to avoid assert&restart except for that you make sure there shouldn't be a problem. 

One thing you can do though, is that if the err_code returns FDS_ERR_BUSY then you can wait and try again after a while instead of reset. But make sure you don't end up in a dead loop. You should trigger a reset if the same error returned for a while. This make sure the product can recover with a reset.