Use of FDS for larger data block

[Peter.J]

Hello,

I am working on an update of older FW implementation: nRF52832, SDK 14.1.0, with SoftDevice S132 and using BLE connection.

The change I need to implement is a functionality for writing a bigger block of data, around 20kB to flash. These data will be written after every use of the device. The flash space available I have is about 160kB - above application and below bootloader (0x4A000 - 0x72000).

Before starting implementation itself I need to clarify several questions.

As I'd like to reflect flash endurance my first choice is using FDS driver utilizing SD. Having the flash space available it could be 8 data blocks written then the driver should start form beginning again.

Questions:

• How the FDS driver knows / checks the flash limits? Does the SD provide this information?
• From some posts I understood that the FDS starts below bootloader area towards lower addresses. Correct?
• How is the unused flash used during DFU over BLE (temporary storing of new image)? I guess that it will conflict with my data. It might not be a problem the data will be erased during DFU.
• Does the DFU ignore that there are some data in the flash or it will not allow running DFU if not enough space?
• Where are BLE data (bonding) stored? Is it somewhere inside SD area or in the free flash space (a page below bootloader)?

Another approach is to use fstorage driver and implement all the flash buffer handling myself. Advantage is that I can handle myself the flash limits and I have better overview on space available and data location.

I am using WDT and the WDT interrupt driver handles writing some important data (48 B) into flash. As there is a short time window (like 36 ms?) while the device restarts, I use fstorage driver for writing the data as I find it a faster driver. 

A question: is it possible to have both flash drivers initialized at the same time and to use the one that is appropriate? 

If there is any documentation on the above questions it would help me.

Thank you for any advices.

Best regards,

peter

[tesc]

Hi,

I see why you think of FDS for this, with regards to flash wear leveling. An alternative could be to do it in a ring buffer fashion.

Regarding your questions:

How the FDS driver knows / checks the flash limits? Does the SD provide this information?

The FDS module reads the address for the bootloader from UICR, to see where the Flash area for FDS ends. For the size (and thus where the area begins) it uses the configuration, in particular FDS_VIRTUAL_PAGE_SIZE and FDS_VIRTUAL_PAGES. (For other configurations related to FDS, please see the "Configuration" section of the FDS Functionality documentation page.)

From some posts I understood that the FDS starts below bootloader area towards lower addresses. Correct?

Correct. Please see the "Memory layout" section of the Bootloader documentation, for the default Flash memory layout in nRF5 SDK. There, FDS data is put in the "Application data" area. FDS uses the area next to the Bootloader, so any other application data must be below the region used by FDS.

How is the unused flash used during DFU over BLE (temporary storing of new image)? I guess that it will conflict with my data. It might not be a problem the data will be erased during DFU.

The DFU Bootloader will keep application data untouched, according to what is defined with APP_DATA_RESERVED. This should be in accordance with the FDS settings (virtual page size times number of virtual pages), plus any additional application (if you have any.) Please note that since the DFU bootloader and the application are two separate builds, you need to set the proper configurations in each of them and make sure that the (size of the) space used by the application correspond to the (size of the) space not touched by the bootloader.

Does the DFU ignore that there are some data in the flash or it will not allow running DFU if not enough space?

There has to be enough space on the device, for performing the given DFU update, in addition to the space used by the bootloader itself, and the space reserved for data storage for the application through APP_DATA_RESERVED. For the technicalities related to DFU in nRF5 SDK, I recommend consulting the Dual-bank and single-bank updates doccumentation page, which explains this thoroughly.

Where are BLE data (bonding) stored? Is it somewhere inside SD area or in the free flash space (a page below bootloader)?

This is typically also stored using FDS, through use of the Peer Manager module (which uses FDS as its backend.) See Peer Manager for details. Your existing project is likely to already use Peer Manager for BLE bonding data.

Another approach is to use fstorage driver and implement all the flash buffer handling myself. Advantage is that I can handle myself the flash limits and I have better overview on space available and data location.

This is of course also an option. It would allow you to handle things yourself, while it also means you must handle potential pitfalls and do a good job at designing and implementing it.

Please note also that there has been some patches to FDS between nRF5 SDK 14.1 (which you use) and nRF5 SDK 17.1.0 (the last nRF5 SDK release). I highly recommend, if you continue development on an nRF5 SDK project, to upgrade to a newer vesion if you can (at least for FDS.)

A question: is it possible to have both flash drivers initialized at the same time and to use the one that is appropriate? 

FDS builds on top of fstorage, and there can be multiple concurrent users of fstorage (using separate areas of Flash.) So yes, you can to a certain degree mix and match. For instance you can have both FDS and one or more uses of fstorage in addition. Just make sure to not have multiple modules try to use the same area of Flash.

Regards,
Terje

[Peter.J]

Hi Terje,

thank you for explaining the details of configuring and using FDS. I have some additional questions.

My understanding of how FDS works is a ring buffer, within its borders. I am now confused when you say “An alternative could be to do it in a ring buffer fashion”. What way the FDS works? Is it a list?

Ad using FDS in normal operation and fstorage in WDT driver – I see a limitation for my use. Those 48 bytes that I want to write in WDT interrupt driver, will normally be written by FDS driver. Thus I would need to know flash address (for fstorage) where FDS would write them. I guess this I cant get via some function?

After thinking over advantages and disadvantages (patch porting, still need to use fstorage) of FDS vs fstorage I am now more in favor of implementing a ring buffer using fstorage.

Best regards,

peter

[tesc]

Hi,

FDS provides an abstraction over fstorage. FDS stores "records", which is a short header followed by data. You are correct those records are written to Flash in a somewhat "ring buffer" fashion, but one where parts of that buffer are compacted (through copying non-deleted records from one virtual flash page, to a free virtual page, records being written back on back, leading to contiguous free space at the end of the virutal flash page.)

With FDS, if you have many small records, you do run the risk of not finding space for a large record, because the space left in each virtual flash page is not enough to hold the record (even though total free space is much larger.)

There can be multiple concurrent FDS users, but you cannot simply write to the FDS area using fstorage and expect things to work. Everything within the FDS area must be of the FDS format, which means a (virutal) flash page header followed by records (with each record consisting of header + data). If you are to use fstorage directly, you would still have to conform to the rules of FDS, if writing within the FDS area, which means it would be just as effective (or time consuming) to use FDS.

With an fstorage ring buffer style of doing it, I expect you still would need some sort of header data (to indicate which data is the newest), but I guess you could spend the time on startup for locating the next area to write to (including erasing flash pages if needed), so that when the data is to be written you can write it right away instead of the FDS library first searching for somewhere to put the record. There is also functionality in FDS to reserve space for a record of a given size, so that part can be done in advance also with FDS.

Regards,
Terje