I am trying to find information on this and it is not clear.
Is it safe to perform FDS read operations (i.e. fds_record_open(), fds_record_find() and fds_record_close()) while FDS is in the middle of a write, delete or garbage collection operation? It seems like these things would interfere with each other.
Hi,
You are correct that those operations that you mention are areas where thought and caution is required. The FDS library is designed with such scenarios in mind, e.g. through the way entries are read by first opening them (preventing garbage collection on the flash page on which they are stored) before accessing the contents.
If you experience - or have reason to believe that there are - any issues with the FDS library in this area, please let us know as that would definitely be a bug.
While there has previously been some issues (that now are patched) related to resets at the wrong time in the middle of e.g. garbage collection, I am not aware of any issues running garbage collection with open records, opening records after initializing a garbage collection, etc. (although some such combinations may be quite unusual.)
So yes, performing those operations should be safe.
Regards,
Terje
Ok, thank you for the answer. I figured that the open and close operations were acting as a lock, but in a scenario where only three flash pages are in use, this seems like it'd be a major obstacle to completing garbage collection effectively. I think I will go ahead and add code to prevent these operations from occurring simultaneously.
Hi,
You are correct that if multiple records are open then operations (including garbage collection, updating and deleting records) may suffer. The best mitigation would be to make sure not to keep records open for longer than needed, and if there is need to keep information from records in memory for a long time, to copy it to temporary variables so that you can close the record as quickly as possible.
Out of curiosity, what did you try for preventing the operations of occurring simultaneously, and how was the results?
Regards,
Terje
I am only getting to that part of the flash implementation now, so I can't tell you how it's going to work. My plan is a proactive one.
I'll register an observer of FDS events using fds_register(). It will listen for the completion of write and update operations. After either of these completes, it will perform a dummy fds_reserve() operation for twice the maximum amount needed for any record. If it fails, it immediately initiates garbage collection. If it succeeds, it immediately cancels the reservation with fds_reserve_cancel().
The hope is that no module encounters an out of space error when attempting a write or update. If it does, that record is lost. Also, as far as reading, this is done briefly at startup and copied into RAM. Hopefully write and update operations should enter the FDS operations queue behind the garbage collection operation, if one has been started.
I am only getting to that part of the flash implementation now, so I can't tell you how it's going to work. My plan is a proactive one.
I'll register an observer of FDS events using fds_register(). It will listen for the completion of write and update operations. After either of these completes, it will perform a dummy fds_reserve() operation for twice the maximum amount needed for any record. If it fails, it immediately initiates garbage collection. If it succeeds, it immediately cancels the reservation with fds_reserve_cancel().
The hope is that no module encounters an out of space error when attempting a write or update. If it does, that record is lost. Also, as far as reading, this is done briefly at startup and copied into RAM. Hopefully write and update operations should enter the FDS operations queue behind the garbage collection operation, if one has been started.
Hi,
Sounds OK. Please be aware of the potential pitfall of doing GC shortly after boot. In certain situations (such as running out of battery power) you may end up with several resets in a short amount of time, which could add to flash wear if GC is run on boot.
Another option for monitoring flash usage is to use fds_stat(), which will provide information not only on remaining free flash, but also on the potential for freeing flash (freeable_words in fds_stat_t.)
Regards,
Terje
Thank you for the suggestion around fds_stat() and boot timing. Those will help make for a reliable design.
