Basic FDS Question

[Birt]

System Details:

Chipset: nRF52840
SDK: nRF52 SDK v17.0.2
Using Zephyr: No

Overview:

The documentation for FDS has been very helpful, and with that, I was able to get something running pretty quickly, but I would like to ask a couple of questions regarding FDS files and records.

As part of a logger design, I want to track the number of resets. Nordic actually has an example of precisely that type of functionality. Basically, they open the record, update the boot count, close the record, and perform the update. This works as expected, but my question is mainly about organizing FDS files and FDS records to get the most out of the design.

Questions:

As part of the logger I mentioned above, I want to write logs to Flash and track the resets. However, I don’t want to marry the two entities. Instead, I would like them to be separate files. As such, I was thinking of having one file to track all my logs, and one file to track the reset counts. That way I can query just the boot count, without having to search through all the logs, as they live in different files.

With that said, my questions are pretty basic:

1. Is there a way to specify how many pages per file? For example, say I wanted to reserve 4K for logs, and 1K for the reset count, each of which would be separate files, with multiple records.
   
   
2. Does my logic above make sense, where you would use one file for (n) number of records, or in my case, logs? If so, is there a maximum number of records per file? Or any sort of rollover that is required?
   
   
3. This could certainly be a misunderstanding on my end, but I noticed that the examples hardcoded the record key. For example:
   

   static void ndef_file_prepare_record(uint8_t const * p_buff, uint32_t size)
   {
       // Set up record.
       m_record.file_id           = FILE_ID;
       m_record.key               = REC_KEY;
       m_record.data.p_data       = p_buff;
       m_record.data.length_words = BYTES_TO_WORDS(size); // Align data length to 4 bytes.
   }

   
   
   In my case, I thought each log would have its own record key to prevent overwriting the last entry, but maybe that's not how it works. I'm just thinking of the case where you have a single file, which contains 100 logs. I thought each log would be considered a record, so I hoped not to specify the key.

[tesc]

Hi,

Is there a way to specify how many pages per file? For example, say I wanted to reserve 4K for logs, and 1K for the reset count, each of which would be separate files, with multiple records.

This is not quite how FDS works. All data in FDS is stored in records, where each record is tagged with a "file ID" and a "record key". The name "file ID" indicates that all records marked with the same file ID is part of the same collection of data, but physically the records can be spread between all FDS data pages. There is no limit to the total flash space used by records of a specific file ID, no. If you want to keep the total amount of records (or space used by records) below a certain threshold, then I am afraid you must implement that yourself.

Does my logic above make sense, where you would use one file for (n) number of records, or in my case, logs? If so, is there a maximum number of records per file? Or any sort of rollover that is required?

Yes, using file ID for discerning between different collections of data stored by FDS makes sense. This is for instance how the peer manager module stores information about BLE peers, bonding data, etc. Each file ID is used for one type of data, and record keys are used for labelling different pieces of data of that same type.

There is no maximum number of records (other than the need for enough flash space for storing the records.) Typically one would want to use the "record key" field for discerning between records belonging to the same file, but there is nothing preventing you from storing multiple records with the same file ID and record key combination (in which case you may want to encode some information in the record data itself, for keeping the data organized.)

If you use the record key for ordering log messages, each message with its own unique record key, then the maximum number of log entries will be 2^16 - 1 = 65535. If you use the record key for marking different levels, sources, etc. of logs, and rather timestamp or number your log entries in the messages themselves, then you will only be limited by the amount of flash pages configured for use by FDS.

This could certainly be a misunderstanding on my end, but I noticed that the examples hardcoded the record key. For example:

(...)

In my case, I thought each log would have its own record key to prevent overwriting the last entry, but maybe that's not how it works. I'm just thinking of the case where you have a single file, which contains 100 logs. I thought each log would be considered a record, so I hoped not to specify the key.

You can choose yourself, whether to use an incrementing record key (for making each log message individually unique) or if you want to use a scheme where multiple (or all) log messages share the same record key (and then fetch out all entries and sort them by the message contents or ortherwise based on the record data.) Both are possible.

If you search for a file ID / record key combination, you will get the first record found in flash, that matches that combination. Then, for consecutive searches, you can start from the previous match and search for the next entry. This way you can fetch all records of a given file ID / record key combination. Please note however that there is no guarantee to the order in which you get the matches, only that if you follow this procedure you will fetch out all matching entries (in some order.) Therefore, if there is an order to the records, then you will either have to use record key for marking this order, or have some information in the record data itself describing the order.

Regards,
Terje

[Birt]

I really appreciate the response!

I see what you mean now with regard to records. Based on the extra info you provided, I think it's safe to say that FDS is really a collection of records indexed by the key/file ID combination, as seen in the Nordic docs below


Since I can't reserve a specific size, I'm wondering if maybe I can reserve a certain number of records instead. 

For example, for wear leveling, let's say I reserve ten records for logging; each record will have a unique record key, 1-10, and share the same file ID.

Kind of something like this:




The logic for those ten slots would be the following:

• If a log already exists in one of the ten records, then the "occurrence count" for that specific log is incremented and the existing log will be updated.
  
  
• If the log doesn't exist, and one of the ten records is available, then the new log will be written. This process continues until all ten records are filled.
  
  
• If all ten slots are filled with unique logs, and a new log that does not exist comes in, then it will be dropped.

I know this is purely an implementation question, but I'm curious if FDS would allow for a paradigm like this.

Also, I say "wear leveling," but I haven't convinced myself that I am using that term correctly. Mainly because an FDS update is really still a write, and I think it invalidates the previous record. So, the FDS design I mentioned about reserving ten records might not actually save me anything.

Again, I know this is more of an implementation question, but can you confirm if I am on the right track? 

[Birt]

I will create a separate ticket for this and close the issue with your verified answer. Thanks!