Maximum number of records per file ?

[QuentinFarizon]

I am writing 3305 records into a single FDS "file" (0x1000)

But when looping through them with `while ((err_code = fds_record_find_in_file(data_type_to_file_id(data_type), &record_desc, &find_token)) == NRF_SUCCESS) {` it loops through only 2304 records.
For the last one, err_code is FDS_ERR_NOT_FOUND, exactly like if there was no more records in this file.

For the first 2304, I have no issue retrieving, writing, updating them, ...

I have launched my program several times, it always loops through 2304 records. Is this a known limit of maximum number of records for one file ?

Code :

    ret_code_t err_code;
    fds_flash_record_t flash_record;
    fds_record_desc_t record_desc;

    fds_find_token_t find_token;
    memset(&find_token, 0x00, sizeof(fds_find_token_t));


    while ((err_code = fds_record_find_in_file(data_type_to_file_id(data_type), &record_desc, &find_token)) == NRF_SUCCESS) {
        < check record >
    }

[Einar Thorsrud]

Hi,

There is no defined maximum number. You can even have multiple records with the same ID within the same file.

Are you sure the additional records are there? Could it be that records after number 2304 are not written for some reason? Perhaps because of no more room in FDS, or some other reason, but you fail to handle an error that occurred before so that it failed silently so that it is not notices before attempting to read?

[QuentinFarizon]

Ok this was tricky to debug. To see correctly the writing process I added a 1 ms delay (nrf_delay_ms(1)) between each write (if I don't do that I lose RTT output)

But in fact this 1 ms delay "fixed" the issue, so I guess there is something wrong when making several FDS write too quickly

Note that for each write I wait for the FDS_EVT_WRITE before proceeding to the next write.

Any idea why this 1ms delay is necessary ?

[QuentinFarizon]

We definitely reproduce 100%. Adding a 1ms delay between each writes completely solved the issue. 

[Einar Thorsrud]

I see. I am not able to explain it, though. Can you upload a project that reproduce this on a DK?

[QuentinFarizon]

In fact, after more tests, adding a 1ms doesn't always solve the issue
We tried with a longer delay, it helps, but randomly it doesn't write everything

[QuentinFarizon]

It's not easy to reproduce, as it always succeeds when having RTT logs plugged (because it slow it down or something ?).

And it doesn't crash/watchdog or anything, it just stops "really writing" records at some point. 

[Einar Thorsrud]

Hi,

Do you by any chance have the Power-fail comparator enabled? I ask because if there if enabled and VDD is lower than the configured threshold, that will cause a POFWARN event, and while active, the NVMC will not write to flash. However, there is no feedback about this, so the flash operation will seemingly have been successful. See Power-fail comparator for details. If this is the case, you should not see the failure if you disable the power-fail comparator (don't enable it), or if the voltage never drops the threshold (I don't know what else happens when you use RTT, but it could be that in that case the voltage never drops below the threshold).

I don't have any particular reason for beliving that this is the issue you are seeing, but it is the only condition I can think of that would make a flash write operation seemingly successful without actually writing data to flash.

[QuentinFarizon]

I don't call sd_power_pof_enable, so I don't think it's activated ? 

[Einar Thorsrud]

That is right, the POF comparator is disabled by default, so if you don't enable it that can not be the issue here.

[QuentinFarizon]

We encounter this problem on ~10% of our customers.

Instead of 3000 writes, they got only a few hundreds (never the same amount)

What would you recommend ?

[Einar Thorsrud]

Hi,

This is not a generic issue or an issue I have seen earlier reports of (assuming it is correct that POF warning is not enabled). We need to understand what the cause of this issue is first in order to be able to fix it. Have you done more testing and debugging to learn more about the issue and in which conditions it is can occur and in which conditions it never occurs (if any)? If that gives some indications, perhaps it is possible to instrument the code with some additional debug printouts or similar, or to inspect the code with that in mind.

[QuentinFarizon]

I have done lots of testing and debugging (not easy since we do not reproduce in Debug mode or with logs enabled).
We pushed in production as-is, and measured which customers had the issue by counting the written records in a separate service. About 10% of them had the issue.

At office, we have one board that reproduce the issue randomly, at a rate of about 15%, and one board where we do not reproduce at all.

I'm out of ideas on what happens, how to reproduce it more often, and on when conditions it occurs (hardware or software).