FDS - data corruption

[Wojciech Jasko]

Hi,

I found a bug in fds module (Flash Data Storage) which lead to data system corruption.

Scenario:

• Given the device started GC(Garbage Collection) procedure

• When the device will reset during the in the certain moment of page swap procedure.

• Then assigning new record id will overlap existing ones.

When just first two words of a record are copied to the swap page, the header of this record will pass header_check function.

During the page_scan procedure, this corrupted header will be used to update m_latest_rec_id. It will result in assigning new record ids from 0.

Solution:

I have updated header_check function to check record_id against 0xFFFFFFFF value, and now I am testing this workaround.

[wael]

Hey Wojciech,

Were you able to fix this issue? We are seeing something that sounds very similar when doing a lot of FDS writes.

Thanks,
Wael

[Wojciech Jasko]

Yes, I have fixed 2 issues.

• devzone.nordicsemi.com/.../fds---data-corruption-during-an-interrupted-swap
• This one by updating function from fds.c

/**@brief   Not initialized record id.
 */
#define FDS_RECORD_ID_NOT_INIT     (0xFFFFFFFF)

fds_header_status_t fds_header_check(fds_header_t const * p_hdr, uint32_t const * p_page_end)
{
    if (((uint32_t*)header_jump(p_hdr) > p_page_end))
    {
        // The length field would jump across the page boundary.
        // FDS won't allow writing such a header, therefore it has been corrupted.
        return FDS_HEADER_CORRUPT;
    }

    if (   (p_hdr->file_id    == FDS_FILE_ID_INVALID)
        || (p_hdr->record_key == FDS_RECORD_KEY_DIRTY)
        || (p_hdr->record_id  == FDS_RECORD_ID_NOT_INIT))
    {
        return FDS_HEADER_DIRTY;
    }

    return FDS_HEADER_VALID;
}

I am still struggling with one more. The one with losing swap page after power failure. 

[LukaszW4]

Have you already solved all the problems connected with FDS? I have similar problem with losing swap page.

[Wojciech Jasko]

Unfortunately no. We decided to drop fds and use something more stable.

[LukaszW4]

Could you tell me solution you used instead of fighting with FDS?

[Wojciech Jasko]

Hi Lukasz!

We are creating our own solution (to be compatible with fds records layout). Anyway, I believe that I found a solution for this swap page. Please see https://devzone.nordicsemi.com/f/nordic-q-a/53687/fds---missing-swap-page

[Jørgen Holmefjord]

There are multiple fixes added to FDS in SDK v16.0.0.

[Wojciech Jasko]

 In version 14.2 I found 3 issues:

• corrupted record_id (this case)
• losing installed pages (https://devzone.nordicsemi.com/f/nordic-q-a/44723/fds---data-corruption-during-an-interrupted-swap)
• losing swap page (https://devzone.nordicsemi.com/f/nordic-q-a/53687/fds---missing-swap-page)

I have checked v16 and it looks like first 2 issues are addressed.

I believe this issue with losing swap page is still present. Please consider reviewing my fix  https://devzone.nordicsemi.com/f/nordic-q-a/53687/fds---missing-swap-page