FDS - data corruption

RE: FDS - data corruption

Wojciech Jasko — Fri, 25 Oct 2019 18:59:42 GMT

In version 14.2 I found 3 issues:

corrupted record_id (this case)
losing installed pages (https://devzone.nordicsemi.com/f/nordic-q-a/44723/fds---data-corruption-during-an-interrupted-swap)
losing swap page (https://devzone.nordicsemi.com/f/nordic-q-a/53687/fds---missing-swap-page)

I have checked v16 and it looks like first 2 issues are addressed.

I believe this issue with losing swap page is still present. Please consider reviewing my fix https://devzone.nordicsemi.com/f/nordic-q-a/53687/fds---missing-swap-page

RE: FDS - data corruption

Jørgen Holmefjord — Fri, 25 Oct 2019 12:50:18 GMT

There are multiple fixes added to FDS in SDK v16.0.0.

RE: FDS - data corruption

Wojciech Jasko — Fri, 25 Oct 2019 11:50:10 GMT

Hi Lukasz!

We are creating our own solution (to be compatible with fds records layout). Anyway, I believe that I found a solution for this swap page. Please see https://devzone.nordicsemi.com/f/nordic-q-a/53687/fds---missing-swap-page

RE: FDS - data corruption

LukaszW4 — Thu, 12 Sep 2019 12:21:04 GMT

Could you tell me solution you used instead of fighting with FDS?

RE: FDS - data corruption

Wojciech Jasko — Tue, 10 Sep 2019 11:55:36 GMT

Unfortunately no. We decided to drop fds and use something more stable.

RE: FDS - data corruption

LukaszW4 — Tue, 10 Sep 2019 11:12:08 GMT

Have you already solved all the problems connected with FDS? I have similar problem with losing swap page.

RE: FDS - data corruption

Wojciech Jasko — Mon, 06 May 2019 05:03:53 GMT

Yes, I have fixed 2 issues.

devzone.nordicsemi.com/.../fds---data-corruption-during-an-interrupted-swap
This one by updating function from fds.c

/**@brief   Not initialized record id.
 */
#define FDS_RECORD_ID_NOT_INIT     (0xFFFFFFFF)

fds_header_status_t fds_header_check(fds_header_t const * p_hdr, uint32_t const * p_page_end)
{
    if (((uint32_t*)header_jump(p_hdr) > p_page_end))
    {
        // The length field would jump across the page boundary.
        // FDS won't allow writing such a header, therefore it has been corrupted.
        return FDS_HEADER_CORRUPT;
    }

    if (   (p_hdr->file_id    == FDS_FILE_ID_INVALID)
        || (p_hdr->record_key == FDS_RECORD_KEY_DIRTY)
        || (p_hdr->record_id  == FDS_RECORD_ID_NOT_INIT))
    {
        return FDS_HEADER_DIRTY;
    }

    return FDS_HEADER_VALID;
}

I am still struggling with one more. The one with losing swap page after power failure.

RE: FDS - data corruption

wael — Thu, 02 May 2019 21:10:11 GMT

Hey Wojciech,

Were you able to fix this issue? We are seeing something that sounds very similar when doing a lot of FDS writes.

Thanks,
Wael

RE: FDS - data corruption

Wojciech Jasko — Mon, 18 Feb 2019 06:36:27 GMT

I am uploading two FDS section dumps. They are binary files.

First one is swap_interrupted (https://drive.google.com/file/d/1YbhHWHZhjc4LcS1HGaIfJf2rJdcdAXHy/view?usp=sharing). It contains data in swap page, which reproduces this issue. If we initialize fds with this data m_latest_rec_id will be set to 0xFFFFFFFF and next created record will have id equal to 0.

The second one is corrupted_data (https://drive.google.com/file/d/1cMJeldRmiGFEwtxTkQgx_KWeFwsv1f-Q/view?usp=sharing). This is how fds looks like, after some operation with m_latest_rec_id initialized to 0xFFFFFFFF.

RE: FDS - data corruption

Wojciech Jasko — Fri, 15 Feb 2019 15:30:25 GMT

We are using:

SDK 14.2, but it should be reproducible with SDK 15.2 (code analysis).
NRF52832 and PCA10040 - but this is purely software bug.

As I described above:

GC procedure must be triggered (for example by a lot of write requests to fds)
This procedure must be interrupted with reset in a certain moment (power off/system reset)
After power on next write, requests will overwrite existing (first will have record id 0 so the second one will corrupt existing records).

It is hard to reproduce. In our setup, it takes about 12 h of certain tests. I believe that code analysis of page_scan and header_check function will help.

I can also share with you a flash dump from the corrupted fds storage.

RE: FDS - data corruption

Jørgen Holmefjord — Fri, 15 Feb 2019 14:34:14 GMT

Hi,

Which SDK version are you using, and what IC are you experiencing this bug on?

Have you reproduced this on any of oru DKs, and can you share code/procedure that can be used to reproduce this bug?

Best regards,
Jørgen