Storing sensor data as circular buffer in non volatile memory

[Artnia]

Hey guys,

I want to store sensor data as circular buffer in non volatile memory. I saw
https://docs.nordicsemi.com/bundle/zephyr-apis-latest/page/group_file_system_storage.html and https://docs.nordicsemi.com/bundle/ncs-latest/page/zephyr/services/storage/index.html. But researching here in the devzone I can see that users have issues like here https://devzone.nordicsemi.com/f/nordic-q-a/110506/nrf5340-fcb-flash-circular-buffer-on-external-flash/530085. 

I was trying to use NVS but it seems that it overwrites data as stated here https://github.com/nrfconnect/sdk-zephyr/blob/a1d9a8db0ea4bbee2e69feec1140e6ce20e62f5e/samples/subsys/nvs/src/main.c#L30
But of course I need the complete history of my data.

So my question is what is currently the best approach for this?

Best

[Hieu]

Hi Artnia,

Michal is out of office so I will take over this case again. Let me sync up and come back to you this week.

Hieu

[Hieu]

Hi Artnia,

I reproduced the "issue" on a nRF52840 DK. I observe that:

• After the sector change to sector 1, each write to the new sector causes the oldest line from sector 0 to disappear.
• sensor_ts and message counters should continue to increase. However, they both restarted at 0 after the sector change for some reason

I first suspect that the timestamp restarting to 0 might be indicative that the logic is reading from the wrong place. However, doubling the sleep duration to 200ms also causes the timestamp to restart to ~1100 instead, so that isn't it.

Without knowing the intended design of the FCB, I can't conclude anything yet. But I can say now that getting to the bottom of this is likely time consuming... I will keep working at it though.

Out of curiosity, was there any issue that motivated you to add this line?
ncs_add_partition_manager_config(pm_static.yml)

pm_static.yml should be picked up by default, without this line. Not that it is any problem.

Hieu

[Artnia]

Hi Hieu,

thanks for the reply.
I cant remember 100% why I put it there. If this is very unusual, it could be worth trying without.

[Hieu]

Hi Artnia,

That line isn't it. I reproduced the issue without it and with a much simpler pm_static.yml, since I am doing it on the nRF52840 DK.

I will keep looking into this.

app:
  address: 0x0
  end_address: 0x0f0000
  region: flash_primary
  size: 0x0f0000
test_partition:
  address: 0xf0000
  end_address: 0xfa000
  region: flash_primary
  size: 0xa000
dummy_partition:
  address: 0xfa000
  end_address: 0x100000
  region: flash_primary
  size: 0x6000
sram_primary:
  address: 0x20000000
  end_address: 0x20040000
  region: sram_primary
  size: 0x40000

[Hieu]

Hi Artnia,

So the issue is that you are using fe_data_off. This offset is relative to the sector start address. 
Therefore, to get the correct flash area offset, you need to add fe_sector->fs_off.

This incorrect usage is why when you attempt to write entries in sector 1, you are actually writing to sector 0 and corrupting the records.

The FCB public header file also provide a convenient macro for this, FCB_ENTRY_FA_DATA_OFF(entry).

I still have some doubts on why the timestamp could be overwritten, since flash memory doesn't really allow overwriting... But right now, the correct fix is identified, so I won't look into that yet.

Hieu