Default storage of persistent info for OpenThread

[tush3094]

Hello,

I wanted to understand what is the default storage place in the internal flash memory region of persistent info for OpenThread. In the previous SDK (Thread and Zigbee) it was clearly mentioned the default memory region as shown below, (reference: Persistent storage of the thread instance)

But with the nrf connect SDK I did not find any such document stating a clarity at this. Currently, I build openthread in my project with the configs shown below,

# Generic networking options

CONFIG_NETWORKING=y

# OpenThread enabling

CONFIG_NET_L2_OPENTHREAD=y

# Openthread build from source

CONFIG_OPENTHREAD_SOURCES=y

CONFIG_OPENTHREAD_NORDIC_LIBRARY_FTD=n

CONFIG_OPENTHREAD_NORDIC_LIBRARY_MTD=y

CONFIG_OPENTHREAD_MTD=y

CONFIG_OPENTHREAD_MTD_SED=y

CONFIG_OPENTHREAD_THREAD_VERSION_1_3=y

# OpenThread config

CONFIG_OPENTHREAD_MANUAL_START=y

CONFIG_OPENTHREAD_JOINER=y

CONFIG_OPENTHREAD_JOINER_AUTOSTART=n

 Partition manager static yml file the nvs storage location is provided as,

nvs_storage:

  address: 0xFA000

  end_address: 0x100000

  placement:

    before:

    - end

  region: flash_primary

  size: 0x6000

As per the NVS documentation OpenThread data should be stored in NVS data partition.The place at which the dataset is stored currently is in internal flash at address: 0xF3000. How is this address selected?

Kind regards,
Tushar D

[Maria Gilje]

Hello,

The defaultt storage location for the OpenThread network data in NCS is in settings storage which again is in NVS. It is the settings subsystem which handles storing in NVS when using Thread in NCS. See the documentation for Configuring Thread in NCS for general information on Thread in NCS, the Settings service documentation, and NVS.

Best regards,

Maria

[tush3094]

Hello Maria,

Thank you for your quick reply. I have add these extra configs in my proj.conf

CONFIG_NVS=y

CONFIG_SETTINGS=y

CONFIG_SETTINGS_NVS=y

Also I have edited my .dts file to have the label "storage" for the storage partition with start address @FA000.

Even though this is set the openthread data is still stored at 0xF3000.

Is the openthread handling the storage by itself or we need to over ride some functions of openthread? 
Currently, I am just not able to store the openthread data at this address.

[AHaug]

Hi,

From what we can see it looks like you're the Partition Manager, and when you are using this the DTS partitioning is no longer used.

Instead you should use "west build -t partition_manager_report" to see your new partitioning, and try to read from that area instead.

See https://academy.nordicsemi.com/courses/nrf-connect-sdk-intermediate/lessons/lesson-8-bootloaders-and-dfu-fota/ for some background on this

Kind regards,
Andreas

 

[tush3094]

Hello Andreas,

Thank you for looking into it. In the last 2 days I tried some more stuff and made some changes into the partition manager .yml file and already reached this point.

As per the info OpenThread data is stored in settings_storage partition which is at 0xF7000 after my changes. This is all good. 

My only concern is if in future the firmware size increases and reaches this point will these stored settings be deleted or automatically handled by the partition manager?

Kind regards,

Tushar 

[AHaug]

tush3094 said:

My only concern is if in future the firmware size increases and reaches this point will these stored settings be deleted or automatically handled by the partition manager?

I assume by this that you mean in case you have to store more data in your settings_storage partition than the declared size, and not if the application firmware size itself increases in the future (correct me if I'm wrong):

If this is the case then he partition manager uses the settings_storage subsystem which in turn should have its own garbage-collection procedure, mentioned https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/zephyr/services/settings/index.html, as described here: "When storage becomes full (FCB) or consumes too much space (file), the backend removes non-recent key-value pairs records and unnecessary key-delete records."

Let me know if this answers your question

Kind regards,
Andreas

[tush3094]

Sorry, I was not clear. I meant if the firmware application size increases and reaches this point. After a firmware update would it be restored again? Because settings_storage address is not something which I declare in the partition manager .yml file. This address 0XF7000 is randomly chosen.

[AHaug]

Aha,

tush3094 said:

Because settings_storage address is not something which I declare in the partition manager .yml file. This address 0XF7000 is randomly chosen.

Given that you define that you have a storage partition or that you use a protocol that requires a storage partition and automatically selects/includes this section from configurations, it will be included by the partition manager automatically. But it's placement will change if you only statically define your application slot partitions and not your settings partitions.

I would recommend that you define all the partitions you need, including settings_storage statically so that you ensure robustness when performing DFU at a later point in time.

tush3094 said:

My only concern is if in future the firmware size increases and reaches this point will these stored settings be deleted or automatically handled by the partition manager?

I understand now what you meant by this question. The short answer is that you should always statically declare all of your partitions with enough room to fit any future firmware updates that you have defined in your products scope/life time

The slightly longer explanation is:

It is also better to allocate a larger mcuboot_primary_application and mcuboot_secondary_application partitions than your applications currently are to ensure robustness w.r.t increasing the application size in the future than it is to rescope your partitions after you've entered production. Of course there are situations where you must update a device since mandatory updates must be done that does not fit your scope, but you should always try to plan ahead of those things. One of the things you can do to plan for the unexpected is to have larger application sizes than your applications actual size.

If you ever intend to update your application size by updating the static partitioning map and only changing the application slot sizes (mcuboot_primary_application and mcuboot_secondary_application) you will have issues with finding and reading the other partitions in your device. This is also the reason for why we recommend that you always define everything statically before going into production.

Firmware updates are typically meant to patch, fix, and add things that are already predefined to a products scope, and it is not really recommended to rescope the product and update it through DFU unless you've already planned ahead for this, i.e preallocated sufficient enough room for said updates.

Let's say for instance if you wish to add a new feature which will translate to an increase of firmware by 30 kB. This will require 60kB since you need to increase both of the application slot sizes with 30kB. To do this you will have to update your static partitioning, but if you don't also update your MCUboot second stage bootloader to be aware of the new partitioning scheme, then the update won't go through since MCUboot still believes that the primary and secondary app starts and ends at the same addresses.

Here's a quick sketch I made to somewhat illustrate this specific point

I.e to summarize: Scope your partition sizes to account for future firmware updates if you believe you will add new features in the future. If you don't have enough flash on your internal flash to do this, then you have two options: Limit your scope/reduce the features or add an external flash and place the mcuboot secondary storage + other storage that does not require any special security on the external flash.

Let me know if this answers your question

Kind regards,
Andreas

[tush3094]

Yes, this clarifies my doubt. Thank you so much for a nice explanation.