Default storage of persistent info for OpenThread

RE: Default storage of persistent info for OpenThread

tush3094 — Mon, 01 Jul 2024 07:37:51 GMT

Yes, this clarifies my doubt. Thank you so much for a nice explanation.

RE: Default storage of persistent info for OpenThread

AHaug — Mon, 01 Jul 2024 06:50:42 GMT

Aha,

[quote user="tush3094"]Because settings_storage address is not something which I declare in the partition manager .yml file. This address 0XF7000 is randomly chosen.[/quote]

Given that you define that you have a storage partition or that you use a protocol that requires a storage partition and automatically selects/includes this section from configurations, it will be included by the partition manager automatically. But it's placement will change if you only statically define your application slot partitions and not your settings partitions.

I would recommend that you define all the partitions you need, including settings_storage statically so that you ensure robustness when performing DFU at a later point in time.

[quote user="tush3094"]My only concern is if in future the firmware size increases and reaches this point will these stored settings be deleted or automatically handled by the partition manager?[/quote]

I understand now what you meant by this question. The short answer is that you should always statically declare all of your partitions with enough room to fit any future firmware updates that you have defined in your products scope/life time

The slightly longer explanation is:

It is also better to allocate a larger mcuboot_primary_application and mcuboot_secondary_application partitions than your applications currently are to ensure robustness w.r.t increasing the application size in the future than it is to rescope your partitions after you've entered production. Of course there are situations where you must update a device since mandatory updates must be done that does not fit your scope, but you should always try to plan ahead of those things. One of the things you can do to plan for the unexpected is to have larger application sizes than your applications actual size.

If you ever intend to update your application size by updating the static partitioning map and only changing the application slot sizes (mcuboot_primary_application and mcuboot_secondary_application) you will have issues with finding and reading the other partitions in your device. This is also the reason for why we recommend that you always define everything statically before going into production.

Firmware updates are typically meant to patch, fix, and add things that are already predefined to a products scope, and it is not really recommended to rescope the product and update it through DFU unless you've already planned ahead for this, i.e preallocated sufficient enough room for said updates.

Let's say for instance if you wish to add a new feature which will translate to an increase of firmware by 30 kB. This will require 60kB since you need to increase both of the application slot sizes with 30kB. To do this you will have to update your static partitioning, but if you don't also update your MCUboot second stage bootloader to be aware of the new partitioning scheme, then the update won't go through since MCUboot still believes that the primary and secondary app starts and ends at the same addresses.

Here's a quick sketch I made to somewhat illustrate this specific point

I.e to summarize: Scope your partition sizes to account for future firmware updates if you believe you will add new features in the future. If you don't have enough flash on your internal flash to do this, then you have two options: Limit your scope/reduce the features or add an external flash and place the mcuboot secondary storage + other storage that does not require any special security on the external flash.

Let me know if this answers your question

Kind regards,
Andreas

RE: Default storage of persistent info for OpenThread

tush3094 — Fri, 28 Jun 2024 13:49:14 GMT

Sorry, I was not clear. I meant if the firmware application size increases and reaches this point. After a firmware update would it be restored again? Because settings_storage address is not something which I declare in the partition manager .yml file. This address 0XF7000 is randomly chosen.

RE: Default storage of persistent info for OpenThread

AHaug — Fri, 28 Jun 2024 13:44:03 GMT

I assume by this that you mean in case you have to store more data in your settings_storage partition than the declared size, and not if the application firmware size itself increases in the future (correct me if I'm wrong):

If this is the case then he partition manager uses the settings_storage subsystem which in turn should have its own garbage-collection procedure, mentioned https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/zephyr/services/settings/index.html, as described here: "When storage becomes full (FCB) or consumes too much space (file), the backend removes non-recent key-value pairs records and unnecessary key-delete records."

Let me know if this answers your question

Kind regards,
Andreas

RE: Default storage of persistent info for OpenThread

tush3094 — Fri, 28 Jun 2024 13:25:50 GMT

Hello Andreas,

Thank you for looking into it. In the last 2 days I tried some more stuff and made some changes into the partition manager .yml file and already reached this point.

As per the info OpenThread data is stored in settings_storage partition which is at 0xF7000 after my changes. This is all good.

My only concern is if in future the firmware size increases and reaches this point will these stored settings be deleted or automatically handled by the partition manager?

Kind regards,

Tushar

RE: Default storage of persistent info for OpenThread

AHaug — Fri, 28 Jun 2024 12:51:24 GMT

Hi,

From what we can see it looks like you're the Partition Manager, and when you are using this the DTS partitioning is no longer used.

Instead you should use "west build -t partition_manager_report" to see your new partitioning, and try to read from that area instead.

See https://academy.nordicsemi.com/courses/nrf-connect-sdk-intermediate/lessons/lesson-8-bootloaders-and-dfu-fota/ for some background on this

Kind regards,
Andreas

RE: Default storage of persistent info for OpenThread

AHaug — Tue, 25 Jun 2024 13:00:01 GMT

Hi,

Maria is currently on vacation so I will be monitoring this case meanwhile. I will need some time to sync up with the information in this case, but I will get back to you later this week.

Kind regards,
Andreas

RE: Default storage of persistent info for OpenThread

tush3094 — Mon, 24 Jun 2024 14:20:47 GMT

Some more information regarding the above issue. OpenThread data is stored correctly on our firmware when partition manager and bootloader are not used. Just the firmware is built and flashed.

So basically this issue comes up when we built the firmware as a whole with application and MCU boot.

Is there any specific change required here that needs to be done with partition manager enabled?

RE: Default storage of persistent info for OpenThread

tush3094 — Fri, 21 Jun 2024 15:11:22 GMT

So I tested with the thread CLI example for nrf52840,

Here the openthread data is stored correctly

I am not sure what I am doing wrong in my build

RE: Default storage of persistent info for OpenThread

tush3094 — Fri, 21 Jun 2024 13:26:17 GMT

I read it out via the debugger (entire flash)

I consider this as the openthread data as I can see the panid (Openthread-e193) which is same as my dataset of the mesh.

I am using my own code. How can I include this settings service in my application?

I have not tried to read out the OpenThread data from NCS sample. I can try it and get back to you.

RE: Default storage of persistent info for OpenThread

Maria Gilje — Fri, 21 Jun 2024 13:16:54 GMT

How and where do you read out the 0xF3000 address for the OpenThread data?

Are you using an NCS sample or your own code? For the Thread samples in NCS, the settings service is included by default. Do you see the same when reading out the address for the OpenThread data for an NCS sample?

RE: Default storage of persistent info for OpenThread

tush3094 — Fri, 21 Jun 2024 12:36:05 GMT

Hello Maria,

Thank you for your quick reply. I have add these extra configs in my proj.conf

CONFIG_NVS=y

CONFIG_SETTINGS=y

CONFIG_SETTINGS_NVS=y

Also I have edited my .dts file to have the label "storage" for the storage partition with start address @FA000.

Even though this is set the openthread data is still stored at 0xF3000.

Is the openthread handling the storage by itself or we need to over ride some functions of openthread?
Currently, I am just not able to store the openthread data at this address.

RE: Default storage of persistent info for OpenThread

Maria Gilje — Fri, 21 Jun 2024 10:22:25 GMT

Hello,

The defaultt storage location for the OpenThread network data in NCS is in settings storage which again is in NVS. It is the settings subsystem which handles storing in NVS when using Thread in NCS. See the documentation for Configuring Thread in NCS for general information on Thread in NCS, the Settings service documentation, and NVS.

Best regards,

Maria