NVS data in internal flash occasionally erased after MCUboot OTA update (nRF5340 + nRF7003, NCS v2.6.2)

[Urvisha Andani]

Hello,

We are using a custom board based on the nRF5340 + nRF7003 Fanstel module with NCS SDK v2.6.2.

Our application uses MCUboot with dual-bank OTA updates. The secondary image slot is located in external flash, while application-specific data is stored permanently in internal flash using NVS.

During production, we write some configuration data to the internal flash only once. This data is then used by the application during normal operation and communication throughout the product lifetime. The data is not modified after production programming.

Issue:

We have observed that after a successful OTA update, the data stored in the NVS partition is occasionally erased.

The issue is intermittent and difficult to reproduce consistently. It occurs approximately once in every 10 OTA update attempts.

Expected behaviour:

The NVS data stored in internal flash should remain intact after OTA updates and device reboots.

Observed behaviour:

• OTA update completes successfully.
• MCUboot swaps the image and boots into the new application.
• The device operates normally after the update.
• However, in some cases, the NVS partition appears to be erased and all stored production data is lost.
• This issue occurs randomly and cannot be reproduced on every OTA attempt

Partition Manager configuration:

EMPTY_0:
  address: 0xfe000
  end_address: 0x100000
  placement:
    after:
    - settings_storage
  region: flash_primary
  size: 0x2000

app:
  end_address: 0xfc000
  region: flash_primary
  size: 0xebe00

external_flash:
  address: 0xec000
  end_address: 0x800000
  region: external_flash
  size: 0x714000

mcuboot:
  address: 0x0
  end_address: 0x10000
  placement:
    before:
    - mcuboot_primary
  region: flash_primary
  size: 0x10000

mcuboot_pad:
  address: 0x10000
  end_address: 0x10200
  placement:
    before:
    - mcuboot_primary_app
  region: flash_primary
  size: 0x200

mcuboot_primary:
  address: 0x10000
  end_address: 0xfc000
  region: flash_primary
  size: 0xec000

mcuboot_primary_app:
  address: 0x10200
  end_address: 0xfc000
  region: flash_primary
  size: 0xebe00

mcuboot_secondary:
  address: 0x0
  device: DT_CHOSEN(nordic_pm_ext_flash)
  end_address: 0xec000
  region: external_flash
  size: 0xec000

nvs_storage:
  address: 0xfa000
  end_address: 0xfc000
  region: flash_primary
  size: 0x2000

settings_storage:
  address: 0xfc000
  end_address: 0xfe000
  region: flash_primary
  size: 0x2000

Additional information:

• Data is stored in nvs_storage (0xFA000 - 0xFC000).
• OTA image is downloaded to external flash (mcuboot_secondary).
• MCUboot performs the image swap during reboot.
• We are not intentionally erasing the NVS partition anywhere in the application.

Request

Could you please help us identify the root cause of this behaviour and suggest a solution?

We already have approximately 1000 devices deployed in the field. Because of this, changing the flash layout or relocating the production data storage is not a practical option for the deployed products.

We are looking for a solution or workaround that can be implemented through a firmware update without affecting the current flash layout or existing device operation.

Any guidance would be greatly appreciated.

Thanks & Regards,

Urvisha Andani

 

[Einar Thorsrud]

Hi Urvisha,

The partition layout is icorrect, as nvs_storage overlaps with mcuboot_primary. As MCUboot here is not updatable, there is no way to fix this in the field for MCUboot. The only proper fix is to make a new partition layout where you ensure no overlaps.

For devices in the field, you do not have any guarantees against loss of data. However, it may be possible to push an update of the app, where the new app has nvs_storage placed at what is now reseved as EMPTY_0. This is just two pages though. You must expect loss of data though, but in cases where the original NVS data is intact, you could then have the new application copy it to the new region. This is not a good approach nor a recommended approach, but it may be a solution.

As the data never changes during the product lifecycle, a better approach could be to put it in UICR OTP. That is well suited for fixed data, and you have no conflict with DFU and it is outside of the normal partition layout. See this thread.

A last option is to use settings storage, as that does not overlap.

Best regards,

Einar

[Urvisha Andani]

Hello Einar,

Thank you for your quick support.

I have two follow-up questions and would appreciate some additional details:

1. Could you please provide the recommended OTP/UICR region and API for the nRF5340 application core that:
   • can be safely programmed from a running application, and
   • retains its contents across MCUboot DFU/OTA firmware updates?
2. The devices are already deployed in the field and contain existing data. If we implement the proposed changes and deliver them through an OTA update, is there a way to ensure that the currently stored data is preserved and not erased during the update process? If any migration or special handling is required, could you please advise on the recommended approach?

Thank you for your guidance.

Thanks & Regards,

Urvisha Andani

[Einar Thorsrud]

Hi Urvisha,

Urvisha Andani said:

can be safely programmed from a running application, and

Yes, programming runtime is no problem assuming you are not using TF-M. Then you can write to UICR.OTP like other low-levels write to flash. Here is one example. You can also use nrfx nvmc APIs as done in the network core bootlaoder here (this is writing to the APPROTECT register, but the writing part is not fundamentally different, all of UICR is essentially flash, but used a bit differently).

On the network core side there is reseved UICR for customers, which is guaranteed to never be touch bo Nordic libraries. On the application core side, you will have to avoid parts used by other compoents, but this is mostly the bootlaoders, and mostly nsib, which I see from your memory layout that you do not use. for MCUboot, that can use part of UICR for downgrade protection, if enabled. See here, so avoid that region as well.

Urvisha Andani said:

retains its contents across MCUboot DFU/OTA firmware updates?

Yes, the UICR.OTP is write once, in and can only be deleted by a full chip erase. This also means that you cannot re-use parts at a later stage for a device in field.

Urvisha Andani said:

The devices are already deployed in the field and contain existing data. If we implement the proposed changes and deliver them through an OTA update, is there a way to ensure that the currently stored data is preserved and not erased during the update process? If any migration or special handling is required, could you please advise on the recommended approach?

No, this is the most tricky part. You cannot control the behaviour of the current firmware (bootlaoder and application) that runs when you perform the DFU to migitate the issue, so there will be a risk here. I am basing my reply on your initial comment that this only fails sometimes, so I believe the best you can achieve with this approach is to recover the devices that "survive" the first DFU procedure needed to write the updated application firmware that migitates the issue.

Br,

Einar

[Urvisha Andani]

Hello Einar.

Thank you for your quick support and for providing the solution.

I am curious about the technical root cause behind this behavior.

In our testing, we performed approximately 50 OTA updates and did not observe this issue. However, one of our customers has now experienced the problem after running the device through many firmware releases (around 11 OTA versions).

Could you please provide a brief technical RCA (Root Cause Analysis) explaining why the NVS data can occasionally be erased after an MCUboot OTA update?

Specifically, I would like to understand:

• What condition triggers the NVS partition to be erased?
• Why does the issue occur only intermittently rather than on every OTA update?
• Is it related to flash wear, interrupted operations, partition handling, MCUboot swap behavior, or some other mechanism?
• Why might the issue appear only after many OTA cycles/releases in the field while being difficult to reproduce during limited testing?

Understanding the underlying cause will help us explain the issue to our customer and assess the risk for our deployed devices.

One additional observation that may be relevant to the RCA:

Our application image size has actually decreased across the recent releases:

• v05.0F: 742.3 KB
• v05.10: 739.6 KB
• v05.11: 707.1 KB

Thank you for your assistance.

Thanks & Regards,

Urvisha Andani

[Einar Thorsrud]

Hi Urvisha,

The root cause is clear, that is the overlapping partitions. Your nvs_storage (0xFA000–0xFC000) is the same two pages MCUboot reserves at the top of mcuboot_primary for its image trailer and swap-status. MCUboot keeps the trailer at the very end of the slot, so the collision is with MCUboot's metadata and not the image itself.

Urvisha Andani said:

What condition triggers the NVS partition to be erased?

MCUboot erasing/rewriting those top sectors to track swap progress (magic, copy-done, image-ok, swap status). Any NVS record in the erased page is lost.

Urvisha Andani said:

Why does the issue occur only intermittently rather than on every OTA update?

The swap-status footprint rounds up to whole sectors, and how far down it reaches depends on the swap state that boot. A clean confirmed boot may touch only 0xFB000; a full swap, revert, or power-interrupted resume reaches 0xFA000. You only lose data when the erased page is the one NVS is currently using.

Urvisha Andani said:

Is it related to flash wear, interrupted operations, partition handling, MCUboot swap behavior, or some other mechanism?

Primarily MCUboot swap behavior given the partition layout overlap.

Urvisha Andani said:

Why might the issue appear only after many OTA cycles/releases in the field while being difficult to reproduce during limited testing?

I believe this is a matter of probability. The more updates means more swap procedures, and more chances to get a write/erase in part of the slot that overlaps with your NVS data. If you deliberately abort updates in the middle (including resetting/power cycling the device), you may be more successfull at reproducing inhouse?

Urvisha Andani said:

Our application image size has actually decreased across the recent releases:

This should not be related, as it is the metadata part that is always at the top/end of the partition/slot that overlaps.

Br,

Einar