OTA DFUs no longer working after recreating build configuration

RE: OTA DFUs no longer working after recreating build configuration

JakeM — Thu, 30 Jan 2025 16:14:03 GMT

We talked to our local sales representative. After not making much progress on non-simultaneous FOTA either, we're planning to get help with the network core changes we intend to make, so those changes can be made before going into production, and try to avoid having to update the network core in the future.

Since we found an explanation for the original discrepancy between builds, and have a plan for addressing future compatibility concerns, I think this case can be closed.

RE: OTA DFUs no longer working after recreating build configuration

Sigurd Hellesvik — Wed, 29 Jan 2025 10:22:57 GMT

[quote user="JakeM"]
Are there any plans to support simultaneous DFU without external flash on nRF5430 in a future SDK release?[/quote]

Devs link me to https://github.com/nrfconnect/sdk-nrf/pull/19551, so this could work.
However, we do not have any tests for this, meaning that we can not claim support for the feature.

So I will just quote our default answer when it comes to timeline questions:

I am not able to answer questions about our timeline. You can try to ask your local sales representative from Nordic Semiconductor for information about our timeline.

[quote user="JakeM"]
For now it looks like our best option is non-simultaneous.[/quote]

Yea seems like it. As always, remember to test FOTA thoroughly on test devices before rolling it out to the field. This is always important of course, but it becomes extra important when you do non-simultaneous instead of simultaneous.

RE: OTA DFUs no longer working after recreating build configuration

Sigurd Hellesvik — Thu, 23 Jan 2025 11:54:40 GMT

Ill check with devs

RE: OTA DFUs no longer working after recreating build configuration

JakeM — Thu, 23 Jan 2025 00:03:59 GMT

Thanks for the response. Those changesets look a little intimidating. Maybe there are some changes we don't need, but I'm seeing a lot of changes that look related to the functionality we want. I don't think we want to maintain a branch of the SDK.

Are there any plans to support simultaneous DFU without external flash on nRF5430 in a future SDK release?

For now it looks like our best option is non-simultaneous.

RE: OTA DFUs no longer working after recreating build configuration

Sigurd Hellesvik — Tue, 21 Jan 2025 11:50:01 GMT

I found these two PRs as a workaround from some time ago:

https://github.com/nrfconnect/sdk-nrf/pull/10060
https://github.com/nrfconnect/sdk-mcuboot/pull/235

As you can see there are some limitations in the system to simultaneous multi-image DFU from internal flash.

Maybe you can try the workaround on your device and and see if that works?

RE: OTA DFUs no longer working after recreating build configuration

JakeM — Mon, 20 Jan 2025 23:53:23 GMT

Sigurd,

Do you have any advice for getting simultaneous DFU working without flash? I still haven't figured out the partition config.

Thanks,
Jake

RE: OTA DFUs no longer working after recreating build configuration

JakeM — Thu, 16 Jan 2025 22:46:06 GMT

I have new insight into this issue that resolves one of my original questions.

Recall, I had OTAs built two ways that were incompatible (versions 0.6–0.9 vs. 0.10–0.11). I'm now able to replicate the 0.6–0.9 style build. What apparently happened is I initially built with CONFIG_NRF53_UPGRADE_NETWORK_CORE=y in prj.conf, then removed that option, and through version 0.9 performed only incremental builds, instead of pristine. The developer who built 0.10 had never enabled the network core option. I recreated my build configuration before building 0.11.

As I understand, my broken build configuration created a DFU that expected a network core image, but didn't actually contain net_core_app_update.bin in dfu_application.zip (only app_update.bin and manifest.json), so it worked, but then wasn't compatible with later builds that neither expected nor contained the network core update. This also explains why the hex files look the way they do in the programmer, yet the network core wasn't being built with the new build configuration.

Your most recent response helps a great deal. I think we want to do a simultaneous DFU if possible, but I'm still struggling to get that working without external flash. In the correspondence between Chen and Andreas, eventually Chen gave up and decided to do non-simultaneous DFUs.

How do I modify the flash partition config to resolve the "Missing partitions?" static assert? Is there some example I can start from?

RE: OTA DFUs no longer working after recreating build configuration

Sigurd Hellesvik — Thu, 09 Jan 2025 08:26:48 GMT

[quote user="JakeM"]Am I understanding the correspondence between Chen and Andreas correctly that we need to evaluate space usage to determinate if multi-image or separate OTAs is the best strategy? [/quote]

Yes.

So the reason why our samples all use external flash for simultaneous multi-image DFU is that you need a lot of memory to store all slots.

As you see you will need to split your internal flash into 4 parts:
mcuboot
application
application backup (for DFU)
netcore backup (for DFU)

This logically results in less space available for DFU.

If you do non-simultaneous multi-core DFU you will get only three slots:

However, with this you cannot update the interface between the application core and network core, so it limits the updatability of the nRF5340 network core.

[quote user="JakeM"]I'm also not sure any of this is helping with the original issue of OTA compatibility breaking for unknown reasons. Is there somewhere private I can post our complete code to get specific recommendations?[/quote]

Create ticket -> Create private ticket is where you go for that.

[quote user="JakeM"]We're mostly looking for guidance on ensuring compatibility for future updates. I appreciate that we need a pm_static.yml, but exactly how that looks is going to depend on our OTA strategy.[/quote]

Compatability for future updates:

pm_static.yml is the most important here. To make that you can just copy build/partitions.yml when you are happy with your project configuration. This will freeze partitioning from that point onwards.

Then the only other thing I that is important for future updates is the network core update, which is why we discussed the network core now.
There are several ways to deal with updating the network core, and they all come with pros and cons. Let me try to summarize these:

No Network core DFU
- Pro: No extra complexity
- Pro: No extra space requirements
- Con: Can never update network core. This can put restrictions on later application updates. (For example cannot update to some new BLE feature)
Non-simultaneous Network core DFU
- Pro: No extra space requirements
- Con: Cannot update interface between network core and application core. This means you must be a bit careful when duing DFU of the network core. Allows for some updates of the network core, but not all.
Simultaneous Network core DFU
- Pro: Lets you update both cores no matter what. Full future-proofing
- Con: Uses extra space. Will put limitations on current application size. Will also limit how much you can increase application size in future DFUs

Oh and one more thing I remember that you need to keep in mind. Application size cannot be more than approx 95% of slot size. Ref Known Issue NCSDK-20567: Partitioning limitation with MCUboot swap move.

RE: OTA DFUs no longer working after recreating build configuration

JakeM — Wed, 08 Jan 2025 23:40:44 GMT

Am I understanding the correspondence between Chen and Andreas correctly that we need to evaluate space usage to determinate if multi-image or separate OTAs is the best strategy? I'm also not sure any of this is helping with the original issue of OTA compatibility breaking for unknown reasons. Is there somewhere private I can post our complete code to get specific recommendations?

We're mostly looking for guidance on ensuring compatibility for future updates. I appreciate that we need a pm_static.yml, but exactly how that looks is going to depend on our OTA strategy.

RE: OTA DFUs no longer working after recreating build configuration

Sigurd Hellesvik — Wed, 08 Jan 2025 09:57:15 GMT

[quote user="JakeM"]To that end, I think I need to lock down a pm_static.yml[/quote]

Yes this is very important!

If partitioning changed between 0.9 and 0.11 this would explain the error you get. However from the flash readouts this does not look like the case so I did not comment on it before.

[quote user="JakeM"]Is there an example using SDK >=2.5 that doesn't require external flash? Will getting multi-image to work help resolve the original confusion over incompatible OTAs?[/quote]

See the answer in NRF5340 MCUBOOT DFU failed: Slot image has no hash TLV . Especially see the restrictions on space with this solution.

RE: OTA DFUs no longer working after recreating build configuration

JakeM — Tue, 07 Jan 2025 23:08:20 GMT

I initially implemented FOTA over BLE (as the guide calls it) several months ago, basically following Exercise 3 through step 5.

Step 6 gets into multi-image, which I hadn't previously attempted. I assume we want that, in case we need to modify the network core in the future. However the example doesn't work, because our hardware doesn't have the external flash.

Is there an example using SDK >=2.5 that doesn't require external flash? Will getting multi-image to work help resolve the original confusion over incompatible OTAs?

My ultimate goal is ensuring compatibility with whatever changes we might need to make in the future via OTA. To that end, I think I need to lock down a pm_static.yml, presumably multi-image, but accounting for the lack of external flash.

RE: OTA DFUs no longer working after recreating build configuration

Sigurd Hellesvik — Mon, 06 Jan 2025 14:55:54 GMT

When you got CONFIG_BOOTLOADER_MCUBOOT enabled on the NRF5340, generally the entwork core bootloader should be automatically enabled.

Can you check our course over at https://academy.nordicsemi.com/courses/nrf-connect-sdk-intermediate/lessons/lesson-8-bootloaders-and-dfu-fota/?
Did you do the steps like this to update the nRF5340?

RE: OTA DFUs no longer working after recreating build configuration

JakeM — Fri, 03 Jan 2025 16:23:53 GMT

0.6 was the version where I initially added OTA DFU support, by adding CONFIG_BOOTLOADER_MCUBOOT and CONFIG_NCS_SAMPLE_MCUMGR_BT_OTA_DFU and a call to smp_bt_register() in main(). We had previously been cable programming exclusively. The hope was for all subsequent versions to be OTA-able over a cable-programmed 0.6 or later.

At some point after making 0.9, I deleted and recreated my build configuration and haven't been able to build OTAs compatible with 0.6–0.9 since; however the builds I've been making with the new build configuration are all compatible with each other.

build/hci_rpmsg does not contain a b0n directory when I make a 0.10–0.11 style build. Unfortunately, I don't have the entire build directory saved for 0.6–0.9, nor do I have ability to recreate them, so I don't know whether they had b0n.

We're trying to understand what broke compatibility to ensure it doesn't happen again.

RE: OTA DFUs no longer working after recreating build configuration

Sigurd Hellesvik — Fri, 03 Jan 2025 09:26:21 GMT

[quote user="JakeM"]I'm trying to make DFUs that can be applied over apps that have originally been cable programmed.[/quote]

Which version did you program before trying DFU? 0.9, 0.10 or something else?

[quote user="JakeM"]The zip files for old and new builds both contain only app_update.bin and manifest.json. I don't see any noteworthy differences.[/quote]

If the files had DFU for netcore ready, the zip would also contain a .bin file for the netcore.

Can you check if there is a b0n folder inside build/hci_rpmsg?

RE: OTA DFUs no longer working after recreating build configuration

JakeM — Thu, 02 Jan 2025 16:43:08 GMT

Sigurd,

I'm trying to make DFUs that can be applied over apps that have originally been cable programmed.

The zip files for old and new builds both contain only app_update.bin and manifest.json. I don't see any noteworthy differences.

I have found something interesting, however. After cable programming to 0.10 or 0.11, we can DFU successfully to 0.11 or 0.10, just not 0.6–0.9. The same is true for the earlier version range. If cable programmed to 0.6–0.9, DFU to another 0.6–0.9 version works, just not 0.10 or 0.11.

I verified the presence of build\hci_rpmsg\, including a zephyr subdirectory and app.hex file.

The top section shows "Core Name Network" in a tooltip when hovered over. Green bands show "Application", and orange "MBR or Application", so it looks like rather than the bootloader being missing, perhaps the application code is being inadvertently built into the bootloader. Certainly the network core is booting with every version.

Thanks,
Jake

RE: OTA DFUs no longer working after recreating build configuration

Sigurd Hellesvik — Mon, 30 Dec 2024 11:52:50 GMT

Hi Jake, and welcome to DevZone!

Here you can get answers from our community, we have a lot of skilled people who like to help. In addition Nordic has a team of support engineers, like me, who are assigned tickets and will assist you guys as best we can.

Alright, I like to start by asking some questions to understand the issue better:

When you changed the application now, did you reflash the nRF5340? Or is it just DFU on top of the old app?

Could you check what is inside the .zip file now?

The top memory is the network core I think, and the small bar there should be the network core bootloader (known as b0n). Looks like maybe b0n is missing in your new build. This is a bit strange though, as that should be added automatically. Can you check your build/hci_rpmsg/ folder and see if you find a folder named b0n in here?

Regards,
Sigurd Hellesvik