Unwanted swap of previously reverted image triggered by updating another image in multi updatable image build

Hello, 

I work on a zephyr project with the MCUBoot and two updatable images: custom firmware loader + TFM and main application.
Update works over BLE. For testing I use an nRF Connect mobile application.
I observed a weird behaviour of the swap mechanism which seems like a bug - I'll explain it in steps:
1. Send firmware loader update with TEST+CONFIRM option (which requires the app to confirm itself before reset to avoid reverting)
2. Reset without confirming the image
3. The firmware loader reverts
4. Send main app update with TEST+CONFIRM option
5. Main app gets swapped with TEST flag (until now everything is as expected) but also, the firmware loader gets swapped with the previously reverted image, and now with the PERM flag

This happens both ways - if I send the main app first, let it revert and then send the firmware loader, the firmware loader gets swapped with the test flag but the main app also gets swapped with the PERM flag with the previously reverted image.

Here are the logs showing the behaviour:

*** Booting MCUboot v2.3.0-dev-fce4dac2e629 ***
*** Using nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 0, Swap type: none
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 1, Swap type: none
I: Bootloader chainload address offset: 0xf000
I: Image version: v0.0.0
I: Jumping to the first image slot
*** Booting nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***

==============================
  Recovery App (nRF54L15 POC)
==============================
Hello from Recovery App!

Normal boot - chain-loading main app (image 1)...
Jumping to 0x000c1800 (MSP=0x20016710, Reset=0x000cbec1)
*** Booting nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***

==============================
  Main App (nRF54L15 POC)
==============================
Hello from Main App!

BLE advertising as "My custom main app"
[main_app] Running... (count=0)
[main_app] Running... (count=1)
*** Booting MCUboot v2.3.0-dev-fce4dac2e629 ***
*** Using nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 0, Swap type: none
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 1, Swap type: none
I: Bootloader chainload address offset: 0xf000
I: Image version: v0.0.0
I: Jumping to the first image slot
*** Booting nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***

==============================
  Recovery App (nRF54L15 POC)
==============================
Hello from Recovery App!

Button 1 pressed - entering recovery mode
*** RECOVERY MODE ***
BLE advertising as "My custom recovery app" (SMP)
BLE connected - DFU ready
BLE disconnected (0x00) - restarting advertising
BLE advertising start failed: -11
BLE advertising start failed: -11
*** Booting MCUboot v2.3.0-dev-fce4dac2e629 ***
*** Using nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 0, Swap type: test
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 1, Swap type: none
I: Starting swap using move algorithm.
I: Bootloader chainload address offset: 0xf000
I: Image version: v0.0.0
I: Jumping to the first image slot
*** Booting nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***

==============================
  Recovery App (nRF54L15 POC)
==============================
Hello from Recovery App!

UPDATED!

Button 1 pressed - entering recovery mode
*** RECOVERY MODE ***
BLE advertising as "My custom recovery app" (SMP)
*** Booting MCUboot v2.3.0-dev-fce4dac2e629 ***
*** Using nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***
I: Starting bootloader
I: Primary image: magic=good, swap_type=0x2, copy_done=0x1, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 0, Swap type: revert
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 1, Swap type: none
I: Starting swap using move algorithm.
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Bootloader chainload address offset: 0xf000
I: Image version: v0.0.0
I: Jumping to the first image slot
*** Booting nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***

==============================
  Recovery App (nRF54L15 POC)
==============================
Hello from Recovery App!

Button 1 pressed - entering recovery mode
*** RECOVERY MODE ***
BLE advertising as "My custom recovery app" (SMP)
BLE connected - DFU ready
BLE disconnected (0x00) - restarting advertising
BLE advertising start failed: -11
BLE advertising start failed: -11
*** Booting MCUboot v2.3.0-dev-fce4dac2e629 ***
*** Using nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***
I: Starting bootloader
I: Primary image: magic=good, swap_type=0x4, copy_done=0x1, image_ok=0x1
I: Secondary image: magic=good, swap_type=0x3, copy_done=0x3, image_ok=0x1
I: Boot source: none
I: Image index: 0, Swap type: perm
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 1, Swap type: test
I: Starting swap using move algorithm.
I: Starting swap using move algorithm.
I: Bootloader chainload address offset: 0xf000
I: Image version: v0.0.0
I: Jumping to the first image slot
*** Booting nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***

==============================
  Recovery App (nRF54L15 POC)
==============================
Hello from Recovery App!

UPDATED!

Normal boot - chain-loading main app (image 1)...
Jumping to 0x000c1800 (MSP=0x20016710, Reset=0x000cbec9)
*** Booting nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***

==============================
  Main App (nRF54L15 POC)
==============================
Hello from Main App!

UPDATED!

BLE advertising as "My custom main app"
[main_app] Running... (count=0)
[main_app] Running... (count=1)
[main_app] Running... (count=2)
*** Booting MCUboot v2.3.0-dev-fce4dac2e629 ***
*** Using nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***
I: Starting bootloader
I: Primary image: magic=good, swap_type=0x3, copy_done=0x1, image_ok=0x1
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 0, Swap type: none
I: Primary image: magic=good, swap_type=0x2, copy_done=0x1, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 1, Swap type: revert
I: Starting swap using move algorithm.
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Bootloader chainload address offset: 0xf000
I: Image version: v0.0.0
I: Jumping to the first image slot
*** Booting nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***

==============================
  Recovery App (nRF54L15 POC)
==============================
Hello from Recovery App!

UPDATED!

Normal boot - chain-loading main app (image 1)...
Jumping to 0x000c1800 (MSP=0x20016710, Reset=0x000cbec1)
*** Booting nRF Connect SDK v3.3.0-ba167d9f3db4 ***
*** Using Zephyr OS v4.3.99-fd9204a02d52 ***

==============================
  Main App (nRF54L15 POC)
==============================
Hello from Main App!

BLE advertising as "My custom main app"
[main_app] Running... (count=0)
[main_app] Running... (count=1)

Here are patches that seem to solve the issue:

diff --git a/boot/bootutil/src/loader.c b/boot/bootutil/src/loader.c
index fcd67afb..e1e0c327 100644
--- a/boot/bootutil/src/loader.c
+++ b/boot/bootutil/src/loader.c
@@ -773,7 +773,7 @@ check_validity:
 #endif
     if (FIH_NOT_EQ(fih_rc, FIH_SUCCESS)) {
         if ((slot != BOOT_SLOT_PRIMARY) || ARE_SLOTS_EQUIVALENT()) {
-            boot_scramble_slot(fap, slot);
+            swap_scramble_trailer_sectors(state, fap);
             /* Image is invalid, erase it to prevent further unnecessary
              * attempts to validate and boot it.
              */
@@ -1679,6 +1679,34 @@ boot_swap_image(struct boot_loader_state *state, struct boot_status *bs)
  *
  * @return                      0 on success; nonzero on failure.
  */
+
+#if !defined(MCUBOOT_OVERWRITE_ONLY) && (BOOT_IMAGE_NUMBER > 1)
+
+static void boot_clear_secondary_trailer_after_revert(struct boot_loader_state *state)
+{
+    const struct flash_area *fap_sec;
+    size_t hdr_off;
+    size_t hdr_size;
+    int rc;
+
+    fap_sec = BOOT_IMG_AREA(state, BOOT_SLOT_SECONDARY);
+
+    hdr_off = 0;
+    hdr_size = ALIGN_UP(sizeof(uint32_t), flash_area_align(fap_sec));
+    rc = boot_scramble_region(fap_sec, hdr_off, hdr_size, false);
+    if (rc != 0) {
+        BOOT_LOG_WRN("Failed to invalidate secondary header after revert"
+                     " (image %d)", BOOT_CURR_IMG(state));
+    }
+
+    rc = swap_scramble_trailer_sectors(state, fap_sec);
+    if (rc != 0) {
+        BOOT_LOG_WRN("Failed to clear secondary trailer after revert"
+                     " (image %d)", BOOT_CURR_IMG(state));
+    }
+}
+#endif /* !MCUBOOT_OVERWRITE_ONLY && BOOT_IMAGE_NUMBER > 1 */
+
 static int
 boot_perform_update(struct boot_loader_state *state, struct boot_status *bs)
 {
@@ -1720,6 +1748,17 @@ boot_perform_update(struct boot_loader_state *state, struct boot_status *bs)
         }
     }
 
+#if (BOOT_IMAGE_NUMBER > 1)
+    if (swap_type == BOOT_SWAP_TYPE_REVERT) {
+        boot_clear_secondary_trailer_after_revert(state);
+
+        rc = swap_set_copy_done(BOOT_CURR_IMG(state));
+        if (rc != 0) {
+            BOOT_SWAP_TYPE(state) = BOOT_SWAP_TYPE_PANIC;
+        }
+    }
+#endif
+
 #ifdef MCUBOOT_HW_ROLLBACK_PROT
     if (swap_type == BOOT_SWAP_TYPE_PERM) {
         /* Update the stored security counter with the new image's security
@@ -1784,7 +1823,14 @@ boot_complete_partial_swap(struct boot_loader_state *state,
         }
     }
 
-    if (BOOT_IS_UPGRADE(bs->swap_type)) {
+#if (BOOT_IMAGE_NUMBER > 1)
+    if (bs->swap_type == BOOT_SWAP_TYPE_REVERT) {
+        boot_clear_secondary_trailer_after_revert(state);
+    }
+#endif
+
+    if (BOOT_IS_UPGRADE(bs->swap_type) ||
+        bs->swap_type == BOOT_SWAP_TYPE_REVERT) {
         rc = swap_set_copy_done(BOOT_CURR_IMG(state));
         if (rc != 0) {
             BOOT_SWAP_TYPE(state) = BOOT_SWAP_TYPE_PANIC;

diff --git a/subsys/mgmt/mcumgr/grp/img_mgmt/src/img_mgmt.c b/subsys/mgmt/mcumgr/grp/img_mgmt/src/img_mgmt.c
index b45b9d26d7..bc7c5aed3c 100644
--- a/subsys/mgmt/mcumgr/grp/img_mgmt/src/img_mgmt.c
+++ b/subsys/mgmt/mcumgr/grp/img_mgmt/src/img_mgmt.c
@@ -14,6 +14,8 @@
 #include <zephyr/logging/log.h>
 #include <zephyr/storage/flash_map.h>
 #include <zephyr/dfu/mcuboot.h>
+#include <bootutil/bootutil_public.h>
+#include "flash_map_backend/flash_map_backend.h"
 
 #include <zcbor_common.h>
 #include <zcbor_decode.h>
@@ -295,6 +297,76 @@ int img_mgmt_active_image(void)
 	return ACTIVE_IMAGE_IS;
 }
 
+static size_t img_mgmt_swap_metadata_size(void)
+{
+	size_t size = BOOT_MAGIC_ALIGN_SIZE + (BOOT_MAX_ALIGN * 4);
+
+#if defined(CONFIG_MCUBOOT_BOOTLOADER_MODE_SWAP_USING_OFFSET)
+	size += BOOT_MAX_ALIGN;
+#endif
+
+	return size;
+}
+
+static void img_mgmt_clear_trailer_for_area(const struct flash_area *fa)
+{
+	size_t fa_size = flash_area_get_size(fa);
+	size_t trailer_erase = MIN(img_mgmt_swap_metadata_size(), fa_size);
+
+	if (trailer_erase == 0) {
+		return;
+	}
+
+	(void)flash_area_erase(fa, fa_size - trailer_erase, trailer_erase);
+}
+
+static bool img_mgmt_ranges_overlap(uint32_t start_a, uint32_t len_a,
+				    uint32_t start_b, uint32_t len_b)
+{
+	return start_a < (start_b + len_b) && start_b < (start_a + len_a);
+}
+
+static void img_mgmt_clear_overlapping_secondary_trailers(int target_area_id,
+						  size_t write_size)
+{
+#if SLOTS_PER_IMAGE > 1
+	const struct flash_area *target_fa;
+	int rc;
+	uint32_t target_start;
+
+	rc = flash_area_open(target_area_id, &target_fa);
+	if (rc != 0) {
+		return;
+	}
+
+	target_start = (uint32_t)target_fa->fa_off;
+
+	for (int image = 0; image < CONFIG_MCUMGR_GRP_IMG_UPDATABLE_IMAGE_NUMBER; image++) {
+		const struct flash_area *secondary_fa;
+		int slot = image * SLOTS_PER_IMAGE + 1;
+		int area_id = img_mgmt_flash_area_id(slot);
+
+		rc = flash_area_open(area_id, &secondary_fa);
+		if (rc != 0) {
+			continue;
+		}
+
+		if (img_mgmt_ranges_overlap(target_start, (uint32_t)write_size,
+						            (uint32_t)secondary_fa->fa_off,
+						            (uint32_t)secondary_fa->fa_size)) {
+			img_mgmt_clear_trailer_for_area(secondary_fa);
+		}
+
+		flash_area_close(secondary_fa);
+	}
+
+	flash_area_close(target_fa);
+#else
+	ARG_UNUSED(target_area_id);
+	ARG_UNUSED(write_size);
+#endif
+}
+
 #ifdef CONFIG_NCS_MCUBOOT_MANIFEST_UPDATES
 /**
  * Checks whether the manifest of the image in the specified slot can be used for booting.
@@ -1079,6 +1151,9 @@ defined(CONFIG_MCUMGR_SMP_COMMAND_STATUS_HOOKS)
 			}
 		}
 #endif
+
+		img_mgmt_clear_overlapping_secondary_trailers(action.area_id, req.size);
+		rc = 0;
 	} else {
 #if defined(CONFIG_MCUMGR_SMP_COMMAND_STATUS_HOOKS)
 		cmd_status_arg.status = IMG_MGMT_ID_UPLOAD_STATUS_ONGOING;

Could you confirm if you observe the same behaviour when working with MCUBoot + 2 updatable images and if it is for some reason expected or not?

NCS: v3.3.0
HW:  nRF54L15-DK

 

In case above I used two secondary slots - one for updating each image. However, I wondered if it is possible to use one secondary slot for updating both images (one at a time). I managed to configure it that way and it seems to work fine (in this case I also needed to fix the bug described above - behaviour was different as it led to brick the device now but the reason was the same).
Is it a correct way to implement it like this to save flash memory?
Beside accepting the fact that the application with the test flag will not revert if there is already another update in progress, is there anything else to worry?

Regards
Filip

Parents Reply Children
Related