Mcuboot swap using move will brick device when power interrupted during move

RE: Mcuboot swap using move will brick device when power interrupted during move

ImaraSpeek — Tue, 14 Feb 2023 13:14:12 GMT

Hi Sigurd,

Mcuboot has been able to reproduce the issue and we have found a fix. Please see this issue link:

https://github.com/mcu-tools/mcuboot/issues/1588

Thanks,

Imara

RE: Mcuboot swap using move will brick device when power interrupted during move

Sigurd Hellesvik — Tue, 14 Feb 2023 11:37:25 GMT

Hi Imara,

Sorry for the wait.

With the excellent information you have provided, I will try to reproduce your issue.

A bootloader should be robust, and it should be tough to brick a device with a reset.
If this turns out to be the case, I will let our developers know, and we will look into how we can fix this.

I will return with more information on my progress on Thursday latest.
Please let me know if you have found anything else since creating this post!

Regards,
Sigurd Hellesvik

RE: Mcuboot swap using move will brick device when power interrupted during move

ImaraSpeek — Tue, 31 Jan 2023 16:37:46 GMT

Another thought though: If the error occurs during the move, not the swap it would result in a state that isn't described in this section of the documentation, since copy_done isn't set yet. Does that mean that when the move (instead of the swap) is interrupted, it will never set the outcome to BOOT_SWAP_TYPE_REVERT and thus not recover?

The bool_status_is_reset will also think the status is reset when the swap is still busy moving the sectors us. Therefor it is required to read all headers, even though the header of the primary image has moved.

bool
boot_status_is_reset(const struct boot_status *bs)
{
    return (bs->op == BOOT_STATUS_OP_MOVE &&
            bs->idx == BOOT_STATUS_IDX_0 &&
            bs->state == BOOT_STATUS_STATE_0);
}

#ifdef MCUBOOT_SWAP_USING_MOVE
        /*
         * Must re-read image headers because the boot status might
         * have been updated in the previous function call.
         */
        rc = boot_read_image_headers(state, !boot_status_is_reset(bs), bs);
#ifdef MCUBOOT_BOOTSTRAP
        /* When bootstrapping it's OK to not have image magic in the primary slot */
        if (rc != 0 && (BOOT_CURR_IMG(state) != BOOT_PRIMARY_SLOT ||
                boot_check_header_erased(state, BOOT_PRIMARY_SLOT) != 0)) {
#else
        if (rc != 0) {
#endif

            /* Continue with next image if there is one. */
            BOOT_LOG_WRN("Failed reading image headers; Image=%u",
                    BOOT_CURR_IMG(state));
            BOOT_SWAP_TYPE(state) = BOOT_SWAP_TYPE_NONE;
            return;
        }
#endif

This then causes the code to return and never reach boot_complete_partial_swap as far as I can tell.

RE: Mcuboot swap using move will brick device when power interrupted during move

ImaraSpeek — Tue, 31 Jan 2023 12:27:05 GMT

Some more info about this issue:

Our reboot is issued whenever the entire device is updated. This causes the power to the nrf52840 to also briefly be cut, causing a reboot of the nrf52840.

Both partitions are on internal flash and the images are not encrypted

The bin image is 0x5FB40 in size, the mcuboot_primary is 0x6D000, the mcuboot_primary_app starts 0x200 later and is 0x6CE00. The mcuboot_secondary starts right after and is 0x6D000. I've already tried increasing the primary partition to 0x6E000 so that it'd be a sector larger than the secondary, but that didn't change anything yet. The hex I've added comes from the prior situation where both partitions are equal size (which should also be allowed according to the documentation).

Please see the attached hexes of the last sectors (from when the partitions were still equally sized) after reproducing the issue.

The image trailer is exactly the same except for the copy_done and image_ok flags (I think these are the values in the memory, but I'm not sure). This means that the faulty image trailer indicates that a swap has been successfully done, but the image hasn't been confirmed. Is is possible that this image trailer is left from a previous swap and isn't correctly cleaned up after a swap? In my tests, I only ever reproduce this issue after > 25 updates, so that'd mean that many successful swaps happened prior to the faulty case.

devzone.nordicsemi.com/.../failed_5F00_swap_5F00_hexes.zip