MCUBOOT won't boot image in 2.4.0

[Anthony Ambuehl]

I have an application I have been developing in NCS v2.3.0.  I'm trying now to updata to v2.4.0.

The application targets NRF5340.

It uses NSIB (b0/b0n) and for the application processor also includes mcuboot.

It also uses TF-M and I have implemented a custom service in a secure partition for security features.

It is also configured for mcuboot to use the external flash partition.

The application is based on smp_server and I uses different build configurations to build it either with the console or with smp / mcumgr support.

I have found it necessary to make quite a changes to be compatible with v2.4.0:  Secure partitions have a bunch of changes and I had to change a variety of Kconfig variables as well.  Since my application is drawing from multiple different samples, its hard to know for sure what other changes I may need.

I was eventually able to get the compile to complete cleanly, but I'm finding the mcuboot doesn't find any valid/bootable partition.

In addition no matter what I try, I cannot disable the LOG_MODE_MINIMAL from mcuboot to get more detailed log information.  Even though in child_image/mcuboot.conf has CONFIG_LOG_MODE_MINIMAL=n    the generated mcuboot/zephyr/.config   says CONFIG_LOG_MODE_MINIMAL=y .   I really don't understand why it switches to this value when I specifically requested the opposite.

As well, I cannot debug, since every time I  try to run the debug action in VSCode, I get a write verification failure.

Erasing page at address 0xAB000.
Erasing page at address 0xAC000.
Erasing page at address 0xAD000.
WARNING: A UICR write operation has been requested but UICR has not been
WARNING: erased. Please verify that the result is correct.
Applying system reset.
Checking that the area to write is not protected.
Programming device.
Verifying programming.
ERROR: Write verify failed.
FATAL ERROR: command exited with status 25: nrfjprog --program 'd:\AAA\prov\provisioning_image\smp_svr\build_shell\zephyr\GENERATED_CP_APPLICATION_merged_domains.hex' --sectorerase --verify -f NRF53 --coprocessor CP_APPLICATION --snr 1050031166

* The terminal process terminated with exit code: 25.
* Terminal will be reused by tasks, press any key to close it.

So in summary no logs, no debug, and it won't boot.

How should I proceed with resolving this problem?

[Anthony Ambuehl]

I modified my config, removing:

CONFIG_LOG_DEFAULT_LEVEL=0

and I now get a few more relevant logs.  I further extended the logging in bootloader/mcuboot/boot/bootutil/loader.c, adding logs:

    BOOT_HOOK_CALL_FIH(boot_image_check_hook, FIH_BOOT_HOOK_REGULAR,
                       fih_rc, BOOT_CURR_IMG(state), slot);
    BOOT_LOG_ERR("boot_image_check_hook returned %d", fih_rc);
    if (FIH_EQ(fih_rc, FIH_BOOT_HOOK_REGULAR))
    {
        FIH_CALL(boot_image_check, fih_rc, state, hdr, fap, bs);
        BOOT_LOG_ERR("boot_image_check returned %d", fih_rc);
    }
    if (!boot_is_header_valid(hdr, fap) || FIH_NOT_EQ(fih_rc, FIH_SUCCESS))
    {
        if ((slot != BOOT_PRIMARY_SLOT) || ARE_SLOTS_EQUIVALENT())
        {
            flash_area_erase(fap, 0, flash_area_get_size(fap));
            /* Image is invalid, erase it to prevent further unnecessary
             * attempts to validate and boot it.
             */
        }
#if !defined(__BOOTSIM__)
        BOOT_LOG_ERR("Image in the %s slot is not valid!",
                     (slot == BOOT_PRIMARY_SLOT) ? "primary" : "secondary");
        BOOT_LOG_ERR("header %s valid and rc is %d",
                     (boot_is_header_valid(hdr, fap) ? "is" : "is not"),
                     fih_rc);
#endif

and here are the logs I'm getting 

*** Booting Zephyr OS build v3.3.99-ncs1 ***
I: Starting bootloader
I: Swap type: none
I: Swap type: none
E: boot_image_check_hook returned 1
E: Image in the primary slot is not valid!
E: header is valid and rc is 1
E: Unable to find bootable image

The log indicates the boot_image_check_hook is returning 1 (e.g. FIH_BOOT_HOOK_REGULAR)  but for some reason I'm not getting the log reporting the return value of boot_image_check..

[Anthony Ambuehl]

It looks like wherever boot_image_check_hook is coming from, it has not been updated to work with the Fault Injection Hardening FIH.  FIH is changing the FIH_BOOT_HOOK_REGULAR to 0x19C1F6E1 as I can see from this modified log:

*** Booting Zephyr OS build v3.3.99-ncs1 ***
I: Starting bootloader
I: Swap type: none
I: Swap type: none
E: boot_image_check_hook returned 0x1 regular is 0x19c1f6e1
E: Image in the primary slot is not valid!
E: header is valid and rc is 1
E: Unable to find bootable image

Are there different boot_image_check_hooks for nrf5340 ? 

Do I need to disable the FIH profile for mcuboot to work?

[Anthony Ambuehl]

Confirmed...

v2.4.0/nrf/modules/mcuboot/hooks/nr53_hooks.c is returning BOOT_HOOK_REGULAR instead of FIH_BOOT_HOOK_REGULAR.

I'm not clear on how/when nrf53_hooks.c gets pulled in, but it seems it has not been tested with mcuboot with FIH.

[Vidar Berg]

Thanks for the updates. I notice that the 'nrf5340dk_nrf5340_cpuapp_ns' target is not being tested for the smp_srv sample; only the 'nrf5340dk_nrf5340_cpuapp' target is being tested. If you patch nrf53_hooks.c, would you be able to make it work for now? I'll discuss this further with the developers when they return to the office on Monday.

[Anthony Ambuehl]

Yes I can confirm that after modifying nrf53_hooks.c mcuboot detected the primary slot as valid.

It would be great if in addition to fixing this, they actually included the nrf5340dk_nrf5340_cpuapp_ns in the regression testing.  

Also it would be wonderful if somehow VSCode would be aware of nrf53_hooks., I had to a perform a global grep of the entire v2.4.0 code base from the shell to locate nrf53_hooks.c, and its still unclear to me how/when this files is used.  With SES at least we always had easy access to all of the source code that was included.  Now its a complete mystery. 

Even worse VSCode actually shows incorrect macro values, I have to determine the correct values by logging them at run time.  For example the value of FIH_BOOT_HOOK_REGULAR reports as 1 in VSCode but reports as 0x19c1f6e1 at run time.  I suppose this is probably an artifact of child images having different config settings.. there needs to be a way to deal with that.