Bug in Secure Boot / b0n related partition configuration for nRF5340 under sysbuild in SDK v2.9.0?

[puz_md]

Hi, I'm having problems with migrating my project to SDK v2.9.0. I'm using mcuboot and I want to use b0n on the net core and update support for the net core application via mcuboot (running on the application core). Everything was running fine on the SDK v2.5.3 with multi-image build, and I managed to port the project to sysbuild on SDK v2.9.0 with mcuboot, but when I try to enable the net core update, the build process fails.

I tried to re-create the configuration with the peripheral_lbs sample (SDK v2.9.0). After checking out the sample project, I added sysbuild.conf to the project folder:

#SB_CONFIG_NETCORE_APP_UPDATE=y
#SB_CONFIG_SECURE_BOOT_NETCORE=y
#SB_CONFIG_SECURE_BOOT_SIGNING_KEY_FILE="C:/<mypath>/mykey.pem"

SB_CONFIG_BOOTLOADER_MCUBOOT=y
SB_CONFIG_MCUBOOT_MODE_SINGLE_APP=n
SB_CONFIG_BOOT_SIGNATURE_KEY_FILE="C:/<mypath>/mykey.pem"
SB_CONFIG_BOOT_SIGNATURE_TYPE_ECDSA_P256=y

SB_CONFIG_NETCORE_HCI_IPC=y

So far, everything compiled without any problem. But when I try to enable the upper three lines (for b0n and net core update support, the build process fails: I receive the following error message seven times before the build process is aborted:

In file included from C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/include/sysflash/sysflash.h:12,
                 from C:/ncs/v2.9.0/bootloader/mcuboot/boot/bootutil/src/bootutil_priv.h:33,
                 from C:/ncs/v2.9.0/bootloader/mcuboot/boot/bootutil/src/tlv.c:24:
C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/include/sysflash/pm_sysflash.h: In function '__flash_area_ids_for_slot':
C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/include/sysflash/pm_sysflash.h:20:37: error: 'PM_MCUBOOT_PRIMARY_1_ID' undeclared (first use in this function); did you mean 'PM_MCUBOOT_PRIMARY_ID'?
   20 | #define FLASH_AREA_IMAGE_1_SLOTS    PM_MCUBOOT_PRIMARY_1_ID, PM_MCUBOOT_SECONDARY_1_ID,
      |                                     ^~~~~~~~~~~~~~~~~~~~~~~
C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/include/sysflash/pm_sysflash.h:38:29: note: in expansion of macro 'FLASH_AREA_IMAGE_1_SLOTS'
   38 |                             FLASH_AREA_IMAGE_1_SLOTS
      |                             ^~~~~~~~~~~~~~~~~~~~~~~~
C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/include/sysflash/pm_sysflash.h:55:9: note: in expansion of macro 'ALL_AVAILABLE_SLOTS'
   55 |         ALL_AVAILABLE_SLOTS
      |         ^~~~~~~~~~~~~~~~~~~
C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/include/sysflash/pm_sysflash.h:20:37: note: each undeclared identifier is reported only once for each function it appears in
   20 | #define FLASH_AREA_IMAGE_1_SLOTS    PM_MCUBOOT_PRIMARY_1_ID, PM_MCUBOOT_SECONDARY_1_ID,
      |                                     ^~~~~~~~~~~~~~~~~~~~~~~
C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/include/sysflash/pm_sysflash.h:38:29: note: in expansion of macro 'FLASH_AREA_IMAGE_1_SLOTS'
   38 |                             FLASH_AREA_IMAGE_1_SLOTS
      |                             ^~~~~~~~~~~~~~~~~~~~~~~~
C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/include/sysflash/pm_sysflash.h:55:9: note: in expansion of macro 'ALL_AVAILABLE_SLOTS'
   55 |         ALL_AVAILABLE_SLOTS
      |         ^~~~~~~~~~~~~~~~~~~
C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/include/sysflash/pm_sysflash.h:20:62: error: 'PM_MCUBOOT_SECONDARY_1_ID' undeclared (first use in this function); did you mean 'PM_MCUBOOT_SECONDARY_ID'?
   20 | #define FLASH_AREA_IMAGE_1_SLOTS    PM_MCUBOOT_PRIMARY_1_ID, PM_MCUBOOT_SECONDARY_1_ID,
      |                                                              ^~~~~~~~~~~~~~~~~~~~~~~~~
C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/include/sysflash/pm_sysflash.h:38:29: note: in expansion of macro 'FLASH_AREA_IMAGE_1_SLOTS'
   38 |                             FLASH_AREA_IMAGE_1_SLOTS
      |                             ^~~~~~~~~~~~~~~~~~~~~~~~
C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/include/sysflash/pm_sysflash.h:55:9: note: in expansion of macro 'ALL_AVAILABLE_SLOTS'
   55 |         ALL_AVAILABLE_SLOTS
      |         ^~~~~~~~~~~~~~~~~~~

It appears like the build system suddenly is using different partition names when I enable b0n?

Under SDK v2.5.3, the app core partitions were just named mcuboot_primary (mcuboot_primary_app) and mcuboot_secondary. Both net core update and app core update were loaded in mcuboot_secondary for flashing. Now, under SDK v2.9.0 and sysbuild, there seems to be a different naming convention (see PM_MCUBOOT_PRIMARY_1_ID error messages).

Some feedback would be appreciated. Is this a bug in the build/configuration system? Or is there a new convention for partition naming that changes when b0n is enabled?

Best regards,
Michael

PS: Update: I tried disabling SB_CONFIG_NETCORE_APP_UPDATE, but leave secure boot for the net core active. In this case, I get the following error messages:

In file included from C:/ncs/v2.9.0/nrf/include/dfu/pcd.h:28,
                 from C:/ncs/v2.9.0/bootloader/mcuboot/boot/zephyr/main.c:95:
C:/ncs/v2.9.0/nrf/include/dfu/pcd_common.h: In function 'pcd_write_cmd_lock_debug':
C:/ncs/v2.9.0/nrf/include/dfu/pcd_common.h:39:25: error: 'PM__PCD_SRAM_ADDRESS' undeclared (first use in this function); did you mean 'PM_SRAM_ADDRESS'?
   39 | #define PCD_CMD_ADDRESS PM__PCD_SRAM_ADDRESS
      |                         ^~~~~~~~~~~~~~~~~~~~

Beside 'pcd_write_cmd_lock_debug', the same error occurs for functions 'pcd_read_cmd_done' and 'pcd_read_cmd_lock_debug'.

[puz_md]

Hi Vidar,

I think I got it working. Thanks a lot!

Concerning my configuration problems, I worked through the following steps to fix them:
(1) Create a pm_static.yml for the app core without the net core update configuration (otherwise it won't compile with SDK v2.9.0)
(2) Add the mcuboot_primary_1 partition
(3) Add the ram_flash partition
(4) Duplicate the mcuboot_secondary partition and rename the copy to mcuboot_secondary_1 with size and (end_address) adjusted to 0x40000
(5) Add "CONFIG_MCUBOOT_VERIFY_IMG_ADDRESS=n" to sysbuild\mcuboot.conf (seems to work without it, too)

In respect to these steps, I have some more questions concerning the further proceeding at Nordic Semiconductor and the nRF Connect SDK:

According to my understanding (and the partitioning training in Nordic Academy), the steps (2) through (4) should not be needed to be performed by the developer. The SDK should create partitions ready for use. Will this issue be fixed in future SDK versions?

Also, the extra (overlapping) partition definitions and the flash_ram partition were not needed in SDK v2.5.3. Will this configuration pattern also be used in future SDK versions, or is this just a workaround for the current problems?

Do I need the "CONFIG_MCUBOOT_VERIFY_IMG_ADDRESS=n"? It seems my project also works without it...

And do I need the Konfig and Kconfig.sysbuild? I've never used these files in my configuration. Currently, I'm using "SB_CONFIG_NETCORE_HCI_IPC=y" in sysbuild.conf. Does this something similar to the ipc_radio configuration in Kconfig.sysbuild?

Best regards,
Michael

[Vidar Berg]

Hi Michael,

Glad to hear that it is working on your end as well. Just remember to test this with your bootloader hex from v2.5.3 if you have devices out in the field with this version of the bootloader.

puz_md said:

According to my understanding (and the partitioning training in Nordic Academy), the steps (2) through (4) should not be needed to be performed by the developer. The SDK should create partitions ready for use. Will this issue be fixed in future SDK versions?

The problem is that dynamic partitioning is not supported with your configuration. This was partially addressed by the PR I linked to earlier. I will report this internally.

puz_md said:

Do I need the "CONFIG_MCUBOOT_VERIFY_IMG_ADDRESS=n"? It seems my project also works without it...

Did you verify that the netcore image got updated even if you had additional address verification enabled? 

puz_md said:

And do I need the Konfig and Kconfig.sysbuild? I've never used these files in my configuration. Currently, I'm using "SB_CONFIG_NETCORE_HCI_IPC=y" in sysbuild.conf. Does this something similar to the ipc_radio configuration in Kconfig.sysbuild?

Kconfig.sysbuild changes the default value to 'y'  instead of explicitly enabling the HCI IPC image in sysbuild.conf. This allows the project to be built for other single core  devices (nRF52/nRF54) without triggering any errors because the symbol can't be selected. You are selecting HCI IPC instead of the IPC radio firmware when you select SB_CONFIG_NETCORE_HCI_IPC.

Best regards,

Vidar

[puz_md]

Hi Vidar,

Vidar Berg said:

Glad to hear that it is working on your end as well. Just remember to test this with your bootloader hex from v2.5.3 if you have devices out in the field with this version of the bootloader.

I tested it already and it worked.

The problem was that I did not program the new mcuboot binary and test the following issue:

Vidar Berg said:

puz_md said:

Do I need the "CONFIG_MCUBOOT_VERIFY_IMG_ADDRESS=n"? It seems my project also works without it...

Did you verify that the netcore image got updated even if you had additional address verification enabled? 

You're right, it doesn't work without turning address verification off! I forgot to program the new mcuboot for testing.

Unfortunately, I ran into another problem now with the new mcuboot: It programs the app core software just fine. But the net core software is not removed/invalidated from the slot after updating the net core (which seems to work by the way): During each power cycle, the system stays for many seconds in mcuboot and probably tries to update the net core again. After the update try, I can connect via Auterm again and the slot readout reports has the following bits set:

The app core is updated just fine and Auterm doesn't report an image in the slot anymore (and the bootloader does not consume extra time during each startup after the first update attempt). Any idea what could be wrong with the net core processing this time?

By the way, only if I erase the flash pages of the secondary slot, in Auterm "Slot 1" disappears and I can upload another update again.

Best regards,
Michael

[Vidar Berg]

Hi Michael,

puz_md said:

Unfortunately, I ran into another problem now with the new mcuboot: It programs the app core software just fine. But the net core software is not removed/invalidated from the slot after updating the net core (which seems to work by the way):

Could you please confirm if you see the same with your original MCUBoot version? I'm able to reproduce the same here but I could not find any relevant changes in loader.c between v2.5.x and v2.9.0.

Best regards,

Vidar

[puz_md]

Hi Vidar,

I did not see this behavior with the old mcuboot (SDK v2.5.3). It might be related to the swap feature (I did not configure the 'overwrite only' parameter, maybe I can try it another time). For now, this is the observation that I make with mcuboot SDK v2.9.0:

(1) App core update: After the update is processed, the first and the last flash page of the secondary partition is erased (all bytes 0xff). The image is NOT swapped as I would expect (and as it worked with SDK v2.3.0).

(2) Net core update: After the update, the secondary partition is untouched! In the last flash page of the secondary partition, the update/confirmed bits are still set! I think this should not happen. Also, considering the time delay after each power cycle in this state, it is possible that the net core is programmed again (I think in the past the keys were compared and if they matched, no extra programming took place...)

Hint: I had to configure this intermediate 'ram_flash' partition for the net core which did not exist in SDK v2.3.0. Maybe mcuboot tries to invalidate the copy in 'ram_flash'?

Here a memory dump of the last flash page of my secondary partition (0x88000-0xf7fff), I will cut the last 16 bytes bytes as they might be a confidential key):

0x000F7FC0 | FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF | ................
0x000F7FD0 | FF FF FF FF FF FF FF FF 03 FF FF FF FF FF FF FF | ................
0x000F7FE0 | FF FF FF FF FF FF FF FF 01 FF FF FF FF FF FF FF | ................

With mcuboot SDK v2.5.3, the application was switched. I did not test it with the net core though, but the slot was available always again after a net core update, so no blocked slots.

My conclusion: There are two problems in mcuboot SDK v2.9.0: First, there swap does not work anymore. Second: For the net core, the image in the secondary slot is not invalidated and triggers endless updates. The update procedure itself seems to work for both cores: The content in the primary slots is updated.

I hope this helps with the debugging.

Best regards,
Michael

Update: I checked it again with the net core update on SDK v2.5.3: I did only compare some bytes with the eye at 1K offset (compared 256 bytes at 0x89000, which is 1K into the secondary slot). The data did not change after the update, but in this case, the last secondary slot page (NOT the first page here!) was deleted again which prevented unwanted additional updates. So: Apparently no swap for the net core, but correct unflagging of the image via the last slot page