nRF5340 non-simultaneous netcore updates with external QSPI

[D Miller]

Development Environment: VS Code on Windows 10, SDK: 2.0.0, HW: nRF5340 Development Kit

I am trying to perform just a network core update on the nRF5340 DK based off of the smp_svr sample where the secondary slot for incoming updates is on an external QSPI. Using the Device Manager app, I can perform application core updates without issue, but when doing a network core update, MCUBoot seems to interpret the incoming image as an application core update no matter what. Which causes a second boot cycle as it swaps the images back.

uart:~$ *** Booting Zephyr OS build v3.0.99-ncs1 ***
I: Starting bootloader
I: Primary image: magic=good, swap_type=0x4, copy_done=0x1, image_ok=0x1
I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: test
I: Starting swap using move algorithm.
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v3.0.99-ncs1 ***
I: Starting bootloader
I: Primary image: magic=good, swap_type=0x2, copy_done=0x1, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: revert
I: Starting swap using move algorithm.
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v3.0.99-ncs1 ***


[00:00:00.017,791] <inf> smp_sample: build time: Jul 6 2022 15:24:46
[00:00:00.041,656] <inf> smp_bt_sample: Bluetooth initialized
[00:00:00.045,532] <inf> smp_bt_sample: Advertising successfully started

I have an overlay both top level and the child images containing:

// Choose the external flash to be used by partition manager (and allow MCUBoot to use external flash)
/ {
	chosen {
		nordic,pm-ext-flash = &mx25r64;
	};
};

And the following config files:
prj.conf:

## Bootloader
# Enable mcumgr.
CONFIG_MCUBOOT_IMAGE_VERSION="1.0.0+0"
CONFIG_MCUMGR=y
CONFIG_MCUBOOT_IMG_MANAGER=y
CONFIG_IMG_MANAGER=y
CONFIG_MCUMGR_CMD_IMG_MGMT=y
CONFIG_MCUMGR_CMD_OS_MGMT=y

# Some command handlers require a large stack.
CONFIG_MAIN_STACK_SIZE=2048

# Ensure an MCUboot-compatible binary is generated andcanp update the net core
CONFIG_BOOTLOADER_MCUBOOT=y

# Enable flash operations.
CONFIG_FLASH=y
CONFIG_FLASH_MAP=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_STREAM_FLASH_ERASE=y

# QSPI for External Flash
CONFIG_SPI=n
CONFIG_NORDIC_QSPI_NOR=y
CONFIG_NORDIC_QSPI_NOR_STACK_WRITE_BUFFER_SIZE=16
CONFIG_NORDIC_QSPI_NOR_FLASH_LAYOUT_PAGE_SIZE=4096

# Enable logging
CONFIG_LOG=y
CONFIG_MCUBOOT_UTIL_LOG_LEVEL_WRN=y

# Enable shell commands.
CONFIG_SHELL=y
CONFIG_MCUMGR_CMD_SHELL_MGMT=y

## BLE
# Allow for large Bluetooth data packets.
CONFIG_BT_L2CAP_TX_MTU=498
CONFIG_BT_BUF_ACL_RX_SIZE=502
CONFIG_BT_BUF_ACL_TX_SIZE=502
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251

# Enable the Bluetooth mcumgr transport (unauthenticated).
CONFIG_MCUMGR_SMP_BT=y
CONFIG_MCUMGR_SMP_BT_AUTHEN=n
CONFIG_MCUMGR_SMP_BT_CONN_PARAM_CONTROL=y

# Enable the Shell mcumgr transport.
CONFIG_MCUMGR_SMP_SHELL=y

# Enable the mcumgr Packet Reassembly feature over Bluetooth and its configuration dependencies.
# MCUmgr buffer size is optimized to fit one SMP packet divided into five Bluetooth Write Commands,
# transmitted with the maximum possible MTU value: 498 bytes.
CONFIG_MCUMGR_SMP_REASSEMBLY_BT=y
CONFIG_MCUMGR_BUF_SIZE=2475
CONFIG_OS_MGMT_MCUMGR_PARAMS=y
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=4096
CONFIG_DEBUG_THREAD_INFO=y
CONFIG_DEBUG_OPTIMIZATIONS=y

child_image/mcuboot.conf:

# MCUboot requires a large stack size, otherwise an MPU fault will occur
CONFIG_MAIN_STACK_SIZE=10240

# Net Core Update Support?
CONFIG_PCD_APP=y
CONFIG_FLASH=y
CONFIG_FLASH_MAP=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_STREAM_FLASH=y
CONFIG_STREAM_FLASH_ERASE=y
CONFIG_FLASH_SIMULATOR=y
CONFIG_FLASH_SIMULATOR_DOUBLE_WRITES=y
CONFIG_FLASH_SIMULATOR_STATS=n
CONFIG_MULTITHREADING=y

# This must be increased to accommodate the bigger images.
CONFIG_BOOT_MAX_IMG_SECTORS=256

# External QSPI secondary slot
CONFIG_NORDIC_QSPI_NOR=y
CONFIG_NORDIC_QSPI_NOR_FLASH_LAYOUT_PAGE_SIZE=4096
CONFIG_NORDIC_QSPI_NOR_STACK_WRITE_BUFFER_SIZE=16

# Various Things to Be small enough but still have logging prints
CONFIG_SIZE_OPTIMIZATIONS=y
CONFIG_DEBUG_OPTIMIZATIONS=n
CONFIG_LOG=y
CONFIG_LOG_MODE_MINIMAL=y # former CONFIG_MODE_MINIMAL
CONFIG_LOG_DEFAULT_LEVEL=0

---

Am I missing something here? I do notice when I debug to MCUBoot's loader.c line 982, where the vector table of the image is grabbed, the loaded data seems to be from the internal flash rather than the external memory. I have gone through the thread here nRF5340 Netcore firmware update using external flash - Nordic Q&A - Nordic DevZone - Nordic DevZone (nordicsemi.com) but my behavior isn't lining up and discussion is all on pre 2.0.0 SDKs. I tried UPGRADE_ONLY but it seems to do the same thing but not be able to revert back and so end up in a boot loop. As said in the title I have no need for simultaneous updates, I am just trying to perform core updates one at a time from the external QSPI.

Attached at the bottom is my minimal reproduction based off of smp_svr. Would greatly appreciate any assistance or pointers on this.

Regards,

D Miller

smp_svr_ext_qspi.zip

[Simon]

Check out this ticket  Busfault when MCUboot reads Network core secondary image header in external flash 

Try applying this patch https://github.com/nrfconnect/sdk-mcuboot/pull/197/commits/b628d6e238bb15eae06a9582fb19b00ea4990e13 and see if you're able to update the netcore using external flash.

Best regards,

Simon 

[D Miller]

I have applied the changes in that commit in the pull request and then tried the removal of the code block also suggested, both to no avail, the system still attempts all updates like they are for the application core.

I see the intention is to use nrf53_hooks.c to trigger a net core update but those hooks are only called in UPGRADE_ONLY configurations and I wish to retain app core reversion. It seems like the code block that was suggested to be removed in the other ticket is supposed to handle the "one core at a time" netcore updates but the flash simulator in RAM just isn't getting set up to pull the data from the external flash like it is in hooks.

So, I applied the patch like before which correctly runs the header check, but instead of removing the further down lines as the other thread suggested, I turned them into something closer to what is done in nrf53_hooks.c:

#if defined(CONFIG_SOC_NRF5340_CPUAPP) && defined(PM_CPUNET_B0N_ADDRESS)
        /* If the update is valid, and it targets the network core: perform the
         * update and indicate to the caller of this function that no update is
         * available
         */
        if (upgrade_valid && reset_addr > PM_CPUNET_B0N_ADDRESS) {
            BOOT_LOG_INF("Starting network core update");
            static const struct device *mock_flash_dev;
            void *mock_flash;
            size_t mock_size;

            BOOT_LOG_INF("Reading update image to flash simulator");
            mock_flash_dev = device_get_binding(PM_MCUBOOT_PRIMARY_DEV_NAME);
            if (mock_flash_dev == NULL) {
                swap_type = BOOT_SWAP_TYPE_NONE;
            }
            mock_flash = flash_simulator_get_memory(mock_flash_dev, &mock_size);
            int rc = flash_area_read(secondary_fa, 0U, mock_flash, hdr->ih_hdr_size + hdr->ih_img_size);
            if (rc != 0) {
                return BOOT_SWAP_TYPE_FAIL;
            }

            BOOT_LOG_INF("Starting PCD update to network core");
            hdr = (struct image_header *) mock_flash;
            uint32_t vtable_addr = (uint32_t)hdr + hdr->ih_hdr_size;
            uint32_t *vtable = (uint32_t *)(vtable_addr);
            rc = pcd_network_core_update(vtable, hdr->ih_img_size);
            if (rc != 0) {
                swap_type = BOOT_SWAP_TYPE_FAIL;
            } else {
                BOOT_LOG_INF("Done updating network core");
#if defined(MCUBOOT_SWAP_USING_SCRATCH) || defined(MCUBOOT_SWAP_USING_MOVE)
                /* swap_erase_trailer_sectors is undefined if upgrade only
                 * method is used. There is no need to erase sectors, because
                 * the image cannot be reverted.
                 */
                rc = swap_erase_trailer_sectors(state,
                    secondary_fa);
#endif
                swap_type = BOOT_SWAP_TYPE_NONE;
            }
        }
#endif /* CONFIG_SOC_NRF5340_CPUAPP */

I also added the needed includes at the top:

#if defined(CONFIG_SOC_NRF5340_CPUAPP) && defined(PM_CPUNET_B0N_ADDRESS)
#include <dfu/pcd.h> 
#include "flash_map_backend/flash_map_backend.h"
#include <zephyr/drivers/flash/flash_simulator.h>
#endif

No other changes to the SDK or project. With this, I can now perform net and app core updates one at a time from a single secondary slot on the external qspi in a way that still allows for reversion of the application core.

I don't feel like I'm doing everything correctly here, especially with the use of the flash simulator, but I can do my update scheme. I've uploaded my loader.c, otherwise it is v2.0.0 SDK.

loader.c

[Simon]

Hello D Miller,

Sorry for the late repsonse in this. A huge part of the support team is on summer vacation and that may cause some delayed responses.

I will test your sample Before the weekend/Monday and investigate the root cause of this.

Best regards,

Simon

[D Miller]

Hello Simon,

Has anyone had a chance to take a look at this matter?

[Simon]

Hello D Miller,

I got affected by the SAS strike in Norway and didn't get home until today. 

I'll look into this the next days

Simon

[Simon]

I've asked internally about your patch. I'll get back to you.

[Simon]

Did you remember to enable CONFIG_BOOT_IMAGE_ACCESS_HOOKS?

Then add CONFIG_BOOT_IMAGE_ACCESS_HOOKS_FILE to your prj.conf file and set it to your custom hook file. That is better than trying to modify loader.c. By default the hook file modules/mcuboot/hooks/nrf53_hooks.c is used: https://github.com/nrfconnect/sdk-nrf/blob/v2.0.0/modules/mcuboot/boot/zephyr/Kconfig#L95.

Best regards,

Simon

[D Miller]

(moved my reply up here)

Hello,

Yes, I did try with and without CONFIG_BOOT_IMAGE_ACCESS_HOOKS. This required I do the whole CONFIG_UPDATEABLE_IMAGE_NUMBER=2 and CONFIG_BOOT_UPGRADE_ONLY=y to get running (the depends for the KConfig on that flag seem to confirm) as there is code in the hooks that assume >1 slots on the secondary and the partitions only seem to be correctly generated with that configuration. But we are not wanting upgrade only updates.

The net core updates in nrf53_hooks.c are hooked to `boot_copy_region_post_hook` and `boot_serial_uploaded_hook`. The first is called in loader.c `boot_copy_image` which is only used in  `MCUBOOT_OVERWRITE_ONLY` configurations. And the latter is used for serial recovery.

In the github patch conversation ([nrf noup] loader: Fix reading reset addr to support ext flash by chrta · Pull Request #197 · nrfconnect/sdk-mcuboot · GitHub), there seems to be some discussion relating to this specifically where hooks do not cover the functionality we are seeking.

[Simon]

D Miller said:

Yes, I did try with and without CONFIG_BOOT_IMAGE_ACCESS_HOOKS. This required I do the whole CONFIG_UPDATEABLE_IMAGE_NUMBER=2 and CONFIG_BOOT_UPGRADE_ONLY=y to get running (the depends for the KConfig on that flag seem to confirm) as there is code in the hooks that assume >1 slots on the secondary and the partitions only seem to be correctly generated with that configuration. But we are not wanting upgrade only updates.

I was able to enable CONFIG_BOOT_IMAGE_ACCESS_HOOKS, I simply set CONFIG_BOOT_IMAGE_ACCESS_HOOKS=y inside smp_svr\child_image\mcuboot.conf (should be set in mcuboot sample, not main application). However, I did get some errors, but that is not related to the Kconfigs, but the nrf53_hooks.c file. The nrf53_hooks.c file tries to access the primary_1 partition, which is not present in your case since you're not doing multi-image updates. I then modified nrf\modules\mcuboot\hooks\nrf53_hooks.c like shown below and was able to build the sample:

diff --git a/modules/mcuboot/hooks/nrf53_hooks.c b/modules/mcuboot/hooks/nrf53_hooks.c
index ebc61666c..fb7213f3c 100644
--- a/modules/mcuboot/hooks/nrf53_hooks.c
+++ b/modules/mcuboot/hooks/nrf53_hooks.c
@@ -24,7 +24,7 @@ int boot_read_image_header_hook(int img_index, int slot,
 	if (img_index == 1 && slot == 0) {
 		img_head->ih_magic = IMAGE_MAGIC;
 		img_head->ih_hdr_size = PM_MCUBOOT_PAD_SIZE;
-		img_head->ih_load_addr = PM_MCUBOOT_PRIMARY_1_ADDRESS;
+		img_head->ih_load_addr = 0x00;//PM_MCUBOOT_PRIMARY_1_ADDRESS;
 		img_head->ih_img_size = PM_CPUNET_APP_SIZE;
 		img_head->ih_flags = 0;
 		img_head->ih_ver.iv_major = 0;
@@ -81,10 +81,10 @@ int network_core_update(bool wait)
 	void *mock_flash;
 	size_t mock_size;
 
-	mock_flash_dev = device_get_binding(PM_MCUBOOT_PRIMARY_1_DEV_NAME);
+	/*mock_flash_dev = device_get_binding(PM_MCUBOOT_PRIMARY_1_DEV_NAME);
 	if (mock_flash_dev == NULL) {
 		return -ENODEV;
-	}
+	}*/
 
 	mock_flash = flash_simulator_get_memory(NULL, &mock_size);
 	hdr = (struct image_header *) mock_flash;

Then I could see that CONFIG_BOOT_IMAGE_ACCESS_HOOKS was set to y and that CONFIG_BOOT_IMAGE_ACCESS_HOOKS_FILE was equal to "nrf/modules/mcuboot/hooks/nrf53_hooks.c" in smp_svr\build\mcuboot\zephyr\.config.

Be aware that the above file is not meant to work, I made it just so the code would compile, and it can be a starting point for creating your own custom hook file.

Of course the proper way would be to add your own file to nrf/modules/mcuboot/hooks/ and set CONFIG_BOOT_IMAGE_ACCESS_HOOKS_FILE (in smp_svr\child_image\mcuboot.conf) accordingly.

D Miller said:

there seems to be some discussion relating to this specifically where hooks do not cover the functionality we are seeking.

Yes, that is true. The nrf53_hooks.c file is tailored towards multi-image, so it needs to be modified to your need, non simultaneous netcore/appcore updates.

I did try to play around and add the changes you made to loader.c into the hooks file, but wasn't able to make it work (the hooks stuff are new to me and I don't have complete control of how it works). I will see if I can get it to work the next days. If you succeed if would be nice if you could share the solution, as I'm sure many other customer will benefit from it as well.

Best regards,

Simon

[Simon]

Hello Miller,

What is the current state on this? Were you able to implement the changes to a hook file? Unfortunately, I haven't gotten time myself to do it yet. I will be gone for the next two weeks, and my colleague Sigurd will watch the case in the meanwhile (he will be gone until 2 of Aug.). 

Best regards,

Simon