Using external flash to store updates for MCUboot & App on nRF5340 which the existing MCUBoot will apply

[blake-bit]

How do I store updated images for both MCUboot and the main App in external flash on the nRF5340 such that the existing MCUboot will apply the updates to the internal flash and execute them? Does the existing MCUboot codebase support this?

I have a nRF5340 with a MX25R6435F flash chip connected via QSPI. It is also connected to an LTE modem (nRF9160) via another SPI bus. The app on the 5340 will download any needed update data from the internet with custom code interfacing to the modem and write the needed update data to the external flash (this functionality is not part of my question here, assume the download and write to external flash of arbitrary data is working).

My main questions are:

1. In what format do the updated MCUboot and App images need to be stored in external flash? (address offsets, etc.)
2. Are any changes required to be made to the MCUboot codebase to support this?
3. Are any additional configuration changes needed other than I what I have listed below?

I have configured my build system to use the B0 Immutable bootloader as the first-stage bootloader and MCUboot as an upgradeable second-stage bootloader. For the flash layout, I believe I have correctly configured the build for a single app image in internal flash, dual MCUboot image slots in internal flash, and the "mcuboot_secondary" slot in external flash. That all appears to be configured, build, and run correctly.

My goal for functionality is:

1. The old App runs, downloads the updated image(s) to external flash, and triggers a reboot.
2. The old MCUboot checks the external flash, detects that an update is available, and begins to copy them to internal flash...
3. The updated MCUboot code is overwritten to the "other" MCUboot slot (the one not currently running).
4. The updated App code is overwritten to the single App slot which had the old App code.
5. The old MCUboot triggers a reboot.
6. B0 detects the updated MCUboot in the "other" slot and executes it.
7. The updated MCUboot runs and executes the updated App code.

I have read over some of the guides for DFU and bootloaders, but they seem to only apply to BLE updates not using external flash, or to LTE modem updates, or update MCUboot only but not the app; and aren't clear to me for how to achieve my goals. For example:

Device Firmware Upgrade module

Device Firmware Upgrade

Serial Recovery 

Adding an upgradable bootloader

I have tried to debug/follow the code of context_boot_go() in workspace/bootloader/mcuboot/boot/bootutil/src/loader.c:1780 to better understand how MCUboot works and its upgrade process, but it's not clear to me how it needs to read from the external flash.

My configuration is as follows

Using NCS 1.6.0

Flash layout from west tool:

  external_flash (0x800000 - 8192kB):
+------------------------------------------------+
| 0x0: mcuboot_secondary (0x100000 - 1024kB)     |
| 0x100000: littlefs_storage (0x700000 - 7168kB) |
| 0x800000: external_flash (0x0 - 0B)            |
+------------------------------------------------+

  flash_primary (0x100000 - 1024kB):
+--------------------------------------------------+
+---0x0: b0_container (0x8000 - 32kB)--------------+
| 0x0: b0 (0x8000 - 32kB)                          |
+---0x8000: s0 (0x10000 - 64kB)--------------------+
| 0x8000: s0_pad (0x200 - 512B)                    |
+---0x8200: s0_image (0xfe00 - 63kB)---------------+
| 0x8200: mcuboot (0xfe00 - 63kB)                  |
+---0x18000: s1 (0x10000 - 64kB)-------------------+
| 0x18000: s1_pad (0x200 - 512B)                   |
| 0x18200: s1_image (0xfe00 - 63kB)                |
+---0x28000: mcuboot_primary (0xd2000 - 840kB)-----+
| 0x28000: mcuboot_pad (0x200 - 512B)              |
+---0x28200: mcuboot_primary_app (0xd1e00 - 839kB)-+
+---0x28200: spm_app (0xd1e00 - 839kB)-------------+
| 0x28200: app (0xd1e00 - 839kB)                   |
+--------------------------------------------------+
| 0xfa000: factory_cfg_1 (0x1000 - 4kB)            |
| 0xfb000: user_cfg_1 (0x1000 - 4kB)               |
| 0xfc000: developer_cfg_1 (0x1000 - 4kB)          |
| 0xfd000: factory_cfg_2 (0x1000 - 4kB)            |
| 0xfe000: user_cfg_2 (0x1000 - 4kB)               |
| 0xff000: developer_cfg_2 (0x1000 - 4kB)          |
+--------------------------------------------------+

  otp (0x2fc - 764B):
+------------------------------------+
| 0xff8100: provision (0x280 - 640B) |
| 0xff8380: otp (0x7c - 124B)        |
+------------------------------------+

  sram_primary (0x80000 - 512kB):
+--------------------------------------------+
| 0x20000000: sram_primary (0x7e000 - 504kB) |
| 0x2007e000: pcd_sram (0x2000 - 8kB)        |
+--------------------------------------------+

prj.conf (relevant sections)

# FLASH
CONFIG_FLASH=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FLASH_MAP=y
CONFIG_FLASH_SHELL=y
CONFIG_SETTINGS=y
CONFIG_MPU_ALLOW_FLASH_WRITE=y

# Secure Immutable Bootloader (B0)
CONFIG_SECURE_BOOT=y

# MCUBOOT
CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_IMG_MANAGER=y
CONFIG_MCUBOOT_IMG_MANAGER=y
CONFIG_IMG_ERASE_PROGRESSIVELY=y
CONFIG_BUILD_S1_VARIANT=

mcuboot.conf (applied to the MCUboot image in CMake system)

CONFIG_NORDIC_QSPI_NOR=y
CONFIG_NORDIC_QSPI_NOR_FLASH_LAYOUT_PAGE_SIZE=4096
CONFIG_NORDIC_QSPI_NOR_STACK_WRITE_BUFFER_SIZE=4
CONFIG_MULTITHREADING=y
CONFIG_BOOT_MAX_IMG_SECTORS=256
CONFIG_PM_EXTERNAL_FLASH=y
CONFIG_PM_EXTERNAL_FLASH_DEV_NAME="MX25R64"
CONFIG_PM_EXTERNAL_FLASH_SIZE=0x800000
CONFIG_PM_EXTERNAL_FLASH_BASE=0x000000

CONFIG_BUILD_S1_VARIANT=y

pm_static.yml

# Config/Settings Partition:
factory_cfg_1:
  address: 0xfa000
  end_address: 0xfb000
  placement:
    before:
    - end
  region: flash_primary
  size: 0x1000

user_cfg_1:
  address: 0xfb000
  end_address: 0xfc000
  placement:
    before:
    - end
  region: flash_primary
  size: 0x1000

developer_cfg_1:
  address: 0xfc000
  end_address: 0xfd000
  placement:
    before:
    - end
  region: flash_primary
  size: 0x1000

#Backup Config Partitions
factory_cfg_2:
  address: 0xfd000
  end_address: 0xfe000
  placement:
    before:
    - end
  region: flash_primary
  size: 0x1000

user_cfg_2:
  address: 0xfe000
  end_address: 0xff000
  placement:
    before:
    - end
  region: flash_primary
  size: 0x1000

developer_cfg_2:
  address: 0xff000
  end_address: 0x100000
  placement:
    before:
    - end
  region: flash_primary
  size: 0x1000


# External Flash Partitions
mcuboot_secondary:
  address: 0x0
  device: MX25R64
  end_address: 0x100000
  region: external_flash
  size: 0x100000

littlefs_storage:
  address: 0x100000
  device: MX25R64
  end_address: 0x800000
  region: external_flash
  size: 0x700000

normal boot log

*** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***
Attempting to boot slot 0.
Attempting to boot from address 0x8200.
Verifying signature against key 0.
Hash: 0xfa...2e
Firmware signature verified.
Firmware version 1
*** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: none
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: none
I: Bootloader chainload address offset: 0x28000
�: Jumping to the first image slot

[00:00:00.020,080] <dbg> fs.fs_register: fs register 1: 0
*** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***
nrf5340:~$

With some debug prints in MCUboot, I was able to see that MCUboot does at least open the external flash partition for access

*** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***
Attempting to boot slot 0.
Attempting to boot from address 0x8200.
Verifying signature against key 0.
Hash: 0xfa...2e
Firmware signature verified.
Firmware version 1
Setting monotonic counter (version: 1, slot: 0)
*** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***
I: Starting bootloader
Flash open 11 -NRF_FLASH_DRV_NAME
Flash open 2 -MX25R64
Flash open 11 -NRF_FLASH_DRV_NAME
Flash open 2 -MX25R64
Flash open 11 -NRF_FLASH_DRV_NAME
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
Flash open 2 -MX25R64
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
Flash open 11 -NRF_FLASH_DRV_NAME
Flash open 2 -MX25R64
Flash open 11 -NRF_FLASH_DRV_NAME
Flash open 2 -MX25R64
I: Swap type: none
Flash open 7 -NRF_FLASH_DRV_NAME
Flash open 2 -MX25R64
Flash open 7 -NRF_FLASH_DRV_NAME
Flash open 2 -MX25R64
Flash open 7 -NRF_FLASH_DRV_NAME
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
Flash open 2 -MX25R64
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
Flash open 7 -NRF_FLASH_DRV_NAME
Flash open 2 -MX25R64
Flash open 7 -NRF_FLASH_DRV_NAME
Flash open 2 -MX25R64
I: Swap type: none
Flash open 11 -NRF_FLASH_DRV_NAME
Flash open 11 -NRF_FLASH_DRV_NAME
I: Bootloader chainload address offset: 0x28000
�: Jumping to the first image slot

[Håkon Alseth]

Hi,

 

How do I store updated images for both MCUboot and the main App in external flash on the nRF5340 such that the existing MCUboot will apply the updates to the internal flash and execute them? Does the existing MCUboot codebase support this?

 We support using an external flash as a secondary slot, but you cannot boot directly from this external flash (ie. it needs to verify, copy, then run). In your current layout, the secondary slot should be 0xd2000 bytes.

In what format do the updated MCUboot and App images need to be stored in external flash? (address offsets, etc.)

This is up to you, and how you define your partition layout (pm_static.yml).

There is a restriction in the mcuboot_secondary size, which must match the size of mcuboot_primary (with a slack of one flash size)

• Are any changes required to be made to the MCUboot codebase to support this?

 You will need to configure mcuboot to understand that it should look for a secondary partition on a external flash device.

• Are any additional configuration changes needed other than I what I have listed below?

 Here's a simplified example using hello_world for nrf5340:

hello_world_mcuboot_qspi_nrf53.zip

Could you try this and see if that works on your end? 

This does not include b0, but the principle is quite similar if you add that "on-top"

 

Kind regards,

Håkon

[blake-bit]

Hi Håkon,

Thanks for your reply, however I'm still confused here.

I was previously able to use the serial flash tool and configure/build/flash MCUboot to accept an application update (app_update.bin) over the serial port using the mcumgr tool. This worked fine and I was able to update the app image as expected via serial port after holding down a button to put MCUboot into serial update mode. However, as stated in my original post, this is not my use case nor my goal; I am looking to store a firmware update in the external flash to be applied by MCUboot upon the next reset, no serial or bluetooth connection required.

I understand I cannot not boot directly from external flash, and that is not my goal.

From a reset I expect the following to run/execute:

1. "B0" bootloader stored in internal flash at 0x0000, which will then launch...
2. "MCUboot" in slot 0 or slot 1, stored in internal flash at 0x8200 (slot0) or 0x18200 (slot1), which will then launch...
3. "App" stored in internal flash at 0x28200.

At some point when the App is running it will get a newer version of the MCUboot and App code and write them to the external flash chip in the "mcuboot_secondary" partition starting at 0x0 of the external flash map (at this point the internal flash in unmodified). Upon the next reset the old/existing MCUboot will copy the updated images from external flash to internal flash, reboot and then the new/upgraded MCUboot & App will run.

My question is what specific bytes/bits and binary image files from the build folder do I need to write to external flash and at what addresses; are there flag bits in the images that also need to be set/cleared?
This would be based on what the existing MCUboot functionality in NCS 1.6.0 is looking for so that MCUboot can detect, verify, and copy the updates from external flash to internal flash. I'm happy to modify MCUboot code if needed to achieve this functionality; I just need to know where in the code to make the changes.

Some questions and file examples, from my build folder (build/zephyr/) :

• "app_update.bin" which I believe contains the vector table, app code, and security signature. Should this be written directly as is to external flash at 0x0, or at 0x28200 which is the same offset used in the internal flash map, or somewhere else. Or should another file like "app_signed.hex" be placed in external flash somewhere instead?
  
  
• "signed_by_b0_s1_image.bin" which I believe contains the vector table, MCUBoot code built to run from slot 1, and security signature. Should this be written to external flash at 0x0, or at 0x18000 which is the slot 1 MCUBoot image offset used for internal flash?
  
  
• Where would the "signed_by_b0_s0_image.bin" MCUboot image go, assuming both (s0 and s1 images) are downloaded to external flash and so that the existing MCUboot can choose the correct one to copy into the correct location in internal flash?
  
  
• With these files in mind, where would all three files: s0-MCUboot, s1-MCUboot, and App images be written to in external flash so that the old/existing MCUboot will copy them from external flash to internal flash? They all can't be written to address 0x0000 in external flash at the same time.
  
  
• Do any of the "image trailers" or such need to be modified to tell the existing MCUboot that these are upgrade images?
  
• I see you recommended that the "mcuboot_secondary" partition in external flash be set to a size of 0xD2000 bytes (840kB) which is the same size as the "mcuboot_primary" partition in internal flash which holds only the App image (despite the confusing container name).
  
  However, since that partition only holds the vector table and App image, it does not include the two MCUboot slots of size 0xfe00 bytes (63kB) each. If the App code image size were large and the size the partition, then the updates to MCUboot would not fit inside that 0xD2000 sized external partition together with the large App code.
  
  Should the "mcuboot_secondary" partition be large enough to accommodate two MCUboot slots (2x 63kb) and the App image slot (840kb)

Thank you for your help!

Blake

[Håkon Alseth]

Hi Blake,

 

blake-bit said:

I was previously able to use the serial flash tool and configure/build/flash MCUboot to accept an application update (app_update.bin) over the serial port using the mcumgr tool. This worked fine and I was able to update the app image as expected via serial port after holding down a button to put MCUboot into serial update mode. However, as stated in my original post, this is not my use case nor my goal; I am looking to store a firmware update in the external flash to be applied by MCUboot upon the next reset, no serial or bluetooth connection required.

 I understand.

 

blake-bit said:

My question is what specific bytes/bits and binary image files from the build folder do I need to write to external flash and at what addresses; are there flag bits in the images that also need to be set/cleared?
This would be based on what the existing MCUboot functionality in NCS 1.6.0 is looking for so that MCUboot can detect, verify, and copy the updates from external flash to internal flash. I'm happy to modify MCUboot code if needed to achieve this functionality; I just need to know where in the code to make the changes.

You can do this in any way that you want, side-by-side while your current application is running, in mcuboot serial mode, etc. MCUboot does not really care how the image gets placed in the secondary slot, it will detect it upon reset and verify/test boot the image.

when the image is "test booted", it will be applied as a primary if the new image calls:

boot_write_img_confirmed();

 

So, it sounds like you want to receive the update image in your application, and place this in the secondary slot.

You can do this using the dfu target library:

https://developer.nordicsemi.com/nRF_Connect_SDK/doc/1.6.1/nrf/include/dfu/dfu_target.html

 

 

blake-bit said:

"app_update.bin"

 This is the application (secure and non-secure), excluding bootloaders etc., and its signed.

blake-bit said:

Should this be written directly as is to external flash at 0x0, or at 0x28200 which is the same offset used in the internal flash map, or somewhere else. Or should another file like "app_signed.hex" be placed in external flash somewhere instead?

This should be fed to mcuboot's secondary slot, not programmed directly. app_test_update.hex can normally be flashed directly to verify that it works.

 

blake-bit said:

"signed_by_b0_s1_image.bin"

 This is the mcuboot (ie. second stage BL since B0 is also enabled), placed in slot 1:

https://developer.nordicsemi.com/nRF_Connect_SDK/doc/1.6.1/nrf/ug_bootloader.html#

  

 

blake-bit said:

Where would the "signed_by_b0_s0_image.bin" MCUboot image go, assuming both (s0 and s1 images) are downloaded to external flash and so that the existing MCUboot can choose the correct one to copy into the correct location in internal flash?

 that should be placed in the s0 address offset.

Note that all address offsets that you mention are unique to your build, unless they are statically defined. You can statically define them in pm_static.yml (which can be a direct copy of build-dir/partitions.yml)

 

blake-bit said:

With these files in mind, where would all three files: s0-MCUboot, s1-MCUboot, and App images be written to in external flash so that the old/existing MCUboot will copy them from external flash to internal flash? They all can't be written to address 0x0000 in external flash at the same time.

 The bootloader stages are booted in order, so if mcuboot + application is updated simultaneously, they will be handled in the order that they boot.

blake-bit said:

Do any of the "image trailers" or such need to be modified to tell the existing MCUboot that these are upgrade images?

No, but; this depends on how hardened your configuration is. if you enable downgrade protection, you must append the version numbering.

blake-bit said:

I see you recommended that the "mcuboot_secondary" partition in external flash be set to a size of 0xD2000 bytes (840kB) which is the same size as the "mcuboot_primary" partition in internal flash which holds only the App image (despite the confusing container name).

However, since that partition only holds the vector table and App image, it does not include the two MCUboot slots of size 0xfe00 bytes (63kB) each. If the App code image size were large and the size the partition, then the updates to MCUboot would not fit inside that 0xD2000 sized external partition together with the large App code.

Should the "mcuboot_secondary" partition be large enough to accommodate two MCUboot slots (2x 63kb) and the App image slot (840kb)

As mentioned, my example is simpler than your use-case. It has only a couple of sections/partitions. Therefore, it is important that you adjust the pm_static.yml file to fit your partition layout. In your use-case, the mcuboot_primary partition will be smaller than in my example, as you have the b0 bootloader and S0+S1 (ie. the two mcuboot slots) and maybe some other parts.

 

Please note that all update-images of your firmware must share the equal pm_static.yml as the original image. This is to ensure that your bootloader(s) know where to place each image, and that the updated images are compiled to start/run from that specific address.

 

Kind regards,

Håkon