Update Firmware (Bootloader and Application) from external flash

[Talisca]

I am using the nRF52840-dk with the nRF Connect SDK v2.0.0 and for the development itself I use Visual Studio Code.

My Goal: Both the bootloader and the actual application should be upgradeable, with firmware updates stored on an external flash. Supposedly this should be possible.
The firmware is stored in the external flash via specially developed methods. Afterwards the bootloader should detect these firmware updates in the external flash and copy/swap them to the internal flash.

According to the documentation I need the nRF Secure Immutable Bootloader and MCUboot as a second stage bootloader.
This actual bootloader chain works fine so far even with my own private keys.

Currently I am getting the following output with my configuration:

*** Booting Zephyr OS build v3.0.99-ncs1  ***
Attempting to boot slot 0.
Attempting to boot from address 0x9200.
Verifying signature against key 0.
Hash: 0xea...6a
Firmware signature verified.
Firmware version 1
Setting monotonic counter (version: 1, slot: 0)
*** Booting Zephyr OS build v3.0.99-ncs1  ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: none
I: Primary image: magic=bad, swap_type=0x2, copy_done=0x2, image_ok=0x2
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: none
I: Bootloader chainload address offset: 0x23000
▒*** Booting Zephyr OS build v3.0.99-ncs1  ***
[00:0Bluetooth initialized

I do not understand the output of MCUboot at all. Why do two passes take place? Why is magic from the primary image suddenly bad on the second pass, but loads the firmware anyway?

I can change the version number of MCUboot. This is also shown correctly by the first bootloader.

But that's it. I can't get it to recognize the firmware updates.
MCUboot doesn't seem to make any attempt at all to read anything from the external flash. Well I don't know. The bootloader doesn't give any further relevant information, even if I adjust the log level.

For now, I'd be happy to at least have MCUboot upgradeable.

The following already works:
The firmware update is in the right position in the external flash. For testing purposes at offset 0x0.
As the firmware update for MCUboot I use the file "signed_by_b0_s0_image.bin" from the "build/zephyr" directory. If I open this binary
I see the first 0x200 bytes with some meta information. See here:

$ head --bytes=$((16#200)) signed_by_b0_s0_image.bin | xxd
00000000: c05d 0020 5dbc 0000 1907 0100 31bc 0000  .]. ].......1...
00000010: 31bc 0000 31bc 0000 31bc 0000 0000 0000  1...1...1.......
00000020: 0000 0000 0000 0000 0000 0000 69b7 0000  ............i...
00000030: 31bc 0000 0000 0000 15b7 0000 31bc 0000  1...........1...
00000040: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
00000050: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
00000060: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
00000070: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
00000080: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
00000090: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
000000a0: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
000000b0: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
000000c0: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
000000d0: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
000000e0: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
000000f0: 19b8 0000 19b8 0000 19b8 0000 19b8 0000  ................
00000100: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000110: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000120: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000130: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000140: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000150: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000160: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000170: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000180: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000190: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................

My expectation would be: Since it is signed_by_b0_s0_image.bin, which is (presumably) a valid MCUboot firmware, the bootloader would have to output some message and validate the update or do something else. But I don't see anything of that.

Below I list the most important configurations I use. These are mainly for the current development.
And yes, due to days of desperate attempts, the configuration files are now very messy.

pm_static.yml (originally taken from a sample project - but couldn't even be built in the first place)

external_flash:
  address: 0x000000
  region: external_flash
  size: 0x0
  device: MX25R64

mcuboot_secondary:
  address: 0x00000
  device: MX25R64
  region: external_flash
  size: 0xdb000

rest_storage:
  address: 0xdb000
  size: 0x725000
  device: MX25R64
  region: external_flash

nrf52840dk_nrf52840.overlay:

/ {
	chosen {
		nordic,pm-ext-flash = &mx25r64;
	};
};

The generated partitions.yml:

EMPTY_0:
  address: 0x15200
  end_address: 0x16000
  placement:
    before:
    - s1_pad
  region: flash_primary
  size: 0xe00
EMPTY_1:
  address: 0x22200
  end_address: 0x23000
  placement:
    before:
    - mcuboot_pad
  region: flash_primary
  size: 0xe00
app:
  address: 0x23200
  end_address: 0xfe000
  region: flash_primary
  size: 0xdae00
app_image:
  address: 0x23200
  end_address: 0xfe000
  orig_span: &id001
  - app
  region: flash_primary
  size: 0xdae00
  span: *id001
b0:
  address: 0x0
  end_address: 0x8000
  placement:
    after:
    - start
  region: flash_primary
  size: 0x8000
b0_container:
  address: 0x0
  end_address: 0x9000
  orig_span: &id002
  - b0
  - provision
  region: flash_primary
  size: 0x9000
  span: *id002
external_flash:
  address: 0x0
  device: MX25R64
  end_address: 0x0
  region: external_flash
  size: 0x0
mcuboot:
  address: 0x9200
  end_address: 0x15200
  placement:
    before:
    - mcuboot_primary
  region: flash_primary
  sharers: 0x1
  size: 0xc000
mcuboot_pad:
  address: 0x23000
  end_address: 0x23200
  placement:
    align:
      start: 0x1000
    before:
    - mcuboot_primary_app
  region: flash_primary
  sharers: 0x2
  size: 0x200
mcuboot_primary:
  address: 0x23000
  end_address: 0xfe000
  orig_span: &id003
  - mcuboot_pad
  - app
  region: flash_primary
  size: 0xdb000
  span: *id003
mcuboot_primary_app:
  address: 0x23200
  end_address: 0xfe000
  orig_span: &id004
  - app
  region: flash_primary
  size: 0xdae00
  span: *id004
mcuboot_secondary:
  address: 0x0
  device: MX25R64
  end_address: 0xdb000
  region: external_flash
  size: 0xdb000
provision:
  address: 0x8000
  end_address: 0x9000
  placement:
    after:
    - b0
    align:
      start: 0x1000
  region: flash_primary
  size: 0x1000
rest_storage:
  address: 0xdb000
  device: MX25R64
  end_address: 0x800000
  region: external_flash
  size: 0x725000
s0:
  address: 0x9000
  end_address: 0x15200
  orig_span: &id005
  - mcuboot
  - s0_pad
  region: flash_primary
  size: 0xc200
  span: *id005
s0_image:
  address: 0x9200
  end_address: 0x15200
  orig_span: &id006
  - mcuboot
  region: flash_primary
  size: 0xc000
  span: *id006
s0_pad:
  address: 0x9000
  end_address: 0x9200
  placement:
    after:
    - b0_container
    align:
      start: 0x1000
  region: flash_primary
  share_size:
  - mcuboot_pad
  size: 0x200
s1:
  address: 0x16000
  end_address: 0x22200
  orig_span: &id007
  - s1_pad
  - s1_image
  region: flash_primary
  size: 0xc200
  span: *id007
s1_image:
  address: 0x16200
  end_address: 0x22200
  placement:
    after:
    - s1_pad
    - s0
  region: flash_primary
  share_size:
  - mcuboot
  size: 0xc000
s1_pad:
  address: 0x16000
  end_address: 0x16200
  placement:
    after:
    - s0
    align:
      start: 0x1000
  region: flash_primary
  share_size:
  - mcuboot_pad
  size: 0x200
settings_storage:
  address: 0xfe000
  end_address: 0x100000
  placement:
    before:
    - end
  region: flash_primary
  size: 0x2000
sram_primary:
  address: 0x20000000
  end_address: 0x20040000
  region: sram_primary
  size: 0x40000

prj.conf

CONFIG_NEWLIB_LIBC=y

CONFIG_CPLUSPLUS=y
CONFIG_STD_CPP2A=y
CONFIG_LIB_CPLUSPLUS=y
CONFIG_NEWLIB_LIBC_NANO=y
CONFIG_EXCEPTIONS=y

CONFIG_SENSOR=y
CONFIG_TEMP_NRF5=y
CONFIG_NRFX_TEMP=y

CONFIG_STDOUT_CONSOLE=y
CONFIG_FLASH=y
CONFIG_I2C=n
CONFIG_SPI=y
CONFIG_NORDIC_QSPI_NOR=y

CONFIG_SPI_SLAVE=n
CONFIG_SPI_NRFX=y
CONFIG_SPI_ASYNC=y
CONFIG_NRFX_SPIS0=y
CONFIG_GPIO=y
CONFIG_SPI_0_NRF_ORC=0x80

CONFIG_THREAD_STACK_INFO=y
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=4096
CONFIG_MAIN_STACK_SIZE=4096
CONFIG_BT_RX_STACK_SIZE=4096

CONFIG_BT_GATT_AUTO_SEC_REQ=n
CONFIG_BT=y
CONFIG_BT_SMP=y
CONFIG_BT_SIGNING=y
CONFIG_BT_PERIPHERAL=y
CONFIG_BT_EXT_ADV=y
CONFIG_BT_PER_ADV=y
CONFIG_BT_DEBUG_LOG=y
CONFIG_BT_DEBUG_SMP=y
CONFIG_BT_DIS=y
CONFIG_BT_ATT_PREPARE_COUNT=5
CONFIG_BT_PRIVACY=n
CONFIG_BT_DEVICE_NAME="MyName"
CONFIG_BT_DEVICE_APPEARANCE=833
CONFIG_BT_DEVICE_NAME_DYNAMIC=y
CONFIG_BT_DEVICE_NAME_MAX=65
CONFIG_BT_SMP_ENFORCE_MITM=y
CONFIG_BT_GATT_DYNAMIC_DB=y
CONFIG_BT_BUF_ACL_RX_SIZE=517
CONFIG_BT_L2CAP_TX_MTU=517
CONFIG_BT_BUF_ACL_TX_SIZE=517
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251

CONFIG_BT_KEYS_OVERWRITE_OLDEST=y
CONFIG_BT_SETTINGS=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FLASH_MAP=y
CONFIG_NVS=y
CONFIG_SETTINGS=y

CONFIG_LZ4=y
CONFIG_HEAP_MEM_POOL_SIZE=16384

CONFIG_SMF=y
CONFIG_DEBUG_THREAD_INFO=y
CONFIG_DEBUG_OPTIMIZATIONS=y
CONFIG_DEBUG_INFO=y

CONFIG_BT_EXT_ADV_MAX_ADV_SET=2
CONFIG_BT_MAX_CONN=10

CONFIG_ASSERT=n

CONFIG_NORDIC_QSPI_NOR=y
CONFIG_NORDIC_QSPI_NOR_FLASH_LAYOUT_PAGE_SIZE=4096
CONFIG_NORDIC_QSPI_NOR_STACK_WRITE_BUFFER_SIZE=16

CONFIG_STDOUT_CONSOLE=y

CONFIG_FPU=y
CONFIG_FPU_SHARING=y

CONFIG_DEBUG_THREAD_INFO=y
CONFIG_DEBUG_OPTIMIZATIONS=y
CONFIG_DEBUG_INFO=y

CONFIG_NRFX_NVMC=y
CONFIG_FW_INFO=y
CONFIG_FW_INFO_OFFSET=0x200

CONFIG_SECURE_BOOT=y
CONFIG_DISABLE_FLASH_PATCH=y

CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_MCUBOOT_IMAGE_VERSION="1.1.1+1"
CONFIG_SB_SIGNING_KEY_FILE="private.pem"
CONFIG_SB_SIGNING_PUBLIC_KEY="public.pem"

CONFIG_TIMING_FUNCTIONS=y
CONFIG_DISABLE_FLASH_PATCH=y

CONFIG_BOOT_BANNER=y

CONFIG_IMG_MANAGER=y
CONFIG_MCUBOOT_IMG_MANAGER=y

CONFIG_UPDATEABLE_IMAGE_NUMBER=1
CONFIG_PM_EXTERNAL_FLASH_BASE=0x0
CONFIG_PM_EXTERNAL_FLASH_MCUBOOT_SECONDARY=y

CONFIG_DFU_TARGET_MCUBOOT=y

child_image/b0.conf:

CONFIG_IS_SECURE_BOOTLOADER=y
CONFIG_MULTITHREADING=n
CONFIG_GPIO=n
CONFIG_ARM_MPU=n
CONFIG_PM=n
CONFIG_TICKLESS_KERNEL=n
CONFIG_ERRNO=n
CONFIG_FPROTECT=y
CONFIG_SECURE_BOOT_CRYPTO=y
CONFIG_SECURE_BOOT_DEBUG=y
CONFIG_SECURE_BOOT_VALIDATION=y
CONFIG_SECURE_BOOT_STORAGE=y
CONFIG_BL_ROT_VERIFY_EXT_API_ENABLED=y
CONFIG_BL_SHA256_EXT_API_ENABLED=y
CONFIG_BL_SECP256R1_EXT_API_ENABLED=y
CONFIG_BL_VALIDATE_FW_EXT_API_ENABLED=y
CONFIG_EXT_API_PROVIDE_EXT_API_ENABLED=y
CONFIG_NRFX_NVMC=y
CONFIG_MAIN_STACK_SIZE=2048
CONFIG_TIMEOUT_64BIT=n

CONFIG_ASSERT=n

CONFIG_NRF_RTC_TIMER=n

CONFIG_FW_INFO=y

CONFIG_PM_EXTERNAL_FLASH_BASE=0x0

child_image/mcuboot.conf:

CONFIG_PM=n

CONFIG_MAIN_STACK_SIZE=10240
CONFIG_MBEDTLS_CFG_FILE="mcuboot-mbedtls-cfg.h"

CONFIG_BOOT_SWAP_SAVE_ENCTLV=n
CONFIG_BOOT_ENCRYPT_RSA=n
CONFIG_BOOT_ENCRYPT_EC256=n
CONFIG_BOOT_ENCRYPT_X25519=n

CONFIG_BOOT_UPGRADE_ONLY=n 
CONFIG_MCUBOOT_DOWNGRADE_PREVENTION=n 
CONFIG_BOOT_BOOTSTRAP=n

CONFIG_FLASH=y
CONFIG_FPROTECT=y

CONFIG_LOG=y
CONFIG_LOG_MODE_MINIMAL=y # former CONFIG_MODE_MINIMAL
CONFIG_LOG_DEFAULT_LEVEL=0
CONFIG_CBPRINTF_NANO=y
CONFIG_NRF_RTC_TIMER_USER_CHAN_COUNT=0

CONFIG_BOOT_SIGNATURE_TYPE_ECDSA_P256=y

CONFIG_BOOT_NRF_EXTERNAL_CRYPTO=y
CONFIG_SECURE_BOOT_CRYPTO=y
CONFIG_SB_CRYPTO_CLIENT_ECDSA_SECP256R1=y
CONFIG_SB_CRYPTO_CLIENT_SHA256=y
CONFIG_BL_SHA256_EXT_API_REQUIRED=y
CONFIG_BL_SECP256R1_EXT_API_REQUIRED=y
CONFIG_EXT_API_PROVIDE_EXT_API_ATLEAST_OPTIONAL=y

CONFIG_BOOT_SIGNATURE_KEY_FILE="private.pem"

CONFIG_FW_INFO_FIRMWARE_VERSION=1

CONFIG_NORDIC_QSPI_NOR=y
CONFIG_NORDIC_QSPI_NOR_FLASH_LAYOUT_PAGE_SIZE=4096
CONFIG_NORDIC_QSPI_NOR_STACK_WRITE_BUFFER_SIZE=16
CONFIG_MULTITHREADING=y
CONFIG_BOOT_MAX_IMG_SECTORS=256

CONFIG_IMG_MANAGER=y
CONFIG_UPDATEABLE_IMAGE_NUMBER=1
CONFIG_SIZE_OPTIMIZATIONS=y

CONFIG_PM_EXTERNAL_FLASH_BASE=0x0
CONFIG_PM_EXTERNAL_FLASH_MCUBOOT_SECONDARY=y

CONFIG_DEBUG_INFO=y

CONFIG_BOOT_SWAP_USING_MOVE=y

CONFIG_DFU_TARGET=y
CONFIG_DEBUG=y

CONFIG_LOG_OVERRIDE_LEVEL=4

[Talisca]

I have found the solution by chance.
As already written, I had debugged mcuboot and had the assumption that you still have to somehow tell the bootloader that it is an upgrade.
I found the confirmation at developer.nordicsemi.com/.../design.html

Then I remembered that I had read in some articles that the application itself can confirm the swap with "boot_write_img_confirmed()". Somehow I had the suspicion that there must be a function to set the swap type. And indeed. All I needed for the last two weeks was a call to the function "boot_request_upgrade()" with "BOOT_UPGRADE_PERMANENT or BOOT_UPGRADE_TEST".

But now I would like to know if there is any setting for the build process, so that this is set directly in the image trailer of the firmware? That would not only be very convinient in my eyes, but also a necessity. What would be the point of uploading a firmware that is not recognized as an upgrade by the bootloader? An uploaded firmware is always an upgrade in my use case.

[Simon]

Talisca said:

But now I would like to know if there is any setting for the build process, so that this is set directly in the image trailer of the firmware? That would not only be very convinient in my eyes, but also a necessity. What would be the point of uploading a firmware that is not recognized as an upgrade by the bootloader? An uploaded firmware is always an upgrade in my use case.

First I would like to inform you that it might be smart to confirm the image from within the image itself, this is how the NCS samples are doing it. For example, take a look at the http application update sample and the asset tracker v2 application. The reason for this is to avoid bricking the device. For example if you perform a DFU update with an unconfirmed image, and a fault happens during the Zephyr init process, the chip will reset (and not reach the point where it confirms itself), and it will swap back to the old image. If it's already confirmed, mcuboot will keep running that image and the device is bricked (unless you have added a possibility to let mcuboot perform the DFU update).

However, to answer your question, I think it should be feasible to confirm the image in the buil process. If you look under Image tool --> signing images, you can see that imagetool sign has a --confirm option. Try using that.

The app_update.bin file is created here and here I believe, so try modifying that fail and append --confirm.

Of course, you can also use app_to_sign.bin (see Using MCUboot in nRF Connect SDK) and sign it yourself following the guide https://github.com/hellesvik-nordic/samples_for_nrf_connect_sdk/tree/main/bootloader_samples/mcuboot_manual_sign 

Best regards,

Simon