MCUBoot + MCUmgr: Image in the secondary slot is not valid!

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Tom_H — Tue, 25 Jan 2022 15:58:19 GMT

Hi Håkon,

the workaround also fixed the issue on my side and helps a lot.

Many thanks for the great support and forwarding the issue for further investigation!

Best regards,
Thomas

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Håkon Alseth — Tue, 25 Jan 2022 13:13:35 GMT

Hi Thomas,

I believe I found the source of the issue, which also explains why it was working with one project, and not the other, atleast on my end of things.

It is this errata:

https://infocenter.nordicsemi.com/topic/errata_nRF52840_Rev2/ERR/nRF52840/Rev2/latest/anomaly_840_198.html#anomaly_840_198

Could you try using "spi2" instead in the boards/nrf52840dk_nrf52840.overlay and child_image/mcuboot/boards/nrf52840dk_nrf52840.overlay and see if this fixes the issue on your end as well? I do not have a better workaround at this moment, but I will ofcourse report this internally for further investigation.

Kind regards,

Håkon

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Tom_H — Tue, 25 Jan 2022 08:05:55 GMT

Hi Håkon,

thanks for the sample code. I am glad you can reproduce the problem. I tested your sample with NCS version v1.8.0 and I see the same issue.

Another strange behaviour is that the image verification works when using the DeviceManger app or the internal hci device of a Linux PC for the image upload, but fails when using an hci_uart device.

Best regards,
Thomas

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Håkon Alseth — Mon, 24 Jan 2022 11:39:26 GMT

Hi Thomas,

I am sorry, I must have misunderstood the scenario here. Given the speed you got, I assumed that you were referring to nRF5 sdk implementation, but yes; you can get this speed with mcumgr if overwriting the same image, it seems.

[quote user="Tom_H"]- Fails: peripheral with secondary slot on external flash (SPI) -> our usecase (example above)[/quote]

I setup hci_uart with this setup, and tried it locally. I can see that the issue is present on my side as well:

*** Booting Zephyr OS build v2.6.99-ncs1  ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: test
E: Image in the secondary slot is not valid!
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v2.6.99-ncs1  ***

The strange part is that if I manually program the "app_update_test.hex" into the primary slot, the hash of the image is equal to when I'm doing the same procedure over-the-air, which indicates that there's a problem with the moving the image from SPI-flash to internal flash. I am not sure where the issue is in your test-application.

Could you try this sample and see if the same issue occurs on your end?

devzone.nordicsemi.com/.../smp_5F00_svr_5F00_nrf52840dk_5F00_spi_5F00_nor.zip

Note: this one advertises as "Zephyr".

Kind regards,

Håkon

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Tom_H — Fri, 21 Jan 2022 14:35:59 GMT

Many thanks for the support.

Does this mean that at the current state we cannot use a dongle as HCI device for BLE OTA if the secondary bootloader slot of the peripheral device is located on external flash and connected via SPI?

I have tested the following combinations with MCUmgr CLI and a nordic dongle as HCI device on a Linux system to update a BLE peripheral device:
- Works: peripheral with secondary slot on internal flash
- Works: peripheral with secondary slot on external flash (QSPI)
- Fails: peripheral with secondary slot on external flash (SPI) -> our usecase (example above)

Our custom senosr node (peripheral device) is already in production and we need a redesign if the update process doesn't work. At the moment we are using the old nRF5 SDK with the included bootloader and are working on porting the application to NCS and Zephyr.

Best regards,
Thomas

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Håkon Alseth — Fri, 21 Jan 2022 12:53:24 GMT

Hi Thomas,

As mentioned, this is unfortunately the state of the image transfer at this time. I have inputted this as an internal improvement report. We are continuously trying to improve our deliveries, and speed of the transfer is one of the areas we're trying to improve. However, as of writing this answer (latest stable version NCS v1.8.0), the speed is approx. 3kB/s.

Kind regards,

Håkon

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Tom_H — Fri, 21 Jan 2022 12:20:07 GMT

Thanks for your response.

Is there a way to reduce the image transfer speed using a Nordic Device with BLE HCI firmware to make BLE OTA work for our intended production environment? Our Gateway is based on a custom board with a Raspberry PI compute module and a pre-certified nRF52840 module.

Best regards,
Thomas

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Håkon Alseth — Thu, 20 Jan 2022 08:47:39 GMT

Hi,

[quote user="Tom_H"]I further investigated the error and I think it is somehow related to the image transfer speed. Instead of a Raspberry Pi I used a PC with Ubuntu and did several tests. Before each test I erased the external flash[/quote]

I am glad to hear that you got it working!

[quote user="Tom_H"]The update process works when using the internal HCI device. Upload speed is around 3.0 KiB/s and it took 1m7s.

When I use the Nordic Dongle with BLE HCI firmware the update process fails. In this case upload speed is about 11,8 KiB/s and takes about 16 seconds.

Do you have any advice?[/quote]

I see the same speed at my end. Unfortunately, this is a current restriction with updating. I have opened up an internal improvement report to look at the speed of the overall DFU process.

Kind regards,

Håkon

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Tom_H — Wed, 19 Jan 2022 18:31:35 GMT

I further investigated the error and I think it is somehow related to the image transfer speed. Instead of a Raspberry Pi I used a PC with Ubuntu and did several tests. Before each test I erased the external flash

nrfjprog --qspieraseall

The update process works when using the internal HCI device. Upload speed is around 3.0 KiB/s and it took 1m7s.

When I use the Nordic Dongle with BLE HCI firmware the update process fails. In this case upload speed is about 11,8 KiB/s and takes about 16 seconds.

Do you have any advice?

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Tom_H — Wed, 19 Jan 2022 14:45:14 GMT

Thanks for the info.

The configuration for the larger MTU size has already been set.

# Allow for large Bluetooth data packets.
CONFIG_BT_L2CAP_TX_MTU=252
CONFIG_BT_BUF_ACL_RX_SIZE=256

CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=4096

Setting the MTU configuration from the throughput example fixes the allocation problem, but the image verification error remains the same

CONFIG_BT_BUF_ACL_RX_SIZE=251
CONFIG_BT_BUF_ACL_TX_SIZE=251
CONFIG_BT_BUF_ACL_TX_COUNT=40
CONFIG_BT_L2CAP_TX_BUF_COUNT=40
CONFIG_BT_ATT_PREPARE_COUNT=2
CONFIG_BT_L2CAP_TX_MTU=247
CONFIG_BT_CTLR_RX_BUFFERS=2
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251

*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
I: Starting bootloader
I: Primary image: magic=good, swap_type=0x4, copy_done=0x1, image_ok=0x1
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: none
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
Starting Bluetooth Peripheral LBS example
build time: Jan 19 2022 15:31:58
I: 2 Sectors of 4096 bytes
I: alloc wra: 0, fd0
I: data wra: 0, 1c
I: SoftDevice Controller build revision: 
I: 3f 47 70 8e 81 95 4e 86 |?Gp...N.
I: 9d d3 a2 95 88 f6 30 0a |......0.
I: 7f 53 49 fd             |.SI.    
I: No ID address. App must call settings_load()
Bluetooth initialized
Advertising successfully started
Connected
I: Swap type: none
Disconnected (reason 19)
Connected
I: Swap type: none
I: Swap type: none
Disconnected (reason 8)
Connected
I: Swap type: none
I: Swap type: none
Disconnected (reason 19)
Connected
I: Swap type: none
I: Swap type: test
I: Swap type: test
Disconnected (reason 19)
*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
I: Starting bootloader
I: Primary image: magic=good, swap_type=0x4, copy_done=0x1, image_ok=0x1
I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: test
E: Image in the secondary slot is not valid!
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
Starting Bluetooth Peripheral LBS example
build time: Jan 19 2022 15:31:58
I: 2 Sectors of 4096 bytes
I: alloc wra: 0, fd0
I: data wra: 0, 1c
I: SoftDevice Controller build revision: 
I: 3f 47 70 8e 81 95 4e 86 |?Gp...N.
I: 9d d3 a2 95 88 f6 30 0a |......0.
I: 7f 53 49 fd             |.SI.    
I: No ID address. App must call settings_load()
Bluetooth initialized
Advertising successfully started

But I think the problem is related to buffer or SPI configuration, cause the image upload throughput is much higher when using MCUmgr and it works in combination with QSPI.

Best regards,
Thomas

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Håkon Alseth — Wed, 19 Jan 2022 13:35:03 GMT

Hi,

Looks like there's a problem allocating TX packets at your end, this could be due to the bluetooth mtu that your bluetooth host uses. Could you try the following configs

CONFIG_BT_L2CAP_TX_MTU=252
CONFIG_BT_BUF_ACL_RX_SIZE=256
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2304

Kind regards,

Håkon

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Tom_H — Wed, 19 Jan 2022 12:54:36 GMT

Many thanks for the fast response!
I followed your steps but still get the same error message.

As a first step, I erased the flash and QSPI flash

nrfjprog -e
nrfjprog --reset
nrfjprog --qspieraseall

Than I build and flashed the sample using NCS v1.7.1

west build -p always -b nrf52840dk_nrf52840
west flash

I rebuilt the sample application, copied the update binary to a Raspberry Pi and uploaded the image using MCUmgr

ubuntu@ubuntu:~$ sudo go/bin/mcumgr -c ble_lbs image upload app_update.bin
 200.50 KiB / 200.50 KiB [======================================================================] 100.00% 9.76 KiB/s 20s
Done

List the images

ubuntu@ubuntu:~$ sudo go/bin/mcumgr -c ble_lbs image list
Images:
 image=0 slot=0
    version: 0.0.0
    bootable: true
    flags: active confirmed
    hash: 30372ead86b3fe06019e4c90d02c2b680bd97195ddc91f20cc5ec68b7516a4a6
 image=0 slot=1
    version: 0.0.0
    bootable: true
    flags:
    hash: 56385894ea602db72850094e158c58afb4925607790ce3d79d64b1024a2a3d3d
Split status: N/A (0)

Triggered an image test and reset the device using nrfjprog

ubuntu@ubuntu:~$ sudo go/bin/mcumgr -c ble_lbs image test 56385894ea602db72850094e158c58afb4925607790ce3d79d64b1024a2a3d
3d
Images:
 image=0 slot=0
    version: 0.0.0
    bootable: true
    flags: active confirmed
    hash: 30372ead86b3fe06019e4c90d02c2b680bd97195ddc91f20cc5ec68b7516a4a6
 image=0 slot=1
    version: 0.0.0
    bootable: true
    flags: pending
    hash: 56385894ea602db72850094e158c58afb4925607790ce3d79d64b1024a2a3d3d
Split status: N/A (0)

However, testing the image fails with the same error message

*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: none
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
Starting Bluetooth Peripheral LBS example
build time: Jan 19 2022 13:30:44
I: 2 Sectors of 4096 bytes
I: alloc wra: 0, fe8
I: data wra: 0, 0
I: SoftDevice Controller build revision: 
I: 3f 47 70 8e 81 95 4e 86 |?Gp...N.
I: 9d d3 a2 95 88 f6 30 0a |......0.
I: 7f 53 49 fd             |.SI.    
I: No ID address. App must call settings_load()
Bluetooth initialized
Advertising successfully started
Connected
I: Swap type: none
Disconnected (reason 19)
Connected
I: Swap type: none
I: Swap type: none
E: Unable to allocate TX context
E: Unable to allocate TX context
E: Unable to allocate TX context
E: Unable to allocate TX context
Disconnected (reason 8)
Connected
I: Swap type: none
I: Swap type: none
Disconnected (reason 19)
Connected
I: Swap type: none
I: Swap type: test
I: Swap type: test
Disconnected (reason 19)
*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: test
E: Image in the secondary slot is not valid!
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
Starting Bluetooth Peripheral LBS example
build time: Jan 19 2022 13:30:44
I: 2 Sectors of 4096 bytes
I: alloc wra: 0, fd0
I: data wra: 0, 1c
I: SoftDevice Controller build revision: 
I: 3f 47 70 8e 81 95 4e 86 |?Gp...N.
I: 9d d3 a2 95 88 f6 30 0a |......0.
I: 7f 53 49 fd             |.SI.    
I: No ID address. App must call settings_load()
Bluetooth initialized
Advertising successfully started

Did I do something wrong?

Best regards,

Thomas

RE: MCUBoot + MCUmgr: Image in the secondary slot is not valid!

Håkon Alseth — Wed, 19 Jan 2022 08:46:52 GMT

Hi,

Thank you for providing your test firmware.

I tried to replicate the issue you are seeing using these commands on the PC side:

sudo /path/to/mcumgr --conntype ble --connstring ctlr_name=hci0,peer_name='Nordic_LBS' image upload zephyr/app_update.bin

Here's the "image list" after successful upload:

sudo /path/to/mcumgr --conntype ble --connstring ctlr_name=hci0,peer_name='Nordic_LBS' image list
Images:
 image=0 slot=0
    version: 0.0.0
    bootable: true
    flags: active confirmed
    hash: 970ee41e6186bebf631aab7aebbc7c7ae0d6b027ac8149f782104c06884673b7
 image=0 slot=1
    version: 0.0.0
    bootable: true
    flags: 
    hash: 24bc32212e7cc352aafc82a5143dd3f79bd9819869c548dacaf41d2e2eddfe11
Split status: N/A (0)

Now test the second image and see that it is set to "pending" successfully:

sudo /path/to/mcumgr --conntype ble --connstring ctlr_name=hci0,peer_name='Nordic_LBS' image test 24bc32212e7cc352aafc82a5143dd3f79bd9819869c548dacaf41d2e2eddfe11
Images:
 image=0 slot=0
    version: 0.0.0
    bootable: true
    flags: active confirmed
    hash: 970ee41e6186bebf631aab7aebbc7c7ae0d6b027ac8149f782104c06884673b7
 image=0 slot=1
    version: 0.0.0
    bootable: true
    flags: pending
    hash: 24bc32212e7cc352aafc82a5143dd3f79bd9819869c548dacaf41d2e2eddfe11
Split status: N/A (0)

And here's the output after reset where you can see that the timestamp has changed (image tested OK):

*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: none
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
Starting Bluetooth Peripheral LBS example
build time: Jan 19 2022 09:25:17
I: 2 Sectors of 4096 bytes
I: alloc wra: 0, fd0
I: data wra: 0, 1c
I: SoftDevice Controller build revision: 
I: 3f 47 70 8e 81 95 4e 86 |?Gp...N.
I: 9d d3 a2 95 88 f6 30 0a |......0.
I: 7f 53 49 fd             |.SI.    
I: No ID address. App must call settings_load()
Bluetooth initialized
Advertising successfully started
Connected
I: Swap type: none
I: Swap type: test
I: Swap type: test
Disconnected (reason 19)
Connected
Disconnected (reason 19)
*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: test
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v2.6.99-ncs1-1  ***
Starting Bluetooth Peripheral LBS example
build time: Jan 19 2022 09:33:22
I: 2 Sectors of 4096 bytes
I: alloc wra: 0, fd0
I: data wra: 0, 1c
I: SoftDevice Controller build revision: 
I: 3f 47 70 8e 81 95 4e 86 |?Gp...N.
I: 9d d3 a2 95 88 f6 30 0a |......0.
I: 7f 53 49 fd             |.SI.    
I: No ID address. App must call settings_load()
Bluetooth initialized
Advertising successfully started

[quote user=""]Debugging MCUBoot with Ozone shows that the image signature/hash is incorrect when using MCUmgr CLI for the image upload. Further investigation of the error shows that the integrity check perforrmed in the bootutil_img_validate function fails. In this case the variable hash is invalid, but the content of the variable buf is correct. When I use the Device Manager App to upload the image, the signature is valid and the update is successful.[/quote]

If the hash is incorrect, then there's a problem with the integrity of the image. Since you are working with a nRF52840-DK, could you try to erase the flash and QSPI flash?

nrfjprog -e
nrfjprog --reset
nrfjprog -qspieraseall

Then flash your sample.

Kind regards,

Håkon