smp_srv example works but my project gives: E: Image in the secondary slot is not valid!

[nvelozsavino]

I'm writing an application for the nrf5340 with nrfConnect v1.6.1,

I've been able to do the FOTA upgrade through BLE with the example smp_srv located inside zephyr/samples/subsys/mgmt/mcumgr/smp_svr

These were the step I've followed:

1. Copy the smp_srv folder to my workspace folder (~/Documents/zephyr_projects/smp_srv)
2. west build -b nrf5340dk_nrf5340_cpuappns -p -- -DOVERLAY_CONFIG=overlay-bt.conf (build pristine for non-secure app)
3. west flash (as far as I know, the mcuboot is compiled as a child image, using the default debug key)
4. Change code (adding some logs to see the changes)
5. execute again  west build -b nrf5340dk_nrf5340_cpuappns -- -DOVERLAY_CONFIG=overlay-bt.conf to generate a new app_update.bin file with the modifications
6. Upload the app_update.bin to my android
7. Open nrfConnect application on my android
8. connect to the board
9. start DFU selecting the new app_update.bin file
10. Transfer completes, Validate
11. I can verify the new logs are applied

Everything works fine in this sample project

Important to notice that I've tested the smp_srv project with the following configuration:

# Enable mcumgr.
CONFIG_MCUMGR=y

# Some command handlers require a large stack.
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048

# Ensure an MCUboot-compatible binary is generated.
CONFIG_BOOTLOADER_MCUBOOT=y

# Enable flash operations.
CONFIG_FLASH=y

# Required by the `taskstat` command.
CONFIG_THREAD_MONITOR=y

# Enable statistics and statistic names.
CONFIG_STATS=y
CONFIG_STATS_NAMES=y

# Enable most core commands.
CONFIG_MCUMGR_CMD_IMG_MGMT=y
CONFIG_MCUMGR_CMD_OS_MGMT=y
# CONFIG_MCUMGR_CMD_STAT_MGMT=y

# Enable logging
CONFIG_LOG=y
CONFIG_MCUBOOT_UTIL_LOG_LEVEL_WRN=y

and the overlay-bt,conf I've stripped everyhing related to LITTLE_FS

# Allow for large Bluetooth data packets.
CONFIG_BT_L2CAP_TX_MTU=252
CONFIG_BT_BUF_ACL_RX_SIZE=256

# Enable the Bluetooth (unauthenticated) and shell mcumgr transports.
CONFIG_MCUMGR_SMP_BT=y
CONFIG_MCUMGR_SMP_BT_AUTHEN=n
CONFIG_MCUMGR_SMP_SHELL=y

# Enable the LittleFS file system.
# CONFIG_FILE_SYSTEM=y
# CONFIG_FILE_SYSTEM_LITTLEFS=y

# Add 256 bytes to accommodate upload command (lfs_stat overflows)
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2304

# Enable file system commands
CONFIG_MCUMGR_CMD_FS_MGMT=y

Then in my project, which I've added the same configuration located on the smp_srv prj.conf and overlay-bt.conf

# Incresed stack due to settings API usage
# CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048

CONFIG_BT=y
CONFIG_BT_DEBUG_LOG=y
CONFIG_BT_SMP=y
CONFIG_BT_SIGNING=y
CONFIG_BT_PERIPHERAL=y
CONFIG_BT_DIS=y
CONFIG_BT_ATT_PREPARE_COUNT=5
CONFIG_BT_BAS=y
CONFIG_BT_PRIVACY=y
CONFIG_BT_DEVICE_NAME="MyProject"
CONFIG_BT_DEVICE_APPEARANCE=833
CONFIG_BT_DEVICE_NAME_DYNAMIC=n
# CONFIG_BT_DEVICE_NAME_MAX=65
CONFIG_BT_USER_DATA_LEN_UPDATE=y
CONFIG_BT_BONDABLE=y
CONFIG_BT_KEYS_OVERWRITE_OLDEST=y
CONFIG_BT_SMP_ALLOW_UNAUTH_OVERWRITE=y
CONFIG_BT_SETTINGS=y
CONFIG_FLASH=y
# CONFIG_FLASH_PAGE_LAYOUT=y
# CONFIG_FLASH_MAP=y
# CONFIG_NVS=y
# CONFIG_SETTINGS=y


CONFIG_SPI=y
CONFIG_SENSOR=y

CONFIG_LIS3DH=y
CONFIG_LIS3DH_ACCEL_RANGE_2G=y
CONFIG_LIS3DH_ODR_5=y
CONFIG_LIS3DH_TRIGGER_OWN_THREAD=y
CONFIG_CBPRINTF_FP_SUPPORT=y
CONFIG_LIS3DH_CLICK_THS=64
CONFIG_LIS3DH_TIME_LIMIT=50
CONFIG_LIS3DH_TIME_LATENCY=2
CONFIG_LIS3DH_TIME_WINDOW=100
CONFIG_PM=y


CONFIG_MAX30005=y
CONFIG_MAX30005_TRIGGER_OWN_THREAD=y

CONFIG_I2C=y
CONFIG_ADP5360=y
CONFIG_ADP5360_TRIGGER_OWN_THREAD=y

# Required to disable default behavior of deep sleep on timeout
CONFIG_PM_DEVICE=y


# Allow for large Bluetooth data packets.
CONFIG_BT_L2CAP_TX_MTU=508
CONFIG_BT_BUF_ACL_RX_SIZE=512

# Enable the Bluetooth (unauthenticated) and shell mcumgr transports.
CONFIG_MCUMGR_SMP_BT=y
CONFIG_MCUMGR_SMP_BT_AUTHEN=n
CONFIG_MCUMGR_SMP_SHELL=y

# Enable the LittleFS file system.
# CONFIG_FILE_SYSTEM=y
# CONFIG_FILE_SYSTEM_LITTLEFS=y

# Add 256 bytes to accommodate upload command (lfs_stat overflows)
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2560

# Enable file system commands
# CONFIG_MCUMGR_CMD_FS_MGMT=y

# Enable mcumgr.
CONFIG_MCUMGR=y


# Ensure an MCUboot-compatible binary is generated.
CONFIG_BOOTLOADER_MCUBOOT=y

# Enable flash operations.
# CONFIG_FLASH=y

# Required by the `taskstat` command.
CONFIG_THREAD_MONITOR=y

# Enable statistics and statistic names.
# CONFIG_STATS=y
# CONFIG_STATS_NAMES=y

# Enable most core commands.
CONFIG_MCUMGR_CMD_IMG_MGMT=y
CONFIG_MCUMGR_CMD_OS_MGMT=y
# CONFIG_MCUMGR_CMD_STAT_MGMT=y

# Enable logging
CONFIG_LOG=y
CONFIG_MCUBOOT_UTIL_LOG_LEVEL_WRN=y
# CONFIG_MCUBOOT_SIGNATURE_KEY_FILE="root-rsa-2048.pem"

I've also added the necesary code to be able to initialize the smp service:

Includes:

#ifdef CONFIG_MCUMGR_SMP_BT
#include <mgmt/mcumgr/smp_bt.h>
#endif
#ifdef CONFIG_MCUMGR_CMD_OS_MGMT
#include "os_mgmt/os_mgmt.h"
#endif
#ifdef CONFIG_MCUMGR_CMD_IMG_MGMT
#include "img_mgmt/img_mgmt.h"
#endif
#ifdef CONFIG_MCUMGR_CMD_STAT_MGMT
#include "stat_mgmt/stat_mgmt.h"
#endif
#ifdef CONFIG_MCUMGR_CMD_SHELL_MGMT
#include "shell_mgmt/shell_mgmt.h"
#endif

in the main function:

#ifdef CONFIG_MCUMGR_CMD_OS_MGMT
os_mgmt_register_group();
#endif
#ifdef CONFIG_MCUMGR_CMD_IMG_MGMT
img_mgmt_register_group();
#endif
#ifdef CONFIG_MCUMGR_CMD_STAT_MGMT
stat_mgmt_register_group();
#endif
#ifdef CONFIG_MCUMGR_CMD_SHELL_MGMT
shell_mgmt_register_group();
#endif

and after enabling ble and registering the callbacks:

#ifdef CONFIG_MCUMGR_SMP_BT
smp_bt_register();
#endif

Then I execute the following steps:

1. west build -b nrf5340dk_nrf5340_cpuappns -p (build pristine for non-secure app)
2. west flash (as far as I know, the mcuboot is compiled as a child image, using the default debug key)
3. Change code (adding some logs to see the changes)
4. execute again  west build -b nrf5340dk_nrf5340_cpuappns to generate a new app_update.bin file with the modifications
5. Upload the app_update.bin to my android
6. Open nrfConnect application on my android
7. connect to the board
8. start DFU selecting the new app_update.bin file
9. Transfer completes.

Then I can see the logs where the mcuboot says that:

*** Booting Zephyr OS build v2.6.0-rc1-ncs1 ***
I: Starting bootloader
I: Primary image: magic=bad, swap_type=0x1, copy_done=0x2, image_ok=0x2
I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: test
E: Image in the secondary slot is not valid!
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v2.6.0-rc1-ncs1 ***

          

and then it loads the old image

What can be wrong?

I've tried enabling and disabling configurations but nothing seems to work

[Sigurd Hellesvik]

Hi

The primary image has "magic=bad". Is this also the case when you run the sample which worked?

In either case, you should be able to fix "magic=bad" by flashing build/zephyr/merged.hex using the Programmer tool in nRF Connect for Desktop instead of "west flash".

Do you still experience the issue after "magic=good" or "magic=unset" for both images?

Regards,
Sigurd Hellesvik

[nvelozsavino]

Hi, sorry for the late response:

I've been doing some tests:

After flashing for the first time I have this:

I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: none
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***

Then, I modify the code, build again, and upload the file app_update.bin using the nrf connect application on my android device, the transfer reaches 100%, it says validating and I see this log:

 *** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Swap type: test
E: Image in the secondary slot is not valid!
I: Bootloader chainload address offset: 0xc000
*** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***

I've tested using the cpuapp and cpuappns and it's the same, the smp_srv project works fine, but my project doesn't

I've cleaned a little bit the prj.conf:

# ============= FOTA ===========

# Enable mcumgr.
CONFIG_MCUMGR=y

# Some command handlers require a large stack.
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2560

# Ensure an MCUboot-compatible binary is generated.
CONFIG_BOOTLOADER_MCUBOOT=y

# Enable flash operations.
CONFIG_FLASH=y

# Required by the `taskstat` command.
CONFIG_THREAD_MONITOR=y

# Enable statistics and statistic names.
# CONFIG_STATS=y
# CONFIG_STATS_NAMES=y

# Enable most core commands.
CONFIG_MCUMGR_CMD_IMG_MGMT=y
CONFIG_MCUMGR_CMD_OS_MGMT=y
# CONFIG_MCUMGR_CMD_STAT_MGMT=y


# Enable logging
CONFIG_LOG=y
CONFIG_MCUBOOT_UTIL_LOG_LEVEL_WRN=y


# ========= FOTA BT ===========

# Allow for large Bluetooth data packets.
CONFIG_BT_L2CAP_TX_MTU=512
CONFIG_BT_BUF_ACL_RX_SIZE=512


# Enable the Bluetooth (unauthenticated) and shell mcumgr transports.
CONFIG_MCUMGR_SMP_BT=y
CONFIG_MCUMGR_SMP_BT_AUTHEN=n
CONFIG_MCUMGR_SMP_SHELL=y

# Enable the LittleFS file system.
# CONFIG_FILE_SYSTEM=y
# CONFIG_FILE_SYSTEM_LITTLEFS=y


# Add 256 bytes to accommodate upload command (lfs_stat overflows)
# CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2304


# Enable file system commands
# CONFIG_MCUMGR_CMD_FS_MGMT=y



CONFIG_BT=y
CONFIG_BT_PERIPHERAL=y
CONFIG_BT_DEVICE_NAME="Elausa"
CONFIG_BT_DEVICE_APPEARANCE=833
CONFIG_BT_MAX_CONN=1
CONFIG_BT_MAX_PAIRED=1



# Enable bonding
CONFIG_BT_SETTINGS=y
CONFIG_FLASH=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FLASH_MAP=y
CONFIG_NVS=y
CONFIG_SETTINGS=y
CONFIG_BT_SMP=y

CONFIG_BT_KEYS_OVERWRITE_OLDEST=y
CONFIG_BT_SMP_ALLOW_UNAUTH_OVERWRITE=y

CONFIG_BT_USER_DATA_LEN_UPDATE=y

# ============ Power management =======
CONFIG_PM=y
CONFIG_PM_DEVICE=y

I've also tested with the default rsa key and with my own and is the same:

for using my key I've added the following to the CMakeList.txt before the line

find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE}):

if (EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/mcuboot.conf")
        message(STATUS "Using mcuboot.conf")
        list(APPEND mcuboot_OVERLAY_CONFIG
                ${CMAKE_CURRENT_SOURCE_DIR}/mcuboot.conf
                )                
endif()

and my mcuboot.conf is:

CONFIG_BOOT_SIGNATURE_TYPE_RSA=y
CONFIG_BOOT_SIGNATURE_KEY_FILE="c:\\path\\to\\my\key\\psl-2048.pem"

The command for building the first time is:

west build --build-dir c:\projects\myproject\build-cpuapp c:\projects\myproject --pristine --board nrf5340dk_nrf5340_cpuapp -- -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=On -DNCS_TOOLCHAIN_VERSION:STRING="NONE" -DCONF_FILE:STRING="c:/projects/myproject/prj.conf" -DDTC_OVERLAY_FILE:STRING="c:/projects/myproject/boards/nrf5340dk_nrf5340_cpuapp.overlay"

This command is what the nrf connect extension for vscode execute when I click build pristine

Then when I clicked flash it uses:

west build --build-dir c:\projects\myproject\build-cpuapp c:\projects\myproject
west flash -d c:\projects\myproject\build-cpuapp --skip-rebuild --snr 960191461

Then I modify the code and execute a new build with the command:

west build --build-dir c:\projects\myproject\build-cpuapp c:\projects\myproject

Finally I grab the file c:\projects\myproject\build-cpuapp\zephyr\app_update.bin and send it through BLE using the nrf connect app from my android device

This is basically the exact same procedure I did for the smp_srv project which always works. The only things that differs is the configuration file because in my project I need to enable the CONF_BT_SETTINGS in order to save the bonding information and the MAC address to prevent changing every time it reset.

[Sigurd Hellesvik]

Hi

I have managed to reproduce your issue, but have not found a solution yet.

I will continue to look into this tomorrow, and return with an update by the end of that day.

If you are able to create a minimal reproduction of the problem, it would speed up the process.
This would be the minimal amount of changes you need to make to the prj.conf of the smp_srv sample to reproduce the issue.

Regards,
Sigurd Hellesvik

[Sigurd Hellesvik]

Hi

Unfortunally, I have not been able to solve this yet.

I will consult with some colleagues to see if I can find out more.

You will hear from me again on Monday.

Regards,
Sigurd Hellesvik

[Sigurd Hellesvik]

Hi

I am sorry for my late answer.

Our expert at this topic have not been available for a few days.
I expect him to return by the end of this week.

Regards,
Sigurd Hellesvik

[Sigurd Hellesvik]

Hi

When I run the smp_srv sample with only one change it works:

I created a folder smp_srv/child_image and a file smp_srv/child_image/mcuboot.conf

CONFIG_BOOT_SIGNATURE_TYPE_ECDSA_P256=n
CONFIG_BOOT_SIGNATURE_TYPE_RSA=y
CONFIG_BOOT_SIGNATURE_KEY_FILE="/c/<full_path_to>/priv.pem"

I build with only prj.conf and overlay-bt.conf.

Can you try to run the sample with only these?

If the sample works with custom certificates, then start adding the configurations you need to prj.conf one by one, until you find the config that crash the program.

Regards,
Sigurd Hellesvik

[nvelozsavino]

Actually what you just suggest solved another issue we were having when we waren't able to compile with the mcuboot.conf using the mcuboot_OVERLAY_CONFIG in the CMakeList.txt.

list(APPEND mcuboot_OVERLAY_CONFIG

"${CMAKE_CURRENT_SOURCE_DIR}/mcuboot.conf"

)

we got an undefined reference when compiling mcuboot

FAILED: zephyr/zephyr_prebuilt.elf zephyr/zephyr_prebuilt.map /home/nico/Documents/nico/otc/zephyr_projects/21191_elausa_alz/build_cpuapp/mcuboot/zephyr/zephyr_prebuilt.map 
: && ccache /home/nico/ncs/v1.6.1/toolchain/bin/arm-none-eabi-gcc   zephyr/CMakeFiles/zephyr_prebuilt.dir/misc/empty_file.c.obj -o zephyr/zephyr_prebuilt.elf  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -Wl,-T  zephyr/linker_zephyr_prebuilt.cmd  -Wl,-Map=/home/nico/Documents/nico/otc/zephyr_projects/21191_elausa_alz/build_cpuapp/mcuboot/zephyr/zephyr_prebuilt.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/aarch32/libarch__arm__core__aarch32.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/cmse/libarch__arm__core__aarch32__cortex_m__cmse.a  zephyr/arch/arch/arm/core/aarch32/mpu/libarch__arm__core__aarch32__mpu.a  zephyr/lib/libc/minimal/liblib__libc__minimal.a  zephyr/lib/posix/liblib__posix.a  zephyr/soc/arm/common/cortex_m/libsoc__arm__common__cortex_m.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/flash/libdrivers__flash.a  zephyr/drivers/serial/libdrivers__serial.a  modules/nrf/lib/fprotect/lib..__nrf__lib__fprotect.a  modules/nrf/lib/fatal_error/lib..__nrf__lib__fatal_error.a  modules/nrf/drivers/hw_cc310/lib..__nrf__drivers__hw_cc310.a  modules/mcuboot/boot/bootutil/zephyr/libmcuboot_util.a  modules/hal_nordic/nrfx/libmodules__hal_nordic__nrfx.a  modules/mbedtls/libmodules__mbedtls.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  -L"/home/nico/ncs/v1.6.1/toolchain/bin/../lib/gcc/arm-none-eabi/9.2.1/thumb/v8-m.main/nofp"  -L/home/nico/Documents/nico/otc/zephyr_projects/21191_elausa_alz/build_cpuapp/mcuboot/zephyr  -lgcc  zephyr/arch/common/libisr_tables.a  /home/nico/ncs/v1.6.1/nrfxlib/crypto/nrf_cc312_platform/lib/cortex-m33/soft-float/no-interrupts/libnrf_cc312_platform_0.9.10.a  -mcpu=cortex-m33  -mthumb  -mabi=aapcs  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -no-pie  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn && cd /home/nico/Documents/nico/otc/zephyr_projects/21191_elausa_alz/build_cpuapp/mcuboot/zephyr && /usr/bin/cmake -E echo
/home/nico/ncs/v1.6.1/toolchain/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld: app/libapp.a(main.c.obj): in function `main':
/home/nico/ncs/v1.6.1/bootloader/mcuboot/boot/zephyr/main.c:583: undefined reference to `pcd_lock_ram'
/home/nico/ncs/v1.6.1/toolchain/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld: app/libapp.a(loader.c.obj): in function `boot_validated_swap_type':
/home/nico/ncs/v1.6.1/bootloader/mcuboot/boot/bootutil/src/loader.c:828: undefined reference to `pcd_network_core_update'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
[335/343] Completed 'hci_rpmsg_subimage'
FAILED: modules/mcuboot/mcuboot_subimage-prefix/src/mcuboot_subimage-stamp/mcuboot_subimage-build mcuboot/zephyr/zephyr.hex mcuboot/zephyr/zephyr.elf /home/nico/Documents/nico/otc/zephyr_projects/21191_elausa_alz/build_cpuapp/modules/mcuboot/mcuboot_subimage-prefix/src/mcuboot_subimage-stamp/mcuboot_subimage-build /home/nico/Documents/nico/otc/zephyr_projects/21191_elausa_alz/build_cpuapp/mcuboot/zephyr/zephyr.hex /home/nico/Documents/nico/otc/zephyr_projects/21191_elausa_alz/build_cpuapp/mcuboot/zephyr/zephyr.elf 

As soon as I commented the code in CMakeList.txt and move the mcuboot.conf to the child_image it solve the compilation issue.

However, regarding the issue we believe we found the configuration that caused the issue:

CONFIG_BT_L2CAP_TX_MTU=512
CONFIG_BT_BUF_ACL_RX_SIZE=512

When we change this to the original values:

CONFIG_BT_L2CAP_TX_MTU=252
CONFIG_BT_BUF_ACL_RX_SIZE=256

We were able to do the upgrade successfully (using the sample project), we're going to try in the whole project to see if that solve the issue.

We would like to have have the MTU as big as we can, with 252 could work for now, but if this is the solution, can you eplain how to use the MTU in 512 bytes? is it possible?

[Sigurd Hellesvik]

Hi

Try to increase the size of CONFIG_MCUMGR_BUF_SIZE.

Regards,
Sigurd Hellesvik

[nvelozsavino]

Should I put that in the child_image/mcuboot.conf or in my prj.conf?

the config says:

MCUMGR_BUF_SIZE >= transport-specific-MTU + transport-overhead

how much is the transport-overhead for BLE?