Mcuboot serial recovery, can't open BOOT_SECONDARY_SLOT flash area, ncs v2.9.0

I am migrating from a child_image type project to the newer sysbuild system used in ncs v2.9.0.

My application is based on the ncs nrf5340_audio application, but my application and hardware target are both out-of-tree.

Using ncs v2.6.1 I was able to build this application so it supported mcuboot serial recovery using USB CDC and the application also supported DFU using USB CDC, but now that I am building this application using ncs v2.9.0 with sysbuild, I can no longer get the serial recovery to work correctly.

After building, I program the nrf5340 with the zephyr.hex file in the build\b0n\zephyr and build\mcuboot\zephyr directories.  After this step, I can reboot the system into serial recovery mode and I see the following output on the logging interface:

[0:0:0.355,773] <inf> spi_nor: mx25r64@0: 8 MiBy flash
*** Booting My Application v2.1.0-dev-12e5ee106034 ***
*** Using nRF Connect SDK v2.9.0-7787b2649840 ***
*** Using Zephyr OS v3.7.99-1f8f3dc29142 ***
[0:0:0.359,161] <inf> mcuboot: Starting bootloader
[0:0:0.359,893] <inf> mcuboot: Enter the serial recovery mode

I then use AuTerm to download the application images.  For the network core I download the file build\signed_by_mcuboot_and_b0_ipc_radio.bin into image #3 and I download the file build\<product>\zephyr\zephyr.signed.bin into image #0.

If I then reboot the system, the application fails to load and the system appears to be hung/crashed.

I debugged the bootloader and discovered that it is failing to open the flash area for BOOT_SECONDARY_SLOT in <ncs>\mcuboot\boot\bootutil\src\loader.c near line 2688.  

This system does use an external spi flash (mx25r64) and I was able to use this flash for DFU using ncs v2.6.1 so I'm sure the hardware is working, but in the serial recover mode, I think mcuboot should download the application image into the on-chip flash so I don't understand why the bootloader can't access the flash BOOT_SECONDARY_SLOT area.

I'll insert my project configuration files below:

In sysbuild_dfu.conf:

SB_CONFIG_BOOTLOADER_MCUBOOT=y
SB_CONFIG_PARTITION_MANAGER=y

# Step 6.3 - Configure project to use external flash for DFU
SB_CONFIG_PM_EXTERNAL_FLASH_MCUBOOT_SECONDARY=y

# Step 7.2 - Add b0n image
SB_CONFIG_SECURE_BOOT_NETCORE=y
# nRF Connect SDK ipc_radio image
SB_CONFIG_NETCORE_IPC_RADIO=y

# Step 7.3 - Set up multiple partitions
SB_CONFIG_MCUBOOT_UPDATEABLE_IMAGES=2

# Step 7.4 - Add support to netcore for DFU
SB_CONFIG_NETCORE_APP_UPDATE=y

# Step 7.5 - Add support to mcuboot for updating
# two cores simultaneously
SB_CONFIG_MCUBOOT_NRF53_MULTI_IMAGE_UPDATE=y

# Step 7.6 - Simultaneous FOTA does not support rollback
SB_CONFIG_MCUBOOT_MODE_OVERWRITE_ONLY=y

In prj_dfu.conf:

#####
# Use nRF5340 IPC BT controller
#CONFIG_BT_LL_ACS_NRF53=y

#CONFIG_MAIN_STACK_SIZE=3000

CONFIG_BT_MAX_CONN=2

#####
# Items added for UPF
CONFIG_BT_AUDIO_CODEC_CAP_MAX_METADATA_SIZE=30
CONFIG_NRF_APPROTECT_USE_UICR=n
CONFIG_BT_L2CAP_TX_MTU=128

#####
# nRF5340 Audio
CONFIG_NRF5340_AUDIO=y
CONFIG_AUDIO_SOURCE_USB=y

# HEADSET = 1, GATEWAY = 2
CONFIG_AUDIO_DEV=2
#CONFIG_TRANSPORT_CIS=y
#CONFIG_TRANSPORT_BIS=y

# The device to scan for is now set in the PtdSettings struct
#CONFIG_BT_DEVICE_NAME="INZONE Buds"
#CONFIG_BT_DEVICE_NAME="Galaxy Buds2 Pro"

# The Sony WF-1000XM5 earbuds require more than the default 5 codec capabilities
CONFIG_CODEC_CAP_COUNT_MAX=12

#####
# We need to enable the STREAM_BIDIRECTIONAL option so the zephyr code will
# compile with this feature enabled.  The g_settings.biDirAudio flag will
# be used in the PTD app to decide what to do at run time.
CONFIG_STREAM_BIDIRECTIONAL=y

#####
# Sample Rate Converter items
# Requied for bi-dir audio with USB because the system 
# uses 24 kHz for both source and sink streams.
CONFIG_SAMPLE_RATE_CONVERTER=y
CONFIG_SAMPLE_RATE_CONVERTER_FILTER_SIMPLE=y

#####
# Debug items

#####
# General
CONFIG_DEBUG=y
CONFIG_DEBUG_INFO=y
CONFIG_WDT_CTLR=n
CONFIG_ASSERT=n
CONFIG_STACK_USAGE=n
CONFIG_THREAD_MONITOR=n
CONFIG_THREAD_NAME=y
CONFIG_THREAD_RUNTIME_STATS=n
CONFIG_THREAD_RUNTIME_STATS_USE_TIMING_FUNCTIONS=n
CONFIG_STACK_SENTINEL=y
CONFIG_INIT_STACKS=y
CONFIG_STATS=n
CONFIG_STATS_SHELL=n

#####
# I2C Eeprom support
CONFIG_I2C=y
CONFIG_EEPROM=y
CONFIG_EEPROM_AT24=y
CONFIG_EEPROM_SHELL=n

#####
# Settings
CONFIG_SETTINGS=y
CONFIG_BT_SETTINGS=y
CONFIG_SETTINGS_RUNTIME=y
CONFIG_SETTINGS_SHELL=n

#####
# Enable USB features
CONFIG_USB_DEVICE_STACK=y
CONFIG_USB_COMPOSITE_DEVICE=y
CONFIG_USB_DEVICE_INITIALIZE_AT_BOOT=n
CONFIG_USB_DEVICE_MANUFACTURER="Portland Tool And Die"
CONFIG_USB_DEVICE_PRODUCT="PTD BQC-4150"
CONFIG_USB_DEVICE_VID=0x1915
CONFIG_USB_DEVICE_PID=0xEA9C

#####
# Enable USB CDC uart support
CONFIG_USB_CDC_ACM=y
CONFIG_UART_LINE_CTRL=y
CONFIG_UART_INTERRUPT_DRIVEN=y

# Uart driver
#CONFIG_SERIAL=y

#####
# Logging
CONFIG_LOG=y
#CONFIG_LOG_BUFFER_SIZE=2048
CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y
CONFIG_LOG_TAG_MAX_LEN=2
CONFIG_LOG_TAG_DEFAULT="--"
CONFIG_PRINTK=y
# Don't mix the log output with the shell uart IO
CONFIG_LOG_BACKEND_UART=y
CONFIG_SHELL_LOG_BACKEND=n
#CONFIG_LOG_DEFAULT_LEVEL=4
#CONFIG_LOG_MODE_DEFERRED=n
#CONFIG_LOG_MODE_IMMEDIATE=y
#CONFIG_LOG_PROCESS_THREAD=y

# Console related defines
CONFIG_CONSOLE=y
CONFIG_UART_CONSOLE=y

# Shell related defines
CONFIG_BOOT_BANNER=n
CONFIG_SHELL=y
CONFIG_USE_SEGGER_RTT=n
## Disable logs on RTT
CONFIG_SHELL_RTT_INIT_LOG_LEVEL_NONE=n
CONFIG_SHELL_BACKEND_RTT=n
CONFIG_SHELL_BACKEND_SERIAL=y
CONFIG_SHELL_BACKEND_SERIAL_CHECK_DTR=y
#CONFIG_SHELL_BACKEND_SERIAL_INIT_PRIORITY=51
CONFIG_SHELL_VT100_COLORS=n
CONFIG_SHELL_STACK_SIZE=2048
CONFIG_SHELL_CMD_BUFF_SIZE=1024
## Reduce shell memory usage
CONFIG_SHELL_WILDCARD=n
CONFIG_SHELL_HELP_ON_WRONG_ARGUMENT_COUNT=n
CONFIG_SHELL_STATS=n
CONFIG_SHELL_CMDS=n
CONFIG_SHELL_HISTORY=n
CONFIG_SHELL_PROMPT_UART="BQC-4150>"

# Turn off default shell commands
CONFIG_I2C_SHELL=n
CONFIG_HWINFO_SHELL=n
CONFIG_CLOCK_CONTROL_NRF_SHELL=n
CONFIG_FLASH_SHELL=n
CONFIG_DEVICE_SHELL=n
CONFIG_KERNEL_SHELL=n
CONFIG_BT_SHELL=n
CONFIG_BT_TESTING=n

# Suppress err msg from sd_check_card_type. Because SPI_SDHC has no card presence method,
# assume card is in slot. Thus error message is always shown if card is not inserted
#CONFIG_SD_LOG_LEVEL_OFF=y

#####
# Serial boot loader items
# Enable mcumgr.
CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_MCUMGR_TRANSPORT_SHELL=y
CONFIG_MCUMGR_GRP_IMG=y
CONFIG_IMG_ERASE_PROGRESSIVELY=y
CONFIG_BASE64=y
CONFIG_CRC=y
#CONFIG_BOOT_SIGNATURE_KEY_FILE="D:\Azure-ptd\BQC-4150\CodeSigningKey\bqc4150_private_key.pem"

# External SPI Flash
CONFIG_SPI=y
CONFIG_SPI_NOR=y
CONFIG_SPI_NOR_SFDP_DEVICETREE=y

# Enable flash operations
CONFIG_FLASH=y
CONFIG_FLASH_MAP=y
CONFIG_FLASH_PAGE_LAYOUT=y

CONFIG_MULTITHREADING=y
CONFIG_SPI_NOR_FLASH_LAYOUT_PAGE_SIZE=4096
CONFIG_SOC_FLASH_NRF_EMULATE_ONE_BYTE_WRITE_ACCESS=y
#CONFIG_FPROTECT=n

# The network core cannot access external flash directly. The flash simulator must be used to
# provide a memory region that is used to forward the new firmware to the network core
CONFIG_PM_EXTERNAL_FLASH_MCUBOOT_SECONDARY=y
CONFIG_PCD_APP=y
CONFIG_NRF53_UPGRADE_NETWORK_CORE=y
CONFIG_UPDATEABLE_IMAGE_NUMBER=2
CONFIG_ADD_MCUBOOT_MEDIATE_SIM_FLASH_DTS=y

# Only enable the external driver check override if we are not
# using the Nordic QSPI flash driver.
#CONFIG_PM_OVERRIDE_EXTERNAL_DRIVER_CHECK=y - moved to sysbuild.conf

#####
# BT security features
# SMP = Security Management Protocol
CONFIG_BT_SMP=y
# If CONFIG_BT_PRIVACY is no, then the system should use public addressees, otherwise RPA.
CONFIG_BT_PRIVACY=n
CONFIG_BT_BONDABLE=y
# we do not want to enforce MITM as this requires authenticated pairing
CONFIG_BT_SMP_ENFORCE_MITM=n
# we want to overwrite old bonds from the same device (optional, security risk)
CONFIG_BT_SMP_ALLOW_UNAUTH_OVERWRITE=n
CONFIG_BT_ID_UNPAIR_MATCHING_BONDS=n
# Enable CONFIG_BT_LOG_SNIFFER_INFO to have the shell print the link keys on pairing events
CONFIG_BT_LOG_SNIFFER_INFO=n

#####
# BLE audio features for unicast client
# MPL = Media Player
CONFIG_BT_BAP_UNICAST_CLIENT=y
CONFIG_BT_CAP_INITIATOR=y
CONFIG_BT_CSIP_SET_COORDINATOR=y
CONFIG_BT_BAP_UNICAST_CLIENT_ASE_SNK_COUNT=2
CONFIG_BT_BAP_UNICAST_CLIENT_ASE_SRC_COUNT=2
CONFIG_BT_BAP_UNICAST_CONFIGURABLE=y
CONFIG_BT_MPL=y
# MCS = Media Control Service
CONFIG_BT_MCS=y
# VCP = Volume Control Profile
CONFIG_BT_VCP_VOL_CTLR=y


#####
# BLE audio features for broadcast source
CONFIG_AURACAST=n
CONFIG_BT_ISO_BROADCASTER=y
CONFIG_BT_BAP_BROADCAST_SOURCE=y
# Need to use the CONFIG_BT_AUDIO_BROADCAST_CONFIGURABLE to support 48kHz audio from USB
CONFIG_BT_AUDIO_BROADCAST_CONFIGURABLE=y
CONFIG_BT_AUDIO_BITRATE_BROADCAST_SRC=96000
#CONFIG_BT_BAP_BROADCAST_16_2_2=n
#CONFIG_BT_BAP_BROADCAST_24_2_1=n
CONFIG_BT_DEVICE_NAME="PTD BIS SOURCE"
CONFIG_BT_AUDIO_BROADCAST_NAME="PTD BIS SOURCE"
#CONFIG_BT_AUDIO_BROADCAST_NAME_ALT="PTD 2"
#CONFIG_BT_AUDIO_BROADCAST_ENCRYPTED=n
#CONFIG_BT_AUDIO_BROADCAST_ENCRYPTION_KEY="PTD_KEY"

#####
# BLE audio features for unicast server
#CONFIG_BT_VCP_VOL_REND=y
#CONFIG_BT_CSIP_SET_MEMBER=y

In sysbuild\mcuboot\prj_dfu.conf:

#CONFIG_DEBUG=y
#CONFIG_DEBUG_INFO=y

CONFIG_SIZE_OPTIMIZATIONS=y
CONFIG_MAIN_STACK_SIZE=20480
CONFIG_MBEDTLS_CFG_FILE="mcuboot-mbedtls-cfg.h"
CONFIG_PCD_APP=y

#CONFIG_MCUMGR=y
#CONFIG_MCUMGR_GRP_STAT=n
#CONFIG_MCUMGR_GRP_OS=y
#CONFIG_MCUMGR_GRP_IMG=y

# Enable mcuboot serial over USB CDC port
CONFIG_MCUBOOT_LOG_LEVEL_INF=y
#CONFIG_MCUBOOT_LOG_LEVEL_DBG=y
CONFIG_MCUBOOT_SERIAL=y
CONFIG_BOOT_SERIAL_CDC_ACM=y
CONFIG_USB_DEVICE_MANUFACTURER="Portland Tool And Die"
CONFIG_USB_DEVICE_PRODUCT="PTD BQC-4150"
CONFIG_USB_DEVICE_VID=0x1915
CONFIG_USB_DEVICE_PID=0xEA9C

# Stay in serial recovery mode if no valid app is found
CONFIG_BOOT_SERIAL_NO_APPLICATION=y

# Configure loggging
CONFIG_CONSOLE=y
CONFIG_CBPRINTF_NANO=y
CONFIG_LOG=y
# We need to enable UART_CONSOLE even though we are using USB CDC.
CONFIG_UART_CONSOLE=y

# External SPI Flash
CONFIG_SPI=y
CONFIG_SPI_NOR=y
CONFIG_SPI_NOR_SFDP_DEVICETREE=y

# Enable flash operations
CONFIG_FLASH=y
#CONFIG_IMG_MANAGER=y
#CONFIG_STREAM_FLASH=y
CONFIG_FLASH_MAP=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_BOOT_MAX_IMG_SECTORS=512

CONFIG_MULTITHREADING=y
# Change default flash erase size for flexible flash layout
# The Nordic QSPI driver does not use the SPI_NOR_FLASH_LAYOUT_PAGE_SIZE variable
CONFIG_SPI_NOR_FLASH_LAYOUT_PAGE_SIZE=4096
CONFIG_BOOT_ERASE_PROGRESSIVELY=y
CONFIG_SOC_FLASH_NRF_EMULATE_ONE_BYTE_WRITE_ACCESS=y
CONFIG_FPROTECT=y

# The following configurations are required to support simultaneous multi-image update
# These options have moved to the top level sysbuild config file.
CONFIG_UPDATEABLE_IMAGE_NUMBER=2
CONFIG_BOOT_UPGRADE_ONLY=y

# The network core cannot access external flash directly. The flash simulator must be used to
# provide a memory region that is used to forward the new firmware to the network core
CONFIG_FLASH_SIMULATOR=y
CONFIG_FLASH_SIMULATOR_DOUBLE_WRITES=y
CONFIG_FLASH_SIMULATOR_STATS=n
CONFIG_PM_EXTERNAL_FLASH_MCUBOOT_SECONDARY=y

# Add support for programming the network processor as part of the DFU system
# These options have moved to the top level sysbuild config file.
CONFIG_NRF53_MULTI_IMAGE_UPDATE=y
CONFIG_BOOT_IMAGE_ACCESS_HOOKS=y
CONFIG_MCUBOOT_SERIAL_DIRECT_IMAGE_UPLOAD=y
#CONFIG_NRF53_RECOVERY_NETWORK_CORE=y

# Only enable the external driver check override if we are not
# using the Nordic QSPI flash driver.
# These options have moved to the top level sysbuild config file.
CONFIG_PM_OVERRIDE_EXTERNAL_DRIVER_CHECK=y

# Use button0 to enter serial recovery mode
CONFIG_BOOT_SERIAL_ENTRANCE_GPIO=y

In pm_static_dfu.yml:

mcuboot:
  address: 0x0
  placement:
    before:
    - mcuboot_primary
  region: flash_primary
  size: 0x16000
mcuboot_pad:
  address: 0x16000
  placement:
    align:
      start: 0x4000
    before:
    - mcuboot_primary_app
  region: flash_primary
  size: 0x200
mcuboot_primary:
  address: 0x16000
  orig_span: &id001
  - mcuboot_pad
  - app
  region: flash_primary
  size: 0xe7000
  span: *id001
app:
  address: 0x16200
  region: flash_primary
  size: 0xe6e00  
mcuboot_primary_app:
  address: 0x16200
  orig_span: &id002
  - app
  region: flash_primary
  size: 0xe6e00
  span: *id002
settings_storage:
  address: 0xfd000
  end_address: 0xff000
  placement:
    align:
      start: 0x4000
    before:
    - end
  region: flash_primary
  size: 0x2000
EMPTY_0:
  address: 0xff000
  placement:
    after:
    - settings_storage
  region: flash_primary
  size: 0x1000
mcuboot_primary_1:
  address: 0x0
  size: 0x40000
  device: flash_ctrl
  region: ram_flash
mcuboot_secondary:
  address: 0x00000
  size: 0xec000
  device: mx25r64
  region: external_flash
  placement:
    align:
      start: 0x4  
mcuboot_secondary_1:
  address: 0xec000
  size: 0x40000
  device: mx25r64
  region: external_flash
external_flash:
  address: 0x12C000
  size: 0x6d4000
  device: mx25r64
  region: external_flash
pcd_sram:
  address: 0x20000000
  size: 0x2000
  region: sram_primary
rpmsg_nrf53_sram:
  address: 0x20070000
  end_address: 0x20080000
  placement:
    before:
    - end
  region: sram_primary
  size: 0x10000
sram_primary:
  address: 0x20002000
  end_address: 0x20070000
  region: sram_primary
  size: 0x6e000
otp:
  address: 0xff8100
  end_address: 0xff83fc
  region: otp
  size: 0x2fc  

Please let me know if you have any ideas for how to fix this issue or if there are other tests I should run to help debug this issue.

Best regards,
Kristopher

Related