CMake/build errors when applying minimal config to nsib and mcuboot

[koos]

I am using NCS v2.9.1 and trying to enable a bootloader chain with NSIB and MCUboot. The regular setup works fine, however if I try to enable the minimal configuration, I get a build errors (for NSIB) and CMake error (for MCUboot). I am using the instructions on this page of the docs.

The issue can be reproduced using the hello_world sample and using the following command:

west build -b nrf52840dk/nrf52840 zephyr/samples/hello_world -- \
-DSB_CONFIG_SECURE_BOOT_APPCORE=y \
-Db0_FILE_SUFFIX=minimal

This results in the following error:

FAILED: zephyr/zephyr_pre0.elf zephyr/zephyr_pre0.map /opt/nordic/ncs/v2.9.1/build/b0/zephyr/zephyr_pre0.map 
: && ccache /opt/nordic/ncs/toolchains/b8efef2ad5/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc -Os -DNDEBUG -gdwarf-4 -gdwarf-4 zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj -o zephyr/zephyr_pre0.elf  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -T  zephyr/linker_zephyr_pre0.cmd  -Wl,-Map=/opt/nordic/ncs/v2.9.1/build/b0/zephyr/zephyr_pre0.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/libarch__arm__core.a  zephyr/arch/arch/arm/core/cortex_m/libarch__arm__core__cortex_m.a  zephyr/lib/libc/picolibc/liblib__libc__picolibc.a  zephyr/lib/libc/common/liblib__libc__common.a  zephyr/soc/soc/nrf52840/libsoc__nordic.a  modules/nrf/lib/fprotect/lib..__nrf__lib__fprotect.a  modules/nrf/subsys/bootloader/bl_boot/lib..__nrf__subsys__bootloader__bl_boot.a  modules/nrf/subsys/bootloader/bl_crypto/lib..__nrf__subsys__bootloader__bl_crypto.a  modules/nrf/subsys/bootloader/bl_validation/lib..__nrf__subsys__bootloader__bl_validation.a  modules/nrf/subsys/bootloader/bl_storage/lib..__nrf__subsys__bootloader__bl_storage.a  modules/nrf/subsys/fw_info/lib..__nrf__subsys__fw_info.a  modules/nrf/drivers/hw_cc3xx/lib..__nrf__drivers__hw_cc3xx.a  modules/hal_nordic/nrfx/libmodules__hal_nordic__nrfx.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  -L/opt/nordic/ncs/v2.9.1/build/b0/zephyr  zephyr/arch/common/libisr_tables.a  /opt/nordic/ncs/v2.9.1/nrfxlib/crypto/nrf_oberon/lib/cortex-m4/hard-float/liboberon_3.0.15.a  /opt/nordic/ncs/v2.9.1/nrfxlib/crypto/nrf_cc310_bl/lib/cortex-m4/hard-float/no-interrupts/libnrf_cc310_bl_0.9.12.a  -mcpu=cortex-m4  -mthumb  -mabi=aapcs  -mfpu=fpv4-sp-d16  -mfloat-abi=hard  -mfp16-format=ieee  -mtp=soft  -fuse-ld=bfd  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -Wl,-no-pie  -specs=picolibc.specs  -DPICOLIBC_DOUBLE_PRINTF_SCANF  /opt/nordic/ncs/v2.9.1/nrfxlib/crypto/nrf_cc310_platform/lib/cortex-m4/hard-float/no-interrupts/libnrf_cc310_platform_0.9.19.a -L"/opt/nordic/ncs/toolchains/b8efef2ad5/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/thumb/v7e-m+fp/hard" -lc -lgcc && cd /opt/nordic/ncs/v2.9.1/build/b0/zephyr && /opt/homebrew/Cellar/cmake/3.29.0/bin/cmake -E true
/opt/nordic/ncs/toolchains/b8efef2ad5/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/zephyr_pre0.elf section `text' will not fit in region `FLASH'
/opt/nordic/ncs/toolchains/b8efef2ad5/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: region `FLASH' overflowed by 5032 bytes
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
[24/38] cd /opt/nordic/ncs/v2.9.1/build/_sysbuild && /opt/homebrew/Cellar/cmake/3.29.0/bin/cmake -E true
FAILED: modules/nrf/b0-prefix/src/b0-stamp/b0-build /opt/nordic/ncs/v2.9.1/build/modules/nrf/b0-prefix/src/b0-stamp/b0-build 
cd /opt/nordic/ncs/v2.9.1/build/b0 && /opt/homebrew/Cellar/cmake/3.29.0/bin/cmake --build .
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: /opt/homebrew/bin/cmake --build /opt/nordic/ncs/v2.9.1/build

When trying the minimal MCUboot configuration I get a CMake error:

Parsing /opt/nordic/ncs/v2.9.1/bootloader/mcuboot/boot/zephyr/Kconfig
Loaded configuration '/opt/nordic/ncs/v2.9.1/zephyr/boards/nordic/nrf52840dk/nrf52840dk_nrf52840_defconfig'
Merged configuration '/opt/nordic/ncs/v2.9.1/bootloader/mcuboot/boot/zephyr/prj_minimal.conf'
Merged configuration '/opt/nordic/ncs/v2.9.1/bootloader/mcuboot/boot/zephyr/boards/nrf52840dk_nrf52840.conf'
Merged configuration '/opt/nordic/ncs/v2.9.1/build/mcuboot/zephyr/.config.sysbuild'

error: Aborting due to Kconfig warnings

CMake Error at /opt/nordic/ncs/v2.9.1/zephyr/cmake/modules/kconfig.cmake:396 (message):
  command failed with return code: 1
Call Stack (most recent call first):
  /opt/nordic/ncs/v2.9.1/nrf/cmake/modules/kconfig.cmake:29 (include)
  /opt/nordic/ncs/v2.9.1/zephyr/cmake/modules/zephyr_default.cmake:133 (include)
  /opt/nordic/ncs/v2.9.1/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:66 (include)
  /opt/nordic/ncs/v2.9.1/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:92 (include_boilerplate)
  CMakeLists.txt:12 (find_package)


-- Configuring incomplete, errors occurred!
CMake Error at cmake/modules/sysbuild_extensions.cmake:514 (message):
  CMake configure failed for Zephyr project: mcuboot

  Location: /opt/nordic/ncs/v2.9.1/bootloader/mcuboot/boot/zephyr
Call Stack (most recent call first):
  cmake/modules/sysbuild_images.cmake:20 (ExternalZephyrProject_Cmake)
  cmake/modules/sysbuild_default.cmake:20 (include)
  /opt/nordic/ncs/v2.9.1/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:75 (include)
  /opt/nordic/ncs/v2.9.1/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:92 (include_boilerplate)
  /opt/nordic/ncs/v2.9.1/zephyr/share/sysbuild-package/cmake/SysbuildConfig.cmake:8 (include)
  template/CMakeLists.txt:10 (find_package)


-- Configuring incomplete, errors occurred!

Do you have any pointers what is going wrong here?

[Hieu]

Hello koos,

The issue with the minimal b0 build was that the application generated is bigger than the size allocated to it. It seems the features have been growing, but the minimal size, which was set years ago, was left unchanged and no longer fit.

The fix for this is adding these configurations to the build:

CONFIG_B0_MIN_PARTITION_SIZE=n
CONFIG_PM_PARTITION_SIZE_B0_IMAGE=0x6000

You can do that in b0's prj_minimal.conf file directly, or add this to the build command:

-Db0_CONFIG_B0_MIN_PARTITION_SIZE=n -Db0_CONFIG_PM_PARTITION_SIZE_B0_IMAGE=0x6000

The problem with minimal MCUboot build is also outdated configurations. In particular, these lines are outdated and must be removed:
https://github.com/nrfconnect/sdk-mcuboot/blob/v2.1.0-ncs3-1/boot/zephyr/prj_minimal.conf#L30-L32
I am not sure why the online commit history shows the file to be 4 months old, but it's actually 4 years old. My git extension on VS Code is able to show it as 4 years old correctly.

After fixing those breaking Kconfig, the build will then also have size issue. This is fixed by setting these Kconfig for MCUboot:

CONFIG_BOOT_USE_MIN_PARTITION_SIZE=n
CONFIG_PM_PARTITION_SIZE_MCUBOOT=0x5e00

Same as for b0, you can do that in MCUboot's prj_minimal.conf file directly, or add this to the build command:

-Dmcuboot_FILE_SUFFIX=minimal -Dmcuboot_CONFIG_BOOT_USE_MIN_PARTITION_SIZE=n -Dmcuboot_CONFIG_PM_PARTITION_SIZE_MCUBOOT=0x5e00

I have reported these issues internally. Our apologies for the inconvenience.

Hieu

[Hieu]

P.s: There will be further issue if you want to combine both b0 and MCUboot later. s0 and s1 partitions will be resolved to different sizes, resulting in build error. 

Due to alignment requirement, there are some empty partitions automatically created to keep s0, s1, and their neighboring partitions aligned. However, the empty partition which corresponds to s0 is placed between its two sub-partitions, s0_pad and s0_image. Meanwhile, the empty partition which corresponds to s1 is placed after s1 entirely. It is likely because s0_image is linked to mcuboot, which has an end-address alignment requirement, while s1_image doesn't have such a requirement.

There are a few ways to fix this.

1. Relocate the empty partition after s1 to between s1_pad and s1_image. 
   This must be done with a static Partition Manager (PM) configuration file. For more information, please refer here: Partition Manager.
   
   
2. Expand s*_pad and/or s*_image to fill the empty space.
   1. This can be done with the static PM file above
   2. This can also be done via Kconfig.
      1. s*_pad size is linked to mcuboot_pad size, which is controlled by CONFIG_PM_PARTITION_SIZE_MCUBOOT_PAD.
      2. s*_image size is linked to mcuboot partition size, which is controlled by CONFIG_PM_PARTITION_SIZE_MCUBOOT.

Overall, I recommend using the static PM configuration to expand both pad and image size, with the image being given a little more size. A system with bootloader should always use static partitioning, and the size increase are in the interest of future proofing.

[koos]

Hi Hieu,

Thanks for the elaborate answer!

I applied to changes you suggested, works perfectly. Do you have any idea how soon these fixes would be integrated in a next release? I'd rather not make local changes to the SDK, and could work with the regular sizes for now.

Could you please explain when these sizes issues could occur? If I build this for the hello_world sample, I get the a partitions.yml file in the build folder which corresponds to the following partition report. This doesn't show any size differences or empty partitions. 

  flash_primary (0x100000 - 1024kB): 
+--------------------------------------------------+
+---0x0: b0_container (0x7000 - 28kB)--------------+
| 0x0: b0 (0x6000 - 24kB)                          |
| 0x6000: provision (0x1000 - 4kB)                 |
+---0x7000: s0 (0x6000 - 24kB)---------------------+
| 0x7000: s0_pad (0x200 - 512B)                    |
+---0x7200: s0_image (0x5e00 - 23kB)---------------+
| 0x7200: mcuboot (0x5e00 - 23kB)                  |
+---0xd000: s1 (0x6000 - 24kB)---------------------+
| 0xd000: s1_pad (0x200 - 512B)                    |
| 0xd200: s1_image (0x5e00 - 23kB)                 |
+---0x13000: mcuboot_primary (0x76000 - 472kB)-----+
| 0x13000: mcuboot_pad (0x200 - 512B)              |
+---0x13200: app_image (0x75e00 - 471kB)-----------+
+---0x13200: mcuboot_primary_app (0x75e00 - 471kB)-+
| 0x13200: app (0x75e00 - 471kB)                   |
+--------------------------------------------------+
| 0x89000: mcuboot_secondary (0x76000 - 472kB)     |
| 0xff000: EMPTY_0 (0x1000 - 4kB)                  |
+--------------------------------------------------+

  sram_primary (0x40000 - 256kB): 
+--------------------------------------------+
| 0x20000000: sram_primary (0x40000 - 256kB) |
+--------------------------------------------+

[Hieu]

Hello koos,

I am glad it worked. 

Regarding the memory layout of your hello_world attempt, it seems that MCUboot size of 0x5e00 is the perfect size so that s0 and s1 fit into flash pages just right, without requiring any empty partition.

I realized I proposed the number 0x5e00 in my reply, but I can't remember if I was aiming for that or not. I think if you change it to 0x5d00 for example, the unfortunate gaps will happen again.

As for the release of a fix, unfortunately we are not allowed to discuss timeline or plan of any kind on DevZone. You will have to talk with your Nordic sales representative instead. My apology for the inconvenience.

[koos]

Hi Hieu,

Clear, indeed if i change the size to 0x5d00, the empty partitions appear. This is caught by an assert though, that checks if s0 and s1 are the same size, so that's very nice!

In file included from /opt/nordic/ncs/v2.9.1/zephyr/include/zephyr/toolchain.h:50,
                 from /opt/nordic/ncs/v2.9.1/nrf/include/fw_info.h:16,
                 from /opt/nordic/ncs/v2.9.1/nrf/include/bl_validation.h:15,
                 from /opt/nordic/ncs/v2.9.1/nrf/subsys/bootloader/bl_validation/bl_validation.c:7:
/opt/nordic/ncs/v2.9.1/nrf/subsys/bootloader/bl_validation/bl_validation.c: In function 'validate_firmware':
/opt/nordic/ncs/v2.9.1/zephyr/include/zephyr/toolchain/gcc.h:87:36: error: static assertion failed: "B0\'s slots aren\'t the same size. Check pm.yml."
   87 | #define BUILD_ASSERT(EXPR, MSG...) _Static_assert((EXPR), "" MSG)
      |                                    ^~~~~~~~~~~~~~
/opt/nordic/ncs/v2.9.1/nrf/subsys/bootloader/bl_validation/bl_validation.c:374:9: note: in expansion of macro 'BUILD_ASSERT'
  374 |         BUILD_ASSERT(PM_S0_SIZE == PM_S1_SIZE,
      |         ^~~~~~~~~~~~
[90/117] Building C object modules/hal_nordic/nrfx/CMakeFiles/modules__hal_...r/opt/nordic/ncs/v2.9.1/modules/hal/nordic/nrfx/drivers/src/nrfx_nvmc.c.obj
ninja: build stopped: subcommand failed.
[29/54] cd /opt/nordic/ncs/v2.9.1/build/_sysbuild && /opt/homebrew/Cellar/cmake/3.29.0/bin/cmake -E true
FAILED: modules/nrf/b0-prefix/src/b0-stamp/b0-build /opt/nordic/ncs/v2.9.1/build/modules/nrf/b0-prefix/src/b0-stamp/b0-build 
cd /opt/nordic/ncs/v2.9.1/build/b0 && /opt/homebrew/Cellar/cmake/3.29.0/bin/cmake --build .
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: /opt/homebrew/bin/cmake --build /opt/nordic/ncs/v2.9.1/build

Thanks for the help!