Bug in NCS v1.7.1 with 32kHz RC Oscillator and MCUboot enabled

Well, this one took me many hours of tracing to track down the root cause, so I hope it helps someone!

Note: I wasn't able to reproduce this directly with a sample included in the nRF Connect SDK. But it happens with my application, based on the "light_ctrl" mesh sample.

It occurs with the following 3-way combination (tested, only if all met);

  • CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC=y
  • CONFIG_BOOTLOADER_MCUBOOT=y
  • nRF Connect SDK v1.7.1 (didn't occur with v1.4.2 I last used)

I also had to enable CONFIG_BT_LL_SW_SPLIT=y  for zephyr controller to be able to fit in the flash of the nRF52833DK I'm using, but not sure if this is related (yes I intend to move to the soft device and know it is now recommended).

To fix I've changed to CONFIG_CLOCK_CONTROL_NRF_K32SRC_SYNTH=y

I've found these two issues which appear to have similar failure logs;

https://github.com/zephyrproject-rtos/zephyr/pull/25583

https://devzone.nordicsemi.com/f/nordic-q-a/76899/zephyr-build-smp_svr-for-thingy52-fails

Build error log below

Kevin

C:\freelance\git\pt118b_firmware>west build --pristine --board pt118a_nrf52833_3-1_3A --build-dir build-release_pt118a-3-1-3A -- -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=On -DNCS_TOOLCHAIN_VERSION:STRING="NONE"
-- west build: making build dir C:\freelance\git\pt118b_firmware\build-release_pt118a-3-1-3A pristine
-- west build: generating a build system
Including boilerplate (Zephyr base): C:/freelance/ncs/v1.7.1/zephyr/cmake/app/boilerplate.cmake
-- Application: C:/freelance/git/pt118b_firmware
-- Zephyr version: 2.6.99 (C:/freelance/ncs/v1.7.1/zephyr), build: v2.6.99-ncs1-1
-- Found Python3: C:/freelance/ncs/v1.7.1/toolchain/opt/bin/python.exe (found suitable exact version "3.8.2") found components: Interpreter
-- Found west (found suitable version "0.11.1", minimum required is "0.7.1")
-- Board: pt118a_nrf52833_3-1_3A
-- Cache files will be written to: C:/freelance/ncs/v1.7.1/zephyr/.cache
-- Found dtc: C:/freelance/ncs/v1.7.1/toolchain/opt/bin/dtc.exe (found suitable version "1.4.7", minimum required is "1.4.6")
-- Found toolchain: gnuarmemb (C:/freelance/ncs/v1.7.1/toolchain/opt)
-- Found BOARD.dts: C:/freelance/git/pt118b_firmware/boards/arm/pt118a_nrf52833_3-1_3A/pt118a_nrf52833_3-1_3A.dts
-- Generated zephyr.dts: C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/zephyr/zephyr.dts
-- Generated devicetree_unfixed.h: C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/zephyr/include/generated/devicetree_unfixed.h
-- Generated device_extern.h: C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/zephyr/include/generated/device_extern.h
-- Including generated dts.cmake file: C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/zephyr/dts.cmake
Parsing C:/freelance/ncs/v1.7.1/zephyr/Kconfig
Loaded configuration 'C:/freelance/git/pt118b_firmware/boards/arm/pt118a_nrf52833_3-1_3A/pt118a_nrf52833_3-1_3A_defconfig'
Merged configuration 'C:/freelance/git/pt118b_firmware/prj.conf'
Configuration saved to 'C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/zephyr/.config'
Kconfig header saved to 'C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/zephyr/include/generated/autoconf.h'
-- The C compiler identification is GNU 9.2.1
-- The CXX compiler identification is GNU 9.2.1
-- The ASM compiler identification is GNU
-- Found assembler: C:/freelance/ncs/v1.7.1/toolchain/opt/bin/arm-none-eabi-gcc.exe

=== child image mcuboot -  begin ===
Including boilerplate (Zephyr base): C:/freelance/ncs/v1.7.1/zephyr/cmake/app/boilerplate.cmake
-- Application: C:/freelance/ncs/v1.7.1/bootloader/mcuboot/boot/zephyr
-- Zephyr version: 2.6.99 (C:/freelance/ncs/v1.7.1/zephyr), build: v2.6.99-ncs1-1
-- Found Python3: C:/freelance/ncs/v1.7.1/toolchain/opt/bin/python.exe (found suitable exact version "3.8.2") found components: Interpreter
-- Found west (found suitable version "0.11.1", minimum required is "0.7.1")
-- Board: pt118a_nrf52833_3-1_3A
-- Cache files will be written to: C:/freelance/ncs/v1.7.1/zephyr/.cache
-- Found dtc: C:/freelance/ncs/v1.7.1/toolchain/opt/bin/dtc.exe (found suitable version "1.4.7", minimum required is "1.4.6")
-- Found toolchain: gnuarmemb (C:/freelance/ncs/v1.7.1/toolchain/opt)
-- Found BOARD.dts: C:/freelance/git/pt118b_firmware/boards/arm/pt118a_nrf52833_3-1_3A/pt118a_nrf52833_3-1_3A.dts
-- Found devicetree overlay: C:/freelance/ncs/v1.7.1/bootloader/mcuboot/boot/zephyr/dts.overlay
-- Generated zephyr.dts: C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/mcuboot/zephyr/zephyr.dts
-- Generated devicetree_unfixed.h: C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/mcuboot/zephyr/include/generated/devicetree_unfixed.h
-- Generated device_extern.h: C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/mcuboot/zephyr/include/generated/device_extern.h
-- Including generated dts.cmake file: C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/mcuboot/zephyr/dts.cmake

warning: The choice symbol BT_CTLR_TX_PWR_PLUS_8 (defined at
subsys/bluetooth\controller/Kconfig:210) was selected (set =y), but no symbol ended up as the choice
selection. See
http://docs.zephyrproject.org/latest/reference/kconfig/CONFIG_BT_CTLR_TX_PWR_PLUS_8.html and/or look
up BT_CTLR_TX_PWR_PLUS_8 in the menuconfig/guiconfig interface. The Application Development Primer,
Setting Configuration Values, and Kconfig - Tips and Best Practices sections of the manual might be
helpful too.
Parsing C:/freelance/ncs/v1.7.1/bootloader/mcuboot/boot/zephyr/Kconfig

Loaded configuration 'C:/freelance/git/pt118b_firmware/boards/arm/pt118a_nrf52833_3-1_3A/pt118a_nrf52833_3-1_3A_defconfig'
Merged configuration 'C:/freelance/ncs/v1.7.1/bootloader/mcuboot/boot/zephyr/prj.conf'
Merged configuration 'C:/freelance/ncs/v1.7.1/nrf/subsys/partition_manager/partition_manager_enabled.conf'
Configuration saved to 'C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/mcuboot/zephyr/.config'
Kconfig header saved to 'C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/mcuboot/zephyr/include/generated/autoconf.h'
-- The C compiler identification is GNU 9.2.1
-- The CXX compiler identification is GNU 9.2.1
-- The ASM compiler identification is GNU
-- Found assembler: C:/freelance/ncs/v1.7.1/toolchain/opt/bin/arm-none-eabi-gcc.exe
MCUBoot bootloader key file: C:/freelance/ncs/v1.7.1/bootloader/mcuboot/root-rsa-2048.pem
-- Configuring done
-- Generating done
-- Build files have been written to: C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/mcuboot
=== child image mcuboot -  end ===

CMake Warning at C:/freelance/ncs/v1.7.1/nrf/modules/mcuboot/CMakeLists.txt:204 (message):


        ---------------------------------------------------------
        --- WARNING: Using default MCUBoot key, it should not ---
        --- be used for production.                           ---
        ---------------------------------------------------------




-- Configuring done
-- Generating done
-- Build files have been written to: C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A
-- west build: building application
[5/319] Performing build step for 'mcuboot_subimage'
[250/257] Linking C executable zephyr\zephyr_prebuilt.elf
FAILED: zephyr/zephyr_prebuilt.elf zephyr/zephyr_prebuilt.map
cmd.exe /C "cd . && C:\freelance\ncs\v1.7.1\toolchain\opt\bin\arm-none-eabi-gcc.exe   zephyr/CMakeFiles/zephyr_prebuilt.dir/misc/empty_file.c.obj -o zephyr\zephyr_prebuilt.elf  -Wl,-T  zephyr/linker_zephyr_prebuilt.cmd  -Wl,-Map=C:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/mcuboot/zephyr/zephyr_prebuilt.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/aarch32/libarch__arm__core__aarch32.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a  zephyr/arch/arch/arm/core/aarch32/mpu/libarch__arm__core__aarch32__mpu.a  zephyr/lib/libc/minimal/liblib__libc__minimal.a  zephyr/lib/posix/liblib__posix.a  zephyr/soc/arm/common/cortex_m/libsoc__arm__common__cortex_m.a  zephyr/soc/arm/nordic_nrf/nrf52/libsoc__arm__nordic_nrf__nrf52.a  zephyr/boards/boards/arm/pt118a_nrf52833_3-1_3A/lib..__..__..__git__pt118b_firmware__boards__arm__pt118a_nrf52833_3-1_3A.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/sensor/nrf5/libdrivers__sensor__nrf5.a  zephyr/drivers/flash/libdrivers__flash.a  zephyr/drivers/serial/libdrivers__serial.a  modules/nrf/lib/fprotect/lib..__nrf__lib__fprotect.a  modules/nrf/lib/fatal_error/lib..__nrf__lib__fatal_error.a  modules/mcuboot/boot/bootutil/zephyr/libmcuboot_util.a  modules/hal_nordic/nrfx/libmodules__hal_nordic__nrfx.a  modules/mbedtls/libmodules__mbedtls.a  modules/segger/libmodules__segger.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -L"c:/freelance/ncs/v1.7.1/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/thumb/v7e-m/nofp"  -LC:/freelance/git/pt118b_firmware/build-release_pt118a-3-1-3A/mcuboot/zephyr  -lgcc  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m4  -mthumb  -mabi=aapcs  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -no-pie  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn && cmd.exe /C "cd /D C:\freelance\git\pt118b_firmware\build-release_pt118a-3-1-3A\mcuboot\zephyr && C:\freelance\ncs\v1.7.1\toolchain\opt\bin\cmake.exe -E echo ""
c:/freelance/ncs/v1.7.1/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: zephyr/drivers/sensor/nrf5/libdrivers__sensor__nrf5.a(temp_nrf5.c.obj): in function `k_sem_give':
C:\freelance\git\pt118b_firmware\build-release_pt118a-3-1-3A\mcuboot/zephyr/include/generated/syscalls/kernel.h:841: undefined reference to `z_impl_k_sem_give'
c:/freelance/ncs/v1.7.1/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: zephyr/drivers/sensor/nrf5/libdrivers__sensor__nrf5.a(temp_nrf5.c.obj): in function `k_sem_init':
C:\freelance\git\pt118b_firmware\build-release_pt118a-3-1-3A\mcuboot/zephyr/include/generated/syscalls/kernel.h:810: undefined reference to `z_impl_k_sem_init'
c:/freelance/ncs/v1.7.1/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: zephyr/drivers/sensor/nrf5/libdrivers__sensor__nrf5.a(temp_nrf5.c.obj): in function `k_mutex_init':
C:\freelance\git\pt118b_firmware\build-release_pt118a-3-1-3A\mcuboot/zephyr/include/generated/syscalls/kernel.h:708: undefined reference to `z_impl_k_mutex_init'
c:/freelance/ncs/v1.7.1/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: zephyr/drivers/sensor/nrf5/libdrivers__sensor__nrf5.a(temp_nrf5.c.obj): in function `k_mutex_lock':
C:\freelance\git\pt118b_firmware\build-release_pt118a-3-1-3A\mcuboot/zephyr/include/generated/syscalls/kernel.h:724: undefined reference to `z_impl_k_mutex_lock'
c:/freelance/ncs/v1.7.1/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: zephyr/drivers/sensor/nrf5/libdrivers__sensor__nrf5.a(temp_nrf5.c.obj): in function `k_sem_take':
C:\freelance\git\pt118b_firmware\build-release_pt118a-3-1-3A\mcuboot/zephyr/include/generated/syscalls/kernel.h:826: undefined reference to `z_impl_k_sem_take'
c:/freelance/ncs/v1.7.1/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: zephyr/drivers/sensor/nrf5/libdrivers__sensor__nrf5.a(temp_nrf5.c.obj): in function `k_mutex_unlock':
C:\freelance\git\pt118b_firmware\build-release_pt118a-3-1-3A\mcuboot/zephyr/include/generated/syscalls/kernel.h:738: undefined reference to `z_impl_k_mutex_unlock'
collect2.exe: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
[212/319] Building C object zephyr/subsys/bluetooth/mesh/CMakeFiles/subsys__bluetooth__mesh.dir/adv.c.obj
FAILED: modules/mcuboot/mcuboot_subimage-prefix/src/mcuboot_subimage-stamp/mcuboot_subimage-build mcuboot/zephyr/zephyr.hex mcuboot/zephyr/zephyr.elf
cmd.exe /C "cd /D C:\freelance\git\pt118b_firmware\build-release_pt118a-3-1-3A\mcuboot && C:\freelance\ncs\v1.7.1\toolchain\opt\bin\cmake.exe --build . --"
[218/319] Building C object zephyr/subsys/bluetooth/host/CMakeFiles/subsys__bluetooth__host.dir/gatt.c.obj
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: 'C:\freelance\ncs\v1.7.1\toolchain\opt\bin\cmake.EXE' --build 'C:\freelance\git\pt118b_firmware\build-release_pt118a-3-1-3A'

Parents
  • I also filed a private support case with Nordic and got some more info;

    Okay, so it might not be due to a bug after all, but rather that the calibration routine for the 32kHz clock was changed (in nRFConnect SDk v1.6.0 I think). Basically, the 32kHz requires a calibration routine that depends on multithreading. So you will need to enable multithreading or disable the calibration in the sub-image that's not compiling. The temp_nrf5 files doesn't seem to handle calibration, but they do require multithreading, so please try enabling that in your application.

    I would think one of the configs CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC_CALIBRATION=n or CONFIG_MULTITHREADING=y would be sufficient here

    I ended up solving with CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC_CALIBRATION=n to avoid the size overhead of multithreading. However someone on the Zephyr bug does say this isn't recommended. I'm not sure why but will post there.

    Note: this only needs to be set for the MCUBOOT image, not the main image, i.e. adding to a \child_image\mcuboot.conf as per https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/ug_multi_image.html which then merges with the default MCUBOOT configuration.

  • Thank you very much for the update!
    Yes, I mentioned there that it isn't recommended, because the Zephyr docs say so: "If calibration is disabled when RC is used for low frequency clock then accuracy of the low frequency clock will degrade. Disable on your own risk.".

    This makes sense for a functionality that requires higher precision clock like BLE, but I guess it's OK to use it in mcuboot (that's only my impression - feel free to correct me).

    I think I found the problem - https://github.com/mcu-tools/mcuboot/blob/main/boot/zephyr/Kconfig#L682

Reply Children
No Data
Related