Enabling CONFIG_BOOTLOADER_MCUBOOT=y and/or CONFIG_BT=y breaks unrelated modules

[janb37896796]

We are attempting to add various modules to our project. For simplicity we've reduced our system to a toy setup closely mimicking the structure of our project and are trying to run the tensorflow lite micro module. 

The state of the code here: flite_main_proj compiles without error and runs the tflite hello world code on the nRF5340 dev kit board as expected. 

Problems arise when we try to enable either the bootloader or bluetooth configs, currently commented out in the file proj.conf

CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_BT=y

Either config produces the same errors on build. The following is from enabling MCUBOOT:

151/299] Building C object CMakeFiles/app.dir/main.c.obj
[152/299] Building CXX object modules/tflite_sub_proj/lib/CMakeFiles/..__tflite_sub_proj__lib.dir/main_functions.cpp.obj
FAILED: modules/tflite_sub_proj/lib/CMakeFiles/..__tflite_sub_proj__lib.dir/main_functions.cpp.obj 
ccache /home/user/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-g++ -DKERNEL -DMBEDTLS_CONFIG_FILE=\"mcuboot-mbedtls-cfg.h\" -DNRF5340_XXAA_APPLICATION -DNRF_SKIP_FICR_NS_COPY_TO_RAM -DUSE_PARTITION_MANAGER=1 -D__PROGRAM_START -D__ZEPHYR__=1 -I/home/user/Code/tf-ws/zephyr/include -I/home/user/Code/tf-ws/tflite_main_proj/app/build/mcuboot/zephyr/include/generated -I/home/user/Code/tf-ws/zephyr/soc/arm/nordic_nrf/nrf53 -I/home/user/Code/tf-ws/zephyr/soc/arm/nordic_nrf/common/. -I/home/user/Code/tf-ws/nrf/include -I/home/user/Code/tf-ws/nrf/tests/include -I/home/user/Code/tf-ws/tflite_sub_proj/inc -I/home/user/Code/tf-ws/modules/hal/cmsis/CMSIS/Core/Include -I/home/user/Code/tf-ws/modules/hal/nordic/nrfx -I/home/user/Code/tf-ws/modules/hal/nordic/nrfx/drivers/include -I/home/user/Code/tf-ws/modules/hal/nordic/nrfx/mdk -I/home/user/Code/tf-ws/zephyr/modules/hal_nordic/nrfx/. -I/home/user/Code/tf-ws/bootloader/mcuboot/boot/zephyr/include -I/home/user/Code/tf-ws/modules/crypto/mbedtls/include -I/home/user/Code/tf-ws/modules/crypto/mbedtls/library -I/home/user/Code/tf-ws/zephyr/modules/mbedtls/configs -I/home/user/Code/tf-ws/zephyr/modules/mbedtls/include -I/home/user/Code/tf-ws/bootloader/mcuboot/boot/bootutil/zephyr/.. -I/home/user/Code/tf-ws/bootloader/mcuboot/boot/bootutil/zephyr/../include -I/home/user/Code/tf-ws/bootloader/mcuboot/boot/bootutil/zephyr/../../zephyr/include -isystem /home/user/Code/tf-ws/zephyr/lib/libc/minimal/include -isystem /home/user/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/include -isystem /home/user/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/include-fixed -isystem /home/user/Code/tf-ws/nrfxlib/crypto/nrf_cc312_platform/include -fno-strict-aliasing -Os -fcheck-new -fno-exceptions -fno-rtti -imacros /home/user/Code/tf-ws/tflite_main_proj/app/build/mcuboot/zephyr/include/generated/autoconf.h -fno-common -g -gdwarf-4 -fdiagnostics-color=always -mcpu=cortex-m33 -mthumb -mabi=aapcs --sysroot=/home/user/zephyr-sdk-0.16.1/arm-zephyr-eabi/arm-zephyr-eabi -imacros /home/user/Code/tf-ws/zephyr/include/zephyr/toolchain/zephyr_stdint.h -Wno-unused-but-set-variable -fno-asynchronous-unwind-tables -fno-reorder-functions --param=min-pagesize=0 -fno-defer-pop -fmacro-prefix-map=/home/user/Code/tf-ws/bootloader/mcuboot/boot/zephyr=CMAKE_SOURCE_DIR -fmacro-prefix-map=/home/user/Code/tf-ws/zephyr=ZEPHYR_BASE -fmacro-prefix-map=/home/user/Code/tf-ws=WEST_TOPDIR -ffunction-sections -fdata-sections -nostdinc -MD -MT modules/tflite_sub_proj/lib/CMakeFiles/..__tflite_sub_proj__lib.dir/main_functions.cpp.obj -MF modules/tflite_sub_proj/lib/CMakeFiles/..__tflite_sub_proj__lib.dir/main_functions.cpp.obj.d -o modules/tflite_sub_proj/lib/CMakeFiles/..__tflite_sub_proj__lib.dir/main_functions.cpp.obj -c /home/user/Code/tf-ws/tflite_sub_proj/lib/main_functions.cpp
/home/user/Code/tf-ws/tflite_sub_proj/lib/main_functions.cpp:19:10: fatal error: tensorflow/lite/micro/all_ops_resolver.h: No such file or directory
   19 | #include <tensorflow/lite/micro/all_ops_resolver.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
[153/299] Linking C static library modules/nrf/lib/fatal_error/lib..__nrf__lib__fatal_error.a
[154/299] Building C object modules/mbedtls/CMakeFiles/modules__mbedtls.dir/home/user/Code/tf-ws/modules/crypto/mbedtls/library/bignum_mod.c.obj
[155/299] Building C object modules/mbedtls/CMakeFiles/modules__mbedtls.dir/home/user/Code/tf-ws/modules/crypto/mbedtls/library/camellia.c.obj
[156/299] Building C object modules/mbedtls/CMakeFiles/modules__mbedtls.dir/home/user/Code/tf-ws/modules/crypto/mbedtls/library/ccm.c.obj
[157/299] Building C object modules/mbedtls/CMakeFiles/modules__mbedtls.dir/home/user/Code/tf-ws/modules/crypto/mbedtls/library/chacha20.c.obj
[158/299] Linking C static library modules/nrf/drivers/hw_cc310/lib..__nrf__drivers__hw_cc310.a
[159/299] Building C object modules/mbedtls/CMakeFiles/modules__mbedtls.dir/home/user/Code/tf-ws/modules/crypto/mbedtls/library/chachapoly.c.obj
[160/299] Building C object modules/mbedtls/CMakeFiles/modules__mbedtls.dir/home/user/Code/tf-ws/modules/crypto/mbedtls/library/cipher.c.obj
[161/299] Building C object modules/mbedtls/CMakeFiles/modules__mbedtls.dir/home/user/Code/tf-ws/modules/crypto/mbedtls/library/cipher_wrap.c.obj
[162/299] Building C object modules/mbedtls/CMakeFiles/modules__mbedtls.dir/home/user/Code/tf-ws/modules/crypto/mbedtls/library/cmac.c.obj
[163/299] Building C object modules/mbedtls/CMakeFiles/modules__mbedtls.dir/home/user/Code/tf-ws/modules/crypto/mbedtls/library/code_share.c.obj
[164/299] Linking C static library modules/mcuboot/boot/bootutil/zephyr/libmcuboot_util.a
[165/299] Building CXX object modules/tflite_sub_proj/lib/CMakeFiles/..__tflite_sub_proj__lib.dir/output_handler.cpp.obj
FAILED: modules/tflite_sub_proj/lib/CMakeFiles/..__tflite_sub_proj__lib.dir/output_handler.cpp.obj 
ccache /home/user/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-g++ -DKERNEL -DMBEDTLS_CONFIG_FILE=\"mcuboot-mbedtls-cfg.h\" -DNRF5340_XXAA_APPLICATION -DNRF_SKIP_FICR_NS_COPY_TO_RAM -DUSE_PARTITION_MANAGER=1 -D__PROGRAM_START -D__ZEPHYR__=1 -I/home/user/Code/tf-ws/zephyr/include -I/home/user/Code/tf-ws/tflite_main_proj/app/build/mcuboot/zephyr/include/generated -I/home/user/Code/tf-ws/zephyr/soc/arm/nordic_nrf/nrf53 -I/home/user/Code/tf-ws/zephyr/soc/arm/nordic_nrf/common/. -I/home/user/Code/tf-ws/nrf/include -I/home/user/Code/tf-ws/nrf/tests/include -I/home/user/Code/tf-ws/tflite_sub_proj/inc -I/home/user/Code/tf-ws/modules/hal/cmsis/CMSIS/Core/Include -I/home/user/Code/tf-ws/modules/hal/nordic/nrfx -I/home/user/Code/tf-ws/modules/hal/nordic/nrfx/drivers/include -I/home/user/Code/tf-ws/modules/hal/nordic/nrfx/mdk -I/home/user/Code/tf-ws/zephyr/modules/hal_nordic/nrfx/. -I/home/user/Code/tf-ws/bootloader/mcuboot/boot/zephyr/include -I/home/user/Code/tf-ws/modules/crypto/mbedtls/include -I/home/user/Code/tf-ws/modules/crypto/mbedtls/library -I/home/user/Code/tf-ws/zephyr/modules/mbedtls/configs -I/home/user/Code/tf-ws/zephyr/modules/mbedtls/include -I/home/user/Code/tf-ws/bootloader/mcuboot/boot/bootutil/zephyr/.. -I/home/user/Code/tf-ws/bootloader/mcuboot/boot/bootutil/zephyr/../include -I/home/user/Code/tf-ws/bootloader/mcuboot/boot/bootutil/zephyr/../../zephyr/include -isystem /home/user/Code/tf-ws/zephyr/lib/libc/minimal/include -isystem /home/user/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/include -isystem /home/user/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/include-fixed -isystem /home/user/Code/tf-ws/nrfxlib/crypto/nrf_cc312_platform/include -fno-strict-aliasing -Os -fcheck-new -fno-exceptions -fno-rtti -imacros /home/user/Code/tf-ws/tflite_main_proj/app/build/mcuboot/zephyr/include/generated/autoconf.h -fno-common -g -gdwarf-4 -fdiagnostics-color=always -mcpu=cortex-m33 -mthumb -mabi=aapcs --sysroot=/home/user/zephyr-sdk-0.16.1/arm-zephyr-eabi/arm-zephyr-eabi -imacros /home/user/Code/tf-ws/zephyr/include/zephyr/toolchain/zephyr_stdint.h -Wno-unused-but-set-variable -fno-asynchronous-unwind-tables -fno-reorder-functions --param=min-pagesize=0 -fno-defer-pop -fmacro-prefix-map=/home/user/Code/tf-ws/bootloader/mcuboot/boot/zephyr=CMAKE_SOURCE_DIR -fmacro-prefix-map=/home/user/Code/tf-ws/zephyr=ZEPHYR_BASE -fmacro-prefix-map=/home/user/Code/tf-ws=WEST_TOPDIR -ffunction-sections -fdata-sections -nostdinc -MD -MT modules/tflite_sub_proj/lib/CMakeFiles/..__tflite_sub_proj__lib.dir/output_handler.cpp.obj -MF modules/tflite_sub_proj/lib/CMakeFiles/..__tflite_sub_proj__lib.dir/output_handler.cpp.obj.d -o modules/tflite_sub_proj/lib/CMakeFiles/..__tflite_sub_proj__lib.dir/output_handler.cpp.obj -c /home/user/Code/tf-ws/tflite_sub_proj/lib/output_handler.cpp
In file included from /home/user/Code/tf-ws/tflite_sub_proj/lib/output_handler.cpp:17:
/home/user/Code/tf-ws/tflite_sub_proj/inc/output_handler.hpp:20:10: fatal error: tensorflow/lite/c/common.h: No such file or directory
   20 | #include <tensorflow/lite/c/common.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

Based on this discussion Setting CONFIG_BOOTLOADER_MCUBOOT=y in prj.config on custom board prevents code working  an attempt was made to include the tf module in the child_image. We added the folder tflite_main_proj/app/child_image which includes a .conf duplicating our proj.conf. Unfortunately this has had no impact on our build and we are still seeing the same errors as before. 

This may be a similar issues to (Missing Forum Thread)

Any advice is appreciated!

[Sigurd Hellesvik]

Hi,

The common factor for CONFIG_BT (specifically for 5340) and CONFIG_BOOTLOADER_MCUBOOT is that they both enable multi imahe builds  which also enable the partition manager.

From your error message, I can not say exactly how this is related yet.

Is it an option for you to send me your minimal project to reproduce this error?
If not, I will give you some tips on how you can debug this yourself.

Regards,
Sigurd Hellesvik

[Guido]

Hi Sigurd,

Thank you for your support. I'm janb37896796 's co-worker, they asked me to follow the issue for this week.

The original post provides a github link with a minimal project. I'll re submit: https://github.com/janfari/tflite_main_proj
We are trying to mimic our current configuration where we use west to pull individual repositories with drivers and libraries. The readme explains the setup, it's a zephyr workspace (workspace folder, clone/west init and west update).
Under tflite_main_proj/app/prj.conf look for:

# # # # BAD
# CONFIG_BOOTLOADER_MCUBOOT=y
# CONFIG_BT=y 
# # # # BAD

Uncomment any of those to replicate.

We are aware of the multi image build and partition manager being enabled when using this with the ncs/zephy. We have used them for the project and they worked without problems until now.

To give you another angle: what's new is that we are looking on using zephyr's tensorflow-lite(tflite) module, which requires C++ to be enabled.
What we have tried:
- tflite and C++ work fine on their own with the nRF5340.
- Enabling tflite and C++ with our current project which resulted in compile time errors.
- After reconstructing the configs from scratchjanb37896796 was able to single out those 2 configs to be the breaking ones when in combination with tflite and C++.
- The original post shows that we tried some solutions that we found on the devzone.

I appreciate you taking the time to review this.

Best regards,
Guido

[Sigurd Hellesvik]

I got the west project set up, but get the error:

fatal error: main_functions.h: No such file or directory
   18 | #include <main_functions.h>

Looks like I am missing some files?

Also, I suggest that you rename your .c files to .cpp to use C++. This will make Zephyr detect some stuff automatically.

[Guido]

I can't reproduce that error.
That file is in a project in the west manifest.
It should be in tflite_sub_proj/inc/main_functions.h after running west update

[Sigurd Hellesvik]

This is how my tflite_main_proj folder looks after "west update"

tflite]$ tree tflite_main_proj/
tflite_main_proj/
├── app
│   ├── child_image
│   │   └── mcuboot
│   │       ├── boards
│   │       │   └── nrf5340dk_nrf5340_cpuapp.overlay
│   │       └── mcuboot.conf
│   ├── CMakeLists.txt
│   ├── Kconfig
│   ├── prj.conf
│   ├── sample.yaml
│   └── src
│       └── main.c
├── CMakeLists.txt
├── Kconfig
├── README.md
├── west.yml
└── zephyr
    └── module.yml

Also, the "inc" folder is not to be seen in https://github.com/janfari/tflite_main_proj either.

Nor reffered to in https://github.com/janfari/tflite_main_proj/blob/main/west.yml

And for the west folder which I inited:

ls
tflite_main_proj  zephyr  bootloader  tools  modules

Sigurd Hellesvik said:

I got the west project set up, but get the error:

The error is from the build btw.

[janb37896796]

Hi Siguard,
Thank you for looking in to this issue. And thank you Guido for helping support.

Siguard, can you confirm that you've followed the step in the readme to update the .west/config file to point to the west.yml in tflite_main_proj? 

The contents of the config should look like this:

[manifest]
path = tflite_main_proj
file = west.yml

[zephyr]
base = zephyr

If you run `west update` after updating this it should pull down the folder tflite_sub_proj repo which contains the inc and src directories