Weird Compilation issue from v2.4.1 to v2.5.0

[Jacky Ortega]

Hi All,

I am trying to migrate my toolchain/sdk v2.4.1 project to toolchain/sdk v2.5.0. And I am having these weird issues related to mbedtls. The same project without any changes is compiling fine on v2.4.1 but on v2.5.0 it is not.

Some of the modules I am using:
- matter

- COAP

- mcu-boot

I tried using the following but they didn't resolve anything:
CONFIG_MBEDTLS_SSL_PROTO_DTLS=y
CONFIG_MBEDTLS_SSL_COOKIE_C=y
CONFIG_NET_SOCKETS_ENABLE_DTLS=y
CONFIG_MBEDTLS_DEBUG=y

[567/587] Linking CXX executable zephyr/zephyr_pre0.elf
FAILED: zephyr/zephyr_pre0.elf zephyr/zephyr_pre0.map
: && ccache /home/user/ncs/toolchains/7795df4459/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-g++ -gdwarf-4 -Wl,--wrap=malloc -Wl,--wrap=calloc -Wl,--wrap=realloc -Wl,--wrap=free -Wl,--wrap=_malloc_r -Wl,--wrap=_calloc_r -Wl,--wrap=_realloc_r -Wl,--wrap=_free_r zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj -o zephyr/zephyr_pre0.elf -fuse-ld=bfd -T zephyr/linker_zephyr_pre0.cmd -Wl,-Map=/home/user/BHT_repos/BHT-1-Sensor-Porting/rev_002_debug/zephyr/zephyr_pre0.map -Wl,--whole-archive app/libapp.a zephyr/libzephyr.a zephyr/arch/common/libarch__common.a zephyr/arch/arch/arm/core/aarch32/libarch__arm__core__aarch32.a zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a zephyr/arch/arch/arm/core/aarch32/mpu/libarch__arm__core__aarch32__mpu.a zephyr/lib/libc/newlib/liblib__libc__newlib.a zephyr/lib/libc/common/liblib__libc__common.a zephyr/lib/posix/liblib__posix.a zephyr/soc/soc/arm/common/cortex_m/libsoc__arm__common__cortex_m.a zephyr/soc/soc/arm/nordic_nrf/nrf52/libsoc__arm__nordic_nrf__nrf52.a zephyr/subsys/mgmt/mcumgr/mgmt/libsubsys__mgmt__mcumgr__mgmt.a zephyr/subsys/mgmt/mcumgr/smp/libsubsys__mgmt__mcumgr__smp.a zephyr/subsys/mgmt/mcumgr/util/libsubsys__mgmt__mcumgr__util.a zephyr/subsys/mgmt/mcumgr/grp/img_mgmt/libsubsys__mgmt__mcumgr__grp__img_mgmt.a zephyr/subsys/mgmt/mcumgr/grp/os_mgmt/libsubsys__mgmt__mcumgr__grp__os_mgmt.a zephyr/subsys/mgmt/mcumgr/grp/zephyr_basic/libsubsys__mgmt__mcumgr__grp__zephyr_basic.a zephyr/subsys/mgmt/mcumgr/transport/libsubsys__mgmt__mcumgr__transport.a zephyr/subsys/random/libsubsys__random.a zephyr/subsys/bluetooth/common/libsubsys__bluetooth__common.a zephyr/subsys/bluetooth/host/libsubsys__bluetooth__host.a zephyr/subsys/dfu/boot/libsubsys__dfu__boot.a zephyr/subsys/net/libsubsys__net.a zephyr/subsys/net/l2/openthread/libsubsys__net__ip__l2__openthread.a zephyr/subsys/net/ip/libsubsys__net__ip.a zephyr/drivers/adc/libdrivers__adc.a zephyr/drivers/clock_control/libdrivers__clock_control.a zephyr/drivers/console/libdrivers__console.a zephyr/drivers/entropy/libdrivers__entropy.a zephyr/drivers/flash/libdrivers__flash.a zephyr/drivers/gpio/libdrivers__gpio.a zephyr/drivers/hwinfo/libdrivers__hwinfo.a zephyr/drivers/i2c/libdrivers__i2c.a zephyr/drivers/ieee802154/libdrivers__ieee802154.a zephyr/drivers/pinctrl/libdrivers__pinctrl.a zephyr/drivers/power_domain/libdrivers__power_domain.a zephyr/drivers/sensor/sht4x/libdrivers__sensor__sht4x.a zephyr/drivers/serial/libdrivers__serial.a zephyr/drivers/timer/libdrivers__timer.a modules/nrf/lib/multithreading_lock/lib..__nrf__lib__multithreading_lock.a modules/nrf/lib/fprotect/lib..__nrf__lib__fprotect.a modules/nrf/subsys/bluetooth/controller/lib..__nrf__subsys__bluetooth__controller.a modules/nrf/subsys/nrf_security/src/zephyr/libmbedtls_zephyr.a modules/nrf/subsys/dfu/dfu_multi_image/lib..__nrf__subsys__dfu__dfu_multi_image.a modules/nrf/subsys/dfu/dfu_target/lib..__nrf__subsys__dfu__dfu_target.a modules/nrf/subsys/mpsl/init/lib..__nrf__subsys__mpsl__init.a modules/nrf/subsys/mpsl/fem/lib..__nrf__subsys__mpsl__fem.a modules/nrf/subsys/ieee802154/lib..__nrf__subsys__ieee802154.a modules/nrf/drivers/mpsl/clock_control/lib..__nrf__drivers__mpsl__clock_control.a modules/nrf/drivers/mpsl/flash_sync/lib..__nrf__drivers__mpsl__flash_sync.a modules/nrf/drivers/mpsl/temp_nrf5/lib..__nrf__drivers__mpsl__temp_nrf5.a modules/nrf/drivers/hw_cc310/lib..__nrf__drivers__hw_cc310.a modules/mcuboot/boot/bootutil/zephyr/libmcuboot_util.a modules/openthread/platform/libopenthread_platform.a modules/hal_nordic/nrf_802154/libnrf-802154-platform.a modules/nrfxlib/nrf_802154/nrf_802154/driver/libnrf-802154-driver.a modules/hal_nordic/nrfx/libmodules__hal_nordic__nrfx.a modules/zcbor/libmodules__zcbor.a modules/BHT-1-BSP_HAL/drivers/bht/lib..__..__..__BHT_repos__BHT-1-Sensor-Porting__BHT-1-BSP_HAL__drivers__bht.a modules/BHT-1-BSP_HAL/drivers/regulator_patch/lib..__..__..__BHT_repos__BHT-1-Sensor-Porting__BHT-1-BSP_HAL__drivers__regulator_patch.a modules/BHT-1-BSP_HAL/modules/lib..__..__..__BHT_repos__BHT-1-Sensor-Porting__BHT-1-BSP_HAL__modules.a /home/user/ncs/v2.5.0/nrfxlib/openthread/lib/cortex-m4/hard-float/v1.3/mtd/oberon/libopenthread-mtd.a -Wl,--no-whole-archive zephyr/kernel/libkernel.a zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj -L"/home/user/ncs/toolchains/7795df4459/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/thumb/v7e-m+fp/hard" -L/home/user/BHT_repos/BHT-1-Sensor-Porting/rev_002_debug/zephyr -lgcc -Wl,--start-group modules/connectedhomeip/lib/libCHIP.a modules/connectedhomeip/lib/libMatterDeviceInfoProviderExample.a -Wl,--end-group zephyr/arch/common/libisr_tables.a modules/nrfxlib/nrf_802154/nrf_802154/common/libnrf-802154-common.a modules/hal_nordic/nrf_802154/libnrf-802154-platform.a /home/user/ncs/v2.5.0/nrfxlib/nrf_802154/sl/sl/lib/nRF52840/hard-float/libnrf-802154-sl.a -Wl,--gc-sections -Wl,--build-id=none -Wl,--sort-common=descending -Wl,--sort-section=alignment -Wl,-u,_OffsetAbsSyms -Wl,-u,_ConfigAbsSyms -nostdlib -static -Wl,-X -Wl,-N -Wl,--orphan-handling=warn -lstdc++ -Wl,-no-pie -lm -Wl,-lc -L"/home/user/ncs/toolchains/7795df4459/opt/zephyr-sdk/arm-zephyr-eabi/arm-zephyr-eabi"/lib/thumb/v7e-m+fp/hard -Wl,-lgcc -lc -specs=nano.specs modules/nrf/subsys/nrf_security/src/libmbedtls.a modules/nrf/subsys/nrf_security/src/libmbedx509.a modules/nrf/subsys/nrf_security/src/libmbedcrypto.a /home/user/ncs/v2.5.0/nrfxlib/crypto/nrf_cc310_platform/lib/cortex-m4/hard-float/no-interrupts/libnrf_cc310_platform_0.9.18.a /home/user/ncs/v2.5.0/nrfxlib/crypto/nrf_oberon/lib/cortex-m4/hard-float/liboberon_mbedtls_3.0.13.a modules/nrf/subsys/nrf_security/src/libmbedcrypto_base.a -mcpu=cortex-m4 -mthumb -mabi=aapcs -mfpu=fpv4-sp-d16 -mfloat-abi=hard -mfp16-format=ieee -lc /home/user/ncs/v2.5.0/nrfxlib/crypto/nrf_oberon/lib/cortex-m4/hard-float/liboberon_3.0.13.a /home/user/ncs/v2.5.0/nrfxlib/softdevice_controller/lib/cortex-m4/hard-float/libsoftdevice_controller_peripheral.a /home/user/ncs/v2.5.0/nrfxlib/mpsl/fem/common/lib/cortex-m4/hard-float/libmpsl_fem_common.a /home/user/ncs/v2.5.0/nrfxlib/mpsl/fem/nrf21540_gpio/lib/cortex-m4/hard-float/libmpsl_fem_nrf21540_gpio.a /home/user/ncs/v2.5.0/nrfxlib/mpsl/fem/nrf21540_gpio_spi/lib/cortex-m4/hard-float/libmpsl_fem_nrf21540_gpio_spi.a /home/user/ncs/v2.5.0/nrfxlib/mpsl/fem/simple_gpio/lib/cortex-m4/hard-float/libmpsl_fem_simple_gpio.a /home/user/ncs/v2.5.0/nrfxlib/mpsl/lib/cortex-m4/hard-float/libmpsl.a && cd /home/user/BHT_repos/BHT-1-Sensor-Porting/rev_002_debug/zephyr && /home/user/ncs/toolchains/7795df4459/usr/local/bin/cmake -E true
/home/user/ncs/toolchains/7795df4459/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/libzephyr.a(sockets_tls.c.obj): in function `tls_release':
/home/user/ncs/v2.5.0/zephyr/subsys/net/lib/sockets/sockets_tls.c:519: undefined reference to `mbedtls_ssl_cookie_free'
/home/user/ncs/toolchains/7795df4459/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/libzephyr.a(sockets_tls.c.obj): in function `tls_mbedtls_init':
/home/user/ncs/v2.5.0/zephyr/subsys/net/lib/sockets/sockets_tls.c:1321: undefined reference to `mbedtls_ssl_cookie_setup'
/home/user/ncs/toolchains/7795df4459/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /home/user/ncs/v2.5.0/zephyr/subsys/net/lib/sockets/sockets_tls.c:1328: undefined reference to `mbedtls_ssl_conf_dtls_cookies'
/home/user/ncs/toolchains/7795df4459/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/libzephyr.a(sockets_tls.c.obj): in function `tls_set_own_cert':
/home/user/ncs/v2.5.0/zephyr/subsys/net/lib/sockets/sockets_tls.c:998: undefined reference to `mbedtls_ssl_cookie_write'
/home/user/ncs/toolchains/7795df4459/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /home/user/ncs/v2.5.0/zephyr/subsys/net/lib/sockets/sockets_tls.c:998: undefined reference to `mbedtls_ssl_cookie_check'
/home/user/ncs/toolchains/7795df4459/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/libzephyr.a(sockets_tls.c.obj): in function `tls_alloc':
/home/user/ncs/v2.5.0/zephyr/subsys/net/lib/sockets/sockets_tls.c:455: undefined reference to `mbedtls_ssl_cookie_init'
/home/user/ncs/toolchains/7795df4459/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/libzephyr.a(sockets_tls.c.obj): in function `dtls_rx':
/home/user/ncs/v2.5.0/zephyr/subsys/net/lib/sockets/sockets_tls.c:868: undefined reference to `mbedtls_ssl_set_client_transport_id'
/home/user/ncs/toolchains/7795df4459/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/libzephyr.a(debug_init.c.obj): in function `_mbedtls_init':
/home/user/ncs/v2.5.0/nrf/subsys/nrf_security/src/zephyr/debug_init.c:13: undefined reference to `mbedtls_debug_set_threshold'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: /home/user/ncs/toolchains/7795df4459/usr/local/bin/cmake --build /home/user/BHT_repos/BHT-1-Sensor-Porting/rev_002_debug

* The terminal process terminated with exit code: 1.
* Terminal will be reused by tasks, press any key to close it.

[Elfving]

Hello,

You can have a look at the migration guide for 2.5 to see if there is anything in particular that needs to be done (note for instance the new application VERSION file), besides that there is unfortunately no go-to way to migrate from one NCS version to another that won't result in any conflicts like these. You'll have to just go through these issues one at the time.

So after searching for one of these issues, I see that enabling CONFIG_MBEDTLS_DEBUG_C might be worth a try.

Regards,

Elfving

[Jacky Ortega]

Hi Elfving,

I already made the VERSION file migration. I also re-viewed the 'migration guide for 2.5' the funny thing is there are no changes related to 'matter' and 'mbedtls'. The same modules,packages and configurations are working fine on v2.4.1. 


Though I also found out that 'mgmt' module API and Callback mechanism were also changed for v2.5.0 and they are not documented in the 'migration guide'.

I am still confused even after enabling all the required 'configuration' flags it is still saying undefined functions for functionality related to use of mbedTLS.

The initial error as you can see from the logs above are pointing to the 'socket_TLS' file present as part of the v2.5.0 zephyr OS SDK.


Note:
The issue is still not resolved.

[Elfving]

Then I would recommend removing one module at the time to isolate the issues, and to see what needs to upgraded/migrated to something else. Migration from one NCS version to another can unfortunately be a bit cumbersome. Especially with multiple large modules like MCUBoot and matter.

Jacky Ortega said:

Though I also found out that 'mgmt' module API and Callback mechanism were also changed for v2.5.0 and they are not documented in the 'migration guide'.

I can add this to the migration guide, but note that not every change is documented.

Jacky Ortega said:

The initial error as you can see from the logs above are pointing to the 'socket_TLS' file present as part of the v2.5.0 zephyr OS SDK.

Seems like mbedtls isn't being linked correctly. Comparing your project to a default sample that uses mbedtls in NCS 2.5 might be useful.

MBEDTLS_ENTROPY_C, CONFIG_MBEDTLS_RSA_C, CONFIG_MBEDTLS_BUILTIN needing to be enabled is mentioned as a culprit in some other cases I've seen here. This entire case here with a similar situation might be interesting to you.

One other potential issue I should mention considering that you are using the '5340 is that some configurations needs to be enabled for the child image as well. If it is a brand new install of the SDK it is also good to make sure that default samples build on it, to make sure that you've indeed cloned the entire SDK (some people have issues installing it completely).

Regards,

Elfving