NCS Uses wrong python version when built with bootloader

Across several NRF Connect SDK versions, I have noticed that when built with the bootloader, the child image is built with the wrong environment and does not use the vendored installation. Here are some logs that the show that the bootloader and child image are built with different python versions when built in VS Code. The build fails because it uses a version of python that does not have the zephyr requirements installed when building the application child image.

My expected behavior would be that both images (mcuboot and b0 image) are built with the python installation under /opt/nordic/ncs/toolchains and the build succeeds regardless of the number of the other python installations.


The workaround I use is to either disable the bootloader or to remove python from my path (editing my shell environment and restarting VS Code).

This happens for sure with the NRF Connect for Desktop Toolchain Manager installation for versions v1.9.1 and v2.1.0. 

```

west build --build-dir /Users/jwtrueb/Desktop/workspace/nirs-monitor/build /Users/jwtrueb/Desktop/workspace/nirs-monitor --pristine --board nrf52840dk_nrf52840 -- -DNCS_TOOLCHAIN_VERSION:STRING="NONE" -DBOARD_ROOT:STRING="/Users/jwtrueb/Desktop/workspace/nirs-monitor;/Users/jwtrueb/Desktop/workspace/nirs-monitor/" -DDTC_OVERLAY_FILE:STRING="/Users/jwtrueb/Desktop/workspace/nirs-monitor/boards/nrf52840dk_nrf52840.overlay" -DCONF_FILE:STRING="/Users/jwtrueb/Desktop/workspace/nirs-monitor/prj.conf"

-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: /Users/jwtrueb/Desktop/workspace/nirs-monitor
-- Found Python3: /opt/nordic/ncs/toolchains/v2.1.0/bin/python3.9 (found suitable exact version "3.9.6") found components: Interpreter
-- Cache files will be written to: /Users/jwtrueb/Library/Caches/zephyr
-- Zephyr version: 3.1.99 (/opt/nordic/ncs/v2.1.0/zephyr)
-- Found west (found suitable version "0.14.0", minimum required is "0.7.1")
-- Board: nrf52840dk_nrf52840
-- Found host-tools: zephyr 0.14.1 (/opt/nordic/ncs/toolchains/v2.1.0/opt/zephyr-sdk)
-- Found dtc: /opt/nordic/ncs/toolchains/v2.1.0/bin/dtc (found suitable version "1.6.1", minimum required is "1.4.6")
-- Found toolchain: zephyr 0.14.1 (/opt/nordic/ncs/toolchains/v2.1.0/opt/zephyr-sdk)
-- Found BOARD.dts: /opt/nordic/ncs/v2.1.0/zephyr/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840.dts
-- Found devicetree overlay: /Users/jwtrueb/Desktop/workspace/nirs-monitor/boards/nrf52840dk_nrf52840.overlay
-- Generated zephyr.dts: /Users/jwtrueb/Desktop/workspace/nirs-monitor/build/zephyr/zephyr.dts
-- Generated devicetree_unfixed.h: /Users/jwtrueb/Desktop/workspace/nirs-monitor/build/zephyr/include/generated/devicetree_unfixed.h
-- Generated device_extern.h: /Users/jwtrueb/Desktop/workspace/nirs-monitor/build/zephyr/include/generated/device_extern.h
-- Including generated dts.cmake file: /Users/jwtrueb/Desktop/workspace/nirs-monitor/build/zephyr/dts.cmake
Parsing /Users/jwtrueb/Desktop/workspace/nirs-monitor/Kconfig
Loaded configuration '/opt/nordic/ncs/v2.1.0/zephyr/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840_defconfig'
Merged configuration '/Users/jwtrueb/Desktop/workspace/nirs-monitor/prj.conf'
Configuration saved to '/Users/jwtrueb/Desktop/workspace/nirs-monitor/build/zephyr/.config'
Kconfig header saved to '/Users/jwtrueb/Desktop/workspace/nirs-monitor/build/zephyr/include/generated/autoconf.h'
-- The C compiler identification is GNU 10.3.0
-- The CXX compiler identification is GNU 10.3.0
-- The ASM compiler identification is GNU
-- Found assembler: /opt/nordic/ncs/toolchains/v2.1.0/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc
-- Found Python3: /opt/nordic/ncs/toolchains/v2.1.0/bin/python3.9 (found version "3.9.6") found components: Interpreter

=== child image mcuboot - begin ===
loading initial cache file /Users/jwtrueb/Desktop/workspace/nirs-monitor/build/mcuboot/child_image_preload.cmake
Loading Zephyr default modules (Zephyr base).
-- Application: /opt/nordic/ncs/v2.1.0/bootloader/mcuboot/boot/zephyr
-- Found Python3: /Users/jwtrueb/miniforge3/bin/python3.10 (found suitable exact version "3.10.6") found components: Interpreter
-- Cache files will be written to: /Users/jwtrueb/Library/Caches/zephyr
-- Zephyr version: 3.1.99 (/opt/nordic/ncs/v2.1.0/zephyr)
-- Board: nrf52840dk_nrf52840
-- Found host-tools: zephyr 0.14.1 (/opt/nordic/ncs/toolchains/v2.1.0/opt/zephyr-sdk)
-- Found dtc: /opt/nordic/ncs/toolchains/v2.1.0/bin/dtc (found suitable version "1.6.1", minimum required is "1.4.6")
-- Found toolchain: zephyr 0.14.1 (/opt/nordic/ncs/toolchains/v2.1.0/opt/zephyr-sdk)
-- Found BOARD.dts: /opt/nordic/ncs/v2.1.0/zephyr/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840.dts
-- Found devicetree overlay: /opt/nordic/ncs/v2.1.0/bootloader/mcuboot/boot/zephyr/dts.overlay
Traceback (most recent call last):
File "/opt/nordic/ncs/v2.1.0/zephyr/scripts/dts/gen_defines.py", line 33, in <module>
from devicetree import edtlib
File "/opt/nordic/ncs/v2.1.0/zephyr/scripts/dts/python-devicetree/src/devicetree/edtlib.py", line 77, in <module>
import yaml
ModuleNotFoundError: No module named 'yaml'

```

I have a mac with the required python installation to use tensorflow, which installs via the Apple script that adds this to the end of ~/.zshrc. https://developer.apple.com/metal/tensorflow-plugin/

```

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/Users/jwtrueb/miniforge3/bin/conda' 'shell.zsh' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/Users/jwtrueb/miniforge3/etc/profile.d/conda.sh" ]; then
. "/Users/jwtrueb/miniforge3/etc/profile.d/conda.sh"
else
export PATH="/Users/jwtrueb/miniforge3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

```

To enable the bootloader, there is a zephyr sample. The key flags used to enable the bootloader build:

```
# Enable mcumgr.
CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_MCUMGR=y
CONFIG_MCUMGR_CMD_IMG_MGMT=y
CONFIG_MCUMGR_CMD_OS_MGMT=y
CONFIG_MCUMGR_SMP_BT=y
CONFIG_MCUMGR_SMP_BT_AUTHEN=n

```

Parents
  • Hi,

    Thank you for the detailed explanation and logs.

    Do you get the same if you build with a terminal opened via Toolchain Manager, or does this only happen in VS Code?

    The issue is the Python version selected by Zephyr, which is used by imgtool among others. Zephyr selects the first Python installation in path, unless the environment is set up correctly. This should be done automatically, but there seems to be a bug in VS Code or some of the other tools causing this to not be set correctly. If you can test with terminal it would be of great help in order to narrow down where the issue is.

    Best regards,

    Marte

  • I can only build with the NCS toolchain via VS Code.

    Nothing happens when I click Open Terminal in the NRF Connect for Desktop (v3.12.1)  Toolchain Manager (v1.2.0).

    There is an error in the logs of the Toolchain Manager



    10:46:11.474
    Initializing environments...
    10:46:11.546
    nrfutil-toolchain-manager 0.8.0 (92e632145e1f406f57a364e4214d7a4528af5a3a 2022-08-31)
    10:46:11.550
    Locally exsisting environment found at /opt/nordic/ncs/v1.9.1/toolchain
    10:46:11.550
    With version: v1.9.1
    10:46:11.550
    With west found: yes
    10:46:11.637
    Toolchain v2.1.0 has been added to the list
    10:46:11.637
    Toolchain v2.0.2 has been added to the list
    10:46:12.416
    Toolchain v2.0.0 has been added to the list
    10:46:12.417
    Toolchain v2.0.0-rc2 has been added to the list
    10:46:12.417
    Toolchain v2.0.1 has been added to the list
    10:46:12.419
    Toolchain v2.0.2 has been added to the list
    10:46:12.419
    Toolchain v2.1.0 has been added to the list
    10:46:12.420
    Toolchain v2.1.0-rc1 has been added to the list
    10:46:12.420
    Toolchain v2.1.0-rc2 has been added to the list
    10:46:12.937
    Failed to get the library versions: RangeError: Maximum call stack size exceeded
    10:46:13.501
    Toolchain v1.4.99-dev1 has been added to the list
    10:46:13.513
    Toolchain v1.4.1 has been added to the list
    10:46:13.519
    Toolchain v1.4.0 has been added to the list
    10:46:13.524
    Toolchain v1.3.2 has been added to the list
    10:46:13.622
    Toolchain v1.3.1 has been added to the list
    10:46:13.669
    Toolchain v1.3.0 has been added to the list
    10:46:13.687
    Toolchain v1.3.0-rc1 has been added to the list
    10:46:13.803
    Toolchain v1.4.2 has been added to the list
    10:46:13.821
    Toolchain v1.5.0-rc1 has been added to the list
    10:46:13.829
    Toolchain v1.5.0 has been added to the list
    10:46:13.832
    Toolchain v1.5.1-rc1 has been added to the list
    10:46:13.841
    Toolchain v1.5.1 has been added to the list
    10:46:13.847
    Toolchain v1.6.0 has been added to the list
    10:46:13.855
    Toolchain v1.6.1 has been added to the list
    10:46:13.909
    Toolchain v1.7.0 has been added to the list
    10:46:13.931
    Toolchain v1.7.1 has been added to the list
    10:46:13.934
    Toolchain v1.8.0 has been added to the list
    10:46:13.937
    Toolchain v1.9.0 has been added to the list
    10:46:13.941
    Toolchain v1.9.1 has been added to the list

    If I manually open a terminal, add the v2.1.0 toolchain to my PATH, source the zephyr-env.sh, set the SDK toolchain variant, and build with the build command from VS Code, the build fails due to CMAKE_PREFIX_PATH issues.



    (base) jwtrueb@dhcp-10-101-37-8 bin % which python3
    /opt/nordic/ncs/toolchains/v2.1.0/bin/python3
    (base) jwtrueb@dhcp-10-101-37-8 bin % which west /opt/nordic/ncs/toolchains/v2.1.0/bin/west

    (base) jwtrueb@dhcp-10-101-37-8 nirs-monitor % echo $ZEPHYR_BASE
    /opt/nordic/ncs/v2.1.0/zephyr
    (base) jwtrueb@dhcp-10-101-37-8 nirs-monitor % echo $ZEPHYR_SDK_INSTALL_DIR
    /opt/nordic/ncs/toolchains/v2.1.0
    (base) jwtrueb@dhcp-10-101-37-8 nirs-monitor % echo $ZEPHYR_TOOLCHAIN_VARIANT
    zephyr
    (base) jwtrueb@dhcp-10-101-37-8 nirs-monitor % west build --build-dir /Users/jwtrueb/Desktop/workspace/nirs-monitor/build /Users/jwtrueb/Desktop/workspace/nirs-monitor --pristine --board nrf52840dk_nrf52840 -- -DNCS_TOOLCHAIN_VERSION:STRING="NONE" -DBOARD_ROOT:STRING="/Users/jwtrueb/Desktop/workspace/nirs-monitor;/Users/jwtrueb/Desktop/workspace/nirs-monitor/" -DCONFIG_DEBUG_OPTIMIZATIONS:STRING="y" -DCONFIG_DEBUG_THREAD_INFO:STRING="y" -DCONF_FILE:STRING="/Users/jwtrueb/Desktop/workspace/nirs-monitor/prj.conf" -DDTC_OVERLAY_FILE:STRING="/Users/jwtrueb/Desktop/workspace/nirs-monitor/boards/nrf52840dk_nrf52840.overlay"
    -- west build: making build dir /Users/jwtrueb/Desktop/workspace/nirs-monitor/build pristine
    -- west build: generating a build system
    Loading Zephyr default modules (Zephyr base).
    -- Application: /Users/jwtrueb/Desktop/workspace/nirs-monitor
    -- Found Python3: /opt/nordic/ncs/toolchains/v2.1.0/bin/python3.9 (found suitable exact version "3.9.6") found components: Interpreter
    -- Cache files will be written to: /Users/jwtrueb/Library/Caches/zephyr
    -- Zephyr version: 3.1.99 (/opt/nordic/ncs/v2.1.0/zephyr)
    -- Found west (found suitable version "0.14.0", minimum required is "0.7.1")
    -- Board: nrf52840dk_nrf52840
    CMake Error at /opt/nordic/ncs/v2.1.0/zephyr/cmake/modules/verify-toolchain.cmake:79 (find_package):
    Could not find a package configuration file provided by "Zephyr-sdk"
    (requested version 0.13.1) with any of the following names:

    Zephyr-sdkConfig.cmake
    zephyr-sdk-config.cmake

    Add the installation prefix of "Zephyr-sdk" to CMAKE_PREFIX_PATH or set
    "Zephyr-sdk_DIR" to a directory containing one of the above files. If
    "Zephyr-sdk" provides a separate development package or SDK, be sure it has
    been installed.
    Call Stack (most recent call first):
    /opt/nordic/ncs/v2.1.0/zephyr/cmake/modules/zephyr_default.cmake:121 (include)
    /opt/nordic/ncs/v2.1.0/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:66 (include)
    /opt/nordic/ncs/v2.1.0/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:92 (include_boilerplate)
    CMakeLists.txt:12 (find_package)


    -- Configuring incomplete, errors occurred!
    FATAL ERROR: command exited with status 1: /opt/nordic/ncs/toolchains/v2.1.0/bin/cmake -DWEST_PYTHON=/opt/nordic/ncs/toolchains/v2.1.0/bin/python3 -B/Users/jwtrueb/Desktop/workspace/nirs-monitor/build -GNinja -DBOARD=nrf52840dk_nrf52840 -DNCS_TOOLCHAIN_VERSION:STRING=NONE '-DBOARD_ROOT:STRING=/Users/jwtrueb/Desktop/workspace/nirs-monitor;/Users/jwtrueb/Desktop/workspace/nirs-monitor/' -DCONFIG_DEBUG_OPTIMIZATIONS:STRING=y -DCONFIG_DEBUG_THREAD_INFO:STRING=y -DCONF_FILE:STRING=/Users/jwtrueb/Desktop/workspace/nirs-monitor/prj.conf -DDTC_OVERLAY_FILE:STRING=/Users/jwtrueb/Desktop/workspace/nirs-monitor/boards/nrf52840dk_nrf52840.overlay -S/Users/jwtrueb/Desktop/workspace/nirs-monitor
    (base) jwtrueb@dhcp-10-101-37-8 nirs-monitor %

    Is there documentation on how to use the NRF Connect SDK in a standalone terminal setting like this? To avoid CMake issues, is Open Terminal or VS Code the only supported way to build with the toolchain with the automatic installation? I know there is documentation on how to use west and whatnot, but I am more so referencing how to make sure that the NRF Connect SDK environment is installed and the terminal session has the shell environment setup to point to the right toolchain utilities. There is documentation on how to manually set everything up, but to my understanding some things were automatically installed by the Toolchain Manager in a way that doesn't match the manual installation documentation.

Reply
  • I can only build with the NCS toolchain via VS Code.

    Nothing happens when I click Open Terminal in the NRF Connect for Desktop (v3.12.1)  Toolchain Manager (v1.2.0).

    There is an error in the logs of the Toolchain Manager



    10:46:11.474
    Initializing environments...
    10:46:11.546
    nrfutil-toolchain-manager 0.8.0 (92e632145e1f406f57a364e4214d7a4528af5a3a 2022-08-31)
    10:46:11.550
    Locally exsisting environment found at /opt/nordic/ncs/v1.9.1/toolchain
    10:46:11.550
    With version: v1.9.1
    10:46:11.550
    With west found: yes
    10:46:11.637
    Toolchain v2.1.0 has been added to the list
    10:46:11.637
    Toolchain v2.0.2 has been added to the list
    10:46:12.416
    Toolchain v2.0.0 has been added to the list
    10:46:12.417
    Toolchain v2.0.0-rc2 has been added to the list
    10:46:12.417
    Toolchain v2.0.1 has been added to the list
    10:46:12.419
    Toolchain v2.0.2 has been added to the list
    10:46:12.419
    Toolchain v2.1.0 has been added to the list
    10:46:12.420
    Toolchain v2.1.0-rc1 has been added to the list
    10:46:12.420
    Toolchain v2.1.0-rc2 has been added to the list
    10:46:12.937
    Failed to get the library versions: RangeError: Maximum call stack size exceeded
    10:46:13.501
    Toolchain v1.4.99-dev1 has been added to the list
    10:46:13.513
    Toolchain v1.4.1 has been added to the list
    10:46:13.519
    Toolchain v1.4.0 has been added to the list
    10:46:13.524
    Toolchain v1.3.2 has been added to the list
    10:46:13.622
    Toolchain v1.3.1 has been added to the list
    10:46:13.669
    Toolchain v1.3.0 has been added to the list
    10:46:13.687
    Toolchain v1.3.0-rc1 has been added to the list
    10:46:13.803
    Toolchain v1.4.2 has been added to the list
    10:46:13.821
    Toolchain v1.5.0-rc1 has been added to the list
    10:46:13.829
    Toolchain v1.5.0 has been added to the list
    10:46:13.832
    Toolchain v1.5.1-rc1 has been added to the list
    10:46:13.841
    Toolchain v1.5.1 has been added to the list
    10:46:13.847
    Toolchain v1.6.0 has been added to the list
    10:46:13.855
    Toolchain v1.6.1 has been added to the list
    10:46:13.909
    Toolchain v1.7.0 has been added to the list
    10:46:13.931
    Toolchain v1.7.1 has been added to the list
    10:46:13.934
    Toolchain v1.8.0 has been added to the list
    10:46:13.937
    Toolchain v1.9.0 has been added to the list
    10:46:13.941
    Toolchain v1.9.1 has been added to the list

    If I manually open a terminal, add the v2.1.0 toolchain to my PATH, source the zephyr-env.sh, set the SDK toolchain variant, and build with the build command from VS Code, the build fails due to CMAKE_PREFIX_PATH issues.



    (base) jwtrueb@dhcp-10-101-37-8 bin % which python3
    /opt/nordic/ncs/toolchains/v2.1.0/bin/python3
    (base) jwtrueb@dhcp-10-101-37-8 bin % which west /opt/nordic/ncs/toolchains/v2.1.0/bin/west

    (base) jwtrueb@dhcp-10-101-37-8 nirs-monitor % echo $ZEPHYR_BASE
    /opt/nordic/ncs/v2.1.0/zephyr
    (base) jwtrueb@dhcp-10-101-37-8 nirs-monitor % echo $ZEPHYR_SDK_INSTALL_DIR
    /opt/nordic/ncs/toolchains/v2.1.0
    (base) jwtrueb@dhcp-10-101-37-8 nirs-monitor % echo $ZEPHYR_TOOLCHAIN_VARIANT
    zephyr
    (base) jwtrueb@dhcp-10-101-37-8 nirs-monitor % west build --build-dir /Users/jwtrueb/Desktop/workspace/nirs-monitor/build /Users/jwtrueb/Desktop/workspace/nirs-monitor --pristine --board nrf52840dk_nrf52840 -- -DNCS_TOOLCHAIN_VERSION:STRING="NONE" -DBOARD_ROOT:STRING="/Users/jwtrueb/Desktop/workspace/nirs-monitor;/Users/jwtrueb/Desktop/workspace/nirs-monitor/" -DCONFIG_DEBUG_OPTIMIZATIONS:STRING="y" -DCONFIG_DEBUG_THREAD_INFO:STRING="y" -DCONF_FILE:STRING="/Users/jwtrueb/Desktop/workspace/nirs-monitor/prj.conf" -DDTC_OVERLAY_FILE:STRING="/Users/jwtrueb/Desktop/workspace/nirs-monitor/boards/nrf52840dk_nrf52840.overlay"
    -- west build: making build dir /Users/jwtrueb/Desktop/workspace/nirs-monitor/build pristine
    -- west build: generating a build system
    Loading Zephyr default modules (Zephyr base).
    -- Application: /Users/jwtrueb/Desktop/workspace/nirs-monitor
    -- Found Python3: /opt/nordic/ncs/toolchains/v2.1.0/bin/python3.9 (found suitable exact version "3.9.6") found components: Interpreter
    -- Cache files will be written to: /Users/jwtrueb/Library/Caches/zephyr
    -- Zephyr version: 3.1.99 (/opt/nordic/ncs/v2.1.0/zephyr)
    -- Found west (found suitable version "0.14.0", minimum required is "0.7.1")
    -- Board: nrf52840dk_nrf52840
    CMake Error at /opt/nordic/ncs/v2.1.0/zephyr/cmake/modules/verify-toolchain.cmake:79 (find_package):
    Could not find a package configuration file provided by "Zephyr-sdk"
    (requested version 0.13.1) with any of the following names:

    Zephyr-sdkConfig.cmake
    zephyr-sdk-config.cmake

    Add the installation prefix of "Zephyr-sdk" to CMAKE_PREFIX_PATH or set
    "Zephyr-sdk_DIR" to a directory containing one of the above files. If
    "Zephyr-sdk" provides a separate development package or SDK, be sure it has
    been installed.
    Call Stack (most recent call first):
    /opt/nordic/ncs/v2.1.0/zephyr/cmake/modules/zephyr_default.cmake:121 (include)
    /opt/nordic/ncs/v2.1.0/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:66 (include)
    /opt/nordic/ncs/v2.1.0/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:92 (include_boilerplate)
    CMakeLists.txt:12 (find_package)


    -- Configuring incomplete, errors occurred!
    FATAL ERROR: command exited with status 1: /opt/nordic/ncs/toolchains/v2.1.0/bin/cmake -DWEST_PYTHON=/opt/nordic/ncs/toolchains/v2.1.0/bin/python3 -B/Users/jwtrueb/Desktop/workspace/nirs-monitor/build -GNinja -DBOARD=nrf52840dk_nrf52840 -DNCS_TOOLCHAIN_VERSION:STRING=NONE '-DBOARD_ROOT:STRING=/Users/jwtrueb/Desktop/workspace/nirs-monitor;/Users/jwtrueb/Desktop/workspace/nirs-monitor/' -DCONFIG_DEBUG_OPTIMIZATIONS:STRING=y -DCONFIG_DEBUG_THREAD_INFO:STRING=y -DCONF_FILE:STRING=/Users/jwtrueb/Desktop/workspace/nirs-monitor/prj.conf -DDTC_OVERLAY_FILE:STRING=/Users/jwtrueb/Desktop/workspace/nirs-monitor/boards/nrf52840dk_nrf52840.overlay -S/Users/jwtrueb/Desktop/workspace/nirs-monitor
    (base) jwtrueb@dhcp-10-101-37-8 nirs-monitor %

    Is there documentation on how to use the NRF Connect SDK in a standalone terminal setting like this? To avoid CMake issues, is Open Terminal or VS Code the only supported way to build with the toolchain with the automatic installation? I know there is documentation on how to use west and whatnot, but I am more so referencing how to make sure that the NRF Connect SDK environment is installed and the terminal session has the shell environment setup to point to the right toolchain utilities. There is documentation on how to manually set everything up, but to my understanding some things were automatically installed by the Toolchain Manager in a way that doesn't match the manual installation documentation.

Children
No Data
Related