Apple M1, SDK v2.5.2, toolchain-supplied Python uses system-installed site-packages

[Mikael Jansson]

MacBook M1. Latest macOS. SDK v2.5.2. Building blinky w/ CONFIG_BOOTLOADER_MCUBOOT=1 enabled.

Doesn't matter if I start an nrfutil shell* or my normal settings.

* nrfutil toolchain-manager launch --shell --ncs-version v2.5.0

$ west build  -b circuitdojo_feather_nrf9160_ns
[0/222] Performing build step for 'tfm'
[4/8] Linking C executable bin/tfm_s.axf
Memory region         Used Size  Region Size  %age Used
           FLASH:       31544 B      32256 B     97.79%
             RAM:       18832 B        32 KB     57.47%
[8/8] Generating ../bin/tfm_s.elf
[1/222] Performing build step for 'mcuboot_subimage'
[18/124] Generating zephyr/autogen-pubkey.c
FAILED: zephyr/autogen-pubkey.c
cd .../nrf-sdk-test-2024-02-28/blinky/build/mcuboot && /opt/nordic/ncs/toolchains/20d68df7e5/bin/python3 /opt/nordic/ncs/v2.5.2/bootloader/mcuboot/scripts/imgtool.py getpub -k /opt/nordic/ncs/v2.5.2/bootloader/mcuboot/root-ec-p256.pem > .../nrf-sdk-test-2024-02-28/blinky/build/mcuboot/zephyr/autogen-pubkey.c
Traceback (most recent call last):
  File "/opt/nordic/ncs/v2.5.2/bootloader/mcuboot/scripts/imgtool.py", line 19, in <module>
    from imgtool import main
  File "/opt/nordic/ncs/v2.5.2/bootloader/mcuboot/scripts/imgtool/main.py", line 26, in <module>
    from imgtool import image, imgtool_version
  File "/opt/nordic/ncs/v2.5.2/bootloader/mcuboot/scripts/imgtool/image.py", line 35, in <module>
    from cryptography.hazmat.primitives.kdf.hkdf import HKDF
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/cryptography/hazmat/primitives/kdf/hkdf.py", line 13, in <module>
    from cryptography.hazmat.primitives import constant_time, hashes, hmac
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/cryptography/hazmat/primitives/hmac.py", line 12, in <module>
    from cryptography.hazmat.backends.openssl.hmac import _HMACContext
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/cryptography/hazmat/backends/openssl/__init__.py", line 6, in <module>
    from cryptography.hazmat.backends.openssl.backend import backend
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 63, in <module>
    from cryptography.hazmat.bindings.openssl import binding
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 14, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: dlopen(/Users/me/Library/Python/3.9/lib/python/site-packages/_cffi_backend.cpython-39-darwin.so, 0x0002): tried: '/Users/me/Library/Python/3.9/lib/python/site-packages/_cffi_backend.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/me/Library/Python/3.9/lib/python/site-packages/_cffi_backend.cpython-39-darwin.so' (no such file), '/Users/me/Library/Python/3.9/lib/python/site-packages/_cffi_backend.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64'))
[29/124] Building C object zephyr/CMakeFiles/zephyr.dir/lib/os/cbprintf_nano.c.obj
ninja: build stopped: subcommand failed.
[2/222] Performing install step for 'tfm'
-- Install configuration: "RelWithDebInfo"
[36/222] Building C object modules/nrf/subsys/nrf_security/src/CMakeFiles/mbedcrypto.dir/opt/nordic/ncs/v2.5.2/modules/crypto/mbedtls/library/gcm.c.obj
FAILED: modules/mcuboot/mcuboot_subimage-prefix/src/mcuboot_subimage-stamp/mcuboot_subimage-build mcuboot/zephyr/zephyr.hex mcuboot/zephyr/zephyr.elf
cd .../nrf-sdk-test-2024-02-28/blinky/build/mcuboot && /opt/nordic/ncs/toolchains/20d68df7e5/Cellar/cmake/3.20.5/bin/cmake --build . --
[48/222] Building C object modules/nrf/subsys/nrf_security/src/CMakeFiles/mbedcrypto.dir/opt/nordic/ncs/v2.5.2/modules/crypto/mbedtls/library/gcm.c.obj
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: /opt/nordic/ncs/toolchains/20d68df7e5/bin/cmake --build .../nrf-sdk-test-2024-02-28/blinky/build
$

The provided Python is x86_64, my system is aarch64.

As you can see, the build system uses its own Python, but links to my system files.

The provided toolchain Python should use its own site-packges.

This is a clean install of v2.5.2 from nRF Connect Desktop.

Thanks in advance,
- Mikael

[Susheel Nuguru]

Thanks for the report Michael,

I do not have Mac to test, but I can forward this to the toolchain team and get you some updates on what can be done. Be back to you when I hear from them.

[Mikael Jansson]

Thanks Sushi.

For other readers, there is a *workaround* set PYTHONPATH=/opt/nordic/ncs/toolchains/20d68df7e5/lib/python3/site-packages before running west build. You do _not_ want to have this globally always set, because it'll mess all your other system apps using Python code up.

Obviously, this is not an acceptable solution, but it allows you to resume your work.