• State Not Answered
  • Replies 5 replies
  • Subscribers 74 subscribers
  • Views 533 views
  • Users 0 members are here
Attachments (0)
Nordic Case Info
  • Case ID: 323083
Options

Apple M1, SDK v2.5.2, toolchain-supplied Python uses system-installed site-packages

Mikael Jansson

MacBook M1. Latest macOS. SDK v2.5.2. Building blinky w/ CONFIG_BOOTLOADER_MCUBOOT=1 enabled.

Doesn't matter if I start an nrfutil shell* or my normal settings.

* nrfutil toolchain-manager launch --shell --ncs-version v2.5.0

$ west build  -b circuitdojo_feather_nrf9160_ns
[0/222] Performing build step for 'tfm'
[4/8] Linking C executable bin/tfm_s.axf
Memory region         Used Size  Region Size  %age Used
           FLASH:       31544 B      32256 B     97.79%
             RAM:       18832 B        32 KB     57.47%
[8/8] Generating ../bin/tfm_s.elf
[1/222] Performing build step for 'mcuboot_subimage'
[18/124] Generating zephyr/autogen-pubkey.c
FAILED: zephyr/autogen-pubkey.c
cd .../nrf-sdk-test-2024-02-28/blinky/build/mcuboot && /opt/nordic/ncs/toolchains/20d68df7e5/bin/python3 /opt/nordic/ncs/v2.5.2/bootloader/mcuboot/scripts/imgtool.py getpub -k /opt/nordic/ncs/v2.5.2/bootloader/mcuboot/root-ec-p256.pem > .../nrf-sdk-test-2024-02-28/blinky/build/mcuboot/zephyr/autogen-pubkey.c
Traceback (most recent call last):
  File "/opt/nordic/ncs/v2.5.2/bootloader/mcuboot/scripts/imgtool.py", line 19, in <module>
    from imgtool import main
  File "/opt/nordic/ncs/v2.5.2/bootloader/mcuboot/scripts/imgtool/main.py", line 26, in <module>
    from imgtool import image, imgtool_version
  File "/opt/nordic/ncs/v2.5.2/bootloader/mcuboot/scripts/imgtool/image.py", line 35, in <module>
    from cryptography.hazmat.primitives.kdf.hkdf import HKDF
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/cryptography/hazmat/primitives/kdf/hkdf.py", line 13, in <module>
    from cryptography.hazmat.primitives import constant_time, hashes, hmac
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/cryptography/hazmat/primitives/hmac.py", line 12, in <module>
    from cryptography.hazmat.backends.openssl.hmac import _HMACContext
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/cryptography/hazmat/backends/openssl/__init__.py", line 6, in <module>
    from cryptography.hazmat.backends.openssl.backend import backend
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 63, in <module>
    from cryptography.hazmat.bindings.openssl import binding
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 14, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: dlopen(/Users/me/Library/Python/3.9/lib/python/site-packages/_cffi_backend.cpython-39-darwin.so, 0x0002): tried: '/Users/me/Library/Python/3.9/lib/python/site-packages/_cffi_backend.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/me/Library/Python/3.9/lib/python/site-packages/_cffi_backend.cpython-39-darwin.so' (no such file), '/Users/me/Library/Python/3.9/lib/python/site-packages/_cffi_backend.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64'))
[29/124] Building C object zephyr/CMakeFiles/zephyr.dir/lib/os/cbprintf_nano.c.obj
ninja: build stopped: subcommand failed.
[2/222] Performing install step for 'tfm'
-- Install configuration: "RelWithDebInfo"
[36/222] Building C object modules/nrf/subsys/nrf_security/src/CMakeFiles/mbedcrypto.dir/opt/nordic/ncs/v2.5.2/modules/crypto/mbedtls/library/gcm.c.obj
FAILED: modules/mcuboot/mcuboot_subimage-prefix/src/mcuboot_subimage-stamp/mcuboot_subimage-build mcuboot/zephyr/zephyr.hex mcuboot/zephyr/zephyr.elf
cd .../nrf-sdk-test-2024-02-28/blinky/build/mcuboot && /opt/nordic/ncs/toolchains/20d68df7e5/Cellar/cmake/3.20.5/bin/cmake --build . --
[48/222] Building C object modules/nrf/subsys/nrf_security/src/CMakeFiles/mbedcrypto.dir/opt/nordic/ncs/v2.5.2/modules/crypto/mbedtls/library/gcm.c.obj
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: /opt/nordic/ncs/toolchains/20d68df7e5/bin/cmake --build .../nrf-sdk-test-2024-02-28/blinky/build
$

The provided Python is x86_64, my system is aarch64.

As you can see, the build system uses its own Python, but links to my system files.

The provided toolchain Python should use its own site-packges.

This is a clean install of v2.5.2 from nRF Connect Desktop.


Thanks in advance,
- Mikael

Parents
  • 0

    Mikael,

    Your workaround shows that your toolchain installation somehow using the global PYTHONPATH which it ideally should not.

    Here are at least some observarions: 

    • The error trace from Python shows that the system installation of some python libraries are being used at one point
    • Ideally, our python installation should never need to interact with the system installation of Python for anything. 
    • We have had issues with the cmake cache before, so trying to clear up interactions with cmake may help, but this might not be related to the issue you are seeing.

    But it could be helpful to see what environment variables there are and if you are affecting the system. Can you run

     

    echo "$PATH"
echo "$PYTHONHOME"
echo "$PYTHONPATH"

    We would also suggest upgrading to version 2.5.2, as some bugs may have been fixed by that. 

    Finally, running 

    • which -a python should show all python installations and if one form inside the toolchain bundle is currently being used
    • pip list -vvv should show where the different pip packages are installed, and therefore which ones may cause issues
      • Here, you will see some issues if you have globally installed packages that you try to use that have not been installed inside the toolchain bundle
Reply
  • 0

    Mikael,

    Your workaround shows that your toolchain installation somehow using the global PYTHONPATH which it ideally should not.

    Here are at least some observarions: 

    • The error trace from Python shows that the system installation of some python libraries are being used at one point
    • Ideally, our python installation should never need to interact with the system installation of Python for anything. 
    • We have had issues with the cmake cache before, so trying to clear up interactions with cmake may help, but this might not be related to the issue you are seeing.

    But it could be helpful to see what environment variables there are and if you are affecting the system. Can you run

     

    echo "$PATH"
echo "$PYTHONHOME"
echo "$PYTHONPATH"

    We would also suggest upgrading to version 2.5.2, as some bugs may have been fixed by that. 

    Finally, running 

    • which -a python should show all python installations and if one form inside the toolchain bundle is currently being used
    • pip list -vvv should show where the different pip packages are installed, and therefore which ones may cause issues
      • Here, you will see some issues if you have globally installed packages that you try to use that have not been installed inside the toolchain bundle
Children
  • 0 in reply to Susheel Nuguru

    Hi,

    This is on v2.5.2.

    PYTHONPATH and PYTHONHOME are empty. The default is for Python to construct it based on the location of the binary.

    PATH=/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin

    which -a python3 gives /usr/bin/python3, i.e. the macOS system-installed python3. Note that its python3, not python. That's reserved for python2.x, which NCS doesnt use.

    It's really easy to test this. Just install the offending crypto libraries that use cffi on your system using pip3 install <libraries>, then try building mcuboot using the ncs toolchain.

Related
Terms of service