Can't install nRF Connect v1.7.0 on macOS Big Sur 11.5 with Python 3.9.6

[Mike Wirth]

Kind folks,

The more things change, the more they stay the same .  At lest it seems that way.  I submitted a similar message seven months ago. I see a recent spate of related messages from other Mac users having nRF Connect installation issues.  Here's my current assessment of the situation with the automatic installation process:

• I tried using the latest update of the nRF Connect app v3.7.1 using the Toolchain Manager v0.9.4 plugin, clicking on nRF Connect SDK v1.7.0 "Install".  This process downloaded the SDK and toolchain files but then halted, without installing the toolchain (e.g., the SEGGER Embedded Studio app).  
• (Documentation note: At this point, the "Install" has changed to "First steps to build". If you single click on this field, it opens a page of instructions for using SES.  Wrong move! it appears that the proper course of action is to click on the downward-pointing triangle to the right, which acts like a 5 element selection, which in fact isn't a selector, but apparently an indication of the five steps to necessary to "get ready" to build with SES.  Very bad UI, ambiguous and misleading!! Would hope your UI designers could come up with something better.
• In any case, on my system, I can contrive to install SES so that the app can be launched in normal mcOS fashion.
• Somewhere in this chain, I got a "CMake not installed" popup.  No sweat.  Took care of that with brew.
• But this process eventually fails, during the toolchain update process, with the following error popup:
• 
  

Gave up on the automatic installation process and went over to your webpage for manual installation of SDK v1.7.0:

• Turns out this was essentially a repeat of of my experience of seven months ago (reported in the message linked above) ending in problems with west issues.

Core Issue: Python version management on macOS

I have come to the conclusion that all of these problems (as well as many of those reported by other Mac users) are due to problematic issues with Python versions on recent macOS versions up through Big Sur (and probably Monterey).  Here are several issues and questions related to this issue:

• Recent macOS releases have come with Python 2.7 and Python 3.6 versions preinstalled and linked to "python" and "python3" respectively.  These installed binaries are in a Read Only section of the system and cannot be changed.
• Additional versions can be installed, but various tricks need to be employed to get them to be used instead of the pre-installed versions
• In particular, virtual Python environments can be setup using the venv tool for Python or the more general asdf tool.
• But this can be problematic if we can't ensure that the same virtual environment is used by both from both the command line and by script handling internal to your code.

So as a step toward solving my and other Mac users' problems with SDK v1.7.0 installation, both automatic and manual, can you please provide us with the following information (which I haven't found in any of your documentation):

• The exact Mac configuration used by your developers to test and qualify SDK v1.7.0 installation, both automatic and manual:
  
  
  • Machine type: e.g., MacBook Pro 10,1 (probably not critical, earliest and latest machine types would be informative)
  • macOSmacOS versions used: earliest and latest
  • Exact software configuration on test Macs prior to SDK v1.7.0 installation, both automatic and manual, especially with regard to:
    • Python installations
    • Virtual environment tools and their use 
    • Any Linux (via brew) or Python packages (via pip) installed and not mentioned in your pre-requisites
• Any suggestions you have for qualification tests that can be used after each substantial step (in the manual installation) to validate proper completion of the prior step.

Note: As a former software development manager of a dozen person team, part of a 150 person engineering group that develops software that runs on the vast majority of PCs, Macs and Linux machines in the world, I appreciate the difficulty your team has.  It's very difficult to anticipate what environments will be encountered by your software, especially automated installation tools. (Sometimes you even have to start with a clean out-of-the-box machines.)  If you can get me answers to the above questions and, better yet, setup a channel for communication with your macOS team, I'll be happy to contribute effort on macOS testing and UI design.

Hoping we can get the installation and use of your software on Macs at least as smooth and easy as the Windows version.

TIA,

Mike

[Mike Wirth]

Hi Torbjørn,

Thanks for the quick reply and suggestions.  Here's my followup:

1. I have been using the latest SDK, 1.7.0.
2. I'm including the log file from my nFR SDK installation process from a couple days ago (2021-09-26T06_50_33.261Z-log.txt).  May or may not be useful given the items below.
3. Since my last update, I installed a new version of Python, 3.9.7, and updated my asdf virtual environment settings. My current asdf settings and PATH variable are detailed in the terminal log, Current asdf settings and PATH,txt.  This will prove to be critical.
4. I then reinstalled the nRF Connect app, v3.7.1, updated all the plugins, including Toolchain Manager, v0.10.0.
5. Removed the current SDK 1.7.0 installation.
6. Finally, started the automatic installation of SDK 1.7.0, which executed and completed without error!
7. Note at this point, that I have included two files to document this process,
   1. The log file, 2021-09-28T23_33_27.406Z-log.txt, as you requested
   2. A PDF file, documenting the process, nRF Connect Execution Notes.pdf, containing my notes, popups, etc., during the process.
8. As requested, I installed Visual Studio Code (actually updated it) and verified that my PATH included it
9. I then tested the buttons created in the Install SDK 1.7.0 box at the completion of the installation process.  Some errors were encountered. Here's a summary of my results (see the PDF file for details):
   1. First steps:
      1. Open VS Code:  Fails (see below for conflicting PATH values)
      2. Open SES: Works
   2. Selection Triangle:
      1. Open terminal: Opens but with a incomplete PATH list (see below)
      2. Open SDK directory: Works
      3. Open toolchain directory: Works
      4. Update SDK: Works
      5. Update toolchain: Fails with two errors: (see PDF notes) 
         1. "Inconsistent directory structure" and
         2. "Error, Cannot copy..."

Analysis:

1. My installation of Python 3.9.7 somehow cleared the errors I was having with west before.  Not sure how, but it would be nice to know how so we could avoid future problems.
2. The current errors noted above appear to be relate to different PATH variables in your "SDK 1.7.0" command line environment (see PATH as setup by Install SDK 1.7.0.txt file) and my normal terminal environment (see my "Current .zshrc.txt" and "Current asdf settings and PATH.txt" files

Hope all this info helps you find the remaining problems.  Will be happy to test further based on your response.

Mike

[ovrebekk]

Hi Mike

For some reason I am unable to load any of your attachments. I will check with one of the devzone admins if this is an issue on our end, and try to get back to you on your comments later in the week. 

Best regards
Torbjørn

[ovrebekk]

Hi again

He was unable to find the attachments as well. Are you able to try and upload them one more time?

For simplicity you could just zip them all into one file. 

Apparently the developers have seen similar issues with Python on Mac, and the solution in that case was to run the following command (in the shell spawned through the Toolchain Manager):

cmake -P toolchain/cmake/NcsToolchainConfig.cmake

The odd thing is that this should be done automatically when you install the SDK through the Toolchain Manager. 

In order to get started with VSCode right away I would recommend this blog, in case you still can't get the Toolchain Manager integration to work:
https://devzone.nordicsemi.com/nordic/nordic-blog/b/blog/posts/nrf-connect-for-visual-studio-code-preview

Best regards
Torbjørn

[Mike Wirth]

Torbjørn,

Here are the files from my last message compressed into a zip archive.  If you still have problems downloading and uncompressing them, I'll put them in a public Dropbox for you.

Will test and respond to your west and VSCode suggestions in a couple hours.

Thanks,

Mike

[Mike Wirth]

Torbjørn,

I tried downloading the zip. Failed.  (Blanks "%20" in URL?) Here's a private link to the zip file in Google Disk.  Hope this works for you:

• https://drive.google.com/file/d/1VYak21oHEmcar1syPdT2DvzDQVFCXStO/view?usp=sharing

I tested it.  Worked for me (but that's no guarantee for you :-)

Mike

[Mike Wirth]

Torbjørn,

You suggested I try this in a shell launched from TM:

ovrebekk said:

cmake -P toolchain/cmake/NcsToolchainConfig.cmake

Here's the log:

Last login: Thu Sep 30 17:19:43 on ttys000
cd /opt/nordic/ncs/v1.7.0 ; export PATH=/opt/nordic/ncs/v1.7.0/toolchain/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin ; export GIT_EXEC_PATH=/opt/nordic/ncs/v1.7.0/toolchain/Cellar/git/2.32.0_1/libexec/git-core ; export ZEPHYR_TOOLCHAIN_VARIANT=gnuarmemb ; export GNUARMEMB_TOOLCHAIN_PATH=/opt/nordic/ncs/v1.7.0/toolchain ; clear
mike@MBP101 ~ % cd /opt/nordic/ncs/v1.7.0 ; export PATH=/opt/nordic/ncs/v1.7.0/toolchain/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin ; export GIT_EXEC_PATH=/opt/nordic/ncs/v1.7.0/toolchain/Cellar/git/2.32.0_1/libexec/git-core ; export ZEPHYR_TOOLCHAIN_VARIANT=gnuarmemb ; export GNUARMEMB_TOOLCHAIN_PATH=/opt/nordic/ncs/v1.7.0/toolchain ; clear

...

mike@MBP101 v1.7.0 % date
Thu Sep 30 17:19:54 PDT 2021
...
mike@MBP101 v1.7.0 % python -V
Python 2.7.16
mike@MBP101 v1.7.0 % python3 -V
Python 3.9.6
mike@MBP101 v1.7.0 % cmake -P toolchain/cmake/NcsToolchainConfig.cmake
mike@MBP101 v1.7.0 %

cmake ran instantaneously.  Suspected it just verified previously execution.

Didn't change anything.  Still got the following error ( #2 only) from invoking "Update toolchain" as before.

Mike Wirth said:

Update toolchain: Fails with two errors: (see PDF notes) 

1. "Inconsistent directory structure" and
2. "Error, Cannot copy..."

Will try the VS Code installation process link you provided ASAP.

Thanks,

Mike

[ovrebekk]

Hi Mike

I sent a request to get access to the drive folder. 

Did you get it?

Best regards
Torbjørn

[Mike Wirth]

Sorry, Torbjørn, didn't get your request.  But noticed that I probably set the Google permissions wrong for access to the zip archive by you.  Here's a new link, which, if I got it right, will allow access to anyone with the link:

https://drive.google.com/file/d/1VYak21oHEmcar1syPdT2DvzDQVFCXStO/view?usp=sharing

Hope this helps,

Mike

[ovrebekk]

Hi Mike

Thanks for the link, it seems now I can access the files both places. 

I have shared the files with the developers and asked them to take a look. Whatever feedback they have I will make sure to forward to you. 

Best regards
Torbjørn