Can't install nRF Connect v1.7.0 on macOS Big Sur 11.5 with Python 3.9.6

[Mike Wirth]

Kind folks,

The more things change, the more they stay the same .  At lest it seems that way.  I submitted a similar message seven months ago. I see a recent spate of related messages from other Mac users having nRF Connect installation issues.  Here's my current assessment of the situation with the automatic installation process:

• I tried using the latest update of the nRF Connect app v3.7.1 using the Toolchain Manager v0.9.4 plugin, clicking on nRF Connect SDK v1.7.0 "Install".  This process downloaded the SDK and toolchain files but then halted, without installing the toolchain (e.g., the SEGGER Embedded Studio app).  
• (Documentation note: At this point, the "Install" has changed to "First steps to build". If you single click on this field, it opens a page of instructions for using SES.  Wrong move! it appears that the proper course of action is to click on the downward-pointing triangle to the right, which acts like a 5 element selection, which in fact isn't a selector, but apparently an indication of the five steps to necessary to "get ready" to build with SES.  Very bad UI, ambiguous and misleading!! Would hope your UI designers could come up with something better.
• In any case, on my system, I can contrive to install SES so that the app can be launched in normal mcOS fashion.
• Somewhere in this chain, I got a "CMake not installed" popup.  No sweat.  Took care of that with brew.
• But this process eventually fails, during the toolchain update process, with the following error popup:
• 
  

Gave up on the automatic installation process and went over to your webpage for manual installation of SDK v1.7.0:

• Turns out this was essentially a repeat of of my experience of seven months ago (reported in the message linked above) ending in problems with west issues.

Core Issue: Python version management on macOS

I have come to the conclusion that all of these problems (as well as many of those reported by other Mac users) are due to problematic issues with Python versions on recent macOS versions up through Big Sur (and probably Monterey).  Here are several issues and questions related to this issue:

• Recent macOS releases have come with Python 2.7 and Python 3.6 versions preinstalled and linked to "python" and "python3" respectively.  These installed binaries are in a Read Only section of the system and cannot be changed.
• Additional versions can be installed, but various tricks need to be employed to get them to be used instead of the pre-installed versions
• In particular, virtual Python environments can be setup using the venv tool for Python or the more general asdf tool.
• But this can be problematic if we can't ensure that the same virtual environment is used by both from both the command line and by script handling internal to your code.

So as a step toward solving my and other Mac users' problems with SDK v1.7.0 installation, both automatic and manual, can you please provide us with the following information (which I haven't found in any of your documentation):

• The exact Mac configuration used by your developers to test and qualify SDK v1.7.0 installation, both automatic and manual:
  
  
  • Machine type: e.g., MacBook Pro 10,1 (probably not critical, earliest and latest machine types would be informative)
  • macOSmacOS versions used: earliest and latest
  • Exact software configuration on test Macs prior to SDK v1.7.0 installation, both automatic and manual, especially with regard to:
    • Python installations
    • Virtual environment tools and their use 
    • Any Linux (via brew) or Python packages (via pip) installed and not mentioned in your pre-requisites
• Any suggestions you have for qualification tests that can be used after each substantial step (in the manual installation) to validate proper completion of the prior step.

Note: As a former software development manager of a dozen person team, part of a 150 person engineering group that develops software that runs on the vast majority of PCs, Macs and Linux machines in the world, I appreciate the difficulty your team has.  It's very difficult to anticipate what environments will be encountered by your software, especially automated installation tools. (Sometimes you even have to start with a clean out-of-the-box machines.)  If you can get me answers to the above questions and, better yet, setup a channel for communication with your macOS team, I'll be happy to contribute effort on macOS testing and UI design.

Hoping we can get the installation and use of your software on Macs at least as smooth and easy as the Windows version.

TIA,

Mike

[Mike Wirth]

So I note that the SDK v1.7.0 contains its own Python interpreter at:

• /opt/nordic/ncs/v1.7.0/toolchain/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/bin/python3.9

which is Python version 3.9.6.  Clearly the intent is that this interpreter is used for all Python scripts inside the SDK installation process.  What in my macOS system is conflicting with this?  Use of virtual environments through venv or asdf?  Any ideas? How can I diagnose this?  

Thanks,

Mike

PS: /opt/nordic/ncs/v1.7.0/ looks like it ought to be a git repository, a local clone of some remote master.  But it doesn't' respond normally to git tools (e.g., SourceTree and GitHub desktop).  Any idea why?

[Mike Wirth]

Just did an automated install of nRF Connect v1.7.0 in Windows 10 using Toolchain Manager v0.9.4, same MacBook Pro as above, but in a Parallels Desktop VM, relatively clean system (i.e., no Python versions installed).  Went without a hitch (as expected) and I can use it as a sample of how the installation process should proceed.

Only problem I noticed was when using the "Update SDK" command, I ran into the following error:

 

Bottom Line:

My comments on Python version clashes on macOS in my first message above still stand and appear to be reinforced by the success of the installation process in other environments, e.g., Windows 10.

Thanks,

Mike

[ovrebekk]

Hi Mike

Thanks for bringing this to our attention, and the detailed report on the issues you have experienced. 

Apparently this is not something that has shown up in our testing, but there are always corner cases that the testing won't catch. 

There is a new version of the Toolchain manager coming out shortly, so things might be different after the update, but since this is a new issue to us it could very well be that the issue is still there. 

If this problem occurs again (in particular after the update), could you share the log as well?
Just click the 'Show log' button at the bottom right corner to bring it up. 

Best regards
Torbjørn

[Mike Wirth]

Hi Torbjørn,

Thanks for the quick reply and suggestions.  Here's my followup:

1. I have been using the latest SDK, 1.7.0.
2. I'm including the log file from my nFR SDK installation process from a couple days ago (2021-09-26T06_50_33.261Z-log.txt).  May or may not be useful given the items below.
3. Since my last update, I installed a new version of Python, 3.9.7, and updated my asdf virtual environment settings. My current asdf settings and PATH variable are detailed in the terminal log, Current asdf settings and PATH,txt.  This will prove to be critical.
4. I then reinstalled the nRF Connect app, v3.7.1, updated all the plugins, including Toolchain Manager, v0.10.0.
5. Removed the current SDK 1.7.0 installation.
6. Finally, started the automatic installation of SDK 1.7.0, which executed and completed without error!
7. Note at this point, that I have included two files to document this process,
   1. The log file, 2021-09-28T23_33_27.406Z-log.txt, as you requested
   2. A PDF file, documenting the process, nRF Connect Execution Notes.pdf, containing my notes, popups, etc., during the process.
8. As requested, I installed Visual Studio Code (actually updated it) and verified that my PATH included it
9. I then tested the buttons created in the Install SDK 1.7.0 box at the completion of the installation process.  Some errors were encountered. Here's a summary of my results (see the PDF file for details):
   1. First steps:
      1. Open VS Code:  Fails (see below for conflicting PATH values)
      2. Open SES: Works
   2. Selection Triangle:
      1. Open terminal: Opens but with a incomplete PATH list (see below)
      2. Open SDK directory: Works
      3. Open toolchain directory: Works
      4. Update SDK: Works
      5. Update toolchain: Fails with two errors: (see PDF notes) 
         1. "Inconsistent directory structure" and
         2. "Error, Cannot copy..."

Analysis:

1. My installation of Python 3.9.7 somehow cleared the errors I was having with west before.  Not sure how, but it would be nice to know how so we could avoid future problems.
2. The current errors noted above appear to be relate to different PATH variables in your "SDK 1.7.0" command line environment (see PATH as setup by Install SDK 1.7.0.txt file) and my normal terminal environment (see my "Current .zshrc.txt" and "Current asdf settings and PATH.txt" files

Hope all this info helps you find the remaining problems.  Will be happy to test further based on your response.

Mike

[ovrebekk]

Hi Mike

For some reason I am unable to load any of your attachments. I will check with one of the devzone admins if this is an issue on our end, and try to get back to you on your comments later in the week. 

Best regards
Torbjørn