nrfutil toolchain-manager "Error: Failed to retrieve toolchain-index" due to corporate SSL/TLS inspection

[ntgabriel]

Could you provide details on how the nrfutil toolchain-manager makes its internet requests?  Or even better, how to add certificates to be trusted in making its TLS connections? 

Our corporate internet security systems are causing the connections to fail, and we need to know what framework is used by the toolchain-manager so that we can work with our IT to resolve or workaround it.  Here are some examples of such workarounds for other development tools (Python PIP, NPM, etc.): https://help.zscaler.com/zia/adding-custom-certificate-application-specific-trust-store 

For context the nrfutil toolchain-manager gives us this error:

C:\>nrfutil toolchain-manager search
Error: Failed to retrieve toolchain-index

Caused by:
0: Failed to download index
1: Get request failed
2: developer.nordicsemi.com/.../index-windows-x86_64.json: Connection Failed: tls connection init failed: invalid peer certificate: UnknownIssuer
3: invalid peer certificate: UnknownIssuer

Or a similar error via the nRF Connect VS Code extension, in that case "Failed to fetch available toolchains." is the high-level error that pops up, but the accompanying console output shows same error as nrfutil toolchain-manager gives on command-line.

I have seen other DevZone posts (such as  "Failed to fetch available toolchains." on Visual Studio Code  ) with same error, but none of them explain how to add trusted certificates so the download works on a network employing SSL/TLS inspection.

Again this is not an issue with the nrfutil toolchain-manager or nRF Connect VS Code extension itself.  But do need to know more about how it works to enable us to get it to work as intended in our corporate environment.

[dejans]

Hi,

Do you use WSL or virtual machine? If so, this discussion might be relevant.

Best regards,
Dejan

[ntgabriel]

Thanks for the response.  No, this is directly in Windows.  We need to find a way to add our certificates into the network call(s) that nrfutil toolchain-manager is making.

[ntgabriel]

Thanks for the input.  Could you say more about how you get or are interpreting the trace output of nrfutil you are referring to?  I'm not seeing any reference to any environment variable in the trace logs I'm getting.

I'm doing this (WIndows cmd):

nrfutil toolchain-manager search --log-level trace


I get notable extra info at trace level, mostly comprising a few TLS message payloads being sent to the server, but doesn't seem to say anything about local certificates or reference the names or contents of environment variables AFAIK.  I'd say the only thing useful to me in this trace log so far is a line near the end which gives me a list of 3 DER-encoded server certificates, right before it fails.

[2024-02-07T17:17:38.621Z] [nrfutil-toolchain-manager] TRACE - Server cert is [Certificate(b"..."), Certificate(b"..."), Certificate(b"...")]

[dejans]

Hi,

I have asked internally. You can find exe file inside attached 3755.nrfutil-toolchain-manager.zip which you could try out. The file should work out-of-the-box. Note that the file is provided as is, for testing purpose, and without any guarantee to be working. The file has not been tested with proxy, and thus might not work with proxy. However, it seems that the file picks up on the SSL_CERT_FILE environment variable. 

Best regards,
Dejan

[rgrr2]

I can confirm, that (for me) this solves the certificate issues under Windows.  Any chance for a Linux version which also obeys proxy rules?

PS: but I'm just the "hijacker"

[ntgabriel]

dejans Indeed, yes that custom nrfutil-toolchain-manager.exe works if I point the SSL_CERT_FILE to the certificate file I use for other dev tools, thank you!

At first I called it directly, for initial testing, then I also temporarily replaced the ~\.nrfutil\bin\nrfutil-toolchain-manager.exe with it for now too, to try this within nrfutil calls, and that works now too.

What's your plan for incorporating this change into the nrfutil / toolchain-manager release? 

Or even better perhaps, to incorporate a change to allow use of the OS-native certificate store?  (The certs I'm passing into this custom exe via SSL_CERT_FILE are in the OS cert store already.)

[dejans]

Hi ntgabriel and rgrr2 ,

rgrr2 said:

Any chance for a Linux version which also obeys proxy rules?

Please find attached Linux binary file (inside provided zip file) which should work in a similar way as previously provided file for Windows. Please note that this is not yet an official version. The same considerations apply as for the Windows file. You would need to test it and verify that it works for you.

  3808.nrfutil-toolchain-manager.zip

ntgabriel said:

What's your plan for incorporating this change into the nrfutil / toolchain-manager release?

For a specific plan or timeline regarding the official release with changes, please contact your regional sales manager.

Best regards,
Dejan

[jgilles]

This linux binary works for me and doesn't require setting SSL_CERT_FILE since I have added the required certs using update-ca-certificates in Ubuntu.  Thank you!

[ntgabriel]

dejans Thanks to you and the development team for your help with this!

[rgrr2]

Unfortunately this version still does not obey to the proxy setup.  Don't know how jgilles  did it

hardy@debian-hardy:~/.nrfutil/bin$ nrfutil --detect-proxy --log-level=trace --log-output=stdout --json toolchain-manager search
{"type":"log","data":{"level":"INFO","message":"nrfutil (version = 7.7.0, platform = x86_64-unknown-linux-gnu) invoked with --detect-proxy --log-level=trace --log-output=stdout --json toolchain-manager search ","timestamp":"2024-02-12T09:33:21.296Z"}}
{"type":"log","data":{"level":"INFO","message":"Trying to detect proxy configuration automatically...","timestamp":"2024-02-12T09:33:21.296Z"}}
{"type":"log","data":{"level":"INFO","message":"Using proxy helper program from '/home/hardy/.nrfutil/proxy_utils/proxy'","timestamp":"2024-02-12T09:33:21.296Z"}}
{"type":"log","data":{"level":"INFO","message":"Running '/home/hardy/.nrfutil/proxy_utils/proxy developer.nordicsemi.com' to detect proxy configuration","timestamp":"2024-02-12T09:33:21.296Z"}}
{"type":"log","data":{"level":"INFO","message":"Proxy configuration detected","timestamp":"2024-02-12T09:33:21.297Z"}}
{"type":"log","data":{"level":"DEBUG","message":"Setting environment variable NRFUTIL_EXEC_PATH to /home/hardy/.nrfutil/bin/nrfutil for subcommand `toolchain-manager`","timestamp":"2024-02-12T09:33:21.297Z"}}
{"type":"log","data":{"level":"INFO","message":"nrfutil-toolchain-manager (version = 0.14.1, platform = x86_64-unknown-linux-gnu) invoked with --json --log-level TRACE --log-output stdout search ","timestamp":"2024-02-12T09:33:21.300Z"}}
{"type":"log","data":{"level":"DEBUG","message":"Constructing the configuration from the defaults","timestamp":"2024-02-12T09:33:21.300Z"}}
{"type":"log","data":{"level":"DEBUG","message":"Current config: Config { install_dir: \"/home/hardy/ncs\", toolchain_index: None }","timestamp":"2024-02-12T09:33:21.300Z"}}
{"type":"log","data":{"level":"INFO","message":"Downloading from "timestamp":"2024-02-12T09:33:21.300Z"}}">developer.nordicsemi.com/.../index-linux-x86_64.json","timestamp":"2024-02-12T09:33:21.300Z"}}
{"type":"log","data":{"level":"DEBUG","message":"connecting to developer.nordicsemi.com:443 at 104.20.251.111:443","timestamp":"2024-02-12T09:33:21.326Z"}}
{"type":"log","data":{"level":"INFO","message":"Failed to retrieve cached index: Failed to get cache","timestamp":"2024-02-12T09:33:51.331Z"}}
Error: Failed to retrieve toolchain-index

Caused by:
   0: Failed to download index
   1: Get request failed
   2: developer.nordicsemi.com/.../index-linux-x86_64.json: Network Error: timed out reading response
   3: timed out reading response

[jgilles]

rgrr2 The new toolchain-manager binary worked for me because I wasn't having proxy-related connections problems (this is a separate issue).  I am behind a corporate TLS inspection system that requires installation of company-specific root certs for clients to trust or cert validation fails.  The new binary is successfully using the locally installed trusted certs, the released one does not.

[dejans]

Hi,

We have released new version which contains updates included in previously provided exe/binary (solving problem with native certificates) and fix for proxy networks. It can be installed using nrfutil command "nrfutil install toolchain-manager".

Best regards,
Dejan