VSCode: replicating build configurations (preset bugs + command line)

[crzyrndm]

OS: Windows

NCS Version: v2.0.2 Installed using toolchain manager

IDE: VSCode 1.70.2 + nordic-semiconductor.nrf-connect-extension-pack 2022.5.1

I have applications created as "freestanding" e.g.

and these work well. I have found the bit about using CMakePresets to share configuration details (https://nrfconnect.github.io/vscode-nrf-connect/connect/ui.html#build-specific-actions) which seems like it should be perfect.

First Issue: Creating a build configuration from the defined CMakePresets ignores the "binaryDir" field (build directory is always default). It is saved from the UI "Save Configuration", just not loaded

1a) The UI doesn't allow the build directory to be nested inside a folder (e.g. " ${sourceDir}/build/${presetName}") which really helps keep things tidy when multiple configurations are needed IMO

Second Issue: Replicating the build in command line for automated / CI builds

This is the output I get running the following commands in "C:\git\central" (west build command is the output of "Copy Build Command" in VSCode)

> C:\git\ncs\v2.0.2\zephyr\zephyr-env.cmd
> west build --build-dir c:\git\central\build_nRF52820 c:\git\central --pristine --board nrf52833dk_nrf52820 -- -DNCS_TOOLCHAIN_VERSION:STRING="NONE" -DCONF_FILE:STRING="c:/git/central/prj-nRF52820.conf"

2a) Is there a Github Actions template/example for building NCS firmware images?

[Edvin]

Hello,

Did you try using the nRF Connect for Desktop -> Toolchain manager

to set up your NCS install? If you didn't, I highly recommend that you give it a go.

It looks like there is a thing or two that is not currently correctly set up on your end.

If you did try to set it up with the toolchain manager, then please be aware that from time to time, it fails silently. Try removing your v2.0.2 folder, and try again.

Best regards,

Edvin

[crzyrndm]

I was using the toolchain manager to install 2.0.2

Reinstalled: still getting the exact same error on command line (VSCode extension builds fine)

The only potential errors I'm seeing in the log file look like this

2022-08-31T20:34:03.046Z DEBUG --- zephyr: initializing
2022-08-31T20:34:03.054Z DEBUG west.manifest: running 'git init 'C:\git\ncs\v2.0.2\zephyr'' in C:\git\ncs\v2.0.2
2022-08-31T20:34:03.077Z DEBUG Initialized empty Git repository in C:/git/ncs/v2.0.2/zephyr/.git/
west.manifest: "git init 'C:\git\ncs\v2.0.2\zephyr'" exit code: 0 stdout: None stderr: None
west.manifest: running 'git remote add -- ncs https://github.com/nrfconnect/sdk-zephyr' in C:\git\ncs\v2.0.2\zephyr
2022-08-31T20:34:03.088Z DEBUG west.manifest: "git remote add -- ncs https://github.com/nrfconnect/sdk-zephyr" exit code: 0 stdout: None stderr: None
2022-08-31T20:34:03.089Z DEBUG west.manifest: running 'git cat-file -t v3.0.99-ncs1-1' in C:\git\ncs\v2.0.2\zephyr
2022-08-31T20:34:03.099Z DEBUG west.manifest: "git cat-file -t v3.0.99-ncs1-1" exit code: 128 stdout: b'' stderr: b'fatal: Not a valid object name v3.0.99-ncs1-1\n'
2022-08-31T20:34:03.099Z DEBUG --- zephyr: fetching, need revision v3.0.99-ncs1-1
west.manifest: running 'git fetch -f --tags -- https://github.com/nrfconnect/sdk-zephyr v3.0.99-ncs1-1' in C:\git\ncs\v2.0.2\zephyr
2022-08-31T20:35:42.125Z DEBUG From https://github.com/nrfconnect/sdk-zephyr

The package installation toward the end of the log looks fairly normal from what I know

2022-08-31T20:41:42.582Z DEBUG ZEPHYR_BASE=C:\git\ncs\v2.0.2\zephyr (origin: configfile)
2022-08-31T20:41:42.603Z DEBUG cmake version 3.20.5 is OK; minimum version is 3.13.1
Running CMake: 'C:\git\ncs\toolchains\v2.0.2\opt\bin\cmake.EXE' -P 'C:\git\ncs\v2.0.2\zephyr\share\zephyr-package\cmake\zephyr_export.cmake'
2022-08-31T20:41:42.643Z DEBUG Zephyr (C:/git/ncs/v2.0.2/zephyr/share/zephyr-package/cmake)
has been added to the user package registry in:
HKEY_CURRENT_USER\Software\Kitware\CMake\Packages\Zephyr
2022-08-31T20:41:42.660Z DEBUG cmake version 3.20.5 is OK; minimum version is 3.13.1
Running CMake: 'C:\git\ncs\toolchains\v2.0.2\opt\bin\cmake.EXE' -P 'C:\git\ncs\v2.0.2\zephyr\share\zephyrunittest-package\cmake\zephyr_export.cmake'
2022-08-31T20:41:42.700Z DEBUG ZephyrUnittest (C:/git/ncs/v2.0.2/zephyr/share/zephyrunittest-package/cmake)
has been added to the user package registry in:
HKEY_CURRENT_USER\Software\Kitware\CMake\Packages\ZephyrUnittest

Examining the v2.0.2/zephyr folder in git it is checked out at the tag v3.0.99-ncs1-1

be360e7 - [nrf fromtree] net: openthread: enable default software tx security in 1.3

RE: Silent failures - Some sort of verification action seems like it would be useful

[Edvin]

crzyrndm said:

Some sort of verification action seems like it would be useful

Tell me about it. 

So I downloaded and tested the NCS v2.0.2 now. I am able to build the samples using the command "west build -b nrf52840dk_nrf52840 -d build".

Remember that you need to open the command line using nRF Connect for Desktop -> Toolchain Manager ->  "Open command prompt"

If you do that, and try to build e.g. the ncs\nrf\samples\bluetooth\peripheral_uart sample using 

west build -b nrf52840dk_nrf52840 -d build

what does it say?

Best regards,

Edvin

[crzyrndm]

That worked exactly as expected so the question then becomes, how do I recreate the nRF command prompt environment for automated CI builds? There's clearly something more than the ZEPHYR_BASE environment variable being set

[Edvin]

crzyrndm said:

There's clearly something more than the ZEPHYR_BASE environment variable being set

Indeed. 

In NCS V1.9.1 and older, there was a .cmd file that you could run (add it to your environment path, and run it from the command line). It was located in e.g. ncs\toolchain\cmd\env.cmd (NB: v1.9.1 and older).

There is also another file that does something similar, but with some limitations. However, for just building samples it should work. It is found in:

ncs\zephyr\zephyr-env.cmd

Also in the later versions.

The env.cmd file that disappeared in v2.0.0 is replaced by some other magic that is ran when you open the CL from nRF Connect for Desktop -> Toolchain Manager. I believe it is some java script running. What I know is that it uses the information stored in the file:

ncs\..\toolchains\v2.0.0\environment.json

(the toolchain folder is at the same level as the ncs root folder).

Perhaps you can use the information in there to set up the CL in order to build if the file found in ncs\zephyr\zephyr-env.cmd is not enough.

Best regards,

Edvin

[crzyrndm]

I can confirm after following through "toolchain/environment.json" that I can recreate the environment and build an image from a fresh shell. 

# Zephyr install root
$Env:ZEPHYR_BASE = "<ncs install directory>/v2.0.2/zephyr"

# Toolchain setup
$toolchain = "<ncs install directory>/toolchains/v2.0.2"

## overwrite ZEPHYR_SDK_INSTALL_DIR
$Env:ZEPHYR_SDK_INSTALL_DIR = "$toolchain/opt/zephyr-sdk"

## overwrite PYTHONPATH with values from environment.json
$pythonPathValues = @(
	"$toolchain/opt/bin",
	"$toolchain/opt/bin/Lib",
	"$toolchain/opt/bin/Lib/site-packages"
)
$pythonPath = $pythonPathValues -join ";"
$Env:PYTHONPATH = $pythonPath

## Prepend PATH with values from environment.json
$pathValues = @(
	"$toolchain/.",
	"$toolchain/mingw64/bin",
	"$toolchain/bin",
	"$toolchain/opt/bin",
	"$toolchain/opt/bin/Scripts",
	"$toolchain/nanopb/generator-bin",
	"$toolchain/opt/zephyr-sdk/arm-zephyr-eabi/bin"
)
$prepPath = $pathValues -join ";"
$Env:PATH = $prepPath + ';' + $Env:PATH

For anyone finding this at a later date ^^ is the script for NCS v2.0.2 (in powershell, adapt as required. What's happening should be fairly obvious)