Project cannot be built in a docker container

[balix]

Hello,

I need to build a project in a Docker container. It built fine under Windows 10 with a connect SDK v1.9.1.

Now I am using a Docker image built with the Dockerfile for nrfconnect-toolchain: https://github.com/NordicPlayground/nrfconnect-chip-docker/tree/master/nrfconnect-toolchain

1. I am not sure which version of the connect SDK is installed with this image but I assume it would be the latest one (v2.0.0). Would that be incompatible with my project which was built with a SDK v1.9.1?

2. My main issue is an error during the build with west that I don't understand:

root@a81dfea4756f:/workspace/nRF52840_zephyr-bsp# west build
-- west build: generating a build system
Loading Zephyr default modules (Zephyr base (cached)).
-- Application: /workspace/nRF52840_zephyr-bsp
-- Cache files will be written to: /workspace/nRF52840_zephyr-bsp/zephyr/.cache
-- Zephyr version: 3.1.99 (/workspace/nRF52840_zephyr-bsp/zephyr)
-- Found west (found suitable version "0.13.1", minimum required is "0.7.1")
-- Board: nrf52840dk_nrf52840
-- Found dtc: /usr/bin/dtc (found suitable version "1.6.0", minimum required is "1.4.6")
-- Found toolchain: gnuarmemb (/opt/gcc-arm-none-eabi-9-2019-q4-major)
-- Found BOARD.dts: /workspace/nRF52840_zephyr-bsp/zephyr/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840.dts
-- Generated zephyr.dts: /workspace/nRF52840_zephyr-bsp/build/zephyr/zephyr.dts
-- Generated devicetree_unfixed.h: /workspace/nRF52840_zephyr-bsp/build/zephyr/include/generated/devicetree_unfixed.h
-- Generated device_extern.h: /workspace/nRF52840_zephyr-bsp/build/zephyr/include/generated/device_extern.h
-- Including generated dts.cmake file: /workspace/nRF52840_zephyr-bsp/build/zephyr/dts.cmake

/workspace/nRF52840_zephyr-bsp/prj.conf:28: warning: attempt to assign the value 'y' to the undefined symbol BT_LL_SOFTDEVICE

/workspace/nRF52840_zephyr-bsp/prj.conf:31: warning: attempt to assign the value 'y' to the undefined symbol BT_GATT_DM

/workspace/nRF52840_zephyr-bsp/prj.conf:33: warning: attempt to assign the value 'y' to the undefined symbol BT_RECV_IS_RX_THREAD
Parsing /workspace/nRF52840_zephyr-bsp/Kconfig
Loaded configuration '/workspace/nRF52840_zephyr-bsp/zephyr/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840_defconfig'
Merged configuration '/workspace/nRF52840_zephyr-bsp/prj.conf'

error: Aborting due to Kconfig warnings

CMake Error at zephyr/cmake/modules/kconfig.cmake:290 (message):
  command failed with return code: 1
Call Stack (most recent call first):
  zephyr/cmake/modules/zephyr_default.cmake:121 (include)
  zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:66 (include)
  zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:96 (include_boilerplate)
  CMakeLists.txt:11 (find_package)


-- Configuring incomplete, errors occurred!
FATAL ERROR: command exited with status 1: /usr/local/bin/cmake -DWEST_PYTHON=/usr/bin/python3 -B/workspace/nRF52840_zephyr-bsp/build -S/workspace/nRF52840_zephyr-bsp -GNinja

I saw this ticket which seems to talk about what appears to be a related issue:  Cannot Replicate Build in Docker Environment

However the workaround addressed in this ticket did not work in my case.

Regards,

Benjamin

[tedhavelka]

Hello Benjamin,

Your Devzone post regarding Docker build errors, specifically Kconfig symbols which the build process finds not defined catches my eye.  While working on an out-of-tree driver for an unsupported sensor, I encountered this same type of error a few months ago early on.  I write because I wonder whether your situation is not a problem with Docker container, but rather something may be missing in your project's included Kconfig files.

While I'm not using any Bluetooth features yet, I text searched from the root of my ncs version 1.6.1 workspace for the symbols your build output shows not defined.  BT_LL_SOFTDEVICE appears in these files in my local copy of Nordic's nrf SDK:

ted@localhost:~/projects/zephyr-based$ grep -nr BT_LL_SOFTDEVICE ./* | grep -v CONFIG_ | grep BT_LL_SOFTDEVICE
./nrf/applications/nrf_desktop/src/modules/Kconfig.ble:212:	depends on BT_LL_SOFTDEVICE
./nrf/applications/nrf_desktop/src/modules/Kconfig.ble:238:	depends on BT_LL_SOFTDEVICE
./nrf/tests/drivers/nrfx_integration_test/Kconfig:86:config NRFX_AND_BT_LL_SOFTDEVICE
./nrf/tests/drivers/nrfx_integration_test/Kconfig:95:	default BT_LL_SOFTDEVICE if NRFX_AND_BT_LL_SOFTDEVICE
./nrf/subsys/mpsl/Kconfig:12:	default SDC_RX_PRIO if BT_LL_SOFTDEVICE
./nrf/subsys/caf/modules/Kconfig.ble_state:11:	select BT_CTLR_TX_PWR_DYNAMIC_CONTROL if BT_LL_SOFTDEVICE
./nrf/subsys/caf/modules/Kconfig.ble_state:30:	depends on BT_LL_SOFTDEVICE
./nrf/subsys/bluetooth/controller/Kconfig:15:	default 1536 if BT_LL_SOFTDEVICE
./nrf/subsys/bluetooth/controller/Kconfig:44:	default 2048 if BT_LL_SOFTDEVICE
./nrf/subsys/bluetooth/Kconfig:12:# BT_LL_SOFTDEVICE which is tested with 20 connections.
./nrf/subsys/bluetooth/Kconfig:16:	range 2 20 if BT_LL_SOFTDEVICE && BT_CENTRAL && BT_PERIPHERAL
./nrf/subsys/bluetooth/Kconfig:17:	range 1 20 if BT_LL_SOFTDEVICE
./nrf/subsys/bluetooth/Kconfig:18:	default 2 if BT_LL_SOFTDEVICE && BT_CENTRAL && BT_PERIPHERAL
./nrf/subsys/bluetooth/Kconfig:20:if BT_LL_SOFTDEVICE
./nrfxlib/softdevice_controller/Kconfig:13:config BT_LL_SOFTDEVICE_VS_INCLUDE
./nrfxlib/softdevice_controller/Kconfig:18:config BT_LL_SOFTDEVICE_DEFAULT
./nrfxlib/softdevice_controller/Kconfig:19:	bool "Select BT_LL_SOFTDEVICE as default BT_LL_CHOICE"
./nrfxlib/softdevice_controller/Kconfig:36:	default BT_LL_SOFTDEVICE
./nrfxlib/softdevice_controller/Kconfig:38:config BT_LL_SOFTDEVICE
./nrfxlib/softdevice_controller/Kconfig:63:if BT_LL_SOFTDEVICE
./nrfxlib/softdevice_controller/Kconfig:104:endif # BT_LL_SOFTDEVICE

The last file in these results is of particular interest to me, because it contains the stanza `./nrfxlib/softdevice_controller/Kconfig:38:config BT_LL_SOFTDEVICE`.  When I faced the same "symbol not defined" errors in my out-of-tree driver work, this is the line I was missing in the most nested Kconfig file of my small driver directory and Kconfig hierarchy.

I did similar searches for the other two symbols with similar results:

ted@localhost:~/projects/zephyr-based$ grep -nr BT_GATT_DM ./* | grep -v CONFIG_ | grep BT_GATT_DM
./nrf/include/bluetooth/gatt_dm.h:7:#ifndef BT_GATT_DM_H_
./nrf/include/bluetooth/gatt_dm.h:8:#define BT_GATT_DM_H_
./nrf/include/bluetooth/gatt_dm.h:338:#endif /* BT_GATT_DM_H_ */
./nrf/subsys/bluetooth/Kconfig.discovery:7:menuconfig BT_GATT_DM
./nrf/subsys/bluetooth/Kconfig.discovery:12:if BT_GATT_DM
./nrf/subsys/bluetooth/Kconfig.discovery:14:config BT_GATT_DM_MAX_ATTRS
./nrf/subsys/bluetooth/Kconfig.discovery:20:config BT_GATT_DM_DATA_PRINT
./nrf/subsys/bluetooth/Kconfig.discovery:26:module = BT_GATT_DM
./nrf/subsys/bluetooth/Kconfig.discovery:30:endif # BT_GATT_DM
./nrf/subsys/bluetooth/services/Kconfig.latency_c:10:	select BT_GATT_DM

ted@localhost:~/projects/zephyr-based$ grep -nr BT_RECV_IS_RX_THREAD ./* | grep -v CONFIG_ | grep BT_RECV_IS_RX_THREAD
./nrf/subsys/bluetooth/controller/Kconfig:23:	depends on BT_HCI_HOST || BT_RECV_IS_RX_THREAD
./zephyr/drivers/bluetooth/hci/Kconfig:18:	select BT_RECV_IS_RX_THREAD
./zephyr/subsys/bluetooth/controller/Kconfig:98:	select BT_RECV_IS_RX_THREAD
./zephyr/subsys/bluetooth/host/Kconfig:73:config BT_RECV_IS_RX_THREAD
./zephyr/subsys/bluetooth/host/Kconfig:85:	depends on BT_HCI_HOST || BT_RECV_IS_RX_THREAD
./zephyr/subsys/bluetooth/host/Kconfig:101:	depends on BT_HCI_HOST || BT_RECV_IS_RX_THREAD
./zephyr/subsys/bluetooth/common/Kconfig:98:	default 3 if BT_RECV_IS_RX_THREAD
./zephyr/subsys/bluetooth/common/Kconfig:129:	default 3 if BT_RECV_IS_RX_THREAD

Symbol BT_GATT_DM has a `menuconfig` type declaration, so that is a little different than the key line I needed (confirmed by Jared Wolff at https://community.circuitdojo.com) but I find these symbols are part of my west workspace.

Are you able to locate these same or comparable Kconfig files under the root of your project in Docker?  If they are present there the question then becomes whether or why the build process is not seeing them included some time soon after the first Kconfig file is processed.

So I am wondering now what tool we have which can more verbosely show us how Kconfig files are included at build time?  After creating your Docker instance, did you both git clone your project(s) and invoke `west init` on them?  While not directly related to Kconfig processing, I know that `west init` has a major impact on where different west-guided elements of the build process look for project dependencies.

I will keep an eye on your Devzone ticket 89477 as I am myself trying to move our team's Zephyr based firmware into a Docker image.  Either way I hope this can help you some, Benjamin.

- Ted

[balix]

Hi Ted,

I tried to build my project without a Docker container. So what I did is follow the manual installation guide on a WSL: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/gs_installing.html

And then I faced the exact same error in this environment, so it appears that you are right and my issue is not related to the Docker container I used.

Now I am a bit confused because if I understand correctly your ncs folder is at the same place than your project?

The guide does not specify where it should be installed so I created the ncs folder in my home directory.

And then your grep commands return nothing if I run them inside my project folder but I got the same result as you if I run it in my ncs folder.

Should these files (ncs) be in my project instead?

Benjamin

[Øyvind]

Hello Benjamin, 

The DevZone team is currently entering summer vacation period, some delay in answers must be expected.

---

From the log output I see that your compiler is using Zephyr version: 3.1.99 (/workspace/nRF52840_zephyr-bsp/zephyr). Please note that nRF Connect SDK v2.0.0 integrates v3.0.99, and nRF Connect SDK v1.9.1 integrates v2.7.99.

This is most likely the reason for why it will fail. Regarding the details of the docker image itself I will need more time.

Kind regards,
Øyvind

[balix]

Thanks for the info Øyvind.

As a matter of fact I managed to solve the issue and it seems to be indeed an issue with the toolchain and sdk version used.

I cleaned the repository containing the project in order to start fresh in the WSL environment.

Then I did the following:

cd nRF52840_zephyr-bsp/

west init -m https://github.com/nrfconnect/sdk-nrf --mr v1.9.1

west update

west build -b nrf52840dk_nrf52840

Before that I was just using west init alone at the root of the project assuming that it would use the nrf-sdk I installed in my  home directory. Or maybe it should have been the case? Maybe I did something wrong following the installation guide?


[Øyvind]

Hello Benjamin, 

balix said:

Before that I was just using west init alone at the root of the project assuming that it would use the nrf-sdk I installed in my  home directory.

After doing west init, you must run west update to clone the project repositories. 

balix said:

Maybe I did something wrong following the installation guide?

Based on the above, I think that is the answer to your question

Kind regards,
Øyvind

[tedhavelka]

Hi Benjamin,

To mirror Øyvind's reply and your question "Should these files (ncs) be in my project instead?", your found solution of running `west init && west update` puts the ncs files where your project needs them.  To run `west update` downloads the ncs pieces along side your project root directory.

To your question just before that, my Zephyr + Nordic based app is also in the same place, same directory as my ncs directory.  And in reality, the ncs code base ends up getting downloaded into a directory named 'nrf'.  But that is the core ncs collection of code libraries, sample apps and related resources from Nordic's repository.

Just a little background:  it took me about three weeks to figure out how to create my first stand alone, Zephyr RTOS based app.  I needed to do this because after getting Nordic ncs sample apps to build (from ncs v1.6.1, in summer of 2021) I needed to bring my own coding efforts under git version control.  I could not do that from the "west updated" copy of Nordic's ncs alone.  That collection of cloned repositories was already under the git control of Nordic's ncs.  With help from both Devzone and CircuitDojo community forums, I was able to pull out Nordic's nRF9160 blinky sample, create a west.yml manifest file for it, and then compile and flash that successfully using the same tools I'd used on the "in ncs" samples.

These are the tools you as well installed, following https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/gs_installing.html.

Once I understood how the west manifest files work, and the purposes they serve, it was easier to craft new Zephyr based apps and download the needed dependencies.

There are a few caveats:  custom apps depending on different versions of Zephyr RTOS, or different versions of Nordic ncs-sdk will each need their own west-initialized workspace, and the roughly two gigabytes of downloaded code projects to support them.  If you have two or more projects which depend on the same version of Zephyr, and non-conflicting same versions of Nordic ncs, it is possible to have just one west workspace for those multiple custom apps.  But only one of the custom app's manifest file should be treated as the arbiter or ruling manifest.  A manifest with fewer external code dependencies in one project won't update all the dependencies of a second project.  And of course we must assure that no two projects sharing one west workspace refer to alternate versions of the same third party code library or code base.

Disk space is pretty cheap these days, so you are likely safe to dedicate one west workspace to each Zephyr based project.

Curious about one thing Benjamin, when you use Docker containers are you able with the container to see USB programmers and other hardware you use to interact with your targeted board?  No issues or hoops to jump through there?

- Ted

[ealtan]

Hi Ted,

I am also interested in your "when you use Docker containers are you able with the container to see USB programmers and other hardware you use to interact with your targeted board? "  question.  I am running  Docker image on a Windows 10 PC.

I built my project successfully but when I do west flash, it doesn't see the board, even though it is connected and I can successfully program it via Visual Studio Code.   I get these errors :

-- west flash: using runner nrfjprog
-- runners.nrfjprog: mass erase requested
FATAL ERROR: "nrfjprog --ids" did not find a board; is the board connected?

Is there a procedure to use USB ports with Docker Container ?  

UPDATE:  I came across this message :  https://github.com/NordicPlayground/nrf-docker#flashing  

So, unless it is changed,  USB access in Windows and MacOS is not supported, yet.  

Thanks,

Ealtan

[tedhavelka]

Hi Ealtan,

In regards to using `west flash` within a virtual machine as provided by Oracle's VirtualBox software, it sounds as though your work station is running either Windows or MacOS, but you are setting up a VM with some flavor of Linux.  If not and you have time, please correct me on this surmise.

The reason I ask is that about a year ago I was using VM images on which I'd set up a basic Ubuntu 20.04 LTS, and some firmware development tools therein.  While the base VirtualBox software had a license free option, I tested out their "Oracle VM VirtualBox Extension Pack" which seemed necessary to provide access to hardware devices like COM ports and USB subsystem.

With this extension pack installed I was able to flash dev kits and custom hardware from my virtual machine running Ubuntu, with this VM running on Windows 10.  I believe I was using `nrfjprog` to flash external hardware.  So while it was not `west` I was able to complete a full "code, compile, flash, debug" work cycle from within a virtual machine.

I think `west` would work in or from this configuration of VM as well, as `west` mostly invokes other tools, and the key thing is for the VM to have a sane view and access to workstation peripherals and physical ports.

This extension pack however I learned at the time required a license.  Shortly after these explorations our team were granted permission to install Ubuntu on dedicated laptops.  This greatly simplified our firmware specific needs to connect programmers and debuggers to external hardware.

At present I don't have time to test the Docker image you're using -- thank you for sharing the link to it! -- yet I suspect that it entails the base, free VirtualBox as part of its configuration.  If it's just VirtualBox without any extension packs, that may be why the ability to flash devices is not yet supported.

Just thought I would share these thoughts, in case it jogs a memory or new idea to help you get tools and environments configured better.  Nice to know there is a Docker image which comes so close to providing a `west` and Zephyr supporting build environment!

Thanks for writing back.  Stay well, Ealtan

- Ted