Out of tree development: Building issues

[erik.johnson]

Hi, I'm trying to migrate from the nRF9160 DK to our own board, and I'm having some trouble getting the west build system to play nicely. I've essentially started with the obvious choice: copy all of the nrf9160_pca10090 files to a new .../boards/arm/<board_name>/ directory with our board's name and changed all instances of BOARD_NRF9160_PCA10090/nrf9160_pca10090 and BOARD_NRF9160_PCA10090NS/nrf9160_pca10090ns with our board's variants. None of the other configurations or Devicetree setup has changed so far. I also updated our CMakeLists.txt file with:

# Set up the location of our custom boards
set(
    BOARD_ROOT
    ${CMAKE_CURRENT_LIST_DIR}
)

# Use our board
set(
    BOARD <board_name>
)

I'm still having some issues with the build, though. I'm building via "west build -p always ." when in our project's directory. I'm trying to avoid having caching issues by using "-p always" for now. I've also tried this with removing set(BOARD ...) and adding "-b <board_name>" during the build, to no avail. I've noticed a few things happening with this build that are different from builds with the nRF9160 board:

1) I get warnings about configurations not being correct. With the non-secure variant, I get this one:

warning: TRUSTED_EXECUTION_NONSECURE (defined at arch/Kconfig:145) was assigned the value 'y' but
got the value 'n'.

In our new <board_name>ns_defconfig file I've left that line as it was in the nRF9160 non-secure file:

CONFIG_TRUSTED_EXECUTION_NONSECURE=y

2) When buiding, I get the following error:

CMake Error at /mnt/c/git_root/test/zephyr/boards/CMakeLists.txt:13 (add_subdirectory):
add_subdirectory not given a binary directory but the given source
directory "/mnt/c/git_root/test/project/boards/arm/<board_name>" is
not a subdirectory of "/mnt/c/git_root/test/zephyr/boards". When
specifying an out-of-tree source a binary directory must be explicitly
specified.

I haven't seen any kind of direction on this in the docs, either Nordic's or Zephyr's. I can apparently resolve this by putting our board files back in ../zephyr/boards/arm/, but that's not exactly a solution for this.

3) If I build with our board in the zephyr directory, I get past #2 but then get quite a lot of build errors:

[35/181] Building C object zephyr/CMakeFiles/zephyr.dir/drivers/clock_control/nrf_power_clock.c.obj

...

In file included from /mnt/c/git_root/test/zephyr/include/toolchain.h:39,
from /mnt/c/git_root/test/zephyr/include/kernel_includes.h:19,
from /mnt/c/git_root/test/zephyr/include/kernel.h:17,
from /mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:7:
/mnt/c/git_root/test/zephyr/include/toolchain/gcc.h:61:28: error: static assertion failed: ""
61 | #define BUILD_ASSERT(EXPR) _Static_assert(EXPR, "")
| ^~~~~~~~~~~~~~
/mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:24:34: note: in expansion of macro 'BUILD_ASSERT'
24 | #define CHECK_ADDRESS(dts, mdk) BUILD_ASSERT((u32_t)(dts) == (u32_t)(mdk))
| ^~~~~~~~~~~~
/mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:35:1: note: in expansion of macro 'CHECK_ADDRESS'
35 | CHECK_ADDRESS(DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS, NRF_CLOCK);
| ^~~~~~~~~~~~~
/mnt/c/git_root/test/zephyr/include/toolchain/gcc.h:61:28: error: static assertion failed: ""
61 | #define BUILD_ASSERT(EXPR) _Static_assert(EXPR, "")

And so on. Same thing happens for:

[38/181] Building C object zephyr/CMakeFiles/zephyr.dir/mnt/c/git_root/test/nrf/subsys/spm/spm.c.obj

4) Compared to the nRF9160 build, I've noticed the build system seems to recurse into the spm project when building b0 and mcuboot:

=== child image spm begin ===

...

=== child image spm end ===

=== child image b0 begin ===

...

=== child image spm begin ===

...

=== child image spm end ===

...

=== child image b0 end ===

Any help is greatly appreciated. Right now I'm out of ideas and shooting in the dark by trying random things.

[erik.johnson]

Welp, figured out where the magic comes from. In the nrf9160_pca10090ns build, I noticed this at the beginning of the SPM build step:

Changed board to secure nrf9160_pca10090 (NOT NS)

This comes from nrf/cmake/multi_image.cmake:

function(image_board_selection board_in board_out)
  # It is assumed that only the root app will be built as non-secure.
  # This is not a valid assumption as there might be multiple non-secure
  # images defined.
  # TODO: Allow multiple non-secure images by using Kconfig to set the
  # secure/non-secure property rather than using a separate board definition.
  set(nonsecure_boards_with_ns_suffix
    nrf9160_pca10090ns
    nrf9160_pca20035ns
    nrf5340_dk_nrf5340_cpuappns
  )
  if(${board_in} IN_LIST nonsecure_boards_with_ns_suffix)
    string(LENGTH ${board_in} len)
    math(EXPR len_without_suffix "${len} - 2")
    string(SUBSTRING ${board_in} 0 ${len_without_suffix} board_in_without_suffix)

    set(${board_out} ${board_in_without_suffix} PARENT_SCOPE)
    message("Changed board to secure ${board_in_without_suffix} (NOT NS)")

  elseif(${board_in} STREQUAL actinius_icarus_ns)
    set(${board_out} actinius_icarus PARENT_SCOPE)
    message("Changed board to secure actinius_icarus (NOT NS)")

  else()
    set(${board_out} ${board_in} PARENT_SCOPE)
  endif()
endfunction()

It's quite a stretch to assume a developer is going to understand that they have to make this sort of a change, especially since it's being kept under source control in a repo that should only need to be checked out to build a project against. I'm still working with the master branch in fw-nrfconnect-nrf, so hopefully this is addressed at some point, either in the source or in documentation.

Hopefully this can help someone else out should they ever stumble upon a similar issue.

[borg]

Thanks a lot, i had the same problem.

The multi_image.cmake is included in the root CMakeLists.txt in the nrf/ folder. Maybe this include could be removed there and added again in the CMakeLists.txt of the examples. For an out-of-tree build one could then just take an example and edit the path in the CMakeLists.txt of your own project (and write your own multi_image.cmake).

Or there could be some variable that is set in the project CMakeLists.txt which is then added to the nonsecure_boards_with_ns_suffix in the multi_image.cmake.