This post is older than 2 years and might not be relevant anymore
More Info: Consider searching for newer posts

Out of tree development: Building issues

Hi, I'm trying to migrate from the nRF9160 DK to our own board, and I'm having some trouble getting the west build system to play nicely. I've essentially started with the obvious choice: copy all of the nrf9160_pca10090 files to a new .../boards/arm/<board_name>/ directory with our board's name and changed all instances of BOARD_NRF9160_PCA10090/nrf9160_pca10090 and BOARD_NRF9160_PCA10090NS/nrf9160_pca10090ns with our board's variants. None of the other configurations or Devicetree setup has changed so far. I also updated our CMakeLists.txt file with:

# Set up the location of our custom boards
set(
    BOARD_ROOT
    ${CMAKE_CURRENT_LIST_DIR}
)

# Use our board
set(
    BOARD <board_name>
)

I'm still having some issues with the build, though. I'm building via "west build -p always ." when in our project's directory. I'm trying to avoid having caching issues by using "-p always" for now. I've also tried this with removing set(BOARD ...) and adding "-b <board_name>" during the build, to no avail. I've noticed a few things happening with this build that are different from builds with the nRF9160 board:

1) I get warnings about configurations not being correct. With the non-secure variant, I get this one:

warning: TRUSTED_EXECUTION_NONSECURE (defined at arch/Kconfig:145) was assigned the value 'y' but
got the value 'n'.

In our new <board_name>ns_defconfig file I've left that line as it was in the nRF9160 non-secure file:

CONFIG_TRUSTED_EXECUTION_NONSECURE=y

2) When buiding, I get the following error:

CMake Error at /mnt/c/git_root/test/zephyr/boards/CMakeLists.txt:13 (add_subdirectory):
add_subdirectory not given a binary directory but the given source
directory "/mnt/c/git_root/test/project/boards/arm/<board_name>" is
not a subdirectory of "/mnt/c/git_root/test/zephyr/boards". When
specifying an out-of-tree source a binary directory must be explicitly
specified.

I haven't seen any kind of direction on this in the docs, either Nordic's or Zephyr's. I can apparently resolve this by putting our board files back in ../zephyr/boards/arm/, but that's not exactly a solution for this.

3) If I build with our board in the zephyr directory, I get past #2 but then get quite a lot of build errors:

[35/181] Building C object zephyr/CMakeFiles/zephyr.dir/drivers/clock_control/nrf_power_clock.c.obj

...

In file included from /mnt/c/git_root/test/zephyr/include/toolchain.h:39,
from /mnt/c/git_root/test/zephyr/include/kernel_includes.h:19,
from /mnt/c/git_root/test/zephyr/include/kernel.h:17,
from /mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:7:
/mnt/c/git_root/test/zephyr/include/toolchain/gcc.h:61:28: error: static assertion failed: ""
61 | #define BUILD_ASSERT(EXPR) _Static_assert(EXPR, "")
| ^~~~~~~~~~~~~~
/mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:24:34: note: in expansion of macro 'BUILD_ASSERT'
24 | #define CHECK_ADDRESS(dts, mdk) BUILD_ASSERT((u32_t)(dts) == (u32_t)(mdk))
| ^~~~~~~~~~~~
/mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:35:1: note: in expansion of macro 'CHECK_ADDRESS'
35 | CHECK_ADDRESS(DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS, NRF_CLOCK);
| ^~~~~~~~~~~~~
/mnt/c/git_root/test/zephyr/include/toolchain/gcc.h:61:28: error: static assertion failed: ""
61 | #define BUILD_ASSERT(EXPR) _Static_assert(EXPR, "")

And so on. Same thing happens for:

[38/181] Building C object zephyr/CMakeFiles/zephyr.dir/mnt/c/git_root/test/nrf/subsys/spm/spm.c.obj

4) Compared to the nRF9160 build, I've noticed the build system seems to recurse into the spm project when building b0 and mcuboot:

=== child image spm begin ===

...

=== child image spm end ===

=== child image b0 begin ===

...

=== child image spm begin ===

...

=== child image spm end ===

...

=== child image b0 end ===

Any help is greatly appreciated. Right now I'm out of ideas and shooting in the dark by trying random things.

  • Interestingly, updating to the latest in master from the fw-nrfconnect-nrf repo will let me get past #2 without having to use Zephyr's tree. Still results in the same chain of assertion failures in #3, however.

  • The assertions also don't make a lot of sense to me, as they shouldn't be failing. Taking the first one -- NRF_CLOCK -- as an example:

    /mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:35:1: note: in expansion of macro 'CHECK_ADDRESS'
    35 | CHECK_ADDRESS(DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS, NRF_CLOCK);
    | ^~~~~~~~~~~~~

    From my generated Devicetree files:

    $ grep -nr "DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS" ./
    ./build/b0/spm/zephyr/include/generated/generated_dts_board.conf:1014:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
    ./build/b0/spm/zephyr/include/generated/generated_dts_board_unfixed.h:1064:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS
    ./build/b0/zephyr/include/generated/generated_dts_board.conf:1014:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
    ./build/b0/zephyr/include/generated/generated_dts_board_unfixed.h:1064:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS
    ./build/mcuboot/spm/zephyr/include/generated/generated_dts_board.conf:1014:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
    ./build/mcuboot/spm/zephyr/include/generated/generated_dts_board_unfixed.h:1064:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS
    ./build/mcuboot/zephyr/include/generated/generated_dts_board.conf:1014:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
    ./build/mcuboot/zephyr/include/generated/generated_dts_board_unfixed.h:1064:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS
    ./build/spm/zephyr/include/generated/generated_dts_board.conf:1014:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
    ./build/spm/zephyr/include/generated/generated_dts_board_unfixed.h:1064:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS
    ./build/zephyr/include/generated/generated_dts_board.conf:1081:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
    ./build/zephyr/include/generated/generated_dts_board_unfixed.h:1133:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS

    That seems to match what should be generated for the non-secure part:

    $ grep -nr "define.*NRF_CLOCK" ../

    ...

    ../modules/hal/nordic/nrfx_config_nrf9160.h:39:#define NRF_CLOCK       NRF_PERIPH(NRF_CLOCK)

    Which is defined as:

    #if defined(NRF_TRUSTZONE_NONSECURE)
    #define NRF_PERIPH(P) P##_NS
    #else
    #define NRF_PERIPH(P) P##_S
    #endif

    That selects:

    ../modules/hal/nordic/nrfx/mdk/nrf9160.h:2016:#define NRF_CLOCK_NS_BASE 0x40005000UL
    ../modules/hal/nordic/nrfx/mdk/nrf9160.h:2136:#define NRF_CLOCK_NS ((NRF_CLOCK_Type*) NRF_CLOCK_NS_BASE)

    So this should pass the check above. Is there maybe something going on with auto-generation of includes or something? Perhaps the NRF_CLOCK value isn't getting properly pulled in or something...

  • Looks like it's not getting set correctly:

    /mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:38:9: note: #pragma message: Our clock: 0x40005000
    38 | #pragma message "Our clock: " XSTR(DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS)
    | ^~~~~~~
    /mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:39:9: note: #pragma message: Their clock: ((NRF_CLOCK_Type*) 0x50005000UL)
    39 | #pragma message "Their clock: " XSTR(NRF_CLOCK)

    Looking at back at the NRF_PERIPH() macro, it's dependent on NRF_TRUSTZONE_NONSECURE, which I guess tracks with the warning about TRUSTED_EXECUTION_NONESECURE having issues during the Kconfig steps. So at this point I would guess I've still got an unknown issue with setting the trusted config, but hopefully once that's resolved the flurry of compilation issues will go away.

  • Digging in a little more, it looks like there must be some kind of hidden magic going on for the nrf9160_pca10090/ns board that rewires SPM. The generated .config file for the nrf9160_pca10090 build has this section:

    CONFIG_BOARD="nrf9160_pca10090"
    CONFIG_FLASH_LOAD_SIZE=0x30000
    CONFIG_FLASH_LOAD_OFFSET=0
    CONFIG_SOC="nRF9160_SICA"
    CONFIG_SOC_SERIES="nrf91"
    CONFIG_NUM_IRQS=65
    CONFIG_SYS_CLOCK_HW_CYCLES_PER_SEC=32768
    # CONFIG_WATCHDOG is not set
    # CONFIG_UART_NS16550 is not set
    # CONFIG_GPIO is not set
    CONFIG_ARCH_HAS_CUSTOM_BUSY_WAIT=y
    # CONFIG_CORTEX_M_SYSTICK is not set
    CONFIG_ISR_STACK_SIZE=2048
    CONFIG_CLOCK_CONTROL=y
    CONFIG_NRF_RTC_TIMER=y
    CONFIG_SYS_CLOCK_TICKS_PER_SEC=32768
    CONFIG_SYS_POWER_MANAGEMENT=y
    CONFIG_BUILD_OUTPUT_HEX=y
    CONFIG_FLOAT=y
    CONFIG_TEXT_SECTION_OFFSET=0
    CONFIG_FLASH_SIZE=1024
    CONFIG_FLASH_BASE_ADDRESS=0x0
    CONFIG_SRAM_SIZE=64
    CONFIG_SRAM_BASE_ADDRESS=0x20000000
    # CONFIG_SPI is not set
    # CONFIG_TINYCRYPT is not set
    CONFIG_SOC_GECKO_EMU=y
    CONFIG_BOARD_NRF9160_PCA10090=y
    # CONFIG_BOARD_NRF9160_PCA10090NS is not set

    But when building for our new board:

    CONFIG_BOARD="<board name>"
    CONFIG_FLASH_LOAD_SIZE=0x40000
    CONFIG_FLASH_LOAD_OFFSET=0x40000
    CONFIG_SOC="nRF9160_SICA"
    CONFIG_SOC_SERIES="nrf91"
    CONFIG_NUM_IRQS=65
    CONFIG_SYS_CLOCK_HW_CYCLES_PER_SEC=32768
    # CONFIG_WATCHDOG is not set
    # CONFIG_UART_NS16550 is not set
    # CONFIG_GPIO is not set
    CONFIG_ARCH_HAS_CUSTOM_BUSY_WAIT=y
    # CONFIG_CORTEX_M_SYSTICK is not set
    CONFIG_ISR_STACK_SIZE=2048
    CONFIG_CLOCK_CONTROL=y
    CONFIG_NRF_RTC_TIMER=y
    CONFIG_SYS_CLOCK_TICKS_PER_SEC=32768
    CONFIG_SYS_POWER_MANAGEMENT=y
    CONFIG_BUILD_OUTPUT_HEX=y
    CONFIG_FLOAT=y
    CONFIG_TEXT_SECTION_OFFSET=0
    CONFIG_FLASH_SIZE=1024
    CONFIG_FLASH_BASE_ADDRESS=0x0
    CONFIG_SRAM_SIZE=128
    CONFIG_SRAM_BASE_ADDRESS=0x20020000
    # CONFIG_SPI is not set
    # CONFIG_TINYCRYPT is not set
    CONFIG_SOC_GECKO_EMU=y
    # CONFIG_BOARD_<BOARD_NAME> is not set
    CONFIG_BOARD_<BOARD_NAME>NS=y

    CONFIG_BOARD in both cases grabs the secure board name -- which is expected, since SPM needs to sit in the secure region -- but then they deviate on things that actually depend on the secure/non-secure definitions. Someone must be changing CONFIG_BOARD_NRF9160_PCA10090NS to the non-NS variant specifically...

  • Welp, figured out where the magic comes from. In the nrf9160_pca10090ns build, I noticed this at the beginning of the SPM build step:

    Changed board to secure nrf9160_pca10090 (NOT NS)

    This comes from nrf/cmake/multi_image.cmake:

    function(image_board_selection board_in board_out)
      # It is assumed that only the root app will be built as non-secure.
      # This is not a valid assumption as there might be multiple non-secure
      # images defined.
      # TODO: Allow multiple non-secure images by using Kconfig to set the
      # secure/non-secure property rather than using a separate board definition.
      set(nonsecure_boards_with_ns_suffix
        nrf9160_pca10090ns
        nrf9160_pca20035ns
        nrf5340_dk_nrf5340_cpuappns
      )
      if(${board_in} IN_LIST nonsecure_boards_with_ns_suffix)
        string(LENGTH ${board_in} len)
        math(EXPR len_without_suffix "${len} - 2")
        string(SUBSTRING ${board_in} 0 ${len_without_suffix} board_in_without_suffix)

        set(${board_out} ${board_in_without_suffix} PARENT_SCOPE)
        message("Changed board to secure ${board_in_without_suffix} (NOT NS)")

      elseif(${board_in} STREQUAL actinius_icarus_ns)
        set(${board_out} actinius_icarus PARENT_SCOPE)
        message("Changed board to secure actinius_icarus (NOT NS)")

      else()
        set(${board_out} ${board_in} PARENT_SCOPE)
      endif()
    endfunction()

    It's quite a stretch to assume a developer is going to understand that they have to make this sort of a change, especially since it's being kept under source control in a repo that should only need to be checked out to build a project against. I'm still working with the master branch in fw-nrfconnect-nrf, so hopefully this is addressed at some point, either in the source or in documentation.

    Hopefully this can help someone else out should they ever stumble upon a similar issue.

Related