Out of tree development: Building issues

RE: Out of tree development: Building issues

borg — Wed, 26 Feb 2020 09:05:55 GMT

Thanks a lot, i had the same problem.

The multi_image.cmake is included in the root CMakeLists.txt in the nrf/ folder. Maybe this include could be removed there and added again in the CMakeLists.txt of the examples. For an out-of-tree build one could then just take an example and edit the path in the CMakeLists.txt of your own project (and write your own multi_image.cmake).

Or there could be some variable that is set in the project CMakeLists.txt which is then added to the nonsecure_boards_with_ns_suffix in the multi_image.cmake.

RE: Out of tree development: Building issues

erik.johnson — Tue, 21 Jan 2020 16:43:39 GMT

Welp, figured out where the magic comes from. In the nrf9160_pca10090ns build, I noticed this at the beginning of the SPM build step:

Changed board to secure nrf9160_pca10090 (NOT NS)

This comes from nrf/cmake/multi_image.cmake:

function(image_board_selection board_in board_out)
# It is assumed that only the root app will be built as non-secure.
# This is not a valid assumption as there might be multiple non-secure
  # images defined.
  # TODO: Allow multiple non-secure images by using Kconfig to set the
  # secure/non-secure property rather than using a separate board definition.
  set(nonsecure_boards_with_ns_suffix
  nrf9160_pca10090ns
  nrf9160_pca20035ns
  nrf5340_dk_nrf5340_cpuappns
  )
  if(${board_in} IN_LIST nonsecure_boards_with_ns_suffix)
  string(LENGTH ${board_in} len)
  math(EXPR len_without_suffix "${len} - 2")
  string(SUBSTRING ${board_in} 0 ${len_without_suffix} board_in_without_suffix)

set(${board_out} ${board_in_without_suffix} PARENT_SCOPE)
message("Changed board to secure ${board_in_without_suffix} (NOT NS)")

  elseif(${board_in} STREQUAL actinius_icarus_ns)
  set(${board_out} actinius_icarus PARENT_SCOPE)
  message("Changed board to secure actinius_icarus (NOT NS)")

  else()
  set(${board_out} ${board_in} PARENT_SCOPE)
  endif()
endfunction()

It's quite a stretch to assume a developer is going to understand that they have to make this sort of a change, especially since it's being kept under source control in a repo that should only need to be checked out to build a project against. I'm still working with the master branch in fw-nrfconnect-nrf, so hopefully this is addressed at some point, either in the source or in documentation.

Hopefully this can help someone else out should they ever stumble upon a similar issue.

RE: Out of tree development: Building issues

erik.johnson — Tue, 21 Jan 2020 16:27:15 GMT

Digging in a little more, it looks like there must be some kind of hidden magic going on for the nrf9160_pca10090/ns board that rewires SPM. The generated .config file for the nrf9160_pca10090 build has this section:

CONFIG_BOARD="nrf9160_pca10090"
CONFIG_FLASH_LOAD_SIZE=0x30000
CONFIG_FLASH_LOAD_OFFSET=0
CONFIG_SOC="nRF9160_SICA"
CONFIG_SOC_SERIES="nrf91"
CONFIG_NUM_IRQS=65
CONFIG_SYS_CLOCK_HW_CYCLES_PER_SEC=32768
# CONFIG_WATCHDOG is not set
# CONFIG_UART_NS16550 is not set
# CONFIG_GPIO is not set
CONFIG_ARCH_HAS_CUSTOM_BUSY_WAIT=y
# CONFIG_CORTEX_M_SYSTICK is not set
CONFIG_ISR_STACK_SIZE=2048
CONFIG_CLOCK_CONTROL=y
CONFIG_NRF_RTC_TIMER=y
CONFIG_SYS_CLOCK_TICKS_PER_SEC=32768
CONFIG_SYS_POWER_MANAGEMENT=y
CONFIG_BUILD_OUTPUT_HEX=y
CONFIG_FLOAT=y
CONFIG_TEXT_SECTION_OFFSET=0
CONFIG_FLASH_SIZE=1024
CONFIG_FLASH_BASE_ADDRESS=0x0
CONFIG_SRAM_SIZE=64
CONFIG_SRAM_BASE_ADDRESS=0x20000000
# CONFIG_SPI is not set
# CONFIG_TINYCRYPT is not set
CONFIG_SOC_GECKO_EMU=y
CONFIG_BOARD_NRF9160_PCA10090=y
# CONFIG_BOARD_NRF9160_PCA10090NS is not set

But when building for our new board:

CONFIG_BOARD="<board name>"
CONFIG_FLASH_LOAD_SIZE=0x40000
CONFIG_FLASH_LOAD_OFFSET=0x40000
CONFIG_SOC="nRF9160_SICA"
CONFIG_SOC_SERIES="nrf91"
CONFIG_NUM_IRQS=65
CONFIG_SYS_CLOCK_HW_CYCLES_PER_SEC=32768
# CONFIG_WATCHDOG is not set
# CONFIG_UART_NS16550 is not set
# CONFIG_GPIO is not set
CONFIG_ARCH_HAS_CUSTOM_BUSY_WAIT=y
# CONFIG_CORTEX_M_SYSTICK is not set
CONFIG_ISR_STACK_SIZE=2048
CONFIG_CLOCK_CONTROL=y
CONFIG_NRF_RTC_TIMER=y
CONFIG_SYS_CLOCK_TICKS_PER_SEC=32768
CONFIG_SYS_POWER_MANAGEMENT=y
CONFIG_BUILD_OUTPUT_HEX=y
CONFIG_FLOAT=y
CONFIG_TEXT_SECTION_OFFSET=0
CONFIG_FLASH_SIZE=1024
CONFIG_FLASH_BASE_ADDRESS=0x0
CONFIG_SRAM_SIZE=128
CONFIG_SRAM_BASE_ADDRESS=0x20020000
# CONFIG_SPI is not set
# CONFIG_TINYCRYPT is not set
CONFIG_SOC_GECKO_EMU=y
# CONFIG_BOARD_<BOARD_NAME> is not set
CONFIG_BOARD_<BOARD_NAME>NS=y

CONFIG_BOARD in both cases grabs the secure board name -- which is expected, since SPM needs to sit in the secure region -- but then they deviate on things that actually depend on the secure/non-secure definitions. Someone must be changing CONFIG_BOARD_NRF9160_PCA10090NS to the non-NS variant specifically...

RE: Out of tree development: Building issues

erik.johnson — Tue, 21 Jan 2020 16:12:35 GMT

Looks like it's not getting set correctly:

/mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:38:9: note: #pragma message: Our clock: 0x40005000
38 | #pragma message "Our clock: " XSTR(DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS)
| ^~~~~~~
/mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:39:9: note: #pragma message: Their clock: ((NRF_CLOCK_Type*) 0x50005000UL)
39 | #pragma message "Their clock: " XSTR(NRF_CLOCK)

Looking at back at the NRF_PERIPH() macro, it's dependent on NRF_TRUSTZONE_NONSECURE, which I guess tracks with the warning about TRUSTED_EXECUTION_NONESECURE having issues during the Kconfig steps. So at this point I would guess I've still got an unknown issue with setting the trusted config, but hopefully once that's resolved the flurry of compilation issues will go away.

RE: Out of tree development: Building issues

erik.johnson — Tue, 21 Jan 2020 15:55:56 GMT

The assertions also don't make a lot of sense to me, as they shouldn't be failing. Taking the first one -- NRF_CLOCK -- as an example:

/mnt/c/git_root/test/zephyr/soc/arm/nordic_nrf/validate_base_addresses.c:35:1: note: in expansion of macro 'CHECK_ADDRESS'
35 | CHECK_ADDRESS(DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS, NRF_CLOCK);
| ^~~~~~~~~~~~~

From my generated Devicetree files:

$ grep -nr "DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS" ./
./build/b0/spm/zephyr/include/generated/generated_dts_board.conf:1014:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
./build/b0/spm/zephyr/include/generated/generated_dts_board_unfixed.h:1064:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS
./build/b0/zephyr/include/generated/generated_dts_board.conf:1014:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
./build/b0/zephyr/include/generated/generated_dts_board_unfixed.h:1064:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS
./build/mcuboot/spm/zephyr/include/generated/generated_dts_board.conf:1014:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
./build/mcuboot/spm/zephyr/include/generated/generated_dts_board_unfixed.h:1064:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS
./build/mcuboot/zephyr/include/generated/generated_dts_board.conf:1014:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
./build/mcuboot/zephyr/include/generated/generated_dts_board_unfixed.h:1064:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS
./build/spm/zephyr/include/generated/generated_dts_board.conf:1014:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
./build/spm/zephyr/include/generated/generated_dts_board_unfixed.h:1064:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS
./build/zephyr/include/generated/generated_dts_board.conf:1081:DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS=0x40005000
./build/zephyr/include/generated/generated_dts_board_unfixed.h:1133:#define DT_INST_0_NORDIC_NRF_CLOCK_BASE_ADDRESS DT_NORDIC_NRF_CLOCK_40005000_BASE_ADDRESS

That seems to match what should be generated for the non-secure part:

$ grep -nr "define.*NRF_CLOCK" ../

...

../modules/hal/nordic/nrfx_config_nrf9160.h:39:#define NRF_CLOCK NRF_PERIPH(NRF_CLOCK)

Which is defined as:

#if defined(NRF_TRUSTZONE_NONSECURE)
#define NRF_PERIPH(P) P##_NS
#else
#define NRF_PERIPH(P) P##_S
#endif

That selects:

../modules/hal/nordic/nrfx/mdk/nrf9160.h:2016:#define NRF_CLOCK_NS_BASE 0x40005000UL
../modules/hal/nordic/nrfx/mdk/nrf9160.h:2136:#define NRF_CLOCK_NS ((NRF_CLOCK_Type*) NRF_CLOCK_NS_BASE)

So this should pass the check above. Is there maybe something going on with auto-generation of includes or something? Perhaps the NRF_CLOCK value isn't getting properly pulled in or something...

RE: Out of tree development: Building issues

erik.johnson — Tue, 21 Jan 2020 15:37:47 GMT

Interestingly, updating to the latest in master from the fw-nrfconnect-nrf repo will let me get past #2 without having to use Zephyr's tree. Still results in the same chain of assertion failures in #3, however.