nRF Connect SDK: add_child_image() does not work for out-of-tree sources

Question

I have several out-of-tree applications targeting the nRF9160 that all build correctly when built individually. One of these applications, named "secure_partition", is a customized clone of the Secure Partition Manager (SPM) sample provided in the SDK. This image targets a "secure board" while the others target "nonsecure" boards. 
 For the non-secure applications, I would like to also build my "secure_partition" application as a child image. However, I have discovered multiple defects in the "Multi-image build" support offered by the NCS that prevent this from being possible. 
 The first defect I discovered is that it's not enough to simply add a call to `add_child_image(....)` in the `CMakeLists.txt` for the non-secure application. This is because the `partition_manager.cmake` script needs to run *after* the last call to `add_child_image(....)`. Currently Zephyr's build system assumes (incorrectly) that child images are only added by in-tree code, and thus Zephyr's build system includes `partition_manager.cmake` in the `app/boilerplate.cmake` file. 
 The work-around for the first defect is to manually include `partition_manager.cmake` again after calling `add_child_image(...)` in the `CMakeLists.txt` for the non-secure application. It would be a much better API if the NCS just handled running the `partition_manager.cmake` script anytime a child image was added. 
 The second defect I discovered is that the `partition_manager.cmake` script makes an incorrect assumption about the location of the hex file for the 'app' image. The `partition_manager.cmake` script assumes that the 'app' image hex file is located at "${PROJECT_BINARY_DIR}/${KERNEL_HEX_NAME}". This assumption is only true when the active CMake project is the 'Zephyr-Kernel' project. If the active CMake project is the application project (aka the application `CMakeLists.txt` calls `project(....)`), then the value of `${PROJECT_BINARY_DIR}` will no longer correspond to the directory containing the Zephyr kernel build artifacts. 
 The `partition_manager.cmake` script should not assume that `${PROJECT_BINARY_DIR}` refers to the binary directory for the Zephyr kernel, as that variable is automatically changed by CMake anytime the `project(...)` function is called. Instead, the `partition_manager.cmake` script should determine the correct location of the `${KERNEL_HEX_NAME}` file via other means. It does not appear that Zephyr defines a variable such as ${ZEPHYR_BINARY_DIR}, though if would certainly be helpful if such a variable existed. 
 The work-around for the second defect is to set the location for the 'app' image hex file to `${CMAKE_BINARY_DIR}/zephyr/${KERNEL_HEX_NAME}`. This work-around also relies on a sometimes-untrue assumption, namely that the binary directory for the application is the same as the root binary directory. In my immediate situation this assumption holds, but nothing guarantees it. A better permanent solution is necessary.

Torsten Rasmussen · Answer

Hi David Palchak

Thank you for your report.

I hope my answer will help you and others facing the same difficulties.

Now, to the head line.

add_child_image() does not work for out-of-tree sources

`add_child_image()` and partition manager does support out-of-tree sources, but what is required is for this to be part of the Zephyr build system in order to work correctly.

Now the `add_child_image()` itself will create a new build of a child image, so that part should in theory work, even if called after `find_package(Zephyr)`.
But making an external build is only a subpart of the whole process of creating a multi-image build as you have noticed.

Correct partitioning is also important, thus partition manager needs to have the correct knowledge on things to combine.
Also when a child image is added to the build then a number of variables are shared from the child back to the parent image.

Those variables are used later by the build system, also making some dependencies as to why the child image must be an integrated part of the Zephyr build.

So how is it possible to have your out-of-tree child image be integrated into the Zephyr build system.

Well, first thing that is needed to hook into the Zephyr build system at the right location is to create a Zephyr module and add it as an extra Zephyr module, see details here:
docs.zephyrproject.org/.../modules.html

Let's say you currently have:

<repo>/applications/
         |--- parent-app
         |     |--- CMakeLists.txt
         |     |--- src/
         |           |--- *.c
         |--- child-app
               |--- CMakeLists.txt
               |--- src/
                     |--- *.c

assuming your parent-app/CMakeLists.txt looks something like this:

cmake_minimum_required(VERSION 3.20.0)

find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(parent-app)

add_child_image(
    NAME child-app
    SOURCE_DIR ${CMAKE_CURRENT_LIST_DIR}/../child-app
)

...

then we need to make a Zephyr module and add the child image from there.
That can be done with minimal changes.

First we need two additional files.
- zephyr/module.yml
- zephyr/CMakeLists.txt

so we have this structure:

<repo>/
   |--- zephyr
   |     |--- module.yml
   |     |--- CMakeLists.txt
   |
   |--- applications/
         |--- parent-app
         |     |--- CMakeLists.txt
         |     |--- src/
         |           |--- *.c
         |--- child-app
               |--- CMakeLists.txt
               |--- src/
                     |--- *.c

and then you add the following content:

# zephyr/module.yml
build:
  cmake: zephyr

and the CMakeList.txt file:

# zephyr/CMakeLists.txt
add_child_image(
    NAME child-app
    SOURCE_DIR ${ZEPHYR_CURRENT_MODULE_DIR}/applications/child-app
)

and the original `applications/parent-app/CMakeLists.txt` must then be updated accordingly:

cmake_minimum_required(VERSION 3.20.0)

set(ZEPHYR_EXTRA_MODULES ${CMAKE_CURRENT_LIST_DIR}/../../)

find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(parent-app)

...

This will now include the zephyr/CMakeLists.txt file in the Zephyr build which the calls the `add_child_image()`.

It does not appear that Zephyr defines a variable such as ${ZEPHYR_BINARY_DIR}, though if would certainly be helpful if such a variable existed.

Actually this variable already exists, :
https://github.com/zephyrproject-rtos/zephyr/blob/81f1225040d317cc9c01ab7d5ef81e098b9ab6e8/cmake/app/boilerplate.cmake#L130

set(ZEPHYR_BINARY_DIR ${PROJECT_BINARY_DIR})

So I will forward your comment to right person, as to take a look if this should be changed.