Zephyr compile speedup

[Tamas Selmeci]

Despite I have a rather modern machine (Intel i7, 12 cores) and a not too big project, compiling a Zephyr application takes too much time. I'm used to FreeRTOS compilation times being 2-5x faster than this. (Of course, Zephyr code base is larger) If my suspicion is right, substantial part of the overall compilation time is consumed by various build system scripts to parse devicetree, regenerate devicetree-related files and so on. It's mostly just a waste of time, because during a regular development the devicetree isn't touched, so there'd be no need to parse again.

Is there any way to have the Zephyr build system recompile the source files only, without executing time-consuming scripts e.g. to regenerate devicetree stuff?

(there's ccache installed on my machine, it's just working for the GCC-phase of compilation)

Any tips to speed up compile are welcome ;)

thanks, regards,

[AHaug]

Hi,

In general: yes, compiling zephyr based applications do take a long time due to the build system scripts you mention. But these scripts are not invoked if you only modify the source code of your project, i.e if you add for instance a line of logging, printing or other arbitrary code to your main.c file. They are only called if you either modify said files (device tree, overlay files, config files) or perfom a pristine build every time you build.

Are you performing a pristine build every time you build? If so, it makes sense that you observe the regeneration and recompilation of every system you've mentioned. Let me know if you need to know how to do this.

The feedback can also be directed to the zephyr groups discord found under Technical resources -> Developing with Zephyr at https://www.zephyrproject.org/#

Kind regards,
Andreas

[Tamas Selmeci]

I'm issuing the following command:

west build -b my_board

You're right, it's going to recompile the changed source (C/H) files only, but before it would get there, spends a lot of time being busy with other things, as these console lines indicate:

-- west build: generating a build system
Loading Zephyr default modules (Zephyr base (cached)).
-- Application: /home/tselmeci/projects/rtls-monolithic.git/src/safetag
-- Cache files will be written to: /home/tselmeci/.cache/zephyr
-- Zephyr version: 3.2.99 (/home/tselmeci/projects/rtls-monolithic.git/packages/nrfconnect/nrfconnect-wlt-v5r1-safetag/zephyr)
-- Found west (found suitable version "0.14.0", minimum required is "0.7.1")
-- Board: wlt_v5r1_app
-- Found host-tools: zephyr 0.15.2 (/home/tselmeci/projects/rtls-monolithic.git/toolchain/wlt-v5r1-safetag/zephyr-sdk-0.15.2)
-- Found toolchain: zephyr 0.15.2 (/home/tselmeci/projects/rtls-monolithic.git/toolchain/wlt-v5r1-safetag/zephyr-sdk-0.15.2)
-- Found BOARD.dts: /home/tselmeci/projects/rtls-monolithic.git/src/libbsp/nrfconnect/boards/arm/wlt-v5r1/wlt_v5r1_app.dts
-- Generated zephyr.dts: /home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/zephyr.dts
-- Generated devicetree_generated.h: /home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/include/generated/devicetree_generated.h
-- Including generated dts.cmake file: /home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/dts.cmake
Parsing /home/tselmeci/projects/rtls-monolithic.git/packages/nrfconnect/nrfconnect-wlt-v5r1-safetag/zephyr/Kconfig
Loaded configuration '/home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/.config'
No change to configuration in '/home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/.config'
No change to Kconfig header in '/home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/include/generated/autoconf.h'
Loading Zephyr default modules (Zephyr base (cached)).
Loading Zephyr default modules (Zephyr base (cached)).
Loading Zephyr default modules (Zephyr base (cached)).
Loading Zephyr default modules (Zephyr base (cached)).
-- Configuring done
-- Generating done

It takes some seconds till completes, and not a single of my .c files have been recompiled. Some interesting lines are in bold.

[AHaug]

Could you state which sample you observed this with? I am assuming that there exists a build folder in this case Here's the output from calling west build -b nrf52840dk_nrf52840 3 times to build mesh/light_switch sample. First time is a pristine build where I expected to see everything called and generated, second call resulting in no work to do is without any changes and third call is by adding a printk to the main function which only recompiles the source code and does not go through any of the lines in the log you attached. Same result is observed with the VSCode extensions build-action button. 

Even with windows it only takes a brief moment to perform the third build where modifications to main.c has been made

C:\Nordic\SDKs\ncs\v2.3.0\nrf\samples\bluetooth\mesh\light_switch>west build -b nrf52840dk_nrf52840
-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch
-- Using NCS Toolchain 2.3.0 for building. (C:/Nordic/SDKs/ncs/toolchains/v2.3.0/cmake)
-- Found Python3: C:/Nordic/SDKs/ncs/toolchains/v2.3.0/opt/bin/python.exe (found suitable exact version "3.8.2") found components: Interpreter
-- Cache files will be written to: C:/Nordic/SDKs/ncs/v2.3.0/zephyr/.cache
-- Zephyr version: 3.2.99 (C:/Nordic/SDKs/ncs/v2.3.0/zephyr)
-- Found west (found suitable version "0.14.0", minimum required is "0.7.1")
-- Board: nrf52840dk_nrf52840
-- Found host-tools: zephyr 0.15.2 (C:/Nordic/SDKs/ncs/toolchains/v2.3.0/opt/zephyr-sdk)
-- Found toolchain: zephyr 0.15.2 (C:/Nordic/SDKs/ncs/toolchains/v2.3.0/opt/zephyr-sdk)
-- Found Dtc: C:/Nordic/SDKs/ncs/toolchains/v2.3.0/opt/bin/dtc.exe (found suitable version "1.4.7", minimum required is "1.4.6")
-- Found BOARD.dts: C:/Nordic/SDKs/ncs/v2.3.0/zephyr/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840.dts
-- Generated zephyr.dts: C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build/zephyr/zephyr.dts
-- Generated devicetree_generated.h: C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build/zephyr/include/generated/devicetree_generated.h
-- Including generated dts.cmake file: C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build/zephyr/dts.cmake
Parsing C:/Nordic/SDKs/ncs/v2.3.0/zephyr/Kconfig
Loaded configuration 'C:/Nordic/SDKs/ncs/v2.3.0/zephyr/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840_defconfig'
Merged configuration 'C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/prj.conf'
Configuration saved to 'C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build/zephyr/.config'
Kconfig header saved to 'C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build/zephyr/include/generated/autoconf.h'
-- The C compiler identification is GNU 12.1.0
-- The CXX compiler identification is GNU 12.1.0
-- The ASM compiler identification is GNU
-- Found assembler: C:/Nordic/SDKs/ncs/toolchains/v2.3.0/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc.exe
CMake Warning at C:/Nordic/SDKs/ncs/v2.3.0/zephyr/CMakeLists.txt:1838 (message):
  __ASSERT() statements are globally ENABLED


-- Configuring done
-- Generating done
-- Build files have been written to: C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build
-- west build: building application
[1/280] Generating include/generated/version.h
-- Zephyr version: 3.2.99 (C:/Nordic/SDKs/ncs/v2.3.0/zephyr), build: v3.2.99-ncs2
[269/280] Linking C executable zephyr\zephyr_pre0.elf

[273/280] Linking C executable zephyr\zephyr_pre1.elf

[279/280] Linking C executable zephyr\zephyr.elf
Memory region         Used Size  Region Size  %age Used
           FLASH:      307488 B       992 KB     30.27%
             RAM:       67066 B       256 KB     25.58%
        IDT_LIST:          0 GB         2 KB      0.00%
[280/280] Generating zephyr/merged.hex

C:\Nordic\SDKs\ncs\v2.3.0\nrf\samples\bluetooth\mesh\light_switch>west build -b nrf52840dk_nrf52840
ninja: no work to do.

C:\Nordic\SDKs\ncs\v2.3.0\nrf\samples\bluetooth\mesh\light_switch>west build -b nrf52840dk_nrf52840
[3/12] Linking C executable zephyr\zephyr_pre0.elf

[6/12] Linking C executable zephyr\zephyr_pre1.elf

[11/12] Linking C executable zephyr\zephyr.elf
Memory region         Used Size  Region Size  %age Used
           FLASH:      307540 B       992 KB     30.28%
             RAM:       67066 B       256 KB     25.58%
        IDT_LIST:          0 GB         2 KB      0.00%
[12/12] Generating zephyr/merged.hex

C:\Nordic\SDKs\ncs\v2.3.0\nrf\samples\bluetooth\mesh\light_switch>

Kind regards,
Andreas

[Tamas Selmeci]

I've verifyed, compiling a sample really won't recompile everything. So there must be an error on my side.

[Tamas Selmeci]

Could you please give me some hints on what the build system investigates when it decides to regenerate everything or to simply recompile the changes C sources?

In fact, I've added a new board under a directory out of NCS directory tree; and west build is invoked like this:

west build -b $(ZEPHYR_BOARD) \
    --build-dir=$(TARGET_OBJ_DIR) \
    -- -DBOARD_ROOT=$(MY_BOARD_ROOT) \
    -DVAR1=value \
    -DVAR2=value

Some custom defines are also added by zephyr_compile_definitions, some options by zephyr_compile_options.

[AHaug]

Tamas Selmeci said:

Could you please give me some hints on what the build system investigates when it decides to regenerate everything or to simply recompile the changes C sources?

I think the way to best understand how this works is to investigate 'build and configuration systems' and 'Build system (Cmake)' under 'Application development' at https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/app_dev.html in your own time

Tamas Selmeci said:

In fact, I've added a new board under a directory out of NCS directory tree; and west build is invoked like this:

Just speculation: it might be related to the board being outside and/or that you're using a custom board that is not listed within ../zephyr/boards or.../nrf/boards

Kind regards,
Andreas