Zephyr compile speedup

RE: Zephyr compile speedup

AHaug — Thu, 04 May 2023 08:59:40 GMT

[quote user="Tamas Selmeci"]Could you please give me some hints on what the build system investigates when it decides to regenerate everything or to simply recompile the changes C sources?[/quote]

I think the way to best understand how this works is to investigate 'build and configuration systems' and 'Build system (Cmake)' under 'Application development' at https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/app_dev.html in your own time

[quote user="Tamas Selmeci"]In fact, I've added a new board under a directory out of NCS directory tree; and west build is invoked like this:[/quote]

Just speculation: it might be related to the board being outside and/or that you're using a custom board that is not listed within ../zephyr/boards or.../nrf/boards

Kind regards,
Andreas

RE: Zephyr compile speedup

Tamas Selmeci — Thu, 04 May 2023 06:21:34 GMT

Could you please give me some hints on what the build system investigates when it decides to regenerate everything or to simply recompile the changes C sources?

In fact, I've added a new board under a directory out of NCS directory tree; and west build is invoked like this:

west build -b $(ZEPHYR_BOARD) \
--build-dir=$(TARGET_OBJ_DIR) \
-- -DBOARD_ROOT=$(MY_BOARD_ROOT) \
-DVAR1=value \
-DVAR2=value

Some custom defines are also added by zephyr_compile_definitions, some options by zephyr_compile_options.

RE: Zephyr compile speedup

Tamas Selmeci — Wed, 03 May 2023 13:47:26 GMT

I've verifyed, compiling a sample really won't recompile everything. So there must be an error on my side.

RE: Zephyr compile speedup

AHaug — Wed, 03 May 2023 12:36:44 GMT

Could you state which sample you observed this with? I am assuming that there exists a build folder in this case Here's the output from calling west build -b nrf52840dk_nrf52840 3 times to build mesh/light_switch sample. First time is a pristine build where I expected to see everything called and generated, second call resulting in no work to do is without any changes and third call is by adding a printk to the main function which only recompiles the source code and does not go through any of the lines in the log you attached. Same result is observed with the VSCode extensions build-action button.

Even with windows it only takes a brief moment to perform the third build where modifications to main.c has been made

C:\Nordic\SDKs\ncs\v2.3.0\nrf\samples\bluetooth\mesh\light_switch>west build -b nrf52840dk_nrf52840
-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch
-- Using NCS Toolchain 2.3.0 for building. (C:/Nordic/SDKs/ncs/toolchains/v2.3.0/cmake)
-- Found Python3: C:/Nordic/SDKs/ncs/toolchains/v2.3.0/opt/bin/python.exe (found suitable exact version "3.8.2") found components: Interpreter
-- Cache files will be written to: C:/Nordic/SDKs/ncs/v2.3.0/zephyr/.cache
-- Zephyr version: 3.2.99 (C:/Nordic/SDKs/ncs/v2.3.0/zephyr)
-- Found west (found suitable version "0.14.0", minimum required is "0.7.1")
-- Board: nrf52840dk_nrf52840
-- Found host-tools: zephyr 0.15.2 (C:/Nordic/SDKs/ncs/toolchains/v2.3.0/opt/zephyr-sdk)
-- Found toolchain: zephyr 0.15.2 (C:/Nordic/SDKs/ncs/toolchains/v2.3.0/opt/zephyr-sdk)
-- Found Dtc: C:/Nordic/SDKs/ncs/toolchains/v2.3.0/opt/bin/dtc.exe (found suitable version "1.4.7", minimum required is "1.4.6")
-- Found BOARD.dts: C:/Nordic/SDKs/ncs/v2.3.0/zephyr/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840.dts
-- Generated zephyr.dts: C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build/zephyr/zephyr.dts
-- Generated devicetree_generated.h: C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build/zephyr/include/generated/devicetree_generated.h
-- Including generated dts.cmake file: C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build/zephyr/dts.cmake
Parsing C:/Nordic/SDKs/ncs/v2.3.0/zephyr/Kconfig
Loaded configuration 'C:/Nordic/SDKs/ncs/v2.3.0/zephyr/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840_defconfig'
Merged configuration 'C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/prj.conf'
Configuration saved to 'C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build/zephyr/.config'
Kconfig header saved to 'C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build/zephyr/include/generated/autoconf.h'
-- The C compiler identification is GNU 12.1.0
-- The CXX compiler identification is GNU 12.1.0
-- The ASM compiler identification is GNU
-- Found assembler: C:/Nordic/SDKs/ncs/toolchains/v2.3.0/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc.exe
CMake Warning at C:/Nordic/SDKs/ncs/v2.3.0/zephyr/CMakeLists.txt:1838 (message):
  __ASSERT() statements are globally ENABLED


-- Configuring done
-- Generating done
-- Build files have been written to: C:/Nordic/SDKs/ncs/v2.3.0/nrf/samples/bluetooth/mesh/light_switch/build
-- west build: building application
[1/280] Generating include/generated/version.h
-- Zephyr version: 3.2.99 (C:/Nordic/SDKs/ncs/v2.3.0/zephyr), build: v3.2.99-ncs2
[269/280] Linking C executable zephyr\zephyr_pre0.elf

[273/280] Linking C executable zephyr\zephyr_pre1.elf

[279/280] Linking C executable zephyr\zephyr.elf
Memory region         Used Size  Region Size  %age Used
           FLASH:      307488 B       992 KB     30.27%
             RAM:       67066 B       256 KB     25.58%
        IDT_LIST:          0 GB         2 KB      0.00%
[280/280] Generating zephyr/merged.hex

C:\Nordic\SDKs\ncs\v2.3.0\nrf\samples\bluetooth\mesh\light_switch>west build -b nrf52840dk_nrf52840
ninja: no work to do.

C:\Nordic\SDKs\ncs\v2.3.0\nrf\samples\bluetooth\mesh\light_switch>west build -b nrf52840dk_nrf52840
[3/12] Linking C executable zephyr\zephyr_pre0.elf

[6/12] Linking C executable zephyr\zephyr_pre1.elf

[11/12] Linking C executable zephyr\zephyr.elf
Memory region         Used Size  Region Size  %age Used
           FLASH:      307540 B       992 KB     30.28%
             RAM:       67066 B       256 KB     25.58%
        IDT_LIST:          0 GB         2 KB      0.00%
[12/12] Generating zephyr/merged.hex

C:\Nordic\SDKs\ncs\v2.3.0\nrf\samples\bluetooth\mesh\light_switch>

Kind regards,
Andreas

RE: Zephyr compile speedup

Tamas Selmeci — Wed, 03 May 2023 12:22:21 GMT

I'm issuing the following command:

west build -b my_board

You're right, it's going to recompile the changed source (C/H) files only, but before it would get there, spends a lot of time being busy with other things, as these console lines indicate:

-- west build: generating a build system
Loading Zephyr default modules (Zephyr base (cached)).
-- Application: /home/tselmeci/projects/rtls-monolithic.git/src/safetag
-- Cache files will be written to: /home/tselmeci/.cache/zephyr
-- Zephyr version: 3.2.99 (/home/tselmeci/projects/rtls-monolithic.git/packages/nrfconnect/nrfconnect-wlt-v5r1-safetag/zephyr)
-- Found west (found suitable version "0.14.0", minimum required is "0.7.1")
-- Board: wlt_v5r1_app
-- Found host-tools: zephyr 0.15.2 (/home/tselmeci/projects/rtls-monolithic.git/toolchain/wlt-v5r1-safetag/zephyr-sdk-0.15.2)
-- Found toolchain: zephyr 0.15.2 (/home/tselmeci/projects/rtls-monolithic.git/toolchain/wlt-v5r1-safetag/zephyr-sdk-0.15.2)
-- Found BOARD.dts: /home/tselmeci/projects/rtls-monolithic.git/src/libbsp/nrfconnect/boards/arm/wlt-v5r1/wlt_v5r1_app.dts
-- Generated zephyr.dts: /home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/zephyr.dts
-- Generated devicetree_generated.h: /home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/include/generated/devicetree_generated.h
-- Including generated dts.cmake file: /home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/dts.cmake
Parsing /home/tselmeci/projects/rtls-monolithic.git/packages/nrfconnect/nrfconnect-wlt-v5r1-safetag/zephyr/Kconfig
Loaded configuration '/home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/.config'
No change to configuration in '/home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/.config'
No change to Kconfig header in '/home/tselmeci/projects/rtls-monolithic.git/src/safetag/build/wlt-v5r1-safetag-obj/zephyr/include/generated/autoconf.h'
Loading Zephyr default modules (Zephyr base (cached)).
Loading Zephyr default modules (Zephyr base (cached)).
Loading Zephyr default modules (Zephyr base (cached)).
Loading Zephyr default modules (Zephyr base (cached)).
-- Configuring done
-- Generating done

It takes some seconds till completes, and not a single of my .c files have been recompiled. Some interesting lines are in bold.

RE: Zephyr compile speedup

AHaug — Wed, 03 May 2023 12:11:19 GMT

Hi,

In general: yes, compiling zephyr based applications do take a long time due to the build system scripts you mention. But these scripts are not invoked if you only modify the source code of your project, i.e if you add for instance a line of logging, printing or other arbitrary code to your main.c file. They are only called if you either modify said files (device tree, overlay files, config files) or perfom a pristine build every time you build.

Are you performing a pristine build every time you build? If so, it makes sense that you observe the regeneration and recompilation of every system you've mentioned. Let me know if you need to know how to do this.

The feedback can also be directed to the zephyr groups discord found under Technical resources -> Developing with Zephyr at https://www.zephyrproject.org/#

Kind regards,
Andreas