Recommended
Related

Getting More Out of Make

Daniel Veilleux

There are many reasons why you may want to modify the Makefiles that are included in the SDK examples. For instance, if you are invoking Make from the command line then you will probably also want the ability to have Make ensure that everything is up-to-date and then start GDB for you. Even if an IDE like Eclipse is invoking Make for you, enabling incremental builds can save you time. The purpose of this post is to introduce you to a few Makefiles that I created to show how Make can do more work for you.

Prereqs

If you've never used Make before then it's probably a good idea to take a look at a tutorial such as this one. The official GNU Make Manual is both intimidating and indispensable.

Links to Makefiles that implement all of the things that are discussed here are included at the bottom of this post. You may find it helpful to refer to a complete example while working your way through.

What We Get For Free

Let's start by taking a look at the existing Makefiles. These files are provided with each example in the SDK and work reasonably well on all platforms. Aside from allowing us to compile and download the examples, they give us some very valuable information:

  1. The exact version of GCC that has been used/tested by the SDK team
  2. The exact compiler/assembler/linker flags that have been used/tested with this version of GCC
  3. The dependencies that are required to build each example (including the order that header files are specified)

NOTE: The safe thing to do is to assume that the flags that aren't included in the default Makefiles are just as interesting as the ones that are. For example, the Link Time Optimization (-flto) flag was dropped starting with SDK v9 when GCC was upgraded to v4.9. The rationale for this change isn't obvious but as far as I know it was intentional (and -flto hasn't been reintroduced).

NOTE: A remduplicates function is used in the default Makefiles instead of the built-in sort function that is commonly used to remove duplicate items from a list. This is probably because there are troublesome header files in the SDK that exist in both the 'components' and 'examples' directories. If a copy of one of these header files is found in the 'components' directory before the project's local version is found then bad things can happen. For SDK v11 these troublesome files include:

  • nrf_drv_config.h
  • pstorage_platform.h
  • ant_stack_config_defs.h
  • device_manager_cnfg.h
  • hci_mem_pool_internal.h
  • hci_transport_config.h
  • nrf_gzp_config.h
  • bootloader_types.h
  • twi_master_config.h
  • bootloader_util.h
  • dfu.h
  • uicr_config.h
  • dfu_types.h
  • dfu_transport.h
  • bootloader.h.

Debug and Release Builds

Debugging is a lot easier if the project is compiled with optimizations disabled and debug information is included in the executable. On the other hand, enabling optimizations can potentially make the code faster and/or smaller and is almost always the correct thing to do before preforming power measurements or releasing the code to production. I prefer to switch between the two build configurations by creating targets that can be called using make (the default target, creates a debug-friendly executable) and make release. There are several ways to accomplish this but I prefer to have separate build directories:

DBG_BUILD_DIR := ./_debug
REL_BUILD_DIR := ./_release

We can tell Make to use these output directories by prepending their names to the object files that need to be created for each of our source files. The easiest way to do this is to create both DBG_OBJ and REL_OBJ lists and then select the correct list to use later when we know which version of the executable to build:

# Convert each source file name into the form '$(OBJ_DIR)/$(SRC_FILE).o'.
OBJ := $(patsubst %,%.o,$(basename $(SRC_FILE_NAMES)))
DBG_OBJ := $(addprefix $(DBG_BUILD_DIR)/,$(OBJ))
REL_OBJ := $(addprefix $(REL_BUILD_DIR)/,$(OBJ))

Furthermore, if we omit the optimization and debug flags from the usual list of CFLAGS then we can add them later when the specified target is executed:

.PHONY: default
default: CFLAGS += -O0 -ggdb
default: OBJ_DIR := $(DBG_BUILD_DIR)
default: $(DBG_BUILD_DIR)/$(PROJECT_NAME).hex

.PHONY: release
release: CFLAGS += -O3 -g3
release: OBJ_DIR := $(REL_BUILD_DIR)
release: $(REL_BUILD_DIR)/$(PROJECT_NAME).bin $(REL_BUILD_DIR)/$(PROJECT_NAME).hex

.PHONY: all
all: default release

One of the consequences of this approach is that each object file that needs to be created has to be referred to using its full path (including the build directory) so we end up with two copies of each target:

$(DBG_BUILD_DIR)/%.o: %.c | $(DBG_BUILD_DIR)
    @echo Compiling file: $(notdir $<)
    @'$(CC)' $(CFLAGS) $(INCLUDES) -c -o $@ $<

...

$(REL_BUILD_DIR)/%.o: %.c | $(REL_BUILD_DIR)
    @echo Compiling file: $(notdir $<)
    @'$(CC)' $(CFLAGS) $(INCLUDES) -c -o $@ $<

As is usual, the object files are specified as prerequisites when it comes time to use the linker. The difference now is that we have to select between the DBG_OJB and REL_OBJ lists that we created before:

$(DBG_BUILD_DIR)/$(PROJECT_NAME).out: $(DBG_OBJ)
    @echo Linking ELF file: $(notdir $@)
    @'$(CC)' $(LDFLAGS) $(DBG_OBJ) -lm -o $@

...

$(REL_BUILD_DIR)/$(PROJECT_NAME).out: $(REL_OBJ)
    @echo Linking ELF file: $(notdir $@)
    @'$(CC)' $(LDFLAGS) $(REL_OBJ) -lm -o $@

The pattern should be pretty clear at this point. Creating the build directories themselves can be done with a simple target like this:

$(DBG_BUILD_DIR) $(REL_BUILD_DIR):; @mkdir $@

Don't forget to make the build directories go away when you invoke make clean:

.PHONY: clean
clean:
    @rm -rf $(DBG_BUILD_DIR)
    @rm -rf $(REL_BUILD_DIR)

Incremental Builds

The default Makefiles simply delete all of the objects that may have been built previously and start from scratch every time you kick off a build. This method works well but is kind of slow. On my machine, the ble_app_hrs example for nRF52 always takes about 7.7 seconds:

$ time make
    ...
    [Compile all the things...]
    ...
    real    0m7.741s
    user    0m7.092s
    sys     0m0.404s

On the other hand, if we enable incremental builds then the first time that the project is built it takes nearly a second longer because some dependency files need to be created before compilation can start:

$ time make release
    ...
    [Create dependencies...]
    [Compile all the things...]
    ...
    real    0m8.687s
    user    0m7.744s
    sys     0m0.508s

But if we call Make again without changing any files it requires less than a second:

$ time make release
    ...
    real    0m0.025s
    user    0m0.016s
    sys     0m0.000s

And if we force Make to think that one of the header files has been updated then it will recompile only the files that could be affected. In this case, a change to bsp.h causes the four files that include it to be recompiled in a little over one second:

$ touch ../../../../../bsp/bsp.h
$ time make release
    Compiling file: nrf_log.c
    Compiling file: main.c
    Compiling file: bsp.c
    Compiling file: bsp_btn_ble.c
    ...
    real    0m1.120s
    user    0m0.996s
    sys     0m0.088s

The trick to incremental builds is to ask GCC to parse each source file and output a list of dependencies that Make can read. These lists are then added to the Makefile itself via the include command. For example, if ./_release/app_button.o needs to be built then we can use GCC to create a list of dependencies that looks like this:

./_release/app_button.o: \
    /home/foolsday/workspace/nRF5_SDK_11.0.0/components/libraries/button/app_button.c \
    /home/foolsday/workspace/nRF5_SDK_11.0.0/components/libraries/button/app_button.h \
    /home/foolsday/workspace/nRF5_SDK_11.0.0/components/device/nrf.h \
    /home/foolsday/workspace/nRF5_SDK_11.0.0/components/device/nrf52.h \
    ...

It's a good idea to make these targets apply to both debug and release targets so they only have to be built once:

./_debug/app_button.o _release/app_button.o: \
    /home/foolsday/workspace/nRF5_SDK_11.0.0/components/libraries/button/app_button.c \
    ...

We can use the echo command to make a text file and then have GCC append its output to the end of the file. This allows the target to apply to both debug and release builds:

$(DEPS_DIR)/%.d: %.c | $(DEPS_DIR)
    @echo Adding dependency for file: $(notdir $<)
    @echo -n "$(DBG_BUILD_DIR)/$(notdir $(<:.c=.o)) " > $@
    @'$(CC)' $(CFLAGS) $(INCLUDES) -c $< -MM \
        -MT $(REL_BUILD_DIR)/$(notdir $(<:.c=.o)) >> $@

We also need to give the dependencies output directory a name, tell Make how to create it, and make sure it gets cleaned:

DEPS_DIR := ./_deps

$(DEPS_DIR):; @mkdir $@

.PHONY: clean
clean:
    @rm -rf $(DEPS_DIR)

The actual magic happens when we pull the individual dependency targets into the Makefile itself. This requires making a list of dependency file names in the same way that we created the DBG_OBJ and REL_OBJ lists above:

DEPS := $(addprefix $(DEPS_DIR)/,$(OBJ:.o=.d))

Then, we simply include them:

-include $(DEPS)

Now whenever Make is called (for any target) it will start by generating dependency files for all of the source files. Unfortunately, if GCC can't generate the dependencies for a given file because of a syntax error, missing dependency, etc. it will exit and leave an incomplete dependency file behind. We can stop it from doing that by including this magic command anywhere in the Makefile:

# This is a special target that tells make to delete a file if an error occurs
# while the file is being generated.
.DELETE_ON_ERROR:

A Target For Starting GDB

If you are not using an IDE then you'll probably want to use GDB for debugging. I prefer to have the ability to invoke make gdb and have Make compile the debug version of my project (if necessary), open a GDBServer, open a GDB client, download the firmware, and then reset the board for me. This requires a little bit of shell wrangling but it's definitely worth the hassle of setting it up.

We can tell Make to use GDB from our cross-compiler in the same way we specify the other GCC commands:

GDB := $(GNU_INSTALL_ROOT)/bin/$(GNU_PREFIX)-gdb

In order to avoid creating unnecessary config files I prefer to create a list of commands that can be read by the GDB client every time it's started. I prefer to create that command in the debug output directory:

# Init commands will be written to a file and then specified when starting GDB
# instead of relying on .gdbinit (to avoid having to enable .gbdinit files).
GDB_CMD_PATH := $(DBG_BUILD_DIR)/gdb.txt

Make is going to have to call several shell commands. We have to use a couple of tricks to ensure that it works similarly between Linux and Cygwin:

ifeq ($(OS),Windows_NT)
    include $(TEMPLATE_PATH)/Makefile.windows
    # The -e option doesn't work in cygwin so -c is used and the job is placed
    # in the background.
    TERMINAL := sh -c
    BG_JOB := &
    NRFJPROG := c:/Program Files (x86)/Nordic Semiconductor/nrf5x/bin/nrfjprog.exe
    RTT_CLIENT := c:/Program Files (x86)/SEGGER/JLink_V512e/JLinkRTTClient.exe
else
    include $(TEMPLATE_PATH)/Makefile.posix
    TERMINAL := gnome-terminal -e
    BG_JOB :=
    NRFJPROG := nrfjprog
    GDBSERVER := JLinkGDBServer
    RTT_CLIENT := JLinkRTTClient
endif

If Make is invoked with a J-Link serial number (e.g. make gdb SN=1234) then it's easier to add it to the individual commands now than it is to accomplish the same thing using conditional statements later. Also, the GDBServer uses a default port number so running multiple GDBServers on the same machine will cause a conflict unless unique port numbers are specified when starting subsequent GDBServers. A simple way to get around this (that works most of the time) is to generate a random port number:

# If multiple J-Links are attached to the computer then "SN=1234" can be used
# to specify a serial number for GDB and flash targets. A random GDB port will
# be generated so multiple targets can be debugged simultaneously.
GDB_PORT := 2331
ifdef SN
    NRFJPROG_SN := --snr $(SN)
    GDB_SERVER_SN := -select usb=$(SN)
    GDB_PORT := $(shell awk 'BEGIN{srand();printf("%4.4s", 1000+9999*rand())}')
endif

The gdb target itself starts by telling Make that we need an up-to-date debug build. It's all pretty straight-forward from there:

.PHONY: gdb
gdb: ELF_PATH = "$(DBG_BUILD_DIR)/$(PROJECT_NAME).out"
gdb: default
    @$(TERMINAL) "'$(GDBSERVER)' $(GDB_SERVER_SN) -device nrf52 \
        -if swd -port $(GDB_PORT)" $(BG_JOB)
        @echo "target remote localhost:$(GDB_PORT)" > $(GDB_CMD_PATH)
        @echo "break main" >> $(GDB_CMD_PATH)
        @echo "mon speed 10000" >> $(GDB_CMD_PATH)
        @echo "mon flash download= 1" >> $(GDB_CMD_PATH)
        @echo "load $(ELF_PATH)" >> $(GDB_CMD_PATH)
        @echo "mon reset 0" >> $(GDB_CMD_PATH)
        @$(TERMINAL) "'$(GDB)' $(ELF_PATH) -x $(GDB_CMD_PATH)"

This will leave both a GDBServer and GDB client running for you to use after Make finishes.

TLDR

The Makefiles that are included in SDK v11 are a little spartan. I keep a few Makefiles on github to use as examples for when you are ready to introduce a few common features.

I use them on Linux and Windows+Cygwin machines (I have never tried them on a Mac). To use them on Windows you'll need to replace your Makefile.windows with mine.

There is an nRF51 example here and an nRF52 example here. You should ensure that the existing SDK Makefiles work on your machine. Then you can get started by copying one of mine into its corresponding 'armgcc' directory and running make help to see the available targets. You will probably need to adjust some of the paths to the executables (e.g. to match your J-Link version in "c:/Program Files (x86)/SEGGER/JLink_V512e").

Questions, comments, and pull requests are welcome.