Unity Test Framework - Adding multiple test files

[Cody]

Hello,

I am trying to integrate the Unity test framework into our project and running into issues with how to add additional test files.
Nordic provides one example project in the nRF Connect SDK repository here: https://github.com/nrfconnect/sdk-nrf/tree/master/tests/unity/example_test.

This sample was helpful but the moment I attempt to add another source file, "example_test_2.c" for example, the generation of the test binary fails.
It seems that I can only generate one test runner per CMake project with the CMake directive: "test_runner_generate(src/example_test.c)". 

Can Nordic provide any help or direction here?

[Amanda Hsieh]

Hi Cody, 

It seems like unity framework limitation: https://github.com/ThrowTheSwitch/Unity/blob/b0032caca4402da692548f2ee296d3b1b1251ca0/auto/generate_test_runner.rb#L484

The script accepts only one file. You should create a test case sample, similar to example_Test, and have there one file with tests. 

-Amanda H.

[Cody]

Have one file with all the tests? What if I have hundreds of tests for my project, that file is going to be huge.

[Amanda Hsieh]

Hi Cody,

In general, you have one file per module, so multiple test binaries for the project.

-Amanda H.

[Cody]

Okay, my project does have multiple modules that I have separated out. Just to clarify, when you mention "module" you are speaking to a grouping of source files referenced in a CMakeLists.txt? For example, in the asset_track sample application, gps_controller and env_sensors would be considered "modules"?

I will try implementing it in this way and get back to you.

[Amanda Hsieh]

Hi Cody, 

Yes, in theory, you can create two tests for the same module.

-Amanda H.

[Cody]

Hey Amanda Hsieh,

I know it has been quite some time since my last update but I am still having troubles with this.

Using the Unity test example located in the nRF Connect SDK here: https://github.com/nrfconnect/sdk-nrf/tree/master/tests/unity/example_test, if the developer wanted to add an additional file to the module, say src/uut/uut_2.c, and the developer wanted to test the functions in this file with the same test project how would the developer go about doing this?

One way to do so would be to modify the CMakeLists.txt:

# add module uut
target_sources(app PRIVATE src/uut/uut.c)
target_sources(app PRIVATE src/uut/uut_2.c) <-- add new source file here
target_include_directories(app PRIVATE ./src/uut)

and then add additional test functions inside example_test.c but that can get huge really fast, even though both source files are part of the same module it is still hard to organize.

If you try to add example_test_2.c to the project to organize all tests for uut_2.c inside of, when you add "test_runner_generate(src/example_test_2.c)" to CMakeLists.txt you will get an error during compilation. This is due to the fact that, as you said previously, the script only accepts one file.

In summary, even though I have nicely organized modules as part of my project, the Unity test project for the modules requires that you only have one test file for the entire module. This "test runner" file gets huge and does not allow for organization of specific files inside the module.

Has anyone at Nordic ran into this and can provide a solution and/or workaround?

Thanks,
Cody

[Amanda Hsieh]

Hi Cody, 

I got the following from the developer:

so it's a limitation of unity, there is one runner for one test file generated. there can be multiple uut_2.c files attached to the project but only one test file. Imo, if there are too many tests for a single file there, I would try to split them logically into multiple test projects for the same module.

-Amanda H.

[Cody]

Okay, just to confirm, every time I "split them out logically into multiple test projects" I am going to need a new CMakeLists.txt, Kconfig, and prj.conf for each test file to which CMake/Make is ran across and a new binary is generated?

Looking at a Unity sample located at https://github.com/ThrowTheSwitch/Unity/tree/master/examples/example_1 it would be nice if I could organize in a similar fashion. I think the limitation might exist as part of the the nRF Connect SDK's Unity helper functions located in https://github.com/nrfconnect/sdk-nrf/blob/master/tests/unity/CMakeLists.txt and less so on Unity based on the example project.

Thanks for your continued assistance,
Cody

[Amanda Hsieh]

Hi Cody,

Cody said:

just to confirm, every time I "split them out logically into multiple test projects" I am going to need a new CMakeLists.txt, Kconfig, and prj.conf for each test file to which CMake/Make is ran across and a new binary is generated?

Yes. 

-Amanda H.