• State Not Answered
  • Replies 5 replies
  • Subscribers 75 subscribers
  • Views 216 views
  • Users 0 members are here
Attachments (0)
Nordic Case Info
  • Case ID: 341489
Options

Introducing ztest for existing application

Artnia

Hi guys,

we are using the nRF52833 and have a running application. Now I want to introduce unit tests and stumbled upon ztest. The idea is to add a folder where all the test are in one place. Started with this tree in the root project:
test
├── CMakeLists.txt
├── README.md
└── sample_test
    ├── CMakeLists.txt
    ├── prj.conf
    ├── testcase.yaml
    └── test_sample.c

test/CMakeLists.txt:

add_subdirectory(sample_test)

test/sample_test/CMakeLists.txt:

cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(sample_test)

# Use the existing application source tree
target_sources(app PRIVATE test_sample.c)

# Automatically inherit includes from the root project
target_include_directories(app PRIVATE ${CMAKE_SOURCE_DIR}/src)

test/sample_test/prj.conf:

CONFIG_ZTEST=y
CONFIG_ZTEST_ASSERT_VERBOSE=2

test/sample_test/testcase.yaml:

tests:
  sample_test.my_suite:
    tags: unit
    platform_allow: native_posix
    extra_args: "APP_DIR=../.."


test/sample_test/test_sample.c:

#include <zephyr/ztest.h>

ZTEST(sample_group, test_missing_information)
{
    type_defined_somewhere_in_application mode = function_defined_somewhere_in_application();
    zassert_equal(mode, type_defined_somewhere_in_application, "Test failed for missing information!");
}

ZTEST_SUITE(sample_group, NULL, NULL, NULL, NULL, NULL);

I would like to call functions defined somewhere in the app and test them here without having to provide all inc/src/etc in e.g. the CMakeLists.txt in the tests as this is already done in the root.

Currently if I run

rm -rf twister-out* && west twister -T test/sample_test/

I get errors like

test/sample_test/test_sample.c:9:8: error: unknown type name ‘type_defined_somewhere_in_application’

Is my assumption correct? Can I inherit the applications build information and "only" add the unit tests without having to provide this info again?

Thanks in advance!

Parents
  • 0

    Hi!

    in CMakeLists.txt try adding:

    FILE(GLOB app_sources src/*.c)
    target_sources(app PRIVATE ${app_sources})
    and add
    #include <your_driver/etc/where_type_is_defined_.h>
  • 0 in reply to Sigurd

    Hi Sigurd,

    /test/sample_test/CMakeLists.txt is adjusted to

    cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(sample_test)

# Use the existing application source tree
# target_sources(app PRIVATE test_sample.c)

FILE(GLOB app_sources ../../src/*.c)
target_sources(app PRIVATE ${app_sources})

# Automatically inherit includes from the root project
target_include_directories(app PRIVATE ${CMAKE_SOURCE_DIR}/src)

    Running

    rm -rf twister-out* && west twister -T test/sample_test/

    now results in

    example_1.c:1:10: fatal error: example_1.h: No such file or directory
    1 | #include "example_1.h"
      |          ^~~~~~~~~~~~~~~~~
      
example_2.c:1:10: fatal error: example_2.h: No such file or directory
    1 | #include "example_2.h"
      |          ^~~~~~~~~~~~~~~~~
      
main.c:1:10: fatal error: example_3.h: No such file or directory
    1 | #include "example_3.h"
      |          ^~~~~~~~~~~~~~~~~
      
      
etc

    So inside the application .c files it complains about not finding application .h files. But of course they are there as the application builds.

    Where to put the

    #include <your_driver/etc/where_type_is_defined_.h>

    If I do /test/sample_test/CMakeLists.txt:

    cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(sample_test)

FILE(GLOB app_sources ../../src/*.c)
target_sources(app PRIVATE ${app_sources})

# Add additional include directories
target_include_directories(app PRIVATE
    ${CMAKE_SOURCE_DIR}/../../src
    ${CMAKE_SOURCE_DIR}/../../inc
    ${CMAKE_SOURCE_DIR}/../../inc/ble_driver
    ${CMAKE_SOURCE_DIR}/../../inc/uart_driver
    ${CMAKE_SOURCE_DIR}/../../modules/nanopb

)

# Link necessary libraries
target_link_libraries(app PRIVATE ztest)

     I get errors like

    test/sample_test/../../inc/uart_driver/uart_driver.h:8:23: error: ‘CONFIG_BT_NUS_UART_BUFFER_SIZE’ undeclared here (not in a function)
    8 | #define UART_BUF_SIZE CONFIG_BT_NUS_UART_BUFFER_SIZE

    This define is generated by the application build and is based on KConfig. It resides in 

    build/app-poc/zephyr/include/generated/zephyr/

    I do not want to manually specify all possible .h files that are generated or in the SDK. I want the ztest system to inherit all. Is this possible?

  • 0 in reply to Artnia

    Hi!

    Did you make any progress?
    Could you please share some examples? I want to use ztest/twister to test my application as well, but I haven't succeeded so far.

  • 0 in reply to Udi Vahaba

    Hi Udi,

    this is how I solved it - not sure how clean it is and how zephyrish. Again the general idea was to have the unit tests as part of the actual code base and not having a separate project where I would have to redo a lot of integration.

    General
    The unit tests are using the Zephyr Test Framework (Ztest). Ztest builds a new Zephyr project where the `main.c` is replaced by a test `main.c` in Ztest. This configuration happens within the SDK so you do not have to provide any `main.c`.

    The unit tests are build for the target platform `nrf52833dk/nrf52833` and also executed there.

    UART console will be switched on and RTT console will be switched off automatically regardless of the root prj.conf. This is needed to get the feedback from the test execution on the target.

    While the actual firmware is written in C, the unit tests are written in C++.

    Root directory
    In the root there was already a file called sample.yaml (can also be named testcase.yaml). It holds tests to some subsystems but I removed them and replaced them with my own:

    sample:
  description: Bluetooth Low Energy UART service sample
  name: BLE UART service
common:
  extra_configs:
    - CONFIG_ZTEST=y
    - CONFIG_UART_CONSOLE=y
    - CONFIG_RTT_CONSOLE=n
    - CONFIG_CPP=y
  platform_allow:
    - nrf52833dk/nrf52833
  integration_platforms:
    - nrf52833dk/nrf52833
  timeout: 10
tests:
  unit.test.advertise:
    extra_args:
      - UNIT_TEST_FILE=unit_test_advertise.cpp
      - CMAKE_EXE_LINKER_FLAGS=-Wl,--wrap=isConnected


    The common/extra_configs are configuring ztest, uart/rtt and cpp.
    Under tests/unit.test.advertise the different tests are listed. Here there is only one. UNIT_TEST_FILE holds the file name of the unit test file and CMAKE_EXE_LINKER_FLAGS is used for mocking functions. If you dont mock functions, you can remove it. The nice thing here is that actual firmware remains unmodified and linker wraps mocked functions.

    Now in the root CMakelists.txt I added something like:

    if(UNIT_TEST_FILE)
  set(SOURCE_FILE tests/${UNIT_TEST_FILE})
else()
  set(SOURCE_FILE src/main.c)
endif()

target_sources(app PRIVATE
  ${SOURCE_FILE}
  ...
)


    This ensures that for the actual firmware the "real" main.c is used while for the unit tests it is replaced with the ztest one and also the unit test file is added.

    Test directory
    Next I created a /test folder in the root and added the unit_test_advertise.cpp as well as a run_test.sh script.

    The unit_test_advertise.cpp looks like this:

    #include <zephyr/ztest.h>
#include "advertise.h"

/*******************************************************************************************************/
extern "C" {
    static bool (*mock_isConnected)(void) = nullptr;

    bool __wrap_isConnected(void) {
        return mock_isConnected ? mock_isConnected() : false; // Default to false if no mock function is set
    }
}

ZTEST(adv_get_advertisement_mode_group, test_adv_get_advertisement_mode_enabled_connectable)
{
    // Mock the isConnected function to return true
    mock_isConnected = []() { return false; };

    advMode mode = adv_get_advertisement_mode();
    zassert_equal(mode, ADV_MODE_CONNECTABLE,
                  "Advertisement mode should be ADV_MODE_CONNECTABLE but got %d!", mode);
}

/*******************************************************************************************************/

// Reset function to initialize variables before each test
static void reset_test_state(void)
{
    // Reset mock functions
    mock_isConnected = nullptr;
}

/*******************************************************************************************************/
// Define the `ztest_suite_before_t` function
static void suite_before(void *data)
{
    reset_test_state();
}

/*******************************************************************************************************/
// Test suites with `ztest_suite_before_t`
ZTEST_SUITE(adv_get_advertisement_mode_group, NULL, NULL, suite_before, NULL, NULL);
/*******************************************************************************************************/


    The run_test.sh looks like this:

    #!/bin/bash

# Clean previous build outputs
rm -rf twister-out*

# Set fixed arguments
DEVICE_SERIAL="/dev/ttyACM0"

# Execute the command with fixed arguments
west twister -v -v -v -T ./.. -p nrf52833dk/nrf52833 --device-testing --device-serial $DEVICE_SERIAL --device-serial-baud 115200


    When running the script ensure you are using the "nRF Connect" terminal, not the regular bash. In VCS this is the "ˇ" next to the "+" in the terminal view:


    Obviously this wont run for you as you do not have my firmware. But it should give a general idea how it can be approached.

    Once it works, the actual firmware is build, unit tests are added to it and it will be flashed to the target. Then the feedback is returned and you see the output.

Reply
  • 0 in reply to Udi Vahaba

    Hi Udi,

    this is how I solved it - not sure how clean it is and how zephyrish. Again the general idea was to have the unit tests as part of the actual code base and not having a separate project where I would have to redo a lot of integration.

    General
    The unit tests are using the Zephyr Test Framework (Ztest). Ztest builds a new Zephyr project where the `main.c` is replaced by a test `main.c` in Ztest. This configuration happens within the SDK so you do not have to provide any `main.c`.

    The unit tests are build for the target platform `nrf52833dk/nrf52833` and also executed there.

    UART console will be switched on and RTT console will be switched off automatically regardless of the root prj.conf. This is needed to get the feedback from the test execution on the target.

    While the actual firmware is written in C, the unit tests are written in C++.

    Root directory
    In the root there was already a file called sample.yaml (can also be named testcase.yaml). It holds tests to some subsystems but I removed them and replaced them with my own:

    sample:
  description: Bluetooth Low Energy UART service sample
  name: BLE UART service
common:
  extra_configs:
    - CONFIG_ZTEST=y
    - CONFIG_UART_CONSOLE=y
    - CONFIG_RTT_CONSOLE=n
    - CONFIG_CPP=y
  platform_allow:
    - nrf52833dk/nrf52833
  integration_platforms:
    - nrf52833dk/nrf52833
  timeout: 10
tests:
  unit.test.advertise:
    extra_args:
      - UNIT_TEST_FILE=unit_test_advertise.cpp
      - CMAKE_EXE_LINKER_FLAGS=-Wl,--wrap=isConnected


    The common/extra_configs are configuring ztest, uart/rtt and cpp.
    Under tests/unit.test.advertise the different tests are listed. Here there is only one. UNIT_TEST_FILE holds the file name of the unit test file and CMAKE_EXE_LINKER_FLAGS is used for mocking functions. If you dont mock functions, you can remove it. The nice thing here is that actual firmware remains unmodified and linker wraps mocked functions.

    Now in the root CMakelists.txt I added something like:

    if(UNIT_TEST_FILE)
  set(SOURCE_FILE tests/${UNIT_TEST_FILE})
else()
  set(SOURCE_FILE src/main.c)
endif()

target_sources(app PRIVATE
  ${SOURCE_FILE}
  ...
)


    This ensures that for the actual firmware the "real" main.c is used while for the unit tests it is replaced with the ztest one and also the unit test file is added.

    Test directory
    Next I created a /test folder in the root and added the unit_test_advertise.cpp as well as a run_test.sh script.

    The unit_test_advertise.cpp looks like this:

    #include <zephyr/ztest.h>
#include "advertise.h"

/*******************************************************************************************************/
extern "C" {
    static bool (*mock_isConnected)(void) = nullptr;

    bool __wrap_isConnected(void) {
        return mock_isConnected ? mock_isConnected() : false; // Default to false if no mock function is set
    }
}

ZTEST(adv_get_advertisement_mode_group, test_adv_get_advertisement_mode_enabled_connectable)
{
    // Mock the isConnected function to return true
    mock_isConnected = []() { return false; };

    advMode mode = adv_get_advertisement_mode();
    zassert_equal(mode, ADV_MODE_CONNECTABLE,
                  "Advertisement mode should be ADV_MODE_CONNECTABLE but got %d!", mode);
}

/*******************************************************************************************************/

// Reset function to initialize variables before each test
static void reset_test_state(void)
{
    // Reset mock functions
    mock_isConnected = nullptr;
}

/*******************************************************************************************************/
// Define the `ztest_suite_before_t` function
static void suite_before(void *data)
{
    reset_test_state();
}

/*******************************************************************************************************/
// Test suites with `ztest_suite_before_t`
ZTEST_SUITE(adv_get_advertisement_mode_group, NULL, NULL, suite_before, NULL, NULL);
/*******************************************************************************************************/


    The run_test.sh looks like this:

    #!/bin/bash

# Clean previous build outputs
rm -rf twister-out*

# Set fixed arguments
DEVICE_SERIAL="/dev/ttyACM0"

# Execute the command with fixed arguments
west twister -v -v -v -T ./.. -p nrf52833dk/nrf52833 --device-testing --device-serial $DEVICE_SERIAL --device-serial-baud 115200


    When running the script ensure you are using the "nRF Connect" terminal, not the regular bash. In VCS this is the "ˇ" next to the "+" in the terminal view:


    Obviously this wont run for you as you do not have my firmware. But it should give a general idea how it can be approached.

    Once it works, the actual firmware is build, unit tests are added to it and it will be flashed to the target. Then the feedback is returned and you see the output.

Children
No Data
Related
Terms of service