Unit testing with Unity and SoftDevice

[Marcus]

Hi,

I'm currently in the process of integrating unit testing into our firmware development process. I'm trying to execute on-target tests with the Unity test framework, using the Nordic-modified version provided in this post.

This has been working fine for modules which don't make calls to the softdevice, but now I'm attempting to write tests for low-level driver modules which interact directly with libraries in the SDK, all the functions return an error code telling me that the softdevice isn't enabled.

Naturally my first response was to attempt to enable the softdevice in the setUp() function called by the test harness, in the same way it's enabled in my production code:

void softdevice_init(void)
{
	ret_code_t err_code;

    nrf_clock_lf_cfg_t clock_lf_cfg =	{.source        = NRF_CLOCK_LF_SRC_RC,            \
                                    	.rc_ctiv       = 16,                                \
										.rc_temp_ctiv  = 2,                                \

										.xtal_accuracy = NRF_CLOCK_LF_XTAL_ACCURACY_20_PPM}; // ignored for NRF_CLOCK_LF_SRC_RC

    NRF_LOG_DEBUG("Inside SoftDevice init\r\n");

    // Initialize the SoftDevice handler module.
    SOFTDEVICE_HANDLER_INIT(&clock_lf_cfg, NULL);

    uint32_t ram_start = 0;
    err_code = softdevice_app_ram_start_get(&ram_start);
    APP_ERROR_CHECK(err_code);

    err_code = softdevice_enable(&ram_start);
	APP_ERROR_CHECK(err_code);

	err_code = softdevice_sys_evt_handler_set(system_evt_dispatch);
    APP_ERROR_CHECK(err_code);
}

void setUp(void)
{
	result      = 0xffff;
	result_code = 0xffffffff;

	softdevice_init();
}

...but this just seems to make things worse, causing the entire test suite to fail due to an "unexpected reset reason", as seen in this UART output:

expected: 0x00000000
../../../../../unity_fw/app_test_runner.c:302:sam_hw_fs_init_SHOULD_return_success_ON_FIRST_CALL:FAIL:FAIL: unexpected_reset
Reset reason: 0x00000004

Do you have any advice on how the test environment can be modified to be softdevice compatible?

Thanks, Marcus

[Yaniv Nis]

Hi,

We are working at Jumper on a virtual device framework that enables what you need. Contact me at [email protected] so I'll better understand your needs and I'll add you to our free beta program.

Best,

Yaniv

[Hung Bui]

Hi Marcus,

Normally, when you unit test a module, you would need to create mock functions outside of the module that being tested. This is applied also for the softdevice API called from the module.

That's what we do when we do unit test for our SDK. We check the params passed to the Softdevice API are correct, check if the APIs called correct number of time and in correct order, etc.

I assume in your case, you are doing on-target testing and you actually need the device to operate to test ? shouldn't it be at system level test, not unit test ? Could you share some more information ?

[Marcus]

Hi Hung,

I would agree that normally a unit test should isolate the module in question by replacing the interfaces it has with other modules with mock functions.

However the module in question is a low-level flash driver which makes use of the FDS from the SDK to perform memory transactions in flash. To test that the memory contents are correctly being manipulated, the SoftDevice is required, and without having performed these tests I would argue that the module hasn't been properly tested.

Mocking SoftDevice function calls so that they retain their core functionality is an option, but would probably take a lot of work. I suppose you could also argue that, as Nordic are already unit testing the SDK modules, that there's no need for me to go that low-level with my tests and I can just drop the calls in as-is - but that seems a little risky in my opinion.

[Hung Bui]

Hi Marcus,

Seems that you are doing system level test. For that purpose I would suggest you do what we do to test fds at system level:

• Create a simple application that take input from UART or RTT for configuration test case and data.

• In that application, you init the softdevice and use the data from input to run the module under test to manipulate the flash.

• After the operation is finished, you can use nrfjprog to verify the flash.

• We created a python script to automate providing test input and check flash output.

• We don't use Unity for testing at system level.

With what you provided and the error log, seems that the softdevice init wasn't run properly. You may want to check if there is any error code when you call softdevice_enable() and softdevice_sys_evt_handler_set(). Have you made sure you configure RAM correctly ? softdevice_app_ram_start_get() only return the application RAM start address configured in the compiler, not the actual RAM needed fort the softdevice.