Testing framework for nRF Connect SDK / Zephyr

[naps]

Hi,
I want to run unit tests on an actual device (nRF52dk) - is it possible?
When I'm trying to run
`west build -b nrf52840dk_nrf52840` on `ncs/nrf/tests/unity/example_test`
the compiler complains about missing `#include <setjmp.h>` file.

On the other hand, trying to run native Zephyr tests, as explained in docs.zephyrproject.org/.../index.html
`./scripts/sanitycheck -v -T tests/foo/bar/ -p nrf52840dk_nrf52840`
I cannot execute test on the platform - the test is getting built only
`0 tests executed on platforms, 1 tests were only built.`

One more question, can you explain why in nRF Connect SDK you are using unity + cmock instead od native ztest from the Zephyr package?

[helsing]

Hi Adam,

Thank you for your questions. I will get back to you after the weekend.

Have a nice weekend!

Best Regards,

Håkon

[naps]

Finally, I found a way to run tests on nRF52DK device.
Maybe it is not very obvious; how to run tests on an actual device (also dealing with native zephyr documentation), but finally, I was able to get there.
The key is to pass additional parameters to `sanitycheck` app.
The following call seems to load and run compiled tests on the device and outputs the proper results:
`./scripts/sanitycheck -T tests/foo/bar/ -p nrf52840dk_nrf52840 --device-testing --device-serial /dev/ttyACM0`

From what I understand the unity/cmock testing framework is for your internal use - correct me as I'm wrong. In our case, we cannot test our code as x86 POSIX env, as parts of c/c++ std lib differs a lot while we are heavily using them.

That's all because I'm relating to my previous experience with the esp32-idf integrated test framework, where I was able to implement the entire multi-device test environment on the target hardware - also using CI.
We are planning to implement such test env - this time in the TDD manner - while porting our old code. We are planning to migrate our ZigBee/FreeRTOS application (nRF5 SDK for Thread and Zigbee ) to Zephyr or a bare-metal application and still looking for the most optimal solution.

Can you also tell me about your plans for future SDK development?
Would it still be possible to write applications using nRF5 SDK for Thread and Zigbee? For how long can we expect your support on that?

Thanks for your help, I'm looking forward to your answers.

[helsing]

Hi Adam,

First of all, I am very sorry you have not heard from me until now.

naps said:

Can you also tell me about your plans for future SDK development?

In general, we are aiming to move all new development to the nRF Connect SDK. All new features will be in nRF Connect SDK while nRF5 SDK for Thread and ZigBee will be in maintenance mode. You might get bugfixes, perhaps new profiles etc. In other words, it is highly recommended to switch over to the nRF Connect SDK. There will be a new release soon.

 

I want to run unit tests on an actual device (nRF52dk) - is it possible?

One more question, can you explain why in nRF Connect SDK you are using unity + cmock instead od native ztest from the Zephyr package?

Here is a related case that touch on these subjects. I will also get back to you on this. 

https://devzone.nordicsemi.com/f/nordic-q-a/54255/how-did-nordic-mock-headers-like-app_timer-h-with-many-dependencies

Best regards,

Håkon

[helsing]

Hi Adam,

naps said:

From what I understand the unity/cmock testing framework is for your internal use - correct me as I'm wrong.

The testing framework is intended for nRF Connect SDK use. Some tests may get public and it is expected that users will use it for their own testing.

naps said:

In our case, we cannot test our code as x86 POSIX env, as parts of c/c++ std lib differs a lot while we are heavily using them.

A comment from one of the developers is that unity in nRF Connect SDK can only be built on native_posix as of now.

the compiler complains about missing `#include <setjmp.h>` file.

The developer confirms seeing the same issue with setjmp.

Both of these issues are expected to be fixed in a future update, but unfortunately we are unable to provide a schedule at this time.