nrf9160 ztest framework : unable to get a unit test to work

[emeralddragon]

Hi. I am working on the nrf9160. I want to use a unit test framework for my application code (which is a combination of C/C++14). I thought about using gtest/gmock or ztest. For now I am trying to get to run a single sample provided: zephyr/sample/testing/integration up and running but unable to do so. I am working on a win 64 laptop.  Can you please explain how I can run a unit test please. I looked at the following webpages:https://docs.zephyrproject.org/1.12.0/subsystems/test/ztest.html, https://docs.zephyrproject.org/latest/reference/kconfig/CONFIG_ZTEST.html, https://developer.nordicsemi.com/nRF_Connect_SDK/doc/0.4.0/zephyr/guides/test/ztest.html. Please dont direct me to the same webpages.

what are the steps to run a unit/integration test on the win 64 enviroment? Do I have to run cmake? Trying to run the sanitycheck script fails due to several reasons:

On cmd: 

xxxxx\zephyr\scripts>python sanitycheck -s ..\samples\testing\integration
Install the anytree module to use the --test-tree option
Install tabulate python module with pip to use --device-testing option.
Renaming output directory to xxx\zephyr\scripts\sanity-out.4
INFO - JOBS: 8
INFO - Selecting default platforms per test case
INFO - Building initial testcase list...
INFO - 0 test configurations selected, 163531 configurations discarded due to filters.
INFO - Adding tasks to the queue...

INFO - [32m0 of 0[39m tests passed (0.00%), [39m0[39m failed, 0 skipped with [39m0[39m warnings in 7.89 seconds
INFO - In total 741 test cases were executed on 0 out of total 225 platforms (0.00%)
'stty' is not recognized as an internal or external command,
operable program or batch file.

On Cygwin: "/usr/bin/env: ‘python3’: No such file or directory" python3 not found. I had to change the shebang in sanitycheck to python instead. After that another error:

$ xxx/zephyr/scripts/sanitycheck -s ../samples/testing/integration
Install the anytree module to use the --test-tree option
Install tabulate python module with pip to use --device-testing option.
Traceback (most recent call last):
File "xxx/zephyr/scripts/sanitycheck", line 213, in <module>
import edtlib
ModuleNotFoundError: No module named 'edtlib'

when I try to source the zephyr env variable on cygwin: here is the error i get: 

source xxx/zephyr/zephyr-env.sh
-bash: $'\r': command not found
-bash: xxx/zephyr/zephyr-env.sh: line 70: syntax error: unexpected end of file

All I want to do is write unit tests and then eventually integration tests using the ztest framework, first on win64 platform. I hope someone can help me out. Please provide a step wise approach with commands to be run. Noob here.

[Simon]

Could you test it again in Cygwin and run the following commands first:

cd <sourcecode_root>\ncs
pip3 install -r zephyr\scripts\requirements.txt
pip3 install -r nrf\scripts\requirements.txt
pip3 install -r bootloader\mcuboot\scripts\requirements.txt

Best regards,

Simon

[emeralddragon]

Thanks Simon. I did as you said,  anytree, tabulate were not yet installed. But the issue remains: 

zephyr/scripts/sanitycheck -s ../samples/testing/integration
Traceback (most recent call last):
File "zephyr/scripts/sanitycheck", line 213, in <module>
import edtlib
ModuleNotFoundError: No module named 'edtlib'

[Simon]

It seems like this line is the one that includes the path where zephyr/scripts/dts/edtlib.py is located.

Have you set the ZEPHYR_BASE?

Could you try to add the following below this line (above import edtlib), to see what the path is equal to:

print("scripts/dts/ path",os.path.join(ZEPHYR_BASE, "scripts", "dts"))

Best regards,

Simon

[emeralddragon]

Hi SImon,

I see what I got previously after adding "dts." to the import paths.

$ zephyr/scripts/sanitycheck -s ../samples/testing/integration

scripts/dts/ path C:\ConnectivityNodes_Cellular\zephyr\scripts\dts
Traceback (most recent call last):
File "zephyr/scripts/sanitycheck", line 1403, in <module>
class Platform:
File "zephyr/scripts/sanitycheck", line 1408, in Platform
platform_schema = scl.yaml_load(os.path.join(ZEPHYR_BASE,
File "C:\ConnectivityNode_Cellular\zephyr\scripts\sanity_chk\scl.py", line 29, in yaml_load
with open(filename, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\ConnectivityNodes_Cellular\\zephyr\\scripts\\sanity_chk\\platform-schema.yaml'

[Simon]

I managed to make this work, I did the following:

• Installed ncs v1.2.0. Check out our newly released app Toolchain Manager, which simplifies the process of installing ncs

• Opened Git Bash
• Typed in:

cd <location_of_ncs>/ncs/zephyr
export ZEPHYR_BASE=c:/Nordic_SDK/ncs/zephyr

• Copied the sample ..\ncs\zephyr\samples\testing\integration into ..\ncs\zephyr\tests
• Ran the following command:

python scripts/sanitycheck -s tests/integration

• Got the following output:

INFO    - JOBS: 8
INFO    - Selecting default platforms per test case
INFO    - Building initial testcase list...
INFO    - 0 test configurations selected, 163755 configurations discarded due to filters.
INFO    - Adding tasks to the queue...
INFO    - 0 of 0 tests passed (0.00%), 0 failed, 0 skipped with 0 warnings in 4.73 seconds
INFO    - In total 742 test cases were executed on 0 out of total 225 platforms (0.00%)
Renaming output directory to C:\Nordic_SDK\ncs\zephyr\sanity-out.4

Best regards,

Simon

[emeralddragon]

Hi Simon. Thanks for the update. After copying the folder to zephyr/tests it seems to be working. Here is my output:

INFO - JOBS: 8
INFO - Selecting default platforms per test case
INFO - Building initial testcase list...
INFO - 0 test configurations selected, 163755 configurations discarded due to filters.
INFO - Adding tasks to the queue...
INFO - 0 of 0 tests passed (0.00%), 0 failed, 0 skipped with 0 warnings in 5.14 seconds
INFO - In total 742 test cases were executed on 0 out of total 225 platforms (0.00%)
scripts/dts/ path xxx/zephyr\scripts\dts
Renaming output directory to xxx\zephyr\sanity-out.1

But I am surprised that the script found 742 tests and it was not run on any platform. It says  "0 test configurations selected" and "0 of 0 tests passed (0.00%), 0 failed" How do I run the testcase from main.c and see the output?

[Vojislav]

Hi,

I am also trying to get unit testing working on nrf91, and also got to this part. I am developing on Ubuntu. What are the next steps to get tests from main.c running?

Best regards,
Vojislav