compiling and running ble-pc-driver tests

[Yorai Levi]

Hi, I am trying to compile and run the ble-pc-driver tests but am struggling.

My best guess based on the documentation is that I am specifying something wrong when configuring the cmake project (im using cmake gui).

based on the 

pc-ble-driver/test at master · NordicSemiconductor/pc-ble-driver (github.com)

For example, if you have two PCA10056 boards, you provide the following defines the CMake when generating the project files: -DBLE_DRIVER_TEST_PCA10056_A=<SEGGER_SERIAL_NUMBER>:<SERIAL_PORT> -DBLE_DRIVER_TEST_PCA10056_B=<SEGGER_SERIAL_NUMBER>:<SERIAL_PORT> -DTEST_SOFTDEVICE_API=1

The test targets needs to be ran manually, the naming of the test targets are "test_run_sdv<SoftDevice_API_version>_<SoftDevice_type>_pca". Each of the targets first program the devices with the correct connectivity firmware and then starts the test. The test result is written in junit format in directory test-reports below the the test build directory.

I added specified these: 

> nrfjprog --com
682179628 COM3 VCOM0
682935919 COM6 VCOM0

which seem to generate these files when built, based on the above text these aren't all the tests that should be generated

even so running test_sd_api_v5.exe also fails on some tests

pc-ble-driver\test\softdevice_api\testcase_security.cpp(329): FAILED:
REQUIRE( p->open() == ((0x0) + 0) )
with expansion:
13 == 0
with message:
serial-ports:
port:COM3 baud-rate:1000000
port:COM6 baud-rate:1000000
retransmission-interval:250
response-timeout:1500
ble-mtu:150
hardware-info:No hardware info provided.
log-level:INFO
iterations(if relevant):10

[18:06:53.850888] [t:6072] [debug] [periph ] sd_rpc_close error code: 0x8018
[18:06:53.851338] [t:6072] [error] 0xffc7ac, in logHandler, not able to find adapter to invoke log handler on, invalid map<K, T> key
[18:06:53.851498] [t:6072] [debug] [periph ] sd_rpc_adapter_delete called and returned.
[18:06:53.852291] [t:6072] [debug] [central][log] severity:INFO message: serial port COM3 closed.
[18:06:53.852457] [t:6072] [debug] [central] sd_rpc_close error code: 0x00
[18:06:53.852593] [t:6072] [debug] [central] sd_rpc_adapter_delete called and returned.
===============================================================================
test cases: 8 | 3 passed | 5 failed
assertions: 177 | 172 passed | 5 failed

* They all fail on the same assertion.

Any assistance on the matter will be of use.

[Einar Thorsrud]

Hi,

I am not able to see exactly what happened. I suggest you use a release and not master, though that may not be related in this case. Also, I suggest follow the instructions on how to build the pc-ble-driver to the point, so that you don't miss anything. 

[Yorai Levi]

The master head is at tag v4.1.4 which means there shouldn't be an issue.

I tried compiling on linux in hope that maybe I am passing some of the arguments wrong on windows but to no avail

cmake -G Ninja -
DCOMPILE_CONNECTIVITY=1 -DCONNECTIVITY_VERSION=1.0.0 -DBLE_DRIVER_TE
ST_PCA10040_A=682935919:COM3 -DBLE_DRIVER_TEST_PCA10040_B=682179628:COM6 -DT
EST_ALL=1 -DTEST_SOFTDEVICE_API=1 -DTEST_TRANSPORT=1 ..

-- NRF_BLE_DRIVER_VERSION not specified, setting it to 0.0.0
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Architecture not set, using native 64-bit toolchain.
-- Building with build type: .
-- Found Git: /usr/bin/git (found version "2.25.1")
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Using unsigned short
-- Check if the system is big endian - little endian
-- SDK directory already exists, reusing that.
-- Found projects: /tmp/pc-ble-driver/sdks/v11/examples/ble_central_and_peripheral/ble_connectivity/pca10028/ser_s130_hci
-- PROJECT_NAME:ser_s130_hci PCA_TYPE:pca10028 SD_TV: s130 TRANSPORT:uart
-- Newer SoftDevice version found (2.0.1) than the SDK provides (2.0.0).
-- Running make:/usr/bin/make in ARMGCC_PROJECT_DIRECTORY:/tmp/pc-ble-driver/sdks/v11/examples/ble_central_and_peripheral/ble_connectivity/pca10028/ser_s130_hci/armgcc
-- Storing files in /home/user/linux-source/pc-ble-driver/hex/sd_api_v2
-- Found projects: /tmp/pc-ble-driver/sdks/v11/examples/ble_central_and_peripheral/ble_connectivity/pca10036/ser_s132_hci
-- PROJECT_NAME:ser_s132_hci PCA_TYPE:pca10036 SD_TV: s132 TRANSPORT:uart
-- Newer SoftDevice version found (2.0.1) than the SDK provides (2.0.0).
-- Running make:/usr/bin/make in ARMGCC_PROJECT_DIRECTORY:/tmp/pc-ble-driver/sdks/v11/examples/ble_central_and_peripheral/ble_connectivity/pca10036/ser_s132_hci/armgcc
-- Storing files in /home/user/linux-source/pc-ble-driver/hex/sd_api_v2
-- Found projects: /tmp/pc-ble-driver/sdks/v11/examples/ble_central_and_peripheral/ble_connectivity/pca10040/ser_s132_hci
-- PROJECT_NAME:ser_s132_hci PCA_TYPE:pca10040 SD_TV: s132 TRANSPORT:uart
-- Newer SoftDevice version found (2.0.1) than the SDK provides (2.0.0).
-- Target connectivity_1.0.0_1m_with_s132_2.0.1 already exists.
-- SDK directory already exists, reusing that.
-- Found projects: /tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10040/ser_s132_hci;/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10040/ser_s132v3_hci;/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10040/ser_s132v5_hci
-- PROJECT_NAME:ser_s132_hci PCA_TYPE:pca10040 SD_TV: s132 TRANSPORT:uart
-- SoftDevice version found is the same version(/home/user/linux-source/pc-ble-driver/hex/sd_api_v6/s132_nrf52_6.1.1_softdevice.hex) as the one in the SDK(6.1.1). Using the one found in the SDK.
-- Running make:/usr/bin/make in ARMGCC_PROJECT_DIRECTORY:/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10040/ser_s132_hci/armgcc
-- Storing files in /home/user/linux-source/pc-ble-driver/hex/sd_api_v6
-- PROJECT_NAME:ser_s132v3_hci PCA_TYPE:pca10040 SD_TV: s132v3 TRANSPORT:uart
-- Newer SoftDevice version found (3.1.0) than the SDK provides (3.0.0).
-- Running make:/usr/bin/make in ARMGCC_PROJECT_DIRECTORY:/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10040/ser_s132v3_hci/armgcc
-- Storing files in /home/user/linux-source/pc-ble-driver/hex/sd_api_v3
-- PROJECT_NAME:ser_s132v5_hci PCA_TYPE:pca10040 SD_TV: s132v5 TRANSPORT:uart
-- SoftDevice version found is the same version(/home/user/linux-source/pc-ble-driver/hex/sd_api_v5/s132_nrf52_5.1.0_softdevice.hex) as the one in the SDK(5.1.0). Using the one found in the SDK.
-- Running make:/usr/bin/make in ARMGCC_PROJECT_DIRECTORY:/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10040/ser_s132v5_hci/armgcc
-- Storing files in /home/user/linux-source/pc-ble-driver/hex/sd_api_v5
-- Found projects: /tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10056/ser_s132v3_usb_hci;/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10056/ser_s132v5_usb_hci;/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10056/ser_s140_hci;/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10056/ser_s140_usb_hci
-- PROJECT_NAME:ser_s132v3_usb_hci PCA_TYPE:pca10056 SD_TV: s132v3 TRANSPORT:usb
-- Newer SoftDevice version found (3.1.0) than the SDK provides (3.0.0).
-- Running make:/usr/bin/make in ARMGCC_PROJECT_DIRECTORY:/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10056/ser_s132v3_usb_hci/armgcc
-- Storing files in /home/user/linux-source/pc-ble-driver/hex/sd_api_v3
-- PROJECT_NAME:ser_s132v5_usb_hci PCA_TYPE:pca10056 SD_TV: s132v5 TRANSPORT:usb
-- SoftDevice version found is the same version(/home/user/linux-source/pc-ble-driver/hex/sd_api_v5/s132_nrf52_5.1.0_softdevice.hex) as the one in the SDK(5.1.0). Using the one found in the SDK.
-- Running make:/usr/bin/make in ARMGCC_PROJECT_DIRECTORY:/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10056/ser_s132v5_usb_hci/armgcc
-- Storing files in /home/user/linux-source/pc-ble-driver/hex/sd_api_v5
-- PROJECT_NAME:ser_s140_hci PCA_TYPE:pca10056 SD_TV: s140 TRANSPORT:uart
-- SoftDevice version found is the same version(/home/user/linux-source/pc-ble-driver/hex/sd_api_v6/s140_nrf52_6.1.1_softdevice.hex) as the one in the SDK(6.1.1). Using the one found in the SDK.
-- Running make:/usr/bin/make in ARMGCC_PROJECT_DIRECTORY:/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10056/ser_s140_hci/armgcc
-- Storing files in /home/user/linux-source/pc-ble-driver/hex/sd_api_v6
-- PROJECT_NAME:ser_s140_usb_hci PCA_TYPE:pca10056 SD_TV: s140 TRANSPORT:usb
-- SoftDevice version found is the same version(/home/user/linux-source/pc-ble-driver/hex/sd_api_v6/s140_nrf52_6.1.1_softdevice.hex) as the one in the SDK(6.1.1). Using the one found in the SDK.
-- Running make:/usr/bin/make in ARMGCC_PROJECT_DIRECTORY:/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10056/ser_s140_usb_hci/armgcc
-- Storing files in /home/user/linux-source/pc-ble-driver/hex/sd_api_v6
-- Found projects: /tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10059/ser_s132v3_usb_hci;/tmp/pc-ble-driver/sdks/v15/nRF5_SDK_15.3.0_59ac345/examples/connectivity/ble_connectivity/pca10059/ser_s140_usb_hci
-- PROJECT_NAME:ser_s132v3_usb_hci PCA_TYPE:pca10059 SD_TV: s132v3 TRANSPORT:usb
-- Newer SoftDevice version found (3.1.0) than the SDK provides (3.0.0).
-- Target connectivity_1.0.0_usb_with_s132_3.1.0 already exists.
-- PROJECT_NAME:ser_s140_usb_hci PCA_TYPE:pca10059 SD_TV: s140 TRANSPORT:usb
-- SoftDevice version found is the same version(/home/user/linux-source/pc-ble-driver/hex/sd_api_v6/s140_nrf52_6.1.1_softdevice.hex) as the one in the SDK(6.1.1). Using the one found in the SDK.
-- Target connectivity_1.0.0_usb_with_s140_6.1.1 already exists.
-- Connectivity artifacts _CONNECTIVITY_ARTIFACTS_SDv2;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v2/connectivity_1.0.0_1m_with_s130_2.0.1.hex;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v2/connectivity_1.0.0_1m_with_s132_2.0.1.hex;_CONNECTIVITY_ARTIFACTS_SDv6;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v6/connectivity_1.0.0_1m_with_s132_6.1.1.hex;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v6/connectivity_1.0.0_1m_with_s140_6.1.1.hex;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v6/connectivity_1.0.0_usb_with_s140_6.1.1.hex;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v6/connectivity_1.0.0_usb_with_s140_6.1.1_dfu_pkg.zip;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v6/connectivity_1.0.0_usb_for_s140_6.1.1.hex;_CONNECTIVITY_ARTIFACTS_SDv3;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v3/connectivity_1.0.0_1m_with_s132_3.1.0.hex;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v3/connectivity_1.0.0_usb_with_s132_3.1.0.hex;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v3/connectivity_1.0.0_usb_with_s132_3.1.0_dfu_pkg.zip;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v3/connectivity_1.0.0_usb_for_s132_3.1.0.hex;_CONNECTIVITY_ARTIFACTS_SDv5;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v5/connectivity_1.0.0_1m_with_s132_5.1.0.hex;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v5/connectivity_1.0.0_usb_with_s132_5.1.0.hex;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v5/connectivity_1.0.0_usb_with_s132_5.1.0_dfu_pkg.zip;|/home/user/linux-source/pc-ble-driver/hex/sd_api_v5/connectivity_1.0.0_usb_for_s132_5.1.0.hex
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Compiled tests are installed in directory "bin"
-- Compiled examples are installed in directory "bin"
-- Did not find clang-tidy, target tidy is disabled.
-- If clang-tidy is installed, make sure run-clang-tidy.py and clang-tidy is in PATH
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:

BLE_DRIVER_TEST_PCA10040_A
BLE_DRIVER_TEST_PCA10040_B
TEST_ALL

-- Build files have been written to: /home/user/linux-source/pc-ble-driver/build

The interesting thing is that BLE_DRIVER_TEST_PCA10040_A,BLE_DRIVER_TEST_PCA10040_B,TEST_ALL seem to do nothing (yes I changed boards) and I didn't see any use of them in any of the cmakefiles.

At this point I am unsure what compiles the test targets "test_run_sdv<SoftDevice_API_version>_<SoftDevice_type>_pca" that the readme in the tests folder hints at.

pc-ble-driver$ tree | grep test

│   └── test
│   │   ├── test_sd_api_v2.dir
│   │   │   │   ├── test.cpp.o
│   │   │   │   ├── testcase_advertising.cpp.o
│   │   │   │   ├── testcase_driver_open_close.cpp.o
│   │   │   │   ├── testcase_issue_128bit_uuid.cpp.o
│   │   │   │   ├── testcase_issue_gh_112.cpp.o
│   │   │   │   ├── testcase_issue_stuck_in_scan_mode.cpp.o
│   │   │   │   ├── testcase_phy_update.cpp.o
│   │   │   │   ├── testcase_rssi.cpp.o
│   │   │   │   ├── testcase_security.cpp.o
│   │   │   │   └── testcase_util.cpp.o
│   │   │   ├── test_environment.cpp.o
│   │   │   ├── test_util.cpp.o
│   │   │   ├── test_util_adapter_wrapper.cpp.o
│   │   │   ├── test_util_conversion.cpp.o
│   │   │   └── test_util_role.cpp.o
│   │   ├── test_sd_api_v3.dir
│   │   │   │   ├── test.cpp.o
│   │   │   │   ├── testcase_advertising.cpp.o
│   │   │   │   ├── testcase_driver_open_close.cpp.o
│   │   │   │   ├── testcase_issue_128bit_uuid.cpp.o
│   │   │   │   ├── testcase_issue_gh_112.cpp.o
│   │   │   │   ├── testcase_issue_stuck_in_scan_mode.cpp.o
│   │   │   │   ├── testcase_phy_update.cpp.o
│   │   │   │   ├── testcase_rssi.cpp.o
│   │   │   │   ├── testcase_security.cpp.o
│   │   │   │   └── testcase_util.cpp.o
│   │   │   ├── test_environment.cpp.o
│   │   │   ├── test_util.cpp.o
│   │   │   ├── test_util_adapter_wrapper.cpp.o
│   │   │   ├── test_util_conversion.cpp.o
│   │   │   └── test_util_role.cpp.o
│   │   ├── test_sd_api_v5.dir
│   │   │   │   ├── test.cpp.o
│   │   │   │   ├── testcase_advertising.cpp.o
│   │   │   │   ├── testcase_driver_open_close.cpp.o
│   │   │   │   ├── testcase_issue_128bit_uuid.cpp.o
│   │   │   │   ├── testcase_issue_gh_112.cpp.o
│   │   │   │   ├── testcase_issue_stuck_in_scan_mode.cpp.o
│   │   │   │   ├── testcase_phy_update.cpp.o
│   │   │   │   ├── testcase_rssi.cpp.o
│   │   │   │   ├── testcase_security.cpp.o
│   │   │   │   └── testcase_util.cpp.o
│   │   │   ├── test_environment.cpp.o
│   │   │   ├── test_util.cpp.o
│   │   │   ├── test_util_adapter_wrapper.cpp.o
│   │   │   ├── test_util_conversion.cpp.o
│   │   │   └── test_util_role.cpp.o
│   │   ├── test_sd_api_v6.dir
│   │   │   │   ├── test.cpp.o
│   │   │   │   ├── testcase_advertising.cpp.o
│   │   │   │   ├── testcase_driver_open_close.cpp.o
│   │   │   │   ├── testcase_issue_128bit_uuid.cpp.o
│   │   │   │   ├── testcase_issue_gh_112.cpp.o
│   │   │   │   ├── testcase_issue_stuck_in_scan_mode.cpp.o
│   │   │   │   ├── testcase_phy_update.cpp.o
│   │   │   │   ├── testcase_rssi.cpp.o
│   │   │   │   ├── testcase_security.cpp.o
│   │   │   │   └── testcase_util.cpp.o
│   │   │   ├── test_environment.cpp.o
│   │   │   ├── test_util.cpp.o
│   │   │   ├── test_util_adapter_wrapper.cpp.o
│   │   │   ├── test_util_conversion.cpp.o
│   │   │   └── test_util_role.cpp.o
│   │   └── test_serial_port_enum_sd_api_v2.dir
│   │   │   └── test_serial_port_enum.cpp.o
│   │   ├── test_environment.cpp.o
│   │   ├── test_util.cpp.o
│   │   ├── test_util_adapter_wrapper.cpp.o
│   │   ├── test_util_conversion.cpp.o
│   │   └── test_util_role.cpp.o
│   ├── test_sd_api_v2
│   ├── test_sd_api_v3
│   ├── test_sd_api_v5
│   ├── test_sd_api_v6
│   └── test_serial_port_enum_sd_api_v2
├── test
│   │   ├── test.cpp
│   │   ├── testcase_advertising.cpp
│   │   ├── testcase_driver_open_close.cpp
│   │   ├── testcase_issue_128bit_uuid.cpp
│   │   ├── testcase_issue_gh_112.cpp
│   │   ├── testcase_issue_stuck_in_scan_mode.cpp
│   │   ├── testcase_phy_update.cpp
│   │   ├── testcase_rssi.cpp
│   │   ├── testcase_security.cpp
│   │   └── testcase_util.cpp
│   │   └── test_serial_port_enum.cpp
│   │   ├── test_environment.h
│   │   ├── test_util.h
│   │   ├── test_util_adapter_wrapper.h
│   │   ├── test_util_adapter_wrapper_scratchpad.h
│   │   ├── test_util_conversion.h
│   │   ├── test_util_role.h
│   ├── test_environment.cpp
│   ├── test_util.cpp
│   ├── test_util_adapter_wrapper.cpp
│   ├── test_util_conversion.cpp
│   └── test_util_role.cpp

 checking the available targets there seem to be no hint of the mentioned tests

cmake --build . --target help | grep test

test/edit_cache: phony
test/package_source: phony
test/install: phony
test/list_install_components: phony
test/rebuild_cache: phony
test/package: phony
test/install/strip: phony
test/install/local: phony
test_sd_api_v2: phony
test_sd_api_v3: phony
test_sd_api_v5: phony
test_sd_api_v6: phony
test_serial_port_enum_sd_api_v2: phony

[Einar Thorsrud]

Hi,

OK, now I think I see. The documentation is not up to date, and TEST_ALL does not do anything (you can search for TEST_ALL in the repository to see it is not used). You need to specify TEST_TRANSPORT and TEST_SOFTDEVICE_API like you did initially. That is all that is used, which you can see from test\CMakeLists.txt. Those are the only two test categories that are there, and all test cases should be built with those two.