How does one run zephyr/tests/drivers/spi/spi_controller_peripheral on two nRF54LM20-DKs?

[tylerh]

Hi,

As the subject states.

I would like to run the tests at zephyr/tests/drivers/spi/spi_controller_peripheral as intended in testcase.yml. I can verify that I have the correct pins setup, as seen in the SPI traces below.

I've tried using the following:

$> ./scripts/twister --generate-hardware-map map.yml #Creates a map.yml with the two nRF54LM20-DKs currently connected
$> ./scripts/twister --device-testing --hardware-map map.yml -T tests/drivers/spi/spi_controller_peripheral/

But this doesn't run, or flash anything. It appears to skip all tests instead.

I can use the following on both boards to run one of the tests (using the 8MHz overlay), and I can see some good output on the SPI lines with a logic analyzer

$> west build -b nrf54lm20dk/nrf54lm20a/cpuapp . -T drivers.spi.spi_8MHz
$> west flash

But I can't see any test results from ztest. Furthermore, it seems to only run test_basic(), which perhaps fails? Because it only does this once.

Is it possible to run all tests on these two boards? What's the command to do that?

[Susheel Nuguru]

I do not think this test is meant to run on two different boards. They seems to be designed to run on the same core with SPIM and SPIS from the same device. The design expects a loopback. What you got when you run west build -b nrf54lm20dk/nrf54lm20a/cpuapp . -T drivers.spi.spi_8MHz might just be lucky as that particular test might just be pushing our data from the master. That said, I am no expert in this setup, so we need to see if there is someone else with better knowledge in twister and ztests can comment here.

[tylerh]

Hi Susheel,

It looks like you're right. This is intended to run as loop-back mode. I'm able to get further by running it on the same board, as opposed to two different boards.

It looks like all of the tests are failing, however. Here's a snippet that I got from J-Link RTT Viewer after building this firmware:

$> west build -p -b nrf54lm20dk/nrf54lm20a/cpuapp . -T drivers.spi.spi_mode0

00> *** Booting nRF Connect SDK v3.2.1-d8887f6f32df ***
00> *** Using Zephyr OS v4.2.99-f3f1c3dacc9c ***
00> Running TESTSUITE spi_controller_peripheral
00> ===================================================================
00> START - test_basic
00> 
00>     Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/spi/spi_controller_peripheral/src/main.c:247: run_test: (periph_rv not equal to srx_len)
00> Got: 0 but expected:16
00>  FAIL - test_basic in 0.011 seconds
00> ===================================================================
00> START - test_basic_async
00> 
00>     Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/spi/spi_controller_peripheral/src/main.c:247: run_test: (periph_rv not equal to srx_len)
00> Got: 0 but expected:16
00>  FAIL - test_basic_async in 0.011 seconds
00> ===================================================================
00> START - test_basic_zero_len
00> 
00>     Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/spi/spi_controller_peripheral/src/main.c:247: run_test: (periph_rv not equal to srx_len)
00> Got: 0 but expected:8
00>     Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/spi/spi_controller_peripheral/src/main.c:247: run_test: (periph_rv not equal to srx_len)
00> Got: 0 but expected:8
00>  FAIL - test_basic_zero_len_async in 0.011 seconds
00> ===================================================================
00> START - test_only_rx
00> 
00>     Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/spi/spi_controller_peripheral/src/main.c:253: run_test: rv not equal to 0
00> 
00>  FAIL - test_only_rx in 0.011 seconds
00> ===================================================================
00> START - test_only_rx_async
00> 
00>     Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/spi/spi_controller_peripheral/src/main.c:253: run_test: rv not equal to 0
00> 
00>  FAIL - test_only_rx_async in 0.011 seconds
00> ===================================================================
00> START - test_only_rx_in_chunks
00> [00:00:06.070,176] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_rx_in_chunks in 0.001 seconds
00> ===================================================================
00> START - test_only_rx_in_chunks_async
00> [00:00:07.070,997] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_rx_in_chunks_async in 0.001 seconds
00> ===================================================================
00> START - test_only_tx
00> 
00>     Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/spi/spi_controller_peripheral/src/main.c:247: run_test: (periph_rv not equal to srx_len)
00> Got: 0 but expected:16
00>  FAIL - test_only_tx in 0.011 seconds
00> ===================================================================
00> START - test_only_tx_async
00> 
00>     Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/spi/spi_controller_peripheral/src/main.c:247: run_test: (periph_rv not equal to srx_len)
00> Got: 0 but expected:16
00>  FAIL - test_only_tx_async in 0.011 seconds
00> ===================================================================
00> START - test_only_tx_in_chunks
00> [00:00:10.094,888] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_tx_in_chunks in 0.001 seconds
00> ===================================================================
00> START - test_only_tx_in_chunks_async
00> [00:00:11.095,709] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_tx_in_chunks_async in 0.001 seconds
00> ===================================================================
00> START - test_short_rx
00> 
00>     Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/spi/spi_controller_peripheral/src/main.c:247: run_test: (periph_rv not equal to srx_len)
00> Got: 0 but expected:12
00>  FAIL - test_short_rx in 0.011 seconds
00> ===================================================================
00> START - test_short_rx_async
00> 
00>     Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/spi/spi_controller_peripheral/src/main.c:247: run_test: (periph_rv not equal to srx_len)
00> Got: 0 but expected:12
00>  FAIL - test_short_rx_async in 0.011 seconds
00> ===================================================================
00> TESTSUITE spi_controller_peripheral failed.
00> 
00> ------ TESTSUITE SUMMARY START ------
00> 
00> SUITE FAIL -   0.00% [spi_controller_peripheral]: pass = 0, fail = 10, skip = 4, total = 14 duration = 0.114 seconds
00>  - FAIL - [spi_controller_peripheral.test_basic] duration = 0.011 seconds
00>  - FAIL - [spi_controller_peripheral.test_basic_async] duration = 0.011 seconds
00>  - FAIL - [spi_controller_peripheral.test_basic_zero_len] duration = 0.011 seconds
00>  - FAIL - [spi_controller_peripheral.test_basic_zero_len_async] duration = 0.011 seconds
00>  - FAIL - [spi_controller_peripheral.test_only_rx] duration = 0.011 seconds
00>  - FAIL - [spi_controller_peripheral.test_only_rx_async] duration = 0.011 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_rx_in_chunks] duration = 0.001 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_rx_in_chunks_async] duration = 0.001 seconds
00>  - FAIL - [spi_controller_peripheral.test_only_tx] duration = 0.011 seconds
00>  - FAIL - [spi_controller_peripheral.test_only_tx_async] duration = 0.011 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_tx_in_chunks] duration = 0.001 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_tx_in_chunks_async] duration = 0.001 seconds
00>  - FAIL - [spi_controller_peripheral.test_short_rx] duration = 0.011 seconds
00>  - FAIL - [spi_controller_peripheral.test_short_rx_async] duration = 0.011 seconds
00> 
00> ------ TESTSUITE SUMMARY END ------
00> 
00> ===================================================================
00> PROJECT EXECUTION FAILED

I'm building with these additions to prj.conf to see the log output.

CONFIG_SPI=y
CONFIG_SPI_SLAVE=y
CONFIG_GPIO=y
CONFIG_POLL=y
CONFIG_SPI_ASYNC=y
CONFIG_ZTEST=y

CONFIG_LOG=y
CONFIG_LOG_MODE_DEFERRED=y
CONFIG_LOG_BACKEND_RTT=y

CONFIG_USE_SEGGER_RTT=y

CONFIG_RTT_CONSOLE=y
CONFIG_CONSOLE=y
CONFIG_PRINTK=y

CONFIG_UART_CONSOLE=n

Do you know why this might be the case?

Additionally, to my question above, do you know if there's someone that might be able to describe how this could be done using Twister on this hardware, in an automated test environment?

Regards,

Tyler

[tylerh]

I did a few tests. It appears that this could be a problem with the nRF54LM20, the implementation of this test on that board, or both.

Running this command works successfully using an nRF54L15-DK, see the output here:

$> west build -p -b nrf54lm20dk/nrf54lm20a/cpuapp . -T drivers.spi.spi_mode0

00> *** Booting nRF Connect SDK v3.2.1-d8887f6f32df ***
00> *** Using Zephyr OS v4.2.99-f3f1c3dacc9c ***
00> Running TESTSUITE spi_controller_peripheral
00> ===================================================================
00> START - test_basic
00>  PASS - test_basic in 0.011 seconds
00> ===================================================================
00> START - test_basic_async
00>  PASS - test_basic_async in 0.011 seconds
00> ===================================================================
00> START - test_basic_zero_len
00>  PASS - test_basic_zero_len in 0.011 seconds
00> ===================================================================
00> START - test_basic_zero_len_async
00>  PASS - test_basic_zero_len_async in 0.011 seconds
00> ===================================================================
00> START - test_only_rx
00>  PASS - test_only_rx in 0.011 seconds
00> ===================================================================
00> START - test_only_rx_async
00>  PASS - test_only_rx_async in 0.011 seconds
00> ===================================================================
00> START - test_only_rx_in_chunks
00> [00:00:05.066,559] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_rx_in_chunks in 0.001 seconds
00> ===================================================================
00> START - test_only_rx_in_chunks_async
00> [00:00:06.067,370] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_rx_in_chunks_async in 0.001 seconds
00> ===================================================================
00> START - test_only_tx
00>  PASS - test_only_tx in 0.011 seconds
00> ===================================================================
00> START - test_only_tx_async
00>  PASS - test_only_tx_async in 0.011 seconds
00> ===================================================================
00> START - test_only_tx_in_chunks
00> [00:00:09.090,196] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_tx_in_chunks in 0.001 seconds
00> ===================================================================
00> START - test_only_tx_in_chunks_async
00> [00:00:10.091,007] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_tx_in_chunks_async in 0.001 seconds
00> ===================================================================
00> START - test_short_rx
00>  PASS - test_short_rx in 0.011 seconds
00> ===================================================================
00> START - test_short_rx_async
00>  PASS - test_short_rx_async in 0.011 seconds
00> ===================================================================
00> TESTSUITE spi_controller_peripheral succeeded
00> 
00> ------ TESTSUITE SUMMARY START ------
00> 
00> SUITE PASS - 100.00% [spi_controller_peripheral]: pass = 10, fail = 0, skip = 4, total = 14 duration = 0.114 seconds
00>  - PASS - [spi_controller_peripheral.test_basic] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_basic_async] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_basic_zero_len] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_basic_zero_len_async] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_only_rx] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_only_rx_async] duration = 0.011 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_rx_in_chunks] duration = 0.001 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_rx_in_chunks_async] duration = 0.001 seconds
00>  - PASS - [spi_controller_peripheral.test_only_tx] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_only_tx_async] duration = 0.011 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_tx_in_chunks] duration = 0.001 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_tx_in_chunks_async] duration = 0.001 seconds
00>  - PASS - [spi_controller_peripheral.test_short_rx] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_short_rx_async] duration = 0.011 seconds
00> 
00> ------ TESTSUITE SUMMARY END ------
00> 
00> ===================================================================
00> PROJECT EXECUTION SUCCESSFUL

[Susheel Nuguru]

I do not think anything is wrong with the chip but i think it is related to zephyr,pm-device-runtime-auto settings for the nRF54LM20.

The right side is for nRF54L15DK default overlay file

try adding this in your project overlay file and see if that helps?

dut_spis: &spi21 {
...
zephyr,pm-device-runtime-auto;
};

[tylerh]

Thanks a lot Susheel, that seems to have worked!

00> *** Booting nRF Connect SDK v3.2.1-d8887f6f32df ***
00> *** Using Zephyr OS v4.2.99-f3f1c3dacc9c ***
00> Running TESTSUITE spi_controller_peripheral
00> ===================================================================
00> START - test_basic
00>  PASS - test_basic in 0.011 seconds
00> ===================================================================
00> START - test_basic_async
00>  PASS - test_basic_async in 0.011 seconds
00> ===================================================================
00> START - test_basic_zero_len
00>  PASS - test_basic_zero_len in 0.011 seconds
00> ===================================================================
00> START - test_basic_zero_len_async
00>  PASS - test_basic_zero_len_async in 0.011 seconds
00> ===================================================================
00> START - test_only_rx
00>  PASS - test_only_rx in 0.011 seconds
00> ===================================================================
00> START - test_only_rx_async
00>  PASS - test_only_rx_async in 0.011 seconds
00> ===================================================================
00> START - test_only_rx_in_chunks
00> [00:00:05.067,707] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_rx_in_chunks in 0.001 seconds
00> ===================================================================
00> START - test_only_rx_in_chunks_async
00> [00:00:06.068,529] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_rx_in_chunks_async in 0.001 seconds
00> ===================================================================
00> START - test_only_tx
00>  PASS - test_only_tx in 0.011 seconds
00> ===================================================================
00> START - test_only_tx_async
00>  PASS - test_only_tx_async in 0.011 seconds
00> ===================================================================
00> START - test_only_tx_in_chunks
00> [00:00:09.091,344] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_tx_in_chunks in 0.001 seconds
00> ===================================================================
00> START - test_only_tx_in_chunks_async
00> [00:00:10.092,165] <err> spi_nrfx_spis: Scattered buffers are not supported
00>  SKIP - test_only_tx_in_chunks_async in 0.001 seconds
00> ===================================================================
00> START - test_short_rx
00>  PASS - test_short_rx in 0.011 seconds
00> ===================================================================
00> START - test_short_rx_async
00>  PASS - test_short_rx_async in 0.011 seconds
00> ===================================================================
00> TESTSUITE spi_controller_peripheral succeeded
00> 
00> ------ TESTSUITE SUMMARY START ------
00> 
00> SUITE PASS - 100.00% [spi_controller_peripheral]: pass = 10, fail = 0, skip = 4, total = 14 duration = 0.114 seconds
00>  - PASS - [spi_controller_peripheral.test_basic] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_basic_async] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_basic_zero_len] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_basic_zero_len_async] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_only_rx] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_only_rx_async] duration = 0.011 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_rx_in_chunks] duration = 0.001 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_rx_in_chunks_async] duration = 0.001 seconds
00>  - PASS - [spi_controller_peripheral.test_only_tx] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_only_tx_async] duration = 0.011 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_tx_in_chunks] duration = 0.001 seconds
00>  - SKIP - [spi_controller_peripheral.test_only_tx_in_chunks_async] duration = 0.001 seconds
00>  - PASS - [spi_controller_peripheral.test_short_rx] duration = 0.011 seconds
00>  - PASS - [spi_controller_peripheral.test_short_rx_async] duration = 0.011 seconds
00> 
00> ------ TESTSUITE SUMMARY END ------
00> 
00> ===================================================================
00> PROJECT EXECUTION SUCCESSFUL

[tylerh]

Hi Susheel,

I've added this fix to the Zephyr project, as this spi driver test sits in the upstream Zephyr project: https://github.com/zephyrproject-rtos/zephyr/pull/103524

If there's anyone from Nordic that can approve it, then it will be fixed in the upcoming nRF Connect SDK.

Regards,

Tyler

[Susheel Nuguru]

Informed internally. Hopefully some maintainers from Nordic will look at that and approve soon.