Running tests with twister produces error for custom boards

Greetings,

I have set up some tests using the ZTest environment based on the test sample in ..\v3.0.2\zephyr\tests\bluetooth\adv\. for my custom project which has custom boards that has been deployed on this HW and is working as expected for 2 years. The custom boards were created using the "Create a new board" feature of the nRF Connect VS Code Extension (from v2.5.2 version of the SDK). I have also followed the file and project configuration of the sample test in my custom application and also looked up all relevant documentation and asked tha same question to the Nordic AI for help.

When running the tests with either twister command or the west command as prompted by the documentation the sample test code runs as expected but the integration test setup (ble initiliazation as performed in the sample bluetooth adv test) for my custom project fails with the following error:

Working directory: c:\ncs\feel_v2_fw\feel_v2_fw
SDK:               nRF Connect SDK v3.0.2
Toolchain:         nRF Connect SDK Toolchain v3.0.1

PS C:\ncs\feel_v2_fw\feel_v2_fw> python C:\ncs\v3.0.2\zephyr\scripts\twister -T . -p feel_v2_prod         
INFO    - Using Ninja..
INFO    - Zephyr version: v4.0.99-ncs1-2
INFO    - Using 'zephyr' toolchain.
ERROR   - integration_platforms in tests/integration/protocols/ble/sample.testing.ztest - unrecognized platform - feel_v2_pre_evt_v1_0_0
PS C:\ncs\feel_v2_fw\feel_v2_fw> cd tests/integration/protocols/ble
PS C:\ncs\feel_v2_fw\feel_v2_fw\tests\integration\protocols\ble> python C:\ncs\v3.0.2\zephyr\scripts\twister -T . -p feel_v2_prod
INFO    - Using Ninja..
INFO    - Zephyr version: v4.0.99-ncs1-2
INFO    - Using 'zephyr' toolchain.
ERROR   - integration_platforms in sample.testing.ztest - unrecognized platform - feel_v2_pre_evt_v1_0_0
PS C:\ncs\feel_v2_fw\feel_v2_fw\tests\integration\protocols\ble> python C:\ncs\v3.0.2\zephyr\scripts\twister -T . -p feel_v2_prod
Renaming output directory to C:\ncs\feel_v2_fw\feel_v2_fw\tests\integration\protocols\ble\twister-out.1
INFO    - Using Ninja..
INFO    - Zephyr version: v4.0.99-ncs1-2
INFO    - Using 'zephyr' toolchain.
ERROR   - integration_platforms in sample.testing.ztest - unrecognized platform - feel_v2_pre_evt_v1_0_0

where feel_v2_pre_evt_v1_0_0 & feel_v2_prod are two of the custom HW boards that my application is used on. Even if ommit specifiyng the board in the twister command I get the same results.

Below are the files included in the tests directory of my custom application for your review:

main.c:

////////////////////////////////////////////////////////////////////////////////
// Includes.
//

#include <zephyr/ztest.h>
#include "ble.h"

////////////////////////////////////////////////////////////////////////////////
// Function definitions.
//

ZTEST_SUITE(ble_tests, NULL, NULL, NULL, NULL, NULL);

ZTEST(ble_tests, test_ble_init, NULL)
{
    zassert_true(ble_init(BLE_CONNECTED), NULL);
}

CMakeLists.txt:

cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(feel_v2_fw_tests)

# Add test sources
FILE(GLOB app_sources src/*.c)
target_sources(app PRIVATE ${app_sources})

# Add include directories
target_include_directories(app PRIVATE ../../include)

prj.conf:

CONFIG_ZTEST=y

testcase.yml:

tests:
  # section.subsection
  sample.testing.ztest:
    build_only: true
    platform_allow:
      - feel_v2_pre_evt_v1_0_0
      - feel_v2_ppg_v1_eda_v2
      - feel_v2_prod
    tags: feel_v2_fw_tests
    slow: true
    timeout: 360
    integration_platforms:
      - feel_v2_pre_evt_v1_0_0
      - feel_v2_ppg_v1_eda_v2
      - feel_v2_prod

Could you provide some help on why this error is being produced? Do I have to setup my custom boards differently o be recognized by the ZTest framework or twister?

Thank you in advance!

Best,
Stavros

Parents
  • Hi Stavros,

    Do you use out-of-tree board?

    Have you tried to migrate your board to hardware model 2 as shown in board migration?

    Best regards,
    Dejan 

  • I believe I already have them ported in hardware model v2, because after I migrated to the v3.0.2 version of the SDK, I used the "Create a new board" tool of the nRF Connect VS Code extension to recreate them, and I have also tried running the conversion script, and after it executes successfully, no files have been changed. So they must have the correct format (HW model 2).

  • Thanks for the prompt response, unfortunately, they are not listed in the command output. Does this mean they are not properly defined? (I have been using them normally for building the application though)

  • Hi,

    clockis said:
    unfortunately, they are not listed in the command output.

    What do you refer to as "they" in this context?
    Can you show the output of the "west boards" command?

    Best regards,
    Dejan

  • Please excuse the confusion. "They" refers to our custom boards.
    They have been created using the "Create a new board" tool of the nRF Connect VS Code extensionand have been extensively used for 2 years now to build and flash our custom application on our custom boards with success. 
    After entering the command, they do not show up on the list of the command output:

    PS C:\ncs\feel_v2_fw\feel_v2_fw> west boards
    esp32c3_042_oled
    96b_aerocore2
    96b_argonkey
    96b_avenger96
    96b_carbon
    96b_meerkat96
    96b_neonkey
    96b_nitrogen
    96b_stm32_sensor_mez
    96b_wistrio
    acn52832
    acrn
    acrn_ehl_crb
    actinius_icarus
    actinius_icarus_bee
    actinius_icarus_som
    actinius_icarus_som_dk
    adafruit_feather_m0_basic_proto
    adafruit_feather_m0_lora
    adafruit_feather_m4_express
    adafruit_feather_nrf52840
    adafruit_feather_stm32f405
    adafruit_grand_central_m4_express
    adafruit_itsybitsy
    adafruit_itsybitsy_m4_express
    adafruit_kb2040
    nrf52_adafruit_feather
    adafruit_qt_py_esp32s3
    adafruit_qt_py_rp2040
    adafruit_trinket_m0
    apard32690
    adi_eval_adin1110ebz
    adi_eval_adin2111ebz
    max32655evkit
    max32655fthr
    max32662evkit
    max32666evkit
    max32666fthr
    max32670evkit
    max32672evkit
    max32672fthr
    max32675evkit
    max32680evkit
    max32690evkit
    max32690fthr
    max78002evkit
    adi_sdp_k1
    pandora_stm32l475
    altera_max10
    apollo3p_evb
    apollo3_evb
    apollo4p_blue_kxr_evb
    apollo4p_evb
    acp_6_0_adsp
    kv260_r5
    adp_xc7k
    arduino_due
    arduino_giga_r1
    arduino_mkrzero
    arduino_nano_33_ble
    arduino_nano_33_iot
    arduino_nicla_sense_me
    arduino_nicla_vision
    arduino_opta
    arduino_portenta_h7
    arduino_uno_r4_minima
    arduino_uno_r4_wifi
    arduino_zero
    fvp_baser_aemv8r
    fvp_base_revc_2xaemv8a
    mps2
    mps3
    v2m_beetle
    v2m_musca_b1
    v2m_musca_s1
    ast1030_evb
    degu_evk
    sam4e_xpro
    sam4l_ek
    sam4s_xplained
    sam_e70_xplained
    sam_v71_xult
    samc21n_xpro
    samd20_xpro
    samd21_xpro
    same54_xpro
    saml21_xpro
    samr21_xpro
    samr34_xpro
    bbc_microbit
    bbc_microbit_v2
    blueclover_plt_demo_v2
    beaglebone_ai64
    beagleconnect_freedom
    beagleplay
    beaglev_fire
    swan_r5
    bcm958401m2
    bcm958402m2
    bytesensi_l
    xt-sim
    circuitdojo_feather
    contextualelectronics_abc
    croxel_cx1825
    ctcc
    cy8ckit_062_ble
    cy8ckit_062_wifi_bt
    arty_a7
    zybo
    walter
    dragino_lsn50
    dragino_nbsn95
    ebyte_e73_tbb
    titanium_ti60_f225
    nrf52840_blip
    nrf52840_papyr
    warp7
    mercury_xu
    kb1200_evb
    litex_vexriscv
    esp32c3_devkitc
    esp32c3_devkitm
    esp32c3_rust
    esp32c6_devkitc
    esp32s2_devkitc
    esp32s2_saola
    esp32s3_devkitc
    esp32s3_devkitm
    esp32s3_eye
    esp32_devkitc_wroom
    esp32_devkitc_wrover
    esp32_ethernet_kit
    esp8684_devkitm
    esp_wrover_kit
    bl5340_dvk
    bl652_dvk
    bl653_dvk
    bl654_dvk
    bl654_sensor_board
    bl654_usb
    bt510
    bt610
    mg100
    pinnacle_100_dvk
    rm1xx_dvk
    fk750m1_vbt6
    fk7b0m1_vbt6
    roc_rk3568_pc
    esp32s2_franzininho
    ucan
    generic_leon3
    gr716a_mini
    sgrm
    gd32a503v_eval
    gd32e103v_eval
    gd32e507v_start
    gd32e507z_eval
    gd32f350r_eval
    gd32f403z_eval
    gd32f407v_start
    gd32f450i_eval
    gd32f450v_start
    gd32f450z_eval
    gd32f470i_eval
    gd32l233r_eval
    gd32vf103c_starter
    gd32vf103v_eval
    google_dragonclaw
    google_icetower
    google_twinkie_v2
    odroid_go
    heltec_wifi_lora32_v2
    heltec_wireless_stick_lite_v3
    holyiot_yj16019
    cy8ckit_062s2_ai
    cy8ckit_062s4
    cy8cproto_062_4343w
    cy8cproto_063_ble
    cyw920829m2evk_02
    xmc45_relax_kit
    xmc47_relax_kit
    innblue21
    innblue22
    intel_adl_crb
    intel_adl_rvp
    intel_adsp
    intel_ehl_crb
    intel_ish_5_4_1
    intel_ish_5_6_0
    intel_ish_5_8_0
    niosv_g
    niosv_m
    intel_rpl_p_crb
    intel_rpl_s_crb
    intel_socfpga_agilex5_socdk
    intel_socfpga_agilex_socdk
    cyclonev_socdk
    it82xx2_evb
    it8xxx2_evb
    khadas_edgev
    kincony_kc868_a32
    ttgo_lora32
    ttgo_t7v1_5
    ttgo_t8c3
    ttgo_t8s3
    opentitan_earlgrey
    esp32c3_luatos_core
    esp32s3_luatos_core
    m5stack_atoms3
    m5stack_atoms3_lite
    m5stack_atom_lite
    m5stack_core2
    m5stack_cores3
    m5stack_stamps3
    m5stickc_plus
    stamp_c3
    mm_feather
    mm_swiftio
    mks_canable_v20
    nrf52832_mdk
    nrf52840_mdk
    nrf52840_mdk_usb_dongle
    mt8186
    mt8188
    mt8195
    mt8196
    ev11l78a
    m2gl025_miv
    mec1501modular_assy6885
    mec15xxevb_assy6853
    mec172xevb_assy6906
    mec172xmodular_assy6930
    mpfs_icicle
    mikroe_clicker_2
    mikroe_clicker_ra4m1
    mikroe_mini_m4_for_stm32
    mikroe_stm32_m4_clicker
    az3166_iotdevkit
    native_posix
    native_sim
    nrf52_bsim
    nrf5340bsim
    nrf54l15bsim
    nrf21540dk
    nrf51dk
    nrf51dongle
    nrf52833dk
    nrf52840dk
    nrf52840dongle
    nrf52dk
    nrf5340dk
    nrf5340_audio_dk
    nrf54h20dk
    nrf54l09pdk
    nrf54l15dk
    nrf54l20pdk
    nrf7002dk
    nrf9131ek
    nrf9151dk
    nrf9160dk
    nrf9161dk
    nrf9280pdk
    thingy52
    thingy53
    octopus_io_board
    octopus_som
    npcm400_evb
    npcx4m8f_evb
    npcx7m6fb_evb
    npcx9m6f_evb
    numaker_m2l31ki
    numaker_pfm_m467
    numaker_pfm_m487
    frdm_k22f
    frdm_k64f
    frdm_k82f
    frdm_ke15z
    frdm_ke17z
    frdm_ke17z512
    frdm_kl25z
    frdm_kw41z
    frdm_mcxa156
    frdm_mcxc242
    frdm_mcxc444
    frdm_mcxn236
    frdm_mcxn947
    frdm_mcxw71
    frdm_rw612
    hexiwear
    imx8mm_evk
    imx8mn_evk
    imx8mp_evk
    imx8mq_evk
    imx8qm_mek
    imx8qxp_mek
    imx8ulp_evk
    imx93_evk
    imx95_evk
    lpcxpresso11u68
    lpcxpresso51u68
    lpcxpresso54114
    lpcxpresso55s06
    lpcxpresso55s16
    lpcxpresso55s28
    lpcxpresso55s36
    lpcxpresso55s69
    ls1046ardb
    mimxrt1010_evk
    mimxrt1015_evk
    mimxrt1020_evk
    mimxrt1024_evk
    mimxrt1040_evk
    mimxrt1050_evk
    mimxrt1060_evk
    mimxrt1062_fmurt6
    mimxrt1064_evk
    mimxrt1160_evk
    mimxrt1170_evk
    mimxrt1180_evk
    mimxrt595_evk
    mimxrt685_evk
    mr_canhubk3
    rddrone_fmuk66
    rd_rw612_bga
    s32z2xxdc2
    twr_ke18f
    twr_kv58f220m
    ucans32k1sic
    usb_kw24d512
    vmu_rt1170
    olimex_lora_stm32wl_devkit
    olimexino_stm32
    olimex_esp32_evb
    olimex_stm32_e407
    olimex_stm32_h103
    olimex_stm32_h405
    olimex_stm32_h407
    olimex_stm32_p405
    rv32m1_vega
    black_f407ve
    black_f407zg_pro
    candlelight
    candlelightfd
    esp32c3_supermini
    icev_wireless
    neorv32
    promicro_nrf52840
    serpente
    stm32f030_demo
    stm32f103_mini
    stm32f401_mini
    stm32_min_dev
    pan1770_evb
    pan1780_evb
    pan1781_evb
    pan1782_evb
    pan1783_evb
    pan1783a_evb
    pan1783a_pa_evb
    particle_argon
    particle_boron
    nrf51_blenano
    nrf52_blenano2
    particle_xenon
    phyboard_electra
    phyboard_lyra
    phyboard_nash
    phyboard_polis
    phyboard_pollux
    reel_board
    pinetime_devkit0
    teensy40
    teensy41
    qemu_arc
    qemu_cortex_a53
    qemu_cortex_a9
    qemu_cortex_m0
    qemu_cortex_m3
    qemu_cortex_r5
    qemu_kvm_arm64
    qemu_leon3
    qemu_malta
    qemu_nios2
    qemu_riscv32
    qemu_riscv32e
    qemu_riscv32_xip
    qemu_riscv64
    qemu_x86
    qemu_x86_lakemont
    qemu_x86_64
    qemu_x86_tiny
    qemu_xtensa
    decawave_dwm1001_dev
    decawave_dwm3001cdk
    qomu
    quick_feather
    rak11720
    rak4631
    rak5010
    rpi_4b
    rpi_5
    rpi_pico
    rpi_pico2
    raytac_mdbt50q_db_33
    raytac_mdbt50q_db_40
    raytac_mdbt53v_db_40
    raytac_mdbt53_db_40
    rts5912_evb
    da14695_dk_usb
    da1469x_dk_pro
    ek_ra2a1
    ek_ra4e2
    ek_ra4m2
    ek_ra4m3
    ek_ra4w1
    ek_ra6e2
    ek_ra6m1
    ek_ra6m2
    ek_ra6m3
    ek_ra6m4
    ek_ra6m5
    ek_ra8d1
    ek_ra8m1
    fpb_ra6e1
    fpb_ra6e2
    mck_ra8t1
    rcar_h3ulcb
    rcar_salvator_x
    rcar_salvator_xs
    rcar_spider_s4
    rzg3s_smarc
    rzt2m_starter_kit
    cortex_r8_virtual
    riscv32_virtual
    ronoth_lodev
    ruuvi_ruuvitag
    scobc_module1
    faze
    legend
    stm32f3_seco_d23
    lora_e5_dev_board
    lora_e5_mini
    seeeduino_xiao
    wio_terminal
    xiao_ble
    xiao_esp32c3
    xiao_esp32c6
    xiao_esp32s3
    xiao_rp2040
    ip_k66f
    segger_trb_stm32f407
    ganymed_bob
    hifive1
    hifive_unleashed
    hifive_unmatched
    sim3u1xx_dk
    sltb004a
    sltb009a
    sltb010a
    xg24_dk2601b
    xg24_ek2703a
    xg27_dk2602a
    slwrb4104a
    slwrb4161a
    slwrb4170a
    slwrb4180a
    slwrb4250b
    slwrb4255a
    slwrb4321a
    xg23_rb4210a
    xg24_rb4187c
    efm32wg_stk3800
    slstk3400a
    slstk3401a
    slstk3402a
    slstk3701a
    longan_nano
    emsdp
    em_starterkit
    hsdk
    hsdk4xd
    iotdk
    nsim
    nsim_arc_v
    micromod
    nrf52_sparkfun
    sparkfun_pro_micro_rp2040
    sparkfun_red_v_things_plus
    sparkfun_thing_plus
    sparkfun_thing_plus_matter_mgm240p
    b_g474e_dpow1
    b_l072z_lrwan1
    b_l4s5i_iot01a
    b_u585i_iot02a
    disco_l475_iot1
    nucleo_c031c6
    nucleo_c071rb
    nucleo_f030r8
    nucleo_f031k6
    nucleo_f042k6
    nucleo_f070rb
    nucleo_f091rc
    nucleo_f103rb
    nucleo_f207zg
    nucleo_f302r8
    nucleo_f303k8
    nucleo_f303re
    nucleo_f334r8
    nucleo_f401re
    nucleo_f410rb
    nucleo_f411re
    nucleo_f412zg
    nucleo_f413zh
    nucleo_f429zi
    nucleo_f446re
    nucleo_f446ze
    nucleo_f722ze
    nucleo_f746zg
    nucleo_f756zg
    nucleo_f767zi
    nucleo_g031k8
    nucleo_g070rb
    nucleo_g071rb
    nucleo_g0b1re
    nucleo_g431kb
    nucleo_g431rb
    nucleo_g474re
    nucleo_h503rb
    nucleo_h533re
    nucleo_h563zi
    nucleo_h723zg
    nucleo_h743zi
    nucleo_h745zi_q
    nucleo_h753zi
    nucleo_h755zi_q
    nucleo_h7a3zi_q
    nucleo_h7s3l8
    nucleo_l011k4
    nucleo_l031k6
    nucleo_l053r8
    nucleo_l073rz
    nucleo_l152re
    nucleo_l412rb_p
    nucleo_l432kc
    nucleo_l433rc_p
    nucleo_l452re
    nucleo_l476rg
    nucleo_l496zg
    nucleo_l4a6zg
    nucleo_l4r5zi
    nucleo_l552ze_q
    nucleo_u031r8
    nucleo_u083rc
    nucleo_u575zi_q
    nucleo_u5a5zj_q
    nucleo_wb05kz
    nucleo_wb09ke
    nucleo_wb55rg
    nucleo_wba52cg
    nucleo_wba55cg
    nucleo_wl55jc
    sensortile_box
    sensortile_box_pro
    st25dv_mb1283_disco
    steval_fcu001v1
    steval_stwinbx1
    stm3210c_eval
    stm32373c_eval
    stm32c0116_dk
    stm32f072b_disco
    stm32f072_eval
    stm32f0_disco
    stm32f3_disco
    stm32f411e_disco
    stm32f412g_disco
    stm32f413h_disco
    stm32f429i_disc1
    stm32f469i_disco
    stm32f4_disco
    stm32f723e_disco
    stm32f746g_disco
    stm32f7508_dk
    stm32f769i_disco
    stm32g0316_disco
    stm32g071b_disco
    stm32g081b_eval
    stm32h573i_dk
    stm32h735g_disco
    stm32h745i_disco
    stm32h747i_disco
    stm32h750b_dk
    stm32h7b3i_dk
    stm32h7s78_dk
    stm32l1_disco
    stm32l152c_disco
    stm32l476g_disco
    stm32l496g_disco
    stm32l4r9i_disco
    stm32l562e_dk
    stm32mp157c_dk2
    stm32u083c_dk
    stm32u5a9j_dk
    stm32vl_disco
    stm32wb5mmg
    stm32wb5mm_dk
    visionfive2
    robokit1
    pico_pi
    tlsr9518adk80d
    cc1352p1_launchxl
    cc1352p7_lp
    cc1352r1_launchxl
    cc1352r_sensortag
    cc26x2r1_launchxl
    cc3220sf_launchxl
    cc3235sf_launchxl
    msp_exp432p401r_launchxl
    sk_am62
    colibri_imx7d
    verdin_imx8mm
    verdin_imx8mp
    ubx_bmd300eval
    ubx_bmd330eval
    ubx_bmd340eval
    ubx_bmd345eval
    ubx_bmd360eval
    ubx_bmd380eval
    ubx_evkannab1
    ubx_evkninab1
    ubx_evkninab3
    ubx_evkninab4
    udoo_neo_full
    up_squared
    up_squared_pro_7000
    yd_esp32
    yd_stm32h750vb
    nrf51_vbluno51
    nrf52_vbluno52
    esp32s3_touch_lcd_1_28
    nrf51_ble400
    waveshare_open103z
    rp2040_zero
    ch32v003evt
    we_ophelia1ev
    we_orthosie1ev
    we_proteus2ev
    we_proteus3ev
    blackpill_f401cc
    blackpill_f401ce
    blackpill_f411ce
    mini_stm32h743
    mini_stm32h7b0
    weact_stm32f405_core
    weact_stm32g431_core
    usb2canfdv1
    esp32s2_lolin_mini
    linum
    w5500_evb_pico
    xenvm
    nrf52820dongle
    nrf52833dongle
    nrf52840gmouse
    nrf52dmouse
    nrf52kbd
    nrf54lm20apdk
    nrf54lv10apdk
    nrf7120pdk
    thingy91
    thingy91x

  • Hi,

    Where is your board located? Can you show its directory structure?

    Best regards,
    Dejan

  • Hello,

    The directory structure of the board location is the following:

    <app_dir>/boards/<vendor>/<board_name> as shown below

    and below are the contents of one of the three boards used currently:

    Best regards,

    Stavros

Reply Children
Related