Matter Test Harness for certification

[Himansu]

Hi,

I'm doing a test with a matter test harness.
An error occurs in test case _TC-CNET-4.16, _TC-CADMIN-1.22 and _TC-DGGEN-2.3.

1. _TC-CNET-4.16: Step 2: should receive NetworkIDNotFound which is '3'
2. _TC-CADMIN-1.22: step 5: got success response instead of Invalid command
3. _TC-DGGEN-2.3: Step 3: got Constraint Error instead of Invalid Command

I would like to know how to solve this problem.

Test Harness Enviornment:

Board: Rraspberry PI4
Version: 2.10.2+spring2024

Sha: 2a350c8

DUT:

Board: nrf52840(BL654) custom board

SDK:2.7.0

Sample Application : Matter Door Lock

Please see attached log and PICS file for more information.

Thanks,

Himansu Donga
FAIL-TC-DGGEN-2.3.logFAIL_TC-CNET-4.16.logAML_PICS_XML_version V_27_SVE_1.3.1.zip

[AHaug]

Hello,

1. _TC-CNET-4.16: Step 2: should receive NetworkIDNotFound which is '3'

Looking through the test steps mentioned here https://github.com/nrfconnect/sdk-connectedhomeip/blob/master/src/app/tests/suites/certification/Test_TC_CNET_4_16.yaml it indicates that you've not set up the PIXIT.CNET.THREAD_2ND_OPERATIONALDATASET correct.

TH sends a remove-network command for PIXIT.CNET.THREAD_2ND_OPERATIONALDATASET, which based on your log should be 1111111122222222. The test expects it to fail with networking status 3 because this PAN ID should not be existing on your device, but instead it sends networkingStatus: 0 (success).

As mentioned this might happen if you either set up your dataset wrong, or it was not given at all since the default input is 1111111122222222 and that is the network ID that actually exists and that the DUT is commissioned on. Providing any other value should work.

2. _TC-CADMIN-1.22: step 5: got success response instead of Invalid command

Could you add the log for this step as well?

3. _TC-DGGEN-2.3: Step 3: got Constraint Error instead of Invalid Command

I might need some more time looking into this one, but in the meanwhile could you refer to 11.12.7.1 in the Matter spec and see if you've configured it according to the spec? 

Kind regards,
Andreas