Communication lost with Zigbee

[mau]

Hi to everybody,

I'm developing over nRF5 SDK v. 3.2.0 a Zigbee node in router role. In my network I've four of my nodes plus several commercial led light driver and a coordinator.

After random amount of time (from one day, to one week) some of my nodes lost communication with coordinator and never come back to communicate, the only way to recover is reboot. (rejoin is not needed).

I attach the sniffer traces, so you can see what happens; unfortunately I can't caught the exact moment in which the issue start because the long time observation period.

trace1 trace2

You have to set this preconfigured key in Wireshark in order to view the trace: 48:35:70:4b:7c:60:63:f1:4f:89:66:f5:04:65:ed:7d

In order to understand the issue I provide to you my devices long and short addresses:

F4:CE:36:53:37:52:8C:10 CBB2 Currently working
F4:CE:36:31:19:B7:A3:25 5F7A Connection lost
F4:CE:36:84:13:85:21:71 BCBC Connection lost
F4:CE:36:8C:78:03:53:9D 30A9 Connection lost

As you can see there are several messages from my devices, so these nodes are not unjoined, but the coordinator ignore every message from them, even "Device Announcement" messages.
Affected nodes only send this kinds of messages: "Network address request", "Link status", "Device Announcement", "Route request".

I can't understand the status that these nodes assumes, it seems that they lose the routing table.

[Marjeris Romero]

Hi,

Sorry for the late reply. I took a look at the traces but without really understanding what could be the reason behind the nodes leaving the network. How do you know they have lost communication with the coordinator? Do they send a Leave message or do they get unresponsive somehow?

Anyways I have also forwarded this case to our Zigbee team to see if they may have any more insight about what could be happening.

Best regards,

Marjeris

[mau]

Hi Marjeris,

Marjeris Romero said:

How do you know they have lost communication with the coordinator?

The coordinator no longer receives any configured report or "device announce" message from affected nodes and when I try to read an attribute (I do it using CLI on coordinator) I see only a "route request" message from coordinator; there is no "route reply" from affected node and coordinator do not proceed to read the attribute.

Best regards,

Maurizio

[Marjeris Romero]

Hi Maurizio,

Could you provide brand and model information on third party nodes?

In the capture you send there are some nodes (e.g 0x5F7A) that are constantly requesting route to the coordinator. Since it doesn't get any reply it can't send any reports. So it might be that the nodes in question don't have a route to the coordinator for some reason or devices along the route misbehave.

We would need a better understanding of the network topology and information about stacks which are used on all nodes so we can have the whole picture.

The best thing would be to get a more isolated and repeatable way to reproduce the issue. Could you try to figure out the smallest network which exposes this behavior? And does this issue also occurs when only ZBOSS nodes are on the network?

Best regards,

Marjeris

[mau]

Hi Marjeris,

in our network there are Sunricher and Harward Zigbee led drivers in 1:10 ratio (1 our device for 10 third party nodes). The coordinator is based on ETRX357 Telegesis (now Silabs) module. Also Harward node is based on that module.

The role of our device is a motion sensor that turn on/off led drivers by sending Move to level with On/Off command and occupancy attribute (0x0000) of occupancy sensing cluster (0x0406) is reported to the coordinator.

Marjeris Romero said:

Could you try to figure out the smallest network which exposes this behavior?

Our smallest network is composed of 4 our device, 40 third party nodes and 1 coordinator.

Marjeris Romero said:

And does this issue also occurs when only ZBOSS nodes are on the network?

The issue never occur in a ZBOSS only network.

Best regards,

Maurizio

[Marjeris Romero]

Hi Maurizio,

Just to update you here. I am waiting for some input from R&D for ideas on how to proceed here, but as I see it it looks like it would be difficult for us to recreate the issue when so many third party devices are involved.

I was thinking, what happens if the affected nodes are in proximity of the coordinator? Does the coordinator reply to the route request of the affected nodes then? The nodes need a route request reply in order to send reports, so that's why they stop reporting. Maybe you can sniff the traffic between the coordinator and one of the affected nodes when the node is placed in proximity to the coordinator?

BR,

Marjeris