Zigbee routing feature problem

[Mateus Vesuvious]

Hi guys, 

 So i have some Nordic DK board including some dongles, NRF52840 DK, NRF52833 DK running with Zigbee and thread SKD 4.1.0 and make a mesh out of them. But I have this problem with mesh topology in Zigbee relating to route discovery. I did 2 experiments

1.The first experiment is set up like this figure 

So I have a mesh of devices including some zigbee routers and 1 coordinator like in (1). I tried to move the router 1 on top away from the coordinator so that it is out of range and would send a message to the coordinator through a hop in the middle like in the (2) . From what i learn in the zigbee spec, the router will try to fix and create a new route if the destination is unreachable. I use a sniffer to monitor the network but i did not see any route request sent even after 3-4 minutes. Then, I reset the device but still nothing happened. So how can I make a out of range router to rebuild a route to the coordinator. Is it suppose to make a new route when the destination address is not in the routing table.

2.The second experiment i did is that I create a mesh with 1 coordinator and 4 routers like the figure below 

So this time, i managed to make router 1 to join through router 2( which is near router 3) and send message to coordinator like in (1). But the problem is that, when i turn off router 2. In theory Router 1 should re route to path (2). But it did not, when it sent out the message, it kept sending to the addr 0x0000 and not start to send any route request. This time i also waited for 4 minutes and reset every device in the network. But still nothing happened. 

For the firmware inside, I ran Cli zigbee example coordinator and ran a custom firmware from light bulb example on all of the routers. I will attach the firmare at the end so that you can take a look at it with all the SDK config and makefile 

Thank you for any help, the code for router is included below 

Mateus 

light_bulb.zip

[tomchy]

Hello Mateus!

Could you point out the section inside the spec that obliges the router to keep an active route to the coordinator?

I was just thinking - could you configure a periodic reporting on the mobile router and rerun your tests? The CLI sample already has all required commands, no recompilation is needed.

The idea here is that sending reports to the coordinator at constant interval will push the router to refresh the route. Additionally - you will see if the router figures out the route, based on other packets.

BR,

Tomchy

[Mateus Vesuvious]

Hi Tomchy, 

Thank you for your answer.

For the first question, may be you misunderstand the figure i do not intend to say that the router should be keeping an active route. But the dash lines just mean that each router is in within the range of each other and it is possible for the router to route pathways along different dash lines . But as i have in the attached code. I only send the data to the coordinator so the   router should find the way to relay the message to the coordinator (a) if the coordinator is not the routing table and (b) if the packet sending not receive an ACK. as in section 3.6.3.5.1 Initiation of Route Discovery of the Zigbee spec line 8981 - 8983. But i do not see that behavior in the test above. 

For the second question, i also tested the configure report a few days ago. I use a the coordinator to bind and then configure the report for every 10 sec, but the result is still the same. Do I have to lower the time to even more to see the route discovery faster? 

And for the last question, can you show me what kind of packets to expect to know if the router figure out the route. For me i only know the about route request and route reply but i did not see any of them in either of the test case above. 

Thank you very much, please reply soon.

Best regards, 

PS: I include the spec that i read with this post so you can check if i misunderstand anything.

zigbeealliance.org/.../docs-05-3474-21-0csg-zigbee-specification.pdf

Tu 

[Marjeris Romero]

Hi Mateus,

Interesting experiments. Let's try to analyze the specification section 3.6.3.5.1 Initiation of Route Discovery

The unicast route discovery procedure for a device shall be initiated on a ZigBee router or ZigBee coordinator by the NWK layer up on receipt of an NLME-ROUTE-DISCOVERY.request primitive from the next higher layer where the DstAddrMode parameter has a value of 0x02. Or, up on receipt of an NLDE-DATA.request primitive from a higher layer with the DstAddrMode set to 0x02 and the discover route sub-field set to 0x01, for which there is no routing table entry corresponding to the routing address of the destination device (the 16-bit network address indicated by the DstAddr parameter).

Or, on receipt of a frame from the MAC sub-layer for which the value of the destination address field in the NWK header is not the address of the current device, the address of an end device child of the current device, or a broadcast address and:
• The discover route sub-field of the frame control field has a value of 0x01, and
• there is no routing table entry corresponding to the routing destination of the frame, and
• either the value of the source address field of the NWK header of the received frame is the same as the 16-bit network address of one of the end device children of the current device, or
• the nwkUseTreeRouting attribute of the NIB has a value of TRUE.

The route discovery procedure for a multicast address shall be initiated by the NWK layer either in response to the receipt of an NLME-ROUTE-DISCOVERY.request primitive from the next higher layer where the DstAddrMode parameter has a value of 0x01, or as specified in section 3.6.6.2.2.

Based on this my thoughts on the results of your experiments are the following:

Experiment nr 1:

From what i learn in the zigbee spec, the router will try to fix and create a new route if the destination is unreachable. I use a sniffer to monitor the network but i did not see any route request sent even after 3-4 minutes.

Did you look into the Link Status of Router 2 before removing Router 1 out of reach of the coordinator? If Router 2 had a link to the coordinator before Router 1 was remove away from the coordinator then I don't know if you will necessary get a Route request when Router 1 gets out of the coordinator reach, since there was a routing table entry corresponding to the routing destination of the frame in Router 2.

Experiment nr 2:

In theory Router 1 should re route to path (2). But it did not, when it sent out the message, it kept sending to the addr 0x0000 and not start to send any route request. This time i also waited for 4 minutes and reset every device in the network. But still nothing happened. 

What was the Link Status in Router 1 before turning Router 2 off? Did Router 1 had Router 2 in his Link list? Why do you spect Router 1 to sent out a Route request message? Is Router 1 receiving a NLME-ROUTE-DISCOVERY.request from a higher layer? Please be more specific as what do you mean with "In theory...."?

Best regards,

Marjeris

[Mateus Vesuvious]

Hi Majeris,

Thank you for your answer. I am very happy to received it and give you back some more information so that i can crack this as soon as possible.  

Did you look into the Link Status of Router 2 before removing Router 1 out of reach of the coordinator? 

- Yes i did check all of the link status of Router 1, 2, and Coordinator. Since i only place them in the vincinity of each other (about 3 meter) each of them already contain links to others and the link cost are ranging from 2-4 for both incoming and outgoing path cost. In addition, from what i saw in your answer, when the router 2 had an entry in its routing table to the destination which router 1 trying to send, then router 1 will not initiate any request to re route. I do not think that this is really the case. Correct me if I am wrong, after 3 -4 minutes of router 1 get out of range, both router 1 and coordinator will not receive any link status update. With each link status periodically send at 15s and routing table aging is 3, the entry of routing table in both of them should be erased by then and then from what i understand, router 1 should init a route request. So, I do not see any how and why router 2 having a entry in routing table which router 1 trying to send to have anything to do with router 1 realising its link is broken and trying to repair itself. So please advise me how to get router 1 to do a route discovery when it is out of range. On which kind of condition that it has to meet before being able to do route discovery since it is pretty confusing to me until this point. If it is possible, can you point out the part where the condition in my test violate the zigbee specs.

What was the Link Status in Router 1 before turning Router 2 off? Did Router 1 had Router 2 in his Link list? Why do you spect Router 1 to sent out a Route request message? Is Router 1 receiving a NLME-ROUTE-DISCOVERY.request from a higher layer? Please be more specific as what do you mean with "In theory...."? 

When i said in theory, I means that with what i read from the zigbee specs. Well, like in the exp#1, i also check for the link status, and read the log coming out of router 1 to see that it did have router 2 and 3 in its link status. Also from the log, it pop up a no parent link failure message when i turn off router 2. That is the reason why I expect to see a route request to 0x0000. But i did not. So in this case, can you implicit when will the router will initiate a route request. Does the process of forming a mesh just slow or need to acquire any specific condition before it start. If it is slow, how can i speed this up, If there is something i could do to make this faster, please enlighten me. I am very happy to receive your detail advices.

Have a good day, 

Best,

Tu

[Marjeris Romero]

Hi,

Mateus Vesuvious said:

Yes i did check all of the link status of Router 1, 2, and Coordinator. Since i only place them in the vincinity of each other (about 3 meter) each of them already contain links to others and the link cost are ranging from 2-4 for both incoming and outgoing path cost.

Perhaos you can supply some sniffer logs from your experiments? I have run your example but I am only able to test with 2 routers and 1 coordinator at the moment.

Mateus Vesuvious said:

With each link status periodically send at 15s and routing table aging is 3, the entry of routing table in both of them should be erased by then and then from what i understand, router 1 should init a route request.

Can you point to the section on the specification that backs your assumptions?

What I have found is that if a device fails to receive nwkRouterAgeLimit link status messages from a router neighbor in a row, the old outgoing cost information is discarded and the neighbor entry is considered stale and may be reused if necessary to make room for a new neighbor.

Here you have a simple snippet, that you may plug into the zigbee_cli_cmd_zdo.c file. Afterwards - you can read the internal neighbor table through zdo neighbors command to check if the neigbour entry for router 1 is considered stale or not by the coordinator:

/**

 * @brief Prints all entries of the neighbor table.

 *

 * @code

 * zdo neighbors

 * @endcode

 *

 * Example:

 * @code

 * zdo neighbors

 * @endcode

 */

static void cmd_zb_neighbors(nrf_cli_t const * p_cli, size_t argc, char **argv)

{

    uint32_t i;

    zb_ieee_addr_t ieee_addr;

    zb_uint16_t addr;

 

    if ((argc > 1) || (nrf_cli_help_requested(p_cli)))

    {

        print_usage(p_cli, argv[0], "");

        return;

    }

 

    nrf_cli_fprintf(p_cli,

                    NRF_CLI_NORMAL,

                    "\r\n[idx] "

                    "ext_addr         "

                    "short_addr "

                    "type "

                    "sleepy "

                    "lqi "

                    "age "

                    "timeout "

                    "relationship\r\n");

 

    for (i = 0; i < gc_neighbor_table_size; i++)

    {

        if ((gc_neighbor[i].used == 0) ||

            (gc_neighbor[i].ext_neighbor != 0))

        {

            continue;

        }

 

        nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, "[%3u] ", i);

 

        zb_address_ieee_by_ref(ieee_addr, gc_neighbor[i].u.base.addr_ref);

        print_eui64(p_cli, ieee_addr);

 

        zb_address_short_by_ref(&addr, gc_neighbor[i].u.base.addr_ref);

        nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " 0x%04x    ", addr);

 

        switch (gc_neighbor[i].device_type) {

            case ZB_NWK_DEVICE_TYPE_COORDINATOR:

                nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " ZC  ");

                break;

 

            case ZB_NWK_DEVICE_TYPE_ROUTER:

                nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " ZR  ");

                break;

 

            case ZB_NWK_DEVICE_TYPE_ED:

                nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " ZED ");

                break;

 

            default:

                nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " ??? ");

                break;

        }

 

        if (gc_neighbor[i].rx_on_when_idle)

        {

            nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " false ");

        }

        else

        {

            nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " true  ");

        }

 

        nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " %03u", gc_neighbor[i].lqi);

 

        if (gc_neighbor[i].device_type != ZB_NWK_DEVICE_TYPE_ED)

        {

            nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " %03u  ---   ", gc_neighbor[i].u.base.age);

        }

        else

        {

            zb_time_t exp_time = ZB_TIME_SUBTRACT(gc_neighbor[i].u.base.time_to_expire, ZB_TIMER_GET());

            nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " --- %07u", ZB_TIME_BEACON_INTERVAL_TO_MSEC(exp_time) / 1000);

        }

 

        switch (gc_neighbor[i].relationship) {

            case 0x00:

                nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " parent");

                break;

 

            case 0x01:

                nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " child");

                break;

 

            case 0x02:

                nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " sibling");

                break;

 

            case 0x03:

                nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " none");

                break;

 

            case 0x04:

                nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " previous child");

                break;

 

            case 0x05:

                nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " unauthenticated child");

                break;

 

            default:

                nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " unknown");

                break;

        }

 

        if ((gc_neighbor[i].device_type != ZB_NWK_DEVICE_TYPE_ED) &&

            (gc_neighbor[i].u.base.outgoing_cost == 0))

        {

            nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, " [STALE]\r\n");

        }

        else

        {

            nrf_cli_fprintf(p_cli, NRF_CLI_NORMAL, "\r\n");

        }

    }

 

    print_done(p_cli, ZB_TRUE);

}

...

    NRF_CLI_CMD(neighbors, NULL, "lists all entries from the neighbor table", cmd_zb_neighbors),

    NRF_CLI_SUBCMD_SET_END

};

Example output for an aged-out router (ZC in this case):

> zdo neighbors
[idx] ext_addr         short_addr type sleepy lqi age timeout relationship
[  0] f4ce36aaaaaaaaaa 0x0000     ZC   false  255 003  ---    parent [STALE]

You can for example check if if the entry for router 1 in the coordinator is really really out of range from the coordinator by checking if the entry is stale.

Mateus Vesuvious said:

So please advise me how to get router 1 to do a route discovery when it is out of range. On which kind of condition that it has to meet before being able to do route discovery since it is pretty confusing to me until this point.

If you have a route from Router 1 <-> Router 2 <-> Coordinator where Router 1 and you turn OFF router 2 and coordinator is out of reach you will then see a route request from Router 1 after a couple of retries when sending a ZCL command with required ack.

Mateus Vesuvious said:

Also from the log, it pop up a no parent link failure message when i turn off router 2. That is the reason why I expect to see a route request to 0x0000. But i did not.

Did you see any retries from router 1 trying to send a message to the coordinator? Just one ZCL command that doesn't reach the destination is not enough to trigger searching a new path in the stack. It can be observed in frames through the sequence number field in Zigbee Network Layer Data. The sequence number should increment while the lower layers perform the retries. After about 10-12 ZCL commands you should see a route discovery request.

Best regards,

Marjeris

[Mateus Vesuvious]

Hi Majeris, 

Thank you for your reply, i have managed to make a route request happen. But I have some questions to ask, 

So if the router 1 create a new route to coordinator but after that it receive a link status from the coordinator. Does it going to send the next packet by the new route through another router or going directly to the cordinator. In my experiment, I have this problem that after a few time making a new route, the node keep sending data to the coordinator directly. So I will a packet sniffer to you so you can take a look at it.

And I also have an other similar question with many to one request, as far as i know, It will tell the routers to reserve an entry for sending the data to the collector (which is in my case is the coordinator, right?) and the router when it send somethinng will send a route record to it to send. But when I try it, routers did not send anything, instead, after a while of sending data through the new route, when MTORR hit, the router go back to use the direct path. Can you guess why it happen?

Thank you very much, hope to hear from you soon and have a nice day,

Here is the capture 3704.Problem with link status.pcapng

Best regards,

Mateus

[Mateus Vesuvious]

Hi Marjeris, 

Do you have any explanation about why the connection is broken after just being formed? I have something to ask you

I use the api that you gave me to log the neighbor table and the routing table to debug the problem, And it is look like router can still receive the link status message one in a long while. So i have logged the tables out, i can see 0x0000 is still in side the neighbor table even when the router can not send anything to the coordinator. The entry is always their even with a very low LQI (like under 15) so i guess the router still see the coordinator as its neighbor and keep sending directly. But it is just my theory and since i can not find any thing on the zigbee spec for that. I want to ask you. 

1.If an entry is available in neighbor table, is it true that the node will send the message directly to the address of that entry without looking at the routing table. 

2. When a new path is formed, what is the behavior of the node if it receive a link status, will it prioritize the neighbor table and not use the corresponding entry in routing table

3. If it is the case, can you point out some ways for me to avoid this?

Thank you and please reply soon,

Best regards, 

Mateus

[Mateus Vesuvious]

Hi Majeris, 

Is there any update on the problem I had?

I have some other things to update to you. So when I printed out the neigbor table and then check the link status, I can see that they are not really the same. Usually, the link status will lack some addresses from the neighbor table. And because i also print out the aging parameter of the node to log, I am pretty sure that all of the missed addresses are still exist (since their aging keeps changing with every 15s) I want to know if this is a normal behavior of the node or is it an uncommon things to see. 

Thank you very much, please let me know if there are some new update. 

Best regards, 

Mateus

[Marjeris Romero]

Hi Mateus,

I am very sorry for the lack of reply here. First thing, I will need a network key for reading the data from the packets in the sniffer trace you sent to me. If you start the sniffer trace before the network is formed this network key will be stored by wireshark automatically, but if the sniffer trace is started later on you should have a network key available to decode the sniffer traces. Right now the packets on the sniffer trace is undecoded:

And can we have a summary of the latest questions you have asked for this ticket? And what scenario (1 or 2 on the figures above) are we talking about? I need to try to summarize the inquiries in this ticket to past them to our developers.

Mateus Vesuvious said:

I have some other things to update to you. So when I printed out the neigbor table and then check the link status, I can see that they are not really the same.

Here I wouldn't expect it to be the same.

Link Status commands are use to keep track of route symmetry. A router will send out a radio 1 broadcast to indicated link status. All the neighboring routers keep track of these messages to determine which routers can hear them and which routers they can hear. So you can have more addresses in the neighbor table withot having them listed in the Link Status command.

While the neighbor table contains information on every device within transmission range, you can have entries in the neighbor table with value of 0 for outgoing cost field if no link status command listing from this device has been received.

The age field indicates the number of nwkLinkStatusPeriod intervals that have passed since the last link status command frame was received, up to a maximum value of nwkRouterAgeLimit.

More information about the neighbor table can be found in the Zigbee specification:
docs-05-3474-21-0csg-zigbee-specification.pdf

Best regards,

Marjeris