ZigBee throughtput test

[Nicolas Kauffmann]

Hi,

I would like to test the ZigBee throughput like the examples ("ATT_MTU Throughput Example" for BLE and "Thread Throughput Measurement Application").

I'm not familiar with ZigBee (I read about it but I'm still learning). I started from the example (Zigbee Light Control with touchlink commissioning). I don't know if it's a good idea.

I implemented a function to send data with the API described in the documentation (zb_zcl_start_command_header, zb_zcl_finish_and_send_packet).

And as a first test, I call periodically ZB_GET_OUT_BUF_DELAYED and it works but, if I send at a too short period (ex: 500ms), the target blocks for a while then receives then it blocks and again and again...

When I say it blocks, it means that I don't get any trace from zcl_device_cb (callback for handling ZCL commands).

Is my method good or am I mistaken ?  Do I need to implement my own cluster ?

Best Regards,

Nicolas

[Jørgen Holmefjord]

Hi,

Your method seems fine, we are looking into implementing something similar ourself. One thing you should do (if not done already) is to disable "ZCL Default Response". Are you checking if you are recieving "RET_OK" status while calling "ZB_GET_OUT_BUF_DELAYED"? The amount of buffers is limited, but checking the status should give you an idea when to stop feeding stack with frames.

There is also a question on the definition of throughput, would like to measure on MAC, NWK, APS, ZCL layer? This will affect results due to increasing amount of ACKs and retransmissions. 

In order to measure throughput without APS ACKs, you have to modify "include/zcl/zb_zcl_commands.h" header:
@lines 264-266 you find:
 

/*ZB_SCHEDULE_CALLBACK(zb_apsde_data_request, ZB_REF_FROM_BUF(buffer));*/                   \
/* NK: Randomize delay for better performance (for example, parallel work of 2 devices). */ \
ZB_SCHEDULE_ALARM(zb_apsde_data_request, ZB_REF_FROM_BUF(buffer), ZB_RANDOM_VALUE(3));      \ 

If you swap commented line with the used one, you should be able to flood network with APS frames:

ZB_SCHEDULE_CALLBACK(zb_apsde_data_request, ZB_REF_FROM_BUF(buffer));                        \
/* NK: Randomize delay for better performance (for example, parallel work of 2 devices).  */ \
/* ZB_SCHEDULE_ALARM(zb_apsde_data_request, ZB_REF_FROM_BUF(buffer), ZB_RANDOM_VALUE(3)); */ \

This will not test on ZCL, but directly on APS.

Best regards,
Jørgen

 

[Nicolas Kauffmann]

Hi,

Here is my function to send a frame:

static zb_void_t send_frame(zb_uint8_t param)
{
    static zb_uint8_t cmd_id = 0;
    zb_uint16_t group_id = DEFAULT_GROUP_ID;
    zb_buf_t * p_buf = ZB_BUF_FROM_REF(param);

    zb_uint32_t data = 0xAAAABBBB;

    cmd_id = !cmd_id;

    /* initialize a ZCL packet */
    zb_uint8_t * _ptr = zb_zcl_start_command_header(p_buf, ZB_ZCL_CONSTRUCT_FRAME_CONTROL(ZB_ZCL_FRAME_TYPE_CLUSTER_SPECIFIC,  
                               ZB_ZCL_NOT_MANUFACTURER_SPECIFIC,                                                               
                               ZB_ZCL_FRAME_DIRECTION_TO_SRV, ZB_ZCL_DISABLE_DEFAULT_RESPONSE), 0, cmd_id, NULL);   

    /* fill in the packet data */
    ZB_ZCL_PACKET_PUT_DATA32(_ptr, &data);

    /* finalize and send the packet */
    zb_ret_t zb_ret = zb_zcl_finish_and_send_packet(p_buf, _ptr,(zb_addr_u*) &group_id, ZB_APS_ADDR_MODE_16_GROUP_ENDP_NOT_PRESENT, 0, LIGHT_SWITCH_ENDPOINT, ZB_AF_HA_PROFILE_ID, ZB_ZCL_CLUSTER_ID_ON_OFF, NULL); 
    if (zb_ret != RET_OK)
    {
        NRF_LOG_ERROR("zb_ret = %d", zb_ret);
    }
}

I call "ZB_GET_OUT_BUF_DELAYED" with app_timer and yes I've seen that there is a memory pool of buffers so I checked the return but I don't get any error even though I send every ms. 

I changed what you said which is in the macro function ZB_ZCL_SEND_COMMAND_SHORT_WITHOUT_ACK but I don't see any call to this function (maybe used in the library ?). And unfortunately, the result doesn't change.

I'd like to measure the throughput on ZCL layer but if I could measure it on APS first, it would be nice.

I enabled all the logs and I don't know if it can help but here is what I get when it blocks:

<info> app: frame_received = 14 param = 11
<info> zboss:  93 C2 D6 C4 00 80 DE AD|........
<info> zboss:  12 02 F3 0C 35 00 1A 08|....5...
<info> zboss:  60 01 06 00 00 00 01 00|`.......
<info> zboss:  00 00 DE AD 12 02 F3 0C|........
<info> zboss:  36 00 1A 08 76 01 06 00|6...v...
<info> zboss:  00 00 01 00 00 00 DE AD|........
<info> zboss:  47 01 13 0D 37 00 00 00|G...7...
<info> zboss:  10 3D 61 88 F5 0D 91 38|.=a....8
<info> zboss:  AA 11 BA 08 12 FD FF 11|........
<info> zboss:  BA 1E 19 16 B6 A7 E8 69|.......i
<info> zboss:  1B 05 69 28 19 00 00 00|..i(....
<info> zboss:  16 B6 A7 E8 69 1B 05 69|....i..i
<info> zboss:  00 45 98 6E 6E 1C 84 CF|.E.nn...
<info> zboss:  68 C8 9A 31 80 7F 60 58|h..1..`X
<info> zboss:  BC DD CF 56 1D E0 FA DE|...V....
<info> zboss:  AD 0F 81 13 0D 37 00 00|.....7..
<info> zboss:  00 10 05 02 00 F5 00 80|........

and when it's OK:

OK
<info> app: frame_received = 18 param = 5
<info> zboss:  00 80 DE AD 12 02 DC 21|.......!
<info> zboss:  3D 00 1A 08 60 01 06 00|=...`...
<info> zboss:  00 00 01 00 00 00 DE AD|........
<info> zboss:  12 02 DC 21 3E 00 1A 08|...!>...
<info> zboss:  76 01 06 00 00 00 01 00|v.......

Thank you,

Best regards,

Nicolas

[Jørgen Holmefjord]

Hi,

I got some help with this from one of our Zigbee developers. We have created an example that you can use for testing throughput.

There is at least one issue in your code, you should not use "ZB_APS_ADDR_MODE_16_GROUP_ENDP_NOT_PRESENT". Groupcast is actually a broadcast and is limited. There is a special table to keep track of broadcast messages, because those packets are just retransmitted by every node couple of times. All nodes try to keep track of which messages they've already transmitted, so they do not cause a broadcast storm.

A couple of things you should consider when measuring throughput:

1. You should not use broadcast transmission to measure throughput (groupcast).
2. It is worth having a second device that will accept those "custom" commands to avoid traffic containing error messages.
3. Going from light sample example with coordinator may be easier (there is an explicit place where you read light bulb's address). Although to avoid going through coordinator to the second node it is worth powering it off after the commissioning and reset all devices.

The attached file contains an example based on the light_switch example in the SDK for Thread and Zigbee v1.0.0. It have been tested with 50ms intervals and there seems to be pretty traffic without crashes at this rate. Please check the patch file for diff towards original example. 

light_switch_throughput.zip

diff.patch

Hope this helps!

Best regards,
Jørgen

[Nicolas Kauffmann]

Hi,

Thank you !

It works but I have a few more questions if you don't mind :

I've read that a ZigBee packet is 127 bytes (payload) (up to 133 bytes (with headers)).

When I send at 50ms during one second (20 frames), it gives me:

133 * 8 = 1064 bits

1064 * 20 (frames) = 21280 bits -> 21kbps

Am I correct ?

If I want to reach (approximatively) the 250 kbps announced by Nordic, I can decrease the period but if it's too low I have memory buffer problem, ZB_GET_OUT_BUF_DELAYED returns RET_ERROR indefinitely.

Does this throughput (250kbps) is reachable only at lower layer as you suggested earlier ?

Best regards,

Nicolas

[Jørgen Holmefjord]

Hi,

When you are talking about 250 kbps, I suppose you are referring to the on-air datarate of the 802.15.4 radio? This is the datarate on the PHY layer, specified in 802.15.4 specifications. You will noe be able to achievable this throughput in application.

This is mainly because of protocol architecture and limitations, i.e.:

• A payload with data is only a part of 802.15.4 frame, a big portion of the frame is consumed by different headers e.g. MAC header, NWK header, etc.
• Before another packet is sent the source node has to wait for an ACK frame from the destination node to get a confirmation that current frame reached the destination.
• 802.15.4 specification implies that two consecutive packets cannot be sent without waiting IFS (Inter-frame spacing) time – this is to give the other side time to process packets.
• CSMA-CA procedure delays sending a packet if channel is busy due to another radio activity.

We do not have throughput measurements for Zigbee yet, but for Thread a point to point throughput in a typical condition is about 80 kbps. Zigbee should have similar numbers.

Best regards,
Jørgen