Chaining - current status

[axel_s]

Hallo,

currently I evaluate a few possibilities to transmit larger data portions in a broadcast way, means using several advertising schemes. I have a few nRF52840-DK and use NCS v1.9.1 and Zephyr v2.7.99.

One at least semi-automatic way to handle data portions bigger than what is possible to transmit in a single packet is chaining. According to the standard it should be possible to transmit up to 1650 Bytes, which will be automatically splitted and re-assembled.

In one of the posts (https://devzone.nordicsemi.com/f/nordic-q-a/80022/chained-advertisement-nordic-52832-or-52840-how-to-generate-the-chained-advertisement-with-the-nrf-connect-sdk) I found the statement that chaining was not yet implemented, but "the feature to add more than 255 bytes is still on its way". Since it is more than a year after the statement I would like to know what the current situation is. At least the struct bt_data - used in bt_le_ext_adv_set_data(..) to handle advertising data - still contains a uint8_t variable data_len, which means there is a max size of 255. Therefore I assume that chaining is still not implemented. Is that correct?

Kind regards

Axel

[Karl Ylvisaker]

Hello Axel,

I am happy to inform you that chaining periodic advertisements is indeed now supported in the SoftDevice controller.
You will need to change the config for the data length to reflect the increased data length, for example:

CONFIG_BT_CTLR_ADV_DATA_LEN_MAX=1650

Additionally, you are correct that there is a limitation on the ad structures to contain more than 255 bytes, but the solution to this is to use multiple ad structures, like this:

static const struct bt_data ad[] = {
    BT_DATA(BT_DATA_MANUFACTURER_DATA, mfg_data, 200),
    BT_DATA(BT_DATA_MANUFACTURER_DATA, mfg_data, 200),
    BT_DATA(BT_DATA_MANUFACTURER_DATA, mfg_data, 200),
    BT_DATA(BT_DATA_MANUFACTURER_DATA, mfg_data, 200)
};

To quickly test this I would recommend starting out by modifying the periodic advertising sample to include this functionality.
Try this, and let me know in the case that you run into any issues or additional questions - please do not hesitate to ask! :) 

Best regards,
Karl

[axel_s]

Hallo Karl,

thanks for this info - I'll test it. But I have a few more related questions:

- The still existing limitation to 255 bytes (uint8_t for length) and the resulting workaround (array of bt_data elements) means in the end that the chaining intention of the standard - I can come up with a bigger portion of data and the stack manages the chaining - is not fully reached. I assume that the reception side will connect all the parts (it does so in first tests with bt_data of up to 255 bytes), but on transmission side it's up to me to do the split. But when testing with a single bt_data of up to 255 byte, one can see that the stack is able to manage the split. Up to 249 bytes there is only one data frame of "normal" extended advertising, and for 250 bytes or more there is one frame with 246 byte and a second chaining frame with the remaining bytes of data. So it seems "only" the length issue (uint8_t), what limits the function of the chaining, but I can imagine that changing this point is not that easy as it seems. Will there be some more work on that issue in the future to meat the intentions?

- In your code example the length of each bt_data is 200. The maximum here is 246 (see above) - correct?

- pure curiosity, and maybe you know some details: The standard mentions a max size for chaining of 1650 bytes. Where does this number come from? As far as I can see there is absolutely no connection to frame sizes or something else, what could lead in a mathematical way to 1650. Assuming a max data size per chaining frame of 246 bytes, the last one will contain 174 bytes only - 72 less than possible. So WHY this 1650 bytes???

[Karl Ylvisaker]

Hello,

Regarding the first question; apart from having to create a convenience function that splits the data into arrays of bt_data there is no other overhead on either side of the link, so the intention of the chained advertising is still met - it is just that you have to package the data before updating the advertising.

axel_s said:

- In your code example the length of each bt_data is 200. The maximum here is 246 (see above) - correct?

Yes, this is due to the overhead (2 bytes) used for each separate datafield in the advertisements, in addition to the chained advertising metadata.

axel_s said:

So WHY this 1650 bytes???

That is an excellent question, but I unfortunately do not have any good answer for it as it stands.. I will ask some of my colleagues and see if they know the reason for it being exactly 1650, and I'll update you here if I find a good answer to it! :)

Best regards,
Karl

[axel_s]

Hallo Karl,

thanks for the information - I'm looking forward to your findings regarding the ominous 1650.

In terms of the data handling, you are right, the effort is not huge. But in an overall view there is a difference in data handling on transmission and reception side - on transmission side I have to package the data, on reception side the stack will do the job. And the stack in general can handle this issue also on transmission side, as a simple example with 255 bytes can show. Such a discrepancy is not a good point, that's why I would prefer to have the stack doing it.

Or is my assumption regarding the reception side wrong and the stack leaves to that point also on reception side to the programmer? In that case there would be no discrepancy.

Kind regards

Axel

[axel_s]

Hallo Karl,

meanwhile we did some tests with data handling according your suggestion. Also the reception is running. But there are some points I have to mention:

- The size of the single BT_DATA elements in ad[] doesn't matter. The stack anyway filles up the data size using two or more of the BT_DATA elements up to a frame size of 296 for AUX_ADV_IND and AUX_CHAIN_IND frames. That leads to the fact that data packets are not conform to the standard anymore - the starting 0xFF 0xFF (company ID) in every single BT_DATA element is not existing in the transmitted frame at the correct position, also the 0xFF (manufacturer specific data) attached from the stack is missing, and maybe also the length information is wrong (I didn't count so far). These informations show up e.g. in the middle of the data in the frame, I assume the stack packages the data according to the BT_DATA elements and later on connects frame data for whatever reason.  Therefore these transmitted frames are not correct, the sniffer reports erroneous frames. The only way for correct frames would be to use max. length packets. But there is no global max length, since the first data portion is in the AUX_ADV_IND and the rest in AUX_CHAIN_IND, which are of different length.

- On reception side, data handling is also tricky. The receiver connects all transmitted frames, the callback function is called only once. The 0xFF (manufacturer specific data) at the beginning as well as the 0xFF 0xFF (company ID) are included several times in the data for a collected chaining data packet. This is maybe simply due to the errors on transmission side. The programmer now should pull out manually these parts, which can't work when the programmer doesn't know the packet size, which is used at transmission side for data splitting.

In my mind there should be some adaptations in the data handling. E.g. the 0xFF 0xFF (company ID) should be a parameter, not included in the data on transmission side. On reception side, the length field should be moved in the info structure as well as the 0xFF (manufacturer specific data) and 0xFF 0xFF (company ID), so that in the end the buf contains user data only.

In general, the mixed handling of user data and standard conform protocol bytes is not a good idea. It should be possible to hand over a single user data field of length X (according standard max 1650 bytes), automatically doing the chaining and hand a complete user data field of length X back on reception side. But than the length parameter data_len can't be an uint8_t any more.

Conclusion: The chaining still doesn't work. And I doubt that there can be a running chaining scheme soon.

Best regards

Axel

[Hung Bui]

Hi Axel, 

I'm taking over the case from Karl as I have played with chained advertising it a little bit. 

I'm not so sure about this : 

axel_s said:

That leads to the fact that data packets are not conform to the standard anymore

From my point of view The data packet is still conform to the standard after you merge them together. 

Could you try testing with my example here. I provided a project at the last reply. 

In the example I added a counter in every 10 bytes to the array. And send the data as 4 200-byte arrays each in one ad struct.

If you use a phone that support chained advertising you can capture the data as follow: 

Note that the nrf Connect app on the phone is a generic app and is not configure to detect the data handling of 200 bytes chunk. It can detect the data structure simply by following the advertising spec/standard. If you click on RAW data, you can see that it has a 0xC9 byte which is the length (201 bytes) before each data structure and then merge them together and send them via the chained advertising packets which has max 255 bytes size. 

On the receiver side, you can simply follow the spec and use the length byte to parse the advertising data you receive. As you can see it's been done on the phone. 

So I don't agree with your conclusion that chained advertising doesn't work in our current stack. 

[axel_s]

Hi Hung Bui,

thanks for your example. I'm sure I'll be back on that topic soon, but currently there are some more important issues. I'll report my results here.

Kind regards

Axel