Sending multiple notify packets per connection interval

RE: Sending multiple notify packets per connection interval

Neil_CPD — Mon, 04 Oct 2021 08:43:23 GMT

Hi Torbjorn,

Thank you for your support with this, I've integrated with the project and it's working well.

...also thank you Simon and Hung 🙂

Many thanks,

Neil

RE: Sending multiple notify packets per connection interval

ovrebekk — Fri, 10 Sep 2021 14:32:50 GMT

Hi Neil

In that case the fix should probably not be needed, if you are sending 160 byte packets.

Migrating to a newer SDK should be less of a hassle now than it was in the old days (with the nRF5 SDK), but this depends on how complex your project is obviously.

The best of luck 🙂

Best regards
Torbjørn

RE: Sending multiple notify packets per connection interval

Neil_CPD — Fri, 10 Sep 2021 07:28:23 GMT

Morning Torbjørn,

The end goal is to be able to send a minimum of 3 x160 byte packets every 15ms interval so this should be fine. My new issue is that the original project is on an older SDK version that the fix doesn't work on but the fix for that is to update the sdk to the latest.

Thanks for all your help.

RE: Sending multiple notify packets per connection interval

ovrebekk — Fri, 10 Sep 2021 07:17:09 GMT

Hi Neil

Looking at his sniffer trace I see that he sent longer packets (35 bytes, including the 3 byte attribute header).

Clearly this is long enough to loosen the timing enough so that you can upload more packets during the connection event.

If your end goal is to maximize throughput, rather than sending many small packets, this is a simple fix to the problem. Just ensure you use a characteristic that supports longer packets (the heart rate measurement characteristic does not).

Best regards
Torbjørn

RE: Sending multiple notify packets per connection interval

Neil_CPD — Thu, 09 Sep 2021 07:12:25 GMT

Thank you for the feedback, I will try this today. Do you know how Hung managed to send so many packets?

RE: Sending multiple notify packets per connection interval

ovrebekk — Thu, 09 Sep 2021 06:47:56 GMT

Hi Neil

Checking my test results again I realized it was still only sending two packets pr connection interval, it is just the interval that was much shorter than I expected. In other words I got the same issue as you.

Discussing this with the software team it seems there is a limit in the SoftDevice controller on how many packets it can queue, which is set by the SDC_DEFAULT_TX_PACKET_COUNT define in \v1.6.1\nrfxlib\softdevice_controller\include\sdc.h:

/** @brief Default maximum Link Layer TX packet count per connection.
 *
 * With the default count, the application is able to refill the buffers during a connection event.
 */
#define SDC_DEFAULT_TX_PACKET_COUNT 3

Normally this isn't a big limitation, as it is possible to upload new data while the connection event is active, but in your case you are sending very short packets, which makes the timing much tighter.

On the nRF52 this works, but on the nRF53 there is some additional overhead because the packets have to be transferred from the Bluetooth host running on the application core to the Bluetooth controller in the network core.

If you change the number above and recompile you should see that you can send more packets pr connection event. I set it to 5, which gave me 4 packets pr connection event consistently.

Alternatively, if you send longer packets you should be able to send more of them, but I don't how long they need to be. The throughput example uses packets > 200 bytes, and doesn't have this problem (if it did the issue would probably have been spotted earlier).

I will discuss this with the team and see if we can get a better solution in the future, such as a configuration parameter that you can set in the project, so that editing the library header files is not necessary.

Best regards
Torbjørn

RE: Sending multiple notify packets per connection interval

Neil_CPD — Wed, 08 Sep 2021 15:55:05 GMT

Thank you Torbjørn, I can only get two packets on any connection interval. it doesn't seem to matter if it's 40ms or 15ms

RE: Sending multiple notify packets per connection interval

ovrebekk — Wed, 08 Sep 2021 15:50:56 GMT

Hi Neil

I can try to reproduce the issue here, yes. My first test wasn't very fruitful as it showed a lot more notifications sent per connection event, but I was testing with a higher connection interval. I will spend some more time with this tomorrow and see if I can make an example demonstrating higher throughput with 15ms connection interval.

Best regards
Torbjørn

RE: Sending multiple notify packets per connection interval

Neil_CPD — Tue, 07 Sep 2021 12:54:27 GMT

Hi Torbjørn,

Thank you for the advice. I can see all the config values that Hung has set in the project that he has attached but this hasn't made any difference I still only get 2 notify packets per connection.

Do you have a nrf5340-dk that you could test with and attach the build folder so I could program the hex as a test please?

Or if you have any other suggestions to try next that would be great.

Thank you.

Neil

RE: Sending multiple notify packets per connection interval

ovrebekk — Tue, 07 Sep 2021 11:45:45 GMT

Hi Neil

[quote user="Neil_CPD"] If there a good way to verify that the values have been built into the project?[/quote]

If you open your build folder, and navigate to the subfolder \hci_rpmsg\zephyr, there should be a file called .config which contains all the pre-compiled configuration settings:

In this file you can search for the config parameter you are interested in (such as CONFIG_BT_CTLR_RX_BUFFERS), and verify that it is set as you expect.

[quote user="Neil_CPD"]if I add the child image folder do I need to regenerate the project in nrf Connect?[/quote]

Yes. This is not picked up by the build automatically, and you need to re-run cmake in order for these settings to be included.

You can do this from the command line by running west build with the -c parameter:

west build -c

If you use Segger Embedded Studio there is a menu option to rerun cmake:

Menu -> Project -> Run CMake...

Best regards
Torbjørn

RE: Sending multiple notify packets per connection interval

Neil_CPD — Mon, 06 Sep 2021 13:51:19 GMT

Thanks for the reply, I've checked the "nrf connect SDK project.." in hci_rpmsg_menuconfig and they seem to be correct, If there a good way to verify that the values have been built into the project?

if I add the child image folder do I need to regenerate the project in nrf Connect?

Thanks

Neil

RE: Sending multiple notify packets per connection interval

ovrebekk — Mon, 06 Sep 2021 13:31:36 GMT

Hung is on leave, and I will help you out in the mean time.

Normally issues with throughput on the nRF53 are caused by not configuring the hci_rpmsg application running on the network core properly.

If you take a look at the throughput example in the nrf repository you can see that they define a separate config file for this application, in a folder called child_image:
https://github.com/nrfconnect/sdk-nrf/tree/master/samples/bluetooth/throughput/child_image

Can you try something similar for your project and see if it improves the number of notifications you can achieve?

Best regards
Torbjørn

RE: Sending multiple notify packets per connection interval

Neil_CPD — Sun, 05 Sep 2021 19:44:54 GMT

Hi,

Sorry for the delay but I been on vacation..

I've spent some time this week trying to re-create the performance that you've managed to find but i've not been able to. I tried with the central and also on my phone but no success.

I've attached your projects with my build folders in. If you could please verify on a nrf5340-dk that would be great.

I've also looked at the log for the rpmsg and the messages seem to also be groups of two.devzone.nordicsemi.com/.../3264.hr_5F00_mp_5F00_NC.zip

devzone.nordicsemi.com/.../nc_5F00_hr_5F00_mp_5F00_test.pcapng

RE: Sending multiple notify packets per connection interval

Hung Bui — Tue, 17 Aug 2021 10:08:31 GMT

Hi Neil,

Attached you can find my example of sending 25 notifications in a 15ms interval connection.

It's not the maximum limit I believe. If you increase CONFIG_BT_CONN_TX_MAX you should have more but the CONFIG_BT_CTLR_RX_BUFFERS is limited to 18.
I found that what slowing down the rate in the original centralhr example was the printk inside notify_func() that causing the central to NACK the packet from the peripheral.

In my test with my Galaxy S10 it achieved similar number of packets.

Please try the example I provided here and if it works please follow the configuration in my projects.

devzone.nordicsemi.com/.../3264.hr_5F00_mp.zip

devzone.nordicsemi.com/.../4745.sniffertrace2.pcapng

RE: Sending multiple notify packets per connection interval

Neil_CPD — Mon, 16 Aug 2021 20:17:27 GMT

I've created the same issue that i've been having with a basic hr example using NCS 1.6.1. in the 1st example I loop round 3 packets then sleep 5ms the other I sleep 30ms. See attached 2 sniffer traces. You can see in the 4ms trace it sends two packets per connection interval to one with 30ms sleep it sends 3 per con interval.devzone.nordicsemi.com/.../hr_5F00_example_5F00_3packetLoop30ms_5F00_sleep.pcapng devzone.nordicsemi.com/.../hr_5F00_example_5F00_3packetLoop5ms_5F00_sleep.pcapng

RE: Sending multiple notify packets per connection interval

Hung Bui — Mon, 16 Aug 2021 12:22:54 GMT

There is a chance that it's hitting the limit of the stack. But we do need a sniffer trace to really understand the situation.
Please also look at the throughput example. As far as I know you can reach around 700kbps at 1Mbps.given that the ATT_MTU is configured correctly and Data length extension is applied.

RE: Sending multiple notify packets per connection interval

Neil_CPD — Mon, 16 Aug 2021 09:56:14 GMT

Hi,

I will try to get you a sniifer trace but I do remember the more data flag being set to false on the second packet.

I also thought that Simon had tried this test with two nrf dk's. Could you do a sniifer trace of it working with an nrf dk central please. If it's a limitation of the central then I can explain this to my customer.

Thanks

RE: Sending multiple notify packets per connection interval

Hung Bui — Mon, 16 Aug 2021 09:14:20 GMT

Hi Neil,
I suspect that the central was not able to handle more than 2 packets per connection event. A sniffer trace would be very helpful to reveal this.

RE: Sending multiple notify packets per connection interval

Neil_CPD — Fri, 13 Aug 2021 15:37:43 GMT

Hi Hung,

Please bear with me as i'm away from my desk for a couple of days but basically my setup was:

NCS 1.6.0 peripheral_uart example with the connection set to 15ms.

Central was my phone Sony XQ-AS52 with Android 11

while(1)

{

loop 3 times

{

bt_gatt_notify_cb(.....)

}

k_sleep( 30 ms) or k_sleep( 5 ms)

}

when I wait 30 ms I see 3 packets on the sniffer per.. but If I reduce the delay so it's every interval only 2 packets.

I'm not going to be able to send an nrf sniffer output for a week or so, but the code is quite basic.

RE: Sending multiple notify packets per connection interval

Hung Bui — Fri, 13 Aug 2021 14:47:53 GMT

Hi again Neil,

Could you send a sniffer trace ?
Could you show how you send the notification in your code ? With and without the delay.

Which firmware you use on the central side ?

Would be nice if you can provide some minimal code so that I can test here (on both sides)

RE: Sending multiple notify packets per connection interval

Neil_CPD — Fri, 13 Aug 2021 13:38:40 GMT

Hi Hung Bui,

Thank you for picking this up. The ultimate aim is to send160 bytes per notification packet three times per 15ms connection interval and this needs to happen every connection interval untill the user stops the device.

I can send two packets but not three, I don't get any errors returned.

As mentioned above Simon and I have used the uart example to test this. I can send 3 notifications in a loop within one connection interval if I wait for a delay of at least one connection interval in-between. Once a shorter delay is used it reverts to only two packets for connection interval. I've tested this using the nrf sniffer. If you need any more explanation please ask.

Kind regards

Neil

RE: Sending multiple notify packets per connection interval

Hung Bui — Fri, 13 Aug 2021 12:06:57 GMT

Hi Neil,
Simon is on vacation so I'm taking over the case.

Could you please explain a little bit more on what you did to " have to sleep for 30ms in the thread to enable this to happen, "

If you keep calling the notification send which error do you have ?

Note that when you change the buffer configuration make sure you reflash the network core. This is not done automatically if you flash using SES.

Do you need to send notification in separate packets ? If you always send 3 notifications per connection interval , can you combine them into one notification with longer payload ? If you have control over both sides of the connection , you can assign a larger ATT_MTU and update the data length to get bigger radio packet.

RE: Sending multiple notify packets per connection interval

Neil_CPD — Tue, 03 Aug 2021 09:30:41 GMT

Hi Simon,

Have you heard any more from the team? Are there any early indications on where the limitations are?

Thanks

RE: Sending multiple notify packets per connection interval

Simon — Fri, 30 Jul 2021 11:21:32 GMT

I tried to increase the buffers in both the ble host as well as the controller, but it didn't help. I've asked internally about this and waiting for an answer.

RE: Sending multiple notify packets per connection interval

Neil_CPD — Wed, 28 Jul 2021 13:45:05 GMT

Thanks for the update Simon.