Fast notifications with nRF5340 and nRF Connect SDK

[Jason Wright]

Board: nRF5340dk

SDK version: v2.1.0

My goal is to transmit 240 bytes of data every 5 ms using GATT notifications and 1 Mb PHY.

In the attached sample, I have my dev kit set up to notify on a characteristic every 5 ms. The connection interval is set to 10 ms, the MTU to 247 bytes, and the data length to 251 bytes. My thought is to fit 240 bytes into a single PDU, and send 2 notification packets per connection interval. I also toggle GPIO P0.04 before and after calling `bt_gatt_notify_cb`.

Looking at a logic analyzer trace of P0.04, I can see that while this setup initially works, eventually `bt_gatt_notify_cb` starts blocking for longer than expected. This happens around 14.5 seconds into the attached capture.

Wireshark capture also attached.

Why is `bt_gatt_notify_cb` blocking this long, and do you have any suggestions to improve the setup?

fast_notifications.zip

[Edvin]

Hello,

bt_gatt_notify_cb() is not a blocking call, so it should not be the reason why the P0.04 isn't toggling. 

Are you sure that bt_gatt_notify_cb() is even called at 14.5 seconds into the trace? The trace doesn't show any notifications being sent around that time. In fact, it doesn't look like any notifications are sent at all in the connection starting at packet no. 7517 (12.89 seconds into the sniffer trace).

Are you sure bt_gatt_notify_cb() is called at all? And if it is, what does it return? I assume it doesn't return 0, since no notifications are actually sent. 

Best regards,

Edvin

[Benno Trutmann]

Hi all.

For me bt_gatt_notify_cb() does block as well.

I try to implement a flow control with the help of the "Notification Value callback" func in struct bt_gatt_notify_params. But for that to work bt_gatt_notify_cb() should  return an error code indicating an out of buffer condition. Whereas in my test bt_gatt_notify_cb() only returns 0.

I use a connection interval of 500ms, the first three calls to bt_gatt_notify_cb() return immediately with return value of 0, but the forth blocks for almost 500ms and then returns also with the value 0.

I too thought that bt_gatt_notify_cb() is non-blocking but the behavior is blocking!

Do I miss some Zephyr BLE config option?

Regards,

Benno

[Edvin]

Hello Benno,

I am still struggling to get the same behavior as you do. Can you please send me a project with the configuration you are using that I can use to reproduce the issue? Are you able to reproduce it e.g. with something based on the peripheral_uart sample? 

BR,
Edvin

[Benno Trutmann]

Hi Edvin.

I have no sample code at hand to demonstrate the blocking behavior. At the moment I also have no time to build one.

Are your tests based on a configuration with the Zephyr Host and the Softdevice Controller as in my case?

By the way: I use the version 2.1.1 of the Connect SDK.

Regards,

Benno

[Edvin]

Hello Benno, 

Can you please explain what you mean by "configuration with the Zephyr Host and the Softdevice Controller"? How do you set the Zephyr Host when you are using the SoftDevice Controller?

BR,
Edvin

[Benno Trutmann]

Hi Edvin.

To quote the nRF Connect SDK documentation:

"The SoftDevice Controller is implemented by Nordic Semiconductor. The Zephyr Bluetooth LE Controller is an open source Link Layer developed in Zephyr, to which Nordic Semiconductor is a contributor.

Both Link Layers integrate with the Zephyr Bluetooth Host, which completes the full Bluetooth LE protocol stack solution in the nRF Connect SDK. It is possible to select either Bluetooth LE Controller for application development. See Usage in samples for more information."

So with both controllers the Zephyr Bluetooth Host is used, the code for this host is contained in the 'zephyr-ncs\subsys\bluetooth\host' directory. And there bt_gatt_notify_cb() is implemented in gatt.c and calls indirectly conn_tx_alloc() from conn.c in the same directory.

For me it is unclear why you cannot see this call stack in your tests. You mention the function bt_gatt_notify_cb() from "ncs\nrf\subsys\bluetooth\rpc\client\bt_rpc_gatt_client.c", but I don't think this is part of the Zephyr Bluetooth Host.

Regards,

Benno

[Benno Trutmann]

Is it possible that you build your test code with CONFIG_BT_RPC=y "Bluetooth over RPC [EXPERIMENTAL]"?

In this case I see inclusion of code from the "ncs\nrf\subsys\bluetooth\rpc' directory. My project does not set CONFIG_BT_RPC!

[Edvin]

For testing, I used the peripheral_uart sample found in ncs\nrf\samples\bluetooth\peripheral_uart. 

According to my build folders, CONFIG_BT_RPC is not set. I didn't find any mentions of it in the autoconf.h, but searching for it in my build folder, these were all the hits:

BR,
Edvin

[Benno Trutmann]

Hi Edvin.

In this case your stack configuration should be the same as mine. But then it is unclear to my why your code calls the bt_gatt_notify_cb() from bt_rpc_gatt_client.c and not from gatt.c.

Can you verify in zephyr.map from where bt_gatt_notify_cb() is really linked?

In my zephyr.map I see:

.text.bt_gatt_notify_cb
                0x000478d0       0xe8 zephyr/subsys/bluetooth/host/libsubsys__bluetooth__host.a(gatt.c.obj)
                0x000478d0                bt_gatt_notify_cb

[Edvin]

Hello,

Actually, it points to the same place. I just navigated using VS Code, and there it pointed to ncs\nrf\subsys\bluetooth\rpc\client\bt_rpc_gatt_client.c. I should have checked the .map file.

I believe the reason I couldn't reproduce it was that I just tried to call bt_nus_send() (bt_gatt_notify_cb()) directly from my main() function. I discussed this with a colleague, who recently had a similar case, which I think you should check out:

 bt_gatt_notify() can block when running on Zephyr workqueue. 

It turns out that it is the workqueue itself locking up. SInce the bt_gatt_notify_cb() is queued by the workqueue, but the workqueue is also required to actually process your BLE queue, you can end up in a deadlock. I guess the takeaway is that you should call bt_gatt_notify from another thread, so where you actually are calling bt_gatt_notify_cb() from now, you should rather trigger a task in another thread, (either the main thread, or a completely separate thread), so that it doesn't end up in a deadlock.

As it is now, this is currently a limitation in the Zephyr Bluetooth Host stack.

Best regards,

Edvin