Fast notifications with nRF5340 and nRF Connect SDK

[Jason Wright]

Board: nRF5340dk

SDK version: v2.1.0

My goal is to transmit 240 bytes of data every 5 ms using GATT notifications and 1 Mb PHY.

In the attached sample, I have my dev kit set up to notify on a characteristic every 5 ms. The connection interval is set to 10 ms, the MTU to 247 bytes, and the data length to 251 bytes. My thought is to fit 240 bytes into a single PDU, and send 2 notification packets per connection interval. I also toggle GPIO P0.04 before and after calling `bt_gatt_notify_cb`.

Looking at a logic analyzer trace of P0.04, I can see that while this setup initially works, eventually `bt_gatt_notify_cb` starts blocking for longer than expected. This happens around 14.5 seconds into the attached capture.

Wireshark capture also attached.

Why is `bt_gatt_notify_cb` blocking this long, and do you have any suggestions to improve the setup?

fast_notifications.zip

[Edvin]

Hello,

bt_gatt_notify_cb() is not a blocking call, so it should not be the reason why the P0.04 isn't toggling. 

Are you sure that bt_gatt_notify_cb() is even called at 14.5 seconds into the trace? The trace doesn't show any notifications being sent around that time. In fact, it doesn't look like any notifications are sent at all in the connection starting at packet no. 7517 (12.89 seconds into the sniffer trace).

Are you sure bt_gatt_notify_cb() is called at all? And if it is, what does it return? I assume it doesn't return 0, since no notifications are actually sent. 

Best regards,

Edvin

[Benno Trutmann]

Hi all.

For me bt_gatt_notify_cb() does block as well.

I try to implement a flow control with the help of the "Notification Value callback" func in struct bt_gatt_notify_params. But for that to work bt_gatt_notify_cb() should  return an error code indicating an out of buffer condition. Whereas in my test bt_gatt_notify_cb() only returns 0.

I use a connection interval of 500ms, the first three calls to bt_gatt_notify_cb() return immediately with return value of 0, but the forth blocks for almost 500ms and then returns also with the value 0.

I too thought that bt_gatt_notify_cb() is non-blocking but the behavior is blocking!

Do I miss some Zephyr BLE config option?

Regards,

Benno

[Benno Trutmann]

Hi Edvin.

Here is the call stack (keep in mind that conn_tx_alloc() is only called when func in struct bt_gatt_notify_params is set to a callback):

bt_gatt_notify_cb(params->func!=NULL) -> gatt_notify(params->func!=NULL) ->

bt_att_send() -> att_send_process() -> process_queue() -> bt_att_chan_send() -> chan_send() ->

bt_l2cap_send_cb(cb!=NULL) ->

bt_conn_send_cb(cb!=NULL) -> conn_tx_alloc() ->

k_fifo_get(&free_tx, K_FOREVER)

Regards,

Benno

[Edvin]

Thank you very much Benno.

Are you using the Zephyr Bluetooth Low Energy controller?

Did you set CONFIG_BT_LL_SPLIT=y in your prj.conf (or any other .conf file that is included in your application)?

I believe the reason I struggled to find it is that if you use the Nordic Bluetooth Low Energy controller (SoftDevice controller), which is either set by default, or if you use CONFIG_BT_LL_SOFTDEVICE=y, then the implementation of bt_gatt_notify_cb() that will be used is found in bt_rpc_gatt_client.c (ncs\nrf\subsys\bluetooth\rpc\client\bt_rpc_gatt_client.c), and it looks like this:

int bt_gatt_notify_cb(struct bt_conn *conn,
		      struct bt_gatt_notify_params *params)
{
	struct nrf_rpc_cbor_ctx ctx;
	int result;
	size_t scratchpad_size = 0;
	size_t buffer_size_max = 8;

	buffer_size_max += bt_gatt_notify_params_buf_size(params);

	scratchpad_size += bt_gatt_notify_params_sp_size(params);

	NRF_RPC_CBOR_ALLOC(&bt_rpc_grp, ctx, buffer_size_max);
	ser_encode_uint(&ctx, scratchpad_size);

	bt_rpc_encode_bt_conn(&ctx, conn);
	bt_gatt_notify_params_enc(&ctx, params);

	nrf_rpc_cbor_cmd_no_err(&bt_rpc_grp, BT_GATT_NOTIFY_CB_RPC_CMD,
		&ctx, ser_rsp_decode_i32, &result);

	return result;
}

It doesn't call gatt_notify(). Do you see the same blocking behavior if you try to set CONFIG_BT_LL_SOFTDEVICE=y in your prj.conf?

Best regards,

Edvin

[Benno Trutmann]

The controller part was not configured by me, so I do not really know what I am talking about. But I was under the impression that I use the Softdevice controller, but via "HCI using RPMsg" (CONFIG_BT_RPMSG=y).

When I look into the .config file in the app build directory I see no definition of CONFIG_BT_LL_SPLIT nor CONFIG_BT_LL_SOFTDEVICE.

In the .config file of the hci_rpmsg build directory I see

# CONFIG_BT_LL_SW_SPLIT is not set

and

CONFIG_BT_LL_SOFTDEVICE=y

Regards,

Benno

[Edvin]

Hello Benno,

I am still struggling to get the same behavior as you do. Can you please send me a project with the configuration you are using that I can use to reproduce the issue? Are you able to reproduce it e.g. with something based on the peripheral_uart sample? 

BR,
Edvin

[Benno Trutmann]

Hi Edvin.

I have no sample code at hand to demonstrate the blocking behavior. At the moment I also have no time to build one.

Are your tests based on a configuration with the Zephyr Host and the Softdevice Controller as in my case?

By the way: I use the version 2.1.1 of the Connect SDK.

Regards,

Benno

[Edvin]

Hello Benno, 

Can you please explain what you mean by "configuration with the Zephyr Host and the Softdevice Controller"? How do you set the Zephyr Host when you are using the SoftDevice Controller?

BR,
Edvin

[Benno Trutmann]

Hi Edvin.

To quote the nRF Connect SDK documentation:

"The SoftDevice Controller is implemented by Nordic Semiconductor. The Zephyr Bluetooth LE Controller is an open source Link Layer developed in Zephyr, to which Nordic Semiconductor is a contributor.

Both Link Layers integrate with the Zephyr Bluetooth Host, which completes the full Bluetooth LE protocol stack solution in the nRF Connect SDK. It is possible to select either Bluetooth LE Controller for application development. See Usage in samples for more information."

So with both controllers the Zephyr Bluetooth Host is used, the code for this host is contained in the 'zephyr-ncs\subsys\bluetooth\host' directory. And there bt_gatt_notify_cb() is implemented in gatt.c and calls indirectly conn_tx_alloc() from conn.c in the same directory.

For me it is unclear why you cannot see this call stack in your tests. You mention the function bt_gatt_notify_cb() from "ncs\nrf\subsys\bluetooth\rpc\client\bt_rpc_gatt_client.c", but I don't think this is part of the Zephyr Bluetooth Host.

Regards,

Benno

[Benno Trutmann]

Is it possible that you build your test code with CONFIG_BT_RPC=y "Bluetooth over RPC [EXPERIMENTAL]"?

In this case I see inclusion of code from the "ncs\nrf\subsys\bluetooth\rpc' directory. My project does not set CONFIG_BT_RPC!

[Edvin]

For testing, I used the peripheral_uart sample found in ncs\nrf\samples\bluetooth\peripheral_uart. 

According to my build folders, CONFIG_BT_RPC is not set. I didn't find any mentions of it in the autoconf.h, but searching for it in my build folder, these were all the hits:

BR,
Edvin

[Benno Trutmann]

Hi Edvin.

In this case your stack configuration should be the same as mine. But then it is unclear to my why your code calls the bt_gatt_notify_cb() from bt_rpc_gatt_client.c and not from gatt.c.

Can you verify in zephyr.map from where bt_gatt_notify_cb() is really linked?

In my zephyr.map I see:

.text.bt_gatt_notify_cb
                0x000478d0       0xe8 zephyr/subsys/bluetooth/host/libsubsys__bluetooth__host.a(gatt.c.obj)
                0x000478d0                bt_gatt_notify_cb