Questions regarding BLE notifications sending in loop

[fr_adv]

Dear Nordic support,

I've done some testing to send many data in BLE notifications, and I have a question regarding the BLE stack implementation. We are using NCS v2.6.1, with the soft device controller.

I have this piece of code running in a thread with higher prioritity than the main thread:

for (auto i = 0U; i < 100; i++) {
    LOG_DBG("TX notification (%u)", i);
    if (auto ret = bt_gatt_notify_uuid(conn, BT_UUID_TEST_CHARAC, attr_service, buffer, len)) {
        LOG_ERR("Cannot send BLE notif (%d)", ret);
        break;
    }
}

I did not imagine this would work properly, and thought that the error log would pop up or the watchdog would trigger a reset since it's refreshed in the main thread running at lower priority.

Especially with this configuration :

BT_L2CAP_TX_BUF_COUNT=3
BT_BUF_ACL_TX_COUNT=3

But I was surprised to see it works fine, sometimes with delay between the notifications: 

[08:29:38.141] [51792.625885] <dbg> test: TX notification (0)
[08:29:38.151] [51792.626678] <dbg> test: TX notification (1)
[08:29:38.162] [51792.627410] <dbg> test: TX notification (2)
[08:29:38.172] [51792.628173] <dbg> test: TX notification (3)
== 5ms gap here 
[08:29:38.182] [51792.673736] <dbg> test: TX notification (4)
== 47ms gap here
[08:29:38.404] [51792.720275] <dbg> test: TX notification (5)
[08:29:38.414] [51792.722473] <dbg> test: TX notification (6)
== 47ms gap here
[08:29:38.424] [51792.769012] <dbg> test: TX notification (7)
== 99ms gap here
[08:29:38.434] [51792.868743] <dbg> test: TX notification (8)
[08:29:38.444] [51792.915252] <dbg> test: TX notification (9)
[08:29:38.457] [51792.917480] <dbg> test: TX notification (10)
[08:29:38.467] [51792.963989] <dbg> test: TX notification (11)
[08:29:38.477] [51792.966247] <dbg> test: TX notification (12)
[08:29:38.598] [51793.012756] <dbg> test: TX notification (13)
[08:29:38.609] [51793.014984] <dbg> test: TX notification (14)
[08:29:38.619] [51793.061492] <dbg> test: TX notification (15)
[08:29:38.629] [51793.063751] <dbg> test: TX notification (16)
[08:29:38.640] [51793.110260] <dbg> test: TX notification (17)
[08:29:38.652] [51793.112487] <dbg> test: TX notification (18)
[08:29:38.662] [51793.158996] <dbg> test: TX notification (19)
[08:29:38.673] [51793.161254] <dbg> test: TX notification (20)

It the thread running the while loop suspended / interrupted / put in 'waiting' mode until a buffer is released in the BLE stack ? This is why lower priority threads can run in the meantime ?

Thanks in advance for your explanations, I'm pretty sure other users will be interested by your feedback.

Best regards

Aurélien

[Susheel Nuguru]

When you call bt_gatt_notify_uuid in a loop, the BLE stack tries to send out each notification using a set of buffers it has for data transmission. Now, you’ve got only a few of these buffers (BT_L2CAP_TX_BUF_COUNT=3 and BT_BUF_ACL_TX_COUNT=3), and they fill up pretty quickly. But the thing is, the BLE stack doesn’t just freeze your thread when all the buffers are in use. Instead, if there are no free buffers, the bt_gatt_notify_uuid call will just return an error, basically saying, “Hey, I’m out of space to send more data right now.”

You’d think that running this in a high-priority thread would hog all the CPU or that it would trigger the watchdog, but that’s not the case. Here’s why:

1. The BLE stack works asynchronously. It sends out the data over the air, and once a buffer is freed up (when the notification is sent), it becomes available for the next notification. This means your loop isn’t blocking everything — it just waits until there’s space again.
2. Zephyr, the real-time operating system you’re using, is pretty clever. Even though your thread has a higher priority, the system uses preemptive multitasking. If your thread can’t do much because it’s waiting for buffers to free up, the system lets lower-priority threads (like your main thread that keeps the watchdog happy) run in the meantime. So, your main thread can still do its job, like resetting the watchdog, and everything stays stable.

Sometimes you’ll notice delays between notifications, and that’s because the BLE stack needs time to send out the data and free up buffers. Your loop doesn’t explicitly wait or yield, but the BLE stack and Zephyr handle this behind the scenes, keeping things from locking up. It is not very straightforward to explain the exact delays as it depends on many things as timings of the application thread and isr context/priorities/pre and post processing. All of these influence the times.

[fr_adv]

Hello M. Nuguru,

Thanks for the explanations. Just remains a questionable point about this statement :

"Instead, if there are no free buffers, the bt_gatt_notify_uuid call will just return an error, basically saying, “Hey, I’m out of space to send more data right now.” "

I think this is contradiction to the next paragraph about the BLE stack and Zephyr. Moreover, I don't see any error log with the snippet in my previous comment. 

The delays are not an issue in my case, it's perfectly clear to me that the data shall be actually sent on the radio link, and acknowledged by the BLE peer (link layer + L2CAP I guess) before filling up BLE stack buffers again and sending more notifications.

Thanks again for your feedback
Best regards

Aurélien