BLE notification takes too long

[Natan ALLARD]

Hello everyone,

I'm developing an application for an nrf52840 with the nrf connect VS Code extension.

The application runs three threads. One to acquire 40 bytes of data from a sensor at a frequency of 400Hz, another to process those data, and a third to send the result through BLE notifications.

It is important to note that I shouldn't lose the synchronization with the sensor for the processing but I can send the processed data slower. Therefore I configured the BLE like this : 

CONFIG_BT=y
CONFIG_BT_DEBUG_LOG=y
CONFIG_BT_PERIPHERAL=y

CONFIG_BT_USER_DATA_LEN_UPDATE=y

CONFIG_BT_ATT_PREPARE_COUNT=9
CONFIG_BT_BUF_ACL_TX_COUNT=9
CONFIG_BT_BUF_ACL_RX_SIZE=47
CONFIG_BT_CTLR_DATA_LENGTH_MAX=47
CONFIG_BT_L2CAP_TX_MTU=47
CONFIG_BT_BUF_ACL_TX_SIZE=47
CONFIG_BT_CTLR_SDC_MAX_CONN_EVENT_LEN_DEFAULT=7812

If my calculations are good, it should take 868µs to send a packet. But when I measure the time taken by the bt_gatt_notify it is around 20ms. 

This slows all of my processes and therefore makes the FIFOs which are in charge of passing the data from one thread to another filled very quickly.

The FIFO is filled like this : 

size_t size = sizeof(struct raw_tx_buf);
char *raw_mem_ptr = k_malloc(size);
__ASSERT_NO_MSG(raw_mem_ptr != 0);

memcpy(raw_mem_ptr, &raw_tx, size);

k_fifo_put(&raw_tx_fifo, raw_mem_ptr);

And read like this : 

struct raw_tx_buf *raw_buf = k_fifo_get(&raw_tx_fifo,K_FOREVER);

bt_gatt_notify(NULL,&mcd_svc.attrs[2],raw->raw_data,sizeof(raw->raw_data);

k_free(raw_buf);

If I can't make the notification function go faster, how can I prevent the overfilling of the FIFO which makes everything crash ?

Thanks in advance for your answer.

[Karl Ylvisaker]

Hello,

If my calculations are good, it should take 868µs to send a packet. But when I measure the time taken by the bt_gatt_notify it is around 20ms. 

To easily verify the on-air time it will take to send a packet you can use the Online Power Profiler, for instance :) 
The reason why you are seeing 20 ms here is likely that it is the Connection Interval used for the connection where the data is sent.
The shortest possible Connection Interval for BLE communication is 7.5 ms, which would mean that you on average will send 120 bytes every connection event (7.5 ms).
Your buffer will need to be able to store this, and potentially some additional packets of this size in case multiple packets of this size suddenly is sent in a single connection event due to a previous packet loss, for instance. 
How large is your receiving buffer currently?
Is this a central or peripheral device you are developing?

Best regards,
Karl

[Natan ALLARD]

Hello, thank you for your answer.

I thought I configured the connection interval to be 7812us (which I believe is the time to let 9 packet be exchanged on each connection event) with CONFIG_BT_CTLR_SDC_MAX_CONN_EVENT_LEN_DEFAULT. Is it the right way to do it ?

My device is a peripheral and I use the Nordic connect mobile app as a central. Therefore I don't know the size of the app buffer. 

Best regards,

Natan 

[Natan ALLARD]

Yes, from what I understood when the FIFO is read, the data is supposed to be removed from it. And after sending the data I use k_free to free the memory of the allocated before with k_malloc.

So after sending there shouldn't be data anymore on the peripheral.

[Natan ALLARD]

I don't manage to get any error code printed even though I have indeed activated the logging with

CONFIG_LOG=y
CONFIG_LOG_PRINTK=y
CONFIG_DEBUG_THREAD_INFO=y

CONFIG_BT=y
CONFIG_BT_DEBUG_LOG=y

[Karl Ylvisaker]

Natan ALLARD said:

I don't manage to get any error code printed even though I have indeed activated the logging with

Is it just the error code that is missing from the logs, or are you not seeing any logs at all?

Natan ALLARD said:

So after sending there shouldn't be data anymore on the peripheral.

Yes, but in the case of a transmission that is not acknowledged (such as if a packet is dropped) then the data is not deleted until the packet is acknowledged by the peer.
So the device will keep trying to send the same packet until it either succeeds, or the connection times out.
In the latter case it will keep trying to send the same packet until the Connection Supervision Timeout occurs, which in the mean time would have filled up the ring buffer with unsent sensor samples.

Best regards,
Karl

[Natan ALLARD]

Thank you for these information.

I am not seeing any log at all when it crashes.

And I think I understand but what can I do to fix this?

[Karl Ylvisaker]

Natan ALLARD said:

I am not seeing any log at all when it crashes.

Yes, but are you seeing logs prior to the crash? Are you seeing any logs at all?

Natan ALLARD said:

And I think I understand but what can I do to fix this?

The best approach to this depends on how important it is whether every sensor measurement makes it across to the peer or not - will the sensor data ever time-out, or stop being useful to the central, or should all data be sent in consecutive order and if it can not then the connection should be terminated?
In case of the latter I would recommend implementing a ring buffer as mentioned in my previous comments.

Best regards,
Karl

[Natan ALLARD]

Yes sorry If I wasn't clear. The I can see logs prior to the crash. When i activate the notification it sends some of the data (I know because I see it on the nrf mobile app and because I print logs) then it crashes.

I need to send them in a consecutive order. I understand the ring buffer but isn't it similar to the FIFO I implemented ?

Thank you,

[Karl Ylvisaker]

Natan ALLARD said:

Yes sorry If I wasn't clear. The I can see logs prior to the crash. When i activate the notification it sends some of the data (I know because I see it on the nrf mobile app and because I print logs) then it crashes.

No need to apologize, I will just ask if anything is unclear :) 

Great to know that you are able to see logs up until the crash. Have you added any local error handling to the notify function call?
For starters, you could add some logging to alert you in the case that the notify function call fails.

Natan ALLARD said:

I understand the ring buffer but isn't it similar to the FIFO I implemented ?

Yes, the difference between the ring buffer and the FIFO is that the ring buffer will start to overwrite itself instead of overflowing, in the case that data is starting to back up significantly. This way, you could avoid crashing due to the buffer overflowing, for instance.

Natan ALLARD said:

I need to send them in a consecutive order.

Thank you for clarifying. Does the data ever 'time out'? I.e is there ever a time threshold where 'old' data no longer will be useful?
If so, then that threshold could be used as a natural limitation on the FIFO buffer size.

Best regards,
Karl

[Natan ALLARD]

Thank you for your help and all the information you gave me.

I finally managed to fix my problem by changing the way I put the data in the FIFO, instead of using k_malloc I used k_fifo_alloc_put and it seems to be fixed.

In any case everything you told me was very helpful to optimize the application.

Best regards 

[Karl Ylvisaker]

Natan ALLARD said:

I finally managed to fix my problem by changing the way I put the data in the FIFO, instead of using k_malloc I used k_fifo_alloc_put and it seems to be fixed.

Great, I am happy to hear that you were able to resolve the issue! :) 

Natan ALLARD said:

Thank you for your help and all the information you gave me.

Natan ALLARD said:

In any case everything you told me was very helpful to optimize the application.

It is no problem at all, I am glad that you found the information helpful! :)

Please do not hesitate to open up another ticket if you should encounter any other issues or questions in the future.

Good luck with your development!

Best regards,
Karl