Radio Buffer's filling up faster then they should NRF_ERROR_RESOURCES on a sd_ble_gatts_hvx(..)

[Dave_couling]

I have a sensor which I'm grabbing FIFO data from every 14msec,  ~240bytes.   I send these FIFO grabs as soon as they arrive. 

My connection parameters:

Connection Interval: 10msec

MTU = 247

DL=251

hvn_tx_queue_size = 36 

event_length = tried 8 ~ 18

PHY = tested both 1Mbps and 2Mbps

According to the timing, the FIFO data grabs occur every 14msec, so under the Connection interval.  Therefore the Buffers should never fill up.  However after about 200 data sends I get the NRF_ERROR_RESOURCES error.  Is there a setting I'm missing, why would the Radio be throttling?  If the MTU is 247 and Interval 10msec, assuming single packet in a Connection Event it should support up to 24kB/sec (with 240byte payloads).  According to the FIFO data grab date it only needs ~17.2kB/sec.  So where am I missing something that would cause the link not to be able to support the Data Rate and hit Buffer FULL errors?    I do handle this by wait on the BLE_GATTS_EVT_HVN_TX_COMPLETE but I don't want to lose any incoming data.

I'm currently using an nRF52840, SDK 16, S140 v7.0.1

Thanks for the help guys,

DC

[Hung Bui]

Hi Dave, 

If there is any interference or other reason a packet get corrupted that requires retransmission it will cause the buffer to build up. 
This interference tends to happen easier on larger packet such as with the 240 bytes payload you are trying to send. 

Please try capture a sniffer trace so that we can have a closer look at what happens over the air. 

[Dave_couling]

Oddly I don't see this issue connecting to the nRF Connect app on a Samsung tablet.  However when using the PCA10059 Dongle I hit this issue almost right away.  When I get the NRF_ERROR_RESOURCES error I loop and wait for the BLE_GATTS_EVT_HVN_TX_COMPLETE flag to hit, inserting a BLE_interval/4 delay between loops.  Connecting to the Samsung tablet acts like a rockstar.  No issues.  However connecting to the nRF Connect SW via PCA10059 hits this issue rather quickly.  Looking at the Connection Settings and Parameters,  the PCA10059 should have higher-throughput but in practice doesn't seem to be the case.  In terms of proximity, both are within 30cm of our Application hardware which is just a PCA10056 DK.

Parameter Connecting to PCA10059

• Interval 7msec
• MTU 247
• DLE 251
• max rx octets 251
• max rx time usec 2120
• max tx octets 251
• max tx time usec 2120

Parameters Connected to Samsung Tablet

• Interval 11msec
• MTU 247
• DLE 123
• max rx octets 123
• max rx time usec 1096
• max tx octets 123
• max tx time usec 1096

[Dave_couling]

My real concern is that when connecting to the PCA10059 Dongle, once I hit a NRF_ERROR_RESOURCES, it never gets a BLE_GATTS_EVT_HVN_TX_COMPLETE.  For example in an interval of 7.5msec,  when I first hit a NRF_ERROR_RESOURCES, I set a volatile boolean flag and enter a loop which delays exactly one connection interval.  The boolean flag clears when a BLE_GATTS_EVT_HVN_TX_COMPLETE event is received.  However I see the loop iterating up to 30 times ( 30 connection intervals ) without ever receiving the BLE_GATTS_EVT_HVN_TX_COMPLETE event.   Again this issue isn't seen connecting to a Samsung Tab 3 via nrf Connect app.

[Dave_couling]

Please find the attached sniffer traces.  The trace file connected to the PCA10059 runs until I see the NRF_ERROR_RESOURCES error and it waits 30 connection intervals before hitting a breakpoint.   The trace connected to the Samsung Tab3 never has any issues and doesn't hit a NRF_ERROR_RESOURCES error ever.

My Assistant Data-Thruput Capture -connected to PCA10059.pcapng

My Assistant Data-Thruput Capture -connected to Samsung Tab3.pcapng

[Dave_couling]

Looking at the PCA10059 Trace you can see once the Data Transmissions over the NUS Rx char start there is a Notification Send every 12~14msec as expected.  The payload should be 243bytes grabbed from an IMU FIFO.  However once I hit the first NRF_ERROR_RESOURCES @41.526 and enter my wait loop,  this loop iterates at the Connection Interval (7.5msec) and seems to keep resending the same payload.  Since my system never receives the BLE_GATTS_EVT_HVN_TX_COMPLETE event this loop spins forever.  So the underlying issue is eventually the BLE_GATTS_EVT_HVN_TX_COMPLETE event stops firing.   Could this be an issue due to using FreeRTOS?  I see that under the hood it uses Event Polling for Stack events rather than interrupts.  I manually set the SDH task priority to assure its the highest in the system (currently at 10).  Also note all my data sends are from Task context of a task with Priority 6 (lower than the SDH task).