nRF5340 BLE Transmission Stack Buffer Overflow

Hi,

We are developing a mesh network. Our requirement is the one device can connect to three other devices. Using one connection as a peripheral and up to two connections as a central for mesh connections.

We are using nRF5340 DK and the nRF Connect SDK Version 2.6.1.

I am using the function bt_gatt_write_without_response_cb(). The stack buffer size is set with CONFIG_BT_BUF_ACL_TX_COUNT=30.

The CONFIG_BT_BUF_ACL_TX_COUNT is set to 30, allowing 10 buffers for each connection. I'm using a write complete callback to monitor buffer usage. If the buffer usage for any connection exceeds 10 due to transmission issues, I stop writing to that connection until it is cleared.

If one device connects to two other devices, with one being very close and the other being far away and out of range, the stack buffer becomes full for both connections, even for the nearby device. Why does this issue occur, causing difficulty in sending packets to the nearby device when the other device goes out of range?

Parents
  • Hi, 

    If one device connects to two other devices, with one being very close and the other being far away and out of range, the stack buffer becomes full for both connections, even for the nearby device. Why does this issue occur, causing difficulty in sending packets to the nearby device when the other device goes out of range?

    When a device is out of range, the BLE stack might still try to resend the data, causing the stack's buffers to fill up. This can prevent new data from being queued for transmission, even for other devices that are within range. 

    Regards,
    Amanda H.

  • Hi,

    Thanks for the response.

    Our project currently runs on the nRF52840 with the nRF5 SDK, and we haven't encountered this issue. We are transitioning to the nRF5340 with the NCS to achieve better performance.

    However, this buffer overflow issue is significantly affecting our device's performance. Since this is a fundamental problem, could you address it on your end? Additionally, can we expect a solution in the next NCS release?

  • Hi, 

    The team said the trace isn't that helpful sadly, and thinks your assumptions are wrong concerning how CONFIG_BT_BUF_ACL_TX_COUNT work. CONFIG_BT_BUF_ACL_TX_COUNT doesn't give 10 buffers per connection because the buffers are shared between all the connections.

    You could try to increase the buffers in the Controller by setting CONFIG_BT_BUF_ACL_RX_COUNT

    -Amanda H.

  • Hi

    CONFIG_BT_BUF_ACL_TX_COUNT doesn't give 10 buffers per connection because the buffers are shared between all the connections.

    I understand that CONFIG_BT_BUF_ACL_TX_COUNT is shared among all connections, so I have set its size to 30. As mentioned earlier, in the application layer, I will write up to 10 for each connection.

    You could try to increase the buffers in the Controller by setting CONFIG_BT_BUF_ACL_RX_COUNT.

    I have tried to increase the CONFIG_BT_BUF_ACL_RX_COUNT up to 30 but the issue is still there.

    TX[0] BufferFull:0, ConnHandle:2, InIndex:270, OutIndex:270, StackBufferUserd:0
    TX[1] BufferFull:0, ConnHandle:0, InIndex:232, OutIndex:232, StackBufferUserd:0
    writeCmdTxCmpltCallback index:1 ConnHandle:0, OutIndex:233
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:271
    TX[0] BufferFull:0, ConnHandle:2, InIndex:271, OutIndex:271, StackBufferUserd:0
    TX[1] BufferFull:0, ConnHandle:0, InIndex:233, OutIndex:233, StackBufferUserd:0
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:272
    writeCmdTxCmpltCallback index:1 ConnHandle:0, OutIndex:234
    TX[0] BufferFull:0, ConnHandle:2, InIndex:272, OutIndex:272, StackBufferUserd:0
    TX[1] BufferFull:0, ConnHandle:0, InIndex:234, OutIndex:234, StackBufferUserd:0
    writeCmdTxCmpltCallback index:1 ConnHandle:0, OutIndex:235
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:273
    TX[0] BufferFull:0, ConnHandle:2, InIndex:273, OutIndex:273, StackBufferUserd:0
    TX[1] BufferFull:0, ConnHandle:0, InIndex:235, OutIndex:235, StackBufferUserd:0
    writeCmdTxCmpltCallback index:1 ConnHandle:0, OutIndex:236
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:274
    TX[0] BufferFull:0, ConnHandle:2, InIndex:274, OutIndex:274, StackBufferUserd:0
    TX[1] BufferFull:0, ConnHandle:0, InIndex:236, OutIndex:236, StackBufferUserd:0
    writeCmdTxCmpltCallback index:1 ConnHandle:0, OutIndex:237
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:275
    TX[0] BufferFull:0, ConnHandle:2, InIndex:275, OutIndex:275, StackBufferUserd:0
    TX[1] BufferFull:0, ConnHandle:0, InIndex:237, OutIndex:237, StackBufferUserd:0
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:276
    TX[0] BufferFull:0, ConnHandle:2, InIndex:276, OutIndex:276, StackBufferUserd:0
    TX[1] BufferFull:0, ConnHandle:0, InIndex:238, OutIndex:237, StackBufferUserd:1
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:277
    TX[0] BufferFull:0, ConnHandle:2, InIndex:277, OutIndex:277, StackBufferUserd:0
    TX[1] BufferFull:0, ConnHandle:0, InIndex:239, OutIndex:237, StackBufferUserd:2
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:278
    TX[0] BufferFull:0, ConnHandle:2, InIndex:278, OutIndex:278, StackBufferUserd:0
    TX[1] BufferFull:0, ConnHandle:0, InIndex:240, OutIndex:237, StackBufferUserd:3
    TX[0] BufferFull:0, ConnHandle:2, InIndex:279, OutIndex:278, StackBufferUserd:1
    TX[1] BufferFull:0, ConnHandle:0, InIndex:241, OutIndex:237, StackBufferUserd:4
    TX[0] BufferFull:0, ConnHandle:2, InIndex:280, OutIndex:278, StackBufferUserd:2
    TX[1] BufferFull:0, ConnHandle:0, InIndex:242, OutIndex:237, StackBufferUserd:5
    TX[0] BufferFull:0, ConnHandle:2, InIndex:281, OutIndex:278, StackBufferUserd:3
    TX[1] BufferFull:0, ConnHandle:0, InIndex:243, OutIndex:237, StackBufferUserd:6
    TX[0] BufferFull:0, ConnHandle:2, InIndex:282, OutIndex:278, StackBufferUserd:4
    TX[1] BufferFull:0, ConnHandle:0, InIndex:244, OutIndex:237, StackBufferUserd:7
    TX[0] BufferFull:0, ConnHandle:2, InIndex:283, OutIndex:278, StackBufferUserd:5
    TX[1] BufferFull:0, ConnHandle:0, InIndex:245, OutIndex:237, StackBufferUserd:8
    TX[0] BufferFull:0, ConnHandle:2, InIndex:284, OutIndex:278, StackBufferUserd:6
    TX[1] BufferFull:0, ConnHandle:0, InIndex:246, OutIndex:237, StackBufferUserd:9
    TX[0] BufferFull:0, ConnHandle:2, InIndex:285, OutIndex:278, StackBufferUserd:7
    TX[1] BufferFull:1, ConnHandle:0, InIndex:247, OutIndex:237, StackBufferUserd:10
    TX[0] BufferFull:0, ConnHandle:2, InIndex:286, OutIndex:278, StackBufferUserd:8
    TX[1] BufferFull:1, ConnHandle:0, InIndex:247, OutIndex:237, StackBufferUserd:10
    TX[0] BufferFull:0, ConnHandle:2, InIndex:287, OutIndex:278, StackBufferUserd:9
    TX[1] BufferFull:1, ConnHandle:0, InIndex:247, OutIndex:237, StackBufferUserd:10
    TX[0] BufferFull:1, ConnHandle:2, InIndex:288, OutIndex:278, StackBufferUserd:10
    TX[1] BufferFull:1, ConnHandle:0, InIndex:247, OutIndex:237, StackBufferUserd:10
    TX[0] BufferFull:1, ConnHandle:2, InIndex:288, OutIndex:278, StackBufferUserd:10
    TX[1] BufferFull:1, ConnHandle:0, InIndex:247, OutIndex:237, StackBufferUserd:10
    
    ####### DISCONNECTED conn_handle:0 for Reason: 8 #######
    
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:279
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:280
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:281
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:282
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:283
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:284
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:285
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:286
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:287
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:288
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:289
    writeCmdTxCmpltCallback index:0 ConnHandle:2, OutIndex:290

    I have attached the relevant portion of the application logs. The TX[0] print occurs before sending, and the write complete callback appears when the write complete callback event is received.

    You can see that the write complete callback is received for two connections.

    However, after the connection handle 0 goes out of range, the write complete callback does not come for connection handle 2, even though connection handle 2 is nearby. Once connection handle 0 disconnects, the write complete callback for connection handle 2 comes through.

    The issue is that I am unable to write to the stack buffer for connection handle 2 until the connection handle 0 disconnects.

  • Could you post the full device (app core) logs with:

    CONFIG_BT_GATT_LOG_LEVEL_DBG=y
    CONFIG_BT_ATT_LOG_LEVEL_DBG=y
    CONFIG_BT_L2CAP_LOG_LEVEL_DBG=y
    

    Would it also be possible to run your app on a 52840 using NCS to rule out misconfiguration issues between the two 5340 cores?

  • Hi,

    Please find the attached log with enabled log.

    CONFIG_BT_GATT_LOG_LEVEL_DBG=y
    CONFIG_BT_ATT_LOG_LEVEL_DBG=y
    CONFIG_BT_L2CAP_LOG_LEVEL_DBG=y

    nRF5340_MultiConnection_BufferOverflow_issue.txt

  • Hi,

    Were you able to verify the log? Are there any updates?

Reply Children
Related