BLE Throughput and Timing Issues

[blue_dew]

Hello, 

I am trying to create an application on the nRF5340 which samples data at 16kHz and packages + sends it over BLE. This requires a BLE throughput ~1Mbps, which I can get very easily when trying to throughput example application, but I can not achieve when communicating with my desktop application and a single nRF5340dk. My nRF53 chip is acting as the peripheral and sends  a notification using bt_gatt_notify() after a "sample_buf" is filled and packaged. I have ensured that my peripheral negotiates a connection interval of 7.5ms, the MTU size is increased, and I have increased the number of L2CAP buffers by settings the following configs:

# bluetooth 
CONFIG_BT_GATT_CLIENT=y
CONFIG_BT_PHY_UPDATE=y
CONFIG_BT=y
CONFIG_BT_RPMSG=y
CONFIG_BT_PERIPHERAL=y
CONFIG_BT_DEVICE_NAME="LiteBEST" 
CONFIG_BT_P1_SECURITY_ENABLED=y
CONFIG_BT_USER_PHY_UPDATE=y
CONFIG_BT_BUF_ACL_RX_SIZE=502
CONFIG_BT_L2CAP_TX_BUF_COUNT=20
CONFIG_BT_L2CAP_TX_MTU=498
CONFIG_BT_L2CAP_DYNAMIC_CHANNEL=y
CONFIG_BT_CONN_TX_MAX=20
CONFIG_BT_BUF_ACL_TX_COUNT=20
CONFIG_BT_BUF_ACL_TX_SIZE=502
CONFIG_BT_PERIPHERAL_PREF_MIN_INT=6
CONFIG_BT_PERIPHERAL_PREF_MAX_INT=6
#CONFIG_BT_PERIPHERAL_PREF_LATENCY=0
#CONFIG_BT_ATT_PREPARE_COUNT=4
#CONFIG_BT_PERIPHERAL_PREF_TIMEOUT=400
CONFIG_BT_GAP_AUTO_UPDATE_CONN_PARAMS=y
CONFIG_BT_CTLR_PHY_2M=y
CONFIG_BT_AUTO_PHY_UPDATE=n

# adding addtional BLE things from random nordic dev form posts
CONFIG_BT_DEBUG_LOG=y
CONFIG_BT_SMP=y
CONFIG_BT_ATT_PREPARE_COUNT=5
CONFIG_BT_CTLR_RX_BUFFERS=18
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_HEAP_MEM_POOL_SIZE=2048
#CONFIG_BT_CONN_TX_MAX=20
#CONFIG_BT_L2CAP_TX_BUF_COUNT=20
#CONFIG_BT_L2CAP_DYNAMIC_CHANNEL=y

# ads131m04
CONFIG_SPI=y
CONFIG_ABC=y
CONFIG_ABC_TRIGGER=y
CONFIG_ABC_TRIGGER_THREAD_PRIO=0
CONFIG_NRFX_SPIM4=y
CONFIG_ABC_SENSOR_LOG_LEVEL=0
CONFIG_ABC_SAMPLE_COUNT=20
CONFIG_ABC_FRAME_SIZE_BYTES=3

# LED stuff
CONFIG_LED_STRIP=y
CONFIG_LED_STRIP_LOG_LEVEL_DBG=y
CONFIG_WS2812_STRIP=y

# fuel gauge stuff
CONFIG_FUEL_GAUGE=y

# remove SWO from J-TAG
CONFIG_NRF_TRACE_PORT=n

# log stuff
CONFIG_LOG=y
CONFIG_SENSOR=y

# over clocking stuff
CONFIG_NRFX_CLOCK=y

# optimization stuff
CONFIG_SPEED_OPTIMIZATIONS=y
CONFIG_SIZE_OPTIMIZATIONS=y
#CONFIG_NO_OPTIMIZATIONS=y
CONFIG_MAIN_STACK_SIZE=4096

# gpio stuff
CONFIG_GPIO_NRFX=y
CONFIG_NRFX_TIMER0=y
CONFIG_NRFX_GPIOTE=y

# prototype stuff
CONFIG_RTT_CONSOLE=y
CONFIG_PRINTK=y
CONFIG_USE_SEGGER_RTT=y
CONFIG_LOG_BACKEND_RTT=y 
CONFIG_STDOUT_CONSOLE=y
CONFIG_CLOCK_CONTROL=y
CONFIG_SOC_HFXO_CAP_INTERNAL=y
CONFIG_SOC_HFXO_CAP_INT_VALUE_X2=27
CONFIG_LOG_MODE_MINIMAL=n
CONFIG_LOG_MODE_DEFERRED=y

The child image config is attached below:

#CONFIG_BT_LL_SOFTDEVICE=y
CONFIG_BT=y
CONFIG_BT_HCI_RAW=y
CONFIG_BT_HCI_RAW_RESERVE=1
CONFIG_BT_MAX_CONN=16
#CONFIG_BT_HCI=y
#CONFIG_BT_CTLR_ADVANCED_FEATURES=y

CONFIG_BT_CTLR_SDC_MAX_CONN_EVENT_LEN_DEFAULT=4000000
#CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
#CONFIG_BT_CTLR_TX_BUFFER_SIZE=251
CONFIG_BT_BUF_ACL_RX_SIZE=502
CONFIG_BT_BUF_ACL_TX_SIZE=502
#CONFIG_BT_CTLR_RX_BUFFERS=10
#CONFIG_BT_CTLR_TX_BUFFERS=10

CONFIG_IPC_SERVICE=y
CONFIG_MBOX=y
CONFIG_HEAP_MEM_POOL_SIZE=4096
CONFIG_MAIN_STACK_SIZE=1024
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=1024


CONFIG_BT_DEBUG_LOG=y
CONFIG_BT_SMP=y
CONFIG_BT_ATT_PREPARE_COUNT=5
CONFIG_BT_CTLR_RX_BUFFERS=18
#CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_BT_CTLR_DATA_LENGTH_MAX=502
CONFIG_HEAP_MEM_POOL_SIZE=2048


# Workaround: Unable to allocate command buffer when using K_NO_WAIT since
# Host number of completed commands does not follow normal flow control.
CONFIG_BT_BUF_CMD_TX_COUNT=10

# Enable and adjust the below value as necessary
CONFIG_BT_BUF_EVT_RX_COUNT=16
CONFIG_BT_L2CAP_TX_BUF_COUNT=10
CONFIG_BT_L2CAP_TX_MTU=498

I know that some config changes need to be copied to the child but I'm not sure which ones exactly so I tried to copy most of them over. 

I begin the application by advertising on the 1M physical layer and then updating the connection parameters manually to 2M PHY after a connection is made. I know that this works because of the nRF BLE Sniffer logs (see below). 

The desktop application is receiving 2000 packets (each is 244 bytes) within 17 seconds (~200kbps) but it is expected to receive it under 4 seconds if we're actually able to get close to the expected throughput of ~1Mbps. 

litebest_blelog.pcapng

This is the BLE sniffer log I was able to collect which clearly shows the PHY layer getting updated but I am struggling to fully understand what I'm seeing. I know that the BLE link layer guarantees delivery and therefore the master must reply even though notify doesn't require a reply from the Host layer but I think there is a problem with the reply from Master (desktop app) as seen in the following snip of the above log:

Any help is greatly appreciated! 

[Susheel Nuguru]

Hi Brady,

Fine tuning throughput is no easy business. But it looks like you have been on the right track all along. You seems to understand the need to calibrate connection interval, parameters and buffer sizes. 

In terms of child image, the below config should be enough with the rest set to defaults

CONFIG_BT_CTLR_SDC_MAX_CONN_EVENT_LEN_DEFAULT=4000000

CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_BT_BUF_ACL_RX_SIZE=502
CONFIG_BT_BUF_ACL_TX_SIZE=502

CONFIG_BT_MAX_CONN=2

I strongly recommend you to take sniffer log of the throughput sample as well and compare that with the sniffer log you got. If the connection interval, 

There can also be bottlenecks in pre and post processing of the data/buffer you are trying to transfer on your peripheral side. I use SystemView to get a visual understanding of timings of each thread/ISR running. It can be possible that you have all the configs set correctly but your peripheral pre/post packet processing is the bottle neck. you can compare the timings of threads by running Throughput example in systemview and also your application to see if you have introduced any processing bottlenecks in some contexts.