Setup HCI controller for maximum data throughput

[OliverB]

Hi,

my firmware is basing on the nRF Connect SDK hci_uart example. I'm using the nRF52840DK as hardware platform and have a STM32 connected to the nRF by HCI with btstack. 

I'm evaluating the maximum possible data throughput of this configuration by using my evaluation board as central device that connects to a peripheral which is a nRF USB stick with a simple Service and characteristic. The central is using write without response. I'm facing that whatever connection parameter I chose only three packet per connection event is sent. 

As I'm not sure sure which component (btstack or nRF Connect SDK) is responsible, I just want to know what I must do to force the controller to use the whole connection event. I can't find find any API to set this up in runtime, I just found the Kconfig parameter CONFIG_BT_CTLR_SDC_MAX_CONN_EVENT_LEN_DEFAULT which is set on compile time and therefore can't be set according the connection parameters that are chosen by the devices in runtime. Here is my current configuration:

CONFIG_UART_INTERRUPT_DRIVEN=y
CONFIG_UART_LINE_CTRL=y
CONFIG_LED=y
CONFIG_LED_GPIO=y

CONFIG_BT=y
CONFIG_BT_HCI_RAW=y
CONFIG_BT_HCI_RAW_H4=y
CONFIG_BT_HCI_RAW_H4_ENABLE=y
CONFIG_BT_DEBUG_LOG=n

CONFIG_BT_MAX_CONN=16
CONFIG_BT_ECC=y
CONFIG_BT_TINYCRYPT_ECC=y
#CONFIG_BT_CTLR_DTM_HCI=y
CONFIG_BT_HCI_VS_EXT=y

CONFIG_BT_CTLR_SDC_MAX_CONN_EVENT_LEN_DEFAULT=400000


# settings for extended MTU
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_BT_DATA_LEN_UPDATE=y
CONFIG_BT_BUF_ACL_RX_SIZE=255
CONFIG_BT_BUF_ACL_TX_SIZE=255
CONFIG_BT_BUF_CMD_TX_SIZE=255
CONFIG_BT_BUF_EVT_DISCARDABLE_SIZE=255

CONFIG_BT_CTLR_PHY_2M=y
CONFIG_BT_CTLR_PHY_CODED=y
CONFIG_BT_CTLR_CHAN_SEL_2=y

# controller settings
#CONFIG_BT_CTLR_ASSERT_HANDLER=y
CONFIG_BT_CTLR_FILTER_ACCEPT_LIST=y
CONFIG_BT_CTLR_PRIVACY=y
CONFIG_BT_CTLR_LE_PING=y

# Workaround: Unable to allocate command buffer when using K_NO_WAIT since
# Host number of completed commands does not follow normal flow control.
CONFIG_BT_BUF_CMD_TX_COUNT=10

CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=512

CONFIG_PM=y
CONFIG_PM_DEVICE=y
#CONFIG_PM_POLICY_CUSTOM=y

CONFIG_BT_WAIT_NOP=n

# LFO configurations
CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC=y
CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC_CALIBRATION=y
CONFIG_CLOCK_CONTROL_NRF_K32SRC_500PPM=n
CONFIG_CLOCK_CONTROL_NRF_K32SRC_50PPM=y
CONFIG_CLOCK_CONTROL_NRF_CALIBRATION_PERIOD=8000
CONFIG_CLOCK_CONTROL_NRF_CALIBRATION_MAX_SKIP=30
CONFIG_CLOCK_CONTROL_NRF_CALIBRATION_TEMP_DIFF=2

I already tried different values for CONFIG_BT_CTLR_SDC_MAX_CONN_EVENT_LEN_DEFAULT  to no avail. 

Thanks and regards,
Oliver

[Karl Ylvisaker]

Hello Oliver,

Do I understand it correctly that it is the STM that is the central in this case?
Please note that it is the central that determines which connection parameters and configurations that will be used - the peripheral may only have a preference that it can prompt the central to use, but it is ultimately the central that decides which parameters to use.

Are you certain that you STM has got the right configuration to allow for longer connection event lengths, so that multiple packets may be sent?
It could also be helpful for you to see this overview on parameters to achieve the highest possible throughput.

Are you familiar with the nRF Sniffer tool? It is a very powerful tool to wield when developing with BLE, and it would be very helpful to see a trace of the communication between the devices when you attempt to set this up so that we may which procedure or configuration that is not being accepted by the central.

Best regards,
Karl

[OliverB]

Karl Ylvisaker said:

Do I understand it correctly that it is the STM that is the central in this case?

Yes, but not completely. The STM implements the host (btstack) which is connected to the nRF52840DK via UART. The DK is programmed with the hci_uart example with the configuration parameters I posted above.

Karl Ylvisaker said:

Are you certain that you STM has got the right configuration to allow for longer connection event lengths, so that multiple packets may be sent?

This is the point. I'm not sure yet which component (STM/host or nRF/controller) is responsible. 

I did a test with the nRF Sniffer. I set a 400 ms connection interval to be able to clearly detect the connection intervals borders. I found out, that in fact the master is just sending 3 packets with the 'More-Data-Flag' set to False in the third packet in every connection interval. So my question gets a little bit more concrete: Is it the controller or the host that decides if the More-Data-Flag gets set? If it is the controller (the nRF), how do I force the controller to send more data?

Here is an excerpt from the packet logs:

No.	Time	Source	            PHY	    Protocol	Length	Delta time 	SN	NESN	More Data	Event counter	Info
117	19.376	Master_0x451814cd	LE 2M	ATT	        251	    395974µs	0	0	    True	    22	            Sent Write Command, Handle: 0x000d (Unknown: Unknown)
118	19.377	Slave_0x451814cd	LE 2M	LE LL	    0	    150µs	    0	1	    False	    22	            Empty PDU
119	19.377	Master_0x451814cd	LE 2M	ATT	        251	    150µs	    1	1	    True	    22	            Sent Write Command, Handle: 0x000d (Unknown: Unknown)
120	19.378	Slave_0x451814cd	LE 2M	LE LL	    0	    150µs	    1	0	    False	    22	            Empty PDU
121	19.378	Master_0x451814cd	LE 2M	ATT	        251	    150µs	    0	0	    False	    22	            Sent Write Command, Handle: 0x000d (Unknown: Unknown)
122	19.380	Slave_0x451814cd	LE 2M	LE LL	    0	    150µs	    0	1	    False	    22	            Empty PDU
123	19.776	Master_0x451814cd	LE 2M	ATT	        251	    395973µs	1	1	    True	    23	            Sent Write Command, Handle: 0x000d (Unknown: Unknown)
124	19.777	Slave_0x451814cd	LE 2M	LE LL	    0	    150µs	    1	0	    False	    23	            Empty PDU
125	19.777	Master_0x451814cd	LE 2M	ATT	        251	    150µs	    0	0	    True	    23	            Sent Write Command, Handle: 0x000d (Unknown: Unknown)
126	19.778	Slave_0x451814cd	LE 2M	LE LL	    0	    150µs	    0	1	    False	    23	            Empty PDU
127	19.778	Master_0x451814cd	LE 2M	ATT	        251	    150µs	    1	1	    False	    23	            Sent Write Command, Handle: 0x000d (Unknown: Unknown)

Thanks and regards,
Oliver

[Karl Ylvisaker]

Hello Oliver,

OliverB said:

Yes, but not completely. The STM implements the host (btstack) which is connected to the nRF52840DK via UART. The DK is programmed with the hci_uart example with the configuration parameters I posted above.

Thank you for the clarification. Even if the host is on the STM side it will still be this device that is the central in the connection to the other device over BLE, and so it is the STM+nRF device that must configure the connection parameters so that additional data gets sent through.

OliverB said:

This is the point. I'm not sure yet which component (STM/host or nRF/controller) is responsible. 

I did a test with the nRF Sniffer. I set a 400 ms connection interval to be able to clearly detect the connection intervals borders. I found out, that in fact the master is just sending 3 packets with the 'More-Data-Flag' set to False in the third packet in every connection interval. So my question gets a little bit more concrete: Is it the controller or the host that decides if the More-Data-Flag gets set? If it is the controller (the nRF), how do I force the controller to send more data?

Here is an excerpt from the packet logs:

Thank you for this update, this is excellent debug information. Now I also understand that the dataflow is from the central to the peripheral in this case (I was under the impression that it was the other way around).
The controller is the one that actually sets the flag, but this will be based on the configurations specified by the host or application during setup. Since it is the central device that determines the number of packets each interval we will need to increase the allowed event length on the central side.
Could you show the BLE configuration on your STM part, just so I can see which parameters are being used?
Alternatively, could you share the full trace of the connection, so that I can see what is being set?

Could you try adding the following kconfigs to your prj.conf, and see if it makes any difference to the number of packets being sent each connection event?

CONFIG_BT_USER_DATA_LEN_UPDATE=y
CONFIG_BT_USER_PHY_UPDATE=y
CONFIG_BT_GAP_AUTO_UPDATE_CONN_PARAMS=n

CONFIG_BT_BUF_ACL_RX_SIZE=502
CONFIG_BT_ATT_PREPARE_COUNT=2
CONFIG_BT_L2CAP_TX_BUF_COUNT=10
CONFIG_BT_L2CAP_TX_MTU=498
CONFIG_BT_L2CAP_DYNAMIC_CHANNEL=y
CONFIG_BT_CONN_TX_MAX=10
CONFIG_BT_BUF_ACL_TX_COUNT=10
CONFIG_BT_BUF_ACL_TX_SIZE=502

CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_BT_CTLR_PHY_2M=y
CONFIG_BT_CTLR_PHY_CODED=y
CONFIG_BT_CTLR_RX_BUFFERS=2
CONFIG_BT_CTLR_SDC_MAX_CONN_EVENT_LEN_DEFAULT=4000000

Best regards,
Karl

[variant]

Regarding the More Data bit: it might be useful to also look at these CONFIG values:  BT_CTLR_SDC_TX_PACKET_COUNT and BT_CTLR_SDC_RX_PACKET_COUNT. 

See devzone.nordicsemi.com/.../is-the-more-data-md-bit-not-being-set-when-it-should-be

[OliverB]

Sorry for the late replay, I was on vacancy and had to do other things when coming back...

The problem is solved in the meantime. Shortly, my hardware setup with two MCU connected by UART with an max baud rate of 1Mbit isn't just capable of transporting data with an data rate more than approx. 600kBit. This is what I'm seeing and this is the reason why the buffers of the controller doesn't get filled fast enough. So the controller behavior is correct and I have to increase the HCI speed by using another bus like USB or SPI.

Thanks to all for your contribution!

Oliver