File transfer/large data transfer example NRF Connect sdk

[cosmotic_instant]

Hi,

I am trying to transfer a dataset of over 100k, via a custom service from a peripheral device.

There are many posts in this forum that describe ways to do this but none seem to apply to the newer nrf Connect SDK. I am using nrf connect 1.7.1 with the VSCode extension.

The peripheral_uart example is often quoted as being a good starting point, but i can't seem to find where i would start some sort of Send data->wait for data sent event-> send next part, kind of cycle. The throughput example appears to transfer a large data set, but i am struggling to interpret that service as it is quite large. 

Is the advised way of doing this still calling notify on a characteristic? I have attempted to call Notify inside an on_sent callback but the code doesn't appear to call callbacks when already executing inside of one.

I would be very grateful if anybody could point me in the right direction towards initiating a large data transfer from a peripheral device.

[Simon]

cosmotic_instant said:

Were the measurements that you have taken done with nRF connect SDK? I know that there is plenty of support on this forum for Nrf5 SDK but i am struggling to get examples for the connect SDK, using Zephyr's Bluetooth implementation. As that is now the recommended way of developing moving forward for Nordic, I started developing using the NCS. I know the BLE theory is the same, but how you setup characteristics etc is not. 

I used the nRF Connect SDK, see https://github.com/nrfconnect/sdk-nrf/tree/main/samples/bluetooth.

cosmotic_instant said:

The answer you linked to This answer, appears to reference a softdevice. The Zephyr implementation doesn't use a soft device correct? it's an open source implementation software implementation that is built along with the zephyr OS? 

In the reply I linked to, the nRF5 SDK was used. It was not meant as a guide exactly how to improve the speed in Zephyr/NCS, but rather what parameters you should use. In NCS BLE samples, we use the SoftDevice Controller (similar to the SoftDevice). You can also use the Zephyr BLE controller.

cosmotic_instant said:

How can i find the size of the TX buffers? I assume these can be altered?

Your test only sends 2000 bytes (or 200 packets @ 20 mtu size. What you have demonstrated is 36kbps but on a data set that is smaller than the tx buffer size. If it were larger, you probably could not add in a for loop as you are showing, without the function failing when it fills. I will try this example until it fails and continue to add the next packet until it is successful. This way, the throughput should be the same. Though, it does seem a shame to not a more event driven method of doing this.

I guess, if i know the TX buffer size, i also know based on the already measured throughput, how long it will take to empty the buffer. So once i receive a fail, i can sleep until i expect it to be say half empty, before attempting to add the next packets. This will give a yield point instead of being stuck in an endless loop of attempting to add to the transmit queue.

Any news on the Long read?

I will get back to you on these question, and will try to put off some time the next days to try to achive a speed close to 1400 kbps.

[cosmotic_instant]

Thank you. 

From my own testing, it seems you can enter a while loop, continuously adding a single byte to the send buffer, and not get a failure to add using bt_gatt_notify_cb. I think it's probably unlikely it is sending faster than the core could add to the buffer, so does this imply that this function call is blocking?

I am more interested in how the code works and how to manipulate it than getting the highest performance at this stage.

So why are you limiting the data size you add in your test function by the MTU size? My understanding is that MTU is a term used in the ATT layer, and the application uses Data Length? Do you have to manually chunk data this way? I thought the point of the MTU, is that is the payload size to which your Data Length will be chunked.

[Simon]

I got some progress here, I added these to the peripheral_uart sample (NCS v1.8.0):

CONFIG_BT_BUF_ACL_RX_SIZE=251
CONFIG_BT_ATT_PREPARE_COUNT=2
CONFIG_BT_L2CAP_TX_BUF_COUNT=10
CONFIG_BT_L2CAP_TX_MTU=247
CONFIG_BT_CTLR_RX_BUFFERS=2
CONFIG_BT_BUF_ACL_TX_COUNT=10
CONFIG_BT_BUF_ACL_TX_SIZE=251
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251

and changed MTU_SIZE to 244, such that bt_nus_send would send 244 block at the time. Then I achieved a speed of 46511.6 bytes/second=372kbps. I communicated with the nRF Connect Mobile app. From the mobile app you have to enable notification for the TX char and request an MTU of 250 bytes.

I also increased TEST_BUF_SIZE to 20000, such that a total of 20k bytes would be sent.

Is this fast enough speed for your purpose? If not try to increase the connection interval to 400ms (tell me if you need help with this).

 Here is the changes I applied to the peripheral_uart sample:

diff --git a/samples/bluetooth/peripheral_uart/prj.conf b/samples/bluetooth/peripheral_uart/prj.conf
index e5262de64..d3e77f24e 100644
--- a/samples/bluetooth/peripheral_uart/prj.conf
+++ b/samples/bluetooth/peripheral_uart/prj.conf
@@ -1,50 +1,65 @@
-#
-# Copyright (c) 2018 Nordic Semiconductor
-#
-# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
-#
-
-# Enable the UART driver
-CONFIG_UART_ASYNC_API=y
-CONFIG_NRFX_UARTE0=y
-CONFIG_SERIAL=y
-
-CONFIG_GPIO=y
-
-# Make sure printk is printing to the UART console
-CONFIG_CONSOLE=y
-CONFIG_UART_CONSOLE=y
-
-CONFIG_HEAP_MEM_POOL_SIZE=2048
-
-CONFIG_BT=y
-CONFIG_BT_PERIPHERAL=y
-CONFIG_BT_DEVICE_NAME="Nordic_UART_Service"
-CONFIG_BT_DEVICE_APPEARANCE=833
-CONFIG_BT_MAX_CONN=1
-CONFIG_BT_MAX_PAIRED=1
-
-# Enable the NUS service
-CONFIG_BT_NUS=y
-
-# Enable bonding
-CONFIG_BT_SETTINGS=y
-CONFIG_FLASH=y
-CONFIG_FLASH_PAGE_LAYOUT=y
-CONFIG_FLASH_MAP=y
-CONFIG_NVS=y
-CONFIG_SETTINGS=y
-
-# Enable DK LED and Buttons library
-CONFIG_DK_LIBRARY=y
-
-# This example requires more workqueue stack
-CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048
-
-# Config logger
-CONFIG_LOG=y
-CONFIG_USE_SEGGER_RTT=y
-CONFIG_LOG_BACKEND_RTT=y
-CONFIG_LOG_BACKEND_UART=n
-
-CONFIG_ASSERT=y
+#
+# Copyright (c) 2018 Nordic Semiconductor
+#
+# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
+#
+
+# Enable the UART driver
+CONFIG_UART_ASYNC_API=y
+CONFIG_NRFX_UARTE0=y
+CONFIG_SERIAL=y
+
+CONFIG_GPIO=y
+
+# Make sure printk is printing to the UART console
+CONFIG_CONSOLE=y
+CONFIG_UART_CONSOLE=y
+
+CONFIG_HEAP_MEM_POOL_SIZE=2048
+
+CONFIG_BT=y
+CONFIG_BT_PERIPHERAL=y
+CONFIG_BT_DEVICE_NAME="Nordic_UART_Service"
+CONFIG_BT_DEVICE_APPEARANCE=833
+CONFIG_BT_MAX_CONN=1
+CONFIG_BT_MAX_PAIRED=1
+
+# Enable the NUS service
+CONFIG_BT_NUS=y
+
+# Enable bonding
+CONFIG_BT_SETTINGS=y
+CONFIG_FLASH=y
+CONFIG_FLASH_PAGE_LAYOUT=y
+CONFIG_FLASH_MAP=y
+CONFIG_NVS=y
+CONFIG_SETTINGS=y
+
+# Enable DK LED and Buttons library
+CONFIG_DK_LIBRARY=y
+
+# This example requires more workqueue stack
+CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048
+
+# Config logger
+CONFIG_LOG=y
+CONFIG_USE_SEGGER_RTT=y
+CONFIG_LOG_BACKEND_RTT=y
+CONFIG_LOG_BACKEND_UART=n
+
+CONFIG_ASSERT=y
+CONFIG_BT_USER_DATA_LEN_UPDATE=y
+
+
+CONFIG_BT_BUF_ACL_RX_SIZE=251
+#CONFIG_BT_GATT_CLIENT=y
+CONFIG_BT_ATT_PREPARE_COUNT=2
+#CONFIG_BT_CONN_TX_MAX=10
+CONFIG_BT_L2CAP_TX_BUF_COUNT=10
+CONFIG_BT_L2CAP_TX_MTU=247
+#CONFIG_BT_L2CAP_DYNAMIC_CHANNEL=y
+#CONFIG_BT_CTLR_PHY_2M=y
+CONFIG_BT_CTLR_RX_BUFFERS=2
+CONFIG_BT_BUF_ACL_TX_COUNT=10
+CONFIG_BT_BUF_ACL_TX_SIZE=251
+CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
\ No newline at end of file
diff --git a/samples/bluetooth/peripheral_uart/src/main.c b/samples/bluetooth/peripheral_uart/src/main.c
index 2acac0051..37ad3724b 100644
--- a/samples/bluetooth/peripheral_uart/src/main.c
+++ b/samples/bluetooth/peripheral_uart/src/main.c
@@ -240,7 +240,7 @@ static int uart_init(void)
 		return -ENXIO;
 	}
 
-	if (IS_ENABLED(CONFIG_USB_DEVICE_STACK)) {
+	if (IS_ENABLED(CONFIG_USB)) {
 		err = usb_enable(NULL);
 		if (err) {
 			LOG_ERR("Failed to enable USB");
@@ -319,6 +319,30 @@ static int uart_init(void)
 	return uart_rx_enable(uart, rx->data, sizeof(rx->data), 50);
 }
 
+static void le_param_updated(struct bt_conn *conn, uint16_t interval,
+			     uint16_t latency, uint16_t timeout)
+{
+	printk("Connection parameters updated.\n"
+	       " interval: %d, latency: %d, timeout: %d\n",
+	       interval, latency, timeout);
+
+}
+
+static void le_data_length_updated(struct bt_conn *conn,
+				   struct bt_conn_le_data_len_info *info)
+{
+	/*if (!data_length_req) {
+		return;
+	}*/
+
+	printk("LE data len updated: TX (len: %d time: %d)"
+	       " RX (len: %d time: %d)\n", info->tx_max_len,
+	       info->tx_max_time, info->rx_max_len, info->rx_max_time);
+
+	/*data_length_req = false;
+	k_sem_give(&throughput_sem);*/
+}
+
 static void connected(struct bt_conn *conn, uint8_t err)
 {
 	char addr[BT_ADDR_LE_STR_LEN];
@@ -377,6 +401,8 @@ static void security_changed(struct bt_conn *conn, bt_security_t level,
 static struct bt_conn_cb conn_callbacks = {
 	.connected    = connected,
 	.disconnected = disconnected,
+	.le_param_updated = le_param_updated,
+	.le_data_len_updated = le_data_length_updated,
 #ifdef CONFIG_BT_NUS_SECURITY_ENABLED
 	.security_changed = security_changed,
 #endif
@@ -415,6 +441,18 @@ static void auth_cancel(struct bt_conn *conn)
 }
 
 
+static void pairing_confirm(struct bt_conn *conn)
+{
+	char addr[BT_ADDR_LE_STR_LEN];
+
+	bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr));
+
+	bt_conn_auth_pairing_confirm(conn);
+
+	LOG_INF("Pairing confirmed: %s", log_strdup(addr));
+}
+
+
 static void pairing_complete(struct bt_conn *conn, bool bonded)
 {
 	char addr[BT_ADDR_LE_STR_LEN];
@@ -441,6 +479,7 @@ static struct bt_conn_auth_cb conn_auth_callbacks = {
 	.passkey_display = auth_passkey_display,
 	.passkey_confirm = auth_passkey_confirm,
 	.cancel = auth_cancel,
+	.pairing_confirm = pairing_confirm,
 	.pairing_complete = pairing_complete,
 	.pairing_failed = pairing_failed
 };
@@ -555,9 +594,29 @@ static void configure_gpio(void)
 		LOG_ERR("Cannot init LEDs (err: %d)", err);
 	}
 }
+#define MTU_SIZE 244
+#define BYTES_TO_SEND 20000
+
+uint8_t dummy[MTU_SIZE];
+uint64_t stamp;
+int64_t delta;
+
+#define PIN_GPIO  (31UL)
 
 void main(void)
 {
+	for(int x=0; x< MTU_SIZE;x++){
+		dummy[x] = x;
+		//printk("dummy[%d] = %d\n", x, dummy[x]);
+	}
+
+	NRF_GPIO->PIN_CNF[PIN_GPIO] = (GPIO_PIN_CNF_DIR_Output << GPIO_PIN_CNF_DIR_Pos) |
+							(GPIO_PIN_CNF_DRIVE_S0S1 << GPIO_PIN_CNF_DRIVE_Pos) |
+							(GPIO_PIN_CNF_INPUT_Connect << GPIO_PIN_CNF_INPUT_Pos) |
+							(GPIO_PIN_CNF_PULL_Disabled << GPIO_PIN_CNF_PULL_Pos) |
+							(GPIO_PIN_CNF_SENSE_Disabled << GPIO_PIN_CNF_SENSE_Pos);
+
+	//LOG_INF("LOG_INF, Delta: %lld ms", delta);
 	int blink_status = 0;
 	int err = 0;
 
@@ -615,12 +674,36 @@ void ble_write_thread(void)
 		/* Wait indefinitely for data to be sent over bluetooth */
 		struct uart_data_t *buf = k_fifo_get(&fifo_uart_rx_data,
 						     K_FOREVER);
+		LOG_INF("Start transmission of %d bytes", BYTES_TO_SEND);
+		stamp = k_uptime_get_32();
+		NRF_GPIO->OUTSET = (1UL << PIN_GPIO);  //turns led (P0.17) off
+		NRF_GPIO->OUTCLR = (1UL << PIN_GPIO); 
+
+		for(int y =0; y< BYTES_TO_SEND; y+=MTU_SIZE){
+			//LOG_INF("y=%d/%d\n", y, BYTES_TO_SEND);
+			if (bt_nus_send(NULL, dummy, MTU_SIZE)) {
+				LOG_WRN("."); //failed
+			}else{
+				LOG_INF("-"); //success
+			}
+		}
+		delta = k_uptime_delta(&stamp); //k_uptime_delta(&stamp);
+		//LOG_INF("Should be 3000 ms| it is %" PRId64"  ms", delta);
+		printk("Delta: %lld ms\n", delta);
+		LOG_INF("Transmission ended");
+		NRF_GPIO->OUTSET = (1UL << PIN_GPIO);  //turns led (P0.17) off
+		NRF_GPIO->OUTCLR = (1UL << PIN_GPIO); 
+		printk("Sent %d bytes (%u KB) in %lld ms\n",
+	       BYTES_TO_SEND, BYTES_TO_SEND / 1024, delta); //, ((uint64_t)data * 8 / delta));
+		/* Wait indefinitely for data to be sent over bluetooth */
+		/*struct uart_data_t *buf = k_fifo_get(&fifo_uart_rx_data,
+						     K_FOREVER);
 
 		if (bt_nus_send(NULL, buf->data, buf->len)) {
 			LOG_WRN("Failed to send data over BLE connection");
 		}
 
-		k_free(buf);
+		k_free(buf);*/
 	}
 }