File transfer/large data transfer example NRF Connect sdk

[cosmotic_instant]

Hi,

I am trying to transfer a dataset of over 100k, via a custom service from a peripheral device.

There are many posts in this forum that describe ways to do this but none seem to apply to the newer nrf Connect SDK. I am using nrf connect 1.7.1 with the VSCode extension.

The peripheral_uart example is often quoted as being a good starting point, but i can't seem to find where i would start some sort of Send data->wait for data sent event-> send next part, kind of cycle. The throughput example appears to transfer a large data set, but i am struggling to interpret that service as it is quite large. 

Is the advised way of doing this still calling notify on a characteristic? I have attempted to call Notify inside an on_sent callback but the code doesn't appear to call callbacks when already executing inside of one.

I would be very grateful if anybody could point me in the right direction towards initiating a large data transfer from a peripheral device.

[Simon]

Is the advised way of doing this still calling notify on a characteristic? I have attempted to call Notify inside an on_sent callback but the code doesn't appear to call callbacks when already executing inside of one.

Try using work queues

Take a look at the peripheral_hr_coded sample to see how to implement a work queue:

https://github.com/nrfconnect/sdk-nrf/blob/v1.8.0/samples/bluetooth/peripheral_hr_coded/src/main.c#L25 

https://github.com/nrfconnect/sdk-nrf/blob/v1.8.0/samples/bluetooth/peripheral_hr_coded/src/main.c#L119

https://github.com/nrfconnect/sdk-nrf/blob/v1.8.0/samples/bluetooth/peripheral_hr_coded/src/main.c#L64

https://github.com/nrfconnect/sdk-nrf/blob/v1.8.0/samples/bluetooth/peripheral_hr_coded/src/main.c#L100-L111

Best regards,

Simon

[cosmotic_instant]

Thank you for the reply Simon. 

I see that this is a good solution for handling ISR generated callbacks.

The questions that i still need answering are as follows:

1. what is a standard way to implement large data transfer from a peripheral to a central device using nrf connect sdk? Should i initiate a read and then have the peripheral notify again and again? how will i know that i am not starting another notify before the previous has been fully sent? Is notify even the right thing to do? could i not just continue to send data chunks from a single data read request? Does this require a characteristic in the central device for the peripheral to write to in a loop?

2. Using the suggested work queue worked fine, but waiting for the on_sent notification before sending another notify was very slow. It took a minute to transfer the word "hello", a 500 times... a payload throughput of 48 Bps. I understand that the data length and mtu can be increased, but even if the payload size was 251, 8 transactions per second is a tiny number. 

3. I see that there is also something referred to as a 'long read' in the peripheral example. What is this and how does it work? should i be using this instead of notify?

I am using the nrf52832 on the nRF52 dk.

Cheers

[Simon]

I have not worked much with Bluetooth that much lately, and need to do some more research to fully answer all your question. But I will try to answer your questions the best I can now.

cosmotic_instant said:

1. what is a standard way to implement large data transfer from a peripheral to a central device using nrf connect sdk? Should i initiate a read and then have the peripheral notify again and again? how will i know that i am not starting another notify before the previous has been fully sent? Is notify even the right thing to do? could i not just continue to send data chunks from a single data read request? Does this require a characteristic in the central device for the peripheral to write to in a loop?

I think notifications would be the most efficient way, see this answer https://devzone.nordicsemi.com/f/nordic-q-a/7052/what-is-the-difference-between-a-ble-central-reading-from-a-peripheral-versus-a-peripheral-notifying-the-central-in-term-of-resources-drained (it's for the nRF5 SDK, but should be relevant for NCS as well)

cosmotic_instant said:

2. Using the suggested work queue worked fine, but waiting for the on_sent notification before sending another notify was very slow. It took a minute to transfer the word "hello", a 500 times... a payload throughput of 48 Bps. I understand that the data length and mtu can be increased, but even if the payload size was 251, 8 transactions per second is a tiny number. 

According to this answer (nRF5 SDK), it should be possible to get a transfer speed of 1400 kbps with the nRF52832, to achieve this you need to set the connection interval to 400ms, the MTU size to 247 bytes and enable DLE. Check out Bluetooth 5 speed maximum throughput.

I will try to do some tests with the peripheral_uart sample (NCS) this week and see if I'm able to reach this speed.

cosmotic_instant said:

3. I see that there is also something referred to as a 'long read' in the peripheral example. What is this and how does it work? should i be using this instead of notify?

I will get back to you on this.

Best regards,

Simon

[Simon]

I did some quick measurements using the peripheral_uart and the central_uart and I measured a speed of 4603bytes/second (36.8kbps), which is certainly better than 48bps.

I sent the data accordingly from the peripheral_uart sample:

#define MTU_SIZE 20
#define TEST_BUF_SIZE 2000

uint8_t test_buf[TEST_BUF_SIZE];
..


for(int y =0; y< TEST_BUF_SIZE; y+=MTU_SIZE){
	if (bt_nus_send(NULL, internal_data, MTU_SIZE)) {
		LOG_WRN("."); //failed
	}else{
		LOG_INF("-"); //success
	}
}

I don't think you need to wait for on_sent, It seems like the Zephyr Bluetooth host will queue the data.

By using the parameters mentions in this answer you can increase the speed even more.

[cosmotic_instant]

Hi Simon,

Were the measurements that you have taken done with nRF connect SDK? I know that there is plenty of support on this forum for Nrf5 SDK but i am struggling to get examples for the connect SDK, using Zephyr's Bluetooth implementation. As that is now the recommended way of developing moving forward for Nordic, I started developing using the NCS. I know the BLE theory is the same, but how you setup characteristics etc is not. 

The answer you linked to This answer, appears to reference a softdevice. The Zephyr implementation doesn't use a soft device correct? it's an open source implementation software implementation that is built along with the zephyr OS? 

How can i find the size of the TX buffers? I assume these can be altered?

Your test only sends 2000 bytes (or 200 packets @ 20 mtu size. What you have demonstrated is 36kbps but on a data set that is smaller than the tx buffer size. If it were larger, you probably could not add in a for loop as you are showing, without the function failing when it fills. I will try this example until it fails and continue to add the next packet until it is successful. This way, the throughput should be the same. Though, it does seem a shame to not a more event driven method of doing this.

I guess, if i know the TX buffer size, i also know based on the already measured throughput, how long it will take to empty the buffer. So once i receive a fail, i can sleep until i expect it to be say half empty, before attempting to add the next packets. This will give a yield point instead of being stuck in an endless loop of attempting to add to the transmit queue.

Any news on the Long read?