Sending a complete file at once from SD card to Mobile App using NRF52840.

[Samiulhaq]

Hello Nordic Family. I hope this post finds you well. 

I am working on a project where I have connected multiple sensors to the NRF52840. I have collected all the sensor's data and saved it on the SD card as a CSV file. The file size is about 240MB which has 60 Hours of data from all the sensors(client requirements). Now I want to send this complete file at once from the SD card to the Mobile App in the fastest possible way. I have run the below program on my board

/cfs-file/__key/communityserver-discussions-components-files/4/8204.ble_5F00_app_5F00_uart_5F00_fatfs_5F00_send_5F00_sdk_5F00_16.0.0.zip.

but this program reads data one by one byte from the SD card and sends it byte by byte to the mobile app which I don't want. I want to send the whole file at once in the fastest possible way not byte by byte. Any help regarding this will be highly appreciated. 

Best Regards,

Sami 

[Håkon Alseth]

Hi,

 

sends it byte by byte to the mobile app

Buffer up the read out data from the SD card and send them in size of "BLE_NUS_MAX_DATA_LEN" instead?

 

Kind regards,

Håkon

[Samiulhaq]

Hi Hakon, thank you for your kind suggestion. Yes, I am sending the data in the size of "BLE_NUS_MAX_DATA_LEN"( 243byte) but it can't solve my problem because the data is sent in chunks of 243bytes per chunk and I want to send the complete file at once, not in chunks. My file size is about 240MB, will it be fully sent to the mobile App at once? If yes Can you please recommend to me the fastest possible way?

Best Regards,

Sami

[Samiulhaq]

Hi Hakon, 

Håkon Alseth said:

Your overall throughput will depend on what the parameters that the central device accepts. It is true that you can technically achieve these numbers, but that also depends on a phone that allows such optimal parameters.

I have applied all the suggestions from multiple similar threads but still, I can only send 220KB(of 234MB file) per 4 seconds which is 55KB/second. This figure is far beyond the 1Mbps speed. 

Håkon Alseth said:

You have to transfer the file in chunks of the max. data length that you can send, ie. 244 bytes in your scenario.

Yes, I am sending in chunks of 244 bytes but the rate of sending chunks is very low. I am very tired of looking at this problem for more than a week and studied a lot of threads but no one help me. I just want to read a 234MB file from the SD card and send it to the mobile app in the fastest possible way(up to 1Mbps)

Best Regards,

Sami Ul Haq. 

[Håkon Alseth]

Hi,

 

Samiulhaq said:

I have applied all the suggestions from multiple similar threads but still, I can only send 220KB(of 234MB file) per 4 seconds which is 55KB/second. This figure is far beyond the 1Mbps speed. 

55 kiloBytes per second -> 440kbit/s, which is quite fast when using the BLE 1Mbit on-air data rate.

Have you checked what the connection parameters that you receive from your phone is? This might be the maximum based on the event length, connection interval, etc. that the central provides you with.

 

Have you tried requesting 2MBit PHY in the BLE_GAP_EVT_CONNECTED event?

Here's a snippet on how to request that:  enabling 2Mb phy Windows 10 (1909) 

 

Kind regards,

Håkon

[Samiulhaq]

Hi Hakan, thanks for the reply.

Håkon Alseth said:

55 kiloBytes per second -> 440kbit/s, which is quite fast when using the BLE 1Mbit on-air data rate.

No, I am not using IMbit on-air data rate, rather I am using 2MBPS as I have set BLE_GAP_EVT_PHY_UPDATE_REQUEST: to 

                .rx_phys = BLE_GAP_PHY_2MBPS,
.                tx_phys = BLE_GAP_PHY_2MBPS,

Is this the correct way to set the 2MBPS on-air data rate? 

Håkon Alseth said:

Have you tried requesting 2MBit PHY in the BLE_GAP_EVT_CONNECTED event?

The below is my Ble_connected Case:

case BLE_GAP_EVT_CONNECTED:
            NRF_LOG_INFO("Connected");
            err_code = bsp_indication_set(BSP_INDICATE_CONNECTED);
            APP_ERROR_CHECK(err_code);
            m_conn_handle = p_ble_evt->evt.gap_evt.conn_handle;
            err_code = nrf_ble_qwr_conn_handle_assign(&m_qwr, m_conn_handle);
            APP_ERROR_CHECK(err_code);
            break;

How do I check whether the 2MBit PHY is requested or not in the BLE_GAP_EVT_CONNECTED case?

Håkon Alseth said:

Have you checked what the connection parameters that you receive from your phone is? This might be the maximum based on the event length, connection interval, etc. that the central provides you with.

Yes Sir, this might be the case, How do I check these parameters in my mobile App? I am using a Vivo phone and the mobile App which I am using to get the data is nRF Toolbox in which I am using UART. How to check connection parameters in my case? 

Best Regards,

Sami Ul Haq

[Håkon Alseth]

Hi Sami,

 

Samiulhaq said:

No, I am not using IMbit on-air data rate, rather I am using 2MBPS as I have set BLE_GAP_EVT_PHY_UPDATE_REQUEST: to 

                .rx_phys = BLE_GAP_PHY_2MBPS,
.                tx_phys = BLE_GAP_PHY_2MBPS,

Is this the correct way to set the 2MBPS on-air data rate? 

You should call the snippet from the BLE_GAP_EVT_CONNECTED, similar to this:

case BLE_GAP_EVT_CONNECTED:
{
    ble_gap_phys_t const phys =
    {
       .rx_phys = BLE_GAP_PHY_2MBPS,
       .tx_phys = BLE_GAP_PHY_2MBPS,
    };
    err_code = sd_ble_gap_phy_update(p_ble_evt->evt.gap_evt.conn_handle, &phys);
    ...
    break;
};

Samiulhaq said:

Yes Sir, this might be the case, How do I check these parameters in my mobile App? I am using a Vivo phone and the mobile App which I am using to get the data is nRF Toolbox in which I am using UART. How to check connection parameters in my case? 

You can use the BLE sniffer to see the details of the link:

https://www.nordicsemi.com/Products/Development-tools/nrf-sniffer-for-bluetooth-le

 

Note that you should also include the pairing process, so that the sniffer can follow (ie. decode) the rest of the communication as well.

You can do this by deleting the bonds on both phone and nRF, then the sniffer shall be able to pick it up.

 

Kind regards,

Håkon

[Samiulhaq]

Hello Hakan, thanks for the replay. 

Håkon Alseth said:

You should call the snippet from the BLE_GAP_EVT_CONNECTED, similar to this:

Yes, I have included these lines in the BLE_GAP_EVT_CONNECTED.  Thanks for the snippet.

Håkon Alseth said:

You can use the BLE sniffer to see the details of the link:

I have set up the Sniffer with Wireshark, but I am new to the sniffer and can't analyze the packets correctly that is how I can find the packet size, number of bits/bytes per packet, connection interval, time taken by1 packet, and other related parameters in the sniffer/Wireshark. I have attached the Wireshark result to this reply, can you please analyze it and extract the required information from it? I will be very thankful for this kind act. 

Sniffer_Result.pcapng

Håkon Alseth said:

Note that you should also include the pairing process, so that the sniffer can follow (ie. decode) the rest of the communication as well.

You can do this by deleting the bonds on both phone and nRF, then the sniffer shall be able to pick it up.

 

I didn't understand this as I am new to Sniffer. I have the sniffer running in one NRF52840 Eval kit and another NRF2840 DK for the main task to send the data to the app. Can you please refer me to a blog/ticket etc for more clarification?

Best Regards,

Sami  

[Håkon Alseth]

Hi Sami,

 

Sniffer trace was good, thanks for sharing this one.

 

The PHY is changed successfully to 2MBit in frame #867.

What is the throughput you're seeing now?

From the traces it seems that the central (phone) does not poll the peripheral (nrf), which can indicate that it is doing something else. The problem with using a phone is that most bluetooth chipsets there are combo-chipsets, and also uses the same radio to server multiple services.

 

Kind regards,

Håkon

[Samiulhaq]

Hi Hakan, thanks for the reply. 

Håkon Alseth said:

The PHY is changed successfully to 2MBit in frame #867.

Yes sir, it is. My mobile phone responded unknown to the PHY update request, so I tried with another mobile and it worked. 

Håkon Alseth said:

What is the throughput you're seeing now?

Unfortunately, the throughput is still the same(220KB per 3-4 seconds).  

Håkon Alseth said:

From the traces it seems that the central (phone) does not poll the peripheral (nrf), which can indicate that it is doing something else. The problem with using a phone is that most bluetooth chipsets there are combo-chipsets, and also uses the same radio to server multiple services.

What do you mean by "the central doesn't poll the peripheral"? If you scroll down the Sniffer file (I shared with you), you will see the data has been exchanged correctly( the green lines in the Sniffer file).

Seems like sending the data file chunk by chunk won't be the best solution to send a large file (about 240MB). Have you watched this video where they are sending images from nRF52 to the mobile app in a single capture and in stream mode like a video? I want to achieve exactly the same of sending the file in bursts, not in chunks. Is it possible with the CSV file or only for the images? 

Best Regards,

Sami

[Håkon Alseth]

Hi Sami,

Samiulhaq said:

What do you mean by "the central doesn't poll the peripheral"? If you scroll down the Sniffer file (I shared with you), you will see the data has been exchanged correctly( the green lines in the Sniffer file).

The data is exchanged properly, but the overall throughput it limited by the central.

 

The phone has "radio silence" for smaller periods (< 50 ms), which indicates that the phone itself is servicing other bluetooth devices or similar on its end.

Even though the peripheral has more data to send, and is signalling this to the central, the central ends the current connection interval earlier than usual.

 

Samiulhaq said:

Seems like sending the data file chunk by chunk won't be the best solution to send a large file (about 240MB). Have you watched this video where they are sending images from nRF52 to the mobile app in a single capture and in stream mode like a video? I want to achieve exactly the same of sending the file in bursts, not in chunks. Is it possible with the CSV file or only for the images? 

The transport both your case, and this video is similar. Data is streamed from a source and over the bluetooth layer, in packets of >200 bytes per payload.

If you're using the same phone as previous test, it will highly likely give you similar throughput.

 

Kind regards,

Håkon

[Samiulhaq]

Thanks Hakan for the quick reply. 

Håkon Alseth said:

The phone has "radio silence" for smaller periods (< 50 ms), which indicates that the phone itself is servicing other bluetooth devices or similar on its end.

Okay got it, what will be the solution for this? Is it depend on the phone?

Håkon Alseth said:

The phone has "radio silence" for smaller periods (< 50 ms), which indicates that the phone itself is servicing other bluetooth devices or similar on its end.

But my phone is only connected to the UART inside the nRF toolbox app and it is only receiving the data from the nRF52. How can I check whether the phone is servicing other Bluetooth devices?

Håkon Alseth said:

The transport both your case, and this video is similar. Data is streamed from a source and over the bluetooth layer, in packets of >200 bytes per payload.

If you're using the same phone as previous test, it will highly likely give you similar throughput.

The sniffer file I have shared with you is the result of a Samsung mobile. It doesn't change(increase) the throughput. 

Another thing I have noticed, if you look at the actual data in the Sniffer file you will see that the data is some like:

UART Rx [truncated]: 6152,15433,36722,45008,23274,41643,855,475,15158,6906,13015,19129,40270,38062,17184,20220111121250\r\n23784,38793,36320,49942,19511,34148,1653,45854,26109,10248,6625,7790,46092,40673,18648,4132,31523,20220111121250\r\

What does [truncated] mean?

How to send 2 lines (about 243Bytes) of the CSV file per chunk and when the \r\n of every 2nd line comes then send the chunk (in short I want to send 2 lines per chunk).

Best Regards,

Sami

[Håkon Alseth]

Hi,

 

Samiulhaq said:

Okay got it, what will be the solution for this? Is it depend on the phone?

Yes, it will depend on the phone.

 

Samiulhaq said:

But my phone is only connected to the UART inside the nRF toolbox app and it is only receiving the data from the nRF52. How can I check whether the phone is servicing other Bluetooth devices?

It could be as simple as the phone needs time to scan for new devices in the background, or that it has a wifi/BLE combo-chipset, where wifi takes priority. It is hard to say exactly what is the issue, unfortunately.

Samiulhaq said:

The sniffer file I have shared with you is the result of a Samsung mobile. It doesn't change(increase) the throughput. 

Another thing I have noticed, if you look at the actual data in the Sniffer file you will see that the data is some like:

UART Rx [truncated]: 6152,15433,36722,45008,23274,41643,855,475,15158,6906,13015,19129,40270,38062,17184,20220111121250\r\n23784,38793,36320,49942,19511,34148,1653,45854,26109,10248,6625,7790,46092,40673,18648,4132,31523,20220111121250\r\

What does [truncated] mean?

How to send 2 lines (about 243Bytes) of the CSV file per chunk and when the \r\n of every 2nd line comes then send the chunk (in short I want to send 2 lines per chunk).

From the log, there's a field called "more data" - this indicates to the central that the peripheral has more data to send.

I have highlighted it here:

 

From the trace, this field will always be set to true, meaning that the peripheral always tries to send more data to the central.

If you look at frame #1226 for instance, where the central uses 27 ms to ask for new data:

There's several of these timing gaps within the trace, which limits your overall throughput, and these are driven by the phone side.

Samiulhaq said:

What does [truncated] mean?

How to send 2 lines (about 243Bytes) of the CSV file per chunk and when the \r\n of every 2nd line comes then send the chunk (in short I want to send 2 lines per chunk).

Truncated means that the wireshark view basically does not show all your data, ie. the graphical view stops listing the data. The data is being sent, it is just a graphical feature of wireshark.

The "\r\n" are line-endings in a file. You will need to parse the datastream before sending it through the bluetooth link to remove or detect these.

 

Kind regards,

Håkon