How to read a large file (About 240MB) from SD card

[Samiulhaq]

Hello all, I hope you are doing well. 

I Have a project on NRF52840 in which I have multiple sensors connected to the NRF52 and the sensor data are storing in the SD card in the form of a CSV file of size 240 MB. I want to read this large file from the SD card and then send it to the mobile app using ble_periperhial. I have used This example and modified it, but the problem is that I can only read 220000 bytes from the SD card in the buffer at once and when I am trying to increase 

   #define FILE_SIZE_MAX (Maximum size in bytes of the file to be read from SDCARD) greater than 220000, while the following;

                                            static uint8_t file_buffer[FILE_SIZE_MAX]; 

                                           ff_result = f_read(&file, file_buffer, FILE_SIZE_MAX, (UINT *) &bytes_read);

 Then it gives the following error:

                                      .bss is too large to fit in RAM1 memory segment
                                     .heap is too large to fit in RAM1 memory segment
                                      section .heap overlaps absolute placed section .stack
                                      section .stack VMA [000000002003e000,000000002003ffff] overlaps section .bss VMA [0000000020002c64,0000000020224b08]   

                                               

I don't know about this error and how to solve it. Simply I want to read the whole file (234MB) from the SD card at once and then send it in chunks to the mobile app, Is this possible or do I have to read in chunks from the SD card too? 

Any help regarding this will be highly appreciated.

[Samiulhaq]

Hi Torbjørn,

**UPDATE:

Samiulhaq said:

I have tested on three different phones and almost every phone has this behavior(decreasing conn. interval leads to more empty packets). when the conn. interval is 40-50 then the number of empty packets is the least but when I decrease the conn. interval then I am facing empty packets which leads to small throughput and when I increase the conn. interval then the throughput decreases( I don't know why).

I have reduced the connection interval and it increases the throughput(in the phone which supports 2M PHY).

Now I am the problem of "When I am using 1M PHY I can send the 2284066 bytes in about 26-27 seconds which is quite good but the problem is that when I switched to 2M PHY then the rate of data transfer(throughput) decreased a lot although I assumed a high throughput in 2M PHY but it is just the opposite of my expectation. What do you think will be the reason for the small throughput in 2M PHY than 1M"?  This happens only in this project (the one you have provided).

What do you think about this?

Best Regards,

Sami 

[ovrebekk]

Hi Sami

Are you able to share sniffer traces for the two cases, with the transfer happening in 1M and 2M modes respectively? 

Also, which phone is showing this behavior?

I actually have seen similar behavior on the Huawei P20 Pro, where the 2M mode showed significantly lower transfer speed. The reason for this was that when enabling 2M phy the phone was only able to send a single packet in each connection event, while in 1M mode it could send multiple packets. In this case it is clearly a bug on the phone side, where the 2M support is not properly implemented. 

Best regards
Torbjørn

[Samiulhaq]

Hi Torbjørn, thanks for the reply.

ovrebekk said:

Are you able to share sniffer traces for the two cases, with the transfer happening in 1M and 2M modes respectively? 

Yes sure,  Actually I am attaching 4 sniffer files:

             1) 1M PHY conn.interval=50

             2) 1M PHY conn.interval=10

             3)  2M PHY conn.interval=50

             4)  2M PHY conn.interval=10

The result I have concluded from the 4 traces is that:

1) Increasing conn. interval in 1M PHY leads to high throughput, that is:

              Throughput at conn. interval 50 >>  Throughput at conn. interval 10

2) Increasing conn. interval in 2M PHY leads to decreased(lower) throughput, that is:

              Throughput at conn. interval 10 >>  Throughput at conn. interval 50

Your comment about that? I am totally confused about what's going on. 

ovrebekk said:

Also, which phone is showing this behavior?

Samsung Galaxy A30, Android version 11.

ovrebekk said:

I actually have seen similar behavior on the Huawei P20 Pro, where the 2M mode showed significantly lower transfer speed. The reason for this was that when enabling 2M phy the phone was only able to send a single packet in each connection event, while in 1M mode it could send multiple packets. In this case it is clearly a bug on the phone side, where the 2M support is not properly implemented. 

 I don't think so the problem is with the phone because I have tested another project (the same project but Modified by me in which I am reading data from an SD card and sent on ble directly without the involvement of fifo or other temporary storage.), in which case the 2M PHY has achieved a decent throughput (2284066 bytes in about 23-24 seconds) with the same phone. 

I don't know why this project has a problem with 2M PHY. Have you tested it on your side? 

If you want more information just let me know and thanks for your through explanation and help.

Best Regards,

Sami

1M_PHY_Sniffer_trace_conn_interval_10.pcapng

1M_PHY_conn_interval_50_sniffer_trace.pcapng

2M_PHY_conn_interval_10_sniffer_trace.pcapng

2M_PHY_conn_interval_50_sniffer_trace.pcapng

[ovrebekk]

Hi Sami

Thanks for sharing all the traces. 

I only had limited time to work on your case today, so I prioritized fixing my example. 

You were correct that there were some issues with it. I wasn't sending full length packets in all cases, and also discovered an issue with the code going to sleep even if you had more data to send. 

I pushed an update to the repo which should fix these issues.

Are you able to check out the updated example and see if it works better?

I will have to come back to you on the traces and the 2M phy issues next week. 

Best regards
Torbjørn 

[Samiulhaq]

Hey Torbjørn , thanks.

ovrebekk said:

Are you able to check out the updated example and see if it works better?

I am going to test it now and let you know about the outcomes. Thanks for updating the project.

ovrebekk said:

I will have to come back to you on the traces and the 2M phy issues next week. 

Okay sir sure, thanks. 

Best Regards,

Sami