Sending a complete file at once from SD card to Mobile App using NRF52840.

[Samiulhaq]

Hello Nordic Family. I hope this post finds you well. 

I am working on a project where I have connected multiple sensors to the NRF52840. I have collected all the sensor's data and saved it on the SD card as a CSV file. The file size is about 240MB which has 60 Hours of data from all the sensors(client requirements). Now I want to send this complete file at once from the SD card to the Mobile App in the fastest possible way. I have run the below program on my board

/cfs-file/__key/communityserver-discussions-components-files/4/8204.ble_5F00_app_5F00_uart_5F00_fatfs_5F00_send_5F00_sdk_5F00_16.0.0.zip.

but this program reads data one by one byte from the SD card and sends it byte by byte to the mobile app which I don't want. I want to send the whole file at once in the fastest possible way not byte by byte. Any help regarding this will be highly appreciated. 

Best Regards,

Sami 

[Håkon Alseth]

Hi,

 

sends it byte by byte to the mobile app

Buffer up the read out data from the SD card and send them in size of "BLE_NUS_MAX_DATA_LEN" instead?

 

Kind regards,

Håkon

[Samiulhaq]

Hi Hakon, thank you for your kind suggestion. Yes, I am sending the data in the size of "BLE_NUS_MAX_DATA_LEN"( 243byte) but it can't solve my problem because the data is sent in chunks of 243bytes per chunk and I want to send the complete file at once, not in chunks. My file size is about 240MB, will it be fully sent to the mobile App at once? If yes Can you please recommend to me the fastest possible way?

Best Regards,

Sami

[Samiulhaq]

Thank you Hakan, for the thorough explanation. 

It concluded that the problem is with the mobile app side(central ) not the peripheral, am I right? 

One last question and sorry for taking your precious time. I want to read a large file (about 240MB) from the SD card but the problem is that when I increase the #define FILE_SIZE_MAX (Maximum size in bytes of the file to be read from SDCARD) greater than 220000, while the following;

                static uint8_t file_buffer[FILE_SIZE_MAX]; 

              ff_result = f_read(&file, file_buffer, FILE_SIZE_MAX, (UINT *) &bytes_read);

 Then it gives the following error:

              .bss is too large to fit in RAM1 memory segment
              .heap is too large to fit in RAM1 memory segment
               section .heap overlaps absolute placed section .stack
               section .stack VMA [000000002003e000,000000002003ffff] overlaps section .bss VMA 

                 [0000000020002c64,0000000020224b08]         

I don't know how to solve this. Simply I want to read the whole file (234MB) from the SD card at once and then send it in chunks to the mobile app, Is this possible or do I have to read in chunks from the SD card too? 

Kind Regards,

Sami 

[Håkon Alseth]

Hi Sami,

Samiulhaq said:

It concluded that the problem is with the mobile app side(central ) not the peripheral, am I right? 

Yes, it is the central that drives the communication and essentially sets all the rules. A peripheral can ask for certain parameters, but it is the central that eventually grants permissions.

Samiulhaq said:

I don't know how to solve this. Simply I want to read the whole file (234MB) from the SD card at once and then send it in chunks to the mobile app, Is this possible or do I have to read in chunks from the SD card too? 

You are running out of RAM by trying to buffer ~220kB of data. You must handle this in a smaller buffer.

 

Kind regards,

Håkon

[Samiulhaq]

Thanks Hakan for helping.

Håkon Alseth said:

Yes, it is the central that drives the communication and essentially sets all the rules. A peripheral can ask for certain parameters, but it is the central that eventually grants permissions.

Okay Sir, I understand, and thank you for the thorough explanation. 

Håkon Alseth said:

You are running out of RAM by trying to buffer ~220kB of data. You must handle this in a smaller buffer.

Can you please refer me to a thread/blog where they have accessed and read large data from SD in small chunks/buffer? I have tried but I am unable to read the large file in a small buffer. I will be very thankful to you. 

Best Regards,

Sami

[Håkon Alseth]

Hi,

 

Samiulhaq said:

Can you please refer me to a thread/blog where they have accessed and read large data from SD in small chunks/buffer? I have tried but I am unable to read the large file in a small buffer. I will be very thankful to you. 

I do not think we have a blog/thread for this scenario.

What you need to handle is a arbitrary byte stream going into a buffer, which again is passed through your bluetooth link. You should not need to buffer many kilobytes of data from the SD card, you need to read SD-data out into your buffer and pass the data through the bluetooth link.

 

Kind regards,

Håkon

[Samiulhaq]

Hi Hakan, thanks for the suggestion.

Yes exactly I have done the same and now I am able to read data from the SD card and send it to the mobile app continuously.  Below code snippet is my approach;

#define FILE_SIZE_MAX                   BLE_NUS_MAX_DATA_LEN  // file size is same as the packet size
static uint8_t file_buffer[FILE_SIZE_MAX];
ff_result = f_open(&file, FILE_NAME, FA_READ);

for (;;)
    {
        if(file_send_to_peripheral)
        {

           file_actual_read_size = (f_size(&file));
           uint32_t remaining_bytes = file_actual_read_size;  
           uint16_t chunk_length = BLE_NUS_MAX_DATA_LEN;
           ret_code_t err_code;

            if(file_actual_read_size > 0)
            {

            while(remaining_bytes > 0)
             {

             ff_result = f_read(&file, file_buffer, sizeof(file_buffer), (UINT *) &bytes_read);
             err_code= ble_nus_data_send(&m_nus, file_buffer, &chunk_length, m_conn_handle);

                if (err_code != NRF_ERROR_RESOURCES)
                {
                    APP_ERROR_CHECK(err_code);

                   remaining_bytes -= chunk_length;
                   if(remaining_bytes < chunk_length)
                    {
                     chunk_length = remaining_bytes;
                    }
                    NRF_LOG_INFO("Remaining bytes to send: %d", remaining_bytes);

                }

                }
                }
            file_send_to_peripheral = false;
        }
        idle_state_handle();
    }
}

Is this the efficient way of reading from the SD card and sending it to the Mobile App?

Best Regards,

Sami

[Håkon Alseth]

Hi Sami,

 

That looks good in terms of algorithm, but this line:

if (err_code != NRF_ERROR_RESOURCES)

Should be expanded to ensure that you do not lose track in cases where the link is disconnected. You should check similar to what is done in the original ble_app_uart sample:

if ((err_code != NRF_ERROR_INVALID_STATE) &&
    (err_code != NRF_ERROR_RESOURCES) &&
    (err_code != NRF_ERROR_NOT_FOUND))

 

Kind regards,

Håkon

[Samiulhaq]

Hi Hakan sir, Thanks for the suggestion.

Do I just have to add the code snippet you provided? 

Best Regards,

Sami

[Håkon Alseth]

Hi Sami,

 

You should test with the proposed change and see if this works for your use-case.

 

Kind regards,

Håkon