Sending a CSV File over ble Gatt connection

[Thomas]

Hi,

I need to send data from a CSV file on the nordic board(Via fatfs library). I have presently gotten a Gatt service up and running.

I have a mobile application which, can see the gatt service and write to the characteristics. From a specific write command from the mobile device I want the nordic board to send the CSV file over.

I can presently read a single line of the CSV file formatted like this: (Time stamp, data1, data 2, data3, data 4, data 5).

What I need to implement is:

• Reading the whole file but, one or multiple lines at a time due to the size of the CSV
• Changing the read pointer of the SD card (So I know where to continue to read from)
• Send this data to the characteristic
• Read this characteristic data.

My code is based upon :  https://github.com/bjornspockeli/custom_ble_service_example

My CSV file size depends on the length of time I have been storing data from. It could be upto around 1-2mbytes. I beleive I need to set up some kind of notification service. Where I read a line(or several) of the SD card, set the GATT characteristic value to this. Then change the SD card pointer. Wait till the data is received by the mobile application. Then repeat until the file is read full.y

Note I am using SDK 16 NRF52840-dk PCA 10056 S140.

This post is similar and mentions using two characteristics one from commication from the mobile app and, one for receiving the data.

https://devzone.nordicsemi.com/f/nordic-q-a/53987/nrf52840-file-transfer-via-ble

[Edvin]

Hello,

So you want to send a CSV file rom the SD card to the connected mobile phone, right?

If I were you, I would start with the ble_app_uart example, and use ble_nus_data_send() to send your notifications. You can probably do it in the application you have started developing now. Just send the data using notifications.

I didn't really see any questions in your post. Have you attempted this? Did you get stuck somewhere?

Best regards,

Edvin

[Thomas]

Yes I want to send a CSV file from the SD card on the nordic board to a mobile phone. 

Is there an example of this?

I have not used the ble_app_uart example for this. However I do have my own custom service and Gatt connection working. My question is

• How do I change the pointer location for the SD card?
• How do I setup the notifications? As in how do I get them to work and change the value to be sent via this notification to the value of the CSV data?

Also does with ble_app_uart example have a custom service? As I will need to move alot of my functionality too it.

Also how does ble_nus_data_send work?

Is there some guidance for how to setup notifications?

Thanks

[Thomas]

So therefore, My bullet point code shown above works via queuing?

This means then I don't need a flag to make it loop 10 times a second.

Therefore, I just need to queue up all my data?

[Thomas]

I have tried adding the parts for BLE_GATTS_EVT_HVN_TX_COMPLETE and setting a volatile bool to true when this event runs. However, I then try to queue up another packet and it still returns NRF_ERROR_RESOURCES.

When I store this data to my phone I receive 533 data out of 600 in my CSV file. Below is my loop code. What is incorrect here. Not my notifyflag runs once every 10ms. Without it I store 524 data.

Note I terminate the reading with an end of file packet.(Just an X)

void SEND_CSV()//Send the CSV file chop off when it gets to \r\n
{
    ret_code_t err_code;
    uint16_t data_length = 0;
    uint16_t offset_length = 11;
    data_length =(sizeof(ble_csv_data)-offset_length);//Only send 40 bytes

    if(End_of_file == 1)//End of the file
    {
        uint8_t End_of_file_data[1]={0x78}; /*End of file*/
        uint16_t end_of_file_length = sizeof(End_of_file_data);
        err_code = ble_cus_send_csv(&m_cus, End_of_file_data,&end_of_file_length, m_conn_handle);
        APP_ERROR_CHECK(err_code);
        End_of_file =0;//Allows it to be re-read
    }
    else
    {
        while(End_of_file != 1)//Loop this till end of file is reached
        {
        if(notify_flag == true)
            {
                SD_CARD_Read_Line();//Gets a line of 51bytes of data
                err_code = ble_cus_send_csv(&m_cus, ble_csv_data,&data_length, m_conn_handle);
                if(err_code == NRF_ERROR_RESOURCES)
                {
                    //Wait
                    while(Ble_gatts_evt_hvn_tx_complete_flag == false)//Gets set to true from interrupt
                    {
                        //Wait here
                    }
                    Ble_gatts_evt_hvn_tx_complete_flag = false;
                    err_code = ble_cus_send_csv(&m_cus, ble_csv_data,&data_length, m_conn_handle);//Error code has passed re send this data
                    //APP_ERROR_CHECK(err_code);//This still returns nrf_error_resources
                }
                
               
              notify_flag = false;
            }
        }
    }
 
     
}

[Edvin]

If the new packet is longer than the one that was ACKed (in the TX_COMPLETE event), then this still may return NRF_ERROR_RESOURCES. The queue isn't a number of packets, but number of bytes. Wait for the next TX_COMPLETE event if it still returns NRF_ERROR_RESOURCES.

 

Thomas said:

When I store this data to my phone I receive 533 data out of 600 in my CSV file.

 Do you have any idea of why you are missing some files? When you called ble_cus_send_csv with these lines, did it return NRF_SUCCESS, or something else?

[Thomas]

Ok So i need to wait for BLE_GATTS_EVT_HVN_TX_COMPLETE twice?

When I get 533 out of 600. I does have the NRF_ERROR_RESOURCES after I have waited for the flag from my BLE_GATTS_EVT_HVN_TX_COMPLETE flag.

case BLE_GATTS_EVT_HVN_TX_COMPLETE:
              Ble_gatts_evt_hvn_tx_complete_flag = true;//Set the flag
              break;

[Edvin]

If it returns NRF_ERROR_RESOURCES, the packet isn't lost. It is never queued. Only packets that are queued with the return value NRF_SUCCESS will be sent. The last snippet you sent will not work, because if it returns NRF_ERROR_RESOURCES then this packet is lost in your case, because I assume your function SD_CARD_Read_Line() will read the next line each time. Do you see where I'm going with this?

Why don't you try to start over with this state machine. Set a variable saying whether or not the last line you read using SD_CARD_Read_Line() was successfully sent or not, and only read a new line from your file when the previous packet was sent successfully.

volatile bool tx_complete_flag = true;
volatile bool last_line_queued = true;

event TX_COMPLETE:
tx_complete_flag = true;

...

send_data(void)
{
    ret_code_t err_code = NRF_SUCCESS;
    
    while (err_code = NRF_SUCCESS)
    {
        err_code = ble_cus_data_send(...);
        if (last_line_queued == true)
        {
            last_line_queued = false;
            SD_CARD_Read_Line();
        }
        if (err_code == NRF_SUCCESS)
        {
            last_line_queued = true;
        }
    }
    if (err_code != NRF_ERROR_RESOURCES)
    {
        APP_ERROR_CHECK(err_code);
    }
    else
    {
        tx_complete_flag = false;
    }
}

This is pseudo code, but hopefully you get the idea.

Best regards,

Edvin

[Thomas]

Thanks for the psudo code. I have adapted it and added to my SD card reading code so I read a block of data and then send 60 lines of data before re reading the data. This code does not seem to crash when processing upto 1000 data.

However, it crashes when I try to send more data I've tried to test 1800. I am not sure why this is. 

For transferring 1000 data (CSV lines) it takes about 30seconds. When I try to send 1800 it gets stuck and when I cancel the connection it crashes. I am not sure why.

void NEW_SEND_CSV()//Send the CSV file chop off when it gets to \r\n
{
    ret_code_t err_code;
   
    uint16_t data_length = 0;
    uint16_t offset_length = 11;
    data_length =(sizeof(ble_csv_data)-offset_length);//Only send 40 bytes

    if(End_of_file == 1)//End of the file
    {
        uint8_t End_of_file_data[1]={0x78}; /*End of file*/
        uint16_t end_of_file_length = sizeof(End_of_file_data);
        err_code = ble_cus_send_csv(&m_cus, End_of_file_data,&end_of_file_length, m_conn_handle);
        APP_ERROR_CHECK(err_code);
    }
    else
    {
        while(End_of_file != 1)//Loop this till end of file is reached
        {
            {
              SD_CARD_READ_DATA();//Read 1 min of data
              #define  LINE_SIZE 51//Single line of data
              #define  DATA_SIZE 3060//1 min of data
              for(int index = 0; index < DATA_SIZE; index += LINE_SIZE)
              {
                    uint8_t data_to_send[LINE_SIZE];//Create an array of 51 bytes to store a single line
                    memcpy(data_to_send, &ble_csv_data_minute[index],LINE_SIZE);//Copy 51 bytes at a time
                    //Send code
                    err_code = ble_cus_send_csv(&m_cus, data_to_send,&data_length, m_conn_handle);
                    if(err_code == NRF_ERROR_RESOURCES)
                    {
                        //Wait
                        while(Ble_gatts_evt_hvn_tx_complete_flag == false)//Gets set to true from interrupt
                        {
                            //Wait here
                        }
                        Ble_gatts_evt_hvn_tx_complete_flag = false;
                        err_code = ble_cus_send_csv(&m_cus, data_to_send,&data_length, m_conn_handle);//Error code has passed re send this data
                        if(err_code == NRF_ERROR_RESOURCES)
                        {
                            while(Ble_gatts_evt_hvn_tx_complete_flag == false)//Gets set to true from interrupt
                            {
                                //Wait here
                            }
                            Ble_gatts_evt_hvn_tx_complete_flag = false;
                            err_code = ble_cus_send_csv(&m_cus, data_to_send,&data_length, m_conn_handle);//Error code has passed re send this data
                        }
                        APP_ERROR_CHECK(err_code);
                    }
                }
            }
        }
    }    
}

 

[Edvin]

Thomas said:

When I try to send 1800 it gets stuck and when I cancel the connection it crashes. I am not sure why.

 I assume it is an APP_ERROR_CHECK(err_code) that receives an err_code != 0. Which one is it? Does your log say something like "fatal error"? If you define DEBUG in your preprocessor defines, the log should say where it is. So where does it point to?

When you find out where it points to, see if you can figure out why the function call that returned the err_code returned that err_code. 

You need to check this yourself. If I just figure out each and every case for you, I basically have to develop the entire application for you. It is better (and faster) if you learn how to deal with the error handler yourself.