Sending a CSV File over ble Gatt connection

[Thomas]

Hi,

I need to send data from a CSV file on the nordic board(Via fatfs library). I have presently gotten a Gatt service up and running.

I have a mobile application which, can see the gatt service and write to the characteristics. From a specific write command from the mobile device I want the nordic board to send the CSV file over.

I can presently read a single line of the CSV file formatted like this: (Time stamp, data1, data 2, data3, data 4, data 5).

What I need to implement is:

• Reading the whole file but, one or multiple lines at a time due to the size of the CSV
• Changing the read pointer of the SD card (So I know where to continue to read from)
• Send this data to the characteristic
• Read this characteristic data.

My code is based upon :  https://github.com/bjornspockeli/custom_ble_service_example

My CSV file size depends on the length of time I have been storing data from. It could be upto around 1-2mbytes. I beleive I need to set up some kind of notification service. Where I read a line(or several) of the SD card, set the GATT characteristic value to this. Then change the SD card pointer. Wait till the data is received by the mobile application. Then repeat until the file is read full.y

Note I am using SDK 16 NRF52840-dk PCA 10056 S140.

This post is similar and mentions using two characteristics one from commication from the mobile app and, one for receiving the data.

https://devzone.nordicsemi.com/f/nordic-q-a/53987/nrf52840-file-transfer-via-ble

[Edvin]

Hello,

So you want to send a CSV file rom the SD card to the connected mobile phone, right?

If I were you, I would start with the ble_app_uart example, and use ble_nus_data_send() to send your notifications. You can probably do it in the application you have started developing now. Just send the data using notifications.

I didn't really see any questions in your post. Have you attempted this? Did you get stuck somewhere?

Best regards,

Edvin

[Thomas]

Yes I want to send a CSV file from the SD card on the nordic board to a mobile phone. 

Is there an example of this?

I have not used the ble_app_uart example for this. However I do have my own custom service and Gatt connection working. My question is

• How do I change the pointer location for the SD card?
• How do I setup the notifications? As in how do I get them to work and change the value to be sent via this notification to the value of the CSV data?

Also does with ble_app_uart example have a custom service? As I will need to move alot of my functionality too it.

Also how does ble_nus_data_send work?

Is there some guidance for how to setup notifications?

Thanks

[Thomas]

How do I increase the size of what is sent via the notification? Each datum I have is 51 bytes. Therefore, I would like to be at least this large.

[Edvin]

Please refer to the ble_app_uart example. look at where it calls ble_nus_data_send, and what this function does.

Are you able to send an array of 51 bytes? If not, what is it that prevents you from doing so? what does sd_ble_gatts_hvx() return?

[Thomas]

I have not tried to send more than 8 bytes. And only with using the notification timer as well. As the parameter that gets passed into is uint8.  How do I change this so I can use a larger size than 51bytes?

I have looked at the ble_app_uart and, its a bit confusing. I do not need its uart functionality. Also I do not need blcm_link_ctx_get. Below is my update code.

uint32_t ble_cus_custom_value_update(ble_cus_t * p_cus, uint8_t custom_value)
{
    NRF_LOG_INFO("In ble_cus_custom_value_update. \r\n"); 
    if (p_cus == NULL)
    {
        return NRF_ERROR_NULL;
    }

    uint32_t err_code = NRF_SUCCESS;
    ble_gatts_value_t gatts_value;

    // Initialize value struct.
    memset(&gatts_value, 0, sizeof(gatts_value));

    gatts_value.len     = sizeof(uint8_t);
    gatts_value.offset  = 0;
    gatts_value.p_value = &custom_value;

    // Update database.
    err_code = sd_ble_gatts_value_set(p_cus->conn_handle,
                                      p_cus->custom_value_handles.value_handle,
                                      &gatts_value);
    if (err_code != NRF_SUCCESS)
    {
        return err_code;
    }

    // Send value if connected and notifying.
    if ((p_cus->conn_handle != BLE_CONN_HANDLE_INVALID)) 
    {
        ble_gatts_hvx_params_t hvx_params;

        memset(&hvx_params, 0, sizeof(hvx_params));

        hvx_params.handle = p_cus->custom_value_handles.value_handle;
        hvx_params.type   = BLE_GATT_HVX_NOTIFICATION;
        hvx_params.offset = gatts_value.offset;
        hvx_params.p_len  = &gatts_value.len;
        hvx_params.p_data = gatts_value.p_value;

        err_code = sd_ble_gatts_hvx(p_cus->conn_handle, &hvx_params);
        NRF_LOG_INFO("sd_ble_gatts_hvx result: %x. \r\n", err_code); 
    }
    else
    {
        err_code = NRF_ERROR_INVALID_STATE;
        NRF_LOG_INFO("sd_ble_gatts_hvx result: NRF_ERROR_INVALID_STATE. \r\n"); 
    }


    return err_code;
}

[Edvin]

uint8_t isn't 8 bytes. It is one byte.

I know that you don't need UART, but what I meant that you could have a look at is how ble_nus_data_send() uses an array of uint8_t.

Please check it out and try to implement something similar.

Also, you don't need to call your ble_cus_custom_value_update() or ble_nus_data_send() from the timeout. But you need to call it from somewhere. Probably, in your case, you want to call it with an array of data that you have just read out from your SD-card.

So (pseudocode) : 

static void sd_card_update(void)
{
    custom_value_update();
}

Please, please look at ble_app_uart. This characteristic has a value that is several bytes long, instead of only one. You need to set this in services_init() Look at how it is done in nus_init() in ble_app_uart.

[Thomas]

I have a function in my main thats called from a command via gatt called SEND_CSV. It crashes with sd_ble_gatts_hvx result: C. This comes from my custom function ble_cus_send_csv. Which I made by looking at the ble_app_uart and my previous update function. Do you know what i could be doing wrong. Also my data i am sending is 51bytes. It is a char array though which I presume is the same as a uint8_t. 

{
    static uint8_t data_array[BLE_NUS_MAX_DATA_LEN];
    static uint8_t index = 0;
    ret_code_t err_code;
    SD_CARD_Read_Line();//Gets a line of 51bytes of data
    NRF_LOG_INFO("D1: %s\r\n",ble_csv_data);
    
    uint16_t data_length =0;
    data_length =sizeof(ble_csv_data);
 
    err_code = ble_cus_send_csv(&m_cus, ble_csv_data,&data_length, m_conn_handle);//Causes error
           
    APP_ERROR_CHECK(err_code);
     
}

uint32_t ble_cus_send_csv(ble_cus_t * p_cus, uint8_t   * p_data, uint16_t  * p_length, uint16_t    conn_handle)
{
    ble_gatts_hvx_params_t hvx_params;
    NRF_LOG_INFO("Sending CSV.\r\n"); 
    if (p_cus == NULL)
    {
        return NRF_ERROR_NULL;
    }

    uint32_t err_code = NRF_SUCCESS;

    // Send value if connected and notifying.
    if ((p_cus->conn_handle != BLE_CONN_HANDLE_INVALID)) 
    {
        
        memset(&hvx_params, 0, sizeof(hvx_params));

        hvx_params.handle = p_cus->custom_value_handles.value_handle;
        hvx_params.p_data = p_data;
        hvx_params.p_len  = p_length;
        hvx_params.type   = BLE_GATT_HVX_NOTIFICATION;

        err_code = sd_ble_gatts_hvx(p_cus->conn_handle, &hvx_params);
        NRF_LOG_INFO("sd_ble_gatts_hvx result: %x. \r\n", err_code); 
    }
    else
    {
        err_code = NRF_ERROR_INVALID_STATE;
        NRF_LOG_INFO("sd_ble_gatts_hvx result: NRF_ERROR_INVALID_STATE. \r\n"); 
    }
    return err_code;
}

[Edvin]

So that means that sd_ble_gatts_hvx_result() returns 0x0C?

You can see the different return values for the softdevice calls either in the documentation on infocenter, or if you search for the softdevice call in your project (in this case sd_ble_gatts_hvx) and look for the line that says SVCALL(SD_BLE_GATTS_HVX, sd_ble_gatts_hvx(...))

I know I can give you the answer straight away, but it is better if you exercise this, so that you know what to do the next time you run into a similar issue. 

If you check the definition for the one of the possible return values, e.g. NRF_SUCCESS, you will see that it is defined in nrf_error.h, with value 0. (NRF_ERROR_BASE_NUM = 0). Most of the return values are listed in this file, but you see that there are some other bases, such as NRF_ERROR_SDM_BASE_NUM, which is used to list errors in another file. Those would be the error codes starting with 0x1000, e.g. 0x1001.

So 0x0C = 12(dec), which is NRF_ERROR_DATA_SIZE. From the description of sd_ble_gatts_hvx() in ble_gatts.h, it says:

* @retval ::NRF_ERROR_DATA_SIZE Invalid data size(s) supplied.

This means that the data size(s) is not valid. You say that you try to send 51 bytes. 

1: What is your MTU size? 

2: When you initialize your characteristic, what is:

add_char_params.max_len     = ?

One of them (or both) is too small to send 51 bytes.

The characteristic's max_len is quite easy to fix. To see how to increase the MTU, please refer to the ble_app_uart example's gatt_init() and gatt_evt_handler() functions.

Best regards,

Edvin

[Thomas]

Ok thanks for explaining how to look up the errors I will try to do so for future reference.

I do not know what my MTU size is but If i need to increase it I shall.

I dont have have add_char_params.max_len but, do have a att_char_value.max_len this is sizeof(uint8_t).

I presume its this that needs changing. If so what to? (BLE_NUS_MAX_DATA_LEN?)

My characteristic add code is below.

static uint32_t custom_value_char_add(ble_cus_t * p_cus, const ble_cus_init_t * p_cus_init)
{
    uint32_t            err_code;
    ble_gatts_char_md_t char_md;
    ble_gatts_attr_md_t cccd_md;
    ble_gatts_attr_t    attr_char_value;
    ble_uuid_t          ble_uuid;
    ble_gatts_attr_md_t attr_md;

    // Add Custom Value characteristic
    memset(&cccd_md, 0, sizeof(cccd_md));

    //  Read  operation on cccd should be possible without authentication.
    BLE_GAP_CONN_SEC_MODE_SET_OPEN(&cccd_md.read_perm);
    BLE_GAP_CONN_SEC_MODE_SET_OPEN(&cccd_md.write_perm);
    
    cccd_md.write_perm = p_cus_init->custom_value_char_attr_md.cccd_write_perm;
    cccd_md.vloc       = BLE_GATTS_VLOC_STACK;

    memset(&char_md, 0, sizeof(char_md));

    char_md.char_props.read   = 1;
    char_md.char_props.write  = 1;
    char_md.char_props.notify = 1; 
    char_md.p_char_user_desc  = NULL;
    char_md.p_char_pf         = NULL;
    char_md.p_user_desc_md    = NULL;
    char_md.p_cccd_md         = &cccd_md; 
    char_md.p_sccd_md         = NULL;
		
    ble_uuid.type = p_cus->uuid_type;
    ble_uuid.uuid = CUSTOM_VALUE_CHAR_UUID;

    memset(&attr_md, 0, sizeof(attr_md));

    attr_md.read_perm  = p_cus_init->custom_value_char_attr_md.read_perm;
    attr_md.write_perm = p_cus_init->custom_value_char_attr_md.write_perm;
    attr_md.vloc       = BLE_GATTS_VLOC_STACK;
    attr_md.rd_auth    = 0;
    attr_md.wr_auth    = 0;
    attr_md.vlen       = 0;

    memset(&attr_char_value, 0, sizeof(attr_char_value));

    attr_char_value.p_uuid    = &ble_uuid;
    attr_char_value.p_attr_md = &attr_md;
    attr_char_value.init_len  = sizeof(uint8_t);
    attr_char_value.init_offs = 0;
    attr_char_value.max_len   = sizeof(uint8_t);

    err_code = sd_ble_gatts_characteristic_add(p_cus->service_handle, &char_md,
                                               &attr_char_value,
                                               &p_cus->custom_value_handles);
    if (err_code != NRF_SUCCESS)
    {
        return err_code;
    }

    return NRF_SUCCESS;
}

I also dont have a bunch of the other setup stuff the ble_app_uart does for it characteristic setup. Will my setup with with changing that value?

 memset(&add_char_params, 0, sizeof(add_char_params));
    add_char_params.uuid              = BLE_UUID_NUS_TX_CHARACTERISTIC;
    add_char_params.uuid_type         = p_nus->uuid_type;
    add_char_params.max_len           = BLE_NUS_MAX_TX_CHAR_LEN;
    add_char_params.init_len          = sizeof(uint8_t);
    add_char_params.is_var_len        = true;
    add_char_params.char_props.notify = 1;

    add_char_params.read_access       = SEC_OPEN;
    add_char_params.write_access      = SEC_OPEN;
    add_char_params.cccd_write_access = SEC_OPEN;

    return characteristic_add(p_nus->service_handle, &add_char_params, &p_nus->tx_handles);

Thanks for being quick and responsive it helps alot.

Edit I changed 

//attr_char_value.max_len = sizeof(uint8_t);
attr_char_value.max_len = BLE_NUS_MAX_TX_CHAR_LEN;

and the crash still persisted.

Hovering over : NRF_SDH_BLE_GATT_MAX_MTU_SIZE it states 23.

To change the MTU size can I go into the sdk_config.h and change it from 23 to 247 as is done in the uart example?

[Edvin]

Yes. It looks like the attr_char_value.max_len is the parameter you want to change. If you look in ble_app_uart, then this is probably the same pointer passed into sd_ble_gatts_characteristic add().

Try to set NRF_SDH_BLE_GATT_MAX_MTU_SIZE to 247, and also check that you have the gatt_init() and gatt_evt_handler() in your function. Does gatt_evt_handler() get called? If so, check the log output from this function. What does it say?

Hint: Try to not call sd_ble_gatts_hvx() before the gatt_evt_handler() is called. Basically, you can't send a notification larger than your MTU size, and the MTU isn't updated before you get this callback. Before that, the maximum payload length you can use is 20 (23 - OPCODE_LENGTH - HANDLE_LENGTH = 23 - 1 -2 = 20).

[Thomas]

When you say have gatt_init() and gatt_evt_handler in my function what do you mean by this?

I call gatt_init in my main before the main loop runs. I  did add the gatt event handler too.

I changed the mtu size to 247 and NRF_SDH_BLE_GAP_DATA_LENGTH from 27 to 251. As this is what is done inside the app_uart example.

Then I programmed it and it crashed. It needed a change in ram start location and ram size.

Then I run it and it goes but, when I call my function to send 51 bytes of data i get :sd_ble_gatts_hvx result: 3401.

Ok sorry this is a dug with my android application. So I have a bug where I need to press notify twice for it to send the notification request to the nordic board. Then I can submit my gatt request to send the data. I pressed it twice and I cansee a line of my CSV on my phone!

The output however is far too long why is this?

The below image is my application. And the top line is the first of my CSV. The rest is junk and I dont know why nor do i want it. How do I get rid of it?

[Edvin]

I can't say what your mobile application outputs. Do you send more than one packet from the nRF? Because the app is printing far more than 51 bytes. Did you check what the return message 0x3401 means?

Hint: Check all the return values found in the description of sd_ble_gatts_hvx(). If you use sdk_16.0.0 this is found in ble_gatts.h line 573-636. Then a short google search reveals threads like this one:

https://devzone.nordicsemi.com/f/nordic-q-a/17688/ble_error_gatts_sys_attr_missing-from-sd_ble_gatts_hvx

Although I give you the answer here, please do the exercise, so that you see how you find the cause of these return values.

Since you print out a lot more than the 51 bytes that you send, I suspect that you just print out random bytes in the RAM in your phone app, and not the actual length of the packet.