Data loss while sending large data by notifications with custom BLE service

[Tai]

Hello everyone,

A nRF52832 with SDK 15.0 was used for my application. I am trying to send a large amount of data via BLE with a custom BLE service. I extended MTU size so that it can meet my demand in both Client and Server. My Android app was developed to get the data. To check whether data sent by BLE device are lost or not, I added an index after a data package of 48-byte long. The following image is about index I got. 

There are 1, 2 or even 3 packages lost during data transmission. In this case, I tried to set the value of interval connection as low as possible (9ms). If I increased this interval as I have tested (24ms), there are more data packages lost.

Interestingly, if I use NUS BLE service instead and UART Android app to collect data,  this issue didn't happen and I can send up to 60 byte data per package without data loss. I double-checked my update data in custom BLE service, it is similar to that in NUS BLE service. 

Thus, are there any suggestion for me to solve this issue on the custom BLE service?

Thanks!

[hmolesworth]

I would suggest that the code is not handling the various busy indications and so the packet is never sent; for example the caller should retry the packet if NRF_ERROR_RESOURCES or NRF_ERROR_BUSY is the returned err_code, otherwise the packet is simply never transmitted because the BLE link is unable to accept the packet. This varies from one central device to another since the BLE stacks on the central can throttle transmissions as they wish (and they do).

/**@brief Notify or Indicate an attribute value.
 *
 * @details This function checks for the relevant Client Characteristic Configuration descriptor value to verify that the relevant operation
 *          (notification or indication) has been enabled by the client. It is also able to update the attribute value before issuing the PDU, so that
 *          the application can atomically perform a value update and a server initiated transaction with a single API call.
 *
 * @note    The local attribute value may be updated even if an outgoing packet is not sent to the peer due to an error during execution.
 *          The Attribute Table has been updated if one of the following error codes is returned: @ref NRF_ERROR_INVALID_STATE, @ref NRF_ERROR_BUSY,
 *          @ref NRF_ERROR_FORBIDDEN, @ref BLE_ERROR_GATTS_SYS_ATTR_MISSING and @ref NRF_ERROR_RESOURCES.
 *          The caller can check whether the value has been updated by looking at the contents of *(@ref ble_gatts_hvx_params_t::p_len).
 *
 * @note    Only one indication procedure can be ongoing per connection at a time.
 *          If the application tries to indicate an attribute value while another indication procedure is ongoing,
 *          the function call will return @ref NRF_ERROR_BUSY.
 *          A @ref BLE_GATTS_EVT_HVC event will be issued as soon as the confirmation arrives from the peer.
 *
 * @note    The number of Handle Value Notifications that can be queued is configured by @ref ble_gatts_conn_cfg_t::hvn_tx_queue_size
 *          When the queue is full, the function call will return @ref NRF_ERROR_RESOURCES.
 *          A @ref BLE_GATTS_EVT_HVN_TX_COMPLETE event will be issued as soon as the transmission of the notification is complete.
 *
 * @note    The application can keep track of the available queue element count for notifications by following the procedure below:
 *          - Store initial queue element count in a variable.
 *          - Decrement the variable, which stores the currently available queue element count, by one when a call to this function returns @ref NRF_SUCCESS.
 *          - Increment the variable, which stores the current available queue element count, by the count variable in @ref BLE_GATTS_EVT_HVN_TX_COMPLETE event.
 *
 * @events
 * @event{@ref BLE_GATTS_EVT_HVN_TX_COMPLETE, Notification transmission complete.}
 * @event{@ref BLE_GATTS_EVT_HVC, Confirmation received from the peer.}
 * @endevents
 *
 * @mscs
 * @mmsc{@ref BLE_GATTS_HVX_SYS_ATTRS_MISSING_MSC}
 * @mmsc{@ref BLE_GATTS_HVN_MSC}
 * @mmsc{@ref BLE_GATTS_HVI_MSC}
 * @mmsc{@ref BLE_GATTS_HVX_DISABLED_MSC}
 * @endmscs
 *
 * @param[in] conn_handle      Connection handle.
 * @param[in,out] p_hvx_params Pointer to an HVx parameters structure. If @ref ble_gatts_hvx_params_t::p_data
 *                             contains a non-NULL pointer the attribute value will be updated with the contents
 *                             pointed by it before sending the notification or indication. If the attribute value
 *                             is updated, @ref ble_gatts_hvx_params_t::p_len is updated by the SoftDevice to
 *                             contain the number of actual bytes written, else it will be set to 0.
 *
 * @retval ::NRF_SUCCESS Successfully queued a notification or indication for transmission, and optionally updated the attribute value.
 * @retval ::BLE_ERROR_INVALID_CONN_HANDLE Invalid Connection Handle.
 * @retval ::NRF_ERROR_INVALID_STATE One or more of the following is true:
 *                                   - Invalid Connection State
 *                                   - Notifications and/or indications not enabled in the CCCD
 *                                   - An ATT_MTU exchange is ongoing
 * @retval ::NRF_ERROR_INVALID_ADDR Invalid pointer supplied.
 * @retval ::NRF_ERROR_INVALID_PARAM Invalid parameter(s) supplied.
 * @retval ::BLE_ERROR_INVALID_ATTR_HANDLE Invalid attribute handle(s) supplied. Only attributes added directly by the application are available to notify and indicate.
 * @retval ::BLE_ERROR_GATTS_INVALID_ATTR_TYPE Invalid attribute type(s) supplied, only characteristic values may be notified and indicated.
 * @retval ::NRF_ERROR_NOT_FOUND Attribute not found.
 * @retval ::NRF_ERROR_FORBIDDEN The connection's current security level is lower than the one required by the write permissions of the CCCD associated with this characteristic.
 * @retval ::NRF_ERROR_DATA_SIZE Invalid data size(s) supplied.
 * @retval ::NRF_ERROR_BUSY For @ref BLE_GATT_HVX_INDICATION Procedure already in progress. Wait for a @ref BLE_GATTS_EVT_HVC event and retry.
 * @retval ::BLE_ERROR_GATTS_SYS_ATTR_MISSING System attributes missing, use @ref sd_ble_gatts_sys_attr_set to set them to a known value.
 * @retval ::NRF_ERROR_RESOURCES Too many notifications queued.
 *                               Wait for a @ref BLE_GATTS_EVT_HVN_TX_COMPLETE event and retry.
 * @retval ::NRF_ERROR_TIMEOUT There has been a GATT procedure timeout. No new GATT procedure can be performed without reestablishing the connection.
 */
SVCALL(SD_BLE_GATTS_HVX, uint32_t, sd_ble_gatts_hvx(uint16_t conn_handle, ble_gatts_hvx_params_t const *p_hvx_params));

[Tai]

Hi hmolesworth,

I am referring a handle from NUS BLE app and they handle each value in the data array. In my case, how should I handle each package which I added an index in the end of each package?  

In my project, I read 12 bytes in every 4ms from a sensor, the data will be stored in an array with 49 bytes long ( last byte of this array is index number). Once this array is full, I send them via BLE which means I call updating data function every 16ms so that I can meet the interval connection range for BLE. The following code is what I tried to add handle. 

"temp == noData" means if the array with 49 bytes long is full.

	if (temp == noData)
		{	
			do 
				{
					err_code = ble_cus_custom_value_update(&m_cus, dataSendBLE, noData);
					temp = 0;
					if ( (err_code != NRF_ERROR_INVALID_STATE) && (err_code != NRF_ERROR_RESOURCES) &&
							 (err_code != NRF_ERROR_NOT_FOUND) )
					{
							APP_ERROR_CHECK(err_code);
					}
				} while (err_code == NRF_ERROR_RESOURCES);
			
		}

I still haven't got correct packages, is the anything wrong in my code?

[hmolesworth]

You missed one: NRF_ERROR_BUSY 

	if (temp == noData)
		{	
			do 
				{
					err_code = ble_cus_custom_value_update(&m_cus, dataSendBLE, noData);
					temp = 0;
					if ( (err_code != NRF_ERROR_INVALID_STATE) && (err_code != NRF_ERROR_RESOURCES) &&
							 (err_code != NRF_ERROR_NOT_FOUND) && (err_code ! NRF_ERROR_BUSY))
					{
							APP_ERROR_CHECK(err_code);
					}
				} while ((err_code == NRF_ERROR_RESOURCES) || (err_code == NRF_ERROR_BUSY) );
			
		}

I tend to just use (err_code == NRF_SUCCESS) to know a packet was sent ok, rather than spinning in a loop, but it's better to be explicit.  Typically - though not required - you would use a queue or other buffer for the sensor packets, since sometimes BLE will be unavailable for significant periods and I assume data must not be lost. That way the code doesn't just spin but any packet that wasn't sent will be tried again on the next interval.

Also I think you want temp = 0; to only happen on a successful transmit ..

[Tai]

The problem hasn't been fixed yet. Yeah, you're right, I need to use a better way ( data structure) to handle a large amount of data from sensor. I'll look at it.

Btw, when I use same way collecting data from sensor with NUS BLE service, data loss never happens even I increase the length of package or interval connection. I am reading this post, in UART handler, what it did in the following code is similar to what you suggested to me.

void uart_event_handle(app_uart_evt_t * p_event)
{
    static uint8_t data_array[BLE_NUS_MAX_DATA_LEN];
    static uint8_t index = 0;
    uint32_t       err_code;

    switch (p_event->evt_type)
    {
        case APP_UART_DATA_READY:
            UNUSED_VARIABLE(app_uart_get(&data_array[index]));
            index++;

            if ((data_array[index - 1] == '\n') || (index >= (m_ble_nus_max_data_len)))
            {
                NRF_LOG_DEBUG("Ready to send data over BLE NUS\r\n");
                NRF_LOG_HEXDUMP_DEBUG(data_array, index);

                do
                {
                    err_code = ble_nus_string_send(&m_nus, data_array, index);
                    if ( (err_code != NRF_ERROR_INVALID_STATE) && (err_code != NRF_ERROR_BUSY) )
                    {
                        APP_ERROR_CHECK(err_code);
                    }
                } while (err_code == NRF_ERROR_BUSY);

                index = 0;
            }
            break;

        case APP_UART_COMMUNICATION_ERROR:
            APP_ERROR_HANDLER(p_event->data.error_communication);
            break;

        case APP_UART_FIFO_ERROR:
            APP_ERROR_HANDLER(p_event->data.error_code);
            break;

        default:
            break;
    }
}

Someone also suggested that "either call ble_nus_data_send() in a loop until the function returns NRF_SUCCESS, or wait for the BLE_GATTS_EVT_HVN_TX_COMPLETE and then send the packet". Should I try to use BLE_GATTS_EVT_HVN_TX_COMPLETE? 

[Tai]

hmolesworth said:

Also I think you want temp = 0; to only happen on a successful transmit ..

Yeah, you're right. For this purpose, I should put it outside the do..while loop then.