Getting error NRF_ERROR_RESOURCES when using more then one notification and the Connection crashes

[danny habot]

Hi, 

I am using the MCU on a board with sensors that are sampled in a very high rate (~100Hz). 

I am having issues when trying to send the data with notification on more then one characteristic. 

The information is sent correctly for a short time (~30 seconds) and then the connection get stuck and unresponsive. 

When the connection "crashes" i start getting NRF_ERROR_RESOURCES  from sd_ble_gatts_hvx(*pointer_conn_handle, &hvx_params);

Initially i thought it is because the queue was very small (the default value of 1) however even after changing it to 10 it still does it.

I use the following code to change it: 

ble_cfg_t ble_cfg;
memset(&ble_cfg, 0, sizeof ble_cfg);
ble_cfg.conn_cfg.conn_cfg_tag = APP_BLE_CONN_CFG_TAG;
ble_cfg.conn_cfg.params.gatts_conn_cfg.hvn_tx_queue_size = 10;
err_code = sd_ble_cfg_set(BLE_CONN_CFG_GATTS, &ble_cfg, ram_start);

I have 2 questions:

1. how can i config the system to allow much more bandwidth as i will need several notification simultaneously?

2. how can i avoid crashes in case of overload? i saw this post but couldn't understand how to use it on my project as the data structs are different.

I am using NRF52832 (custom board) with freeRTOS (based on the HRS freeRTOS example) SDK15, and SD 6.0

BR, 

Danny

[Sigurd]

Hi,

danny habot said:

1. how can i config the system to allow much more bandwidth as i will need several notification simultaneously?

Please try to increase the event length(NRF_SDH_BLE_GAP_EVENT_LENGTH). See this post.

danny habot said:

2. how can i avoid crashes in case of overload? i saw this post but couldn't understand how to use it on my project as the data structs are different.

The quick and easy solution is just to wait for the function to return something else than NRF_ERROR_RESOURCES.

err_code = your_send_function / sd_ble_gatts_hvx

if ( (err_code != NRF_ERROR_INVALID_STATE) && (err_code != NRF_ERROR_RESOURCES) &&
	 (err_code != NRF_ERROR_NOT_FOUND) )
{
	APP_ERROR_CHECK(err_code);
}
} while (err_code == NRF_ERROR_RESOURCES);

You could also set a flag when you get NRF_ERROR_RESOURCES, and wait for the BLE_GATTS_EVT_HVN_TX_COMPLETE event to occur, and try again. This is done in the ble_app_att_mtu_throughput example. But try to increase the event_length and do the easy solution first.

Let me know how it goes.

BR,

Sigurd

[danny habot]

Hi Sigurd,

I have increased the NRF_SDH_BLE_GAP_EVENT_LENGTH from 6 to 100 but still no luck i still get NRF_ERROR_RESOURCES after several seconds. 

In addition i tried to add the while loop in order to retry the sd_ble_gatts_hvx it get stuck in the loop and never recover. 

It feels like the resource shortage cause the soft device to stop responding. in addition, i don't know if it is related, but when i connect to the device the sd_ble_gatts_hvx function returns 13313 which i couldn't find the meaning of..

i am sending 200 packets per second which shouldn't be a too large amount. 

[danny habot]

Hi bjorn, 

Thanks for the update. 

regarding your questions:

1. we also based our project on the ble_app_hrs_freertos  so it is good to hear that the issue is not with the code that we added but with the original example.

2. the logs are disable in our project as we use ITM for printf instead of using the NRF_LOG module. so this does not help us unfortunately. However it means that it might happen when the system is more "loaded" with tasks as our project has many periodic activities that might take the resources that the log module take in your system. 

3. I am not checking if I got the event BLE_GATTS_EVT_HVN_TX_COMPLETE  however i did try to do it in my debugging process without success. can you send me a code snippet with this condition so that i would add it as a precaution? my current code for notification is:

case SENSOR_VAL_UUID:
if (len == SENSOR_VAL_SIZE)
{
//check if notification/indication is configured and send the data if it does.
if(*pointer_conn_handle != BLE_CONN_HANDLE_INVALID)
{
ble_gatts_hvx_params_t hvx_params;
memset(&hvx_params, 0, sizeof(hvx_params));

hvx_params.handle = Sensor_val_handles.value_handle;
hvx_params.type = BLE_GATT_HVX_NOTIFICATION;
hvx_params.offset = 0;
hvx_params.p_len = &value.len;
hvx_params.p_data = (uint8_t*)data;

sd_ble_gatts_hvx(*pointer_conn_handle, &hvx_params);
}
sd_ble_gatts_value_set(*pointer_conn_handle, Sensor_val_handles.value_handle, &value);
}

 

As you can see in previous posts i tried other precautions as well but to no avail so i am a little skeptic.

 

[bjorn-spockeli]

Hi Danny, 

how many tasks do you create in your project and which priority have you given them? I am starting to think that this is not a bug, but rather a consquence of how the FreeRTOS scheduler is configured and which priority the different tasks are given. I am basing this on another post, here is the link, on DevZone. 

FreeRTOS uses prioritised preemptive scheduling with time slicing. That means the RTOS scheduler will always run the highest priority task that is in the Ready state, and will switch between tasks of equal priority on every RTOS tick interrupt. If time slicing is disabled then the RTOS scheduler will still run the highest priority task that is in the Ready state, but will not switch between tasks of equal priority just because a tick interrupt has occurred.

Time slicing appears to be disabled in the ble_app_hrs_freertos example, so could you try to enable time slicing by setting the following define

#define configUSE_TIME_SLICING 1

in the /external/freertos/config folder? I have enabled it in my code and i have ran 30min+ traces and have not seen the issue occuring. 

Best regards

Bjørn 

[danny habot]

Hi bjorn, 

I also think that it is somewhat related to the FreeRTOS configuration however the task handling was one of the first things that i checked. 

My task run with priority 2 which shouldn't starve the SD. in addition, because the SD is not sending anything new it shouldn't starve my task as well (and it doesn't because the UI is still working correctly) . 

I tried to config it to use time slicing but it has no effect on the behavior i see. Verified it again now. 

Did you change anything else in the example? This is very weird because we have based our project on it. 

How can we continue? would it help if i send you a copy of my project with instructions how to reproduce? 

[bjorn-spockeli]

Hi Danny, can you check which priority the timer task(s) are given in the FreeRTOSConfig.h, i.e. configTIMER_TASK_PRIORITY, is it set to 2 as well?

I tried increasing the priority of the main (ble)task from 2 to 3, i.e. 

xTaskCreate(softdevice_task,
"BLE",
NRF_BLE_FREERTOS_SDH_TASK_STACK,
p_context,
3,
&m_softdevice_task);

and that seemed to impreove things on my end. Can you try that?

[danny habot]

Hi Bjorn, 

I have added more priority levels so now i have one for each task with the softdevice on the highest. It seems to improve this. I am still running tests and if all goes well i will close the ticket. 

Thanks for all the help and finger crossed i will have good news soon