Error 22 (0x16) GATT_CONN TERMINATE LOCAL HOST ISSUE

[Siddhesh]

Platform NRF52832 SoftDevice S132 V6.1 SDK 15.0 FreeRTOS implemented

Hi,
In our application, once the mobile app is connected to the peripheral, it makes continuous writes to a characteristic every 1 second and then forwards the message to a different chip over UART. We expect this behavior to work for at least 2 hours.

Here are the connection parameters we are using
 MIN_CONN_INTERVAL MSEC_TO_UNITS(500, UNIT_1_25_MS)
MAX_CONN_INTERVAL MSEC_TO_UNITS(2000, UNIT_1_25_MS) 
SLAVE_LATENCY 0 
CONN_SUP_TIMEOUT MSEC_TO_UNITS(10000, UNIT_10_MS) 

However after some time (ranging from 3 minutes to 25 minutes at random), we see an error on nrf_connect app stating Error 22 (0x16) GATT_CONN_TERMINATE_LOCAL_HOST and the phone is disconnected. But on the dev board, we do not see any disconnect event and the board becomes unresponsive to any BLE command and remains stuck. The only way to recover is pressing the reset switch. This issue is quite problematic since the production version will not have any I/O for reset.

Could RTOS be causing any issue. I have increased the priority of the BLE task to 4 (Total of 4 tasks running)  from the default of 2 in nrf_sdh_freertos.c 

[MartinBL]

Hi,

1. You say that "we do not see any disconnect event and the board becomes unresponsive " does this mean that the device continue to work perfectly normal, except for the BLE stuff? It sounds like your code has asserted and just ended up in an endless while loop in an assert handler somewhere. Have you debugged your code? Do you get any debug log information out of your nRF52 device?
2. It could be useful with a sniffer trace so that we can check what packets are transmitted last. If you have an extra Nordic dev kit to spare you can use our RF Bluetooth Sniffer solution. 

[Siddhesh]

After using both your suggestions, the sniffer concluded there was nothing wrong on the BLE end.
After debugging the code was stuck without any error. After I paused the debugger, the control was stuck in vPortSuppressTicksAndSleep function in port_cmsis_systick.c. Specifically the lines 254-255.

I enabled the part for softdevice ignoring the internal ticket NRFFOSDK-11174, and the board was connected for around 2 hours but got the same error 22

I disabled tickless idle in FreeRTOSConfig.h and the board was connected for around 2 hours and 40 mins after which I saw GATT_CONN_TIMEOUT error 0x08 on the phone side, but no disconnect on the board side.

Any thoughts on this ?

[MartinBL]

Can you upload your sniffer logs?

Are you checking the return codes of all your Softdevice API calls?

What kind of mobile phone do you use as a central and what OS does it run? Have you tried other devices?

[Siddhesh]

1. Highlights of the sniffer log. The peripheral advertises and scans at the same time. The scan is for a different device not active while testing. Connection established at 15.809 Connection lost without a proper disconnect at 406.29

2. I put a break point in the app_error handler. Should I put a breakpoint on every SD API call ?

3. Currently we are testing using Samsung Galaxy S8 running Android Oreo. I found a slightly different background behavior with a OnePlus device where the phone stops sending data when the screen is off and then resumes sending on waking up.

7002.test2.pcapng

EDIT: I also came across this blog. Could this be an issue ?
blog.classycode.com/a-short-story-about-android-ble-connection-timeouts-and-gatt-internal-errors-fa89e3f6a456

[MartinBL]

It seems like two weird things happen around frame 3355:

1. In frame 3356 the slave responds to an empty PDU, but the master doesn't acknowledge. This is indicated by the wrong Sequence Number in frame 3357. There seems to be a fair amount of packet loss in your trace so this could be coincidental. 
2. The slave stops responding to the master's PDUs after frame 3356. There are many reasons this could happen. Maybe the code asserted, a BLE event isn't handled properly, the device is out of range, etc. 

After 5 seconds the supervision timout is out and the Sniffer starts listening on the advertising channels again. At the end your link seems to use ~1 s connection intervals. With a supervision timout of 5 seconds this allows for 4 dropped packets before the link times out. It should be enough i most cases, but it could be interesting to see what happens if you increase the supervision timout or reduce the connection parameters a little.

Should I put a breakpoint on every SD API call
No. Most SD functions, as well as most of the functions in the libraries and drivers in the SDK, return an error code. You should check all error codes and handle them properly. In the SDK the codes are usually checked with the macro APP_ERROR_CHECK(err_code). For example:

err_code = sd_ble_gap_disconnect(...);
APP_ERROR_CHECK(err_code);

Are you doing this? If you do that and debug like shown in my previous link, you will end up in an error handler if something unexpected happen. 

[...]OnePlus device where the phone stops sending data when the screen is off
This sounds related to the implementation of the App. Do you lose connections with the OnePlus as well?

Can you upload a couple of more sniffer logs? It could be interesting to compare. 

[Siddhesh]

Quick update on the issue
I changed the connection intervals to Min: 100ms Max 300ms Supervisory timeout: 5000ms
On Oneplus 3, the connection was stable for around 8 hours overnight. It was still active when I woke up and manually closing the app caused a clean disconnect (Reason 0x13).
I will run some more tests with Samsung phones and upload sniffer logs if there are any issues.

[MartinBL]

That is interesting news. Please let me know how it goes. 

-Martin