Why having "nRF Connect - Bluetooth Low Energy" open makes my connection faster and more stable

[jpo38]

I am using a nRF52840-PCA10059 dongle to acquire data at a high rate (between 50Hz and 400Hz).

The connection is very unstable, after some seconds of acquisition, it sort of freezes: looks like the softdevice crashs as we stop receiving notifications and it won't respond to read/write requests (however we do not receive any information regarding a connection being lost).

I already reported a case here and it has never been fixed:

https://devzone.nordicsemi.com/f/nordic-q-a/44922/s140-pc-ble-driver-4-0-0-freezes-when-receiving-too-many-data

Surprisingly, we just noticed that having "nRF Connect - Bluetooth Low Energy" app opened (even if not connected to the dongle, just opened in Windows) makes everything work better:

- The services dicovery is much faster

- The connection is prefectly stable. I can receive notifications for hours

As soon as I close "nRF Connect - Bluetooth Low Energy", my other application stops receiving norifications an I observe the freeze.

Apparently, "nRF Connect - Bluetooth Low Energy" does something in the background that makes the dongle and/or the soft device be more stable. What could it be?

[Jørgen Holmefjord]

Hi,

This sounds very strange. I now have a PC with an OS that has not been used for Bluetooth before. I will try to reproduce the behavior you describe on this one this week. Unfortunately, I have not received any feedback from our developers if they have managed to reproduce this yet.

Best regards,
Jørgen

[Jørgen Holmefjord]

Hi,

I did, in fact, manage to reproduce it. I tested both on the nRF52840 Dongle and the nRF52840 DK using the nRF_USB connector (both running the USB connectivity firmware), and they both stop after 15-20 notifications. In order to get more logs, I built the connectivity FW with RTT logging enabled. When I retried the test with the J-Link RTT Viewer open, I was no longer able to reproduce the issue.

I will try to do some more debugging, and also check with the developers again if they have had time to look into this.

Best regards,
Jørgen

[Jørgen Holmefjord]

You can find that in the case I linked above.

[jpo38]

Hello Jorgen,

Unfortunately, this new .hex file does not resolve the issue. Note that there is not only "nRF Connect - Bluetooth Low Energy" being opened that makes the issue disappear, we also noticed that having Keil or Visual Studio opened makes the issue disappear. Maybe you had one opened when you could run the app for 5 hours with no problem. To be sure you reproduce our issue, the best is to reboot the system and only run heart_rate_collector_v3 or heart_rate_collector_v6 without starting any other program, it always freezes within a few seconds.

On my laptop, just after softdevice stops transmitting notifications, I receive a "serial port read on port COM5 aborted" "Debug" notification through the log_handler passed to sd_rpc_open. I was not seeing on my desktop computer. Maybe it can help.

Please confirm that you can still reproduce the issue, as you did some weeks ago.

Kind regards,

Jean 

[Jørgen Holmefjord]

Hi Jean,

I did only verify the fix with sd_api_v3, as it was my understanding that you did not manage to reproduce this issue with sd_api_v6. Reading over the cases again, I now see that this assumption may not have been correct. I did a new test with both sd_api_v3 (where I again reproduced the issue with the old firmware, and confirmed that it's working with new firmware), and with sd_api_v6. In sd_api_v6, I can both reproduce the issue, and the new connectivity hex does not seem to solve the issue.

This computer does not have any Nordic tools installed (nRF Connect, nRF Command Line Tools, etc), or any compilers (SES, Keil, VS, GCC, etc), so there should not be anything interfering in that way,

I will try to run connectivity firmware with logging enabled to try to find any other issues. Can you confirm if you were able to reproduce the issue with the new connectivity hex on sd_api_v3?

Best regards,
Jørgen

[jpo38]

OK, good to know you can still reproduce the issue.

I can reproduce the issue with both SD API v3 and v6, and, surprisingly, none of the .hex file you posted fix it (with *_critical_region_fix.hex, I can still see notification freezing with SD API v3 and v6). So for API v3, we experience a different behaviour.

In all 4 situations, I get this "serial port read on port COM5 aborted" "Debug" notification after notifications are stopped.

Kind regards,

Jean

[Jørgen Holmefjord]

Ok, I enabled logs in the connectivity firmware and ran the test again. I see a fatal error in the log, which I traced to a NRF_ERROR_NO_MEM when a new event is added to the scheduler queue. I suspect that the high frequency of notifications cause the scheduler queue to be filled before it can be processed.

I tried doubling the configured queue size, and I'm now not able to reproduce the issue again. Can you test with this new hex-file? connectivity_4.1.1_usb_with_s140_6.1.1_critical_region_fix_increased_sched_queue_size.hex

[jpo38]

Hello Jorgen,

I just tested and this does not fix the issue for me, I still see the freeze after few seconds. DId you reboot your system before testing when you saw the issue fixed?

We can try to increase the size more, but this seems more like a "workaround" than a "fix", and the queue size to workaround the issue may depend on the device configuration (amount of data sent and throughput) and computer speed.

I mean increasing a buffer size may prevent the freeze but the root issue is that things are going slower, not even notifications.

On a particular device I have here:

- when my app runs without "nRF Connect - Bluetoth Low Energy" app being opened, device service discovery takes 4733ms.

- when my app runs with "nRF Connect - Bluetoth Low Energy" app being opened, device service discovery takes 2302ms.

Note that in both cases a descriptor read takes the same time (~20ms).

So when "nRF Connect - Bluetoth Low Energy" is not alive, the softdevice is slower and this is likelly to cause the freeze at some point, freeze that could be workarounded by playing with buffer sizes but things will remain slower than they should. Did you get a chance to check with the developers if they are doing anything special (like changing port priority) in "Bluetooth Low Energy" app?

Kind regards,

Jean

[Jørgen Holmefjord]

Hi,

jpo38 said:

We can try to increase the size more, but this seems more like a "workaround" than a "fix", and the queue size to workaround the issue may depend on the device configuration (amount of data sent and throughput) and computer speed.

I'm not sure I agree that this is not a fix. When the problem occurs due to an error from a too-small queue size, the queue size needs to be increased to the maximum number of events to be scheduled. The queue size has worked well for other transports for many years, but the USB transport seems to require larger queues to work well with slower computers.

I posted an updated FW with a queue size of 64 in this post. None of the affected customers have managed to reproduce this issue with this queue size, on any computers. I would again recommend you to try this out. The firmware has logging enabled, an case you are facing any other errors we would like to have a look at the logs.

jpo38 said:

So when "nRF Connect - Bluetoth Low Energy" is not alive, the softdevice is slower and this is likelly to cause the freeze at some point

This makes no sense to me or any of the developers that I have spoken with about this issue. There should be no communication between nRF Connect and the chip before the COM-port is opened. If the port is opened by the pc-ble-driver application, it should also not be possible for nRF Connect to open the port. I have also not been able to reproduce this behavior. Can you show the exact process you used to produce this issue, and how the timings are measured?

[jpo38]

Hello Jorgen,

Thank you again for following this issue.

We are still experiencing the same here and there's been no progress:

- Reboot the PC
- Start our software using NRF's pc-ble-driver, connect a APIv6 dongle, connect a device
- Service discovery is slow (up to 4s)
- Start notifications for data being sent by the device at high rate (100Hz)
- After less than a 5s, driver reports "serial port read on port COM5 aborted." and I don't get any more notifications (I see the dongle LED blink once)
- Kill our software, unplug/replug the dongle
- Now start NRF connect, "Bluetooth Low Energy" app (I just open the app, do nothing more)
- Start our software using NRF's pc-ble-driver, connect a APIv6 dongle, connect a device
- Service discovery is at least twice faster (~2s)
- Start notifications for data being sent by the device at high rate (100Hz)
- After more than a minute, no error is reported
- I close NRF connect, "Bluetooth Low Energy" app
- Instantly, I get the "serial port read on port COM5 aborted." and I don't get any more notifications (I see the dongle LED blink once)

I agree "Bluetooth Low Energy" can't open the port while my app uses it, however, it can change some Windows priority or anything else.

I can do a video if you don't believe me, this is 100% reproductible.

I tried bot hthe original hex file packaged with the NRF's pc-ble-driver and connectivity_4.1.1_usb_with_s140_6.1.1_critical_region_fix_increased_sched_queue_size.hex. With both I can reproduce the problem.

Jean

[Jørgen Holmefjord]

I will test out your instructions next week. Did you try the hex-file with scheduler queue size set to 64, which I posted in the linked thread (200505_ble_connectivity_s140_usb_hci_pca10056_lfxo_mergedsoftdevice.hex)?

[jpo38]

Hi Jorgen,

I originally only tested connectivity_4.1.1_usb_with_s140_6.1.1_critical_region_fix_increased_sched_queue_size.hex, this did not fix the issue.

I just tested 200505_ble_connectivity_s140_usb_hci_pca10056_lfxo_mergedsoftdevice.hex, and this apparently fixes the issue!

I'll ask other developers from my team to test this last hex file and confirm next week if this solves the problem for good or if I was just lucky when I tested it today.

What's the difference between those tow files? Are they related to different issues?

Please do not loose time trying to investigate this on Monday untill I come back to you.

Kind regards,

Jean