No UART irq for several ms after regular BLE radio activity

RE: No UART irq for several ms after regular BLE radio activity

MartinBL — Wed, 05 Sep 2018 08:05:49 GMT

Hi,

No worries.

Yes, it affects the S132 Softdevices too.

RE: No UART irq for several ms after regular BLE radio activity

mkuiken — Wed, 05 Sep 2018 07:37:49 GMT

Hi Martin,

Sorry for not getting back to you sooner. Holiday and some personal stuff kept me out of the office for some time.

I will see if I can easily change the code to use the normal requests. I'll get back to you on that.

Also, is it clear if this issue is also affecting the S132 softdevice (used with the nRF52832)?

/Martijn

RE: No UART irq for several ms after regular BLE radio activity

MartinBL — Thu, 26 Jul 2018 12:10:43 GMT

Hi,

That is interesting. This is what the Softdevice team suggests:

For the S110 with only a slave link running, I would guess that a safe value could be timeslot_event_length * 2 + 7.5 ms (<- slave max event length). Maybe give it some extra headroom. I think increasing the interrupt priority of the UART0 interrupt would be a better solution. The problem could be avoided entirely by using normal requests instead.

RE: No UART irq for several ms after regular BLE radio activity

mkuiken — Tue, 17 Jul 2018 14:01:53 GMT

Hi Martin,

This is definitely making a difference! The timeout value was set to 50ms and I have now increased it by a factor 6 (300ms) and the UART errors are a lot less. They're still there but definitely an improvement.

From these first tests I would say that this is same issue as for the S140 then. Can you elaborate a bit on this recently discovered bug? Is there a minimal timeout value to use which can prevent this bug from happening? Or are there other workarounds? Is this bug also existing for the S132/nRF52832?

/Martijn

RE: No UART irq for several ms after regular BLE radio activity

MartinBL — Tue, 17 Jul 2018 11:03:21 GMT

Hi,

I was made aware of a recently discovered bug in the timeslot API today. The bug was discovered using S140 and nRF52840 though, but the symptoms and circumstances look similar. Are you using the timeslot API to perform "earliest possible" requests? Can you try to increase the timeout on the timeslot request and see if that has an effect?

PS: I'll be on holiday until next Tuesday. As it is holiday season in Norway it is not certain that any of my colleagues will have time to pick up the case in the meantime.

RE: No UART irq for several ms after regular BLE radio activity

mkuiken — Fri, 13 Jul 2018 13:58:32 GMT

[quote userid="3500" url="~/f/nordic-q-a/36223/no-uart-irq-for-several-ms-after-regular-ble-radio-activity/140047"]Ok, so BLE and timeslot doesn't work, but either one by itself works?[/quote]

Well not exactly. BLE and timeslot perfectly work together radio wise. It's just that when both are active/functional I get UART errors as soon as the BLE connection becomes active....

[quote userid="3500" url="~/f/nordic-q-a/36223/no-uart-irq-for-several-ms-after-regular-ble-radio-activity/140047"]Are you handling this?[/quote]

Yup. Right after the softdevice is enabled I configure the Radio Notification. Also, this notifcation I only implemented to debug the UART error. It's not used for the timeslot functionality or anything. And I will remove it as soon as the UART problem is fixed.

[quote userid="3500" url="~/f/nordic-q-a/36223/no-uart-irq-for-several-ms-after-regular-ble-radio-activity/140047"]So there are more handlers to check?[/quote]

Not really. It was just a way to say I'm out of options what else to check :-)

[quote userid="3500" url="~/f/nordic-q-a/36223/no-uart-irq-for-several-ms-after-regular-ble-radio-activity/140047"]Could this be a pointer towards an event handler taking care of received data?[/quote]

Well I checked the events I get from the softdevice via SD_EVT_IRQn/SWI2_IRQn and the sd_evt_get() function. The time it takes to handle those events/received data from the softdevice is not the reason for blocking the UART IRQ. I made a screenshot:

You can see that when the UART interrupts are missed, the system is not in the SWI2_IRQn ISR to handle the ble events.

[quote userid="3500" url="~/f/nordic-q-a/36223/no-uart-irq-for-several-ms-after-regular-ble-radio-activity/140047"]Have you tried to reduce the baud rate?[/quote]

No I haven't. I'm not sure this would be a feasible solution since it would mean I also have to lower the baudrate of the device on the remote side of the UART. To overcome the UART problem by lowering the UART baudrate would mean I have to buffer those 13 ms in the 6 bytes FIFO receiver buffer. That would be a really low baudrate of ~460 bytes/sec :-(

[quote userid="3500" url="~/f/nordic-q-a/36223/no-uart-irq-for-several-ms-after-regular-ble-radio-activity/140047"] but at least you can get the current level by doing something similar[/quote]

I will check to see if that can help me ...

[quote userid="3500" url="~/f/nordic-q-a/36223/no-uart-irq-for-several-ms-after-regular-ble-radio-activity/140047"]you will not get a 100 % reliable UART transfer without flow control when using BLE at the same time[/quote]

I understand what you're saying and I agree there are limits to what to expect from a UART without flowcontrol. That being said, I don't think the baudrate I'm using, is pushing the system to the limit as can also be seen in the screenshot above. The system doesn't seemed to be more "stressed" the moment interrupts are missed

[quote userid="3500" url="~/f/nordic-q-a/36223/no-uart-irq-for-several-ms-after-regular-ble-radio-activity/140047"]Any chance it would be possible to upgrade to nRF52?[/quote]

Actually I have this system running on nRF52 as well, with the same uart implementation etc. No problem there. Because of the already established user base (nRF51 products), I'm making it run on nRF51 as well.

RE: No UART irq for several ms after regular BLE radio activity

MartinBL — Fri, 13 Jul 2018 10:24:21 GMT

Ok, so BLE and timeslot doesn't work, but either one by itself works?

I read through the radio notification chapter i the Softdevice Specification again, and since I'm under the impression that you turn on and off the timeslot API dynamically at run-time I took notice of this quote:

Are you handling this?

So far all event handlers I've checked
So there are more handlers to check?

The moment the UART interrupts are blocked, always seems to coincides with the radio deactivating (and each time it was after regular BLE traffic, not timeslot traffic)
Could this be a pointer towards an event handler taking care of received data?

Is it possible to retrieve the "previous" priority level somehow?
Not sure about the privious level, but at least you can get the current level by doing something similar to what is being discussed here: https://community.arm.com/processors/f/discussions/5387/get-current-active-interrupt-priority

Or are there possibly some other hooks in the soft device
Unfortunately the Softdevice is Nordic's proprietary black box, and there aren't many ways to peek inside it. But you can assume that all Softdevice generated events forwarded to your application is running in priority 2 as illustrated here.

Have you tried to reduce the baud rate?

What SDK are you using and are you using our UART drivers?

The bottom line is that, because of the asynchronous nature of UART and BLE, you will not get a 100 % reliable UART transfer without flow control when using BLE at the same time. And using the timeslot API and proprietary radio protocol on top of that again certainly won't make it better.

Any chance it would be possible to upgrade to nRF52? It has a much more advanced UART capable of writing data directly to RAM while the CPU is busy or sleeping.

RE: No UART irq for several ms after regular BLE radio activity

mkuiken — Thu, 12 Jul 2018 07:22:31 GMT

Hi Martin, thanks for your reply.

In my implementation the use of the timeslot functionality can be activated and deactivated when the device is up and running. When it is deactivated (or when the device boots with timeslot functionality deactivated), the UART errors do not occur. But as soon as the timeslot functionality is activated the UART errors occur when a BLE connection is set up. However I've not seen the UART error when actually being in a timeslot.

I've been trying to find if any of the event handlers is consuming excessive time but unfortunately this hasn't resulted in a suspect. So far all event handlers I've checked use their normal timing and shouldn't be the reason the UART is not serviced anymore. The moment the UART interrupts are blocked, always seems to coincides with the radio deactivating (and each time it was after regular BLE traffic, not timeslot traffic).

The common factor I guess is the radio; issue only occurs when timeslot functionality is active (though not actually being in a timeslot), issue occurs after radio activity. Hence my remark about "what could be the relation to the Radio Timeslot...".

Is it possible to retrieve the "previous" priority level somehow? If so I was thinking about running a TIMER on priority 1 and then logging the priority level the system was in prior to the TIMER IRQ. Then I would at least see what priority level was active when the UART is not serviced (being either an event handler running on priority 3 or the soft device running in priority level 2).

Or are there possibly some other hooks in the soft device I can use to get information on the priority level the system is in?

/Martijn

RE: No UART irq for several ms after regular BLE radio activity

MartinBL — Tue, 10 Jul 2018 13:33:57 GMT

[quote user=""]-Is it normal for the softdevice to block the UART interrupt (with low prio) for several milliseconds? This seems very long to me?[/quote]

Yes, it is normal for the Softdevice to block other interrupts, but maybe not for 13 milliseconds. It depends on your application. As you can see here, the Softdevice uses interrupt priority 0 (highest) for timing critical "under-the-hood" stuff. It uses priority 2 for less critical stuff though, like forwarding events to your application when you receive data for example. So if you have implemented time consuming code inside such event handlers, it might be that you are blocking other things running at lower interrupt priorities. When you change UART priority to 1, it means that it gets higher priority than all the Softdevice events which is probably why it works better. Maybe you can try to toggle some GPIOs inside suspicious Softdevice event handler functions and see if you can find the sinner.

[quote user=""]Futhermore, even though the timeslot is not active at the moment the overrun error occurs, the problem is non-existent when the Radio Timeslot functionality is not used (not initialized/started).[/quote]

Can you elaborate on this? Have you been using the timeslot API, but uninitialized it? Or do you mean if you remove everything related to the timeslot API and proprietary radio from your application entirely?

[quote user=""]-What could be the relation to the Radio Timeslot (maybe some left over, not re-initialized radio setting?)?[/quote]

Not sure if it is relevant here, but the radio timeslot API also works in high priority interrupts that will block your UART as you can see here: Radio Timeslot API processor usage patterns

[quote user=""]Using UART flow control is not an option in this scenario.[/quote]

That is unfortunate. Using BLE and UART on the nRF51 with 19k2 baud rate and no flow control is risky.

PS: It is holiday season in Norway these days and the response time might be slower than usual.

Best regards,
Martin