Hardware Timer interrupt delayed significantly by SoftDevice (Or even missed)

RE: Hardware Timer interrupt delayed significantly by SoftDevice (Or even missed)

droberson — Wed, 26 Jun 2019 20:29:18 GMT

Hey,

Take a look at my response to vittopascu in this ticket for how this was (mostly) resolved: https://devzone.nordicsemi.com/f/nordic-q-a/37731/nrf52840-spi-easydma-double-buffering

It ended up not being a timer issue, but rather an issue with reading and updating TX.PTR. In short, you cannot update the TX.PTR while a SPI transaction is in progress despite what the datasheet says.

RE: Hardware Timer interrupt delayed significantly by SoftDevice (Or even missed)

kdubovenko — Tue, 25 Jun 2019 21:39:33 GMT

Hello, I am running into the exact same bug with my NRF52840. I am using code based on this thread to read and write from SPI NAND via DMA using timer counter (timer2 in my case). I have tried it with SPIM 0-3 and all of them run into this issue. if BLE is off I can read/write 10s of thousands of files without an issue. When the BLE goes on, the device will miss an interrupt within the first 100 files, somewhat randomly. Was this issue ever resolved with droberson? I am happy to create a private case for this but figured others might experience this as well, have tired all sorts of ways to fix this to no avail.

RE: Hardware Timer interrupt delayed significantly by SoftDevice (Or even missed)

MartinBL — Tue, 27 Nov 2018 09:17:51 GMT

Hi,

Thanks.

I saw your other case. I think it is best that we proceed over there.

-Martin

RE: Hardware Timer interrupt delayed significantly by SoftDevice (Or even missed)

droberson — Mon, 26 Nov 2018 20:41:38 GMT

How often does it happen, and can you say something about the frequency of which it occurs?

It appears to be random. It can happen within the first 10 minutes of it running or up to an hour later. It always occurs in less than 2 hours. I can perform the same operation when connected to my application via USB and I have absolutely no issue after 10 hours of continuous use, so it definitely seems related to the SoftDevice.

Is this easy to reproduce?

Yes, just allow the application to run and wait an hour. It usually occurs within an hour.

Can you upload your code? We can make the case private if you prefer confidentiality.

I have actually created a private ticket per recommendation from my direct FAE contact that another engineer was assigned to. I will PM you the details.

The code must remain confidential. I will continue my discussion in that thread in regards to how to upload my project.

Thanks!

RE: Hardware Timer interrupt delayed significantly by SoftDevice (Or even missed)

MartinBL — Fri, 23 Nov 2018 12:54:41 GMT

Hi,

I have read on the forums that the TXD.PTR pointer is double buffered. Is there any rule as to when this should/can be updated?
Please refer to this documentation: "The .PTR and .MAXCNT registers are double-buffered. They can be updated and prepared for the next transmission immediately after having received the STARTED event."
Is there a way to check if the SPIM peripheral is currently busy?
Unfortunately the only way is to control it in software
Is there a way to automatically reset the TX.PTR to the beginning of the ArrayList without CPU intervention? I ask this again because you were wandering why I implemented this mechanism.
No. I didn't quite understand your reasoning at first, but I think I do now, and I think your strategy is good.
Errata 198 shouldn't be relevant unless you are using SPIM instance #3. Are you? If so, can you try with instance 0,1, or 2?

How often does it happen, and can you say something about the frequency of which it occurs?

Is this easy to reproduce?

Would I be able to run your code and reproduce it on my desk?

Can you upload your code? We can make the case private if you prefer confidentiality.

-Martin

RE: Hardware Timer interrupt delayed significantly by SoftDevice (Or even missed)

droberson — Mon, 19 Nov 2018 05:14:08 GMT

Martin,

I did a bit more digging over the weekend and found something interesting. In my TIMER_COUNTER callback I added the following code:

// Reset the transmit buffer pointer offset to the start plus the number of bytes we went over
txPointerUpdated = ((uint32_t) buffer) + bufferOver;
spi3.p_reg->TXD.PTR = txPointerUpdated;

// Make sure the TX pointer was actually updated
if(spi3.p_reg->TXD.PTR != txPointerUpdated)
{
    cprintf("TX.PTR should have been updated to 0x%x", txPointerUpdated);
    cprintf("TX.PTR is actually 0x%x", spi3.p_reg->TXD.PTR);
}

Sure enough, it appears that the TX.PTR is sometimes not getting updated as seen in the log below.

(17:02:19.094) bufferOver 6
(17:02:19.094) TXD.PTR = 0x2001782c and COUNTER CC0 = 1021
(17:02:27.612) bufferOver 2
(17:02:27.612) TXD.PTR = 0x20017826 and COUNTER CC0 = 1023
(17:02:33.511) bufferOver 2
(17:02:33.511) TXD.PTR = 0x20017826 and COUNTER CC0 = 1023
(17:02:36.492) TX.PTR should have been updated to 0x20017822
(17:02:36.492) TX.PTR is actually 0x2001803a
(17:02:36.525) bufferOver 2046
(17:02:36.525) TXD.PTR = 0x20018028 and COUNTER CC0 = 1

Note that I increased the size of my buffer and overflow region to see if it made any difference before adding the TX pointer update check, which it didn't of course. Hence why the numbers are larger than in my original post.

A couple things:

1. I have read on the forums that the TXD.PTR pointer is double buffered. Is there any rule as to when this should/can be updated? Is it possible that writing to the TXD.PTR while the DMA controller is reading it will cause it to not be updated since the DMA controller is also incrementing it?

2. Is there a way to check if the SPIM peripheral is currently busy?

3. Is there a way to automatically reset the TX.PTR to the beginning of the ArrayList without CPU intervention? I ask this again because you were wandering why I implemented this mechanism.

I also took a look at the latest errata (http://infocenter.nordicsemi.com/pdf/nRF52840_Rev_1_Errata_v1.1.pdf) and the only one I see of interest is "3.30 [198] nRF52840: SPIM3 transmit data might be corrupted". It talks about not accessing the data in the TX data buffer where TXD.PTR is pointing at the same time the DMA controller is fetching data. This is specific to the data itself, not the pointer to the data but I imagine the same restrictions apply.

Any ideas, suggestions, or known workarounds for this?

Thanks again for any help!

Derek

RE: Hardware Timer interrupt delayed significantly by SoftDevice (Or even missed)

droberson — Thu, 15 Nov 2018 16:22:40 GMT

Hey Martin!

My timer interrupts are set to priority 2 (Highest possible) for this reason. Normally, I am not printing any information to a debug UART during this time, but have added a few debug statements as highlighted in my original post in an attempt to debug why this is happening. Ie, it was happening prior to adding any debug statements.

Based on the BLE peripheral timing diagram, I agree in that the SoftDevice shouldn't block for more than a few hundred usec at a time, hence why I am stumped as to why this is happening.

In regards to the buffer "mechanism" that I am using, the purposes of this is because the DAC writes need to occur indefinitely until the application tells it to stop. Given my understanding of ArrayLists, PPI, timers, and the SPI driver, the TXD.PTR must be reset by the CPU once all data in the ArrayList has been transmitted. In other words, the SPI driver cannot loop through the ArrayList without CPU intervention. This was confirmed by another FAE in my post here: https://devzone.nordicsemi.com/f/nordic-q-a/37731/nrf52840-spi-easydma-double-buffering (Read the last 3 replies). If there is a way to indefinitely loop through an ArrayList without CPU intervention, that would be great! (As this is what I had hoped could be done originally). Or perhaps there is an easier way to achieve my desired behavior?

I will look into the Errata list. I am using a Rigado branded nRF52840, so I need to figure out which revision Nordic chipset it is.

Thanks!

RE: Hardware Timer interrupt delayed significantly by SoftDevice (Or even missed)

MartinBL — Thu, 15 Nov 2018 14:57:07 GMT

Hi,

Higher priority interrupts will block lower priority interrupts and if you have an interrupt that runs for 20 ms or longer you will miss a timer interrupt (if it has lower priority). However, 20 ms is a relatively long time, so unless you are doing some serious processing or printing a lot of data on a slow UART line inside a high priority interrupt context I find it difficult to believe that this is what happens.

The Softdevice alone should never block your application for 20 ms straight. You can see what latency to expect from the Softdevice here.

But I don't understand why you need this whole mechanism? With your 32 us timer and PPI you can transfer your buffer to the DAC autonomously. The Softdevice will never block your timer from starting the SPI.

Have you looked through the Errata list?

Best regards,
Martin