USBD endpoint transfer completion event.

RE: USBD endpoint transfer completion event.

Hung Bui — Tue, 03 Dec 2024 09:26:10 GMT

Hi Ronald,
The legacy driver you pointed to will be retired.
You can find the current driver here, without the loop:
https://github.com/zephyrproject-rtos/zephyr/blob/main/drivers/usb/common/nrf_usbd_common/nrf_usbd_common.c#L897C4-L904C20

RE: USBD endpoint transfer completion event.

Ronald Hoogenboom — Fri, 29 Nov 2024 11:22:30 GMT

Thanks for the feedback. It is weird that his conclusion is not aligning with what I experience. For me, the hardware is a black box. All I have is whatever information Nordic provides and the inconsistencies I can detect in it. Anyway, my modification seems to solve my issue, I haven't had any issues with usbd since.

"the Zephyr usbd driver no longer does busy loop waiting in the interrupt context"

What I see in sdk-hal_nordic/nrfx/drivers/src/nrfx_usbd.c at ncs_2_7_0_system_nrf54l_mdk_8_67_fix · nrfconnect/sdk-hal_nordic (github.com) , there is still the busy wait loop, exactly the same as in NRF5 SDK. Maybe this is not part of NCS? Can you tell me where the nrfx_usbd driver for NCS is? I would be happy to transplant the newer code in my NRF5 SDK tree and get rid of the busy wait in interrupt context.

RE: USBD endpoint transfer completion event.

Hung Bui — Thu, 28 Nov 2024 13:23:42 GMT

Hi Ronald,
I'm really sorry for the late response. It's my mistake that the feedback from our R&D engineer didn't get back here. Here is his thought:

STARTED event is completely independent from host issuing the IN token. The host may not issue IN token at all, and yet the STARTED event will be generated.

The customer seems to be chasing something, but the conclusion that this in any way allows to determine whether IN token is sent by host is wrong. It does not, never did and never will. Can he observe the issue with Zephyr or NCS?

By the way, the Zephyr usbd driver no longer does busy loop waiting in the interrupt context. If the customer can reproduce it with Zephyr or NCS, I would be very interesting in investigating this. Otherwise, I just assume there's some bug somewhere in the NRF5 SDK or the customer application and not in the silicon.

Without a reproducer (full software package both on nRF52840 and host) I cannot check what's going on.

In the ticket customer was asking "What could be the reason why a DMA action isn't captured? Could it be because a previous DMA isn't finished yet?" - there can only be one DMA transfer (between main system memory and USBD endpoint buffer) active at a time. If the previous one hasn't finished before new one is started then for sure things won't work. But considering that nrfx does busy loop until the transfer ends, I don't think it is possible to trigger next DMA before previous one finished with nrfx.

A long shot at tackling this problem could be to try to remove the *((volatile uint32_t *)0x40027C1C) = 0x00000000; from usbd_dma_pending_clear() completely. This would result in increased current consumption, but could provide some insight whether or not the issue is related to errata 199 or not.

RE: USBD endpoint transfer completion event.

Ronald Hoogenboom — Thu, 28 Nov 2024 11:22:55 GMT

Is there any progress on this?

RE: USBD endpoint transfer completion event.

Hung Bui — Mon, 30 Sep 2024 14:29:03 GMT

Hi Ronald,

I would need to check internally to see if this can cause any potential issue. I will get back when I have a reply.

RE: USBD endpoint transfer completion event.

Ronald Hoogenboom — Fri, 27 Sep 2024 08:46:34 GMT

Hi Hung Bui,

There are just no IN tokens coming from the host, because it didn't listen for the interrupts from the endpoint yet. This apparently results in the DMA for the triggered endpoint task to never 'capture' (no STARTED event) and thus no END event either (which is what the loop in the driver is waiting for).

I think it's better to check a few times for the STARTED event (which should come quickly, as all DMA requests are serialized by the driver software already) than have a longer timeout on the END event. ISRs are timing critical, because they block all other parts of the code. I have seen that the STARTED event always comes within 5 loops, so timing out for the STARTED event can be done with as little as 10 loops or so. Then when the STARTED loop times out, it's useless to wait for END. I'm not sure if the triggered endpoint task needs to be canceled in the timeout case. I didn't do that and it seems to not harm anything.

The fragment in the nrfx_usbd.c around line 1440 looks like this now:

/* Start transfer to the endpoint buffer */
nrf_usbd_ep_easydma_set(ep, transfer.p_data.addr, (uint32_t)transfer.size);

usbd_dma_start(ep);
for (int i=0; ; i++)
{
if (nrf_usbd_event_check(NRF_USBD_EVENT_STARTED)) break;
if (i>8)
{
if (NRFX_USBD_DMAREQ_PROCESS_DEBUG)
{
NRFX_LOG_DEBUG("USB DMA process - not started");
}
/* Transfer won't capture - abort */
return;
}
}
usbd_dma_pending_set();

/* There is a lot of USBD registers that cannot be accessed during EasyDMA transfer.
* This is quick fix to maintain stability of the stack.

This seems to make the behavior stable for both when the host sends IN tokens to the interrupt endpoint and when it doesn't.

What do you think?

RE: USBD endpoint transfer completion event.

Hung Bui — Fri, 27 Sep 2024 08:22:34 GMT

Hi Ronald,
So from my understanding, it's the issue with the host that it doesn't send IN requests. But I still don't understand why that would cause the driver to end in a deadloop as in your initial question. Would that because the host is unresponsive and leading to END event never come ?
Maybe we can safeguard this loop by adding a way to exit it if it stuck there for a while ?

RE: USBD endpoint transfer completion event.

Ronald Hoogenboom — Thu, 26 Sep 2024 08:27:52 GMT

I now have the real root cause of the problem. In some situations, the host doesn't do any IN request on the interrupt endpoint after configuring the usb device. When it doesn't, and I send an interrupt transfer to the host, this phenomenon in the dmareq_process function occurs and it still does after the change I described above. I'm not sure why I initially saw that the change solved the issue.

I don't see a way to check whether the host 'activated' the interrupt endpoint (by sending IN requests) before doing the endpoint tranfer. I think a valid way to handle such situations is to just ignore the transfer, possibly returning an error, not hang in an interrupt routine in the driver code.

Maybe polling a few times for the 'STARTED' event before setting dma_pending?

RE: USBD endpoint transfer completion event.

Hung Bui — Wed, 25 Sep 2024 09:23:56 GMT

Hi Ronald,
Thanks for the report. I have forwarded your finding internally. I will keep you updated when I get a feedback.

RE: USBD endpoint transfer completion event.

Ronald Hoogenboom — Tue, 24 Sep 2024 09:21:17 GMT

I have found the reason for the issue. Look at the comments on the function usbd_dma_pending_set:

/**
* @brief Mark that EasyDMA is working.
*
* Internal function to set the flag informing about EasyDMA transfer pending.
* This function is called always just after the EasyDMA transfer is started.
*/
static inline void usbd_dma_pending_set(void)
{
if (nrfx_usbd_errata_199())
{
*((volatile uint32_t *)0x40027C1C) = 0x00000082;
}
m_dma_pending = true;
}

It says that this function is (to be) called just AFTER the DMA is started, but in the function usbd_dmareq_process, it is called a little BEFORE the DMA is started. This isn't a problem for setting the value of m_dma_pending (of course) but it is a problem for the errata 199 workaround.

I moved the call to the pending_set function from line 1429 to right after usbd_dma_start(ep), line 1443 and it solves the issue with raspberry pi. It also still works fine with the windows laptop.

I do think this should be rectified in the NRF5 SDK and in the NRF Connect SDK (sdk-hal_nordic/nrfx/drivers/src/nrfx_usbd.c).

RE: USBD endpoint transfer completion event.

Ronald Hoogenboom — Mon, 23 Sep 2024 15:41:50 GMT

I dove a bit deeper in the situation and it seems the notification ep transfer isn't started at all (or at least it doesn't arrive at the host side) and therefore the end event doesn't occur either. The driver code just calls 2 inline functions (nrf_usbd_ep_easydma_set and usbd_dma_start) and assumes the transfer is started without checking if it really did.

According to the nRF52840_PS_v1.8.pdf, there should be a 'STARTED' event when the PTR and MAXCOUNT have been 'captured'. Only when they have, it makes sense to wait for the ENDEPIN event to ensure the buffer is 'free' again (although I disagree that this should be done in the interrupt routine, alas).

What could be the reason why a DMA action isn't captured? Could it be because a previous DMA isn't finished yet?

RE: USBD endpoint transfer completion event.

Ronald Hoogenboom — Fri, 20 Sep 2024 12:16:37 GMT

This is for an outgoing notification to the host, so it should be ENDEPIN. The endpoint address is 0x83 and the ep_to_endevent(ep) results in 0x114.

Since this is involving interrupts, it's hard to see if anything else reset the end event before the handler code has the chance to look at it, but since this is code run in interrupt context, I guess the likelihood is very minimal.

I will try to find an example that is suitable to demonstrate this issue. Maybe you have a suggestion which one?

I don't have a working NC SDK at my disposal now. But it looks like that code didn't change much between nRF5 SDK and NRF Connect.

RE: USBD endpoint transfer completion event.

Hung Bui — Fri, 20 Sep 2024 12:03:10 GMT

Hi Ronald,
Please correct me if I'm wrong, what you are saying is that the hardware event ENDEPIN or ENDEPOUT didn't come as expected ? Could you check exactly which one ?
Is there any chance that it's been cleared by something else and got stuck there ? Are you able to reproduce the issue with an example in the nRF5 SDK ? This way we can be sure it's not something else with the application may cause the problem.

As you know the nRF5 SDK is discontinued for a long time, but if you can reproduce with NCS then it's easier for us to request investigation/bug fix in NCS than on nRF5 SDK.

RE: USBD endpoint transfer completion event.

Ronald Hoogenboom — Fri, 20 Sep 2024 08:53:27 GMT

I see that in the nrfconnect SDK, this piece of code is still there: sdk-hal_nordic/nrfx/drivers/src/nrfx_usbd.c at ncs_2_7_0_system_nrf54l_mdk_8_67_fix · nrfconnect/sdk-hal_nordic (github.com)