SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

Håkon Alseth — Mon, 25 May 2020 13:23:16 GMT

Hi,

[quote user="ryerye120"]Do you think these missed packets are related to how often I run _queue_process()?[/quote]

It could be, but without seeing the whole picture, I would only be guessing.

[quote user="ryerye120"]Also what did you mean by "retry function". I interpreted as checking the usb_write() return code. So now, my code will try to such that whenever a USB transfer returns something NOT(NRF_SUCCESS) then I run _queue_process() instead and then wait till the next iteration of my main loop before trying another USB transfer.[/quote]

Since you're using the queue, events and handling related to usbd is performed in iterations, as per how often the _queue_process() function is called. If there's something blocking the function from executing (like a delay in your main loop or similar), handling of the USBD traffic would also be delayed.

Could you share more of your code so I can have a look, or even better; be able to reproduce the problem?

Do you have a log file or similar which has timestamped data on the host side, to see which transfers are missing? Is the data missing, as in you see "skipped packets", or is the data delayed?

Kind regards,

Håkon

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

ryerye120 — Thu, 21 May 2020 20:25:26 GMT

Hi Hakon,

So at first I thought it was fixed, but ultimately no. Something is slowing down the USB transfer such that the FIFO fills up and then I start missing packets.

Do you think these missed packets are related to how often I run _queue_process()?

Also what did you mean by "retry function". I interpreted as checking the usb_write() return code. So now, my code will try to such that whenever a USB transfer returns something NOT(NRF_SUCCESS) then I run _queue_process() instead and then wait till the next iteration of my main loop before trying another USB transfer.

This way my USB_transfer doesn't block me from pushing SPI data into my FIFO.

Regardless ... I still can't empty out my FIFO fast enough.

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

Håkon Alseth — Tue, 19 May 2020 11:39:40 GMT

Hi,

[quote user="ryerye120"]I only call app_usbd_event_queue_process() once because I was scared of putting a blocking while loop before the spi_processing block. Again, I'm assuming this loop will be called enough times in between interrupts that it should be fine. Do you think this may cause a problem down the line?[/quote]

Sorry, I didn't think of the queue handler. try catching the "retry function" in a non-blocking fashion and setup the logic to run the _queue_process().

Has there been any progress compared to the original race-condition/missing packets? Its a bit hard to follow by looking solely on code snippets.

Kind regards,

Håkon

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

ryerye120 — Fri, 15 May 2020 18:47:23 GMT

[quote userid="2115" url="~/f/nordic-q-a/61202/spi-usb-will-drop-large-multiple-of-packets-in-a-row-can-t-keep-up-with-1-mbps-on-nrf52840-dk/250090"]It is therefore very important that you check the return code from app_usbd_cdc_acm_write().[/quote]

Of course! I didn't appreciate that app_usbd_cdc_acm_write() was non-blocking and wasn't checking the return value. This is definitely a major issue that I've now fixed in my code.

Unfortunately, that alone didn't seem to fix the issue - nor did the FIFO - nor did the combination.

I played with my FIFO depth and in the end, putting a 2000 entry FIFO between the SPI and USB seemed to do the trick (while of course checking to make sure the usb_write() returned NRF_SUCCESS).

Unfortunately, I worry that I haven't solved the root issue and that putting such a deep FIFO has just pushed the error further down the road.

My code currently looks like:

app_usbd_event_queue_process();

if (spis_xfer_done)
{
  spis_xfer_done = 0;
  data = process_data();
  fifo_status = fifoWrite(data); //if FIFO full, doesn't do anything
  if (fifo_status == 0) 
  {
    NRF_LOG_INFO("ERROR: FIFO FULL");
  }
}

if (usb_xfer_done)
{
  if (fifoRead()) //if FIFO empty, function returns 0 and skips if block
  {
    // fifo updates pointers so that usb_data points to the right place
    size_t size = sprintf(usb_data)
    while (app_usbd_cdc_acm_write(&m_app_cdc_acm, m_tx_buffer, size) != NRF_SUCCESS);    
    usb_xfer_done = 0;
  }
  else
  {
    NRF_LOG_INFO("FIFO Empty, nothing to send");
  }
}

I only call app_usbd_event_queue_process() once because I was scared of putting a blocking while loop before the spi_processing block. Again, I'm assuming this loop will be called enough times in between interrupts that it should be fine. Do you think this may cause a problem down the line?

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

Håkon Alseth — Fri, 15 May 2020 07:24:51 GMT

Hi Ryan,

[quote user="ryerye120"]Am I right to assume that the USB driver finishes really fast? Or is there overhead that I'm not accounting for.[/quote]

The call to app_usbd_cdc_acm_write() will not be a blocking call (ie: will return before the transaction is done), and eventually you will get a callback in your event handler with event "APP_USBD_CDC_ACM_USER_EVT_TX_DONE". At this point, the transaction is done and fully transferred to the USB host.

It is therefore very important that you check the return code from app_usbd_cdc_acm_write(). If your timing on SPI / USB is asynchronous to each other, then you can get into a scenario where you push several packets through USB, and this function returning NRF_ERROR_BUSY (0x11, 17 decimal). At this point, you should wait, then try to call the _acm_write() function again. Its simplest form for testing purposes is: while (app_usbd_cdc_acm_write(...) != NRF_SUCCESS);

These types of while-loops should not be called from interrupts (including callback handlers)

[quote user="ryerye120"]Right now I'm considering putting a FIFO in that the spi will push data into and the USB driver will pop from? Unfortunately, I don't know if there's enough space to make this FIFO worth it since it would need to be 520 bits x 1024 at least...[/quote]

If you need to buffer up 1024 entries, this indicates a very large delay in USB communications. Having a buffer that is between 2-10 entries deep should be sufficient, unless you plan to buffer while the USB is inactive (not plugged in)

[quote user="ryerye120"]Also thanks Hakon and everyone who's chimed in. I really appreciate all of your help. Otherwise, I'd be on my lonesome and be a sad boy.[/quote]

Happy to help! No one wants you to be sad and lonesome.

Cheers,

Håkon

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

ryerye120 — Thu, 14 May 2020 18:03:48 GMT

Hi Hakon!

[quote userid="2115" url="~/f/nordic-q-a/61202/spi-usb-will-drop-large-multiple-of-packets-in-a-row-can-t-keep-up-with-1-mbps-on-nrf52840-dk/249862"]Did you call app_usbd_cdc_acm_write() directly from the SPI interrupt callback prior to this? [/quote]

I tried this a long time ago but ultimately moved everything to the main specifically because of what you were saying. I worried that if I put the data processing and the usb_write() in the SPI interrupt handler, then I'd run into issues down the line. Thanks for confirming this because I'm kind of going off the cuff here.

[quote userid="2115" url="~/f/nordic-q-a/61202/spi-usb-will-drop-large-multiple-of-packets-in-a-row-can-t-keep-up-with-1-mbps-on-nrf52840-dk/249862"]Do you clear the usb_xfer_done flag somewhere else in that loop?[/quote]

Yea sorry, in my actual code, I actually clear usb_xfer_done in that inner if-sentence. I've updated the above blurb to reflect this.

Am I right to assume that the USB driver finishes really fast? Or is there overhead that I'm not accounting for.

Right now I'm considering putting a FIFO in that the spi will push data into and the USB driver will pop from? Unfortunately, I don't know if there's enough space to make this FIFO worth it since it would need to be 520 bits x 1024 at least...

Also thanks Hakon and everyone who's chimed in. I really appreciate all of your help. Otherwise, I'd be on my lonesome and be a sad boy.

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

Håkon Alseth — Thu, 14 May 2020 08:48:22 GMT

Did you call app_usbd_cdc_acm_write() directly from the SPI interrupt callback prior to this? By moving it to main, with a flag "usb_xfer_done", you are essentially forcing the usbd process to occur in main priority instead of interrupt priority, which is a good thing, as you want interrupts to finish processing as quick as possible.

Do you clear the usb_xfer_done flag somewhere else in that loop? Right now, your code will run the inner if-sentence regardless of if the usb is finished or not.

Kind regards,

Håkon

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

Mojo — Wed, 13 May 2020 18:49:56 GMT

You can send a packet containing prbs code. Google 'prbs'. On the receiving side check the code for error. You only printout when there is an error. Then you'll know whether your spirits transfer is good. I am assuming you are handling Master/Slave.

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

ryerye120 — Wed, 13 May 2020 18:37:22 GMT

The SPI clock? I set it to 4MHz because I was getting set up errors at 8MHz.

I should note that I found a bug in my code. Now if I save all the SPI codes and send them over USB *after* the fact, I don't drop any packets. So is there a recommended way of transferring data from SPI to USB on a packet-per-packet basis?

Earlier I had a naive nested conditional:

app_usbd_event_queue_process();
if (spi_xfer_done)
{
    spis_xfer_done = 0;
    if (usb_xfer_done)
    {
        size = sprintf(some data);
        app_usbd_cdc_acm_write(some data, size);
        usb_xfer_done = 0;
    }
}

Where spi_xfer_done is set to 1 by the spi_event_handler & usb_xfer_done is set to 1 by a handler looking out for the APP_USBD_CDC_ACM_USER_EVT_TX_DONE event.

This assumes that the USB driver is much faster than the SPI driver and will finish before the next SPI transaction is fired off/complete. Unfortunately, I must be missing something because experimentally speaking, this doesn't work.

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

Mojo — Wed, 13 May 2020 18:30:08 GMT

What is your SPI speed settings ?

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

ryerye120 — Wed, 13 May 2020 15:42:00 GMT

Sorry I did mean 500us. That was a silly mistake on my part. 512 bits every 500 us.

I'm using v16.0.0 of the SDK (which I believe is the latest). In this setup, it's purely SPIM and USBD. At least, those are the only things that I'm making calls to.

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

Håkon Alseth — Wed, 13 May 2020 09:16:33 GMT

Hi Ryan,

[quote user="ryerye120"]Does CDC have a similar limit?[/quote]

No, that should be able to push more data through the link.

[quote user="ryerye120"]I send 520 bits every 500 ms[/quote]

65 bytes will trigger a 64 byte + 1 byte transaction over USB, as your EP size is 64. Sending that every 500 milliseconds should be plenty time, or do you mean every 500 us?

Which SDK version are you using? What else than SPIM and USBD is running in your application? Do you have any BLE links active?

Kind regards,

Håkon

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

ryerye120 — Tue, 12 May 2020 17:19:12 GMT

The SPIM is running on the nRF.

I can try with RTT to see if the chip select issue continues (you're right in that I was using UART debugging) - and if not, I can look into debugging with that.

I send 520 bits every 500 us (so throughput is slightly higher than 1 Mbps). The software will send/receive 520 bits over SPI, then after a flag (spi_xfr_complete flag) is tripped in the main() body. After this flag is tripped, the data is processed (minorly) and then sent via USB to the host pc.

I've built this off the CDC example so it enumerates as CDC with whatever settings were in the usb_CDC example. Does CDC have a similar limit?

Thank you for the help!

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

Håkon Alseth — Tue, 12 May 2020 09:19:47 GMT

Hi,

Is the SPIM or SPIS running on the nRF?

If logging is an issue, you could use RTT instead of UART as the logger backend (see sdk_config.h::NRF_LOG_BACKEND_RTT_ENABLED and clearing NRF_LOG_BACKEND_UART_ENABLED) to see if something is caught and printed.

How large are the USB packets and how often do you send? How does your device enumerate, HID, CDC, libusb, winusb?

Note that HID is limited to 64 byte per 1 ms frame.

Kind regards,

Håkon

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

ryerye120 — Tue, 12 May 2020 04:35:40 GMT

Yea I agree that there seems to be an issue with the SPI code. I tried writing all the SPI data to a buffer and then reading that out over USB after the fact (in order to decouple the two) and I still had missing packets. I rarely missed a single packet, rather large chunks of adjacent packets were missed so it seems to me that there is some process interrupting the SPI.

But even that seems odd since the SPI should mostly be handled by hardware.

Do you have any suggestions for testing the SPI out on its own? Using the NRF Log is a no go because it will sometimes cause issues with the SPI's chip select driver and limit data throughput to <1Mbps.

RE: SPI/USB will drop large multiple of packets in a row. Can't keep up with 1 Mbps on nrf52840 dk?

Nguyen Hoan Hoang — Tue, 12 May 2020 00:49:40 GMT

USB is 12MHz no way USB can be slower than SPI. The problem is more likely in your SPI code. Have you validate that you got all the data through SPI before sending to USB ? Are you also sure the SPI sender actually sent all the data correctly. Validate each step of the way in order to find where the issue is.