How to transfer data through EPOUT2 in usbd-custom-hid-ep example

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

ovrebekk — Thu, 15 Dec 2022 12:00:30 GMT

Hi Ning

[quote user="425Ning"]If reliability is not considered, is it possible to use an isochronous endpoint to get a higher data rate than CDC?[/quote]

Isochronous endpoints essentially give you a guaranteed and stable data rate, but without the guarantee that all the packets are received. The main use case for this is audio or video streaming where it is more important to keep latency low and throughput stable, rather than make sure that all the data is successfully transmitted.

Our examples only use isochronous endpoints for CD quality audio, so we haven't really tried to push the data rates to the limit.

Best regards
Torbjørn

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

425Ning — Wed, 14 Dec 2022 08:23:03 GMT

Hi Torbjørn,

[quote userid="2116" url="~/f/nordic-q-a/91465/how-to-transfer-data-through-epout2-in-usbd-custom-hid-ep-example/399894"]You should be able to get a higher speed if you use a different USB class, such as CDC or MSC. [/quote]

I'd used CDC and PySerial(in Python) before. But I want to transfer the data through the endpoint directly but not through the virtual COM port. So I tried to use HID and PyUSB(in Python) to implement my project.

[quote userid="2116" url="~/f/nordic-q-a/91465/how-to-transfer-data-through-epout2-in-usbd-custom-hid-ep-example/399894"]if you use an isochronous endpoint you are not guaranteed to have all the data come through, and you would have to accept a certain amount of packet loss. [/quote]

If reliability is not considered, is it possible to use an isochronous endpoint to get a higher data rate than CDC?

Best regards,

Ning

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

ovrebekk — Fri, 09 Dec 2022 11:07:59 GMT

Hi Ning

You should be able to get a higher speed if you use a different USB class, such as CDC or MSC.

We have done some testing internally to evaluate the transfer speed using these classes, and got the following results:

MSC write zeros to an empty drive (Linux): 513 kB/s (4104 kbit/s)

CDC ACM binary file transfer (Linux): 323 kB/s (2584 kbit/s)
CDC ACM binary file transfer (Windows): 215 kB/s (1720 kbit/s)

I don't know where the 8Mbps number comes from, but keep in mind that if you use an isochronous endpoint you are not guaranteed to have all the data come through, and you would have to accept a certain amount of packet loss.

Best regards
Torbjørn

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

425Ning — Fri, 09 Dec 2022 05:41:19 GMT

Hi Torbjørn,

[quote userid="2116" url="~/f/nordic-q-a/91465/how-to-transfer-data-through-epout2-in-usbd-custom-hid-ep-example/396661"]If you set this to 1 instead you should be able to send a packet every millisecond.[/quote]

After I set this parameter to 1, I could get the same speed as HID full speed(64,000 bytes per second).

Thanks for the answer. It is helpful.

Because HID defines the (interrupt) endpoint polling rate in the endpoint descriptor, the minimum is 1 ms. I can't change the parameter to less than 1 ms to get a higher data rate.

So I searched online to confirm if I could use this example to get a higher data rate than HID full speed.

Then I found this page. The questioner mentioned that he read in another post in the forum said the USB examples could achieve 8,184 Mbps. (I think he means 8.184Mbps.)

Could the program that I used achieve such a high data rate? Or it can be higher than HID full speed is fine. (More than 2 Mbps is better.)

What method should I use to achieve it?

Best regards,

Ning

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

ovrebekk — Mon, 21 Nov 2022 13:10:58 GMT

Hi Ning

This sounds like an issue with the bInterval setting in the endpoint descriptor, as shown here.

In my example this was set to 8, which will limit the polling interval to 8ms.

If you set this to 1 instead you should be able to send a packet every millisecond.

Best regards
Torbjørn

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

425Ning — Sun, 20 Nov 2022 12:39:31 GMT

Hi Torbjørn,

Sorry for the late reply due to some peresonal reasons.

[quote userid="2116" url="~/f/nordic-q-a/91465/how-to-transfer-data-through-epout2-in-usbd-custom-hid-ep-example/388908"]You shouldn't need to change the USB libraries to implement this[/quote]

OK. I got it. Thank you so much for the answer.

Recently I encountered a problem when testing the speed, when I transfer data through pyUSB, I could only get a 64,000 bits per second data rate, but I cannot reach the 64,000 bytes per second data rate of HID full speed.

How could I get a higher data rate?

Or...Will this problem improve if I use ISO EP instead? Because ISO EP can receive more bytes at one time, I think the data rate may be higher because of this reason.

Best regards,

Ning

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

ovrebekk — Mon, 03 Oct 2022 08:06:06 GMT

Hi Ning

What is the problem of just sending two 64 byte payloads and assembling them to a single 128 byte payload on the receiving side?

You shouldn't need to change the USB libraries to implement this.

Best regards
Torbjørn

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

425Ning — Fri, 30 Sep 2022 15:55:23 GMT

Hi Torbjørn,

[quote userid="2116" url="~/f/nordic-q-a/91465/how-to-transfer-data-through-epout2-in-usbd-custom-hid-ep-example/387274"]you could create multiple alternative reports with different lengths and differentiate between them by using a report ID. This method is described here.[/quote]

After reading the content of the webpage, I understood that the report ID could be used to differentiate the data length when the packet size less than 64 bytes. It fits the needs I mentioned. Thank you so much!

But I have another question: this code doesn't seem to receive the data larger than 64 bytes. When the data size is larger than 64 bytes, I would like to know if it is possible to give more space for the data. (For example, 128 bytes of data must use two 64-byte packets, whether HID can achieve continuous reception of two 64-byte packets, instead of discarding the following 64 bytes of data.)

If the above method could work, I must find the function that handles the data receiving, and modify it. I think the function is usbd_dmareq_process() which in the nrfx_usbd.c file. Is the understanding correct? Or the function is at another place.

The code of usbd_dmareq_process() function is below.

/**
 * @brief Process all DMA requests.
 *
 * Function that have to be called from USBD interrupt handler.
 * It have to be called when all the interrupts connected with endpoints transfer
 * and DMA transfer are already handled.
 */
static void usbd_dmareq_process(void)
{
    if (!m_dma_pending)
    {
        uint32_t req;
        while (0 != (req = m_ep_dma_waiting & m_ep_ready))
        {
            uint8_t pos;
            if (NRFX_USBD_CONFIG_DMASCHEDULER_ISO_BOOST && ((req & USBD_EPISO_BIT_MASK) != 0))
            {
                pos = usbd_dma_scheduler_algorithm(req & USBD_EPISO_BIT_MASK);
            }
            else
            {
                pos = usbd_dma_scheduler_algorithm(req);
            }
            nrfx_usbd_ep_t ep = bit2ep(pos);
            usbd_ep_state_t * p_state = ep_state_access(ep);

            nrfx_usbd_ep_transfer_t transfer;
            bool continue_transfer;

            NRFX_STATIC_ASSERT(offsetof(usbd_ep_state_t, handler.feeder) ==
                offsetof(usbd_ep_state_t, handler.consumer));
            NRFX_ASSERT((p_state->handler.feeder) != NULL);

            if (NRF_USBD_EPIN_CHECK(ep))
            {
                /* Device -> Host */
                continue_transfer = p_state->handler.feeder(
                    &transfer,
                    p_state->p_context,
                    p_state->max_packet_size);

                if (!continue_transfer)
                {
                    p_state->handler.feeder = NULL;
                }
            }
            else
            {
                /* Host -> Device */
                const size_t rx_size = nrfx_usbd_epout_size_get(ep);
                continue_transfer = p_state->handler.consumer(
                    &transfer,
                    p_state->p_context,
                    p_state->max_packet_size,
                    rx_size);
                
                if (transfer.p_data.rx == NULL)
                {
                    /* Dropping transfer - allow processing */
                    NRFX_ASSERT(transfer.size == 0);
                }
                else if (transfer.size < rx_size)
                {
                    NRFX_LOG_DEBUG("Endpoint %x overload (r: %u, e: %u)", ep, rx_size, transfer.size);
                    p_state->status = NRFX_USBD_EP_OVERLOAD;
                    (void)(NRFX_ATOMIC_FETCH_AND(&m_ep_dma_waiting, ~(1U << pos)));
                    NRFX_USBD_EP_TRANSFER_EVENT(evt, ep, NRFX_USBD_EP_OVERLOAD);
                    m_event_handler(&evt);
                    /* This endpoint will not be transmitted now, repeat the loop */
                    continue;
                }
                else
                {
                    /* Nothing to do - only check integrity if assertions are enabled */
                    NRFX_ASSERT(transfer.size == rx_size);
                }

                if (!continue_transfer)
                {
                    p_state->handler.consumer = NULL;
                }
            }

            usbd_dma_pending_set();
            m_ep_ready &= ~(1U << pos);
            if (NRFX_USBD_ISO_DEBUG || (!NRF_USBD_EPISO_CHECK(ep)))
            {
                NRFX_LOG_DEBUG(
                    "USB DMA process: Starting transfer on EP: %x, size: %u",
                    ep,
                    transfer.size);
            }
            /* Update number of currently transferred bytes */
            p_state->transfer_cnt += transfer.size;
            /* Start transfer to the endpoint buffer */
            nrf_usbd_ep_easydma_set(ep, transfer.p_data.addr, (uint32_t)transfer.size);

            usbd_dma_start(ep);
            /* There is a lot of USBD registers that cannot be accessed during EasyDMA transfer.
             * This is quick fix to maintain stability of the stack.
             * It cost some performance but makes stack stable. */
            while (!nrf_usbd_event_check(nrfx_usbd_ep_to_endevent(ep)) &&
                   !nrf_usbd_event_check(NRF_USBD_EVENT_USBRESET))
            {
                /* Empty */
            }

            if (NRFX_USBD_DMAREQ_PROCESS_DEBUG)
            {
                NRFX_LOG_DEBUG("USB DMA process - finishing");
            }
            /* Transfer started - exit the loop */
            break;
        }
    }
    else
    {
        if (NRFX_USBD_DMAREQ_PROCESS_DEBUG)
        {
            NRFX_LOG_DEBUG("USB DMA process - EasyDMA busy");
        }
    }
}

Best regards,

Ning

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

ovrebekk — Wed, 21 Sep 2022 11:23:44 GMT

Hi Ning

Sorry for the slow response.

After digging into this a bit more I realized this is probably just a limitation of the HID stack. Essentially HID reports operate with static fields in terms of size, you can't have dynamically sized fields like you can do with USB CDC or libusb for instance.

In other words you either need to pad your messages to 64 bytes, even if you don't have 64 byte to send, or you could create multiple alternative reports with different lengths and differentiate between them by using a report ID. This method is described here.

The drawback of the report ID method is that you lose one byte of the payload to the report ID itself (the report ID is always the first byte of the HID report).

Best regards
Torbjørn

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

425Ning — Mon, 12 Sep 2022 05:34:33 GMT

Hi Torbjørn,

OK. I understand. I'll try my best, too. Look forward to your reply.

Best regards,

Ning

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

ovrebekk — Tue, 06 Sep 2022 07:10:52 GMT

Hi Ning

Thanks for sharing the code. I will be out of office for a couple of days and won't be able to look into this until Friday unfortunately, and I will do my best to get back to you then with more information.

Please note that the example you are using is not an official example, and as such is not officially supported by us. Still, I will do my best to help once I am back in the office.

Best regards
Torbjørn

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

425Ning — Thu, 01 Sep 2022 13:25:41 GMT

Hi Torbjørn,

[quote userid="2116" url="~/f/nordic-q-a/91465/how-to-transfer-data-through-epout2-in-usbd-custom-hid-ep-example/384054"]Would you be able to share your python script with me so I can test it on my end? [/quote]

Sure! This is my pyUSB code. devzone.nordicsemi.com/.../pyusb_5F00_HID.py

[quote userid="2116" url="~/f/nordic-q-a/91465/how-to-transfer-data-through-epout2-in-usbd-custom-hid-ep-example/384054"]Have you tried to send exactly 64 bytes to see if that works?
As far as I recall I only tested the example with 64 byte packets[/quote]

I didn't try it before getting your suggestion.

Thanks to your suggestion, I tried sending 64-byte packets, then it works.

[quote userid="2116" url="~/f/nordic-q-a/91465/how-to-transfer-data-through-epout2-in-usbd-custom-hid-ep-example/384054"]it might need some modifications to support any packet size. [/quote]

I need to modify the packet size so that the chip could receive the data from the computer regardless of the packet size.

When the chip read the char '\n', then it has to transfer the packet through EPOUT2.

But I can not find where the chip receives the packet to make the NRF_DRV_USBD_EVT_EPTRANSFER event called.

So... Now I have another question: "Where should I modify the packet size in this example?"

Best regards,

Ning

RE: How to transfer data through EPOUT2 in usbd-custom-hid-ep example

ovrebekk — Wed, 31 Aug 2022 07:45:58 GMT

Would you be able to share your python script with me so I can test it on my end?

Have you tried to send exactly 64 bytes to see if that works?
As far as I recall I only tested the example with 64 byte packets, it might need some modifications to support any packet size.

Best regards
Torbjørn