NRF52840 USB CDC Serial is missing first character when SOF = 2 and QUEUE_EN = 0 only

RE: NRF52840 USB CDC Serial is missing first character when SOF = 2 and QUEUE_EN = 0 only

OPCODE — Sat, 13 Nov 2021 00:18:18 GMT

Just wanted to come back to this and say that everything is working well. Your initial code helped me understand the way the SDK code was triggering on new data and work from there.

Had some timing issues when a lot of data was flowing between the USB and UARTs but that was solved with a nice circ buffer.

Thanks for taking the time to help me and sorry for replying late.

RE: NRF52840 USB CDC Serial is missing first character when SOF = 2 and QUEUE_EN = 0 only

Jørgen Holmefjord — Fri, 19 Feb 2021 10:32:56 GMT

[quote user="OPCODE"]

Regarding the missing first character during RX, I tried improving the code as per your suggestions but I cant get it to go away either. Only the first string is received well.

- Do you think its something related to the CDC APP? Shall I start poking in app_usbd_cdc_acm.c?

[/quote]

It is related to how the API works and how you are using it. Like I said, the problem is that the app_usbd_cdc_acm_read() function only provides a buffer to the CDC ACM library, that will be filled with received data sometime in the future. This means that when you reset the pointer to start receiving bytes at the start of the buffer again, there will already be one byte later in the buffer that is already provided to the library. You can work around this by copying the last byte into the first index when you have preformed the write operation:

#include <stdint.h>
#include <string.h>
#include "nordic_common.h"
#include "nrf.h"
#include "nrfx.h"

#include "nrf_sdh.h"
#include "nrf_sdh_soc.h"
#include "nrf_sdh_ble.h"
// #include "nrf_ble_gatt.h"
#include "app_timer.h"
// #include "ble_nus.h"
#include "app_uart.h"
#include "app_util_platform.h"

#include "nrf_drv_usbd.h"
#include "nrf_drv_clock.h"
#include "nrf_gpio.h"
#include "nrf_delay.h"
#include "nrf_drv_power.h"

#include "app_error.h"
#include "app_util.h"
#include "app_usbd_core.h"
#include "app_usbd.h"
#include "app_usbd_string_desc.h"
#include "app_usbd_cdc_acm.h"
#include "app_usbd_serial_num.h"

#define RECV_BUF_LEN 256
static char m_cdc_data_array[RECV_BUF_LEN] = {0};
volatile uint32_t buffer_available_idx = 0;
volatile uint32_t last_buffer_idx = 0;
static bool m_usb_connected = false;
static bool usbd_tx_done = true;

// USB DEFINES START
static void cdc_acm_user_ev_handler(app_usbd_class_inst_t const * p_inst,
                                    app_usbd_cdc_acm_user_event_t event);
#define CDC_ACM_COMM_INTERFACE  0
#define CDC_ACM_COMM_EPIN       NRF_DRV_USBD_EPIN2
#define CDC_ACM_DATA_INTERFACE  1
#define CDC_ACM_DATA_EPIN       NRF_DRV_USBD_EPIN1
#define CDC_ACM_DATA_EPOUT      NRF_DRV_USBD_EPOUT1

/** @brief CDC_ACM class instance */
APP_USBD_CDC_ACM_GLOBAL_DEF(m_app_cdc_acm,
                            cdc_acm_user_ev_handler,
                            CDC_ACM_COMM_INTERFACE,
                            CDC_ACM_DATA_INTERFACE,
                            CDC_ACM_COMM_EPIN,
                            CDC_ACM_DATA_EPIN,
                            CDC_ACM_DATA_EPOUT,
                            APP_USBD_CDC_COMM_PROTOCOL_AT_V250);


/** @brief User event handler @ref app_usbd_cdc_acm_user_ev_handler_t */
static void cdc_acm_user_ev_handler(app_usbd_class_inst_t const * p_inst,
                                    app_usbd_cdc_acm_user_event_t event)
{
    //app_usbd_cdc_acm_t const * p_cdc_acm = app_usbd_cdc_acm_class_get(p_inst);

    switch (event)
    {
        case APP_USBD_CDC_ACM_USER_EVT_PORT_OPEN:
        {
            /*Set up the first transfer*/
            //ret_code_t ret = 
            app_usbd_cdc_acm_read(&m_app_cdc_acm, &m_cdc_data_array[buffer_available_idx++], 1);  // necessary it seems
            break;
        }

        case APP_USBD_CDC_ACM_USER_EVT_PORT_CLOSE:
            break;

        case APP_USBD_CDC_ACM_USER_EVT_TX_DONE:
            usbd_tx_done = true;
            //app_usbd_cdc_acm_read_any(&m_app_cdc_acm, m_cdc_data_array, 1);
            //app_usbd_cdc_acm_read(&m_app_cdc_acm, &m_cdc_data_array[0], 1);  
            break;

        case APP_USBD_CDC_ACM_USER_EVT_RX_DONE:
        {
            ret_code_t ret;

            if(last_buffer_idx != 0)
            {
                memcpy(&m_cdc_data_array[buffer_available_idx], &m_cdc_data_array[last_buffer_idx], 1);
                last_buffer_idx = 0;
                buffer_available_idx++; 
            }

            do{
                    ret = app_usbd_cdc_acm_read(&m_app_cdc_acm, &m_cdc_data_array[buffer_available_idx++], 1);  
            }while (ret == NRF_SUCCESS);
 
            break;
        }
        default:
            break;
    }
}

static void usbd_user_ev_handler(app_usbd_event_type_t event)
{
    switch (event)
    {
        case APP_USBD_EVT_DRV_SUSPEND:
            break;

        case APP_USBD_EVT_DRV_RESUME:
            break;

        case APP_USBD_EVT_STARTED:
            break;

        case APP_USBD_EVT_STOPPED:
            app_usbd_disable();
            break;

        case APP_USBD_EVT_POWER_DETECTED:
            //NRF_LOG_INFO("USB power detected");
            if (!nrf_drv_usbd_is_enabled())
            {
                app_usbd_enable();
            }
            break;

        case APP_USBD_EVT_POWER_REMOVED:
        {
            m_usb_connected = false;
            app_usbd_stop();
        }
            break;

        case APP_USBD_EVT_POWER_READY:
        {
            m_usb_connected = true;
            app_usbd_start();
        }
            break;

        default:
            break;
    }
}


void INIT_usb_cdc(){

    ret_code_t ret;
    static const app_usbd_config_t usbd_config = {
        .ev_state_proc = usbd_user_ev_handler
    };

    app_usbd_serial_num_generate();

    ret = nrf_drv_clock_init();
    APP_ERROR_CHECK(ret);

    ret = app_usbd_init(&usbd_config);
    APP_ERROR_CHECK(ret);

    app_usbd_class_inst_t const * class_cdc_acm = app_usbd_cdc_acm_class_inst_get(&m_app_cdc_acm);
    ret = app_usbd_class_append(class_cdc_acm);
    APP_ERROR_CHECK(ret);

    ret = nrf_sdh_enable_request();
    APP_ERROR_CHECK(ret);

    ret = app_usbd_power_events_enable();
    APP_ERROR_CHECK(ret);
}




int main(void){
    INIT_usb_cdc();
    ret_code_t ret;
    while(1)
    {
        if (buffer_available_idx>1){            
            while(usbd_tx_done == false);
            usbd_tx_done = false;
            ret = app_usbd_cdc_acm_write(&m_app_cdc_acm, ">> ", 3);
            if(ret == NRF_SUCCESS)
            {              
                while(usbd_tx_done == false);
                usbd_tx_done = false;

                ret = app_usbd_cdc_acm_write(&m_app_cdc_acm, m_cdc_data_array, buffer_available_idx-1);
                if(ret == NRF_SUCCESS)
                {
                    last_buffer_idx = buffer_available_idx - 1;
                    buffer_available_idx=0;
                }
            }
        }
    }
}

You may also use a similar approach to store multiple bytes received during the write in main (as I described in point 4 above).

Another solution would be to use a ring buffer, instead of resetting the index to the start of the buffer after every read. This will allow you to store the current read and write index of the buffer, to always be able to write the last read bytes. See Ring buffer library in the SDK.

RE: NRF52840 USB CDC Serial is missing first character when SOF = 2 and QUEUE_EN = 0 only

OPCODE — Thu, 18 Feb 2021 19:23:42 GMT

Thanks so much for looking at it Jorgen!

Adding a delay of 100uS or a while(flag) check, after the app_usbd_cdc_acm_write also fixes the loss of any TX data (my original issue no2). I have confirmed that with my larger codebase and this example. I dont see any loss when transmitting in bursts anymore. I think it solves the same issue as the usbd_tx_done flag does in your code above.

Regarding the missing first character during RX, I tried improving the code as per your suggestions but I cant get it to go away either. Only the first string is received well.

- Do you think its something related to the CDC APP? Shall I start poking in app_usbd_cdc_acm.c?

- Shall I try a different SDK?

I am basically a little unclear as to what I should try next. Any pointers would be welcome!

RE: NRF52840 USB CDC Serial is missing first character when SOF = 2 and QUEUE_EN = 0 only

Jørgen Holmefjord — Thu, 18 Feb 2021 16:39:42 GMT

Hi,

I spent some time looking into your example today. I think I found the root cause of all issues, but I do not have a good solution for all:

Initially, the first missing character is caused by how you call app_usbd_cdc_acm_read(). This function will only provide a buffer to the USBD library, which will be filled when data is coming in over USB. When the port is opened, you provide the first byte of the buffer as the initial buffer to the library. When the first byte is received, you provide the same byte of the buffer to the library (since buffer_available_idx is 0). You can resolve this by also incrementing buffer_available_idx in APP_USBD_CDC_ACM_USER_EVT_PORT_OPEN, and wait until buffer_available_idx > 1 before writing back the data in main().
When you write back the data in main(), you do not check the return codes of the calls to app_usbd_cdc_acm_write(). This function will return an error if another write operation is ongoing. You need to wait for the APP_USBD_CDC_ACM_USER_EVT_TX_DONE event before starting a new transfer. You can resolve this by setting a flag in the event handler and wait for this before starting the next transfer.
When you are done writing the data back, you reset buffer_available_idx to 0, but you do still have 1 byte buffer passed to the library from the latest call to app_usbd_cdc_acm_read(). When you send the second string from the terminal, the first byte will be store at the last buffer_available_idx, while subsequent bytes will be stored from 0 and upwards. This will possibly overwrite the first byte (if the string is longer than the previous). I cannot see any API to cancel a buffer, so this issue I do not have a good solution to at the moment.
One final issue is that the USB running at interrupt may preempt the execution of the main loop while writing the data. If data is received between the second call to app_usbd_cdc_acm_write and the reset of the buffer_available_idx variable, you may also lose these bytes.

Attaching my main.c which works apart from the missing first byte of second and subsequent strings:

#include <stdint.h>
#include <string.h>
#include "nordic_common.h"
#include "nrf.h"
#include "nrfx.h"

#include "nrf_sdh.h"
#include "nrf_sdh_soc.h"
#include "nrf_sdh_ble.h"
// #include "nrf_ble_gatt.h"
#include "app_timer.h"
// #include "ble_nus.h"
#include "app_uart.h"
#include "app_util_platform.h"

#include "nrf_drv_usbd.h"
#include "nrf_drv_clock.h"
#include "nrf_gpio.h"
#include "nrf_delay.h"
#include "nrf_drv_power.h"

#include "app_error.h"
#include "app_util.h"
#include "app_usbd_core.h"
#include "app_usbd.h"
#include "app_usbd_string_desc.h"
#include "app_usbd_cdc_acm.h"
#include "app_usbd_serial_num.h"

#define RECV_BUF_LEN 256
static char m_cdc_data_array[RECV_BUF_LEN] = {0};
volatile uint32_t buffer_available_idx = 0;
static bool m_usb_connected = false;
static bool usbd_tx_done = true;

// USB DEFINES START
static void cdc_acm_user_ev_handler(app_usbd_class_inst_t const * p_inst,
                                    app_usbd_cdc_acm_user_event_t event);
#define CDC_ACM_COMM_INTERFACE  0
#define CDC_ACM_COMM_EPIN       NRF_DRV_USBD_EPIN2
#define CDC_ACM_DATA_INTERFACE  1
#define CDC_ACM_DATA_EPIN       NRF_DRV_USBD_EPIN1
#define CDC_ACM_DATA_EPOUT      NRF_DRV_USBD_EPOUT1

/** @brief CDC_ACM class instance */
APP_USBD_CDC_ACM_GLOBAL_DEF(m_app_cdc_acm,
                            cdc_acm_user_ev_handler,
                            CDC_ACM_COMM_INTERFACE,
                            CDC_ACM_DATA_INTERFACE,
                            CDC_ACM_COMM_EPIN,
                            CDC_ACM_DATA_EPIN,
                            CDC_ACM_DATA_EPOUT,
                            APP_USBD_CDC_COMM_PROTOCOL_AT_V250);


/** @brief User event handler @ref app_usbd_cdc_acm_user_ev_handler_t */
static void cdc_acm_user_ev_handler(app_usbd_class_inst_t const * p_inst,
                                    app_usbd_cdc_acm_user_event_t event)
{
    //app_usbd_cdc_acm_t const * p_cdc_acm = app_usbd_cdc_acm_class_get(p_inst);

    switch (event)
    {
        case APP_USBD_CDC_ACM_USER_EVT_PORT_OPEN:
        {
            /*Set up the first transfer*/
            //ret_code_t ret = 
            app_usbd_cdc_acm_read(&m_app_cdc_acm, &m_cdc_data_array[buffer_available_idx++], 1);  // necessary it seems
            break;
        }

        case APP_USBD_CDC_ACM_USER_EVT_PORT_CLOSE:
            break;

        case APP_USBD_CDC_ACM_USER_EVT_TX_DONE:
            usbd_tx_done = true;
            //app_usbd_cdc_acm_read_any(&m_app_cdc_acm, m_cdc_data_array, 1);
            //app_usbd_cdc_acm_read(&m_app_cdc_acm, &m_cdc_data_array[0], 1);  
            break;

        case APP_USBD_CDC_ACM_USER_EVT_RX_DONE:
        {
            ret_code_t ret;

            //if (app_usbd_cdc_acm_bytes_stored(&m_app_cdc_acm) > 0){
            //ret = app_usbd_cdc_acm_read(&m_app_cdc_acm, &m_cdc_data_array[buffer_available_idx++], 1);
                        
            do{
                    ret = app_usbd_cdc_acm_read(&m_app_cdc_acm, &m_cdc_data_array[buffer_available_idx++], 1);  
            }while (ret == NRF_SUCCESS);
 
            break;
        }
        default:
            break;
    }
}

static void usbd_user_ev_handler(app_usbd_event_type_t event)
{
    switch (event)
    {
        case APP_USBD_EVT_DRV_SUSPEND:
            break;

        case APP_USBD_EVT_DRV_RESUME:
            break;

        case APP_USBD_EVT_STARTED:
            break;

        case APP_USBD_EVT_STOPPED:
            app_usbd_disable();
            break;

        case APP_USBD_EVT_POWER_DETECTED:
            //NRF_LOG_INFO("USB power detected");
            if (!nrf_drv_usbd_is_enabled())
            {
                app_usbd_enable();
            }
            break;

        case APP_USBD_EVT_POWER_REMOVED:
        {
            m_usb_connected = false;
            app_usbd_stop();
        }
            break;

        case APP_USBD_EVT_POWER_READY:
        {
            m_usb_connected = true;
            app_usbd_start();
        }
            break;

        default:
            break;
    }
}


void INIT_usb_cdc(){

    ret_code_t ret;
    static const app_usbd_config_t usbd_config = {
        .ev_state_proc = usbd_user_ev_handler
    };

    app_usbd_serial_num_generate();

    ret = nrf_drv_clock_init();
    APP_ERROR_CHECK(ret);

    ret = app_usbd_init(&usbd_config);
    APP_ERROR_CHECK(ret);

    app_usbd_class_inst_t const * class_cdc_acm = app_usbd_cdc_acm_class_inst_get(&m_app_cdc_acm);
    ret = app_usbd_class_append(class_cdc_acm);
    APP_ERROR_CHECK(ret);

    ret = nrf_sdh_enable_request();
    APP_ERROR_CHECK(ret);

    ret = app_usbd_power_events_enable();
    APP_ERROR_CHECK(ret);
}




int main(void){
    INIT_usb_cdc();
    ret_code_t ret;
    while(1)
    {
        if (buffer_available_idx>1){            
            while(usbd_tx_done == false);
            usbd_tx_done = false;
            ret = app_usbd_cdc_acm_write(&m_app_cdc_acm, ">> ", 3);
            if(ret == NRF_SUCCESS)
            {              
                while(usbd_tx_done == false);
                usbd_tx_done = false;

                ret = app_usbd_cdc_acm_write(&m_app_cdc_acm, m_cdc_data_array, buffer_available_idx-1);
                if(ret == NRF_SUCCESS)
                {
                    buffer_available_idx=0;
                }
            }
        }
    }
}

Best regards,
Jørgen

RE: NRF52840 USB CDC Serial is missing first character when SOF = 2 and QUEUE_EN = 0 only

OPCODE — Wed, 10 Feb 2021 17:10:18 GMT

Hi Jorgen,

Thanks for your reply. When I made this example, I had the same issue. When I sent characters down the terminal I also get >> and sometimes I get the characters minus the first. But sometimes I don't get any response.

Looking at the code do you see any issue? Basically I am looking for a barebones interrupt CDC example with SD (no cli, no logging, no bsp, no uart) that can echo back characters so I can build on top of it. I cant run the examples as they have BSP/uart in them that might turn on and off random mosfets on our PCB. When I modify the examples to remove the bsp and uart things, I basically end up with the example I sent you, which looks like a pretty logic CDC interrupt based CDC example, but I have no idea why it doesnt work.

Regarding the terminal, I use many different ones like coolterm, minicom, custom one, and they all show the same behaviour.

Thanks!

RE: NRF52840 USB CDC Serial is missing first character when SOF = 2 and QUEUE_EN = 0 only

Jørgen Holmefjord — Wed, 10 Feb 2021 16:24:01 GMT

Unfortunately this example does not enumerate as a serial device on my setup, it only shows up as a generic USB device with a warning triangle in Windows Device manager. I was able to resolve this by modifying the VID/PID configs in sdk_config.h. Maybe this could also have been resolved by manually installing drivers, etc.

When connecting the serial terminal, I only see >> printed in most cases when I send strings. How are you testing this? Which serial terminal are you using? What is the input and output that you get?

RE: NRF52840 USB CDC Serial is missing first character when SOF = 2 and QUEUE_EN = 0 only

OPCODE — Wed, 10 Feb 2021 03:15:05 GMT

Thanks for looking at it Jorgen. I have spent some hours today trimming the code down to the minimum where the bug occurs.

The code just enumerates a CDC serial and should echo back whatever is typed. Please see the attached zip!

Setup details:

Compiler: gcc-arm-none-eabi-9-2019-q4-major-x86_64-linux

SDK version: nRF5_SDK_17.0.0_9d13099

Softdevice version: s140_nrf52_7.0.1_softdevice.hex

Let me know what you find!

devzone.nordicsemi.com/.../usbd_5F00_cdc_5F00_acm2.zip

www.dropbox.com/.../usbd_cdc_acm2.zip

RE: NRF52840 USB CDC Serial is missing first character when SOF = 2 and QUEUE_EN = 0 only

Jørgen Holmefjord — Tue, 09 Feb 2021 16:02:02 GMT

Hi,

I tried to reproduce this behavior by modifying the configs in the app_usbd_cdc_acm example in the SDK, but I'm not able to reproduce it. Maybe I missed something else required to reproduce this?

Could you share the full project that can be used to reproduce the issue?

Best regards,
Jørgen

RE: NRF52840 USB CDC Serial is missing first character when SOF = 2 and QUEUE_EN = 0 only

OPCODE — Tue, 09 Feb 2021 09:39:04 GMT

I have nRF5_SDK_17.0.0_9d13099 btw on NRF52840 rev2.

Also maybe this is an issue that resurfaced?
devzone.nordicsemi.com/.../52840-usb-cdc-missed-first-byte-in-rx-event