USBD_CDC_ACM 1st byte missing on second message sent when messages are greater than 64 Bytes

[Jes]

Dear Nordic Support team,

Currently I am in the process of adding the USBD_CDC_ACM driver code to our existing application following your usbd_ble_uart example. I can receive messages okay if they are less than 64 bytes long but when a message is greater than 64 bytes then the next message received has the first byte missing. I am using SDK 15.2 and running a SES project.

This is the message I am sending with a line feed at the end.

{"mac":"68B9D3D0BEC0","idonly":0,"duration":10,"fctrl":1,"cloud":1,"utc":1606456028,"type":"MQTT","fsrvid":"0%0","msg":"shake","rssi":-40,"scanenable":0}

Here is the RX event code

case APP_USBD_CDC_ACM_USER_EVT_RX_DONE:
{
   ret_code_t ret;
   NRF_LOG_INFO("Bytes waiting: %d, Index %d", app_usbd_cdc_acm_bytes_stored(p_cdc_acm), rxIndex);

do
{
   /*Get amount of data transfered*/
   size_t size = app_usbd_cdc_acm_rx_size(p_cdc_acm);
   // NRF_LOG_INFO("RX: size: %lu char: %c", size, m_rx_buffer[0]);
   //NRF_LOG_INFO("RX: %c", m_rx_buffer[0]);
   /* Fetch data until internal buffer is empty */

   if(firstRx)       /* Breakpoint B */
   {
      ret = app_usbd_cdc_acm_read(&m_app_cdc_acm,
                                                         &m_rx_buffer[++rxIndex],
                                                         1);
       //NRF_LOG_INFO("P2 %d", rxIndex);
   }
   else
   {
      ret = app_usbd_cdc_acm_read(&m_app_cdc_acm,
                                                         &m_rx_buffer[rxIndex++],
                                                         1);
      //NRF_LOG_INFO("P3 %d",rxIndex );
   }

   //nrf_delay_ms(1);
   } while (ret == NRF_SUCCESS);

   if(m_rx_buffer[rxIndex-1] == '\n')
   {

      
      NRF_LOG_INFO("Rx %s",m_rx_buffer);    /* breakpoint A */
      NRF_LOG_INFO("P1");
      rxIndex =0;
      firstRx = false;
      memset(m_rx_buffer, 0x00, sizeof(m_rx_buffer));

   }
      //bsp_board_led_invert(LED_CDC_ACM_RX);
   break;
}

To reproduce this problem I first place a breakpoint at position A

I then ran the application and sent the message via a serial terminal with a Line Feed at the end.

It hits breakpoint A and the contents is as shown below with the complete message.

image 1

I then activate breakpoint B and click play.

I then resend the message changing it slightly i.e.

{"maw":"68B9D3D0BEC0","idonly":0,"duration":10,"fctrl":1,"cloud":1,"utc":1606456028,"type":"MQTT","fsrvid":"0%0","msg":"shake","rssi":-40,"scanenable":0}

At breakpoint B I look into my message buffer which should be NULLED from the memset but see it has the first character of the second message at index 154

image 2

I then step into function 'app_usbd_cdc_acm_read' as shown below

image 3

This shows that the message was received by the USB_CDC_ACM HAL layer okay but the index last_read is off by 1 byte thereby causing m_rx_buffer to be off by one and the 2nd byte of the sent message be in the first byte of my m_rx_buffer.

image 4

It looks like once the APP_USBD_CDC_ACM_USER_EVT_RX_DONE is hit, the first byte of the sent message has already been read by the HAL layer before function 'app_usbd_cdc_acm_read' has been executed.

Can you please fix this issue or suggest a way around it when processing APP_USBD_CDC_ACM_USER_EVT_RX_DONE events for messages greater than 64 bytes

Regards,

Jes

[Dmitry]

Hi Jes,

I couldn't get your code completely (it's not clear what exactly does firstRx and why you're using post-increment and pre-increment in different cases), but the reason of lost character seems clear. The function app_usbd_cdc_acm_read() doesn't simply read characters from internal buffer - it tells the driver that you're allocated a buffer to receive characters, and this buffer will be filled right after return (if there are enough charachers in the internal buffers), or upon a next APP_USBD_CDC_ACM_USER_EVT_RX_DONE when the next portion of data is received. At the end of your handler you're clearing m_rx_buffer and rx index, but the driver still has a one-byte buffer passed in the last call to app_usbd_cdc_acm_read() - so the first byte of your second message comes there.

[Jes]

Hi Dmitry,

Firstly thank-you for your quick reply. I have just given you a section of code for you to look at. I understand what you are saying that a buffer is passed to the driver to be filled. I have been trying various methods to get a completed message from start to end. The first message comes thru okay it is the second message onward that has the problem.

Each message is '\n' terminated which is then passed to a processing function. I cleared the memory here to indicate a successful message has been received.(just breaking up my problem)

The application is a command / response type where the Nordic device responds to certain commands from the main CPU.

Have you tried receiving messages greater than 64 Bytes like the example I have given you to see if you receive the messages correctly from start to end?

This is no different to your sample code I just can't see what I am doing wrong. What steps in the RX_Done should I be taking to fulfill my requirements.

Regards,

Jes

[Dmitry]

Jes said:

Firstly thank-you for your quick reply. I have just given you a section of code for you to look at. I understand what you are saying that a buffer is passed to the driver to be filled. I have been trying various methods to get a completed message from start to end. The first message comes thru okay it is the second message onward that has the problem.

Yes, after '\n' you set rxIndex=0 but app_usbd_cdc_acm_read() is already called to prepare receiving a next character somewhere at the middle of your buffer, you should change the logic to pass a pointer to first byte when the buffer is cleared. BTW, reading characters one by one is not a best way in your case, take a look at app_usbd_cdc_acm_read_any() - it will fetch all pending data without waiting the buffer to fill completely.

Jes said:

Have you tried receiving messages greater than 64 Bytes like the example I have given you to see if you receive the messages correctly from start to end?

I did a reception of character stream to a circular buffer using app_usbd_cdc_acm_read_any(), there were no issues with lost characters.

[Jes]

Dmitry can you please share this code with me. I need to show something working to management in the next few days otherwise the project will be cancelled and we will need to look at another solution. This is the only issue we have holding up the roll out of our product.

[Hung Bui]

Hi Jes, 

Is your data always sending at 64 bytes chunk ? 

If it is, you should use app_usbd_cdc_acm_read() with the length of 64 bytes, not 1 bytes as you are doing now. 

As Dmitry already explained to you app_usbd_cdc_acm_read() doesn't read the byte(s) immediately, it only provide a buffer for the next "byte" (or bytes depends on the buffer size you provided) to come. 

So in your case you can call app_usbd_cdc_acm_read() with the buffer size of 64 bytes. See APP_USBD_CDC_ACM_USER_EVT_PORT_OPEN event in the cdc example. 

When you receive APP_USBD_CDC_ACM_USER_EVT_RX_DONE event this meant that there are 64 bytes already in the buffer and you can start using it. You don't need to call app_usbd_cdc_acm_read() again to read them. You call app_usbd_cdc_acm_read() again to prepare the buffer for the next 64 bytes. 

And since app_usbd_cdc_acm_read() can handle double buffer, you can call app_usbd_cdc_acm_read() two times with two different buffers in APP_USBD_CDC_ACM_USER_EVT_PORT_OPEN event. This way you have double buffer and can handle if there are more than 64 bytes comes before you can process them.