USB UART output buffer remains full after serial disconnect

[douglas.malnati]

Windows 11, NCS v2.3.0, VSCode

Hi, I am using the uart_poll_out API for communicating via USB serial.

For each character I want to output, I do this:

uint32_t dtr;
uart_line_ctrl_get(DEV_UART_USB, UART_LINE_CTRL_DTR, &dtr);
if (dtr)
{
    uart_poll_out(DEV_UART_USB, c);
}

The other side of the serial connection can disconnect at any time.

I don't want my code to hang trying to write, that's why I checked the DTS beforehand, and basically throw away any data that won't get sent immediately.

The problem I have is:

- I am writing continuously

- The other side disconnects

- I attempt to detect that, and do, eventually, but not before the uart_poll_out puts a few characters into what I assume is an output buffer which fills up a short time later

Then, when the other side re-connects, the bytes that are sitting in the output buffer get sent.  This corrupts the datastream to the receiver as those bytes were for a different connection.

Is there a way to deal with this?

I would basically like to detect that I can't send data, and clear any existing output buffer at that time.

Or any other way to deal with this situation?

I'm not a serial expert and I only had the line_ctrl code because I saw it in an example.  Please let me know.

Thanks.

[Kenneth]

Hello,

I don't see any better way than to always check the uart_line_ctrl_get(dev, UART_LINE_CTRL_DTR, &dtr) before trying to send data. 

I assume you are calling usb_enable() here, can you try to add a callback function such as:

static void status_cb(enum usb_dc_status_code status, const uint8_t *param)
{
	LOG_INF("Status %d", status);
	// Clear buffers here if status is USB_DC_DISCONNECTED.
}

...
ret = usb_enable(status_cb);
...

I do believe you should be able to clear all buffers on disconnect event.

The complete list of status codes are:

/**
 * @brief USB Driver Status Codes
 *
 * Status codes reported by the registered device status callback.
 */
enum usb_dc_status_code {
	/** USB error reported by the controller */
	USB_DC_ERROR,
	/** USB reset */
	USB_DC_RESET,
	/** USB connection established, hardware enumeration is completed */
	USB_DC_CONNECTED,
	/** USB configuration done */
	USB_DC_CONFIGURED,
	/** USB connection lost */
	USB_DC_DISCONNECTED,
	/** USB connection suspended by the HOST */
	USB_DC_SUSPEND,
	/** USB connection resumed by the HOST */
	USB_DC_RESUME,
	/** USB interface selected */
	USB_DC_INTERFACE,
	/** Set Feature ENDPOINT_HALT received */
	USB_DC_SET_HALT,
	/** Clear Feature ENDPOINT_HALT received */
	USB_DC_CLEAR_HALT,
	/** Start of Frame received */
	USB_DC_SOF,
	/** Initial USB connection status */
	USB_DC_UNKNOWN
};

Maybe you can use those to also investigate your other issue you have:
usb_cdc_acm: cdc_acm_poll_out: Ring buffer full, drain buffer 

Best regards,
Kenneth

[douglas.malnati]

Thank you.

Kenneth said:

I assume you are calling usb_enable() here, can you try to add a callback function such as:

Yes I receive a callback in that way.

Kenneth said:

I do believe you should be able to clear all buffers on disconnect event.

How do I do that?  Is there documentation or an example?  I don't understand how to do this from your reply.

Kenneth said:

I don't see any better way than to always check the uart_line_ctrl_get(dev, UART_LINE_CTRL_DTR, &dtr) before trying to send data. 

Are you saying there is no way to stop this from happening?

Is the problem description clear enough?

I am saying I believe several bytes get loaded into some area in memory before I am notified of disconnect (usb callback) and also during that time I am still getting the dtr (probably because the bytes are in ram to be sent out later but aren't blocking yet.

Thanks.

Doug

[Kenneth]

Maybe you can share some soure code I can look at?

Kenneth

[douglas.malnati]

Hi, what code do you want to see to help you answer this?

Let's be specific so we can get to the answer quickly.

[Kenneth]

Ideally I want a project I can run on an nRF52840-DK to recreate this.

Kenneth

[douglas.malnati]

I don't think I can do that, at least not in a practical sense.  The nordic library is layered under many layers of abstraction (zephyr and my own) that you will find difficult to navigate.  What do you think?

Can we start with some thinking about what in principle is the issue here?  How well do you understand the interplay between the systems described in the problem statement?  What in principle can cause that error, and how does that relate to using the APIs as I have explained?

Thanks.

[Kenneth]

It would be good to get at least something to go on, but typically you need to clear the buffers somehow, for instance maybe you can call ring_buf_init() again to get clear them, similar for any other buffers you may have.

Kenneth

[douglas.malnati]

Hi, 

Kenneth said:

It would be good to get at least something to go on

Ok here is the .cpp file where all the UART activity happens.  There's a lot going on there.

Ultimately though, UartInit is called, and all it does is register an interrupt handler function for RX.

The interrupt handler function loads that data into a queue.

A thread consumes from that queue and uart_poll_out the data.

Everything else is scaffolding.

0804.UART.cpp

Kenneth said:

maybe you can call ring_buf_init() again to get clear them, similar for any other buffers you may have.

I don't know which ring buffer you're talking about.  Is there an outbound ring buffer the UART uses which receives bytes from uart_poll_out?  My code never does anything with a ring buffer, no idea if any uart-related libs do.

Please have a look at the code and please give some specific suggestions about next steps.

Thanks.

Doug

[Kenneth]

Hi,

I can't see that you are actually checking the dtr line before each uart_poll_out(), so a start would be to at least check dtr register before each call to uart_poll_out(). I can see that the cdc_acm_driver_api also have an irq_tx_ready() api, not sure if you have access to this api, but if you do it make sense to check this api before trying to call uart_poll_out(). Maybe you have access to uart_tx_abort(), so that is something you can try when you get a USB disconnected event.

Best regards,
Kenneth

[douglas.malnati]

Hi,

What is this code doing if not testing the dtr?  Again, I'm not an expert at this, but this code seems to be working, because it notices the difference between a connected USB serial connection and not one.

static void ThreadFnUARTUSBOutput()
{
    bool skippedLast = false;
    while (true)
    {
        char c;
        if (UART_USB_OUTPUT_PIPE.Get(c, K_FOREVER))
        {
            // consume the bytes, but only attempt to output if USB serial connected
            uint32_t dtr;
            uart_line_ctrl_get(DEV_UART_USB, UART_LINE_CTRL_DTR, &dtr);
            if (dtr)
            {
                // sad kludge to protect against:
                // - sending a stream of data to an endpoint which disconnects midway through
                // - unknowingly queuing a few additional bytes in the USB output buffer
                // - those bytes lingering and later getting sent on a new connection, corrupting the stream
                //
                // this approach hopes:
                // - the other side is line-oriented and simply discards the malformed message
                // - too few bytes get in the buffer to form a complete message
                if (skippedLast)
                {
                    skippedLast = false;

                    for (int i = 0; i < 6; ++i)
                    {
                        uart_poll_out(DEV_UART_USB, '\n');
                    }
                }
                uart_poll_out(DEV_UART_USB, c);
            }
            else
            {
                skippedLast = true;
            }
        }
    }
}

Kenneth said:

 irq_tx_ready() api

I'm using the polling API.  I don't want to use the IRQ-driven API.

Kenneth said:

Maybe you have access to uart_tx_abort()

Would uart_tx_abort() fix the problem?

douglas.malnati said:

Is the problem description clear enough?

I am saying I believe several bytes get loaded into some area in memory before I am notified of disconnect (usb callback) and also during that time I am still getting the dtr (probably because the bytes are in ram to be sent out later but aren't blocking yet.

I asked this question before but didn't get a reply.  Can you answer?

I want to know if you do understand the problem, and are saying that uart_tx_abort will clear the outbound buffer of unsent bytes.  Have you tested this?

Thanks.

Doug

[Kenneth]

Hello,

I assume the genereal theory here is that you are feeding the USB with data (by using the poll api) after USB is disconnected. Considering you are only reading the dtr register one time before you send several bytes, and I believe the poll() api is blocking for each bytes sent (while waiting for previous to be sent), you will very likely experience the problem you are. How to workaround this? I expect it will slightly depend on the priority you have of both interrupts and threads here. It is at least very likely that the disconnect will happen asynchonrous to your current scheme of things. Here are a few things I would have looked into:

- Ideally you want to make sure that the previous byte is sent before you try to send the next, a combination of checking that it is indeed connected and that the previous transfer have occured before writing the next byte seems like something I would have looked into. Though I suggest you may also consider using the uart_tx_abort() in addition if you find that it just disconnected.

Kenneth