Multiple virtual COM ports (CDC ACM connections) via USB

[puz_md]

Hello,

my goal is to enable two virtual COM ports via USB, where one is used for the Zephyr console and debug output, and the other is used for my software's communication interface. The debug console is already running fine. I'm using the nRF5340 SoC.

I already had a look at the cdc_acm and cdc_acm_composite samples, but I have trouble combining the configurations with my existing debug console configuration.

I use the following proj.conf:

CONFIG_PRINTK=y
CONFIG_ASSERT=y
CONFIG_GPIO=y
CONFIG_ZERO_LATENCY_IRQS=y

CONFIG_USB_DEVICE_STACK=y
CONFIG_USB_DEVICE_PRODUCT="My USB product"
CONFIG_USB_DEVICE_PID=0x0004
CONFIG_USB_DEVICE_INITIALIZE_AT_BOOT=y

CONFIG_SERIAL=y
CONFIG_CONSOLE=y
CONFIG_UART_CONSOLE=y
CONFIG_UART_LINE_CTRL=y
#CONFIG_UART_INTERRUPT_DRIVEN=y

CONFIG_BT=y
CONFIG_BT_PERIPHERAL=y
CONFIG_BT_DEVICE_NAME="My BT Device"
CONFIG_BT_HCI=y

CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048

And a extended my devicetree overlay with the cdc_acm_uart1 entry. (cdc_acm_uart0 was used for the Zephyr debug console and referenced in chosen {...} ).

&zephyr_udc0 {
	cdc_acm_uart0: cdc_acm_uart0 {
		compatible = "zephyr,cdc-acm-uart";
	};
	cdc_acm_uart1: cdc_acm_uart1 {
		compatible = "zephyr,cdc-acm-uart";
	};
};

First, I have two questions about the configuration:

1. I'm using CONFIG_USB_DEVICE_INITIALIZE_AT_BOOT=y for the debug console. The samples have this setting disabled (=n). Is there anything I have to take care of?
2. The samples seem to enable some interrupt driven drivers (CONFIG_UART_INTERRUPT_DRIVEN=y). I originally had this configuration disabled (not mentioned) in my zephyr console version, and it worked just fine. What is the impact of this setting?

Second, I have to find a way how to access the cdc_acm_uart1 instance in my software. I didn't succeed yet as the samples access the driver by choosing just some device that fits the device class, but I need the specific one. The composite sample is so confusing on accessing the device instances that it didn't help either. I tried dev = DEVICE_DT_GET(cdc_acm_uart1); but this line fails during compiling. I tried dev = DEVICE_DT_GET_OR_NULL(cdc_acm_uart1); but the macro returned zero/null. How can I access the device for handling the data?

Also, I'm a bit stressed by seeing the uart implementation in the samples, using "interrupt handlers" while the cdc_acm interface is just a software layer and the actual interrupts should happen on the USB interface's driver, not the uart implementation. Isn't there a more convenient driver implementation? If I implemented such a driver in FreeRTOS, I would just define queues that handle the data (a send queue that invokes a higher priority send task once data is queued, and a receive queue that just holds all received data for polling by the software. Some virtual uart (cdc_acm) interface like this would be much more convenient than simulating uart with interrupts (and dynamically enabling/disabling interrupts, which is kind of opaque in the samples, too).

[ovrebekk]

Hi Michael

I'm using CONFIG_USB_DEVICE_INITIALIZE_AT_BOOT=y for the debug console. The samples have this setting disabled (=n). Is there anything I have to take care of?

As long as you make sure not to call usb_enable(..) from main, like the examples, enabling this configuration should work fine. 

The samples seem to enable some interrupt driven drivers (CONFIG_UART_INTERRUPT_DRIVEN=y). I originally had this configuration disabled (not mentioned) in my zephyr console version, and it worked just fine. What is the impact of this setting?

Without this configuration enabled you are left having to use a polling API for CDC ACM which is not very efficient, using the interrupt driven API is better. Even if you didn't include it in your project configuration I expect it was included indirectly by some other config parameter. To check whether or not this configuration is set you can look for it in the autoconf.h file in the build/zephyr/include/generated folder, and see if it is set to 1 or 0. 

but this line fails during compiling. I tried dev = DEVICE_DT_GET_OR_NULL(cdc_acm_uart1); but the macro returned zero/null. How can I access the device for handling the data?

Since you have node labels defined in your devicetree overlay you can get a reference to a specific instance like this:

const struct device * dev_cdc1 = DEVICE_DT_GET(DT_NODELABEL(cdc_acm_uart1));

Also, I'm a bit stressed by seeing the uart implementation in the samples, using "interrupt handlers" while the cdc_acm interface is just a software layer and the actual interrupts should happen on the USB interface's driver, not the uart implementation. Isn't there a more convenient driver implementation?

The terms interrupt handler and event handler are used a bit interchangeably in the docs. Essentially a lot of the asynchronous drivers in Zephyr use such a model where you register a callback which will be triggered asynchronously when a hardware interrupt happens. It doesn't mean that there is a one to one mapping between the interrupts being triggered, and the callback being run. Usually many interrupts will be handled by the driver directly, while some interrupts are forwarded to the registered event/interrupt handler for processing. 

As mentioned you can use a pure polling interface as well, but this is less efficient since you don't get any notification when there is data to receive. 

Best regards
Torbjørn

[puz_md]

Hi ovrebekk,

thanks a lot for the reply! I managed to access the new device successfully with your code. I'm still a bit unsure about the abstraction levels in the device tree...

In the meantime, I found a nice Nordic Academy course that explains UART basics pretty well, so I know about the three different interfaces for normal UART interfaces (polling, interrupt, async). I assume async is using the EasyDMA feature of the UART devices, which is pretty convenient (but I wouldn't want to include the PPI driver in Zephyr, as I use it bare metal in my code at the moment.)

As I mentioned before, I am only using the USB interface (and I hope the USB driver uses EasyDMA already), so I am wondereing which effect have CONFIG_UART_INTERRUPT_DRIVEN=y and/or CONFIG_UART_ASYNC_API=y in this scenario. Is it just the software interface? Would it be wise to use the ASYNC API in this case? Will the PPI driver also be needed if I use UART only via the CDC_ACM service?

Generally speaking, I am just planning to implement a simple service interface over the CDC_ACM interface, and have the debug outputs from the Kernel over the other CDC_ACM instance. (In our prototype hardware, we usually don't have access to the JTAG signals, at least after I finish integrating mcuboot and the bootloader, So, a simple polling interface would be most efficient in my case. But it would need a large enough FIFO... I have not seen where I can configure the FIFO size for the polling interface and am not sure how efficiently it is handled via USB and CDC_ACM service. It would be nice if you could give me some feedback about this.

One more question, does the configuration of the UART interface also affect the efficiency of printk() and the LOG interface, if they are linked ("chosen") to a CDC_ACM device? Today, I created a high throughput debug output (probably more than 100 kb/s or 1 mbps via printk(), and sometimes, the CDC_ACM connection to PuTTY crashed awfully. (I think even the software / thread itself crashed.) Might this change if I configure the ASYNC API instead of the Polling API for UART?

By the way, is there a way to receive data via the Zephyr console connection? Something like poll_inputk() ... or is the input data handled by one of the consoles or debug interfaces already?

Best regards,

Michael

[ovrebekk]

Hi Michael

puz_md said:

I assume async is using the EasyDMA feature of the UART devices, which is pretty convenient (but I wouldn't want to include the PPI driver in Zephyr, as I use it bare metal in my code at the moment.)

The async driver utilizes the EasyDMA feature extensively, yes, and is a good choice if you need to push a lot of data through the UART interface. 

I know it will use PPI in certain configurations, but I am not sure PPI is always used. Either way, I would recommend using the nrfx_ppi driver in case you need to use PPI in your application, then you can dynamically allocate PPI channels and ensure that there is no conflict between your own channels and those used by other drivers or libraries. 

puz_md said:

As I mentioned before, I am only using the USB interface (and I hope the USB driver uses EasyDMA already), so I am wondereing which effect have CONFIG_UART_INTERRUPT_DRIVEN=y and/or CONFIG_UART_ASYNC_API=y in this scenario. Is it just the software interface? Would it be wise to use the ASYNC API in this case?

The USB driver uses EasyDMA, yes. 

The CDC_ACM class uses CONFIG_UART_INTERRUPT_DRIVEN for the interface only, it does not reuse any part of the implementation. In other words the only thing it shares with the regular UART interrupt driver is the API (interface), nothing else. 

Whether or not you enable CONFIG_UART_ASYNC_API will have no impact on the cdc_acm driver, it doesn't support this interface at all (and as such is not dependent on it either). 

puz_md said:

I have not seen where I can configure the FIFO size for the polling interface and am not sure how efficiently it is handled via USB and CDC_ACM service. It would be nice if you could give me some feedback about this.

I have to do a bit of checking internally. I believe one way to do this is to use the shell module, and have it use the cdc_acm backend. Then you should get buffering and logging more or less for free. 

puz_md said:

One more question, does the configuration of the UART interface also affect the efficiency of printk() and the LOG interface, if they are linked ("chosen") to a CDC_ACM device? Today, I created a high throughput debug output (probably more than 100 kb/s or 1 mbps via printk(), and sometimes, the CDC_ACM connection to PuTTY crashed awfully. (I think even the software / thread itself crashed.) Might this change if I configure the ASYNC API instead of the Polling API for UART?

It should not make a difference, no. If something crashes on the host PC I don't think the issue is on the nRF side, it sounds more like a problem on the PC side. Have you tried with a different serial terminal?

puz_md said:

By the way, is there a way to receive data via the Zephyr console connection? Something like poll_inputk() ... or is the input data handled by one of the consoles or debug interfaces already?

The shell module would solve this issue surely. Again I will try to collect a bit more information about this module and see if it would be a good fit in your case. 

Best regards
Torbjørn

[puz_md]

Hi Torbjørn

Thanks for your support so far.

ovrebekk said:

Whether or not you enable CONFIG_UART_ASYNC_API will have no impact on the cdc_acm driver, it doesn't support this interface at all (and as such is not dependent on it either). 

The ASYNC UART interface indeed looks interesting, but if it is not supported by the cdc_acm driver, there is no need for me to switch over. I think the interrupt driven frontend is too complex for my scenario, so for now, I decided to continue with the polling interface. It's just a bit sad that you have to manually read each byte and cannot copy over a larger buffer in one step.

By the way, when I checked the ASYNC interface, It seemed to me that receiving partially filled buffers can only be done with a timeout (using timer and ppi) which switches the buffers. I think it would be nice if you could manually trigger a buffer switch by "polling" the current buffer content, causing a receive buffer switch. Please have a look at the diagram under question 2 in my older ticket where I just described this kind of implementation. It doesn't need any timer or ppi. Is there any chance that the ASYNC interface will be extended with such a feature? Might be useful for future projects that use physical UART again.

ovrebekk said:

I have to do a bit of checking internally. I believe one way to do this is to use the shell module, and have it use the cdc_acm backend. Then you should get buffering and logging more or less for free. 

I already found out that the cdc_acm driver does some buffering with the polling interface (sending many characters from my PC shell and reading one byte after another with a delay of 1 second), it just would be nice to know how much buffer is available (or where I can configure the size).

I'm not sure if the shell module is the way to go, as I have to implement an interface for machine communication, with packet synchronization etc., so no data should be handled in the background by the shell.

ovrebekk said:

It should not make a difference, no. If something crashes on the host PC I don't think the issue is on the nRF side, it sounds more like a problem on the PC side. Have you tried with a different serial terminal?

This might have been a misunderstanding, the software that crashed was actually on the embedded side, when I sent too much data via printk(). I sent logging data with 1 kHz which was intended to be written into a csv file via the terminal.

Although I noticed yesterday that the nRF Serial Termial in nRF Connect plugin for Visual Studio Code seems to work pretty unreliably in some cases... it did not find my COM ports from the nRF5340 anymore, or I could not open a second cdc_acm port once I connected to the first one, or there was extensive lag before I could open the second console, or the cdc_acm ports were not shown at all anymore for selection... while I had these problems in Visual Studio Code, I could connect to the cdc_acm ports vial PuTTY without any problem. At the moment, I don't know what triggered this problem. Today, it seems to work better again. Well... just wanted to mention that the nRF Commect for VS Code plugin seems to be a bit unreliable here.

ovrebekk said:

puz_md said:

By the way, is there a way to receive data via the Zephyr console connection? Something like poll_inputk() ... or is the input data handled by one of the consoles or debug interfaces already?

The shell module would solve this issue surely. Again I will try to collect a bit more information about this module and see if it would be a good fit in your case. 

I solved this topic in the meantime: I can just use uart_poll_in() and uart_poll_out() with the device which is linked to the system outputs.

Conclusion:

I'm pretty much on my way with the implementation at the moment. For the time being, I will continue with the polling interface for cdc_acm. If you still manage to find out anything about the internals like fifo/buffer sizes, please let me know. Otherwise, I'll just try and see.

For any other developers coming across this ticket, I advise to check out Lesson 5 – Serial communication (UART) in the DevAcademy, course _nRF Connect SDK Fundamentals', and the API link in the driver chapter.

Best regards,

Michael

[ovrebekk]

Hi Michael

puz_md said:

It's just a bit sad that you have to manually read each byte and cannot copy over a larger buffer in one step.

Wouldn't this be just as cumbersome as using the interrupt driver? 

puz_md said:

By the way, when I checked the ASYNC interface, It seemed to me that receiving partially filled buffers can only be done with a timeout (using timer and ppi) which switches the buffers. I think it would be nice if you could manually trigger a buffer switch by "polling" the current buffer content, causing a receive buffer switch.

What problem would this feature solve? 

If you are polling on a timer it will work more or less exactly the same way as a timeout. The only difference I can see with the polling method is that you have more exact control with the timing, since you can choose to poll whenever you want. 

There is no way to read the number of received bytes unfortunately. The only way to force a flush of the RX buffers is to disable RX, but unless you use HWFC this is risky since the UART RX will be disabled for a moment and you might lose incoming data. If HWFC is used the flow control pins will be de-asserted and communication should be delayed until the UART RX is enabled again. 

puz_md said:

Is there any chance that the ASYNC interface will be extended with such a feature?

Generally we try not to touch low level drivers, since there is always a risk of introducing bugs or unintended side effects that could affect everyone that uses the driver. Also, like most engineering companies we are chronically low on software resources, and have to prioritize where to focus our time

A strength of the open source development model is that you can actually promote updates yourself, and you could issue a pull request with the changes you would like to see in the driver, but obviously there is no guarantee that the PR will be accepted. 

puz_md said:

I sent logging data with 1 kHz which was intended to be written into a csv file via the terminal.

Interesting. Did you do the math to check whether or not the UART bandwidth is sufficient for the data you are trying to send? 

Possibly this would work better if you use the log API, it provides a warning if the log buffers overflow, and you also have a deferred mode to avoid the log calls blocking further code execution. 

puz_md said:

Well... just wanted to mention that the nRF Commect for VS Code plugin seems to be a bit unreliable here.

If you want us to look into this please open a separate ticket on it, and provide a bit more details regarding what you are doing, which versions of the tools you are running and so forth. 

puz_md said:

I'm pretty much on my way with the implementation at the moment. For the time being, I will continue with the polling interface for cdc_acm. If you still manage to find out anything about the internals like fifo/buffer sizes, please let me know. Otherwise, I'll just try and see.

Good to hear that you got something that works fine (if not perfectly). 

I am a bit curious how you are using polling mode though, as far as I can tell the CDC ACM driver will only work with the interrupt driven API, and actively selects it when you enable the USB_CDC_ACM configuration as shown here.

The buffers should be scaled after the USB bulk endpoint buffers, which are 64 bytes maximum. 

Best regards
Torbjørn 

[puz_md]

Hi Torbjørn,

ovrebekk said:

Wouldn't this be just as cumbersome as using the interrupt driver? 

ovrebekk said:

What problem would this feature solve? 

If you are polling on a timer it will work more or less exactly the same way as a timeout. The only difference I can see with the polling method is that you have more exact control with the timing, since you can choose to poll whenever you want.

Benefit #1: No preemption, which means better performance in realtime systems

Benefit #2: Less complexity and more performance: You can parse the data immediately in the task's context, byte by byte, without the need to copy over the data to a second buffer in the interrupt handler and handle it later with at a lower priority.

Let's say I parse data byte by byte to detect my packet boundaries. In a polling task, I can immediately handle the packet after parsing the last byte, then start over with my packet buffer, continuing with the next byte of the recently obtained polling buffer.

Benefit #3: I do not need the timer and ppi modules, they will be fully available for other functions. Thus, less complexity in the driver implementation, too.

ovrebekk said:

There is no way to read the number of received bytes unfortunately. The only way to force a flush of the RX buffers is to disable RX, but unless you use HWFC this is risky since the UART RX will be disabled for a moment and you might lose incoming data. If HWFC is used the flow control pins will be de-asserted and communication should be delayed until the UART RX is enabled again. 

Oh, I resolved that already in the ticket I linked last time. Let me copy over the flow chart:

In short, you manually trigger the STOPRX task, and the ENDRX event automatically triggers STARTRX, thus performing a buffer switch. All you have to do is wait for the ENDRX bit in software, and the old buffer is available for processing. You can do this without interrupts/preemption.

Nordic builds such beautifully designed hardware. But it seems most times, you have to go for bare metal to get all of the benefits...

By the way, you're right, the RXD.AMOUNT value is only updated after STOPRX is triggered. If there would also be a dynamically updated variable, it would be possible to use a single EasyDMA buffer as a circular fifo (with the tail being implemented in software).

ovrebekk said:

Interesting. Did you do the math to check whether or not the UART bandwidth is sufficient for the data you are trying to send? 

I'm using the USB interface, so configured UART speed does not matter... or do you mean the USB speed limit? My data was sent at less than 1 Mbps, the USB device should work at 12 Mbps (full-speed) according to product specification.

ovrebekk said:

I am a bit curious how you are using polling mode though, as far as I can tell the CDC ACM driver will only work with the interrupt driven API, and actively selects it when you enable the USB_CDC_ACM configuration as shown here.

Now I'm a bit confused, too. If I set CONFIG_UART_INTERRUPT_DRIVEN=y in prj.conf, the line becomes bright blue and I don't get the hover message that this value was already set elsewhere that I get on the dark blue configuration lines, so I assumed it was set to =n. But when I check for the definition in VS Code (F12), I get forwarded to the sparkfun_sara_r4's kconfig.defconfig which sets it to =y (although I do not use this shield). Maybe the lookup feature of VS Code is not so reliably...

That said, whether I have CONFIG_UART_INTERRUPT_DRIVEN=y in my proj.conf or not, I can still use the polling interface with uart_poll_in() and uart_poll_out(). I guess it is still possible as I don't register an interrupt handler...

ovrebekk said:

Also, like most engineering companies we are chronically low on software resources, and have to prioritize where to focus our time

Haha, I know too well... it is so sad. I would implement a nice driver myself if I had the time...

Best regards,

Michael