ZBOSS NCP debugging

[DaKa]

Hi,

I am using the ZBOSS NCP production ready firmware from NCS v2.0 and I have some issue with serial communication.

I am able to run my application, and I don't have any issue with my test gateway. Meanwhile, the application is extremely unstable at a customer and requires restart every day almost.
The difference between my test hub and at the customer is that he is using the MicroMatic ZB250 device only (arround 15 devices). And I have 11 devices of different types.

Symptoms are following:
- Runs well for some hours
- NCP stops replying to any request
- NCP continues to send ACK when I send requests from host

Is there a way to see logs from the NCP side in order to debug this issue? I am using the nrf52840 dongle.

BR,
Damien

[Jørgen Holmefjord]

Hi,

Did you not manage to enable logging over a second USB CDC ACM instance in your previous ticket (#289809)?

Can you post the configuration you are currently using? Are you not seeing any logs?

Best regards,
Jørgen

[DaKa]

Hi,

I did, but I was using NCS v1.8, and now I moved to v2.0 and I am not able to find the right configuration.
I tried this config:

nrf52840dongle_nrf52840.overlay

/ {
	chosen {
		zephyr,entropy = &rng;
		zephyr,console = &cdc_acm_uart1;
	};

	buttons {
		compatible = "gpio-keys";
		rst_button0: rst_button_0 {
			gpios = <&gpio0 19 (GPIO_PULL_UP | GPIO_ACTIVE_LOW)>;
			label = "Reset button";
		};
	};

	aliases {
		rst0 = &rst_button0;
	};
};

&zephyr_udc0 {
    cdc_acm_uart0: cdc_acm_uart0 {
        compatible = "zephyr,cdc-acm-uart";
        label = "CDC_ACM_0";
    };

    cdc_acm_uart1: cdc_acm_uart1 {
        compatible = "zephyr,cdc-acm-uart";
        label = "CDC_ACM_1";
    };
};

prj_usb.conf

CONFIG_ZIGBEE_UART_DEVICE_NAME="CDC_ACM_0"
CONFIG_ZIGBEE_UART_SUPPORTS_FLOW_CONTROL=y

CONFIG_USB_COMPOSITE_DEVICE=y

CONFIG_USB_DEVICE_STACK=y
CONFIG_USB_DEVICE_REMOTE_WAKEUP=n
CONFIG_USB_DEVICE_PRODUCT="Zigbee NCP"
CONFIG_USB_CDC_ACM=y
CONFIG_UART_LINE_CTRL=y

CONFIG_ZIGBEE=y

CONFIG_ZIGBEE_LIBRARY_NCP_DEV=y

CONFIG_ZIGBEE_HAVE_SERIAL=y
CONFIG_ZIGBEE_HAVE_ASYNC_SERIAL=y
CONFIG_ZIGBEE_UART_RX_BUF_LEN=256
CONFIG_ZIGBEE_UART_TX_TIMEOUT=1000
CONFIG_ZIGBEE_UART_RX_TIMEOUT=1000

CONFIG_ZIGBEE_USE_LEDS=y
CONFIG_ZIGBEE_TIME_KTIMER=y

CONFIG_ZIGBEE_ROLE_COORDINATOR=y

CONFIG_HEAP_MEM_POOL_SIZE=2048

CONFIG_MAIN_THREAD_PRIORITY=7

CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048

# Networking
CONFIG_NET_IPV6_MLD=n
CONFIG_NET_IPV6_NBR_CACHE=n
CONFIG_NET_IPV6_RA_RDNSS=n
CONFIG_NET_IP_ADDR_CHECK=n
CONFIG_NET_UDP=n

CONFIG_LOG=y
CONFIG_LOG_DEFAULT_LEVEL=3
CONFIG_CONSOLE=y
CONFIG_UART_CONSOLE=y
CONFIG_LOG_BACKEND_UART=y

CONFIG_ASSERT=n
CONFIG_ASSERT_NO_COND_INFO=n
CONFIG_ASSERT_NO_MSG_INFO=n
CONFIG_HW_STACK_PROTECTION=y

This is what I get when I try it out on my system (using dmesg)

[5359837.126645] usb 1-1.1.3: new full-speed USB device number 47 using dwc_otg
[5359852.356630] usb 1-1.1.3: device descriptor read/64, error -110
[5360319.656040] usb 1-1.1.2: new full-speed USB device number 48 using dwc_otg
[5360334.916008] usb 1-1.1.2: device descriptor read/64, error -110

BR,
Damien

[DaKa]

I can't figure out how to see logs from the NCP side. I would like to see RX and TX data going from NCP to Host, but I don't know how to enable this. I can only enable ZBOSS trace logs, and not even app logs (from app module) nor zboss_osif logs.

[Jørgen Holmefjord]

Did you try the instructions in the Zigbee NCP "Communication through USB" documentation?

Alternatively, you can configure Zigbee stack logs to be printed in binary format using an independent CDC ACM device of the same nRF USB. Complete the following steps:

1. Set the following Kconfig options:

   • CONFIG_ZBOSS_TRACE_BINARY_LOGGING - This option enables the binary format.

   • CONFIG_ZBOSS_TRACE_USB_CDC_LOGGING - This option selects nRF USB serial over UART serial.

   • CONFIG_ZBOSS_TRACE_LOGGER_DEVICE_NAME - This option selects the serial device for printing Zigbee stack logs, set to "CDC_ACM_1".

2. Create two instances of USB CDC ACM for the application:

   1. Create two entries in the DTS overlay file for the selected board, one for each USB CDC ACM instance. See USB device stack CDC ACM support for more information.

   2. Extend the zephyr_udc0 node in the DTS overlay file to also configure the second USB CDC ACM instance "CDC_ACM_1":

      &zephyr_udc0 {
         cdc_acm_uart0: cdc_acm_uart0 {
            compatible = "zephyr,cdc-acm-uart";
            label = "CDC_ACM_0";
         };
      
         cdc_acm_uart1: cdc_acm_uart1 {
            compatible = "zephyr,cdc-acm-uart";
            label = "CDC_ACM_1";
         };
      };
      

3. Enable the composite USB device driver using the CONFIG_USB_COMPOSITE_DEVICE Kconfig option.

With this configuration, you have two serial ports created by the NCP sample. Use the first one for NCP communication. Use the second serial port for collecting Zigbee stack logs.

[DaKa]

Hello,

This worked well, and I am able to see ZBOSS traces.
But I would like to see zephyr logs as well since I don't know if my issue comes from ZBOSS or from the system. So, I tried to enable a third CDC ACM in the dts overlay and referenced it to the zephyr console like this

chosen {
		zephyr,entropy = &rng;
		zephyr,console = &cdc_acm_uart2;
	};

Then I configure prj_usb.conf like this:

# Configure NCP sample to use USB CDC ACM 0 serial for communication with Host
# and inform application that CDC ACM requires the line control signals to be configured.
CONFIG_ZIGBEE_UART_DEVICE_NAME="CDC_ACM_0"
CONFIG_ZIGBEE_UART_SUPPORTS_FLOW_CONTROL=n

# Configure NCP sample to print ZBOSS stack logs in binary format using "UART_1" device
# CONFIG_ZBOSS_TRACE_BINARY_LOGGING=y
# CONFIG_ZBOSS_TRACE_LOGGER_DEVICE_NAME="UART_1"

#### Use this configuration to print ZBOSS stack logs in binary format using USB "CDC_ACM_1" device
CONFIG_ZBOSS_TRACE_BINARY_LOGGING=y
CONFIG_ZBOSS_TRACE_USB_CDC_LOGGING=y
CONFIG_ZBOSS_TRACE_LOGGER_DEVICE_NAME="CDC_ACM_1"
CONFIG_ZIGBEE_ENABLE_TRACES=y
CONFIG_ZBOSS_TRACE_LOG_LEVEL_DBG=y
CONFIG_ZBOSS_TRACE_MASK=0x00100001

# Two USB CDC ACM instances configured for this sample: CDC_ACM_0 and CDC_ACM_1
CONFIG_USB_COMPOSITE_DEVICE=y
####

CONFIG_USB_DEVICE_STACK=y
CONFIG_USB_DEVICE_REMOTE_WAKEUP=n
CONFIG_USB_DEVICE_PRODUCT="Zigbee NCP"
CONFIG_USB_CDC_ACM=y
CONFIG_UART_LINE_CTRL=y

# Those settings below are common with prj.conf

# CONFIG_NCS_SAMPLES_DEFAULTS=y
CONFIG_LOG=y
CONFIG_LOG_MODE_DEFERRED=y
CONFIG_CONSOLE=y
CONFIG_UART_CONSOLE=y
CONFIG_LOG_DEFAULT_LEVEL=4
CONFIG_USB_MAX_NUM_TRANSFERS=8

CONFIG_ZIGBEE=y

# Add libncp-dev library
CONFIG_ZIGBEE_LIBRARY_NCP_DEV=y

CONFIG_ZIGBEE_HAVE_SERIAL=y
CONFIG_ZIGBEE_HAVE_ASYNC_SERIAL=y
CONFIG_ZIGBEE_UART_RX_BUF_LEN=256
CONFIG_ZIGBEE_UART_TX_TIMEOUT=1000
CONFIG_ZIGBEE_UART_RX_TIMEOUT=1000

CONFIG_ZIGBEE_USE_LEDS=y
CONFIG_ZIGBEE_TIME_KTIMER=y

CONFIG_ZIGBEE_ROLE_COORDINATOR=y

CONFIG_HEAP_MEM_POOL_SIZE=2048

CONFIG_MAIN_THREAD_PRIORITY=7

CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048

# Networking
CONFIG_NET_IPV6_MLD=n
CONFIG_NET_IPV6_NBR_CACHE=n
CONFIG_NET_IPV6_RA_RDNSS=n
CONFIG_NET_IP_ADDR_CHECK=n
CONFIG_NET_UDP=n

And unfortunately this doesn't work. When enabling CONFIG_UART_CONSOLE, the dongle is not showing correctly in dmesg.

Output of dmesg when CONFIG_UART_CONSOLE is disabled (application works but no zephyr logs are showing):

[5966393.200551] usb 1-1.1.3: new full-speed USB device number 101 using dwc_otg
[5966393.345622] usb 1-1.1.3: New USB device found, idVendor=2fe3, idProduct=0100, bcdDevice= 3.00
[5966393.345644] usb 1-1.1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[5966393.345660] usb 1-1.1.3: Product: Zigbee NCP
[5966393.345675] usb 1-1.1.3: Manufacturer: ZEPHYR
[5966393.345690] usb 1-1.1.3: SerialNumber: 1E2D14F7A543460F
[5966393.348321] cdc_acm 1-1.1.3:1.0: ttyACM0: USB ACM device
[5966393.352672] cdc_acm 1-1.1.3:1.2: ttyACM1: USB ACM device
[5966393.356679] cdc_acm 1-1.1.3:1.4: ttyACM2: USB ACM device
[5966393.679857] usb 1-1.1.3: USB disconnect, device number 101


Output of dmesg when CONFIG_UART_CONSOLE is enabled:

[5966460.280544] usb 1-1.1.2: new full-speed USB device number 102 using dwc_otg
[5966475.540510] usb 1-1.1.2: device descriptor read/64, error -110
[5966490.900496] usb 1-1.1.2: device descriptor read/64, error -110
[5966491.120488] usb 1-1.1.2: new full-speed USB device number 103 using dwc_otg

BR,
Daka