Openthread RCP NoBufs

Hi,

I'm upgrading our RCP firmware that runs on a custom nrf52840 dongle. It was using openthread bare metal on an old version (clone of openthread repo and run make -f examples/Makefile-nrf52840 USB=1) and I'm trying to use NCS v1.8.0 samples/openthread/coprocessor as a start for the upgrade.

The main reason for the upgrade is some platform NoBufs we are seeing on our border router log. We have a network with 150 devices, with each one sending 2 packages per minute, with ~600 bytes each package, in a noisy environment (random WiFi transactions).

We could reduce the NoBufs errors by increasing some UART buffer macro configurations on old firmware (openthread bare metal), but when using the NCS (without bootloader for now) we have a worse result, even after optimizing some kconfig from the sample.
At the sample, I've set `CONFIG_LOG`, added kconfigs `OPENTHREAD_THREAD_VERSION_1_2`, `NRF_802154_ENCRYPTION`, `IEEE802154_2015`, `NET_PKT_TXTIME`;
Also the following modifications:  
```
CONFIG_OPENTHREAD_COPROCESSOR_UART_RING_BUFFER_SIZE=4096
CONFIG_NET_PKT_RX_COUNT=64
CONFIG_NET_BUF_RX_COUNT=64
CONFIG_NET_PKT_TX_COUNT=256
CONFIG_NET_BUF_TX_COUNT=256
CONFIG_NRF_802154_RX_BUFFERS=32
```

I'm building the RCP with:
```
west build --board ${OUR_CUSTOM_BOARD} -- -DCONF_FILE=overlay-minimal_rcp.conf -DOVERLAY_CONFIG="overlay-usb.conf overlay-logging.conf"
```

In order to increase UART baudrate, I'm running the border router container with env var: `RADIO_URL: spinel+hdlc+uart:///dev/ttyACM0?uart-baudrate=460800`. The ot-br-posix image used is the one that points to the same Openthread version used by the NCS 1.8.0.

The logs on the border router container were filtered to catch the `NoBufs` messages:
```
openthread_otbr | Apr 1 21:54:27 predito-default4 otbr-agent[549]: 00:02:24.561 [WARN]-PLAT----: Error waiting response: NoBufs
openthread_otbr | Apr 1 21:54:27 predito-default4 otbr-agent[549]: 00:02:24.596 [WARN]-PLAT----: Error processing result: NoBufs
openthread_otbr | Apr 1 21:54:27 predito-default4 otbr-agent[549]: 00:02:24.596 [WARN]-PLAT----: Error waiting response: NoBufs
openthread_otbr | Apr 1 21:54:27 predito-default4 otbr-agent[549]: 00:02:24.629 [WARN]-PLAT----: Error processing result: NoBufs
openthread_otbr | Apr 1 21:54:27 predito-default4 otbr-agent[549]: 00:02:24.629 [WARN]-PLAT----: Error waiting response: NoBufs
openthread_otbr | Apr 1 21:54:27 predito-default4 otbr-agent[549]: 00:02:24.650 [WARN]-PLAT----: Error processing result: NoBufs
openthread_otbr | Apr 1 21:54:27 predito-default4 otbr-agent[549]: 00:02:24.651 [WARN]-PLAT----: Error waiting response: NoBufs
openthread_otbr | Apr 1 21:54:28 predito-default4 otbr-agent[549]: 00:02:25.277 [WARN]-PLAT----: Error processing result: NoBufs
openthread_otbr | Apr 1 21:54:28 predito-default4 otbr-agent[549]: 00:02:25.277 [WARN]-PLAT----: Error waiting response: NoBufs
```

We could not see any log related to buffer fullness on the RCP though. The logs caught with SWD/RTT debug on the RCP has only Ack losses:
```
[00:01:25.402,008] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:0/2, len:126, seqnum:136, type:Data, src:d29d3f26e645ffa1, dst:56bc652a6~
[00:01:25.409,729] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:1/2, len:126, seqnum:136, type:Data, src:d29d3f26e645ffa1, dst:56bc652a6~
[00:01:25.415,863] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:2/2, len:126, seqnum:136, type:Data, src:d29d3f26e645ffa1, dst:56bc652a6~
[00:01:25.535,552] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:0/2, len:126, seqnum:136, type:Data, src:d29d3f26e645ffa1, dst:56bc652a6~
[00:01:25.542,785] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:1/2, len:126, seqnum:136, type:Data, src:d29d3f26e645ffa1, dst:56bc652a6~
[00:01:25.553,771] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:2/2, len:126, seqnum:136, type:Data, src:d29d3f26e645ffa1, dst:56bc652a6~
[00:01:25.569,763] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:0/2, len:126, seqnum:136, type:Data, src:d29d3f26e645ffa1, dst:56bc652a6~
[00:01:25.577,484] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:1/2, len:126, seqnum:136, type:Data, src:d29d3f26e645ffa1, dst:56bc652a6~
[00:01:25.583,648] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:2/2, len:126, seqnum:136, type:Data, src:d29d3f26e645ffa1, dst:56bc652a6~
[00:01:25.616,821] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:0/2, len:126, seqnum:137, type:Data, src:d29d3f26e645ffa1, dst:56bc652a6~
[00:01:25.623,229] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:1/2, len:126, seqnum:137, type:Data, src:d29d3f26e645ffa1, dst:56bc652a6~
[00:01:25.659,729] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:0/2, len:21, seqnum:139, type:Data, src:0x1000, dst:0x1011, sec:yes, ack~
[00:01:25.663,879] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:1/2, len:21, seqnum:139, type:Data, src:0x1000, dst:0x1011, sec:yes, ack~
[00:01:25.666,442] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:2/2, len:21, seqnum:139, type:Data, src:0x1000, dst:0x1011, sec:yes, ack~
[00:01:26.513,610] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:0/2, len:123, seqnum:141, type:Data, src:d29d3f26e645ffa1, dst:76449dd88~
[00:01:26.519,989] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:1/2, len:123, seqnum:141, type:Data, src:d29d3f26e645ffa1, dst:76449dd88~
[00:01:26.527,282] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:2/2, len:123, seqnum:141, type:Data, src:d29d3f26e645ffa1, dst:76449dd88~
[00:01:26.578,125] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:0/2, len:123, seqnum:141, type:Data, src:d29d3f26e645ffa1, dst:76449dd88~
[00:01:26.586,059] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:1/2, len:123, seqnum:141, type:Data, src:d29d3f26e645ffa1, dst:76449dd88~
[00:01:26.593,719] <inf> net_openthread: [INFO]-MAC-----: Frame tx failed, error:NoAck, retries:2/2, len:123, seqnum:141, type:Data, src:d29d3f26e645ffa1, dst:76449dd88~
```


1) Is there any recommendation on how we could change our configurations so we could vanish or at least reduce the NoBufs errors?
2) Or is there some extra logs we should enable in order to see where and why the NoBufs are happening?
3) Could it be possible for the NoBuf to be at the border router side, although the logs say it is on the PLAT layer?

Thanks!
Best regards,
Rodrigo

Parents Reply
  • Hi,

    CONFIG_OPENTHREAD_NUM_MESSAGE_BUFFERS was already increased to 4096 on border router. At the RCP side, I understand that the RCP stack doesn't use the Openthread message buffers right?

    I increased it anyway to 4096 on RCP and it didn't work either. I'm suspecting the problem is on the RCP because the border router logs the error as [WARN]-PLAT---- and plat would mean platform.

    Besides, the RCP seems to be crashing some times.

    In overlay-minimal_rcp.conf I did not set the sl opensource and mbedtls kconfig flags, so it could use NCS proprietary ones (CONFIG_NRF_802154_SL_OPENSOURCE and CONFIG_OPENTHREAD_MBEDTLS_CHOICE lines deleted). I suppose it should work, right?

    Thanks!

Children
No Data
Related