OpenThread and MQTT over TLS is single program.

[Grzegorz Blach]

Hi,

I use nrf52840 processor with esp32c3 as WiFi modem. I need to run OpenThread network and MQTT client on these processors. When I run only OpenThread network, it works correctly. When I run only MQTT client via WiFi, it works correctly too. But when I try to merge this two programs into one, only unencrypted MQTT works.

When I try to transfer some data via OpenThread network I get this error:
<err> coap_utils: Failed to create socket 106

And when I try to connect to MQTT broker over TLS I get this error:

<err> net_sock_tls: TLS handshake error: -7780
<err> net_sock: invalid access on sock 0 by thread 0x2000137

Here is my OpenThread program: https://github.com/Whirla/whbr_demos/tree/main/whbr_ot_coap
Here is my MQTT program: https://github.com/Whirla/whbr_demos/tree/main/whbr_wifi_mqtt

Here is merged program: https://github.com/Whirla/whbr_demos/tree/main/whbr_both
And here is a hardware I use: https://github.com/gblach/HexGateway

Could you point me out why encryption is not working in my program?

[Sigurd Hellesvik]

Hi Grzegorz,

Could the solution be as simple as adding more available sockets by increasing CONFIG_NET_SOCKETS_TLS_MAX_CONTEXTS?

Regards,
Sigurd Hellesvik

[Grzegorz Blach]

I tried to set CONFIG_NET_SOCKETS_TLS_MAX_CONTEXTS to 2 and to 10, but that didn't change anything. I still have the same two errors.

[Sigurd Hellesvik]

Hi

I did try the configuration which I was able to build for with our coap server+client samples, and I were not able to reproduce this yet.

I were able to build fine with CONFIG_WIFI, but CONFIG_WIFI_ESP_AT does not work for me.
Therefore, I tried to set different configurations selected by CONFIG_WIFI_ESP_AT manually, but did not fine the problematic one.

You could test some configurations set by of CONFIG_WIFI_ESP_AT, to find which specific config causes the error? (iterate trough the ""config selected by"-tree")

When you send CoAP messages over OpenThread, where do you send the data to? (another node in the network, or an external server)

From zephyr errno, 106 looks like EAFNOSUPPORT, which means that the address family is not supported.
Is IPv6 still supported after you select CONFIG_WIFI_ESP_AT?

Regards,
Sigurd Hellesvik

[Grzegorz Blach]

I didn't found any other options other than CONFIG_WIFI_ESP_AT that cause this issue. But I found -EAFNOSUPPORT cames from drivers/wifi/esp_at/esp_offload.c:608 because family variable has the same value that I pass to coap_init() function.

I've added /boards/nrf52840dk_nrf52840.overlay to OpenThread only sample, so you can reproduce this issue on nrf52840dk. You don't need to have esp32c3 connected, because it is not used by this program, only driver is enabled.

Sigurd Hellesvik said:

When you send CoAP messages over OpenThread, where do you send the data to? (another node in the network, or an external server)

I'm trying to send data to another node in the same OT network.

Sigurd Hellesvik said:

Is IPv6 still supported after you select CONFIG_WIFI_ESP_AT?

"net iface" command shows that OpenThread interface has assigned seven IPv6 addresses:

rtt:~$ net iface

Interface 0x20000a08 (IP Offload) [1]
=====================================
Interface is down.

Interface 0x20000a14 (OpenThread) [2]
=====================================
Link addr : F4:CE:36:EF:04:8D:D0:E1
MTU       : 1280
Flags     : AUTO_START,IPv4,IPv6
IPv6 unicast addresses (max 6):
  fd04:93c0:e52a:0:fa7d:45d:d6e:95ca autoconf preferred infinite meshlocal
  fe80::b8e9:def7:b096:806c autoconf preferred infinite
IPv6 multicast addresses (max 8):
  ff33:40:fd04:93c0:e52a::1
  ff32:40:fd04:93c0:e52a::1
  ff02::1
  ff03::1
  ff03::fc
  ff02::2
  ff03::2
IPv6 prefixes (max 2):
  <none>
IPv6 hop limit           : 64
IPv6 base reachable time : 30000
IPv6 reachable time      : 23751
IPv6 retransmit timer    : 0
IPv4 unicast addresses (max 1):
  <none>
IPv4 multicast addresses (max 1):
  <none>

[Sigurd Hellesvik]

Hi

Since you enabled the nrf52840dk, I were able to load your project into VS Code and debug it,
So I have found the issue, but not a fix for it yet.

When the code tries to get a socket/context from the offloaded IP stack at zephyr/subsys/net/ip/net_context.c:331, it will call the esp_get function from zephyr/drivers/wifi/esp_at/esp_offload.c:597. This function returns  EAFNOSUPPORT (error 106), and the coap fails to create the socket.

You can also see a fixme in net_context.c, which is likely what causes the issue:
"
/* FIXME - Figure out a way to get the correct network interface
* as it is not known at this point yet.
*/
"

I hope this helps you some.

We here on support will he at vacation the rest of the Easter.
If you want some more help, update me sometime next week and let me know if you found anything. Then I will have another look.

Happy Easter! Regards,
Sigurd Hellesvik

[Grzegorz Blach]

Hi,

I've applied this small patch and cleared NET_IF_IPV6 flag on default interface. Now I have working OpenThread and unencrypted MQTT in single program. But when I've tried to enable encryption for MQTT communication I still get this error:

<err> net_sock_tls: TLS handshake error: -7780
<err> net_sock: invalid access on sock 0 by thread 0x2000137

Do you have any idea where should I look to solve this problem?

Unfortunately it is impossible to reproduce this error without having esp32c3 connected to nrf52840dk. I'm guessing you don't have appropriate device, do you?

[Sigurd Hellesvik]

Hi

I have not been able to find a lot more about this quite yet.
As you say, it is hard to reproduce for me, as I do not have an esp32c3.

When looking around, I found a couple of other forums [1], [2] saying error 7780(MBEDTLS_ERR_SSL_FATAL_ALERT_MESSAGE) is due to refusal on the peer side.
Are you able to get some logs from the peers showing why they refuse the connection?

The weird thing here is that you say it works without OpenThread.
OpenThread is closely connect to the crytpo libraries, so this may be related somehow.
Have a look at Hardware-accelerated cryptography.

For our old nRF SDK for Thread and Zigbee, we had documentation on Mandatory Mbed TLS library, and that thread libraries include some of mbedTLS. Keep in mind that this is for the old SDK, and might not be as relevant for the nRF Connect SDK.
I am looking for similar documentation for the nRF Connect SDK, but have not found it yet.

This is all I got for now, but I will continue looking around a bit.
Let me know if you have found anything in the meantime.

Regards,
Sigurd Hellesvik

[Grzegorz Blach]

Ok, I got it.

I need to enable CONFIG_MBEDTLS_CIPHER_MODE_CBC_ENABLED, because it defaults to y only if !CONFIG_NET_L2_OPENTHREAD.

Now I have OpenThread and encrypted MQTT working in single program.

Thanks you for your help.