MQTT over Thread

[Frog]

Today I am attempting to set up an MQTT connection from a Thread device to an MQTT server on the Internet (for better or for worse, if this is a terrible idea I'm open to suggestions).

I've done some preliminary investigation using the Thread CLI example and am now starting to develop code using the Thread API in SDK 2.6.0.  I have a border router running on an RPi with NAT64.

At this point I can do a DNS lookup using otDnsClientResolveIp4Address, sending the request to fd34:e64f:ccd9:02:0:0:808:808 (i.e. 8.8.8.8) and I get back the address of the MQTT server as a synthesized IPv6 address.  I can ping this address and get a response.

The next step is to attempt an MQTT connection. 

I start by calling getaddrinfo() to resolve the IP address of the broker, this fails with -ENOENT, it's the same whether I use hints.ai_family = AF_INET or AF_INET6.

So instead I tried using the address returned by the earlier DNS lookup.  Then I call mqtt_connect() but get -EPROTOTYPE (protocol wrong type for socket).

I see some discussion in DevZone from people trying to do similar things but none quite the same as this.  Perhaps I'm being naive thinking that it should 'just work'?

Any suggestions will be gladly received.

Thanks,

  Frog

[Charlie]

Hi Frog,

This has been discussed in many places before. Just share the latest discussion I had with another developer on the MQTT over Thread topic: https://devzone.nordicsemi.com/f/nordic-q-a/105692/nrf52840-mqtt-coap/455746.

This is neither suggested nor easy to make directly since some configurations need to explore. You can refer to the following implementations to review your codes. If the problem still exists, we can continue the discussion here.

 RE: Error -22 in mqtt_connect() - nRF52840dk with Azure IoT Hub using OpenThread and TCP 

fgervais/project-nrf-thread-switch: Home Assistant (MQTT) wireless button with 10 years of battery life. (github.com)

Best regards,

Charlie

[Frog]

Thanks Charlie,

  fgervais' project has got me some way down the road; I'm able to connect to a border router, perform a DNS lookup and start to open a connection to my MQTT server.  However, the server requires some specific ciphersuites so the TLS negotiation is failing.

I seem to be having difficulty in getting the ciphersuites set up correctly without breaking the encryption for openThread - at the time of writing my code is failing to set the OT network key because it's stored in an encrypted form within openThread and the encryption is broken.

What I'm not clear on is how the encryption libraries used by openThread and MQTT/TLS/SSL should be configured to work together - is there a recommended approach to this?

Here's my prj.conf as it currently is, if there are any obvious shortcomings please let me know, thanks.

# OpenThread

CONFIG_OPENTHREAD_NORDIC_LIBRARY_MASTER=y

CONFIG_NET_L2_OPENTHREAD=y

CONFIG_OPENTHREAD_SOURCES=y

CONFIG_OPENTHREAD_NORDIC_LIBRARY=n

CONFIG_OPENTHREAD_CSL_RECEIVER=y

CONFIG_OPENTHREAD_SLAAC=y

CONFIG_OPENTHREAD_CHANNEL=11

CONFIG_OPENTHREAD_TCP_ENABLE=y

CONFIG_OPENTHREAD_MBEDTLS_DEBUG=y

CONFIG_OPENTHREAD_TIME_SYNC=y

# Networking options

CONFIG_NETWORKING=y

CONFIG_NET_CONFIG_SETTINGS=y

CONFIG_NET_TCP=y

CONFIG_NET_UDP=y

CONFIG_NET_SOCKETS=y

CONFIG_NET_IPV6=y

CONFIG_NET_IPV4=n

CONFIG_NET_CONFIG_NEED_IPV6=y

CONFIG_NET_CONFIG_NEED_IPV4=n

CONFIG_NET_LOG=y

CONFIG_NET_IF_LOG_LEVEL_DBG=y

CONFIG_NET_IPV6_LOG_LEVEL_DBG=y

CONFIG_NET_CONFIG_LOG_LEVEL_DBG=y

CONFIG_NET_CONN_LOG_LEVEL_DBG=y

CONFIG_NET_SOCKETS_LOG_LEVEL_DBG=y

#CONFIG_NET_CORE_LOG_LEVEL_DBG=y

CONFIG_NET_SOCKETS_SOCKOPT_TLS=y

#CONFIG_NET_TCP_LOG_LEVEL_DBG=y

CONFIG_NET_BUF_TX_COUNT=64

CONFIG_NET_TX_STACK_SIZE=4096

CONFIG_NET_RX_STACK_SIZE=4096

CONFIG_NET_SHELL=y

CONFIG_NET_SOCKETS_POSIX_NAMES=y

CONFIG_NET_CONNECTION_MANAGER=n

CONFIG_NET_CONTEXT_NET_PKT_POOL=y

# MQTT

CONFIG_MQTT_LIB=y

CONFIG_MQTT_LIB_TLS=y

CONFIG_MQTT_CLEAN_SESSION=y

CONFIG_MQTT_LOG_LEVEL_DBG=y

CONFIG_MQTT_KEEPALIVE=600

# Crypto

CONFIG_PSA_CRYPTO_DRIVER_CC3XX=y

CONFIG_MBEDTLS=y

CONFIG_MBEDTLS_SHA1_C=y

CONFIG_MBEDTLS_LEGACY_CRYPTO_C=y

CONFIG_MBEDTLS_X509_CRT_PARSE_C=y

CONFIG_NORDIC_SECURITY_BACKEND=n

CONFIG_ENTROPY_GENERATOR=y

CONFIG_MBEDTLS_ENABLE_HEAP=y

CONFIG_MBEDTLS_HEAP_SIZE=32768

CONFIG_MBEDTLS_TLS_LIBRARY=y

[Charlie]

Hi Frog,

Thanks for the update, good to hear that you have explored 

For the MQTT server side, you should be able to configure the cipher suites from server side, this is important because the device wouldn't support all cipher suites and TLS version due to resource limitation and mbed TLS library implemenation.

You can find the following page related to AWS IoT MQTT server TLS configuration.

https://docs.aws.amazon.com/iot/latest/developerguide/transport-security.html#tls-policy-table

https://docs.aws.amazon.com/iot/latest/developerguide/iot-endpoints-tls-config.html#custom-tls-console

For the device side, you need TLS library to load and use the certifications when you build TLS connection with server. Zephyr has samples about how to use mbed TLS library. You can refer to the following discussion about its implementation with OpenThread. 

(+) Adaption to TCP+TLS on top of OpenThread. - Nordic Q&A - Nordic DevZone - Nordic DevZone (nordicsemi.com)

Best regards,

Charlie

[Frog]

Sadly I don't have control over the server; there's a list of about a dozen supported ciphersuites.  I've chosen to go with ECDHE-RSA-AES256-SHA384.  I'd hoped that this would be as simple as adding a few lines to prj.conf but have gone down something of a rabbit hole.

Starting with CONFIG_MBEDTLS_KEY_EXCHANGE_RSA_ENABLE

This has dependencies:

CONFIG_NRF_SECURITY=y
CONFIG_MBEDTLS_RSA_C=y
CONFIG_MBEDTLS_PKCS1_V15=y
CONFIG_MBEDTLS_LEGACY_CRYPTO_C=y

However, setting these breaks OpenThread, I get the message:
"Current nrf_security configuration does not provide all MBEDTLS options which are required by precompiled OpenThread libraries."

Investigating further I find that setting CONFIG_MBEDTLS_LEGACY_CRYPTO_C=y is the culprit.

Intriguingly, if I specify CONFIG_MBEDTLS_RSA_C=y I get a warning that it "was assigned the value 'y' but got the value 'n'".  This appears to depend on CONFIG_MBEDTLS_LEGACY_CRYPTO_C too.

It looks as though CONFIG_MBEDTLS_KEY_EXCHANGE_ECDHE_RSA_ENABLED=y might be a step forward, I'm working through the dependencies but that also appears to depend on CONFIG_MBEDTLS_LEGACY_CRYPTO_C.

The ciphersuite list is currently

TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
TLS_ECDHE_ECDSA_WITH_AES_256_CCM
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8
TLS_ECDHE_ECDSA_WITH_AES_128_CCM
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8
TLS_EMPTY_RENEGOTIATION_INFO_SCSV

Presumably this is the list that OpenThread has chosen to compile in.  Do you happen to know where that's defined?  I'm not above modifying the OT source if that's what it takes.

[Frog]

So today I decided to go somewhat offroad; I set

CONFIG_MBEDTLS_CFG_FILE="config-tls-generic.h"
CONFIG_MBEDTLS_USER_CONFIG_FILE="my-own-nrf-config.h"

and copied the content of config-thread.h into my-own-nrf-config.h with the intention of taking over control of the crypto settings without losing any of the functionality that OpenThread needs.  With a couple of minor tweaks I got it to compile and run.

Now I have OpenThread working normally, but when I go to open an MQTT session it's using TCP rather than TLS, which is being rejected by the server.  At this point I'm trying to work out how to get TLS going again.

I'll keep working away at that, comparing what I have with some examples unless you have any specific advice.