MQTT over Thread

RE: MQTT over Thread

Charlie — Wed, 17 Jul 2024 13:34:05 GMT

Hi Frog,

Yes, you are correct. By default, OpenThread in the nRF Connect SDK uses the nRF Security module for cryptographic operations. This module provides hardware-accelerated cryptographic functionality on selected Nordic Semiconductor SoCs as well as alternate software-based implementations of the Mbed TLS APIs.

If you choose to build OT libraries from source, you can define additional configuration options one by one. By default, the Feature sets option is set to custom, which allows you to create your own OpenThread stack configuration. However, you can select other feature sets as a basis. Rebuilding OT libraries from source may need more certification steps for the final products.

The unsupported cipher suite may have some reason to be unsupported. It could consume more memory or computing power, which may not fit for MCU usage.

It is common that server support a list of cipher suite which more than client side. The negotiation between client and server decide one that fit both of them. Do you have control on the server side? Maybe it is easier to support cipher suite from MQTT broker server side.

Best regards,

Charlie

RE: MQTT over Thread

Frog — Wed, 17 Jul 2024 04:41:35 GMT

It's not possible to set NRF_SECURITY_LEGACY_AND_PSA directly, so I set

CONFIG_NRF_CC3XX_PLATFORM=y
CONFIG_TRUSTED_STORAGE=y
CONFIG_BUILD_WITH_TFM=n

which conflicted with many of the crypto library options I'd already requested, e.g.

#CONFIG_MBEDTLS_AES_C=y

so I commented out all the offending definitions. This left me with:

warning: HW_UNIQUE_KEY (defined at C:/nordic/v2.6.1/nrf\lib\hw_unique_key/Kconfig:35) has direct dependencies HW_UNIQUE_KEY_SUPPORTED && (NRF_CC3XX_PLATFORM || BUILD_WITH_TFM) && NRF_SECURITY && (MPU_ALLOW_FLASH_WRITE || BUILD_WITH_TFM) && (!BUILD_WITH_TFM || TFM_CRYPTO_BUILTIN_KEYS) with value n, but is currently being y-selected by the following symbols:
- TRUSTED_STORAGE_BACKEND_AEAD_KEY_DERIVE_FROM_HUK (defined at C:/nordic/v2.6.1/nrf\subsys\trusted_storage/Kconfig:132), with value y, direct dependencies HW_UNIQUE_KEY_SUPPORTED && <choice TRUSTED_STORAGE_BACKEND_AEAD_KEY> (value: y), and select condition HW_UNIQUE_KEY_SUPPORTED && <choice TRUSTED_STORAGE_BACKEND_AEAD_KEY> (value: y)

warning: SOC_NRF_GPIO_FORWARDER_FOR_NRF5340 (defined at soc/arm/nordic_nrf\nrf53\Kconfig.soc:147) has direct dependencies NRF_SOC_SECURE_SUPPORTED && SOC_NRF5340_CPUAPP && SOC_SERIES_NRF53X && SOC_FAMILY_NRF with value n, but is currently being y-selected by the following symbols:
- BOARD_ENABLE_CPUNET (defined at C:/nordic/v2.6.1/zephyr/boards/arm/nrf5340dk_nrf5340/Kconfig:23), with value y, direct dependencies BOARD_NRF5340DK_NRF5340_CPUAPP || BOARD_NRF5340DK_NRF5340_CPUAPP_NS (value: y), and select condition BOARD_NRF5340DK_NRF5340_CPUAPP || BOARD_NRF5340DK_NRF5340_CPUAPP_NS (value: y)

Given that this is all internal to the SDK I don't know if there's anything I can do to work around it.

I'm worried about getting too far into the weeds without having a really good understanding of what OpenThread needs. If I understand correctly there's one crypto library that's used system-wide, and this needs to provide the superset of what OT needs and what my TLS connection needs. Is that correct? If so it would make sense to start with the default OT configuration and then add the ciphersuite that I need, although I suppose that may mean I have to build the library from source rather than linking a precompiled library.

RE: MQTT over Thread

Charlie — Tue, 16 Jul 2024 14:03:34 GMT

Hi Frog,

Could you have a quick try to enable configure NRF_SECURITY_LEGACY_AND_PSA?

config NRF_SECURITY_LEGACY_AND_PSA

bool

default y

select EXPERIMENTAL

depends on MBEDTLS_LEGACY_CRYPTO_C && MBEDTLS_PSA_CRYPTO_C

# This configuration doesn't affect TF-M builds since the PSA

# APIs are provided by TF-M.

# When this configuration is enabled we manually enable

# some symbols in the build_config.h file in the Oberon PSA core.

# This requires only the Oberon PSA crypto driver to be enabled,

# it requires the CC3XX platform library to get random data and

# the trusted storage for ITS support. The depenedencies here

# match what we enable in the build_config.h file so if we need to

# modify the dependencies here we also need to modify the build_config.h.

depends on PSA_CRYPTO_DRIVER_OBERON && !PSA_CRYPTO_DRIVER_CC3XX

depends on NRF_CC3XX_PLATFORM

depends on TRUSTED_STORAGE

depends on !BUILD_WITH_TFM

help

This is an option to support legacy mbedTLS and PSA crypto APIs

at the same time. This is not recommended as it is not fully

supported in our system. This feature might get changed/removed at

any time in the future. You are advised to use the PSA APIs

for any new developments.

This option doesn't use the nrf_security for the internal

PSA configuration. It always use the Oberon PSA driver

for all the crypto operations expect for the PRNG which

uses the nrf_cc3xx_platform library.

Best regards,

Charlie

RE: MQTT over Thread

Frog — Fri, 12 Jul 2024 04:57:58 GMT

Hi Charlie,

Since last time I have moved from a Thingy91 to a Fanstel BT40NE module which is based on an nRF5340 and has a 21540 FEM, which I've got working with some SDK adjustments as recommended by Fanstel. I'm more or less back where I was with the MQTT connection - the device sends a hello request to the server but with the default list of ciphersuites.

I've copied the config from here
RE: Error -22 in mqtt_connect() - nRF52840dk with Azure IoT Hub using OpenThread and TCP
and resolved a few issues, now I'm down to three problems that appear to be closely related: MBEDTLS_CIPHER_AES_ENABLED, MBEDTLS_CIPHER_CCM_ENABLED and MBEDTLS_MAC_CMAC_ENABLED. The error messages such as

warning: MBEDTLS_CIPHER_AES_ENABLED (defined at C:/nordic/v2.6.1/zephyr/modules/mbedtls\Kconfig.tls-generic:257, modules\mbedtls\Kconfig.tls-generic:257) has direct dependencies (!(NRF_SECURITY || NORDIC_SECURITY_BACKEND) && MBEDTLS_BUILTIN && MBEDTLS_CFG_FILE = "config-tls-generic.h" && MBEDTLS) || (!(NRF_SECURITY || NORDIC_SECURITY_BACKEND) && MBEDTLS_BUILTIN && MBEDTLS_CFG_FILE = "config-tls-generic.h" && MBEDTLS && 0) with value n, but is currently being y-selected by the following symbols:
- OPENTHREAD_MBEDTLS (defined at subsys/net/l2/openthread/Kconfig:179), with value y, direct dependencies NET_L2_OPENTHREAD && NETWORKING (value: y), and select condition NET_L2_OPENTHREAD && NETWORKING (value: y)

tell me that OpenThread requires these set to 'y' but they are forced to 'n' because CONFIG_MBEDTLS_BUILTIN=n, in turn because I don't have MBEDTLS_IMPLEMENTATION set and I'm not sure how to do that.

I'm currently using SDK v2.6.1 and getting ready to move to v2.7.0.

Are you able to give any guidance please?

RE: MQTT over Thread

Charlie — Fri, 31 May 2024 09:37:19 GMT

Hi Frog,

Welcome back!

[quote user="Frog"]I'm back to the problem of wanting to specify one additional RSA-based cipersuite in addition to those that OT needs. As far as I can see (please tell me if I'm wrong) I should be able to use the precompiled openThread library provided that I also provide a PSA suite that at least satisfies OT.[/quote]

You are free to do this on your application codes. The certification should be OK according to Certification by inheritance without modifications to binaries, but if you use deprecated Mbed TLS support by setting the CONFIG_OPENTHREAD_NRF_SECURITY_CHOICE Kconfig option to y, but you must build the Thread libraries from source, the Certification by inheritance with modifications to binaries need to be followed.

[quote user="Frog"]Since I need an RSA-based cipersuite, is it essential to enable the legacy APIs (which the dependency tree seems to suggest) or is there another way to do this?[/quote]

As far as I know, using legacy APIs is the shortest way to achieve your target.

However, it's important to note that combining legacy and PSA crypto APIs in the same application might work, but it's not a recommended or maintained solution.

Best regards,

Charlie

RE: MQTT over Thread

Frog — Fri, 31 May 2024 01:45:03 GMT

Thanks for your patience, I've been dealing with some other matters for a few days.

Looking at the CLI example some more, the default configuration opens a TCP connection and then attempts TLS negotiation with the default ciphersuites (using SDK2.5.1). However, when I set CONFIG_MBEDTLS_LEGACY_CRYPTO_C=y it seems to break - the initial TCP connection packet should just have the SYN flag set but with LEGACY_C it has SYN, ECN and CWR according to Wireshark, and there's no response from the server.

I know from this post PSA crypto features not enabled when CONFIG_MBEDTLS_LEGACY_CRYPTO_C is enabled that specifying LEGACY_C causes some PSA functionality to be lost. Given that the legacy APIs are deprecated it makes sense that I would aim not to use them but should use the PSA APIs, and logically I'd use SDK 2.6.0 where OpenThread uses PSA too.

So I'm back to the problem of wanting to specify one additional RSA-based cipersuite in addition to those that OT needs. As far as I can see (please tell me if I'm wrong) I should be able to use the precompiled openThread library provided that I also provide a PSA suite that at least satisfies OT.

Since I need an RSA-based cipersuite, is it essential to enable the legacy APIs (which the dependency tree seems to suggest) or is there another way to do this?

RE: MQTT over Thread

Frog — Wed, 22 May 2024 04:41:16 GMT

I looked at the CLI sample again and found that I could request CONFIG_MBEDTLS_LEGACY_CRYPTO_C without breaking OpenThread. I noticed that the CLI example was built with SDK version 2.5.1 rather than 2.6.0 that I'm using for the project I'm working on. Dropping back to 2.5.1 in my own project appears to resolve the problem of breaking OpenThread, which is a step forward, but for some reason the TLS negotiation is no longer working - MQTT opens the TCP port but no 'client hello' packet is sent. I'll continue to investigate that.

RE: MQTT over Thread

Charlie — Tue, 21 May 2024 12:23:08 GMT

CONFIG_MBEDTLS_KEY_EXCHANGE_ECDHE_RSA_ENABLED dependencies are listed here: Kconfig search — Kconfig reference (nordicsemi.com)

According to the NCS v2.6.0 release notes:

"The default cryptography backend for Thread is now Arm PSA Crypto API instead of Mbed TLS, which was used in earlier versions. You can still build all examples with deprecated Mbed TLS support by setting the CONFIG_OPENTHREAD_NRF_SECURITY_CHOICE Kconfig option to y, but you must build the Thread libraries from sources. To inherit Thread certification from Nordic Semiconductor, you must use the PSA Crypto API backend."

The tcat command in CLI sample actually enables support for Thread commissioning over authenticated TLS.

RE: MQTT over Thread

Frog — Tue, 21 May 2024 02:14:08 GMT

Currently I'm able to request a subset of the default ciphersuites, which is a step in the right direction. The suite that I want to use isn't in that list though. Looking in ssl_cipersuites.c I see that I need (at least) CONFIG_MBEDTLS_KEY_EXCHANGE_ECDHE_RSA_ENABLED

although this depends on CONFIG_MBEDTLS_RSA_C and CONFIG_MBEDTLS_PKCS1_V15, and ultimately on CONFIG_MBEDTLS_LEGACY_CRYPTO_C. The last of these breaks OpenThread.

I see that this is a reversal of the situation described here
PSA crypto features not enabled when CONFIG_MBEDTLS_LEGACY_CRYPTO_C is enabled
where CONFIG_MBEDTLS_LEGACY_CRYPTO_C used to be required by OT.

It's my intention to use PSA throughout, so I have CONFIG_OPENTHREAD_CRYPTO_PSA and CONFIG_MBEDTLS_PSA_CRYPTO_C and a few CONFIG_PSA_WANT... so I'm not clear on why I would need LEGACY_CRYPTO_C - are you able to shed any light on this?

RE: MQTT over Thread

Charlie — Thu, 16 May 2024 14:21:49 GMT

Hi Frog,

If you search MQTT_TRANSPORT_SECURE in NCS source codes, you will find some samples about to properly set up TLS connection in MQTT.

Hope this will give you some hint.

Best regards,

Charlie

RE: MQTT over Thread

Frog — Thu, 16 May 2024 05:07:39 GMT

So today I decided to go somewhat offroad; I set

CONFIG_MBEDTLS_CFG_FILE="config-tls-generic.h"
CONFIG_MBEDTLS_USER_CONFIG_FILE="my-own-nrf-config.h"

and copied the content of config-thread.h into my-own-nrf-config.h with the intention of taking over control of the crypto settings without losing any of the functionality that OpenThread needs. With a couple of minor tweaks I got it to compile and run.

Now I have OpenThread working normally, but when I go to open an MQTT session it's using TCP rather than TLS, which is being rejected by the server. At this point I'm trying to work out how to get TLS going again.

I'll keep working away at that, comparing what I have with some examples unless you have any specific advice.

RE: MQTT over Thread

Frog — Wed, 15 May 2024 04:56:39 GMT

Sadly I don't have control over the server; there's a list of about a dozen supported ciphersuites. I've chosen to go with ECDHE-RSA-AES256-SHA384. I'd hoped that this would be as simple as adding a few lines to prj.conf but have gone down something of a rabbit hole.

Starting with CONFIG_MBEDTLS_KEY_EXCHANGE_RSA_ENABLE

This has dependencies:

CONFIG_NRF_SECURITY=y
CONFIG_MBEDTLS_RSA_C=y
CONFIG_MBEDTLS_PKCS1_V15=y
CONFIG_MBEDTLS_LEGACY_CRYPTO_C=y

However, setting these breaks OpenThread, I get the message:
"Current nrf_security configuration does not provide all MBEDTLS options which are required by precompiled OpenThread libraries."

Investigating further I find that setting CONFIG_MBEDTLS_LEGACY_CRYPTO_C=y is the culprit.

Intriguingly, if I specify CONFIG_MBEDTLS_RSA_C=y I get a warning that it "was assigned the value 'y' but got the value 'n'". This appears to depend on CONFIG_MBEDTLS_LEGACY_CRYPTO_C too.

It looks as though CONFIG_MBEDTLS_KEY_EXCHANGE_ECDHE_RSA_ENABLED=y might be a step forward, I'm working through the dependencies but that also appears to depend on CONFIG_MBEDTLS_LEGACY_CRYPTO_C.

The ciphersuite list is currently

TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
TLS_ECDHE_ECDSA_WITH_AES_256_CCM
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8
TLS_ECDHE_ECDSA_WITH_AES_128_CCM
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8
TLS_EMPTY_RENEGOTIATION_INFO_SCSV

Presumably this is the list that OpenThread has chosen to compile in. Do you happen to know where that's defined? I'm not above modifying the OT source if that's what it takes.

RE: MQTT over Thread

Charlie — Tue, 14 May 2024 12:04:37 GMT

Hi Frog,

Thanks for the update, good to hear that you have explored

For the MQTT server side, you should be able to configure the cipher suites from server side, this is important because the device wouldn't support all cipher suites and TLS version due to resource limitation and mbed TLS library implemenation.

You can find the following page related to AWS IoT MQTT server TLS configuration.

https://docs.aws.amazon.com/iot/latest/developerguide/transport-security.html#tls-policy-table

https://docs.aws.amazon.com/iot/latest/developerguide/iot-endpoints-tls-config.html#custom-tls-console

For the device side, you need TLS library to load and use the certifications when you build TLS connection with server. Zephyr has samples about how to use mbed TLS library. You can refer to the following discussion about its implementation with OpenThread.

(+) Adaption to TCP+TLS on top of OpenThread. - Nordic Q&A - Nordic DevZone - Nordic DevZone (nordicsemi.com)

Best regards,

Charlie

RE: MQTT over Thread

Frog — Tue, 14 May 2024 05:20:08 GMT

Thanks Charlie,

fgervais' project has got me some way down the road; I'm able to connect to a border router, perform a DNS lookup and start to open a connection to my MQTT server. However, the server requires some specific ciphersuites so the TLS negotiation is failing.

I seem to be having difficulty in getting the ciphersuites set up correctly without breaking the encryption for openThread - at the time of writing my code is failing to set the OT network key because it's stored in an encrypted form within openThread and the encryption is broken.

What I'm not clear on is how the encryption libraries used by openThread and MQTT/TLS/SSL should be configured to work together - is there a recommended approach to this?

Here's my prj.conf as it currently is, if there are any obvious shortcomings please let me know, thanks.

# OpenThread
CONFIG_OPENTHREAD_NORDIC_LIBRARY_MASTER=y
CONFIG_NET_L2_OPENTHREAD=y
CONFIG_OPENTHREAD_SOURCES=y
CONFIG_OPENTHREAD_NORDIC_LIBRARY=n
CONFIG_OPENTHREAD_CSL_RECEIVER=y
CONFIG_OPENTHREAD_SLAAC=y
CONFIG_OPENTHREAD_CHANNEL=11
CONFIG_OPENTHREAD_TCP_ENABLE=y
CONFIG_OPENTHREAD_MBEDTLS_DEBUG=y
CONFIG_OPENTHREAD_TIME_SYNC=y

# Networking options
CONFIG_NETWORKING=y
CONFIG_NET_CONFIG_SETTINGS=y
CONFIG_NET_TCP=y
CONFIG_NET_UDP=y
CONFIG_NET_SOCKETS=y
CONFIG_NET_IPV6=y
CONFIG_NET_IPV4=n
CONFIG_NET_CONFIG_NEED_IPV6=y
CONFIG_NET_CONFIG_NEED_IPV4=n

CONFIG_NET_LOG=y
CONFIG_NET_IF_LOG_LEVEL_DBG=y
CONFIG_NET_IPV6_LOG_LEVEL_DBG=y
CONFIG_NET_CONFIG_LOG_LEVEL_DBG=y
CONFIG_NET_CONN_LOG_LEVEL_DBG=y
CONFIG_NET_SOCKETS_LOG_LEVEL_DBG=y
#CONFIG_NET_CORE_LOG_LEVEL_DBG=y
CONFIG_NET_SOCKETS_SOCKOPT_TLS=y
#CONFIG_NET_TCP_LOG_LEVEL_DBG=y

CONFIG_NET_BUF_TX_COUNT=64
CONFIG_NET_TX_STACK_SIZE=4096
CONFIG_NET_RX_STACK_SIZE=4096
CONFIG_NET_SHELL=y
CONFIG_NET_SOCKETS_POSIX_NAMES=y
CONFIG_NET_CONNECTION_MANAGER=n
CONFIG_NET_CONTEXT_NET_PKT_POOL=y

# MQTT
CONFIG_MQTT_LIB=y
CONFIG_MQTT_LIB_TLS=y
CONFIG_MQTT_CLEAN_SESSION=y
CONFIG_MQTT_LOG_LEVEL_DBG=y
CONFIG_MQTT_KEEPALIVE=600

# Crypto
CONFIG_PSA_CRYPTO_DRIVER_CC3XX=y
CONFIG_MBEDTLS=y
CONFIG_MBEDTLS_SHA1_C=y
CONFIG_MBEDTLS_LEGACY_CRYPTO_C=y
CONFIG_MBEDTLS_X509_CRT_PARSE_C=y
CONFIG_NORDIC_SECURITY_BACKEND=n
CONFIG_ENTROPY_GENERATOR=y
CONFIG_MBEDTLS_ENABLE_HEAP=yCONFIG_MBEDTLS_HEAP_SIZE=32768
CONFIG_MBEDTLS_TLS_LIBRARY=y

RE: MQTT over Thread

Charlie — Wed, 24 Apr 2024 12:50:26 GMT

Hi Frog,

This has been discussed in many places before. Just share the latest discussion I had with another developer on the MQTT over Thread topic: https://devzone.nordicsemi.com/f/nordic-q-a/105692/nrf52840-mqtt-coap/455746.

This is neither suggested nor easy to make directly since some configurations need to explore. You can refer to the following implementations to review your codes. If the problem still exists, we can continue the discussion here.

RE: Error -22 in mqtt_connect() - nRF52840dk with Azure IoT Hub using OpenThread and TCP

fgervais/project-nrf-thread-switch: Home Assistant (MQTT) wireless button with 10 years of battery life. (github.com)

Best regards,

Charlie