Enabling and testing TLS in mqtt_simple

The MQ Telemetry Transport (MQTT) protocol is hugely popular in IoT because it is lightweight and versatile. In this post we will enable Transport Layer Security (TLS) in the mqtt_simple sample from the nRF Connect SDK (NCS) and then connect it to a MQTT test server that is hosted by the Eclipse Mosquitto project.

Working with TLS credentials

The simplest approach to providing TLS credentials to an nRF91 application is to compile them into the application's hex file. However, there are several reasons why compiling TLS credentials into production firmware is not a good idea:

• The application must copy the credentials to the "modem side" of the SoC so they end up occupying space on both cores.
  • Individual certificates can approach 4KB in length.
  • This also means that the application has to contain extra code to perform the copying.
• Key material that is part of the application doesn't benefit from all of the extra security that is provided by the modem core.
• Compiling credentials into application hex files requires generating unique application hex files for every device.

These disadvantages are fine during development but ideally a production server would require only the current version of the application firmware, credentials to use for the SoC, and an SWD interface for programming. To accomplish this, we will build the mqtt_simple sample without any credentials and then load the credentials as a separate step.

Tools

The following tools will be used:

• Nordic nRF9160 DK
• nRF Connect SDK (NCS)
• OpenSSL command line utility
• Mosquitto command line tools (mosquitto-clients for Linux)
• git and Python3
• Python program called cred for writing credentials to the nRF9160 DK

NOTE: Most Linux distributions come with the openssl utility preinstalled. For Windows users the easiest way to access this utility is via the "Git BASH" tool that is installed with Git for Windows.

NOTE: When installing Mosquitto on Windows the command line utilities work fine even if the "Service" checkbox is unchecked in the installer.

Enabling TLS

We will start by modifying the "ncs/nrf/samples/nrf9160/mqtt_simple" example from NCS to enable TLS. Then we will create credentials that allow the nRF9160 DK to talk to the Mosquitto test server using standard RSA certificates.

Check out the v1.1.0 release in a new branch called "devzone_mqtt_simple":

$ git checkout v1.1.0 -b devzone_mqtt_simple
$ west -update

Although optional, it's nice to be able to define the application's security tag and peer verification requirement in its prj.conf. We can create new CONFIG variables and assign them reasonable defaults by defining them in the project's Kconfig file.

config SEC_TAG
    int "Security tag to use for the connection"
    default 16842753

config PEER_VERIFY
    int "Peer verify parameter for mqtt_client"
    default 1
    help
        Set to 0 for VERIFY_NONE, 1 for VERIFY_OPTIONAL, and 2 for VERIFY_REQUIRED.

Now we can assign a SEC_TAG the same way we can assign to the other useful configs like MQTT_BROKER_HOSTNAME. Switch to prj.conf. Starting at the top, we need to enable TLS functionality in the MQTT lib:

CONFIG_MQTT_LIB_TLS=y

Then we can customize the application-specific CONFIGs, including the security tag -- avoid using 16842753 if nrf_cloud certificates are present:

CONFIG_MQTT_CLIENT_ID="devzone_client"
CONFIG_MQTT_BROKER_HOSTNAME="test.mosquitto.org"
CONFIG_MQTT_BROKER_PORT=8884
    
CONFIG_SEC_TAG=51966
CONFIG_PEER_VERIFY=1

The MQTT_CLIENT_ID is separate from the TLS credentials so it can be set to anything. It should be unique to a particular nRF91 application. In this case we use the Eclipse test server for the MQTT_BROKER_HOSTNAME and change MQTT_BROKER_PORT to 8884, the Mosquitto port for TLS when providing a client certificate.

NOTE: PEER_VERIFY must not be set to VERIFY_REQUIRED (2) because test.mosquitto.org does not verify correctly, probably because the certificates it uses are not strong enough (SHA1 and 1024-bit RSA).

At this point the sample should compile but won't use TLS because we haven't enabled it in the mqtt_client in main.c. First, we grab the security tag from prj.conf and write it to a variable at the top of main.c that can then be given to the mqtt_client:

#if defined(CONFIG_MQTT_LIB_TLS)
static sec_tag_t sec_tag_list[] = { CONFIG_SEC_TAG };
#endif /* defined(CONFIG_MQTT_LIB_TLS) */ 

Using CONFIG_MQTT_LIB_TLS as a preproccesor guard will allow the finished project to be compiled with or without TLS support. Next, we drop into the client_init function and initialize the TLS parameters:

#if defined(CONFIG_MQTT_LIB_TLS)
    struct mqtt_sec_config *tls_config = &client->transport.tls.config;
    
    client->transport.type = MQTT_TRANSPORT_SECURE;
    
    tls_config->peer_verify = CONFIG_PEER_VERIFY;
    tls_config->cipher_count = 0;
    tls_config->cipher_list = NULL;
    tls_config->sec_tag_count = ARRAY_SIZE(sec_tag_list);
    tls_config->sec_tag_list = sec_tag_list;
    tls_config->hostname = CONFIG_MQTT_BROKER_HOSTNAME;
#else /* MQTT transport configuration */
    client->transport.type = MQTT_TRANSPORT_NON_SECURE;
#endif /* defined(CONFIG_MQTT_LIB_TLS) */

Here we're telling the mqtt_client to connect to a specific server using an encrypted socket. The security tag allows the modem's TLS library to find and load the credentials that have been (or will be) added separately. Note that the client->password and client->user_name are not used by the socket for TLS purposes -- they are optionally used for authenticating users with the MQTT server regardless of whether or not the connection has been authenticated via TLS.

That's it for main.c so the project can be compiled:

$ west build -b nrf9160_pca10090ns

A patch is attached for reference in case any of these edits are ambiguous. The resulting hex file is ready to go and is located at "/build/zephyr/merged.hex". Without credentials the printk output from this application will look something like this:

LTE Link Connected!
IPv4 Address found XX.XX.XXX.X
ERROR: mqtt_connect -45

The error code is EOPNOTSUP and it is returned because the socket failed to enable TLS.

Generating an RSA client certificate

First, a private key is required. A 2048-bit RSA key can be created using the openssl utility:

openssl genrsa -out client.key 2048

NOTE: This key is not encrypted using a passphrase.

One of the features of using certificates is that they can uniquely identify the device to which they were issued via the 'Common Name' field. The modem's IMEI is useful for supporting this feature.

The cred program will be used for managing the credentials on the nRF9160 DK (that is connected to a PC via the onboard J-Link debug probe). The repository can be cloned:

$ [email protected]:inductivekickback/cred.git
$ cd cred

The "Requirements" section of the README.md explains how to install the required Python modules.

The cred program works by writing a simple, prebuilt hex file to the nRF9160 DK, allowing that hex file to run, and then reading the result. The whole thing runs over the SWD interface so no serial ports are required. Because the flash memory of the nRF9160 DK is modified as part of the process the cred program can be instructed to program an application when it finishes.

The IMEI of the modem can be printed to the console:

$ python3 cred.py --imei_only
123456789012345

The cred program always prints the modem's IMEI to stdout when it finishes. We can use it to create a unique identity to use for the client certificate's 'Common Name', e.g. "nrf-123456789012345". The 'Common Name' will be requested in the next step when we create a Certificate Signing Request (CSR). The individual fields don't matter from a TLS perspective so just wing it. Avoid using the default "Country", "Organizational Name", and "Common Name", however, because doing so might cause the Certificate Authority (CA) to reject the CSR:

openssl req -out client.csr -key client.key -new

The newly-created client.csr must then be signed by the Mosquitto test server's CA by pasting the contents of client.csr into the text box on this page. This will cause a client.crt file to download.

The Mosquitto test server's root CA certificate must also be downloaded. The best way to accomplish this is by right-clicking the "mosquitto.org.crt (PEM format)" link on this page and selecting "Save Link As..." with the name CA.crt.

Writing the certificates

Now it's time to write the credentials to the nRF9160 DK. Make sure that the DK is plugged in and turned on. Copy the CA.crt, client.crt, and client.key to the cred repository directory along with the "merged.hex" from the "mqtt_simple/build/zephyr" directory.

$ python3 cred.py --CA_cert CA.crt --client_cert client.crt --client_private_key client.key --sec_tag 51966 --program_app merged.hex 123456789012345

If everything worked correctly then the printk output should look something like this:

LTE Link Connected!
IPv4 Address found XX.XX.XXX.X
[mqtt_evt_handler:194] MQTT client connected!
Subscribing to: my/subscribe/topic len 18
[mqtt_evt_handler:244] SUBACK packet id: 1234

Now the nRF9160 DK will echo any messages that it receives on the "my/subscribe/topic" topic. This can be tested using the Mosquitto command line utilities:

mosquitto_pub -h test.mosquitto.org -t "my/subscribe/topic" -m "Devzone test!"

The message should then be visible in the printk output:

Received: Devzone test!
Publishing: Devzone test!
to topic: my/publish/topic len: 16
[mqtt_evt_handler:234] PUBACK packet id: 32133