nRF9160 Dev Kit MQTT simple + TLS : exception fault while deleting keys

RE: nRF9160 Dev Kit MQTT simple + TLS : exception fault while deleting keys

Julien — Fri, 15 Nov 2019 15:45:38 GMT

Hi,

Thank you for your support.

I don't know which CA certificate field I intentionally corrupted.

I made another test that behaved as expected, where I did not use the same CA certificate on both sides. Just after the Server Hello Done step in the TLS connection establishment, the client sends an alert "unknown CA" leading to a failed connection as expected.

Best regards,

Julien

RE: nRF9160 Dev Kit MQTT simple + TLS : exception fault while deleting keys

Didrik Rokhaug — Tue, 12 Nov 2019 08:31:56 GMT

Hi.

I have talked with our modem developers, and here are our findings:

A PEM file is a Base64 encoded ASCII file, and a full file integrity check is not possible. A PEM file consits of many different parts, e.g. version, serial number, issuer, validity, key info, signature, etc. Not all parts of the certificate are used in all cases. Therefore, a modified certificate can be used if the use case does not use the modified part of the certificate.

For example, the signature field of a certificate is used to prove that the certificate is legit. It is signed by someone higher up the trust chain, as proof that it is trusted. However, a root certificate is at the top of the trust chain. Therefore no one can sign the root certificate, but as the PEM format requires the signature, it must be there. What is then done is that the root certificate is signed by itself. This leads to two ways of viewing the signature of a root certificate: Either it is worthless, as anyone can issue a self-signed certificate, or the signature is irrelevant, as you trust the certificate regardless (it is, after all, the "root of trust").

In our case, this is exactly what happens. As we only are corrupting the signature of a root certificate, we are still able to use the certificate because the signature is not used. However, if we corrupt other, used, parts of the certificate, we do not get a connection. We also tested with OpenSSL and could see the same behavior there.

In conclusion, everything works as it is supposed to do, even if it looks a bit strange at first glance.

Hope this helps, if you any more questions, feel free to ask them.

Didrik

RE: nRF9160 Dev Kit MQTT simple + TLS : exception fault while deleting keys

Didrik Rokhaug — Fri, 08 Nov 2019 12:41:05 GMT

Hi.

I just did some testing with modem firmware version 1.0.1 and NCS version 1.1.0-rc2.

1. First I ran asset_tracker with correct CA certificate, public and private key => Everything OK (as expected).

2. Then, I changed the CA certificate to the certificate of mqtt.eclipse.org => Could not connect to nRF Cloud with error 95 (EOPNOTSUPP) (as expected).

3. After that, I changed the peer_verify field in the TLS configuration from 2 (always) to 0 (never) => Could connect with the wrong CA certificate (as expected).

4. I then reverted to the old firmware, but changed a couple of letters at the beginning of the CA certificate => Could not connect to nRF Cloud with error 95 (as expected).

5. Finally, I fixed the changed letters at the beginning and instead changed some letters at the end of the CA certificate => Could connect with the wrong CA certificate (not as expected).

In the end, I got this "corrupted" CA certificate to work, but if I change the "C" at the end to an "a", I am no longer able to connect:

You can find the original here: https://www.amazontrust.com/repository/AmazonRootCA1.pem

I have informed our developers about the unexpected result in case 5.

Have you changed the peer_verify field (on line 359 in main.c from the original post)?
Which part of the CA certificate did you "corrupt"?

Best regards,

Didrik

RE: nRF9160 Dev Kit MQTT simple + TLS : exception fault while deleting keys

Julien — Wed, 06 Nov 2019 15:40:21 GMT

Thank you. I managed to establish the secure connection using the provision function in the code and the CA certificate in the certificates.h file.

Nevertheless, I just made a small check that let me doubt about the setup. I "corrupted" the content of the CA certificate in the certificates.h file, cleaned the solution, rebuilt and reprogram the nRF. The connection nevertheless established and so I wonder if the nRF really validates the server certificate sent by the certificate during the TLS connection establishment.

Is the verification of the server certificate by the client done only of the CA certificate stored in the modem is valid?

Any idea?

Best regards,

Julien

RE: nRF9160 Dev Kit MQTT simple + TLS : exception fault while deleting keys

Didrik Rokhaug — Thu, 17 Oct 2019 09:00:05 GMT

Hi.

Can you check what commit you are on in the nrf repo (and run "west update" afterward)?

I tried running your code (had to make a couple of changes to make it compile, but nothing big). Instead of an exception, it just hanged at nrf_inbuilt_key_delete.

I tried removing and disabling the modem_info module, and then everything worked as expected.

Best regards,

Didrik

RE: nRF9160 Dev Kit MQTT simple + TLS : exception fault while deleting keys

Didrik Rokhaug — Wed, 16 Oct 2019 13:52:12 GMT

[quote user="Julien Mercenier"]You'll see in autoconf.h that the port 8883 is used, not 1883.[/quote]

I see now that the offending line in your prj.conf is commented out. Sorry for the confusion.

When getting new certificates from nRF Cloud, these certificates come in JSON format (client ID, private key, public key, CA certificate). For simplicity, the Certificate Manager can import the certificates directly from this JSON file. However, the certificates themselves are PEM formatted, and you should be able to copy a PEM formatted key/certificate into the corresponding text box.

Again, note that you probably have to increase the buffer size in the at_client sample by setting CONFIG_AT_CMD_RESPONSE_MAX_LEN (3000 should be more than enough).

One tip for reducing the problem with writing certificates from the application is to only do it when you want to update the certificates, so set CONFIG_PROVISION_CERTIFICATES to "n" after successfully writing certificates.

I'll try your code to see if I can reproduce your problem.

RE: nRF9160 Dev Kit MQTT simple + TLS : exception fault while deleting keys

Julien — Wed, 16 Oct 2019 13:15:09 GMT

Hi Didrik,

Thank you very much for your rapid answer.

As I tried several codes, with and without key deletion, I may have copied the wrong log. Sorry for that.

I saw the "Certificate Manager" in the LTE Link Monitor. If I understand it correctly, this manager requires JSON formats for certificates. The CA certificate I own was generated by openssl and has a CRT/PEM format. I put it in a defined string in the certificates.h file. How can I convert it in a format accepted by the "Certificate Manager" so that I can at least try this tool?

I am well aware of the flash wearing problematic. My code is just a proof of concept. I want to master both ways of writing certificates.

You'll see in autoconf.h that the port 8883 is used, not 1883.

Thank you for your advice regarding NB-IoT and TCP. Once I manage to make my application MQTT+TLS work, I'll take a look at UDP based protocol.

Will you nevertheless try my code to help finding the problem?

Best regards,

Julien

RE: nRF9160 Dev Kit MQTT simple + TLS : exception fault while deleting keys

Didrik Rokhaug — Wed, 16 Oct 2019 12:54:36 GMT

Hi.

While I have not tried to run your code yet, I cannot see anything that looks wrong. However, one thing I noticed in your log is that you do not print "Deleting certs sec_tag: %d\n". Is there a reason for these five lines missing from the log?

In general, we do not recommend writing certificates from the application. This has several downsides:

The certificates are present in the flash, they will be present in modem traces, and writing the certificates on every reboot leads to rapid flash wear.

Instead, what we recommend is that you use the "Certificate Manager" in the LTE Link Monitor application in nRF Connect for Desktop, using the at_client (with increased buffers) on the nRF9160.

This way, you only write the certificates when you need to, they are not stored easily accessible in flash, and will not show up in modem traces (unless you take a trace of the certificate writing).

You should also consider using another sec_tag, as the nrf_cloud library uses 16842753. This means that if you delete or write new certificates to that sec_tag, you will not be able to connect to nRF Cloud without getting new certificates (guide here: https://devzone.nordicsemi.com/nordic/cellular-iot-guides/b/getting-started-cellular/posts/nrf-cloud-certificate-update).

Also, if you want to connect to mqtt.eclipse.org with TLS, you must use port 8883, not 1883.

Finally, NB-IoT is not well suited for TCP traffic. Instead of using MQTT, which relies on TCP. You might want to take a look at UDP based protocols with STLS for security.

Best regards,

Didrik