SSL: Certificate verify failure

[Thom van Zeijl]

Hello,

I'm using the nRF9160DK (PCA10090 0.8.2). I was able to see the data coming from the nRF9160DK in the nrfcloud, but now I want to be able to see the data published by the nRF9160DK in another place (for example in a terminal). Using the SDK located in: https://github.com/aws/aws-iot-device-sdk-python I was able to create a small script in python:

# Import SDK packages
from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTClient

# For certificate based connection
myMQTTClient = AWSIoTMQTTClient("nrf-xxxxxxxxxxxxxxx")
#myMQTTClient = AWSIoTMQTTClient("arn:aws:iot:us-east-1:xxxxxxxxxxxx:cert/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")
# For Websocket connection
# myMQTTClient = AWSIoTMQTTClient("myClientID", useWebsocket=True)
# Configurations
# For TLS mutual authentication
myMQTTClient.configureEndpoint("xxxxxxxxxxxwix-ats.iot.us-east-1.amazonaws.com", 8883)
# For Websocket
# myMQTTClient.configureEndpoint("YOUR.ENDPOINT", 443)
# For TLS mutual authentication with TLS ALPN extension
# myMQTTClient.configureEndpoint("YOUR.ENDPOINT", 443)
myMQTTClient.configureCredentials("ca.txt", "privatekey.txt", "certificate.txt")
# For Websocket, we only need to configure the root CA
# myMQTTClient.configureCredentials("YOUR/ROOT/CA/PATH")
myMQTTClient.configureOfflinePublishQueueing(-1)  # Infinite offline Publish queueing
myMQTTClient.configureDrainingFrequency(2)  # Draining: 2 Hz
myMQTTClient.configureConnectDisconnectTimeout(10)  # 10 sec
myMQTTClient.configureMQTTOperationTimeout(5)  # 5 sec
print("Step1")
myMQTTClient.connect()
print("Connect")
myMQTTClient.subscribe("$SYS/#", 1, customCallback)

When I run this script in a terminal using "python ./client3.py" (name of the python file is client3.py) I get the following error message:

Step1
Traceback (most recent call last):
  File "./client3.py", line 24, in <module>
    myMQTTClient.connect()
  File "/home/thom/Documents/nRF9160devkit/eclipse/aws-iot-device-sdk-python/AWSIoTPythonSDK/MQTTLib.py", line 485, in connect
    return self._mqtt_core.connect(keepAliveIntervalSecond)
  File "/home/thom/Documents/nRF9160devkit/eclipse/aws-iot-device-sdk-python/AWSIoTPythonSDK/core/protocol/mqtt_core.py", line 192, in connect
    self.connect_async(keep_alive_sec, self._create_blocking_ack_callback(event))
  File "/home/thom/Documents/nRF9160devkit/eclipse/aws-iot-device-sdk-python/AWSIoTPythonSDK/core/protocol/mqtt_core.py", line 219, in connect_async
    raise e
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)

So there is a certificate verification failure and I do not know how to solve this. The operating system is Ubuntu 18.04.1 LTS running on a virtual machine (using Oracle Virtualbox) with native os Windows 10.

The client ID, endpoint, certificate and privatekey all are retrieved using the commands from https://docs.api.nrfcloud.com/MQTT/ 

The certificate and privatekey are provided in the relative .txt files.

The certificate.txt file looks like:

-----BEGIN CERTIFICATE-----
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-----END CERTIFICATE-----

The privatekey.txt looks exactly like the certificate.txt file except for the fact that it is 27 lines instead of 20 lines and the first and last line is different (-----BEGIN RSA PRIVATE KEY----- and -----END RSA PRIVATE KEY-----). The content of the ca.txt file is supplied by the link https://github.com/aws/aws-iot-device-sdk-python#id3 under "Credentials/X.509 certificate", and then the link "AWS IoT root CA". 

Is possible to use this script to subscribe to the topics on which the nRF9160DK publishes (simulated location and simulated orientation) or is this impossible to do or am I overlooking something?

[Markus Tacker (he/him)]

The root certificate mentioned in https://github.com/aws/aws-iot-device-sdk-python#id3 is deprecated and should not be used.

Go to https://docs.aws.amazon.com/iot/latest/developerguide/managing-device-certs.html and download the key for the Amazon Trust Services Endpoints.

This one should work for you: https://www.amazontrust.com/repository/AmazonRootCA1.pem 

[Thom van Zeijl]

Thanks for your response. Using the new root certificate I no longer get a verification failure. However, a new error occurs. See:

Step1
Connect
No handlers could be found for logger "AWSIoTPythonSDK.core.protocol.mqtt_core"
Traceback (most recent call last):
  File "./client3.py", line 31, in <module>
	myMQTTClient.subscribe("$SYS/#", 1, customCallback)
  File "/home/thom/Documents/nRF9160devkit/eclipse/aws-iot-device-sdk-python/AWSIoTPythonSDK/MQTTLib.py", line 668, in subscribe
	return self._mqtt_core.subscribe(topic, QoS, callback)
  File "/home/thom/Documents/nRF9160devkit/eclipse/aws-iot-device-sdk-python/AWSIoTPythonSDK/core/protocol/mqtt_core.py", line 303, in subscribe
	raise subscribeTimeoutException()
AWSIoTPythonSDK.exception.AWSIoTExceptions.subscribeTimeoutException
Exception in thread Thread-1 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
  File "/usr/lib/python2.7/threading.py", line 754, in run
  File "/home/thom/Documents/nRF9160devkit/eclipse/aws-iot-device-sdk-python/AWSIoTPythonSDK/core/protocol/internal/workers.py", line 147, in _dispatch
  File "/usr/lib/python2.7/threading.py", line 289, in __exit__
  File "/usr/lib/python2.7/threading.py", line 216, in __exit__
  File "/usr/lib/python2.7/threading.py", line 203, in release
<type 'exceptions.TypeError'>: 'NoneType' object is not callable

The "No handlers could be found for logger "AWSIoTPythonSDK.core.protocol.mqtt_core"" error is not important. The error below it is. "subscribeTimeoutException()" is a problem. Somehow the script is connected to the aws server but it can not subscribe to any topic. The topic the script should subscribe to is "$SYS/#" which is as far as I know the topic where basic information about the broker is published. The same error also occurs when the topic is different (for example the "alerts" and "messagePrefix" topics retrieved from the commands at  https://docs.api.nrfcloud.com/MQTT/ ). 

The code for the script is slightly extended from the one I posted originally. The extension is as follows:

def customCallback(client, userdata, message):
	print("Received a new message: ")
	print(message.payload)
	print("from topic: ")
	print(message.topic)
	print("--------------\n\n")

I've also tested the script located at https://github.com/aws/aws-iot-device-sdk-python/blob/master/samples/basicPubSub/basicPubSub.py This script gives basically the same error:

Traceback (most recent call last):
  File "./basicpubsub.py", line 110, in <module>
	myAWSIoTMQTTClient.subscribe(topic, 0, customCallback)
  File "/home/thom/Documents/nRF9160devkit/eclipse/aws-iot-device-sdk-python/AWSIoTPythonSDK/MQTTLib.py", line 668, in subscribe
	return self._mqtt_core.subscribe(topic, QoS, callback)
  File "/home/thom/Documents/nRF9160devkit/eclipse/aws-iot-device-sdk-python/AWSIoTPythonSDK/core/protocol/mqtt_core.py", line 303, in subscribe
	raise subscribeTimeoutException()
AWSIoTPythonSDK.exception.AWSIoTExceptions.subscribeTimeoutException
Exception in thread Thread-1 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
  File "/usr/lib/python2.7/threading.py", line 754, in run
  File "/home/thom/Documents/nRF9160devkit/eclipse/aws-iot-device-sdk-python/AWSIoTPythonSDK/core/protocol/internal/workers.py", line 147, in _dispatch
  File "/usr/lib/python2.7/threading.py", line 289, in __exit__
  File "/usr/lib/python2.7/threading.py", line 216, in __exit__
  File "/usr/lib/python2.7/threading.py", line 203, in release
<type 'exceptions.TypeError'>: 'NoneType' object is not callable

Does anyone know how to solve this?

[Markus Tacker (he/him)]

1. on the nrfcloud.com MQTT broker you are only authorized to subscribe to topics below your mqttTopicPrefix, e.g. "prod/762bfc32-d338-4d5a-bd03-1e504e013846/" (762bfc32-d338-4d5a-bd03-1e504e013846 is a v4 UUID which will be different for you). So make sure that you concatenate your personal mqttTopicPrefix with any topic you want to subscribe to. You can request your personal mqttTopicPrefix using the https://api.nrfcloud.com/v1/account endpoint as explained here: https://docs.api.nrfcloud.com/MQTT 
2. if you subscribe to an MQTT topic which you are not authorized to, the broker will disconnect you. Since you are seeing a timeout on your end, I suspect something else is going on.

[Thom van Zeijl]

Thanks for the reply.

I've tried subscribing to:

prod/MY_UUID/

prod/MY_UUID/a

prod/MY_UUID/a/

prod/MY_UUID/a/alerts

prod/MY_UUID/a/alerts/

prod/MY_UUID/a/alerts/notifications

But none of them worked. I always got the timeout error. So as you have indicated it looks like something else is going on but I do not know what. Any help would be appreciated!

[Markus Tacker (he/him)]

A timeout typically means that there is something not working with the physical connection. Try the Link Monitor to verify that you are connected: devzone.nordicsemi.com/.../getting-started-with-nrf9160-dk

[Thom van Zeijl]

I used the Link Monitor (on native Windows 10 instead of the ubuntu VM) and I followed the steps in the link you mentioned. When I send an AT command to the device, it returns: Error: 'AT+CFUN? ' timed out. I am only able to send AT commands to the device when the AT client sample app is installed. In the top right, only the UART box is green, the Modem, UICC and LTE boxes are red and the PDN box is green. I changed the periodic signal quality request to 1sec. In the chart tab, it does not give any data. This indicates that there is no connection. But when I go to nrfcloud, I can see the device updating and it also returns the simulated orientation of the nrf9160dk. So there is a connection, otherwise the nrfcloud would not return any data. When the nrf9160dk is connected and gives information in nrfcloud, led 4 is turned on.

I am using a KPN simcard (of type K019) instead of the iBasis sim card that came with the dev kit. The iBasis sim card does not work here in the Netherlands. I don't think this should matter.