NRF9160 Azure IoT hub DPS Certification Connection Rejected

[jczacharia]

Hi Nordic

Sample: asset_tracker_v2 (with only changes to overlay-azure.conf)
SDK: 2.5.2
Modem FW: nrf9160_1.3.5

I have followed this tutorial https://learn.microsoft.com/en-us/azure/iot-dps/tutorial-custom-hsm-enrollment-group-x509?tabs=windows&pivots=programming-language-ansi-c for getting my device connected to IOT Hub via DPS. Which I have followed and re-tried many times.

On Azure, in my DPS Certificates, I've uploaded azure-iot-test-only.root.ca.cert.pem, per tutorial.
And in my Managed Enrollments, I've added an Enrollment Group with the azure-iot-test-only.intermediate.cert.pem, per tutorial.

But, I am struggling with the certification and getting a valid connection.

The error that keeps persisting is:

[00:00:15.799,896] <err> azure_iot_hub_dps: Connection was rejected with return code 5
[00:00:15.799,926] <wrn> azure_iot_hub_dps: Is the device certificate valid?
[00:00:15.799,987] <err> mqtt_helper: Library is in the wrong state (MQTT_STATE_DISCONNECTED), MQTT_STATE_CONNECTED required
[00:00:15.800,018] <err> azure_iot_hub_dps: Failed to disconnect gracefully, error: -95
[00:00:15.800,048] <err> azure_iot_hub_dps: There was a failure during DPS registration, process is stopped
[00:00:15.801,086] <err> mqtt_helper: Cloud MQTT input error: -111

Here is what I have successfully uploaded to the modem: 

And my overlay-azure.conf looks like:

CONFIG_AZURE_IOT_HUB=y
CONFIG_AZURE_IOT_HUB_DPS=y
CONFIG_AZURE_IOT_HUB_AUTO_DEVICE_TWIN_REQUEST=y

# Increase the number of maximum message properties that can be parsed by the Azure IoT Hub library.
# Needed to be able to parse P-GPS responses.
CONFIG_AZURE_IOT_HUB_MSG_PROPERTY_RECV_MAX_COUNT=4

# Azure IoT Hub options that must be configured in order to establish a connection.
CONFIG_AZURE_IOT_HUB_DPS_HOSTNAME="global.azure-devices-provisioning.net"
CONFIG_AZURE_IOT_HUB_DPS_ID_SCOPE="0ne********" # Stars replaces with my ID scope

# MQTT helper library
CONFIG_MQTT_HELPER_RX_TX_BUFFER_SIZE=2048
CONFIG_MQTT_HELPER_STACK_SIZE=4096
CONFIG_MQTT_HELPER_SEC_TAG=11

# MQTT Transport library
# Maximum specified MQTT keepalive timeout for Azure IoT Hub is 1177 seconds.
CONFIG_MQTT_KEEPALIVE=1177

The full trace is:

*** Booting nRF Connect SDK v2.5.2 ***
[00:00:00.257,446] <inf> app_event_manager: APP_EVT_START
[00:00:00.506,896] <inf> app_event_manager: MODEM_EVT_INITIALIZED
[00:00:00.516,571] <inf> azure_fota: Current firmware version: 0.0.0-development
[00:00:00.570,159] <inf> app_event_manager: MODEM_EVT_LTE_CONNECTING
[00:00:01.293,548] <inf> app_event_manager: DATA_EVT_CONFIG_INIT
%CESQ: 48,2,10,1
+CEREG: 2,"AEFE","005C4C16",7
[00:00:02.401,519] <inf> app_event_manager: MODEM_EVT_LTE_CELL_UPDATE
+CSCON: 1
+CGEV: ME PDN ACT 0,0
[00:00:03.967,681] <inf> app_event_manager: MODEM_EVT_LTE_CONNECTED
[00:00:03.968,902] <inf> app_event_manager: CLOUD_EVT_CONNECTING
+CNEC_ESM: 50,0
%MDMEV: SEARCH STATUS 2
+CEREG: 5,"AEFE","005C4C16",7,,,"00001010","11000001"
%XTIME: "0A","4220328121620A","00"
[00:00:04.073,364] <inf> app_event_manager: MODEM_EVT_LTE_PSM_UPDATE
[00:00:04.073,913] <inf> app_event_manager: DATA_EVT_DATE_TIME_OBTAINED
%CESQ: 47,2,5,0
%CESQ: 47,2,10,1
+CSCON: 0
+CSCON: 1
[00:00:17.401,397] <err> azure_iot_hub_dps: Connection was rejected with return code 5
[00:00:17.401,397] <wrn> azure_iot_hub_dps: Is the device certificate valid?
[00:00:17.401,458] <err> mqtt_helper: Library is in the wrong state (MQTT_STATE_DISCONNECTED), MQTT_STATE_CONNECTED required
[00:00:17.401,489] <err> azure_iot_hub_dps: Failed to disconnect gracefully, error: -95
[00:00:17.401,519] <err> azure_iot_hub_dps: There was a failure during DPS registration, process is stopped
[00:00:17.402,832] <err> mqtt_helper: Cloud MQTT input error: -111
+CSCON: 0
%CESQ: 46,2,4,0
%CESQ: 255,0,255,0

Hope you can help.

Thank you!
-Jeremy

[Sigurd Hellesvik]

Hi,

Have you tried to connect your device to Azure IoT Hub without DPS?

Regards,
Sigurd Hellesvik

[jczacharia]

Yes, I have.

I followed this tutorial: developer.nordicsemi.com/.../azure_iot_hub.html

I have tried 3 different ways of creating credentials:

• https://learn.microsoft.com/en-us/azure/iot-hub/tutorial-x509-test-certs?tabs=windows
• https://learn.microsoft.com/en-us/azure/iot-dps/tutorial-custom-hsm-enrollment-group-x509?tabs=windows&pivots=programming-language-ansi-c
• https://github.com/Azure/azure-iot-sdk-c/blob/main/tools/CACertificates/CACertificateOverview.md

But have not been able to get any to work. I still always see the "Is the device certificate valid?" error.

My overlay-azure.conf for one device:

CONFIG_AZURE_IOT_HUB=y
CONFIG_AZURE_IOT_HUB_DPS=n
CONFIG_AZURE_IOT_HUB_AUTO_DEVICE_TWIN_REQUEST=y

# Increase the number of maximum message properties that can be parsed by the Azure IoT Hub library.
# Needed to be able to parse P-GPS responses.
CONFIG_AZURE_IOT_HUB_MSG_PROPERTY_RECV_MAX_COUNT=4

CONFIG_AZURE_IOT_HUB_HOSTNAME="******.azure-devices.net" # Replace with my IoT Hub hostname
CONFIG_AZURE_IOT_HUB_DEVICE_ID="device-01"

# MQTT helper library
CONFIG_MQTT_HELPER_RX_TX_BUFFER_SIZE=2048
CONFIG_MQTT_HELPER_STACK_SIZE=4096
CONFIG_MQTT_HELPER_SEC_TAG=11

# MQTT Transport library
# Maximum specified MQTT keepalive timeout for Azure IoT Hub is 1177 seconds.
CONFIG_MQTT_KEEPALIVE=1177

I have tried each combination of creating credentials from each above source with both DPS and single device using both the asset_tracker_v2 codebase and as well as the https://github.com/NordicSemiconductor/asset-tracker-cloud-firmware-azure codebase.

But no matter which combination I pick, I cannot get the certificates provisioned correctly to establish a valid communication.

[jczacharia]

I have followed the Azure IoT Hub tutorial and that is also where I got the Baltimore and DigiCert CA certificates from.

I tried using both the Baltimore and DigiCert individually and together in the CA certificate textarea in the Certificate Manager in the Cellular Monitor.

If I do not include the Balitmore certificate in the CA certificate, then I get the error:

[00:00:04.746,582] <err> mqtt_helper: mqtt_connect, error: -111
[00:00:04.746,612] <err> azure_iot_hub_dps: mqtt_helper_connect failed, error: -111
[00:00:04.746,612] <err> azure_iot_hub: azure_iot_hub_dps_start failed, error: -111
[00:00:04.746,643] <err> azure_iot_hub_integration: azure_iot_hub_connect, error: -111

Which looks like no connection is established whereas if I include the Baltimore certificate, I then get:

[00:00:08.990,051] <err> azure_iot_hub: Connection was rejected with return code 5
[00:00:08.990,081] <wrn> azure_iot_hub: Is the device certificate valid?
[00:00:08.990,081] <dbg> azure_iot_hub_integration: azure_iot_hub_event_handler: AZURE_IOT_HUB_EVT_CONNECTION_FAILED
[00:00:08.991,119] <wrn> azure_iot_hub: DISCONNECT, result: -111
[00:00:08.991,149] <dbg> azure_iot_hub_integration: azure_iot_hub_event_handler: AZURE_IOT_HUB_EVT_DISCONNECTED
[00:00:08.991,180] <dbg> cloud_module: cloud_wrap_event_handler: CLOUD_WRAP_EVT_DISCONNECTED
[00:00:08.991,241] <inf> app_event_manager: CLOUD_EVT_DISCONNECTED
[00:00:08.991,882] <err> mqtt_helper: Cloud MQTT input error: -111

That appears to establish some sort of connection but then gets rejected because the device certificate is not valid.

So following the Azure IoT Hub tutorial (for a single device; no DPS) that instructs me to use Create and upload certificates for testing, I then

• take the Test Subordinate CA from rootca/certs and add that certificate as verified in the IoT hub certificates
• create a device certificate called device-01 and put that device's certificate from subca/certs into the Client certificate textarea in the Certificate Manager in the Cellular Monitor (also tried including the Test Subordinate CA along with it)
• add a new enabled device (non-IoT Edge) in the IoT hub called device-01 with X.509 CA Signed authentication type
• take device-01's private key subca/private/device-01.key and put that in the Private key textarea in the Certificate Manager in the Cellular Monitor

I certainly feel like I have followed these tutorials very precisely but cannot get a proper connection.

I have been able to establish proper communication with the nRF Cloud so I know that the SIM card is working. However, I am not sure if having the device on nRF Cloud is affecting Azure communication.

[Sigurd Hellesvik]

Hi,

From https://infocenter.nordicsemi.com/topic/comp_matrix_nrf9160/COMP/nrf9160/nrf9160_modem_fw.html?cp=2_1_3_5:

I see you use modem firmware 1.3.5 with NCS 2.5.2.

Maybe this could be related?

[jczacharia]

I've tried it with:

• SDK v2.5.2 - MFW 1.3.5
• SDK v2.5.2 - MFW 1.3.6
• SDK v2.5.1 - MFW 1.3.5
• SDK v2.5.1 - MFW 1.3.6
• SDK v2.5.99-dev1 - MFW 1.3.6
• SDK v2.5.99-dev1 - MFW 1.3.5

I even tried this on a couple of brand-new nRF9160-DKs and same issue.

Though, connecting is nRF Cloud has been successful for all devices.

I was planning on creating a video going step by step but I feel as though it might be redundant since it's just going through the documentation.

Is there a direct line of contact with the Nordic engineering team to address this challenge efficiently?

I am working with a company that is entering the market with a custom IoT hardware product, which is built on the Nordic platform. Over the past few years, we have developed a deep appreciation for the robustness and versatility of this platform but seeing that Azure and Nordic are changing rapidly and are currently facing this firmware/certification issue that is hampering our progress. Despite diligently following the latest documentation, even including the new tutorial and cert_tool.py script released on February 29, 2024, we have been unable to resolve this issue, which we suspect may be external.

I would be more than willing to contribute to the software and documentation to get a sound solution to this seeing that others are having similar problems.

Any help will be met with reciprocation.

Thanks!

[Sigurd Hellesvik]

jczacharia said:

Is there a direct line of contact with the Nordic engineering team to address this challenge efficiently?

Although it is often normal to have first-line support, we in technical support here at Nordic are application engineers in R&D at the same level as the rest of our engineering team, and have similar competence.
That being said, you are onto something: Most of our nRF9160 experts are at a conference this week. I do not have the most experience in this topic, although I have indeed used Azure with both DPS and normal provisioning myself a couple of years ago.

Next week I will ask for help from the people back from the conference and we will see what we can figure out here.

[jczacharia]

Just as an update: I have successfully connected to AWS. Still have no idea why Azure doesn't recognize my certificates.

[Mikael Jansson]

Did you make sure to copy the ETAG from the JSON response for each invocation of "az iot hub"? 

az iot hub certificate create --hub-name "$HUBNAME" --name "$CERTNAME" --path ca/root-ca-cert.pem

> read the ETAG, use in following command:

az iot hub certificate generate-verification-code --hub-name $HUBNAME --name MYROOTCA --etag "$ETAG"

> read the ETAG from the above command, use it in the following:

az iot hub certificate verify --hub-name $HUBNAME --name $CERTNAME --etag $ETAG2 --path certs/client-cert.pem

and so on.

[Byter]

Hello, I'm receiving the exact same issue. Followed the tutorial 1:1, copied the ETAG from the JSON response at each invocation and all. Not sure what the issue is at this point

[jczacharia]

Yes, I have done that. I can't seem to shake the -111 error (Is the device certificate valid?). I have a feeling it might be related to Azure DigiCert migration.