API registration of certification for Azure IoT Hub / DPS

[Thomas]

I am trying to connect a custom device to the Azure IoT Hub using DPS service. Because I am using a custom board I am trying to write certificates to the modem using api calls to the modem_key_mgmg functions. Theses functions have a tag and a credential type.

But trying to get around the different certificates in use.and what tags and credential types i should use when writing the certificates.

I have registered Baltimore and DigiCert root CA certificates using the tag CONFIG_MQTT_HELPER_SEC_TAG and CONFIG_MQTT_HELPER_SECONDARY_SEC_TAG and both of them with credential type set to MODEM_KEY_MGMT_CRED_TYPE_PUBLIC_CERT.

For the device certificate I am not really sure what the tag should be, so just set it to 12 for both types.

Then the private certificate is registered with type MODEM_KEY_MGMT_CRED_TYPE_PRIVATE_CERT.

But what about the public part? Should it be MODEM_KEY_MGMT_CRED_TYPE_PUBLIC_CERT?

Is the above at all correct? I am wondering about the tag field since I find no description except a reserrved tag for nRFCloud. Is it used somewhere in the Nordic libraries for Azure and has to have a specific value?

I am getting mqtt_helper, mqtt_connect, error: -111 when trying to connect.

[Charlie]

Hi Thomas,

Thanks for checking with us. The error is also reported in the following case. Have you read through it before to find some hint?

(+) Sample Azure IoT Hub (Thingy9.1) - Nordic Q&A - Nordic DevZone - Nordic DevZone (nordicsemi.com)

Best regards,

Charlie

[Thomas]

Hi Charlie

I have not tested different things to see if I can find a solution....

I have gone through the the Microsoft tutorial on creating and uploading certificates from the beginning again. This was to test that I did not have a problem with the certificates. I have created new internal root CA, subordinate and device certificates.

From this I have

• Internal root CA
• Subordinate certificate used for creating device certificates
• Device certificate(s)
• Device private key(s)

I also have two other public certificates old and new IoT hub

• Baltimore
• DigiCert

There are a number of different ways I can register these certificates in the device.... Is it possible that someone can describe this process from within the application.

I think the only certificates needed in the device is

• Device certificate(s)
• Device private key(s)
• Baltimore
• DigiCert

So I'm doing:

modem_key_mgmt_write(CONFIG_MQTT_HELPER_SEC_TAG, MODEM_KEY_MGMT_CRED_TYPE_CA_CHAIN, digicert, sizeof(digicert));

modem_key_mgmt_write(CONFIG_MQTT_HELPER_SECONDARY_SEC_TAG,MODEM_KEY_MGMT_CRED_TYPE_CA_CHAIN, baltimore, sizeof(baltimore));

modem_key_mgmt_write(CONFIG_MQTT_HELPER_SEC_TAG, MODEM_KEY_MGMT_CRED_TYPE_PUBLIC_CERT, devicecert, sizeof(devicecert));

modem_key_mgmt_write(CONFIG_MQTT_HELPER_SEC_TAG, MODEM_KEY_MGMT_CRED_TYPE_PRIVATE_CERT, devicecertpriv, sizeof(devicecertpriv))

[Thomas]

Great :-) Thanks a lot I will look forward to hearing the result :-)

[Charlie]

Hi Thomas,

Here is the brief report from my tests.

1) No DSP

I got the same -111 error as you did on the first try, but when I replaced BaltimoreCyberTrustRoot.crt.pem with DigiCertGlobalRootG2.crt.pem as root CA or set DigiCertGlobalRootG2.crt.pem rootCA, client certificate and private key in second security tag 11,

then the issue is solved.

Fullscreen azureIoT_NO_DPS.log Download

*** Booting nRF Connect SDK v2.5.0 ***
[00:00:00.253,723] <inf> azure_iot_hub_sample: Azure IoT Hub sample started
[00:00:00.253,723] <inf> azure_iot_hub_sample: Bringing network interface up and connecting to the network
[00:00:00.502,197] <inf> azure_iot_hub_sample: Device ID: testdevice
[00:00:00.502,349] <inf> azure_iot_hub_sample: Host name: ncs-sample.azure-devices.net
+CEREG: 2,"81A3","03238D0A",7
+CSCON: 1
+CGEV: ME PDN ACT 0
+CEREG: 1,"81A3","03238D0A",7,0,2,"11100000","11100000"
[00:00:03.939,117] <inf> azure_iot_hub_sample: Network connectivity established and IP address assigned
[00:00:03.939,178] <inf> azure_iot_hub_sample: Connected to network
[00:00:03.940,063] <inf> azure_fota: Current firmware version: 0.0.0-dev
[00:00:03.940,093] <dbg> azure_iot_hub: azure_iot_hub_init: Azure FOTA initialized
[00:00:03.940,124] <dbg> azure_iot_hub: iot_hub_state_set: State transition: STATE_UNINIT --> STATE_DISCONNECTED
[00:00:03.940,155] <inf> azure_iot_hub_sample: Azure IoT Hub library initialized
[00:00:03.940,216] <dbg> azure_iot_hub: iot_hub_state_set: State transition: STATE_DISCONNECTED --> STATE_CONNECTING
[00:00:03.940,216] <inf> azure_iot_hub_sample: AZURE_IOT_HUB_EVT_CONNECTING
[00:00:03.940,307] <dbg> azure_iot_hub: azure_iot_hub_connect: User name: ncs-sample.azure-devices.net/testdevice/?api-version=2020-09-30&DeviceClientType=azsdk-c%2F1.4.0-beta.2
[00:00:03.940,338] <dbg> azure_iot_hub: azure_iot_hub_connect: User name buffer size is 160, actual user name size is: 103
+CGEV: IPV6 0
[00:00:07.177,734] <inf> azure_iot_hub_sample: Connection request sent to IoT Hub
[00:00:07.316,162] <dbg> azure_iot_hub: iot_hub_state_set: State transition: STATE_CONNECTING --> STATE_CONNECTED
[00:00:07.316,162] <dbg> azure_iot_hub: on_connack: MQTT mqtt_client connected
[00:00:07.317,138] <dbg> azure_iot_hub: topic_subscribe: Successfully subscribed to default topics
[00:00:07.317,169] <inf> azure_iot_hub_sample: AZURE_IOT_HUB_EVT_CONNECTED
[00:00:07.394,104] <inf> azure_iot_hub_sample: AZURE_IOT_HUB_EVT_READY
[00:00:07.394,226] <dbg> azure_iot_hub: request_id_create_and_get: Request ID not specified, using "739"
[00:00:07.394,470] <inf> azure_iot_hub_sample: Sending event:{"temperature":25.4,"timestamp":7394}
[00:00:07.395,294] <dbg> azure_iot_hub: device_twin_request: Device twin requested
[00:00:07.396,270] <inf> azure_iot_hub_sample: Event was successfully sent
[00:00:07.396,270] <inf> azure_iot_hub_sample: Next event will be sent in 20 seconds
[00:00:07.502,960] <dbg> azure_iot_hub: on_publish: Message type: AZ_IOT_HUB_CLIENT_TWIN_RESPONSE_TYPE_GET
[00:00:07.502,990] <dbg> azure_iot_hub: on_publish: Device twin data received
[00:00:07.503,051] <dbg> azure_iot_hub: device_twin_result_process: Request ID: 739
[00:00:07.503,479] <wrn> azure_fota: No job ID found
[00:00:07.503,723] <dbg> azure_iot_hub: fota_evt_handler: AZURE_FOTA_EVT_REPORT
[00:00:07.503,814] <dbg> azure_iot_hub: request_id_create_and_get: Request ID not specified, using "750"
[00:00:07.504,882] <dbg> azure_iot_hub: fota_report_send: FOTA report sent
[00:00:07.505,493] <inf> azure_iot_hub_sample: AZURE_IOT_HUB_EVT_TWIN_RECEIVED
[00:00:07.505,920] <inf> azure_iot_hub_sample: No 'telemetryInterval' object in the device twin
[00:00:07.849,578] <dbg> azure_iot_hub: on_publish: Message type: AZ_IOT_HUB_CLIENT_TWIN_RESPONSE_TYPE_REPORTED_PROPERTIES
[00:00:07.849,609] <dbg> azure_iot_hub: on_publish: Device twin data received
[00:00:07.849,639] <dbg> azure_iot_hub: device_twin_result_process: Request ID: 750
[00:00:07.849,700] <inf> azure_iot_hub_sample: AZURE_IOT_HUB_EVT_TWIN_RESULT_SUCCESS, ID: 750
+CSCON: 0
+CEREG: 1,"81A3","031B4301",7,,,"11100000","11100000"
[00:00:27.396,453] <inf> azure_iot_hub_sample: Sending event:{"temperature":25.6,"timestamp":27396}
[00:00:27.405,883] <inf> azure_iot_hub_sample: Event was successfully sent
[00:00:27.405,914] <inf> azure_iot_hub_sample: Next event will be sent in 20 seconds
+CSCON: 1
+CSCON: 0
+CEREG: 1,"81A3","03238D0A",7,,,"11100000","11100000"
[00:00:47.406,066] <inf> azure_iot_hub_sample: Sending event:{"temperature":25.5,"timestamp":47405}
[00:00:47.415,466] <inf> azure_iot_hub_sample: Event was successfully sent
[00:00:47.415,466] <inf> azure_iot_hub_sample: Next event will be sent in 20 seconds
+CSCON: 1
+CSCON: 0
[00:01:07.415,649] <inf> azure_iot_hub_sample: Sending event:{"temperature":25.5,"timestamp":67415}
[00:01:07.425,048] <inf> azure_iot_hub_sample: Event was successfully sent
[00:01:07.425,048] <inf> azure_iot_hub_sample: Next event will be sent in 20 seconds
+CSCON: 1
+CSCON: 0
[00:01:27.425,231] <inf> azure_iot_hub_sample: Sending event:{"temperature":25.5,"timestamp":87425}
[00:01:27.434,631] <inf> azure_iot_hub_sample: Event was successfully sent
[00:01:27.434,631] <inf> azure_iot_hub_sample: Next event will be sent in 20 seconds
+CSCON: 1
+CSCON: 0

2) With DSP

I test with BaltimoreCyberTrustRoot.crt.crt.pem as root CA, client certificate and private key in security tag 10 and set DigiCertGlobalRootG2.crt.pem rootCA, client certificate and private key in second security tag 11, also set CONFIG_AZURE_IOT_HUB_DEVICE_ID="testdevice" in prj.conf since "the client certificate must have the value of its Subject Common Name (CN) field set to the value of the device ID." according to Tutorial - Create and upload certificates for testing - Azure IoT Hub | Microsoft Learn.

Fullscreen azureIoT_DPS.log Download

*** Booting nRF Connect SDK v2.5.0 ***
[00:00:00.253,753] <inf> azure_iot_hub_sample: Azure IoT Hub sample started
[00:00:00.253,784] <inf> azure_iot_hub_sample: Bringing network interface up and connecting to the network
[00:00:00.502,166] <inf> azure_iot_hub_sample: Device ID: testdevice
+CEREG: 2,"81A3","03238D0A",7
+CSCON: 1
+CGEV: ME PDN ACT 0
+CEREG: 1,"81A3","03238D0A",7,0,2,"11100000","11100000"
[00:00:04.147,155] <inf> azure_iot_hub_sample: Network connectivity established and IP address assigned
[00:00:04.147,186] <inf> azure_iot_hub_sample: Connected to network
[00:00:04.147,216] <inf> azure_iot_hub_sample: Starting DPS
[00:00:04.147,766] <inf> azure_iot_hub_sample: DPS registration status: AZURE_IOT_HUB_DPS_REG_STATUS_NOT_STARTED
+CGEV: IPV6 0
[00:00:08.272,613] <inf> azure_iot_hub_sample: The DPS process has started, timeout is set to 90 seconds
[00:00:08.416,717] <inf> azure_iot_hub_sample: DPS registration status: AZURE_IOT_HUB_DPS_REG_STATUS_ASSIGNING
[00:00:08.738,616] <inf> azure_iot_hub_sample: DPS registration status: AZURE_IOT_HUB_DPS_REG_STATUS_ASSIGNING
[00:00:11.944,274] <inf> azure_iot_hub_sample: DPS registration status: AZURE_IOT_HUB_DPS_REG_STATUS_ASSIGNED
[00:00:11.944,396] <inf> azure_iot_hub_sample: Device ID "testdevice" assigned to IoT hub with hostname "ncs-sample.azure-devices.net"
[00:00:11.945,159] <inf> azure_fota: Current firmware version: 0.0.0-dev
[00:00:11.945,190] <inf> azure_iot_hub_sample: Azure IoT Hub library initialized
[00:00:11.945,220] <inf> azure_iot_hub_sample: AZURE_IOT_HUB_EVT_CONNECTING
[00:00:15.054,077] <inf> azure_iot_hub_sample: Connection request sent to IoT Hub
[00:00:15.181,396] <inf> azure_iot_hub_sample: AZURE_IOT_HUB_EVT_CONNECTED
[00:00:15.281,585] <inf> azure_iot_hub_sample: AZURE_IOT_HUB_EVT_READY
[00:00:15.281,890] <inf> azure_iot_hub_sample: Sending event:{"temperature":25.1,"timestamp":15281}
[00:00:15.283,660] <inf> azure_iot_hub_sample: Event was successfully sent
[00:00:15.283,691] <inf> azure_iot_hub_sample: Next event will be sent in 20 seconds
[00:00:15.381,835] <wrn> azure_fota: No job ID found
[00:00:15.383,789] <inf> azure_iot_hub_sample: AZURE_IOT_HUB_EVT_TWIN_RECEIVED
[00:00:15.384,246] <inf> azure_iot_hub_sample: No 'telemetryInterval' object in the device twin
[00:00:15.728,973] <inf> azure_iot_hub_sample: AZURE_IOT_HUB_EVT_TWIN_RESULT_SUCCESS, ID: 153
+CSCON: 0
[00:00:35.283,843] <inf> azure_iot_hub_sample: Sending event:{"temperature":25.3,"timestamp":35283}
[00:00:35.293,243] <inf> azure_iot_hub_sample: Event was successfully sent
[00:00:35.293,273] <inf> azure_iot_hub_sample: Next event will be sent in 20 seconds
+CSCON: 1

What I have observed is fit for the CA migration timeline you can find from the following page. BaltimoreCyberTrustRoot.crt.pem is not valid for IoT Hub any more after October 15, 2023, but still valid for DSP, until February 15, 2024.

Azure IoT TLS: Critical changes are almost here! (…and why you should care) - Microsoft Community Hub

Best regards,

Charlie

[Thomas]

Hi Charlie

Thanks a lot for the effort :-)

Unfortunately I am still unable to connect. 

Could you share the test certificates and the exact way you register them?

I do know about the Baltimore/Digicert certificate change as I have several other device types and many devices running on the IoT Hub. The IoT hub I want to connect to is still using the Baltimore certificate. So I am registering this as  CONFIG_MQTT_HELPER_SEC_TAG. The certificates (PEM) was downloaded from a Microsoft page with a description of the two certificates.

I am now going to spin up a new test IoT Hub in a region where I know it will run Digicert and test that situation.

[Thomas]

A few questions (I think I have tried the different combinations, but....)

When uploading a certificate to Azure IoT hub certificates list, are you using the PEM file with only the generated subordinate certificate or a manually combined file with the certificate section for both subordinate and slef-signed root CA?

When adding your testdevice do you set authentificate to "X.509 Self-Signed" or "X.509 CA Signed"?

If you select "X.509 Self-Signed", what secondary thump print do you use?

[Charlie]

Hi Thomas,

Thomas said:

Could you share the test certificates and the exact way you register them?

I can share more details about how I register them.

1. Follow Tutorial - Create and upload certificates for testing - Azure IoT Hub | Microsoft Learn you will get the following keys.

C:\NCS\AzureIoTCerts>dir /s
 Volume in drive C has no label.
 Volume Serial Number is 7C9B-FC43

 Directory of C:\NCS\AzureIoTCerts

18/12/2023  15:21    <DIR>          .
18/12/2023  15:21    <DIR>          ..
18/12/2023  15:18    <DIR>          rootca
18/12/2023  16:08    <DIR>          subca
               0 File(s)              0 bytes

 Directory of C:\NCS\AzureIoTCerts\rootca

18/12/2023  15:18    <DIR>          .
18/12/2023  15:18    <DIR>          ..
18/12/2023  16:04    <DIR>          certs
18/12/2023  16:04    <DIR>          db
18/12/2023  15:10    <DIR>          private
18/12/2023  15:16             2,099 rootca.conf
18/12/2023  15:18             4,135 rootca.crt
18/12/2023  15:16             1,022 rootca.csr
               3 File(s)          7,256 bytes

 Directory of C:\NCS\AzureIoTCerts\rootca\certs

18/12/2023  16:04    <DIR>          .
18/12/2023  16:04    <DIR>          ..
18/12/2023  15:18             4,135 88A8F7A2E59F52BF8F8EA450CDBB125E.pem
18/12/2023  16:04             4,484 88A8F7A2E59F52BF8F8EA450CDBB1260.pem
               2 File(s)          8,619 bytes

 Directory of C:\NCS\AzureIoTCerts\rootca\db

18/12/2023  16:04    <DIR>          .
18/12/2023  16:04    <DIR>          ..
18/12/2023  15:03                 5 crlnumber
18/12/2023  16:04               231 index
18/12/2023  16:04                21 index.attr
18/12/2023  15:25                21 index.attr.old
18/12/2023  15:25               157 index.old
18/12/2023  16:04                34 serial
18/12/2023  15:25                34 serial.old
               7 File(s)            503 bytes

 Directory of C:\NCS\AzureIoTCerts\rootca\private

18/12/2023  15:10    <DIR>          .
18/12/2023  15:10    <DIR>          ..
18/12/2023  15:16             1,884 rootca.key
               1 File(s)          1,884 bytes

 Directory of C:\NCS\AzureIoTCerts\subca

18/12/2023  16:08    <DIR>          .
18/12/2023  16:08    <DIR>          ..
18/12/2023  16:08    <DIR>          certs
18/12/2023  16:08    <DIR>          db
18/12/2023  16:05    <DIR>          private
18/12/2023  16:02             2,094 subca.conf
18/12/2023  16:04             4,484 subca.crt
18/12/2023  16:03             1,018 subca.csr
18/12/2023  16:08             4,552 testdevice.crt
18/12/2023  16:07             1,002 testdevice.csr
               5 File(s)         13,150 bytes

 Directory of C:\NCS\AzureIoTCerts\subca\certs

18/12/2023  16:08    <DIR>          .
18/12/2023  16:08    <DIR>          ..
18/12/2023  16:08             4,552 EE3E911808A245024E9E824AC8F90B4E.pem
               1 File(s)          4,552 bytes

 Directory of C:\NCS\AzureIoTCerts\subca\db

18/12/2023  16:08    <DIR>          .
18/12/2023  16:08    <DIR>          ..
18/12/2023  15:58                 5 crlnumber
18/12/2023  16:08               120 index
18/12/2023  16:08                21 index.attr
18/12/2023  15:58                 0 index.old
18/12/2023  16:08                34 serial
18/12/2023  15:58                34 serial.old
               6 File(s)            214 bytes

 Directory of C:\NCS\AzureIoTCerts\subca\private

18/12/2023  16:05    <DIR>          .
18/12/2023  16:05    <DIR>          ..
18/12/2023  16:03             1,884 subca.key
18/12/2023  16:05             1,732 testdevice.key
               2 File(s)          3,616 bytes

     Total Files Listed:
              27 File(s)         39,794 bytes
              26 Dir(s)  101,159,108,608 bytes free

2. Upload subordinate CA certificate C:\NCS\AzureIoTCerts\rootca\certs\88A8F7A2E59F52BF8F8EA450CDBB1260.pem to IoT Hub or DSP Certificates page.

3. Copy and write the following keys into the device.

Server rootCA:  DigiCertGlobalRootG2.crt.pem or  BaltimoreCyberTrustRoot.crt.pem

Client certificate:  C:\NCS\AzureIoTCerts\subca\certs\EE3E911808A245024E9E824AC8F90B4E.pem

Client private key: C:\NCS\AzureIoTCerts\subca\private\testdevice.key

Program AT client or SLM sample first, then use Certificate Manager to write keys.

If you are testing IoT Hub without DSP, remember to set the following configuration

CONFIG_AZURE_IOT_HUB_DEVICE_ID="testdevice" # same as Subject Common Name (CN) field when you generate certificates

CONFIG_AZURE_IOT_HUB_HOSTNAME=""

Just let me know if there is anything else you feel confused.

Best regards,

Charlie

[Thomas]

If just using IoT hub, how do you add the device, especially authentification!