Cannot connect custom board to nRF Cloud

[Burt]

Hi, I have followed the instructions in Securely generating credentials for nRF91 Series devices - nRF Cloud but cannot connect to nRF Cloud.

I think the instructions on that page are WRONG!!! Because in step 9 the bold statement is made:

You can also perform all previous steps using just the device_credentials_installer.py script. For details, see the Device Credentials Installer documentation.

But as I look at my call to device_credentials_installer.py below, where did I get ca and ca_key from? Didn't I have to do some previous steps to get those? But that contradicts the instructions that clearly state "You can also perform ALL previous steps using just the device_credentials_installer.py script. Something does not jive. Edit: I must have used create_ca_cert.py using the IMEI (not UUID) for the CN. Hopefully that is okay: I did not see any instructions stating that a particular CN should be used (I started off trying to work with IMEI prior to seeing that UUID is more "natural."

I did modify prj.conf to use CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID=y and commented out #CONFIG_NRF_CLOUD_CLIENT_ID_SRC_IMEI=y.

In the running log I see this regularly:

[00:16:36.450,927] <inf> app_event_manager: CLOUD_EVT_CONNECTING
%CESQ: 36,1,24,3
%MDMEV: SEARCH STATUS 2
+CSCON: 1
[00:16:42.157,897] <err> nrf_cloud_transport: MQTT input error: -128
[00:16:42.157,958] <err> nrf_cloud_transport: Error disconnecting from cloud: -128
[00:16:42.158,050] <inf> app_event_manager: CLOUD_EVT_DISCONNECTED

On nrf_cloud my device shows only Name and ID, both of which are set to the UUID. So four columns are missing when I compare to my DK or Thingy.

I have done this in order to set things up, installing credentials and also onboarding. Please help. Thank you. I am quite confused given the dificulty level.

C:\Users\joe\nrfcloud\utils\python\modem-firmware-1.3+>python device_credentials_installer.py --port COM13 --rtscts_off --ca 0x79200d11779235442d9179e8635fb4922a536756_ca.pem --ca_key 0x79200d11779235442d9179e8635fb4922a536756_prv.pem
Opening port COM13 as generic device...
Disabling LTE and GNSS...
-> AT+CFUN=4
<- OK
-> AT+CGSN
<- 350457794634418
<- OK
Device IMEI: 350457794634418
-> AT+CGMR
<- mfw_nrf9160_1.3.1
<- OK
Modem FW version: mfw_nrf9160_1.3.1
Generating private key and requesting a CSR for sectag 16842753...
-> AT%KEYGEN=16842753,2,0
<- %KEYGEN: "MIIBCTCBrwIBADAvMS0wKwYDVQQDDCQ1MDRiNTgzOS0zNzM2LTQ4NzctODBmOC0xZjFkNzQwZmE5YWEwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARibVcjsI8WZbyYV9WTrTzbc-6xZmkxZeVd4I4xsb-aA5RYWN2i2PwOjYTt6ewpeRsIKYBPoyfGPhGo8j23BIJhoB4wHAYJKoZIhvcNAQkOMQ8wDTALBgNVHQ8EBAMCA-gwDAYIKoZIzj0EAwIFAANHADBEAiACnNCAq2YAwAfd3BuWicLuDa1TTPm26ay4xUKGHcK09AIgZq4NsK-QImFLaasOiTSnXRNjBSLBh4xZWpVmFpp-mNQ.0oRDoQEmoQRBIVhP2dn3hQlQUEtYOTc2SHeA-B8ddA-pqkUaAQEAAVgg67pvCUJ0GoE1-tmRSw9c_f2VJLjDhqmy4n80uRCs_lJQBjLHh5B6YshlintKr_4hElhAMXmonvYw9FHDVF1fwSf7Js3Lx89KIcrN-WcYM4TsDFaTshhnMt96nM2WIiT0ia28JzqmlqR1Z7UDUApbrAJPMw"
<- OK

Parsing AT%KEYGEN output:

-----BEGIN CERTIFICATE REQUEST-----
MIIBCTCBrwIBADAvMS0wKwYDVQQDDCQ1MDRiNTgzOS0zNzM2LTQ4NzctODBmOC0x
ZjFkNzQwZmE5YWEwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARibVcjsI8WZbyY
V9WTrTzbc+6xZmkxZeVd4I4xsb+aA5RYWN2i2PwOjYTt6ewpeRsIKYBPoyfGPhGo
8j23BIJhoB4wHAYJKoZIhvcNAQkOMQ8wDTALBgNVHQ8EBAMCA+gwDAYIKoZIzj0E
AwIFAANHADBEAiACnNCAq2YAwAfd3BuWicLuDa1TTPm26ay4xUKGHcK09AIgZq4N
sK+QImFLaasOiTSnXRNjBSLBh4xZWpVmFpp+mNQ=
-----END CERTIFICATE REQUEST-----

Device public key:
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEYm1XI7CPFmW8mFfVk60823PusWZp
MWXlXeCOMbG/mgOUWFjdotj8Do2E7ensKXkbCCmAT6Mnxj4RqPI9twSCYQ==
-----END PUBLIC KEY-----

SHA256 Digest:
ebba6f0942741a8135fad9914b0f5cfdfd9524b8c386a9b2e27f34b910acfe52

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
COSE:
Prot Hdr: 1 : -7 (ECDSA w/ SHA-256)
Unprot Hdr: 4 : -2 (identity_key)
---------------
Attestation:
Payload ID: CSR_msg_v1
Dev UUID: 504b5839-3736-4877-80f8-1f1d740fa9aa
sec_tag: 16842753
SHA256: ebba6f0942741a8135fad9914b0f5cfdfd9524b8c386a9b2e27f34b910acfe52
Nonce: 0632c787907a62c8658a7b4aaffe2112
---------------
Sig:
3179a89ef630f451c3545d5fc127fb26cdcbc7cf4a21cacdf967183384ec0c5693b2186732df7a9ccd962224f489adbc273aa696a47567b503500a5bac024f33

COSE digest matches payload
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Device ID: 504b5839-3736-4877-80f8-1f1d740fa9aa
Loading CA and key...
Creating device certificate...
Writing CA cert(s) to device...
-> AT%CMNG=0,16842753,0,"-----BEGIN CERTIFICATE-----
MIIDQTCCAimgAwIBAgITBmyfz5m/jAo54vB4ikPmljZbyjANBgkqhkiG9w0BAQsF
ADA5MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRkwFwYDVQQDExBBbWF6
b24gUm9vdCBDQSAxMB4XDTE1MDUyNjAwMDAwMFoXDTM4MDExNzAwMDAwMFowOTEL
MAkGA1UEBhMCVVMxDzANBgNVBAoTBkFtYXpvbjEZMBcGA1UEAxMQQW1hem9uIFJv
b3QgQ0EgMTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ4gHHKeNXj
ca9HgFB0fW7Y14h29Jlo91ghYPl0hAEvrAIthtOgQ3pOsqTQNroBvo3bSMgHFzZM
9O6II8c+6zf1tRn4SWiw3te5djgdYZ6k/oI2peVKVuRF4fn9tBb6dNqcmzU5L/qw
IFAGbHrQgLKm+a/sRxmPUDgH3KKHOVj4utWp+UhnMJbulHheb4mjUcAwhmahRWa6
VOujw5H5SNz/0egwLX0tdHA114gk957EWW67c4cX8jJGKLhD+rcdqsq08p8kDi1L
93FcXmn/6pUCyziKrlA4b9v7LWIbxcceVOF34GfID5yHI9Y/QCB/IIDEgEw+OyQm
jgSubJrIqg0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMC
AYYwHQYDVR0OBBYEFIQYzIU07LwMlJQuCFmcx7IQTgoIMA0GCSqGSIb3DQEBCwUA
A4IBAQCY8jdaQZChGsV2USggNiMOruYou6r4lK5IpDB/G/wkjUu0yKGX9rbxenDI
U5PMCCjjmCXPI6T53iHTfIUJrU6adTrCC2qJeHZERxhlbI1Bjjt/msv0tadQ1wUs
N+gDS63pYaACbvXy8MWy7Vu33PqUXHeeE6V/Uq2V8viTO96LXFvKWlJbYK8U90vv
o/ufQJVtMVT8QtPHRh8jrdkPSHCa2XV4cdFyQzR1bldZwgJcJmApzyMZFo6IQ6XU
5MsI+yMRQ+hDKXJioaldXgjUkK642M4UwtBV8ob2xJNDd2ZhwLnoQdeXeGADbkpy
rqXRfboQnoZsG4q5WTP468SQvvG5
-----END CERTIFICATE-----
"
<- OK
Writing dev cert to device...
-> AT%CMNG=0,16842753,1,"-----BEGIN CERTIFICATE-----
MIIBNTCB3QIUHKda42NToWPdU1s8qIaXzGot+20wCgYIKoZIzj0EAwIwDTELMAkG
A1UEBhMCVVMwHhcNMjQwNTAyMDMwNzMwWhcNMzQwNDMwMDMwNzMwWjAvMS0wKwYD
VQQDDCQ1MDRiNTgzOS0zNzM2LTQ4NzctODBmOC0xZjFkNzQwZmE5YWEwWTATBgcq
hkjOPQIBBggqhkjOPQMBBwNCAARibVcjsI8WZbyYV9WTrTzbc+6xZmkxZeVd4I4x
sb+aA5RYWN2i2PwOjYTt6ewpeRsIKYBPoyfGPhGo8j23BIJhMAoGCCqGSM49BAMC
A0cAMEQCICxfFZmiZT67sBdmqAWoFLMgZ7CaeHtNWb6LoCjILN/QAiAB7txcZI8W
9nbESKFwyXvFeZIQi7nIjJ3GBzvn7X61Vg==
-----END CERTIFICATE-----
"
<- OK
Saving nRF Cloud device onboarding CSV file onboard.csv...
--- File onboard.csv exists; overwrite, append, or quit (y,a,n)? y
Onboarding CSV file saved

And then I have done this

python nrf_cloud_onboard.py --apikey f8b0026db89c7d09a29567d56b7ec30aaf1b4ca6
Devices to be onboarded: 1
Onboarding API call result: 202 - Accepted
Response: {"bulkOpsRequestId":"01HWVMK63WZB5MZKPKWVHSRHM3"}
Fetching results for bulkOpsRequestId: 01HWVMK63WZB5MZKPKWVHSRHM3
Waiting 5s...
Onboarding status: SUCCEEDED

CSV-formatted results:
bulkOpsRequestId,01HWVMK63WZB5MZKPKWVHSRHM3
status,SUCCEEDED
endpoint,PROVISION_DEVICES
requestedAt,2024-05-02T03:12:01.148Z
completedAt,2024-05-02T03:12:05.766Z
uploadedDataUrl,bulk-ops-requests.nrfcloud.com/.../01HWVMK63WZB5MZKPKWVHSRHM3.csv
Error count,0

Device ID,Result
504b5839-3736-4877-80f8-1f1d740fa9aa,OK

[Burt]

I got things to work. In addition to the CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID=y change, I also had to add CONFIG_HW_ID_LIBRARY_SOURCE_UUID=y and most likely comment out CONFIG_HW_ID_LIBRARY_SOURCE_IMEI=y. But in order to set CONFIG_HW_ID_LIBRARY_SOURCE_UUID=y I also had to add CONFIG_MODEM_JWT=y. None of this is explained in nRF Cloud (nordicsemi.com) under Connecting. Well, maybe it is further down on the page, but you really have to look hard, and certainly CONFIG_MODEM_JWT is not mentioned. Wait, actually someone reading the document would think they are okay with CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID=y but would have no way of understanding the subtle difference with that option vs CONFIG_NRF_CLOUD_CLIENT_ID_SRC_HW_ID. Anyway, someplace there is a suggestion that UUID is the default, but that's hard to square with the way that asset_tracker_v2 comes up. Something like

cloud_wrap_init() HW_ID_LEN=37 hw_id_buf=504b5839-3736-4877-80f8-1f1d740fa9aa
nrf_cloud_init() entered... param->client_id=nrf-504b5839-3736-4
nct_init() entered... client_id=nrf-504b5839-3736-4
[00:00:00.448,699] <inf> nrf_modem_lib_trace: Trace thread ready
[00:00:00.450,073] <inf> nrf_modem_lib_trace: Trace level override: 2
[00:00:00.450,683] <wrn> lte_lc: About to lte_lc_system_mode_set
[00:00:00.453,765] <inf> app_event_manager: MODEM_EVT_INITIALIZED
[00:00:00.559,814] <wrn> nrf_cloud_transport: Not configured for runtime client ID, ignoring
[00:00:00.653,045] <dbg> nrf_cloud_transport: nct_client_id_set: client_id = 504b5839-3736-4877-80f8-1f1d740fa9aa

after I threw enough options into prj.conf. Maybe you can show me a simplification.

One more question: the documentation states that nrf-<IMEI> will work as a client id only for Thingy and DK boards. Is that still true? I imagine so, but it sure would be a lot easier if I didn't have to jump through so many hoops for a custom board. So I ask the question.

Thanks,

Burt

[Burt]

Looking more at asset_tracker_v2/src/cloud/nrf_cloud_integration.c in v2.5.0, this looks a bit corny:

#define IMEI_LEN 15
#define CLOUD_CLIENT_ID_IMEI_PREFIX_LEN (sizeof(CONFIG_CLOUD_CLIENT_ID_IMEI_PREFIX) - 1)

#if !defined(CONFIG_CLOUD_CLIENT_ID_USE_CUSTOM)
#define NRF_CLOUD_CLIENT_ID_LEN (IMEI_LEN + CLOUD_CLIENT_ID_IMEI_PREFIX_LEN)
static char client_id_buf[NRF_CLOUD_CLIENT_ID_LEN + 1] = CONFIG_CLOUD_CLIENT_ID_IMEI_PREFIX;
#else
#define NRF_CLOUD_CLIENT_ID_LEN (sizeof(CONFIG_CLOUD_CLIENT_ID) - 1)
static char client_id_buf[NRF_CLOUD_CLIENT_ID_LEN + 1];
#endif

So they don't seem to acknowledge the existence of UUID based cloud client id, but then later in nrf_cloud_transport nct_client_id_set() they are smart enough to know that based on IS_ENABLED(CONFIG_NRF_CLOUD_CLIENT_ID_SRC_RUNTIME) not being true that they should ignore the malformed client_id. Okay, they clearly do the right thing, but it's pretty complicated so maybe that corresponds to me having had to jump through hoops to get things working!

[Burt]

Okay, I see that with an improvement one could make to asset_tracker_v2 application (ignore the added printf)

diff --git a/applications/asset_tracker_v2/src/cloud/nrf_cloud_integration.c b/applications/asset_tracker_v2/src/cloud/nrf_cloud_integration.c
index c624fd544..156cba1a6 100644
--- a/applications/asset_tracker_v2/src/cloud/nrf_cloud_integration.c
+++ b/applications/asset_tracker_v2/src/cloud/nrf_cloud_integration.c
@@ -303,10 +303,12 @@ int cloud_wrap_init(cloud_wrap_evt_handler_t event_handler)
#endif
};

-#if !defined(CONFIG_CLOUD_CLIENT_ID_USE_CUSTOM)
+#if !defined(CONFIG_CLOUD_CLIENT_ID_USE_CUSTOM) \
+ && !defined(CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID)
char hw_id_buf[HW_ID_LEN];

err = hw_id_get(hw_id_buf, ARRAY_SIZE(hw_id_buf));
+printf("cloud_wrap_init() HW_ID_LEN=%d hw_id_buf=%s\n",HW_ID_LEN, hw_id_buf);//BMS

if (err) {
LOG_ERR("Failed to retrieve device ID");
@@ -314,7 +316,7 @@ int cloud_wrap_init(cloud_wrap_evt_handler_t event_handler)
}

strncat(client_id_buf, hw_id_buf, IMEI_LEN);
-#else
+#elif defined(CONFIG_CLOUD_CLIENT_ID_USE_CUSTOM)
snprintf(client_id_buf, sizeof(client_id_buf), "%s", CONFIG_CLOUD_CLIENT_ID);
#endif

Then one could greatly simplify the prj.conf for the case of UUID: in prj.conf you just need to add

CONFIG_MODEM_JWT=y
CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID=y

and you can get rid of the use of the HW LIBRARY, like
-CONFIG_HW_ID_LIBRARY=y
-CONFIG_HW_ID_LIBRARY_SOURCE_IMEI=y

Burt

[simensr]

Hi Burt ! Are you willing to make a PR towards sdk-nrf with your suggested changes?

github.com/.../sdk-nrf

Simen

[Pete Skeggs]

Hello Burt.  Thank you for your feedback.

In reply to:

"I must have used create_ca_cert.py using the IMEI (not UUID) for the CN. Hopefully that is okay: I did not see any instructions stating that a particular CN should be used (I started off trying to work with IMEI prior to seeing that UUID is more "natural.""

The CN doesn't matter. The example in the docs here: https://github.com/nRFCloud/utils/tree/master/python/modem-firmware-1.3%2B#create-ca-cert shows:
-cn example.com

The next section explains how to use the device credentials installer: https://github.com/nRFCloud/utils/tree/master/python/modem-firmware-1.3%2B#device-credentials-installer. This section does say that create_ca_cert.py is required first.

We will add clarifying statements to the Securely generating credentials page.