Cannot connect custom board to nRF Cloud

[Burt]

Hi, I have followed the instructions in Securely generating credentials for nRF91 Series devices - nRF Cloud but cannot connect to nRF Cloud.

I think the instructions on that page are WRONG!!! Because in step 9 the bold statement is made:

You can also perform all previous steps using just the device_credentials_installer.py script. For details, see the Device Credentials Installer documentation.

But as I look at my call to device_credentials_installer.py below, where did I get ca and ca_key from? Didn't I have to do some previous steps to get those? But that contradicts the instructions that clearly state "You can also perform ALL previous steps using just the device_credentials_installer.py script. Something does not jive. Edit: I must have used create_ca_cert.py using the IMEI (not UUID) for the CN. Hopefully that is okay: I did not see any instructions stating that a particular CN should be used (I started off trying to work with IMEI prior to seeing that UUID is more "natural."

I did modify prj.conf to use CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID=y and commented out #CONFIG_NRF_CLOUD_CLIENT_ID_SRC_IMEI=y.

In the running log I see this regularly:

[00:16:36.450,927] <inf> app_event_manager: CLOUD_EVT_CONNECTING
%CESQ: 36,1,24,3
%MDMEV: SEARCH STATUS 2
+CSCON: 1
[00:16:42.157,897] <err> nrf_cloud_transport: MQTT input error: -128
[00:16:42.157,958] <err> nrf_cloud_transport: Error disconnecting from cloud: -128
[00:16:42.158,050] <inf> app_event_manager: CLOUD_EVT_DISCONNECTED

On nrf_cloud my device shows only Name and ID, both of which are set to the UUID. So four columns are missing when I compare to my DK or Thingy.

I have done this in order to set things up, installing credentials and also onboarding. Please help. Thank you. I am quite confused given the dificulty level.

C:\Users\joe\nrfcloud\utils\python\modem-firmware-1.3+>python device_credentials_installer.py --port COM13 --rtscts_off --ca 0x79200d11779235442d9179e8635fb4922a536756_ca.pem --ca_key 0x79200d11779235442d9179e8635fb4922a536756_prv.pem
Opening port COM13 as generic device...
Disabling LTE and GNSS...
-> AT+CFUN=4
<- OK
-> AT+CGSN
<- 350457794634418
<- OK
Device IMEI: 350457794634418
-> AT+CGMR
<- mfw_nrf9160_1.3.1
<- OK
Modem FW version: mfw_nrf9160_1.3.1
Generating private key and requesting a CSR for sectag 16842753...
-> AT%KEYGEN=16842753,2,0
<- %KEYGEN: "MIIBCTCBrwIBADAvMS0wKwYDVQQDDCQ1MDRiNTgzOS0zNzM2LTQ4NzctODBmOC0xZjFkNzQwZmE5YWEwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARibVcjsI8WZbyYV9WTrTzbc-6xZmkxZeVd4I4xsb-aA5RYWN2i2PwOjYTt6ewpeRsIKYBPoyfGPhGo8j23BIJhoB4wHAYJKoZIhvcNAQkOMQ8wDTALBgNVHQ8EBAMCA-gwDAYIKoZIzj0EAwIFAANHADBEAiACnNCAq2YAwAfd3BuWicLuDa1TTPm26ay4xUKGHcK09AIgZq4NsK-QImFLaasOiTSnXRNjBSLBh4xZWpVmFpp-mNQ.0oRDoQEmoQRBIVhP2dn3hQlQUEtYOTc2SHeA-B8ddA-pqkUaAQEAAVgg67pvCUJ0GoE1-tmRSw9c_f2VJLjDhqmy4n80uRCs_lJQBjLHh5B6YshlintKr_4hElhAMXmonvYw9FHDVF1fwSf7Js3Lx89KIcrN-WcYM4TsDFaTshhnMt96nM2WIiT0ia28JzqmlqR1Z7UDUApbrAJPMw"
<- OK

Parsing AT%KEYGEN output:

-----BEGIN CERTIFICATE REQUEST-----
MIIBCTCBrwIBADAvMS0wKwYDVQQDDCQ1MDRiNTgzOS0zNzM2LTQ4NzctODBmOC0x
ZjFkNzQwZmE5YWEwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARibVcjsI8WZbyY
V9WTrTzbc+6xZmkxZeVd4I4xsb+aA5RYWN2i2PwOjYTt6ewpeRsIKYBPoyfGPhGo
8j23BIJhoB4wHAYJKoZIhvcNAQkOMQ8wDTALBgNVHQ8EBAMCA+gwDAYIKoZIzj0E
AwIFAANHADBEAiACnNCAq2YAwAfd3BuWicLuDa1TTPm26ay4xUKGHcK09AIgZq4N
sK+QImFLaasOiTSnXRNjBSLBh4xZWpVmFpp+mNQ=
-----END CERTIFICATE REQUEST-----

Device public key:
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEYm1XI7CPFmW8mFfVk60823PusWZp
MWXlXeCOMbG/mgOUWFjdotj8Do2E7ensKXkbCCmAT6Mnxj4RqPI9twSCYQ==
-----END PUBLIC KEY-----

SHA256 Digest:
ebba6f0942741a8135fad9914b0f5cfdfd9524b8c386a9b2e27f34b910acfe52

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
COSE:
Prot Hdr: 1 : -7 (ECDSA w/ SHA-256)
Unprot Hdr: 4 : -2 (identity_key)
---------------
Attestation:
Payload ID: CSR_msg_v1
Dev UUID: 504b5839-3736-4877-80f8-1f1d740fa9aa
sec_tag: 16842753
SHA256: ebba6f0942741a8135fad9914b0f5cfdfd9524b8c386a9b2e27f34b910acfe52
Nonce: 0632c787907a62c8658a7b4aaffe2112
---------------
Sig:
3179a89ef630f451c3545d5fc127fb26cdcbc7cf4a21cacdf967183384ec0c5693b2186732df7a9ccd962224f489adbc273aa696a47567b503500a5bac024f33

COSE digest matches payload
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Device ID: 504b5839-3736-4877-80f8-1f1d740fa9aa
Loading CA and key...
Creating device certificate...
Writing CA cert(s) to device...
-> AT%CMNG=0,16842753,0,"-----BEGIN CERTIFICATE-----
MIIDQTCCAimgAwIBAgITBmyfz5m/jAo54vB4ikPmljZbyjANBgkqhkiG9w0BAQsF
ADA5MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRkwFwYDVQQDExBBbWF6
b24gUm9vdCBDQSAxMB4XDTE1MDUyNjAwMDAwMFoXDTM4MDExNzAwMDAwMFowOTEL
MAkGA1UEBhMCVVMxDzANBgNVBAoTBkFtYXpvbjEZMBcGA1UEAxMQQW1hem9uIFJv
b3QgQ0EgMTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ4gHHKeNXj
ca9HgFB0fW7Y14h29Jlo91ghYPl0hAEvrAIthtOgQ3pOsqTQNroBvo3bSMgHFzZM
9O6II8c+6zf1tRn4SWiw3te5djgdYZ6k/oI2peVKVuRF4fn9tBb6dNqcmzU5L/qw
IFAGbHrQgLKm+a/sRxmPUDgH3KKHOVj4utWp+UhnMJbulHheb4mjUcAwhmahRWa6
VOujw5H5SNz/0egwLX0tdHA114gk957EWW67c4cX8jJGKLhD+rcdqsq08p8kDi1L
93FcXmn/6pUCyziKrlA4b9v7LWIbxcceVOF34GfID5yHI9Y/QCB/IIDEgEw+OyQm
jgSubJrIqg0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMC
AYYwHQYDVR0OBBYEFIQYzIU07LwMlJQuCFmcx7IQTgoIMA0GCSqGSIb3DQEBCwUA
A4IBAQCY8jdaQZChGsV2USggNiMOruYou6r4lK5IpDB/G/wkjUu0yKGX9rbxenDI
U5PMCCjjmCXPI6T53iHTfIUJrU6adTrCC2qJeHZERxhlbI1Bjjt/msv0tadQ1wUs
N+gDS63pYaACbvXy8MWy7Vu33PqUXHeeE6V/Uq2V8viTO96LXFvKWlJbYK8U90vv
o/ufQJVtMVT8QtPHRh8jrdkPSHCa2XV4cdFyQzR1bldZwgJcJmApzyMZFo6IQ6XU
5MsI+yMRQ+hDKXJioaldXgjUkK642M4UwtBV8ob2xJNDd2ZhwLnoQdeXeGADbkpy
rqXRfboQnoZsG4q5WTP468SQvvG5
-----END CERTIFICATE-----
"
<- OK
Writing dev cert to device...
-> AT%CMNG=0,16842753,1,"-----BEGIN CERTIFICATE-----
MIIBNTCB3QIUHKda42NToWPdU1s8qIaXzGot+20wCgYIKoZIzj0EAwIwDTELMAkG
A1UEBhMCVVMwHhcNMjQwNTAyMDMwNzMwWhcNMzQwNDMwMDMwNzMwWjAvMS0wKwYD
VQQDDCQ1MDRiNTgzOS0zNzM2LTQ4NzctODBmOC0xZjFkNzQwZmE5YWEwWTATBgcq
hkjOPQIBBggqhkjOPQMBBwNCAARibVcjsI8WZbyYV9WTrTzbc+6xZmkxZeVd4I4x
sb+aA5RYWN2i2PwOjYTt6ewpeRsIKYBPoyfGPhGo8j23BIJhMAoGCCqGSM49BAMC
A0cAMEQCICxfFZmiZT67sBdmqAWoFLMgZ7CaeHtNWb6LoCjILN/QAiAB7txcZI8W
9nbESKFwyXvFeZIQi7nIjJ3GBzvn7X61Vg==
-----END CERTIFICATE-----
"
<- OK
Saving nRF Cloud device onboarding CSV file onboard.csv...
--- File onboard.csv exists; overwrite, append, or quit (y,a,n)? y
Onboarding CSV file saved

And then I have done this

python nrf_cloud_onboard.py --apikey f8b0026db89c7d09a29567d56b7ec30aaf1b4ca6
Devices to be onboarded: 1
Onboarding API call result: 202 - Accepted
Response: {"bulkOpsRequestId":"01HWVMK63WZB5MZKPKWVHSRHM3"}
Fetching results for bulkOpsRequestId: 01HWVMK63WZB5MZKPKWVHSRHM3
Waiting 5s...
Onboarding status: SUCCEEDED

CSV-formatted results:
bulkOpsRequestId,01HWVMK63WZB5MZKPKWVHSRHM3
status,SUCCEEDED
endpoint,PROVISION_DEVICES
requestedAt,2024-05-02T03:12:01.148Z
completedAt,2024-05-02T03:12:05.766Z
uploadedDataUrl,bulk-ops-requests.nrfcloud.com/.../01HWVMK63WZB5MZKPKWVHSRHM3.csv
Error count,0

Device ID,Result
504b5839-3736-4877-80f8-1f1d740fa9aa,OK

[Burt]

I got things to work. In addition to the CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID=y change, I also had to add CONFIG_HW_ID_LIBRARY_SOURCE_UUID=y and most likely comment out CONFIG_HW_ID_LIBRARY_SOURCE_IMEI=y. But in order to set CONFIG_HW_ID_LIBRARY_SOURCE_UUID=y I also had to add CONFIG_MODEM_JWT=y. None of this is explained in nRF Cloud (nordicsemi.com) under Connecting. Well, maybe it is further down on the page, but you really have to look hard, and certainly CONFIG_MODEM_JWT is not mentioned. Wait, actually someone reading the document would think they are okay with CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID=y but would have no way of understanding the subtle difference with that option vs CONFIG_NRF_CLOUD_CLIENT_ID_SRC_HW_ID. Anyway, someplace there is a suggestion that UUID is the default, but that's hard to square with the way that asset_tracker_v2 comes up. Something like

cloud_wrap_init() HW_ID_LEN=37 hw_id_buf=504b5839-3736-4877-80f8-1f1d740fa9aa
nrf_cloud_init() entered... param->client_id=nrf-504b5839-3736-4
nct_init() entered... client_id=nrf-504b5839-3736-4
[00:00:00.448,699] <inf> nrf_modem_lib_trace: Trace thread ready
[00:00:00.450,073] <inf> nrf_modem_lib_trace: Trace level override: 2
[00:00:00.450,683] <wrn> lte_lc: About to lte_lc_system_mode_set
[00:00:00.453,765] <inf> app_event_manager: MODEM_EVT_INITIALIZED
[00:00:00.559,814] <wrn> nrf_cloud_transport: Not configured for runtime client ID, ignoring
[00:00:00.653,045] <dbg> nrf_cloud_transport: nct_client_id_set: client_id = 504b5839-3736-4877-80f8-1f1d740fa9aa

after I threw enough options into prj.conf. Maybe you can show me a simplification.

One more question: the documentation states that nrf-<IMEI> will work as a client id only for Thingy and DK boards. Is that still true? I imagine so, but it sure would be a lot easier if I didn't have to jump through so many hoops for a custom board. So I ask the question.

Thanks,

Burt

[Burt]

Looking more at asset_tracker_v2/src/cloud/nrf_cloud_integration.c in v2.5.0, this looks a bit corny:

#define IMEI_LEN 15
#define CLOUD_CLIENT_ID_IMEI_PREFIX_LEN (sizeof(CONFIG_CLOUD_CLIENT_ID_IMEI_PREFIX) - 1)

#if !defined(CONFIG_CLOUD_CLIENT_ID_USE_CUSTOM)
#define NRF_CLOUD_CLIENT_ID_LEN (IMEI_LEN + CLOUD_CLIENT_ID_IMEI_PREFIX_LEN)
static char client_id_buf[NRF_CLOUD_CLIENT_ID_LEN + 1] = CONFIG_CLOUD_CLIENT_ID_IMEI_PREFIX;
#else
#define NRF_CLOUD_CLIENT_ID_LEN (sizeof(CONFIG_CLOUD_CLIENT_ID) - 1)
static char client_id_buf[NRF_CLOUD_CLIENT_ID_LEN + 1];
#endif

So they don't seem to acknowledge the existence of UUID based cloud client id, but then later in nrf_cloud_transport nct_client_id_set() they are smart enough to know that based on IS_ENABLED(CONFIG_NRF_CLOUD_CLIENT_ID_SRC_RUNTIME) not being true that they should ignore the malformed client_id. Okay, they clearly do the right thing, but it's pretty complicated so maybe that corresponds to me having had to jump through hoops to get things working!

[Burt]

Okay, I see that with an improvement one could make to asset_tracker_v2 application (ignore the added printf)

diff --git a/applications/asset_tracker_v2/src/cloud/nrf_cloud_integration.c b/applications/asset_tracker_v2/src/cloud/nrf_cloud_integration.c
index c624fd544..156cba1a6 100644
--- a/applications/asset_tracker_v2/src/cloud/nrf_cloud_integration.c
+++ b/applications/asset_tracker_v2/src/cloud/nrf_cloud_integration.c
@@ -303,10 +303,12 @@ int cloud_wrap_init(cloud_wrap_evt_handler_t event_handler)
#endif
};

-#if !defined(CONFIG_CLOUD_CLIENT_ID_USE_CUSTOM)
+#if !defined(CONFIG_CLOUD_CLIENT_ID_USE_CUSTOM) \
+ && !defined(CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID)
char hw_id_buf[HW_ID_LEN];

err = hw_id_get(hw_id_buf, ARRAY_SIZE(hw_id_buf));
+printf("cloud_wrap_init() HW_ID_LEN=%d hw_id_buf=%s\n",HW_ID_LEN, hw_id_buf);//BMS

if (err) {
LOG_ERR("Failed to retrieve device ID");
@@ -314,7 +316,7 @@ int cloud_wrap_init(cloud_wrap_evt_handler_t event_handler)
}

strncat(client_id_buf, hw_id_buf, IMEI_LEN);
-#else
+#elif defined(CONFIG_CLOUD_CLIENT_ID_USE_CUSTOM)
snprintf(client_id_buf, sizeof(client_id_buf), "%s", CONFIG_CLOUD_CLIENT_ID);
#endif

Then one could greatly simplify the prj.conf for the case of UUID: in prj.conf you just need to add

CONFIG_MODEM_JWT=y
CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID=y

and you can get rid of the use of the HW LIBRARY, like
-CONFIG_HW_ID_LIBRARY=y
-CONFIG_HW_ID_LIBRARY_SOURCE_IMEI=y

Burt