Cannot connect custom board to nRF Cloud

[Burt]

Hi, I have followed the instructions in Securely generating credentials for nRF91 Series devices - nRF Cloud but cannot connect to nRF Cloud.

I think the instructions on that page are WRONG!!! Because in step 9 the bold statement is made:

You can also perform all previous steps using just the device_credentials_installer.py script. For details, see the Device Credentials Installer documentation.

But as I look at my call to device_credentials_installer.py below, where did I get ca and ca_key from? Didn't I have to do some previous steps to get those? But that contradicts the instructions that clearly state "You can also perform ALL previous steps using just the device_credentials_installer.py script. Something does not jive. Edit: I must have used create_ca_cert.py using the IMEI (not UUID) for the CN. Hopefully that is okay: I did not see any instructions stating that a particular CN should be used (I started off trying to work with IMEI prior to seeing that UUID is more "natural."

I did modify prj.conf to use CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID=y and commented out #CONFIG_NRF_CLOUD_CLIENT_ID_SRC_IMEI=y.

In the running log I see this regularly:

[00:16:36.450,927] <inf> app_event_manager: CLOUD_EVT_CONNECTING
%CESQ: 36,1,24,3
%MDMEV: SEARCH STATUS 2
+CSCON: 1
[00:16:42.157,897] <err> nrf_cloud_transport: MQTT input error: -128
[00:16:42.157,958] <err> nrf_cloud_transport: Error disconnecting from cloud: -128
[00:16:42.158,050] <inf> app_event_manager: CLOUD_EVT_DISCONNECTED

On nrf_cloud my device shows only Name and ID, both of which are set to the UUID. So four columns are missing when I compare to my DK or Thingy.

I have done this in order to set things up, installing credentials and also onboarding. Please help. Thank you. I am quite confused given the dificulty level.

C:\Users\joe\nrfcloud\utils\python\modem-firmware-1.3+>python device_credentials_installer.py --port COM13 --rtscts_off --ca 0x79200d11779235442d9179e8635fb4922a536756_ca.pem --ca_key 0x79200d11779235442d9179e8635fb4922a536756_prv.pem
Opening port COM13 as generic device...
Disabling LTE and GNSS...
-> AT+CFUN=4
<- OK
-> AT+CGSN
<- 350457794634418
<- OK
Device IMEI: 350457794634418
-> AT+CGMR
<- mfw_nrf9160_1.3.1
<- OK
Modem FW version: mfw_nrf9160_1.3.1
Generating private key and requesting a CSR for sectag 16842753...
-> AT%KEYGEN=16842753,2,0
<- %KEYGEN: "MIIBCTCBrwIBADAvMS0wKwYDVQQDDCQ1MDRiNTgzOS0zNzM2LTQ4NzctODBmOC0xZjFkNzQwZmE5YWEwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARibVcjsI8WZbyYV9WTrTzbc-6xZmkxZeVd4I4xsb-aA5RYWN2i2PwOjYTt6ewpeRsIKYBPoyfGPhGo8j23BIJhoB4wHAYJKoZIhvcNAQkOMQ8wDTALBgNVHQ8EBAMCA-gwDAYIKoZIzj0EAwIFAANHADBEAiACnNCAq2YAwAfd3BuWicLuDa1TTPm26ay4xUKGHcK09AIgZq4NsK-QImFLaasOiTSnXRNjBSLBh4xZWpVmFpp-mNQ.0oRDoQEmoQRBIVhP2dn3hQlQUEtYOTc2SHeA-B8ddA-pqkUaAQEAAVgg67pvCUJ0GoE1-tmRSw9c_f2VJLjDhqmy4n80uRCs_lJQBjLHh5B6YshlintKr_4hElhAMXmonvYw9FHDVF1fwSf7Js3Lx89KIcrN-WcYM4TsDFaTshhnMt96nM2WIiT0ia28JzqmlqR1Z7UDUApbrAJPMw"
<- OK

Parsing AT%KEYGEN output:

-----BEGIN CERTIFICATE REQUEST-----
MIIBCTCBrwIBADAvMS0wKwYDVQQDDCQ1MDRiNTgzOS0zNzM2LTQ4NzctODBmOC0x
ZjFkNzQwZmE5YWEwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARibVcjsI8WZbyY
V9WTrTzbc+6xZmkxZeVd4I4xsb+aA5RYWN2i2PwOjYTt6ewpeRsIKYBPoyfGPhGo
8j23BIJhoB4wHAYJKoZIhvcNAQkOMQ8wDTALBgNVHQ8EBAMCA+gwDAYIKoZIzj0E
AwIFAANHADBEAiACnNCAq2YAwAfd3BuWicLuDa1TTPm26ay4xUKGHcK09AIgZq4N
sK+QImFLaasOiTSnXRNjBSLBh4xZWpVmFpp+mNQ=
-----END CERTIFICATE REQUEST-----

Device public key:
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEYm1XI7CPFmW8mFfVk60823PusWZp
MWXlXeCOMbG/mgOUWFjdotj8Do2E7ensKXkbCCmAT6Mnxj4RqPI9twSCYQ==
-----END PUBLIC KEY-----

SHA256 Digest:
ebba6f0942741a8135fad9914b0f5cfdfd9524b8c386a9b2e27f34b910acfe52

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
COSE:
Prot Hdr: 1 : -7 (ECDSA w/ SHA-256)
Unprot Hdr: 4 : -2 (identity_key)
---------------
Attestation:
Payload ID: CSR_msg_v1
Dev UUID: 504b5839-3736-4877-80f8-1f1d740fa9aa
sec_tag: 16842753
SHA256: ebba6f0942741a8135fad9914b0f5cfdfd9524b8c386a9b2e27f34b910acfe52
Nonce: 0632c787907a62c8658a7b4aaffe2112
---------------
Sig:
3179a89ef630f451c3545d5fc127fb26cdcbc7cf4a21cacdf967183384ec0c5693b2186732df7a9ccd962224f489adbc273aa696a47567b503500a5bac024f33

COSE digest matches payload
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Device ID: 504b5839-3736-4877-80f8-1f1d740fa9aa
Loading CA and key...
Creating device certificate...
Writing CA cert(s) to device...
-> AT%CMNG=0,16842753,0,"-----BEGIN CERTIFICATE-----
MIIDQTCCAimgAwIBAgITBmyfz5m/jAo54vB4ikPmljZbyjANBgkqhkiG9w0BAQsF
ADA5MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRkwFwYDVQQDExBBbWF6
b24gUm9vdCBDQSAxMB4XDTE1MDUyNjAwMDAwMFoXDTM4MDExNzAwMDAwMFowOTEL
MAkGA1UEBhMCVVMxDzANBgNVBAoTBkFtYXpvbjEZMBcGA1UEAxMQQW1hem9uIFJv
b3QgQ0EgMTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ4gHHKeNXj
ca9HgFB0fW7Y14h29Jlo91ghYPl0hAEvrAIthtOgQ3pOsqTQNroBvo3bSMgHFzZM
9O6II8c+6zf1tRn4SWiw3te5djgdYZ6k/oI2peVKVuRF4fn9tBb6dNqcmzU5L/qw
IFAGbHrQgLKm+a/sRxmPUDgH3KKHOVj4utWp+UhnMJbulHheb4mjUcAwhmahRWa6
VOujw5H5SNz/0egwLX0tdHA114gk957EWW67c4cX8jJGKLhD+rcdqsq08p8kDi1L
93FcXmn/6pUCyziKrlA4b9v7LWIbxcceVOF34GfID5yHI9Y/QCB/IIDEgEw+OyQm
jgSubJrIqg0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMC
AYYwHQYDVR0OBBYEFIQYzIU07LwMlJQuCFmcx7IQTgoIMA0GCSqGSIb3DQEBCwUA
A4IBAQCY8jdaQZChGsV2USggNiMOruYou6r4lK5IpDB/G/wkjUu0yKGX9rbxenDI
U5PMCCjjmCXPI6T53iHTfIUJrU6adTrCC2qJeHZERxhlbI1Bjjt/msv0tadQ1wUs
N+gDS63pYaACbvXy8MWy7Vu33PqUXHeeE6V/Uq2V8viTO96LXFvKWlJbYK8U90vv
o/ufQJVtMVT8QtPHRh8jrdkPSHCa2XV4cdFyQzR1bldZwgJcJmApzyMZFo6IQ6XU
5MsI+yMRQ+hDKXJioaldXgjUkK642M4UwtBV8ob2xJNDd2ZhwLnoQdeXeGADbkpy
rqXRfboQnoZsG4q5WTP468SQvvG5
-----END CERTIFICATE-----
"
<- OK
Writing dev cert to device...
-> AT%CMNG=0,16842753,1,"-----BEGIN CERTIFICATE-----
MIIBNTCB3QIUHKda42NToWPdU1s8qIaXzGot+20wCgYIKoZIzj0EAwIwDTELMAkG
A1UEBhMCVVMwHhcNMjQwNTAyMDMwNzMwWhcNMzQwNDMwMDMwNzMwWjAvMS0wKwYD
VQQDDCQ1MDRiNTgzOS0zNzM2LTQ4NzctODBmOC0xZjFkNzQwZmE5YWEwWTATBgcq
hkjOPQIBBggqhkjOPQMBBwNCAARibVcjsI8WZbyYV9WTrTzbc+6xZmkxZeVd4I4x
sb+aA5RYWN2i2PwOjYTt6ewpeRsIKYBPoyfGPhGo8j23BIJhMAoGCCqGSM49BAMC
A0cAMEQCICxfFZmiZT67sBdmqAWoFLMgZ7CaeHtNWb6LoCjILN/QAiAB7txcZI8W
9nbESKFwyXvFeZIQi7nIjJ3GBzvn7X61Vg==
-----END CERTIFICATE-----
"
<- OK
Saving nRF Cloud device onboarding CSV file onboard.csv...
--- File onboard.csv exists; overwrite, append, or quit (y,a,n)? y
Onboarding CSV file saved

And then I have done this

python nrf_cloud_onboard.py --apikey f8b0026db89c7d09a29567d56b7ec30aaf1b4ca6
Devices to be onboarded: 1
Onboarding API call result: 202 - Accepted
Response: {"bulkOpsRequestId":"01HWVMK63WZB5MZKPKWVHSRHM3"}
Fetching results for bulkOpsRequestId: 01HWVMK63WZB5MZKPKWVHSRHM3
Waiting 5s...
Onboarding status: SUCCEEDED

CSV-formatted results:
bulkOpsRequestId,01HWVMK63WZB5MZKPKWVHSRHM3
status,SUCCEEDED
endpoint,PROVISION_DEVICES
requestedAt,2024-05-02T03:12:01.148Z
completedAt,2024-05-02T03:12:05.766Z
uploadedDataUrl,bulk-ops-requests.nrfcloud.com/.../01HWVMK63WZB5MZKPKWVHSRHM3.csv
Error count,0

Device ID,Result
504b5839-3736-4877-80f8-1f1d740fa9aa,OK

[Pete Skeggs]

The need for various CONFIG_HW_ID_LIBRARY_SOURCE values is not required when using CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID=y, but only when CONFIG_NRF_CLOUD_CLIENT_ID_SRC_HW_ID=y.

We will add a note to the documentation about the requirement for CONFIG_MODEM_JWT=y.

[Burt]

I'm trying to review everything in this ticket since it has been a while since I looked at it. I have found a bug in https://docs.nrfcloud.com/Devices/Security/Credentials.html : the statement "For details, see the Device Credentials Installer documentation." in the second note points to the python script rather than the documentation. It should be https://github.com/nRFCloud/utils/blob/master/python/modem-firmware-1.3+/README.md#device-credentials-installer.

Burt

[Burt]

Hi Simen, thank you for the opportunity to improve the code, but the short answer to your question is "No!" I am chuckling, because it sounds so rude for me to say that so bluntly--even more rude would be to use all caps or even say "NO WAY." The longer answer: Looking again at the issue, I don't like the design in which the choice HW_ID_LIBRARY_SOURCE and the choice NRF_CLOUD_CLIENT_ID_SRC are set up as independent entities in the Kconfig files, but in most cases HW_ID_LIBRARY_SOURCE depends on NRF_CLOUD_CLIENT_ID_SRC or else one is not using NRF_CLOUD and it depends on something different. To do things really nicely, one has to consider every case. If I tried and missed a case, I'd look like a jackass. And even if I didn't miss any case, a restructuring of configuration might not be acceptable when existing configurations required changes. I like to point out that throughout my career I have had a knack for noticing these types of design issues. "These types" translates to "things that have to be fixed globally." One time I was "the guy in charge" and I was able to make some good build system changes. But I had to block a lot of people from checking in code to our ancient, locking source control system for 3 days. So even when I was able to make the changes, I caught a lot of flak for those 3 days. Fortunately, I was able to speed up the build system so people changed their tune.

If it is not practical to make big changes, I recommend that a documentation change be made to nRF Cloud (nordicsemi.com). There are a few places in "Configuration options for device ID" that should explicitly mention the corresponding required HW_ID_LIBRARY_SOURCE option.

Just for fun, here is an example of what might be nice: 1) I take the options HW_ID_LIBRARY_SOURCE_IMEI and HW_ID_LIBRARY_SOURCE_UUID out of the HW_ID_LIBRARY_SOURCE choice. That allows me to do something like:

config NRF_CLOUD_CLIENT_ID_SRC_IMEI
depends on NRF_MODEM_LIB
# depends on HW_ID_LIBRARY_SOURCE_IMEI
select HW_ID_LIBRARY_SOURCE_IMEI
bool "IMEI based value"

and

config NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID
bool "Modem internal UUID value"
depends on MODEM_JWT
select HW_ID_LIBRARY_SOURCE_UUID
help
Requires modem firmware version 1.3 or later.

Burt

[Burt]

I still have a question about below, and the question is how I make use of a custom prefix for a custom board if when I try to add the custom board to nRF Cloud I will be asked for the PIN or HWID. How can my custom board have a HWID when it has no nRF52840 or anything that looks like a MAC address?

• CONFIG_NRF_CLOUD_CLIENT_ID_SRC_IMEI - If you enable this option, the ID is automatically generated using a prefix and the modem’s IMEI (<prefix><IMEI>). This is the default. You can configure the prefix by using CONFIG_NRF_CLOUD_CLIENT_ID_PREFIX. The default format of the prefix is nrf- and it is valid only for Nordic devices such as Thingy:91 or an nRF91 Series DK. For custom hardware, use a prefix other than nrf- by modifying CONFIG_NRF_CLOUD_CLIENT_ID_PREFIX.

[Burt]

Yeah, they make zero sense, that crazy menu brought up by Add Devices. The only way I can add my bms-350457794634418 board (where I am using bms as a prefix) is to use the nrf_cloud_onboard.py script. So, what the heck is the purpose of the useless Add Devices menu? Once I use nrf_cloud_onboard.py, there is no need to use Add Devices. Prior to using nrf_cloud_onboard.py, Add Devices comes up with a 40142 error. It's crazy!! I'm tempted to think that Add Devices is only for Nordic DK and Thingy, but that's not what it claims. And that Add Devices button is in my face when I go to the Devices page (that lists current devices).

Burt