cloud_send fail AWS_IOT with nRF9160

[Farhang]

Hi,

I'm trying to convert the NCS - cloud client sample to connect to AWS_IOT rather than NRF cloud.

After a connection achieved, i press Btn1 and I get the following error:

*** Booting Zephyr OS build v2.1.99-ncs1-8-g367eec2dd60d ***
Cloud client has started
Connecting to LTE network. This may take several minutes.
Connected to LTE network
Publishing message: {"state":{"reported":{"message":"Hello Internet of Things!"}}}
cloud_send failed, error: -128
E: mqtt_connect, error: -116
cloud_connect, error: -10
Publishing message: {"state":{"reported":{"message":"Hello Internet of Things!"}}}
cloud_send failed, error: -128

....

from nrf_errno.h

#define NRF_EKEYREVOKED     128 ///< Key has been revoked

Sometimes the errors

E: mqtt_connect, error: -116
cloud_connect, error: -10

show up first, so i think those are the root problem, however, there's no error definition for 116! I do not know what that error means, can't find the number 116 anywhere in the project source.

Any idea what these errors mean? and where to find documentation?

My guess is credentials haven't been programmed properly into the modem.

Here are the steps I took to change the creds over to AWS_IOT:

Once an AWS "thing" is created, it gives 1x pem + 1x public and 1x private key, and I activate that on AWS IOT Core.

On the nRF Connect app -> LTE-LINK Monitor -> certificate manager there are a number of fields.

My best guess (because this page doesn't exactly say what cert/key goes into what field in Cert Manager) was copy pasting the contents of the AWS certs as follows into nRF Cert manager

pem file into "CA certificate"

public key into "Client Certificate"

private key into "Private key" 

I gave that a random security tag number and I changed the aws_io/KConfig to reflect the AWS_IOT_SEC_TAG.

Also I changed the AWS_IOT_BROKER_HOST_NAME to something like:

xyzabcefgblah-xxx.iot.us-west-2.amazonaws.com

Also changed AWS_IOT_CLIENT_ID_STATIC to the name I gave my "thing" on AWS IOT CORE.

Question:  Is this parameter Amazon's thing name (arbitrary chosen name by me) or Thing ARN (Amazon Resource Name), which is a way longer string with the name i chose at the end, e.g. : arn:aws:iot:us-west-1:11111111111111:thing/nRF9160DK-1? (nRF9160DK-1  is the name i chose earlier)

I have tried both name formats and it doesn't change the error.

There are no "Activity" on my AWS Core thing.

Anyone has any troubleshooting ideas? 

Much appreciated thanks.

[Heidi-Irene Harvey Sollie]

Hi!

So when looking up error codes from NCS, you need to know if the sample is using Newlib or not, which you can see from the prj.conf file. The Cloud Client sample that you are using, uses Newlib, because the NEWLIB_LIBC configuration is enabled here.

When this is the case, you need to refer to the newlib error codes found here.

This is different from the default minimal C implementation which uses error codes found here. 

The error file you referred to (nrf_errno.h) is used directly by bsdlib but is then translated to one of the two files I linked to above when returned from the application. So only refer to the nrf_errno.h file if you're debugging and seeing returning error codes from directly inside bsdlib.

When that's said, it looks like cloud_send is returning ENOTCONN 128 /*Socket is not connected*/, mqtt_connect is returning ETIMEDOUT 116 */Connection timed out*/ and cloud_connect is returning ECHILD 10 */No children*/.

I think you are correct in your analysis that the problem is your credentials when trying to connect to AWS. 

 

Also I changed the AWS_IOT_BROKER_HOST_NAME to something like:

xyzabcefgblah-xxx.iot.us-west-2.amazonaws.com

"1. In the AWS IoT console, navigate to IoT core -> Manage -> things and click on the entry for the thing, created during the steps of Creating a thing in AWS IoT.
2. Navigate to interact, find the Rest API Endpoint and set the configurable option CONFIG_AWS_IOT_BROKER_HOST_NAME to this address."

Is that address the Rest API Endpoint?

 

I gave that a random security tag number and I changed the aws_io/KConfig to reflect the AWS_IOT_SEC_TAG.

 Not sure what you meant by this. You don't have to change anything in the KConfig file located in \ncs\nrf\subsys\net\lib\aws_iot\KConfig. Set the AWS_IOT_SEC_TAG in the prj.conf file of the application you are running and remember to use the same number when flashing the certificates to the modem via LTE Link Monitor. Just make sure not to use the same security tag as the nRF Cloud certificates use: 16842753. 

Have you seen this documentation on how to create a thing in AWS IoT? Step 9 describes which certificate you need for which category. 

"You need the thing certificate (*-certificate.pem.crt), the private key (*.private.pem.key), and the root CA (choose the Amazon Root CA 1, AmazonRootCA1.pem)."

The Thing certificate is the client certificate and the root CA is the CA certificate. 

Also changed AWS_IOT_CLIENT_ID_STATIC to the name I gave my "thing" on AWS IOT CORE.

 See this documentation, specifically, step 3 where it asks you to set the AWS_IOT_CLIENTID_STATIC config to the name of the thing you created in the AWS IoT Console. 

Sorry about the messy format. I hope I answered all of your questions. Let me know if this works for you!

Best regards,

Heidi

[Farhang]

Heidi,

This was extremely helpful thank you. I am now one step closer to getting some data into AWS.

The error was in cert programming, I was not using AmazonCA1. So thanks for clarifying that.

Now my device does connect to AWS IoT server, ping is OK, however in the cloud_client app after connect and ping, the polls flags are checked. No incoming data (i wasn't expecting any either) but the POLLHUP flag is set. I changed the code to try and connect again if POLLHUP was set. it does that for a couple of times until it hangs at mqtt_connect and doesn't return.

From AWS IoT Core Monitor i can tell my device successfully connected.

Perhaps the problem now is that my device is connecting but there's no outgoing data so AWS hangs up? Or do you think it could still be authentication related?

I wish there was a more comprehensive demo/tutorial on this cloud_client with AWS IOT. All I want to do is to demo that i can get some data from nRF160 into amazon.. a table or dynamoDB.

Tomorrow i will focus on creating the DynamoDB and creating a rule so MQTT messages with a known topic are routed to the DB. And will change the example message to something with a proper topic.

RE your comment about Kconfig, i have checked out the latest commit on the dev branch of NCS and things my be different than the latest release. AWS_IOT_SEC_TAG is definitely in Kconfig not prj.conf now, and so are so many other settings the documentation calls for.

I am writing this past midnight and from my phone. Forgot to respond earlier.. so sorry about the scattered thoughts, I will be more precise in my next response.

[Farhang]

The other thing I'm wondering about is whether or not i have to remove footer and header from.private key/certs before copy pasting into Link monitor-> cert manager?

It did not complain when i copy pasted entire contents of each file into each corresponding field and updated my modem.

If the cert manager is not meant to work with an entire cert/key file contents and just the body, it will mess things up.

Thoughts?

[Heidi-Irene Harvey Sollie]

Hi! I'm glad you were able to connect. 

Farhang said:

RE your comment about Kconfig, i have checked out the latest commit on the dev branch of NCS and things my be different than the latest release. AWS_IOT_SEC_TAG is definitely in Kconfig not prj.conf now, and so are so many other settings the documentation calls for.

 Yes, configurations are always defined in the Kconfig files, but it is the convention that when setting or changing them from their default value you add them to the prj.conf file of your application like this, for example:

AWS_IOT_SEC_TAG=45

The reason to do it like that is so you can have different values for this config for different applications using it.

If you're getting POLLHUP, could you take a modem trace for me so I can take a closer look? This tutorial shows you how to take a modem trace. And remember to include the log output from when you took the trace.

Best regards,

Heidi

[Farhang]

I sent you the requested traces in a DM. Let me know if there are any missing peices. Thank you!