mqtt_simple connect to AWS

[Timothy]

I have an AWS account and I am comfortable with the AWS IOT operation in general. I have used it in the past. I am trying to get the sample mqtt_simple to connect to AWS IOT and be able to send packets and receive packets to the nRF9160dk board.

 I am not sure how to set up mqtt_simple parameters? I have created a thing and certificates. I have named the thing "nrf-IMEI" as recommended so it has the board ID in it. All I want to do is to be able to publish a packet to a topic on my AWS account and receive a packet from a topic in my AWS account. I have done this several times in the past with other systems so I am comfortable with AWS IOT operations and formats. I just do not understand how to get mqtt_simple to publish from my board to AWS and subscribe to messages from AWS.

Is this the correct program to use. It seems like all the pieces are there I just do not know how to use them.

I appreciate any help you can give, This is an essential requirement of our production system. right now we are using the pre-production system with a modified version of asset_tracker to send sensor data to the nRF Connect site. We need to be able to send sensor data to our AWS site.

[Timothy]

Martin

The problem of not connecting was the SIM cards were full on both boards. I am loading and trying things a hundred times a day and I forgot to check the SIM cards. That still lives the original problem that it connects to AWS and crashes as soon as I do the first publish. I still need any insight you have into solving that  problem.

[Martin Lesund]

Hi Timothy,
The issue is most likely that cJSON_Init() never gets called.
If it's not called malloc/free from newlib will then be used, and this causes issues since malloc/free gets mixed up with k_malloc/k_free.

So if you call cJSON_init() someplace in your code it should work.

[Jan Tore Guggedal]

Hi Timothy, 

I think the issue is that there's nothing actually reading incoming data to the cloud socket. For the asset tracker, the assumption is that it's done by the cloud backend. That is the case for nRF Cloud backend, but not for AWS IoT backend. Support for polling the AWS IoT socket in the backend is coming in this PR: https://github.com/nrfconnect/sdk-nrf/pull/2351/commits

For now you can cherry-pick the commit introducing this addition from that PR and enable it with Kconfig.

Another option is to solve this in the asset tracker like it's done in the cloud client sample: https://github.com/nrfconnect/sdk-nrf/blob/3710213bc65cb64fbb59fb07ae84285ab1c2755b/samples/nrf9160/cloud_client/src/main.c#L221

Best regards, 

Jan Tore 

[Timothy]

Jan

thanks you very much. I added the POLL loop like was in the V1.2.0 asset_tracker from the cloud_client example.

I can now connect to mqtt but I see two problems as shown below. first is I get AWS shadow topics subscribe error. I am not sure what topics I am supposed to configure to subscribe to. I just selected the first few. the second is that the program crashes when it gets fota messages. I think I ma very close now. it is connecting and subscribing but some scribe errors and a crash with fota message.

any ideas on what is going on now?

[00:00:08.087,371] <inf> asset_tracker: Cloud connection request sent.
[00:00:08.094,360] <inf> asset_tracker: Connection response timeout is set to 30 seconds.
[00:00:08.103,027] <dbg> watchdog.secondary_feed_worker: Feeding watchdog
[00:00:13.110,382] <dbg> watchdog.secondary_feed_worker: Feeding watchdog
mqtt_evt_handler called 0
[00:00:15.937,072] <inf> aws_jobs: Subscribe: $aws/things/nrf-352656100378150/jobs/notify-next
+CSCON: 0
[00:00:15.947,387] <dbg> lte_lc.at_handler: +CSCON notification
[00:00:15.954,071] <inf> aws_jobs: Subscribe: $aws/things/nrf-352656100378150/jobs/$next/get/#
[00:00:15.964,172] <dbg> aws_iot.mqtt_evt_handler: MQTT client connected!
[00:00:15.971,496] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/get/accepted
[00:00:15.984,039] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/get/rejected
[00:00:15.996,643] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/update/accepted
[00:00:16.009,429] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/update/rejected
[00:00:16.022,216] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/update/delta
[00:00:16.034,820] <err> aws_iot: AWS shadow topics subscribe, error: -12
[00:00:16.042,358] <inf> asset_tracker: CLOUD_EVT_CONNECTED
[00:00:16.048,400] <inf> asset_tracker: Persistent Sessions = 0
[00:00:16.054,779] <inf> asset_tracker: CLOUD_EVT_READY
+CSCON: 1
[00:00:16.061,950] <dbg> lte_lc.at_handler: +CSCON notification
[00:00:16.068,725] <dbg> nrf9160_gps.init: GPS socket created, fd: 1232491587
[00:00[00:00:16.070,251] <dbg> aws_iot.aws_iot_send: Publishing to topic: $aws/things/nrf-352656100378150/shadow/update
:16.077,178] <inf> gps_control: GPS initialized
mqtt_evt_handler called 7
[00:00:16.586,517] <inf> aws_fota: subscribed to notify-next topic
[00:00:16.593,231] <inf> aws_jobs: Publish topic: $aws/things/nrf-352656100378150/jobs/$next/get
[00:00:16.602,478] <inf> aws_jobs: Publish payload {"clientToken": ""}
mqtt_evt_handler called 7
[00:00:16.750,946] <dbg> aws_iot.mqtt_evt_handler: MQTT_EVT_SUBACK: id = 2114 result = 0
mqtt_evt_handler called 3
[00:00:16.967,163] <dbg> aws_iot.mqtt_evt_handler: MQTT_EVT_PUBACK: id = 19972 result = 0
mqtt_evt_handler called 2
[00:00:17.012,084] <dbg> aws_fota.on_publish_evt: Received topic: $aws/things/nrf-352656100378150/jobs/$next/get/accepted
[00:00:17.023,498] <inf> aws_fota: Checking for an available job
[00:00:17.030,059] <dbg> aws_fota.get_job_execution: Job doc: {"clientToken":"","timestamp":1593124752}
[00:00:17.040,191] <dbg> aws_fota.get_job_execution: Got only one field
[00:00:17.047,271] <inf> aws_fota: No queued jobs for this device
*** Booting Zephyr OS build v2.3.0-rc1-ncs1 ***

[Timothy]

Jan

I got rid of the subscribe error by only subscribing to the first 4 AWS topics as shown below.

I also am receiving only the flip data on my AWS account as shown below.

{
  "appId": "FLIP",
  "data": "NORMAL",
  "messageType": "DATA"
}

I am not receiving the device info or sensor information packets. I am sure that 
is because it crashed before or during the transmission of those JSON packets.
it crashes at the same place each time.

[00:00:16.418,701] <dbg> aws_fota.get_job_execution: Job doc: {"clientToken":"","timestamp":1593126964}
[00:00:16.428,833] <dbg> aws_fota.get_job_execution: Got only one field
[00:00:16.435,913] <inf> aws_fota: No queued jobs for this device

any ideas where I can look to fix this. maybe this is all I need and I will be 
working again with V1.3.0. I am sure I am very close. I would be still banging my
head without your help. I had no idea the POLL loop was needed.

mqtt_evt_handler called 0
[00:00:14.879,913] <inf> aws_jobs: Subscribe: $aws/things/nrf-352656100378150/jobs/notify-next
+CSCON: 0
[00:00:14.890,228] <dbg> lte_lc.at_handler: +CSCON notification
[00:00:14.896,881] <inf> aws_jobs: Subscribe: $aws/things/nrf-352656100378150/jobs/$next/get/#
[00:00:14.906,768] <dbg> aws_iot.mqtt_evt_handler: MQTT client connected!
[00:00:14.914,062] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/get/accepted
[00:00:14.926,605] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/get/rejected
[00:00:14.939,147] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/update/accepted
[00:00:14.951,995] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/update/rejected
+CSCON: 1
[00:00:14.966,094] <dbg> lte_lc.at_handler: +CSCON notification
[00:00:14.972,656] <inf> asset_tracker: CLOUD_EVT_CONNECTED
[00:00:14.978,698] <inf> asset_tracker: Persistent Sessions = 0
[00:00:14.985,107] <inf> asset_tracker: CLOUD_EVT_READY
[00:00:14.991,668] <dbg> nrf9160_gps.init: GPS socket created, fd: 1232491587
[00:00:15.000,152] <inf> gps_control: GPS initialized
[00:00:14.993,713] <dbg> aws_iot.aws_iot_send: Publishing to topic: $aws/things/nrf-352656100378150/shadow/update
mqtt_evt_handler called 7
[00:00:15.698,669] <inf> aws_fota: subscribed to notify-next topic
[00:00:15.705,413] <inf> aws_jobs: Publish topic: $aws/things/nrf-352656100378150/jobs/$next/get
[00:00:15.714,660] <inf> aws_jobs: Publish payload {"clientToken": ""}
mqtt_evt_handler called 7
[00:00:16.025,726] <dbg> aws_iot.mqtt_evt_handler: MQTT_EVT_SUBACK: id = 2114 result = 0
mqtt_evt_handler called 7
[00:00:16.072,052] <dbg> aws_iot.mqtt_evt_handler: MQTT_EVT_SUBACK: id = 29952 result = 0
mqtt_evt_handler called 3
[00:00:16.389,495] <dbg> aws_iot.mqtt_evt_handler: MQTT_EVT_PUBACK: id = 56415 result = 0
mqtt_evt_handler called 2
[00:00:16.400,756] <dbg> aws_fota.on_publish_evt: Received topic: $aws/things/nrf-352656100378150/jobs/$next/get/accepted
[00:00:16.412,170] <inf> aws_fota: Checking for an available job
[00:00:16.418,701] <dbg> aws_fota.get_job_execution: Job doc: {"clientToken":"","timestamp":1593126964}
[00:00:16.428,833] <dbg> aws_fota.get_job_execution: Got only one field
[00:00:16.435,913] <inf> aws_fota: No queued jobs for this device
*** Booting Zephyr OS build v2.3.0-rc1-ncs1 ***

[Yusuke]

Hi Jan

Let me confirm. In ncs v1.3.0 you are supposed to use asset_tracker connecting nRF cloud and AWS IoT is not supported yet right?

[Timothy]

Jan

any update on this? is it supported and if so do you understand why I am crashing? If not can you tell me when it is supported and I will continue to work with V1.2.0 until it is supported.

[Jan Tore Guggedal]

Hi, 

Yusuke, only nRF Cloud backend is officially supported and tested in NCS releases. I'm not aware of timeline for official AWS IoT support. I would expect it to work, however. 

Timothy, it looks like the client token object in received jobs message is empty. You could use debugger to check if that's the reason for crash.

I'm out of office the next few weeks, so I will unfortunately not be able to follow up much at this time. Hopefully Martin can pick it up or forward it to someone who can. If not, I will pick it up when I'm back. 

Best regards, 

Jan Tore 

[Timothy]

Can anyone tell where to look for the client token object in received jobs. How do I tell if jobs are empty and if so why are they empty? is that fatal? If they are empty who is supposed to put data in them? I appreciate any help you can give. I am sure it is something simple I have not added or set correctly. It does send the Flip data JSON packet so a lot has to be working for that to go to my AWS account.

[Timothy]

There is another problem with the V1.2.0 asset_tracker for AWS that is a show stopper. The Asset_tracker connects to AWS and sends the FLIP, device info, RSRP, and status information to our AWS site, This runs without a problem forever on any single board. We have seen no problem running for days at a time. I recently got a second board working and a colleague got a board working. We are all running the same code. When any one of the boards starts up and connects to AWS and tries to publish the other boards see a POLLHUP which resets the boards. These boards will then in turn reach the same connection point and reset the other boards. It is a death spiral.

How can two or more AWS connections be able to disconnect the MQTT connection on different boards no matter if they are located in different states. The only common denominator is the AWS account. All boards are connecting to the same AWS account which is a requirement since lambda functions on that account are putting all the data from the various boards into dynamodb tables.  any thing I could do to fix this problem would be appreciated.

[Timothy]

I forgot to mention that my two boards have two different sets of certificates.