mqtt_simple connect to AWS

[Timothy]

I have an AWS account and I am comfortable with the AWS IOT operation in general. I have used it in the past. I am trying to get the sample mqtt_simple to connect to AWS IOT and be able to send packets and receive packets to the nRF9160dk board.

 I am not sure how to set up mqtt_simple parameters? I have created a thing and certificates. I have named the thing "nrf-IMEI" as recommended so it has the board ID in it. All I want to do is to be able to publish a packet to a topic on my AWS account and receive a packet from a topic in my AWS account. I have done this several times in the past with other systems so I am comfortable with AWS IOT operations and formats. I just do not understand how to get mqtt_simple to publish from my board to AWS and subscribe to messages from AWS.

Is this the correct program to use. It seems like all the pieces are there I just do not know how to use them.

I appreciate any help you can give, This is an essential requirement of our production system. right now we are using the pre-production system with a modified version of asset_tracker to send sensor data to the nRF Connect site. We need to be able to send sensor data to our AWS site.

[Timothy]

Martin

The problem of not connecting was the SIM cards were full on both boards. I am loading and trying things a hundred times a day and I forgot to check the SIM cards. That still lives the original problem that it connects to AWS and crashes as soon as I do the first publish. I still need any insight you have into solving that  problem.

[Martin Lesund]

Hi Timothy,
The issue is most likely that cJSON_Init() never gets called.
If it's not called malloc/free from newlib will then be used, and this causes issues since malloc/free gets mixed up with k_malloc/k_free.

So if you call cJSON_init() someplace in your code it should work.

[Timothy]

Martin

yes that fixed the problem. I am able to publish to my AWS account and send messages from my AWS account.

AWESOME!!! thank you.

[Martin Lesund]

Hi Timothy,
No problem, I am happy to help.
Glad to hear that you got it up and running,

[Timothy]

Martin

I am back again. I loaded V1.3.0 and I am trying to get the AWS connection to my account working.

I made all of the changes (I think) we talked about above and I have mqtt_simple working for V1.3.0. when I make the same changes for asset_tracker I cannot connect to my account. I get the following messages. As before it is crucial that I connect asset_tracker to my AWS account so I can send data to my tables. I am sure I missed something or something changed in V1.3.0. any help you could give would be greatly appreciated. I am dead in the water right now.

[[00:00:00.237,579] <inf> asset_tracker: Connecting to LTE network.
[00:00:00.244,232] <inf> asset_tracker: This may take several minutes.
[00:00:00.251,312] <dbg> lte_lc.lte_lc_system_mode_set: Sending AT command to set system mode: AT%XSYSTEMMODE=1,0,1,0
[00:00:00.270,599] <dbg> lte_lc.lte_lc_system_mode_set: Sending AT command to set system mode: AT%XSYSTEMMODE=1,0,1,0
+CEREG: 2,"9E03","026B8102",7,0,0,"11100000","11100000"
[00:00:02.253,540] <dbg> lte_lc.at_handler: +CEREG notification: +CEREG: 2,"9E03","026B8102",7,0,0,"11100000","11100000"

[00:00:02.265,319] <dbg> lte_lc.parse_psm_cfg: TAU: -1 sec, active time: -1 sec

+CSCON: 1
[00:00:02.305,145] <dbg> lte_lc.at_handler: +CSCON notification
+CEREG: 5,"9E03","026B8102",7,,,"11100000","11100000"
[00:00:05.878,753] <dbg> lte_lc.at_handler: +CEREG notification: +CEREG: 5,"9E03","026B8102",7,,,"11100000","11100000"

[00:00:05.890,350] <dbg> lte_lc.parse_psm_cfg: TAU: -1 sec, active time: -1 sec

[00:00:05.898,345] <inf> asset_tracker: Connected to LTE network.
[00:00:05.904,937] <inf> asset_tracker: Connecting to cloud, attempt 1 of 8
[00:00:10.265,930] <inf> asset_tracker: Cloud connection request sent.
[00:00:10.272,918] <inf> asset_tracker: Connection response timeout is set to 30 seconds.
+CSCON: 0
[00:00:17.348,327] <dbg> lte_lc.at_handler: +CSCON notification
[00:00:40.281,646] <err> asset_tracker: LTE link disconnect
+CSCON: 1
[00:00:40.404,907] <dbg> lte_lc.at_handler: +CSCON notification
+CEREG: 0,"9E03","026B8102",7,0,0,"11100000","11100000"
[00:00:40.416,503] <dbg> lte_lc.at_handler: +CEREG notification: +CEREG: 0,"9E03","026B8102",7,0,0,"11100000","11100000"

[00:00:40.428,253] <dbg> lte_lc.parse_psm_cfg: TAU: -1 sec, active time: -1 sec

+CSCON: 0
[00:00:41.808,380] <dbg> lte_lc.at_handler: +CSCON notification
[00:00:42.136,688] <err> asset_tracker: Shutdown modem

[Martin Lesund]

Hi Timothy,
Have you provisioned the correct certificates to the modem?

So the asset_tracker from v1.2.0 you were able to connect to your server, but not the NCS v1.3.0 version?

Could you maybe share the Modem trace ?

[Timothy]

Martin

yes it is the same board with the same certificates. it connects and runs fine with V1.2.0 but does not with V1.3.0. I think I made all of the settings in kconfig and prj.conf that I did in V1.2.0 to get ti to connect.

I am very confused. I must be missing something simple.

[Martin Lesund]

Please first try to enable more logging from the application:

Add in prj.conf:

ASSET_TRACKER_LOG_LEVEL_DBG=y

What we could do is to make this case private so I could look more into your configurations in your application.

[Timothy]

Martin

I added the line but not much more was shown. we can make private or do whatever else is needed.

I could send you the directory zipped up but you would need the  certificates to connect to my board. I am willing to try whatever you suggest

[00:00:00.238,250] <inf> asset_tracker: Connecting to LTE network.
[00:00:00.244,903] <inf> asset_tracker: This may take several minutes.
[00:00:00.251,953] <dbg> lte_lc.lte_lc_system_mode_set: Sending AT command to set system mode: AT%XSYSTEMMODE=1,0,1,0
[00:00:00.271,453] <dbg> lte_lc.lte_lc_system_mode_set: Sending AT command to set system mode: AT%XSYSTEMMODE=1,0,1,0
+CEREG: 2,"9E03","026B8102",7,0,0,"11100000","11100000"
[00:00:02.561,798] <dbg> lte_lc.at_handler: +CEREG notification: +CEREG: 2,"9E03","026B8102",7,0,0,"11100000","11100000"

[00:00:02.573,577] <dbg> lte_lc.parse_psm_cfg: TAU: -1 sec, active time: -1 sec

+CSCON: 1
[00:00:02.611,602] <dbg> lte_lc.at_handler: +CSCON notification
+CEREG: 5,"9E03","026B8102",7,,,"11100000","11100000"
[00:00:04.247,039] <dbg> lte_lc.at_handler: +CEREG notification: +CEREG: 5,"9E03","026B8102",7,,,"11100000","11100000"

[00:00:04.258,636] <dbg> lte_lc.parse_psm_cfg: TAU: -1 sec, active time: -1 sec

[00:00:04.266,601] <inf> asset_tracker: Connected to LTE network.
[00:00:04.273,193] <inf> asset_tracker: Connecting to cloud, attempt 1 of 8
client_id_buf nrf-352656100378150
CONFIG_AWS_IOT_BROKER_HOST_NAME a2uy3w9dvx40gv-ats.iot.us-east-1.amazonaws.com
client_broker_init passed
[00:00:09.697,937] <inf> asset_tracker: Cloud connection request sent.
[00:00:09.704,925] <inf> asset_tracker: Connection response timeout is set to 30 seconds.
[00:00:09.713,592] <dbg> watchdog.secondary_feed_worker: Feeding watchdog
[00:00:14.720,916] <dbg> watchdog.secondary_feed_worker: Feeding watchdog
+CSCON: 0
[00:00:17.323,913] <dbg> lte_lc.at_handler: +CSCON notification
[00:00:19.728,271] <dbg> watchdog.secondary_feed_worker: Feeding watchdog
[00:00:24.735,656] <dbg> watchdog.secondary_feed_worker: Feeding watchdog
[00:00:29.743,072] <dbg> watchdog.secondary_feed_worker: Feeding watchdog
[00:00:34.750,457] <dbg> watchdog.secondary_feed_worker: Feeding watchdog
[00:00:39.713,623] <err> asset_tracker: LTE link disconnect
+CSCON: 1
[00:00:39.833,801] <dbg> lte_lc.at_handler: +CSCON notification
+CEREG: 0,"9E03","026B8102",7,0,0,"11100000","11100000"
[00:00:39.845,367] <dbg> lte_lc.at_handler: +CEREG notification: +CEREG: 0,"9E03","026B8102",7,0,0,"11100000","11100000"

[00:00:39.857,147] <dbg> lte_lc.parse_psm_cfg: TAU: -1 sec, active time: -1 sec

[Timothy]

Martin

I added some print statements to help see what is going on. see below that everything is passing (client_broker_init -> mqtt_connect -> aws_iot -> cloud_connect).  the client structure returned from client_broker_init is identical to the one returned in mqtt_simple I am pretty sure. I printed out some of the same information in both. it looks like mqtt is never connecting and calling the call back handler. it is the same board with the same credentials and security tag.

could it be some other configuration is different that needs to be set or changed? the mqtt_simple has a very simple client_init interface while the asset_tracker now has several layers and uses aws_iot.c functions. I am at a loss and can not see what I am missing. any ideas or suggestions would be helpful.

if you have a working example of asset_tracker for V1.3.0 connecting to aws that would help also. I am sure it is something very simple I am missing.

[00:00:03.533,050] <inf> asset_tracker: Connecting to cloud, attempt 1 of 8
[00:00:03.742,462] <dbg> aws_iot.broker_init: IPv4 Address found 3.221.33.115
client_id_buf nrf-352656100378150
CONFIG_AWS_IOT_BROKER_HOST_NAME a2uy3w9dvx40gv-ats.iot.us-east-1.amazonaws.com
client_broker_init passed
mqtt_connect passed
aws_iot_connect 0
cloud_connect 0
[00:00:08.274,841] <inf> asset_tracker: Cloud connection request sent.
[00:00:08.281,829] <inf> asset_tracker: Connection response timeout is set to 30 seconds.

[Jan Tore Guggedal]

Hi Timothy, 

I think the issue is that there's nothing actually reading incoming data to the cloud socket. For the asset tracker, the assumption is that it's done by the cloud backend. That is the case for nRF Cloud backend, but not for AWS IoT backend. Support for polling the AWS IoT socket in the backend is coming in this PR: https://github.com/nrfconnect/sdk-nrf/pull/2351/commits

For now you can cherry-pick the commit introducing this addition from that PR and enable it with Kconfig.

Another option is to solve this in the asset tracker like it's done in the cloud client sample: https://github.com/nrfconnect/sdk-nrf/blob/3710213bc65cb64fbb59fb07ae84285ab1c2755b/samples/nrf9160/cloud_client/src/main.c#L221

Best regards, 

Jan Tore 

[Timothy]

Jan

thanks you very much. I added the POLL loop like was in the V1.2.0 asset_tracker from the cloud_client example.

I can now connect to mqtt but I see two problems as shown below. first is I get AWS shadow topics subscribe error. I am not sure what topics I am supposed to configure to subscribe to. I just selected the first few. the second is that the program crashes when it gets fota messages. I think I ma very close now. it is connecting and subscribing but some scribe errors and a crash with fota message.

any ideas on what is going on now?

[00:00:08.087,371] <inf> asset_tracker: Cloud connection request sent.
[00:00:08.094,360] <inf> asset_tracker: Connection response timeout is set to 30 seconds.
[00:00:08.103,027] <dbg> watchdog.secondary_feed_worker: Feeding watchdog
[00:00:13.110,382] <dbg> watchdog.secondary_feed_worker: Feeding watchdog
mqtt_evt_handler called 0
[00:00:15.937,072] <inf> aws_jobs: Subscribe: $aws/things/nrf-352656100378150/jobs/notify-next
+CSCON: 0
[00:00:15.947,387] <dbg> lte_lc.at_handler: +CSCON notification
[00:00:15.954,071] <inf> aws_jobs: Subscribe: $aws/things/nrf-352656100378150/jobs/$next/get/#
[00:00:15.964,172] <dbg> aws_iot.mqtt_evt_handler: MQTT client connected!
[00:00:15.971,496] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/get/accepted
[00:00:15.984,039] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/get/rejected
[00:00:15.996,643] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/update/accepted
[00:00:16.009,429] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/update/rejected
[00:00:16.022,216] <dbg> aws_iot.topic_subscribe: Subscribing to AWS shadow topic: $aws/things/nrf-352656100378150/shadow/update/delta
[00:00:16.034,820] <err> aws_iot: AWS shadow topics subscribe, error: -12
[00:00:16.042,358] <inf> asset_tracker: CLOUD_EVT_CONNECTED
[00:00:16.048,400] <inf> asset_tracker: Persistent Sessions = 0
[00:00:16.054,779] <inf> asset_tracker: CLOUD_EVT_READY
+CSCON: 1
[00:00:16.061,950] <dbg> lte_lc.at_handler: +CSCON notification
[00:00:16.068,725] <dbg> nrf9160_gps.init: GPS socket created, fd: 1232491587
[00:00[00:00:16.070,251] <dbg> aws_iot.aws_iot_send: Publishing to topic: $aws/things/nrf-352656100378150/shadow/update
:16.077,178] <inf> gps_control: GPS initialized
mqtt_evt_handler called 7
[00:00:16.586,517] <inf> aws_fota: subscribed to notify-next topic
[00:00:16.593,231] <inf> aws_jobs: Publish topic: $aws/things/nrf-352656100378150/jobs/$next/get
[00:00:16.602,478] <inf> aws_jobs: Publish payload {"clientToken": ""}
mqtt_evt_handler called 7
[00:00:16.750,946] <dbg> aws_iot.mqtt_evt_handler: MQTT_EVT_SUBACK: id = 2114 result = 0
mqtt_evt_handler called 3
[00:00:16.967,163] <dbg> aws_iot.mqtt_evt_handler: MQTT_EVT_PUBACK: id = 19972 result = 0
mqtt_evt_handler called 2
[00:00:17.012,084] <dbg> aws_fota.on_publish_evt: Received topic: $aws/things/nrf-352656100378150/jobs/$next/get/accepted
[00:00:17.023,498] <inf> aws_fota: Checking for an available job
[00:00:17.030,059] <dbg> aws_fota.get_job_execution: Job doc: {"clientToken":"","timestamp":1593124752}
[00:00:17.040,191] <dbg> aws_fota.get_job_execution: Got only one field
[00:00:17.047,271] <inf> aws_fota: No queued jobs for this device
*** Booting Zephyr OS build v2.3.0-rc1-ncs1 ***