mqtt_simple connect to AWS

[Timothy]

I have an AWS account and I am comfortable with the AWS IOT operation in general. I have used it in the past. I am trying to get the sample mqtt_simple to connect to AWS IOT and be able to send packets and receive packets to the nRF9160dk board.

 I am not sure how to set up mqtt_simple parameters? I have created a thing and certificates. I have named the thing "nrf-IMEI" as recommended so it has the board ID in it. All I want to do is to be able to publish a packet to a topic on my AWS account and receive a packet from a topic in my AWS account. I have done this several times in the past with other systems so I am comfortable with AWS IOT operations and formats. I just do not understand how to get mqtt_simple to publish from my board to AWS and subscribe to messages from AWS.

Is this the correct program to use. It seems like all the pieces are there I just do not know how to use them.

I appreciate any help you can give, This is an essential requirement of our production system. right now we are using the pre-production system with a modified version of asset_tracker to send sensor data to the nRF Connect site. We need to be able to send sensor data to our AWS site.

[Martin Lesund]

Hi Timothy,

I see that you have added some configurations in the Kconfig file, but would instead invite you to instead add the Kconfig options from the AWS fota sample:
https://github.com/nrfconnect/sdk-nrf/blob/master/samples/nrf9160/aws_fota/Kconfig#L13

And insert your correct MQTT_BROKER_HOSTNAME etc..

and remember to 

CONFIG_USE_NRF_CLOUD=y

I also see changes that you have done in the main.c file
e.g. cjson_init etc. --> https://github.com/nrfconnect/sdk-nrf/pull/2035/files recommended to just have that in the cloud_codec instead.

[Timothy]

Martin

I added the lines you suggested and matched up things for my AWS account in Kconfig. I removed the Cjson_int since it is already in cloud_codec.c. no change. I get the exact same output.

[Martin Lesund]

Hi Timothy,
I apologize, I seem to have misunderstood your goal.

From what I now have understood you just want to run the "Asset Tracker" Application, but it should connect your host instead.

So let's start with a clean plate.

Make sure you are running NCS v1.3.0:

cd ncs/nrf
git checkout master
git pull
git checkout v1.3.0
west update

Then open an unmodified Asset Tracker application.

Then you can add the Configurations so that the Application should connect to your host.

So you can add the following (with some modifications based on your preference):

CONFIG_NRF_CLOUD_HOST_NAME="a2uy3w9dvx40gv-ats.iot.us-east-1.amazonaws.com"
CONFIG_NRF_CLOUD_PORT=8883
CONFIG_NRF_CLOUD_SEC_TAG=16842753

Then you utilize the features of the nrf cloud library itself instead of mixing in other modules like before.

(e.g. CONFIG_NRF_CLOUD=y, already enables CONFIG_MQTT_LIB=y, CONFIG_MQTT_LIB_TLS=y which you set separately before)

Best regards,
Martin L.

[Timothy]

Martin

It looks like we are only using the "NRF_CLOUD" client and not the "AWS_IOT" as in the original. I just changed the HOST and the security tag in Kconfig. I also added the lines suggested to prj.conf  I do not see a CLIENT_ID that was unique and essential in the AWS_IOT implementation in V1.2.0. we will have many boards sending sensor information up to AWS and the CLIENT_ID was how they were unique topics for subscribe and publish.

things are different but still not working correctly see the printout below. It looks like I am connecting to my AWS account but I do not see device information getting sent as a JSON packet and I do not see status being sent at all. I am using the AWS IoT "test" to subscribe to topics and see what is getting sent. I have included my Kconfig and prj.conf files. It does NOT crash but does NOT continue to send any data to AWS.

with the "AWS_IOT" client and implementation I did get the FLIP JSON packet but then it crashed. It was connected to my account and was sending JSON packets. This was the implementaion I used in V1.2.0. I would be more comfortable with getting that to work but right now neither implmentation is working.

on the AWS topic

$aws/things/nrf-352656100378150/shadow/get

which is used to send device information JSON packet I get the following message: "We cannot display the message as JSON, and are instead displaying it raw as Hexadecimal.". but there is nothing displayed. It is blank no hexadecimal display. so something got sent to the correct topic on my account. just not a good JSON packet.

on the AWS topic 

$aws/things/nrf-352656100378150/shadow/update

which is used to send the sensor data JSON packet I get nothing.

on the AWS topic 

$aws/things/nrf-352656100378150/shadow/get/accepted

which is used for I do not what I get a JSON packet. this is what is used for a test publish. I did not send it.

{
  "state": {
    "reported": {
      "message": "Hello Internet of Things!"
    }
  },
  "metadata": {
    "reported": {
      "message": {
        "timestamp": 1591533259
      }
    }
  },
  "version": 19,
  "timestamp": 1593786764
}

[00:00:00.190,887] <dbg> nrf9160_gps.setup: MAGPIO set: AT%XMAGPIO=1,0,0,1,1,1574,1577
[00:00:00.199,798] <dbg> nrf9160_gps.setup: COEX0 set: AT%XCOEX0=1,1,1570,1580
[00:00:00.207,519] <inf> asset_tracker: Asset tracker started TJM
[00:00:00.214,141] <inf> watchdog: Watchdog timeout installed. Timeout: 10000
[00:00:00.221,740] <inf> watchdog: Watchdog started
[00:00:00.227,142] <inf> watchdog: Watchdog feed enabled. Timeout: 5000
[00:00:00.241,455] <dbg> nrf_cloud_transport.nct_client_id_get: client_id = nrf-352656100378150
[00:00:00.250,671] <dbg> nrf_cloud_transport.nct_topics_populate: shadow_base_topic: $aws/things/nrf-352656100378150/shadow
[00:00:00.262,298] <dbg> nrf_cloud_transport.nct_topics_populate: accepted_topic: nrf-352656100378150/shadow/get/accepted
[00:00:00.273,773] <dbg> nrf_cloud_transport.nct_topics_populate: rejected_topic: $aws/things/nrf-352656100378150/shadow/get/rejected
[00:00:00.286,254] <dbg> nrf_cloud_transport.nct_topics_populate: update_delta_topic: $aws/things/nrf-352656100378150/shadow/update/delta
[00:00:00.299,133] <dbg> nrf_cloud_transport.nct_topics_populate: update_topic: $aws/things/nrf-352656100378150/shadow/update
[00:00:00.310,943] <dbg> nrf_cloud_transport.nct_topics_populate: shadow_get_topic: $aws/things/nrf-352656100378150/shadow/get
[00:00:00.330,047] <inf> asset_tracker: Connecting to LTE network.
[00:00:00.336,700] <inf> asset_tracker: This may take several minutes.
[00:00:00.343,780] <dbg> lte_lc.lte_lc_system_mode_set: Sending AT command to set system mode: AT%XSYSTEMMODE=1,0,1,0
[00:00:00.363,433] <dbg> lte_lc.lte_lc_system_mode_set: Sending AT command to set system mode: AT%XSYSTEMMODE=1,0,1,0
+CEREG: 2,"9E03","026B8102",7,0,0,"11100000","11100000"
[00:00:02.168,304] <dbg> lte_lc.at_handler: +CEREG notification: +CEREG: 2,"9E03","026B8102",7,0,0,"11100000","11100000"

[00:00:02.180,084] <dbg> lte_lc.parse_psm_cfg: TAU: -1 sec, active time: -1 sec

+CSCON: 1
[00:00:02.479,370] <dbg> lte_lc.at_handler: +CSCON notification
+CEREG: 5,"9E03","026B8102",7,,,"11100000","11100000"
[00:00:05.803,588] <dbg> lte_lc.at_handler: +CEREG notification: +CEREG: 5,"9E03","026B8102",7,,,"11100000","11100000"

[00:00:05.815,216] <dbg> lte_lc.parse_psm_cfg: TAU: -1 sec, active time: -1 sec

[00:00:05.823,181] <inf> asset_tracker: Connected to LTE network.
[00:00:05.829,772] <inf> asset_tracker: Connecting to cloud, attempt 1 of 8
[00:00:05.837,280] <inf> asset_tracker: Cloud connection request sent.
[00:00:05.844,268] <inf> asset_tracker: Connection response timeout is set to 30 seconds.
[00:00:05.853,057] <inf> asset_tracker: CLOUD_EVT_CONNECTING
[00:00:09.782,958] <dbg> nrf_cloud_transport.nct_connect: IPv4 address: 3.223.0.180
[00:00:09.791,076] <dbg> nrf_cloud_transport.nct_mqtt_connect: mqtt_connect requesting persistent session
[00:00:30.025,054] <inf> nrf_cloud_transport: Using non-blocking socket
[00:00:30.032,196] <dbg> nrf_cloud.nrf_cloud_run: Cloud connection request sent.
[00:00:30.598,175] <dbg> aws_fota.aws_fota_mqtt_evt_handler: Previous session valid; skipping FOTA subscriptions
[00:00:30.608,917] <inf> aws_fota: Created notify_next_topic $aws/things/nrf-352656100378150/jobs/notify-next
[00:00:30.619,354] <inf> aws_fota: Created get_topic $aws/things/nrf-352656100378150/jobs/$next/get/#
[00:00:30.629,028] <inf> aws_fota: previously subscribed to notify-next topic
[00:00:30.636,718] <inf> aws_jobs: Publish topic: $aws/things/nrf-352656100378150/jobs/$next/get
[00:00:30.645,935] <inf> aws_jobs: Publish payload {"clientToken": ""}
[00:00:30.654,022] <dbg> nrf_cloud_transport.nct_mqtt_evt_handler: MQTT_EVT_CONNACK: result 0
[00:00:30.663,055] <dbg> nrf_cloud.nfsm_set_current_state_and_notify: state: 2
[00:00:30.670,715] <dbg> nrf_cloud.api_event_handler: NRF_CLOUD_EVT_TRANSPORT_CONNECTED
[00:00:30.679,199] <inf> asset_tracker: CLOUD_EVT_CONNECTED
[00:00:30.685,241] <inf> asset_tracker: Persistent Sessions = 1
[00:00:30.691,650] <dbg> nrf_cloud_fsm.connection_handler: Previous session valid; skipping nct_cc_connect()
[00:00:30.701,934] <dbg> nrf_cloud.nfsm_set_current_state_and_notify: state: 4
[00:00:30.709,655] <dbg> nrf_cloud_transport.nct_cc_send: mqtt_publish: id = 5678 opcode = 0 len = 0
[00:00:30.720,153] <dbg> nrf_cloud.nfsm_set_current_state_and_notify: state: 5
[00:00:31.317,993] <dbg> nrf_cloud_transport.nct_mqtt_evt_handler: MQTT_EVT_PUBACK: id = 21395 result = 0
[00:00:31.328,247] <dbg> aws_fota.on_publish_evt: Received topic: $aws/things/nrf-352656100378150/jobs/$next/get/accepted
[00:00:31.339,660] <inf> aws_fota: Checking for an available job
[00:00:31.346,252] <dbg> aws_fota.get_job_execution: Job doc: {"clientToken":"","timestamp":1593786763}
[00:00:31.356,414] <dbg> aws_fota.get_job_execution: Got only one field
[00:00:31.363,586] <inf> aws_fota: No queued jobs for this device
[00:00:31.703,094] <dbg> nrf_cloud_transport.nct_mqtt_evt_handler: MQTT_EVT_PUBACK: id = 5678 result = 0
[00:00:31.713,073] <dbg> nrf_cloud.nfsm_set_current_state_and_notify: state: 5
+CSCON: 0
[00:00:38.732,910] <dbg> lte_lc.at_handler: +CSCON notification
+CSCON: 1
[00:02:30.857,788] <dbg> lte_lc.at_handler: +CSCON notification
+CSCON: 0
[

2514.Kconfig.zip8877.prj.zip

[Timothy]

Martin

I went back to the AWS_IOT backend where it sends flip data and then crashed. This matches what we have in V1.2.0 and what you had suggested using. I did some experimenting and found the following.

if I return immediately in the call device_status_send(struct k_work *work) in main it does NOT crash and sends back the FLIP data and the sensor data.

I then went further and returned immediately in the call modem_info_params_get in modem_info_parameters.c it does NOT crash and sends the FLIP and sensor data.

I then went further and returned immediately in the call modem_data_get in modem_info_parameters.c and it does NOT crash and sends back FLIP and sensor data.

Summary: I am using AWS_IOT as I am using n V1.2.0 as the backend. The only change to the code is a return immediate in the call as shown below.

static int modem_data_get(struct lte_param *param)
{
return 0; /* XXX added TJM */

So I am very close to getting V1.3.0 working. all works except it crashes when I get the modem information is what it looks like. Any ideas what I am missing or doing wrong at this point.

[Timothy]

Martin

I believe I narrowed it down to where it is failing. I have added print statements as shown below. I will try to explain the steps and teh printout. my prints have an 'XXX' in them.

If I do nothing (return immediately) in the device_status_send function all works fine and it does not crash. data is snet to mt AWS account as expected. If I only allow the AT_PARAM_TYPE_nUM_SHORT to be allowed things fail on the second call to the handler. I have the calles to AT_PARAM_TYPE_STRING disabled (no call to handler function).

as you can see the first call to modem_data_get is type 1. this calles modem_info_short_get which gets the string "%XCBAND 13' from the at_cmd_write. this then gets decoded to retunr the value 13 in at_params_short_get. this looks correct to me. the next is a call for AT_PARAM_TYPE_STRING which is ignored.

the next call is again a call for AT_PARAM_TYPE_NUM_SHORT but it has a param->type 4. at this point the at_cmd_write never returns and you reboot Zephyr.

I belive I am very close and a lot is working and talking to my AWS account. It seems like it fails trying to get all of the modem information or issue AT commands or something. do I have some configuration incorrect? any ideas what is going on. I cannot proceed until I have all of teh modem information correctly. I use the IEMI, date, etc throughout my program. I would apreciate any help you can give and thank you for all of your help so far. remeber I am using teh AWS_IOT backend like first setup in this ticket not the NRF cloud used later.

XXX device_status_send called
XXX modem_info_params_get called
XXX modem_data_get called param->type: 1 data_type: 1
AT_PARAM_TYPE_NUM_SHORT
XXX modem_info_short_get called info: 1
[00:00:15.705,871] <inf> aws_fota: subscribed to notify-next topic
[00:00:15.712,615] <inf> aws_jobs: Publish topic: $aws/things/nrf-352656100378150/jobs/$next/get
[00:00:15.721,893] <inf> aws_jobs: Publish payload {"clientToken": ""}
XXX at_cmd_write err: 0 recv_buf: %XCBAND: 13

XXX modem_info_parse err: 0
XXX at_params_short_get err: 0 value: 13
XXX modem_data_get called param->type: 2 data_type: 3
AT_PARAM_TYPE_STRING
XXX modem_data_get called param->type: 4 data_type: 1
AT_PARAM_TYPE_NUM_SHORT
XXX modem_info_short_get called info: 4
[00:00:16.003,173] <dbg> aws_iot.mqtt_evt_handler: MQTT_EVT_SUBACK: id = 2114 result = 0
[00:00:16.021,759] <dbg> aws_iot.mqtt_evt_handler: MQTT_EVT_SUBACK: id = 31697 result = 0
[00:00:16.254,486] <dbg> aws_iot.mqtt_evt_handler: MQTT_EVT_PUBACK: id = 56652 result = 0
[00:00:16.307,220] <dbg> aws_fota.on_publish_evt: Received topic: $aws/things/nrf-352656100378150/jobs/$next/get/accepted
[00:00:16.318,664] <inf> aws_fota: Checking for an available job
[00:00:16.325,195] <dbg> aws_fota.get_job_execution: Job doc: {"clientToken":"","timestamp":1594064733}
[00:00:16.335,327] <dbg> aws_fota.get_job_execution: Got only one field
[00:00:16.342,407] <inf> aws_fota: No queued jobs for this device
*** Booting Zephyr OS build v2.3.0-rc1-ncs1 ***