AWS_FOTA: ERROR: mqtt_connect -22

[icampbell]

Hello, I've been having problems getting the AWS_FOTA sample working on a nrf9160 DK.

At first, the device got stuck as it was downloading the firmware update, at 2%. I was able to fix this by setting CONFIG_DFU_TARGET_MCUBOOT_SAVE_PROGRESS=n. After that, the download got stuck at around 11%, though that number changed a bit each time I ran it. While I was trying to fix that, the device stopped being able to connect to AWS at all, even though I didn't change anything. The device waits for a long time after the "client_id:" line, and then seems to time out and print "ERROR: mqtt_connect -22". The timeout/error seems to come from mqtt_connect -> mqtt_client_tls_connect -> connect (line 87 in mqtt_transport_socket_tls.c). I am able to connect to AWS IoT from a python client on my computer using the same credentials and MQTT broker, so I don't think it's a problem with the AWS server.

Here are the changes I made to prj.conf:
CONFIG_DFU_TARGET_MCUBOOT_SAVE_PROGRESS=n
CONFIG_CLOUD_CERT_SEC_TAG=5
CONFIG_APP_VERSION="v1"
CONFIG_MQTT_BROKER_HOSTNAME="ahykgwaormc93-ats.iot.us-west-2.amazonaws.com"
CONFIG_USE_CLOUD_CLIENT_ID=y
CONFIG_CLOUD_CLIENT_ID="nrf9160dk"
CONFIG_USE_NRF_CLOUD=n

Besides that and the credentials (ran once with CONFIG_PROVISION_CERTIFICATES), I didn't make any changes to the AWS_FOTA sample.

I tried both the master branch and the v1.2.0 branch. I also stuff like turning CONFIG_DFU_TARGET_MCUBOOT_SAVE_PROGRESS on again, and re-flashing the credentials with CONFIG_PROVISION_CERTIFICATES again.

[Øyvind]

Hello, 

From nrfxlib\bsdlib\include\nrf_errno.h

#define NRF_EINVAL          22  /**< Invalid argument */

I've had no issues with the AWS FOTA sample (tested yesterday). Can you please provide the output from i.e. LTE Link Monitor? 

Kind regards,
Øyvind

[icampbell]

OK, here's the output

[Øyvind]

Ok, that did not give much more information on how to solve this. 

Did you try to revert back to the original project? Were you trying to download from S3 bucket? 

1. Is the client ID the same as the client ID of the AWS thing?
2. Have you registered correct policy?
3. Is the hostname correct?

Please confirm all settings in AWS given in the documentation

[icampbell]

"git diff" only shows the changes to prj.conf that I showed above and certificates.h, which are necessary to get it to work.

I am using an S3 bucket for the update and the job file but mqtt_connect is before it gets to that point.

1. Yes, the client ID on AWS is also "nrf9160dk"

2. Yes, I am using the sample policies from https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/samples/nrf9160/aws_fota/README.html#nrf9160-aws-fota for both the thing and the S3 bucket

3. Yes, the endpoint at AWS IoT -> Test -> View Endpoint is also "ahykgwaormc93-ats.iot.us-west-2.amazonaws.com".

The device was able to connect to AWS MQTT before, and I haven't changed anything, so the client ID, hostname, and certificates should work.

[Øyvind]

icampbell said:

The device was able to connect to AWS MQTT before, and I haven't changed anything, so the client ID, hostname, and certificates should work

 It the device was able to connect before, then you must revert any changes done before losing connection.

 

At first, the device got stuck as it was downloading the firmware update, at 2%. I was able to fix this by setting CONFIG_DFU_TARGET_MCUBOOT_SAVE_PROGRESS=n. After that, the download got stuck at around 11%, though that number changed a bit each time I ran it. While I was trying to fix that, the device stopped being able to connect to AWS at all, even though I didn't change anything.

 I suggest reverting the sample to the original git hash and provide information on what happens when the download gets stuck. 

Are you using NB-IoT or LTE-M?

KInd regards,
Øyvind

[icampbell]

I did not make any changes between when it was able to connect and when it wasn't. I even tried reverting the CONFIG_DFU_TARGET_MCUBOOT_SAVE_PROGRESS change, even though it was able to connect after that change until it stopped working, but it didn't work even with CONFIG_DFU_TARGET_MCUBOOT_SAVE_PROGRESS=y. Even if I were to revert it to the original version from the repo, I would need to make the same changes to connect to my account (broker url, certificates, id, etc.)

I'm pretty sure I'm using LTE-M.

Here are the changes to the project; I've been able to connect with this configuration before:

diff --git a/samples/nrf9160/aws_fota/prj.conf b/samples/nrf9160/aws_fota/prj.conf
index 33b27ca..0d2a6a6 100644
--- a/samples/nrf9160/aws_fota/prj.conf
+++ b/samples/nrf9160/aws_fota/prj.conf
@@ -67,3 +67,13 @@ CONFIG_BOOTLOADER_MCUBOOT=y
 CONFIG_IMG_MANAGER=y
 CONFIG_FLASH=y
 CONFIG_IMG_ERASE_PROGRESSIVELY=y
+
+# CONFIG_PROVISION_CERTIFICATES=y
+CONFIG_CLOUD_CERT_SEC_TAG=5
+CONFIG_APP_VERSION="v1"
+CONFIG_MQTT_BROKER_HOSTNAME="ahykgwaormc93-ats.iot.us-west-2.amazonaws.com"
+CONFIG_USE_CLOUD_CLIENT_ID=y
+CONFIG_CLOUD_CLIENT_ID="nrf9160dk"
+CONFIG_USE_NRF_CLOUD=n
+# CONFIG_FOTA_DOWNLOAD_PROGRESS_EVT=y
+# CONFIG_AWS_FOTA_LOG_LEVEL_DBG=y
diff --git a/samples/nrf9160/aws_fota/src/certificates.h b/samples/nrf9160/aws_fota/src/certificates.h
index e0f316b..fdf9ba4 100644
--- a/samples/nrf9160/aws_fota/src/certificates.h
+++ b/samples/nrf9160/aws_fota/src/certificates.h
<my certificates>

[Øyvind]

My apologies for the late reply.

icampbell said:

I'm pretty sure I'm using LTE-M.

Are you using iBasis, or another SIM card? If you are using iBasis, please verify that you have enough data. 

icampbell said:

Even if I were to revert it to the original version from the repo, I would need to make the same changes to connect to my account (broker url, certificates, id, etc.)

 Yes, that is expected. We need to verify if the issue is on your side or ours. Also, please verify that all steps are followed/done correctly in AWS console. You could try to add a more restrictive policy:

{
   "Version": "2012-10-17",
   "Statement": [
    {
      "Effect": "Allow",
      "Action": "iot:Connect",
      "Resource": "arn:aws:iot:*:*:client/${iot:ClientId}"
      }
    },
    {
      "Effect": "Allow",
      "Action": "iot:Receive",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "iot:Subscribe",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "iot:Publish",
      "Resource": "*"
    }
  ]
 }

icampbell said:

OK, here's the output

 From this screenshot, I can see you have a weak signal. Can you please try to program your device with the AT Client and provide the output from LTE Link Monitor. 

Thanks! 

Kind regards,
Øyvind

[icampbell]

I am using iBasis, and I seem to have enough data. I also tried a test application that sends and receives on a UDP socket, which also works.

I tried the more restrictive policy, but that didn't fix the problem, still an error with mqtt_connect -22.

About the default configuration - I wasn't aware that this example even worked with nrf cloud until just now.  Reverting the repository doesn't work (the broker is set to your_aws_mqtt_broker_hostname.amazonaws.com by default, which causes "getaddrinfo failed -11"), and when I set CONFIG_USE_NRF_CLOUD=y, I get the same problem as before (mqtt_connect -22)

Here is the output of the AT Client.

[Øyvind]

icampbell said:

I am using iBasis, and I seem to have enough data

 Yes, should be enough. 

 

icampbell said:

About the default configuration - I wasn't aware that this example even worked with nrf cloud until just now

 No, this should not work with nRF Cloud from what I am told. It is designed to work nRF Cloud in the beginning, but some changes were introduced to nRF Cloud which made it not work with nRF Cloud. nRF Cloud is based on AWS, and will, therefore, have a similar address. 

 

icampbell said:

Here is the output of the AT Client.

 Can you please click on Open Log File, then save and send a full log file?

From the information you give, there seems to be a mismatch between your prj.conf and the AWS MQTT settings. Are you using SES when building and programming? What I am fearing, is that the settings you add in prj.conf are overwritten by kconfig. E.g. your client-id does not match in the latest output: