nrf9160 AWS FOTA fails after OTA job starts

[hmichel]

My nrf9160 connects to AWS when using the AWS FOTA example. However, when I start the custom job as setup from the following link:

https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/samples/nrf9160/aws_fota/README.html#configuring

I get the following in the termite terminal window:

see the first error AWS_FOTA_EVT_ERROR below

LTE Link Connected!
IPv4 Address 52.15.223.13
client_id: nrf-352656100047698
[mqtt_evt_handler:209] MQTT client connected!
[mqtt_evt_handler:262] PUBACK packet id: 28212
[mqtt_evt_handler:272] SUBACK packet id: 2114
[00:00:38.854,400] [1B][0m<inf> aws_jobs: Subscribe: $aws/things/nrf-352656100047698/jobs/notify-next[1B][0m
[00:00:38.856,109] [1B][0m<inf> aws_jobs: Subscribe: $aws/things/nrf-352656100047698/jobs/$next/get/#[1B][0m
[00:00:39.259,735] [1B][0m<inf> aws_fota: subscribed to notify-next topic[1B][0m
[00:00:39.259,826] [1B][0m<inf> aws_jobs: Publish topic: $aws/things/nrf-352656100047698/jobs/$next/get[1B][0m
[00:00:39.259,826] [1B][0m<inf> aws_jobs: Publish payload {"clientToken": ""}[1B][0m
[mqtt_evt_handler:262] PUBACK packet id: 41283
[00:00:40.077,911] [1B][0m<inf> aws_fota: Received topic: $aws/things/nrf-352656100047698/jobs/$next/get/accepted[1B][0m
[00:00:40.078,125] [1B][0m<inf> aws_fota: Got only one field: {"clientToken":"","timestamp":1580875619}[1B][0m
[mqtt_evt_handler:277] default: 8
[00:00:59.907,440] [1B][0m<inf> aws_fota: Received topic: $aws/things/nrf-352656100047698/jobs/notify-next[1B][0m
[00:00:59.908,050] [1B][0m<inf> aws_jobs: Unsubscribe: $aws/things/nrf-352656100047698/jobs/notify-next[1B][0m
[00:00:59.909,149] [1B][0m<inf> aws_jobs: Subscribe: $aws/things/nrf-352656100047698/jobs/nrf9150fwota_020220_04/update/#[1B][0m
[00:01:00.715,301] [1B][0m<inf> aws_jobs: Publish topic: $aws/things/nrf-352656100047698/jobs/nrf9150fwota_020220_04/update[1B][0m
[00:01:00.715,332] [1B][0m<inf> aws_jobs: Publish payload {"status":"IN_PROGRESS","statusDetails": {"nextState":"download_firmware"},"expectedVersion": "1","clientToken": ""}[1B][0m
[mqtt_evt_handler:262] PUBACK packet id: 23440
[00:01:01.559,356] [1B][0m<inf> aws_fota: Received topic: $aws/things/nrf-352656100047698/jobs/nrf9150fwota_020220_04/update/accepted[1B][0m
[00:01:01.559,448] [1B][0m<dbg> aws_fota.aws_fota_on_publish_evt: Job document update was accepted[1B][0m
[00:01:01.559,539] [1B][0m<inf> aws_fota: Start downloading firmware from s3.us-east-2.amazonaws.com/app_update.bin[1B][0m
[00:01:01.915,649] [1B][0m<inf> download_client: Attempting to connect over IPv4[1B][0m
[00:01:02.235,290] [1B][0m<inf> download_client: Connected to s3.us-east-2.amazonaws.com[1B][0m
[00:01:02.235,321] [1B][0m<inf> download_client: Downloading: /app_update.bin [0][1B][0m
AWS_FOTA_EVT_ERROR
[mqtt_evt_handler:262] PUBACK packet id: 20563
[00:01:02.625,671] [1B][1;31m<err> download_client: Server did not send "Content-Range" in response[1B][0m
[00:01:02.627,288] [1B][1;31m<err> fota_download: Download client error[1B][0m
[00:01:02.627,288] [1B][1;31m<err> fota_download: Unable to deinitialze resources used by dfu_target.[1B][0m
[00:01:02.627,288] [1B][1;31m<err> aws_fota: FOTA download failed, report back[1B][0m
[00:01:02.627,502] [1B][0m<inf> aws_jobs: Publish topic: $aws/things/nrf-352656100047698/jobs/nrf9150fwota_020220_04/update[1B][0m
[00:01:02.627,532] [1B][0m<inf> aws_jobs: Publish payload {"status":"FAILED","statusDetails": {"nextState":"download_firmware"},"expectedVersion": "2","clientToken": ""}[1B][0m
[00:01:03.546,295] [1B][0m<inf> aws_fota: Received topic: $aws/things/nrf-352656100047698/jobs/nrf9150fwota_020220_04/update/accepted[1B][0m
[00:01:03.546,356] [1B][0m<dbg> aws_fota.aws_fota_on_publish_evt: Job document update was accepted[1B][0m
[00:01:03.546,447] [1B][0m<inf> aws_fota: Start downloading firmware from s3.us-east-2.amazonaws.com/app_update.bin[1B][0m
[00:01:03.547,424] [1B][0m<inf> download_client: Attempting to connect over IPv4[1B][0m
[00:01:03.989,349] [1B][0m<inf> download_client: Connected to s3.us-east-2.amazonaws.com[1B][0m
[00:01:03.989,379] [1B][0m<inf> download_client: Downloading: /app_update.bin [0][1B][0m
AWS_FOTA_EVT_ERROR
[00:01:05.036,529] [1B][1;31m<err> download_client: Server did not send "Content-Range" in response[1B][0m
[00:01:05.041,290] [1B][1;31m<err> fota_download: Download client error[1B][0m
[00:01:05.041,290] [1B][1;31m<err> fota_download: Unable to deinitialze resources used by dfu_target.[1B][0m
[00:01:05.041,290] [1B][1;31m<err> aws_fota: FOTA download failed, report back[1B][0m
[00:01:05.041,503] [1B][0m<inf> aws_jobs: Publish topic: $aws/things/nrf-352656100047698/jobs/nrf9150fwota_020220_04/update[1B][0m
[00:01:05.041,534] [1B][0m<inf> aws_jobs: Publish payload {"status":"FAILED","statusDetails": {"nextState":"download_firmware"},"expectedVersion": "3","clientToken": ""}[1B][0m
[mqtt_evt_handler:262] PUBACK packet id: 34129
AWS_FOTA_EVT_ERROR
aws_fota_mqtt_evt_handler: Failed! -14
[mqtt_evt_handler:227] MQTT PUBLISH result=0 len=281
mqtt_read_publish_payload: Failed! -36
Disconnecting MQTT client...
Received:
[mqtt_evt_handler:220] MQTT client disconnected 0
POLLNVAL
Disconnecting MQTT client...
Could not disconnect MQTT client. Error: -57
[00:01:06.747,833] [1B][0m<inf> aws_fota: Received topic: $aws/things/nrf-352656100047698/jobs/nrf9150fwota_020220_04/update/rejected[1B][0m
[00:01:06.747,924] [1B][1;31m<err> aws_fota: Job document update was rejected[1B][0m

[Øyvind]

Hello, 

First of all, what version of NCS are you running? 

 

Server did not send "Content-Range" in response[1B][0m

In the AWS_FOTA documentation, it states under troubleshooting: 

Content range is not defined:

If you host the firmware image on a different server than in an S3 bucket, this error indicates that the Content-Range field is missing in the HTTP GET header. To fix this problem, configure the host server to provide this field. If you are using an S3 bucket, make sure that you have configured it as described in Setting up an AWS S3 bucket. Also, confirm that your file is available from the browser without being logged into your AWS account.

Can you please check and verify that this is the issue? If not I will try to reproduce here.

Kind regards,
Øyvind

[hmichel]

1) I am running v1.1.0 of NCS.

2) Per the link for setting up Aws S3 Bucket and as a result addressing the "Content-Range field is missing in the HTTP GET header"

Here are the bucket policy settings

And here is the content of the json file

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::mynrf9160fwupdatev2/*"
        }
    ]
}

3) I am able to put the url for the binary in a browser and it will pop up a message in the browser for downloading the file.

I must be missing something simple

Kind regards,

Hiilary

[Øyvind]

Hello Hiilary, 

I was able to reproduce the error, but could not see what caused it until I looked over your log one more time, and compared it to mine. I think your job file is the issue. 

In the host part of the job file, you are missing a part of your Object URL of update.bin:

Start downloading firmware from s3.us-east-2.amazonaws.com/app_update.bin

While mine is:

Start downloading firmware from nordicoeyvind.s3.eu-north-1.amazonaws.comapp_update.bin

After configuring the S3 Bucket permissions and Bucket policy, your Object URL should look like this:

https://<bucket-name>.s3.<region>.amazonaws.com/app_update.bin

It may be somewhat clumsy explained in the documentation, but the job file should look like this:

{
  "operation": "app_fw_update",
  "fwversion": "v1.0.2",
  "size": 181124,
  "location": {
    "protocol": "http:",
    "host": "<bucket-name>.s3.<region>.amazonaws.com",
    "path": "app_update.bin"
   }
}

Please try this and let me know how it goes. 

Remember to delete the old job under Manage --> Jobs in IoT Console.

Kind regards,
Øyvind

 

[hmichel]

That worked. Thank you for all the help. I have learned that it is good to delete the old jobs. Thanks for the reminder.

Kind regards,

Hilary

[Øyvind]

Great to hear!

Regards,
Øyvind