nrf9160 fota download with Microsft Blob storage issue

[Nikola Markovic]

Hi,

I'm able to successful download and flash my application firmware using AWS S3, but unable to use Microsoft Blob Storage successfully.

I'm using Thingy91 with SDK 1.3.0 in a custom project.

My relevant prj.conf settings:

# Download client
CONFIG_DOWNLOAD_CLIENT=y
CONFIG_DOWNLOAD_CLIENT_TLS=y
CONFIG_DOWNLOAD_CLIENT_STACK_SIZE=8192
CONFIG_DOWNLOAD_CLIENT_SOCK_TIMEOUT_MS=10000

(also tried default -1 - forever timeout and 4k stack default size)

A  tiny file download succeeds, but recognizes that it's not a valid firmware image:

Starting OTA download for host:itoctonnectstore.blob.core.windows.net path:nrf/test/test.zip, sec_tag:10703
[00:00:19.566,986] <dbg> download_client.download_thread: Receiving up to 4096 bytes at 0x2002060c...
IoTConnect MQTT Connected
[00:00:19.789,031] <dbg> download_client.download_thread: Read 661 bytes from socket
[00:00:19.797,515] <dbg> download_client.header_parse: GET header size: 500
[00:00:19.805,084] <dbg> download_client.header_parse: client->buf HTTP/1.1 206 Partial Content
Content-Length: 161
Content-Type: application/octet-stream
Content-Range: bytes 0-160/161
Last-Modified: Fri, 07 Aug 2020 22:32:11 GMT
Accept-Ranges: bytes
ETag: "0x8D83B21B9F0CAB2"
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: f3993d2f-c01e-0069-0b0a-6dd9b2000000
x-ms-version: 2014-02-14
x-ms-lease-status: unlocked
x-ms-lease-state: available
x-ms-blob-type: BlockBlob
Content-Disposition:
Date: Fri, 07 Aug 2020 22:33:57 GMT

PK

[00:00:19.856,597] <dbg> download_client.header_parse: File size = 161
[00:00:19.863,861] <wrn> download_client: Copying 161 payload bytes
[00:00:19.871,002] <inf> download_client: Downloaded 161/161 bytes (100%)
[00:00:19.878,448] <err> dfu_target: No supported image type found
[00:00:19.885,498] <err> dfu_target: Unknown image type
[00:00:19.891,571] <err> fota_download: dfu_target_init error -134

A file of size 20Kbytes:

[00:00:21.046,295] <inf> download_client: Connected to itoctonnectstore.blob.core.windows.net
[00:00:21.055,450] <inf> download_client: Configuring socket timeout (10 s)
[00:00:21.063,140] <dbg> download_client.get_request_send: Send Buffer: GET /nrf/test/random.bin HTTP/1.1
Host: itoPacket id: 2 acknowledged
ctonnectstore.blob.core.windows.net
Connection: keep-alive
Range: bytes=0-2047

[00:00:21.084,442] <dbg> download_client.get_request_send: Sending HTTP request
[00:00:21.093,475] <dbg> download_client.download_thread: Receiving up to 4096 bytes at 0x2002060c...
[00:00:21.325,836] <wrn> download_client: Peer closed connection!
[00:00:21.332,763] <wrn> fota_download: Download socket error. 1 retries left...
[00:00:21.341,033] <inf> download_client: Reconnecting..
[00:00:21.348,297] <inf> download_client: Attempting to connect over IPv4
[00:00:21.355,987] <inf> download_client: Setting up TLS credentials
[00:00:23.047,271] <inf> download_client: Connected to itoctonnectstore.blob.core.windows.net
[00:00:23.056,457] <inf> download_client: Configuring socket timeout (10 s)
[00:00:23.064,147] <dbg> download_client.get_request_send: Send Buffer: GET /nrf/test/random.bin HTTP/1.1
Host: itoctonnectstore.blob.core.windows.net
Connection: keep-alive
Range: bytes=0-2047

[00:00:23.082,794] <dbg> download_client.get_request_send: Sending HTTP request
[00:00:23.091,796] <dbg> download_client.download_thread: Receiving up to 4096 bytes at 0x2002060c...
[00:00:23.325,836] <wrn> download_client: Peer closed connection!
[00:00:23.332,855] <err> fota_download: Download client error

Also, the same download works successfully with curl mimicking the same header and remote doesn't disconnect

curl -v -X GET \
-H 'Accept:' \
-H 'User-Agent:' \
-H 'Host: itoctonnectstore.blob.core.windows.net' \
-H 'Connection: keep-alive' \
-H 'Range: bytes=0-2047' \
'itoctonnectstore.blob.core.windows.net/.../random.bin'

This is difficult to debug because the disconnect may be coming from the modem. The read()  fails with 0 bytes in download_client.c and that ause the "Peer closed" error.

Can you please advise?

Thanks,

Nik

[Didrik Rokhaug]

Hi.

I believe the problem is due to too large packet sizes.

The HTTP headers from Azure can be quite big (especially if SAS tokens are used in addition to TLS), which can cause problems when requesting large file fragments.

Could you try to lower CONFIG_DOWNLOAD_CLIENT_MAX_TLS_FRAGMENT_SIZE, and see if that helps?

Best regards,

Didrik

[Nikola Markovic]

Hi,

I lowered it to 1024 from the default 2048 and it took care of the problem.

I don't completely understand what the underlying cause for the issue is.I can understand that "if something is to too small, then app should increase it" but this is the opposite. I can only guess that 2048 fragment size is too large for modem's MTU size or something similar to that. Shouldn't 1024 be the software default then on your end?

I'm all set, but just here trying to understand the problem and see if reducing the default would help other Nordic customers who may run into this issue.

Thanks Didrik!

[stes]

Hi Nikola, have you been using SAS (shared access signature) for downloading the firmware? I currently see the "Peer closed" error, but my settings seem to the ones you listed..

Regards

Stefan

[Nikola Markovic]

I was passing the the temporary token in the URL.

This was probably in sdk 1.3.1. Maybe double-check if the names of settings changed in newer SDKs by checking in menuconfig. I think Nordic even increased the default size too.

Sorry, I only remember this much about the issue.