nRF9161 FOTA + TLS Download Error

[OhmMyGod]

Hi,

We are using `fota_download` library on our nRF9161 design running mfw-nrf91x1_2.0.1 to download a firmware upgrade image from our Amazon AWS server location.

When trying to download with TLS enabled, fota_donwload_start function errors out with errno 122: EMSGSIZE, however everything works fine when TLS is not being used. We are also able to download the image if TLS is setup on our local PC/server.

Our current understanding is that the nRF9161 delegates TLS/socket processing to the modem firmware. Adjusting the fragment size in our firmware to a smaller value hasn't resolved the problem, and error 122 persists.
We’ve looked into modifying the TLS settings on AWS server, however that doesn’t seem to be an option as of now.

Wondering if you have any recommendations on how to work around the problem?

[Øyvind]

Thanks for your patience. 

OhmMyGod said:

We have tried to setup another server and set fragmentation size to 1024 on the server side and found out that download does works for  both CONFIG_DOWNLOAD_CLIENT_HTTP_FRAG_SIZE_1024 and CONFIG_DOWNLOAD_CLIENT_HTTP_FRAG_SIZE_2048.

No updates yet to this issue. Still ongoing investigation. 

Could you please try adding the following configs for more debug logs

CONFIG_NRF_MODEM_LIB_LOG_LEVEL_DBG=y
CONFIG_NRF_MODEM_LIB_NET_IF_LOG_LEVEL_DBG=y
CONFIG_NET_CONNECTION_MANAGER_LOG_LEVEL_DBG=y
CONFIG_NET_LOG=y
CONFIG_DOWNLOAD_CLIENT_LOG_LEVEL_DBG=y

Thanks!

Kind regards,
Øyvind

[OhmMyGod]

Please see attached the logs with requested configs enabled

We were not able to find the CONFIG_NRF_MODEM_LIB_NET_IF_LOG_LEVEL_DBG (see below)

Fullscreen application_logs_7_16.txt Download

*** Booting nRF Connect SDK v2.5.0 ***
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, cop
y_done=0x3, image_ok=0x3
I: Secondary image: magic=unset, swap_type=0x1, c
opy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 0, Swap type: none
I: Bootloader chainload address offset: 0x10000
*** Booting nRF Connect SDK v2.5.0 ***
D: (0x200163a0): Connection Manager started
I: LTE Node application v1.0.7
I: Trace thread ready
I: Trace level override: 2
D: Modem library has initialized, ret 0
D: Modem init callback: 0x2f871
D: Modem init callback: 0x2eff5
D: Modem init callback: 0x2f161
I: IMEI: 1400D8DEC570E
I: Serial ID: 99999985
I: Hardware Revision: N/A - nRF 9160 DK
I: Node config URL: config.api.waites.net/test/no
de/99999985
I: Entered state ST_DISCONNECTED
E: Failed to find pin: 19
I: Entered state ST_INIT
D: iface 0x20010fd4 connect
+CEREG: 2,"3BAF","0D85A206",7
+CSCON: 1
+CGEV: ME PDN ACT 0,0
+CNEC_ESM: 50,0
+CEREG: 5,"3BAF","0D85A206",7,,,"00000000","10101
111"
D: (0x20016190): IPv4 event 3758358529 received o
n iface 1 (0x20010fd4)
D: (0x20016190): Iface index 0
%XTIME: "8A","4270611223508A","01"
D: (0x20016190): Iface event 3489726466 received
on iface 1 (0x20010fd4)
D: (0x20016190): Iface index 0
I: L4 Event: Connected
I: Entered state ST_NETWORK_UP
I: Entered state ST_CONNECTED
I: Subscribing to: on-config-changed/n/1400D8DEC5
70E
E: Library is in the wrong state (MQTT_STATE_DISC
ONNECTED), MQTT_STATE_CONNECTED required
E: Failed to subscribe to topics, err: -95
I: Connected successfully
I: Upgrading self with image: https://config.api.
waites.net/v1/sensor-core/Sensor-IEP-N1_217A7_v1.
6.0.bin
D: URI checksums 191005400,1987551763,0,0
D: state = 1
I: Downloading: v1/sensor-core/Sensor-IEP-N1_217A
7_v1.6.0.bin [0]
> AT+CFUN=1
D: Protocol not specified, defaulting to HTTP(S)
D: Port not specified, using default: 443
D: family: 1, type: 1, proto: 258
I: Setting up TLS credentials, sec tag count 1
I: Connecting to config.api.waites.net
D: fd 0, addrlen 8, fam IPv4, port 443
D: state = 2
E: len = 137
D: HTTP request
D: 47 45 54 20 2f 76 31 2f |GET /v1/
D: 73 65 6e 73 6f 72 2d 63 |sensor-c
D: 6f 72 65 2f 53 65 6e 73 |ore/Sens
D: 6f 72 2d 49 45 50 2d 4e |or-IEP-N
D: 31 5f 32 31 37 41 37 5f |1_217A7_
D: 76 31 2e 36 2e 30 2e 62 |v1.6.0.b
D: 69 6e 20 48 54 54 50 2f |in HTTP/
D: 31 2e 31 0d 0a 48 6f 73 |1.1..Hos
D: 74 3a 20 63 6f 6e 66 69 |t: confi
D: 67 2e 61 70 69 2e 77 61 |g.api.wa
D: 69 74 65 73 2e 6e 65 74 |ites.net
D: 0d 0a 52 61 6e 67 65 3a |..Range:
D: 20 62 79 74 65 73 3d 30 | bytes=0
D: 2d 31 30 32 33 0d 0a 43 |-1023..C
D: 6f 6e 6e 65 63 74 69 6f |onnectio
D: 6e 3a 20 6b 65 65 70 2d |n: keep-
D: 61 6c 69 76 65 0d 0a 0d |alive...
D: 0a                      |.
D: Receiving up to 4096 bytes at 0x200166e8...
E: Error in recv(), errno 122
W: Download socket error. 2 retries left...
I: Reconnecting...
D: Protocol not specified, defaulting to HTTP(S)
D: Port not specified, using default: 443
D: family: 1, type: 1, proto: 258
I: Setting up TLS credentials, sec tag count 1
I: Connecting to config.api.waites.net
D: fd 0, addrlen 8, fam IPv4, port 443
E: len = 137
D: HTTP request
D: 47 45 54 20 2f 76 31 2f |GET /v1/
D: 73 65 6e 73 6f 72 2d 63 |sensor-c
D: 6f 72 65 2f 53 65 6e 73 |ore/Sens
D: 6f 72 2d 49 45 50 2d 4e |or-IEP-N
D: 31 5f 32 31 37 41 37 5f |1_217A7_
D: 76 31 2e 36 2e 30 2e 62 |v1.6.0.b
D: 69 6e 20 48 54 54 50 2f |in HTTP/
D: 31 2e 31 0d 0a 48 6f 73 |1.1..Hos
D: 74 3a 20 63 6f 6e 66 69 |t: confi
D: 67 2e 61 70 69 2e 77 61 |g.api.wa
D: 69 74 65 73 2e 6e 65 74 |ites.net
D: 0d 0a 52 61 6e 67 65 3a |..Range:
D: 20 62 79 74 65 73 3d 30 | bytes=0
D: 2d 31 30 32 33 0d 0a 43 |-1023..C
D: 6f 6e 6e 65 63 74 69 6f |onnectio
D: 6e 3a 20 6b 65 65 70 2d |n: keep-
D: 61 6c 69 76 65 0d 0a 0d |alive...
D: 0a                      |.
D: Receiving up to 4096 bytes at 0x200166e8...
E: Error in recv(), errno 122
W: Download socket error. 1 retries left...
I: Reconnecting...
D: Protocol not specified, defaulting to HTTP(S)
D: Port not specified, using default: 443
D: family: 1, type: 1, proto: 258
I: Setting up TLS credentials, sec tag count 1
I: Connecting to config.api.waites.net
D: fd 0, addrlen 8, fam IPv4, port 443
E: len = 137
D: HTTP request
D: 47 45 54 20 2f 76 31 2f |GET /v1/
D: 73 65 6e 73 6f 72 2d 63 |sensor-c
D: 6f 72 65 2f 53 65 6e 73 |ore/Sens
D: 6f 72 2d 49 45 50 2d 4e |or-IEP-N
D: 31 5f 32 31 37 41 37 5f |1_217A7_
D: 76 31 2e 36 2e 30 2e 62 |v1.6.0.b
D: 69 6e 20 48 54 54 50 2f |in HTTP/
D: 31 2e 31 0d 0a 48 6f 73 |1.1..Hos
D: 74 3a 20 63 6f 6e 66 69 |t: confi
D: 67 2e 61 70 69 2e 77 61 |g.api.wa
D: 69 74 65 73 2e 6e 65 74 |ites.net
D: 0d 0a 52 61 6e 67 65 3a |..Range:
D: 20 62 79 74 65 73 3d 30 | bytes=0
D: 2d 31 30 32 33 0d 0a 43 |-1023..C
D: 6f 6e 6e 65 63 74 69 6f |onnectio
D: 6e 3a 20 6b 65 65 70 2d |n: keep-
D: 61 6c 69 76 65 0d 0a 0d |alive...
D: 0a                      |.
D: Receiving up to 4096 bytes at 0x200166e8...
E: Error in recv(), errno 122
E: Download client error
D: No DFU target was initialized
D: state = 4
D: state = 0
E: Upgrade start failed
I: Sleeping...
E: Received error from fota downloadHandler, err:
1
D: Connection closed
+CSCON: 0

modem_trace_7_16.mtrace

[Øyvind]

Thanks for your patience. Our TLS experts have had a look at this issue and the last modem trace you provided. They verify that the data sent it too large from the server. Still waiting for more details and suggestions to a work-around

While waiting for more information, as you pointed out, it is difficult to alter the fragmentation size on the AWS server. But have you tested all possible combinations of HTTP_FRAG_SIZE and CLIENT_BUF_SIZE? I.e.:

config DOWNLOAD_CLIENT_BUF_SIZE
	int "Buffer size (static)"
	range 256 4096
	default 2048
	help
	  Size of the internal buffer used for incoming and
	  outgoing packets. It must be large enough to
	  acommodate for the largest between the HTTP fragment
	  and CoAP block. In case of CoAP, the CoAP header
	  length of 20 bytes should be taken into account.

config DOWNLOAD_CLIENT_HTTP_FRAG_SIZE
	int
	default 256 if DOWNLOAD_CLIENT_HTTP_FRAG_SIZE_256
	default 512 if DOWNLOAD_CLIENT_HTTP_FRAG_SIZE_512
	default 1024 if DOWNLOAD_CLIENT_HTTP_FRAG_SIZE_1024
	default 2048 if DOWNLOAD_CLIENT_HTTP_FRAG_SIZE_2048
	default 4096 if DOWNLOAD_CLIENT_HTTP_FRAG_SIZE_4096

choice
	prompt "HTTP(S) fragment size"
	default DOWNLOAD_CLIENT_HTTP_FRAG_SIZE_2048
	help
	  Size of the data fragments reported to the application when
	  using HTTP or HTTPS. Use the largest fragment size when using HTTPS
	  to reduce the bandwidth overhead due to the HTTP headers, which are
	  sent for each fragment. When using the Modem library and TLS,
	  the fragment size may not exceed 2 kB minus the largest HTTP header
	  the application expects to receive.

Did you test with e.g. CONFIG_DOWNLOAD_CLIENT_HTTP_FRAG_SIZE_512 (or 256) and CONFIG_DOWNLOAD_CLIENT_BUF_SIZE=4096?

Kind regards,
Øyvind