nRF5340 + download_client + HTTPS guide needed

[Tamas Selmeci]

Hello!

I need to use download_client to get files from an HTTPS server. I'm stuck in the implementation process - I'm lost when it comes to TLS etc. I tried to extract some precious details from the example in samples/net/sockets/http_client. MBEDTLS etc are enabled in Kconfig.

The following steps were taken:

1. generated a CA certificate:

openssl genrsa 2048 > ca-key.pem

openssl req -new -x509 -nodes -days 365000 -key ca-key.pem -out ca-cert.pem

2. generated the .DER file:

openssl base64 -d -in ca-cert.pem -out ca-cert.der

3. with my tool (which is quite similar to file2hex.py) it was converted to a C array (static const uint8_t ca_cert[] = {...});

4. included in my source (#include "ca_cert.h");

5. initiated the download:

int cfg_sec_tag_list[] = {1};
struct download_client_cfg cfg = {
 .sec_tag_list = cfg_sec_tag_list,
 .sec_tag_count = ARRAY_SIZE(cfg_sec_tag_list),
 .pdn_id = 0,
 .frag_size_override = 0,
 .set_tls_hostname = 0,
};

...
...

tls_credential_add(cfg_sec_tag_list[0],
 TLS_CREDENTIAL_CA_CERTIFICATE,
 ca_cert,
 sizeof(ca_cert));
download_client_init(&dlc, c_cb);
download_client_get(&dlc, "">https://my.url/file", &cfg, NULL, 0);

6. got these results:

[00:00:28.969,543] <dbg> download_client: set_state: state = 1
[00:00:28.969,573] <inf> download_client: Downloading: https://my.url/file [0]
[00:00:29.183,044] <dbg> download_client: client_connect: Port not specified, using default: 443
[00:00:29.183,074] <dbg> download_client: client_connect: family: 1, type: 1, proto: 258
[00:00:29.183,715] <inf> download_client: Setting up TLS credentials, sec tag count 1
[00:00:29.183,746] <inf> download_client: Connecting to https://my.url/file
[00:00:29.183,776] <dbg> download_client: client_connect: fd 5, addrlen 8, fam IPv4, port 443
[00:00:29.412,231] <err> download_client: Unable to connect, errno 22
[00:00:29.412,658] <dbg> download_client: set_state: state = 0

--------------------------------------------------------------------

I believe this must be something TLS-related, but I'm very rookie in this field.

Please give me a guide how to setup download_client to be able to get files via HTTPS on the simplest way. 

[Sigurd Hellesvik]

Hi,

First, try to run the HTTPS Client sample out of the box. Does this work?
Then, I can suggest some things depending on how this goes.

Regards,
Sigurd Hellesvik

[Tamas Selmeci]

Hello Sigurd!

Thanks for the hint. Some hardening conditions:

• I can't run it, because we don't have such a HW (nRF91);
• our modem is based on a custom driver, so can't use nRF modem lib;
• we're using the oldish ncs-2.4.1, so the sample you referred to doesn't exist, instead there's something probably similar in nrf/samples/nrf9160/https_client/src.  Please confirm if I'm looking at the right program;
  • however, there are some interesting steps in tls_setup(...) which appear to setup up socket with TLS;
• so this all must be working with our custom modem driver (based on top of gsm_ppp.c) + without modemlib + without nRF modem tools + with MbedTLS + with download client;
• I had another glance on download_client code, appears to do the same steps as the sample;

As I mentioned before, I'm very rookie in the field of TLS etc. From what I've found I believe that I need to provide download_client the appropriate root certificate for each HTTPS site I want to connect to. This explains why download_client was unable to connect to an HTTPS-site with my self-generated cert :D Please confirm if I'm right.

Additionally, if I'm not totally wrong, I can use my browser to download the CA cert from the HTTPS site I want to connect to with download_client. Then this PEM must be converted to DER (binary format), which must be included in my program and must be passed to TLS subsystem (tls_credential_add(...)), and finally download_client will be able to make use of it? Is it correct?

---------------------------------------------------------------------

UPDATE

I tried to connect to example.com with the CA cert from NCS and got the following:

[00:00:23.333,282] <dbg> download_client: set_state: state = 1
[00:00:23.333,312] <inf> download_client: Downloading: www.example.com/index.html [0]
[00:00:23.526,885] <dbg> download_client: client_connect: Port not specified, using default: 443
[00:00:23.526,916] <dbg> download_client: client_connect: family: 1, type: 1, proto: 258
[00:00:23.527,496] <inf> download_client: Setting up TLS credentials, sec tag count 1
[00:00:23.527,526] <inf> download_client: Connecting to www.example.com/index.html
[00:00:23.527,557] <dbg> download_client: client_connect: fd 5, addrlen 8, fam IPv4, port 443
[00:00:23.637,054] <wrn> mbedtls: ssl_msg.c:2937: <= write record
--- 33 messages dropped ---
[00:00:23.637,084] <wrn> mbedtls: ssl_msg.c:2754: <= write handshake message
[00:00:23.637,145] <wrn> mbedtls: ssl_client.c:0996: <= write client hello
[00:00:23.637,176] <wrn> mbedtls: ssl_msg.c:2177: => flush output
[00:00:23.637,268] <wrn> mbedtls: ssl_msg.c:2194: message length: zu, out_left: zu
[00:00:23.637,817] <wrn> mbedtls: ssl_msg.c:2201: ssl->f_send() returned 77 (-0xffffffb3)
[00:00:23.637,878] <wrn> mbedtls: ssl_msg.c:2229: <= flush output
[00:00:23.637,939] <wrn> mbedtls: ssl_tls.c:3709: client state: MBEDTLS_SSL_SERVER_HELLO
[00:00:23.638,000] <wrn> mbedtls: ssl_tls12_client.c:1231: => parse server hello
[00:00:23.638,031] <wrn> mbedtls: ssl_msg.c:4002: => read record
[00:00:23.638,061] <wrn> mbedtls: ssl_msg.c:1962: => fetch input
[00:00:23.638,153] <wrn> mbedtls: ssl_msg.c:2116: in_left: zu, nb_want: zu
[00:00:23.638,214] <wrn> mbedtls: ssl_msg.c:2141: in_left: zu, nb_want: zu
[00:00:23.638,275] <wrn> mbedtls: ssl_tls.c:3801: <= handshake
[00:00:23.776,672] <wrn> mbedtls: ssl_msg.c:4002: => read record
--- 41 messages dropped ---
[00:00:23.776,733] <wrn> mbedtls: ssl_msg.c:1962: => fetch input
[00:00:23.776,794] <wrn> mbedtls: ssl_msg.c:2116: in_left: zu, nb_want: zu
[00:00:23.777,038] <wrn> mbedtls: ssl_msg.c:2141: in_left: zu, nb_want: zu
[00:00:23.777,130] <wrn> mbedtls: ssl_msg.c:2144: ssl->f_recv(_timeout)() returned 5 (-0xfffffffb)
[00:00:23.777,160] <wrn> mbedtls: ssl_msg.c:2164: <= fetch input
[00:00:23.777,465] <inf> mbedtls: ssl_msg.c:3733: input record: msgtype = 22, version = [0x303], msglen = zu
[00:00:23.777,526] <wrn> mbedtls: ssl_msg.c:1962: => fetch input
[00:00:23.777,587] <wrn> mbedtls: ssl_msg.c:2116: in_left: zu, nb_want: zu
[00:00:23.777,923] <wrn> mbedtls: ssl_msg.c:2141: in_left: zu, nb_want: zu
[00:00:23.778,015] <wrn> mbedtls: ssl_msg.c:2144: ssl->f_recv(_timeout)() returned 544 (-0xfffffde0)
[00:00:23.778,106] <wrn> mbedtls: ssl_msg.c:2141: in_left: zu, nb_want: zu
[00:00:23.778,167] <wrn> mbedtls: ssl_tls.c:3801: <= handshake
00:00:00:23.863|GCT|DEV main.c:1724 !!! conn klozet
00:00:00:23.864|GCT|DEV main.c:1715 !!! DL error -113
[00:00:23.843,322] <wrn> mbedtls: ssl_msg.c:2116: in_left: zu, nb_want: zu
--- 11 messages dropped ---
[00:00:23.843,719] <wrn> mbedtls: ssl_msg.c:2141: in_left: zu, nb_want: zu
[00:00:23.843,811] <wrn> mbedtls: ssl_msg.c:2144: ssl->f_recv(_timeout)() returned 480 (-0xfffffe20)
[00:00:23.843,872] <wrn> mbedtls: ssl_msg.c:2164: <= fetch input
[00:00:23.861,572] <inf> mbedtls: ssl_msg.c:3088: handshake message: msglen = zu, type = 1024, hslen = zu
[00:00:23.861,633] <err> mbedtls: ssl_msg.c:3162: TLS handshake fragmentation not supported
[00:00:23.861,724] <err> mbedtls: ssl_msg.c:4060: mbedtls_ssl_handle_message_type() returned -28800 (-0x7080)
[00:00:23.861,816] <err> mbedtls: ssl_tls.c:7348: mbedtls_ssl_read_record() returned -28800 (-0x7080)
[00:00:23.861,877] <wrn> mbedtls: ssl_tls.c:3801: <= handshake
[00:00:23.862,823] <err> download_client: Unable to connect, errno 113
[00:00:23.862,854] <wrn> mbedtls: ssl_msg.c:5953: => write close notify
[00:00:23.862,915] <wrn> mbedtls: ssl_msg.c:5966: <= write close notify
[00:00:23.864,135] <dbg> download_client: set_state: state = 0

See the bolded lines. Momentarily I'm unable to enable TLS handshake fragmentation...

Thanks,

[Sigurd Hellesvik]

Tamas Selmeci said:

I tried to connect to example.com with the CA cert from NCS and got the following:

Ah. In future versions of the NCS, the https client sample is generic, and works for both nRF9160 and nRF5340+nRF7002 (Wi-Fi).

However, since you are on an older version and has a custom setup, I understand that you cannot test this sample.

Tamas Selmeci said:

tried to connect to example.com with the CA cert from NCS and got the following

22 is in my experience often that the certs are wrong.

And yes, I agree that your understanding is correct, you need the root CA certificate for the webpage you want to connect to, so that your application can verify the trust of the webpage.

However, from the fragmentation error you got, try perhaps to change CONFIG_MBEDTLS_SSL_MAX_FRAGMENT_LENGTH