Enabling the TLS layer to get a HTTPS connection going.

[Tudor B.]

7343.nrf7002dk_nrf5340_cpuapp_ns.conf3124.prj.confHello everyone.

WE're trying to make a https connection with google.com and execute a GET request.

Wifi connection is working; DHCP seems to be working (my personal assumption given the log message we get: "Resolved: [(1, 1, 6, '', ('142.250.201.206', 443))]" which indicates that getaddrinfo() works); but when trying to initiate the socket via TLS, something strange happens: we get the error "OSError: 109".

Inserting some debug prints inside subsys/net/lib/sockets/, we found the culprit to be the function "int zsock_setsockopt_ctx(struct net_context *ctx, int level, int optname, const void *optval, socklen_t optlen)".

The function call that triggers error 109 is:  res = setsockopt(socket->ctx, SOL_TLS, TLS_PEER_VERIFY, &verify, sizeof(verify));

No matter what other option we try to set via setsockopt(), it will fail with the 109 error since the implementation for setsockopt() is somehow set to sockets_inet.c (whose implementation does not recognise SOL_TLS as a valid in its switches) instead of sockets_tls.c (which has handling for SOL_TLS in its switches). My personal hunch is that the config options set in the project are somehow wrong. Can someone please take a look over our .conf files? Maybe we can find the culprit. :)

We can provide any extra code snippets that are necessary for debugging and/ or run any tests. Have a great day and hope to hear from you soon!

[Håkon Alseth]

Hi,

 

I used net/https_client for this exercise.

You need to download r1.pem from here: https://pki.goog/repository/

 

Place this in certs/ folder, and make sure that you change the file in CMakeLists.txt, change the domain in kconfig, and add the required configurations in the board .conf file:

diff --git a/samples/net/https_client/CMakeLists.txt b/samples/net/https_client/CMakeLists.txt
index 2a937786ed..39276fd2e2 100644
--- a/samples/net/https_client/CMakeLists.txt
+++ b/samples/net/https_client/CMakeLists.txt
@@ -14,7 +14,7 @@ set(gen_dir ${CMAKE_CURRENT_BINARY_DIR}/certs)
 zephyr_include_directories(${gen_dir})
 generate_inc_file_for_target(
     app
-    cert/DigiCertGlobalG2.pem
+    cert/r1.pem
     ${gen_dir}/DigiCertGlobalG2.pem.inc
     )
 
diff --git a/samples/net/https_client/Kconfig b/samples/net/https_client/Kconfig
index 90ad33f42e..bb22e82794 100644
--- a/samples/net/https_client/Kconfig
+++ b/samples/net/https_client/Kconfig
@@ -15,7 +15,7 @@ config SAMPLE_TFM_MBEDTLS
 
 config HTTPS_HOSTNAME
        string "HTTPS hostname"
-       default "example.com"
+       default "google.com"
 
 endmenu
 
diff --git a/samples/net/https_client/boards/nrf7002dk_nrf5340_cpuapp_ns.conf b/samples/net/https_client/boards/nrf7002dk_nrf5340_cpuapp_ns.conf
index 9eb362cb16..8366313af8 100644
--- a/samples/net/https_client/boards/nrf7002dk_nrf5340_cpuapp_ns.conf
+++ b/samples/net/https_client/boards/nrf7002dk_nrf5340_cpuapp_ns.conf
@@ -69,3 +69,20 @@ CONFIG_MBEDTLS_TLS_LIBRARY=y
 CONFIG_TFM_PROFILE_TYPE_SMALL=y
 CONFIG_PM_PARTITION_SIZE_TFM_SRAM=0xc000
 CONFIG_PM_PARTITION_SIZE_TFM=0x20000
+
+CONFIG_MBEDTLS_SSL_SERVER_NAME_INDICATION=y
+CONFIG_MBEDTLS_SSL_RENEGOTIATION=y
+CONFIG_MBEDTLS_SSL_MAX_FRAGMENT_LENGTH=y
+CONFIG_MBEDTLS_SSL_SESSION_TICKETS=y
+CONFIG_PSA_WANT_RSA_KEY_SIZE_4096=y
+CONFIG_MBEDTLS_MPI_MAX_SIZE=512
+
+CONFIG_LOG=y
+CONFIG_MBEDTLS_DEBUG=y
+CONFIG_MBEDTLS_SSL_DEBUG_ALL=y
+CONFIG_MBEDTLS_LOG_LEVEL_DBG=y
+CONFIG_MBEDTLS_DEBUG_C=y
+CONFIG_MBEDTLS_DEBUG_LEVEL=4
+# Handle the large influx of prints
+CONFIG_LOG_BUFFER_SIZE=16384
+CONFIG_LOG_BACKEND_UART=y

I also need to add CONFIG_NET_IPV6=n due to a local network issue at my end.

 

Kind regards,

Håkon

[Tudor B.]

There are many options and suboptions in the link you sent me. Which one is the correct one?

When attempting to get it working, I got r1.der and then created r1.der.inc. But I'm not sure which option I chose.

[Tudor B.]

Hey Håkon,

We just had a call with Alexander Rawstone, in which we provided him with the local version of the https_client sample project. If you recall from earlier in this thread, I tried setting the host in the https_client  to google.com and adding the certificate that you mentioned but the connection didn't work out. I undid the hostname in the Kconfig for the https_sample, so it's back to example.com. I left some google certs in the project there (mainly I used r1, but I tried others also). So, when I wanted to try google.com as a host, I changed the Kconfig, made a copy of r1.pem and then renamed that copy to DigiCertGlobalG3.pem.

 

[Håkon Alseth]

Hi,

  

Tudor B. said:

We just had a call with Alexander Rawstone, in which we provided him with the local version of the https_client sample project. If you recall from earlier in this thread, I tried setting the host in the https_client  to google.com and adding the certificate that you mentioned but the connection didn't work out.

If you are running NCS v3.0.0, you need to add:

CONFIG_MBEDTLS_RSA_C=y

Since you already have the changes in-place, I suspect that this is the only option that you're missing. It shall then print something like this:

*** Booting nRF Connect SDK v3.0.2-89ba1294ac9b ***
*** Using Zephyr OS v4.0.99-f791c49f492c ***
HTTPS client sample started
Bringing network interface up
Provisioning certificate
CA certificate already exists, sec tag: 42
Connecting to the network
Connected
Network connectivity established and IP address assigned
Looking up google.com
Resolved 216.58.207.238 (AF_INET)
Connecting to google.com:443
Sent 60 bytes
Received 631 bytes

>        HTTP/1.1 301 Moved Permanently

Finished, closing socket.
Disconnected
Network connectivity lost
Disconnected from the network
uart:~$ 

I will post the full change, for simplicity.

Here's a git .patch file showing the changes needed for net/https_client in ncs v3.0.0:

https_client_ncs3.0_google.patch 

place this in the path/to/ncs3.0.0/nrf/samples/net/https_client/ folder, and write:

git apply https_client_ncs3.0_google.patch

 

If your tree is unmodified, it shall apply cleanly.

 

Note: I am disabling IPv6 on my end. This is an local network issue with the test-network that I am using, so disabling CONFIG_NET_IPV6 is optional.

 

Kind regards,

Håkon

[Tudor B.]

Sadly it still doesn't work:

*** Booting nRF Connect SDK v3.0.0-3bfc46578e42 ***
*** Using Zephyr OS v4.0.99-a0e545cb437a ***
HTTPS client sample started
Bringing network interface up
Provisioning certificate
CA certificate already exists, sec tag: 42
Connecting to the network
uart:~$
uart:~$
uart:~$
uart:~$
uart:~$
uart:~$
Connected
Network connectivity established and IP address assigned
Looking up google.com
Resolved 142.251.39.78 (AF_INET)
Connecting to google.com:443
Dead here...56. ret value = -9984
Dead here...66. cert_failed = 1
connect() failed, err: 113
Disconnected
Network connectivity lost
Disconnected from the network

Edit 1: the two "Dead here..." messages that you see are prints that I included somewhere in ssl_tls.c to see what can potentially cause the TLS handshake to fail. I can comment them but I'm pretty sure the behaviour would be the same. I also want to confirm: the r1.pem is the one that was in that archive, right? More precisely this:

r1.pem.zip

Edit 2: Since I have no more ideas, I applied your changes, built and flashed the target and it didn't work, I decided to simply archive and add the sample project here, which will include the build folder:

1581.https_client.zip

[Håkon Alseth]

Hi,

  

this line:

Dead here...56. ret value = -9984

is -0x2700 in hex, ie, mbedtls verification error.

The r1.pem.zip holds something that differs from the one downloaded from google:

https://i.pki.goog/r1.pem

 

Could you triple check that you're using the correct root CA?

 

Kind regards,

Håkon

[Tudor B.]

Checking now! :D

[Tudor B.]

You were right indeed: the cert that you linked was different to the one I had.

I downloaded twice the certificate from the link you just gave me. Replaced the one in the cert folder twice, Did a --pristine build and then a flash twice and I still got exactly the same error.

[Håkon Alseth]

Hi,

 

Here's my https_client precompiled .hex file:

33171.merged.hex

 

Could you try this one and share the results?

you can flash it either via the nRF connect for desktop -> Programmer application, or use nrfjprog directly:

nrfjprog --program merged.hex --sectorerase --verify -f nrf53

 

Kind regards,

Håkon 

[Tudor B.]

Thanks for the hasty replies Håkon!

Log of flashing:

root@Tudors-MacBook https_client # nrfjprog --program merged.hex --sectorerase --verify -f nrf53
[ #################### ]  23.186s | Erase file - Done erasing
[ #################### ]   5.406s | Program file - Done programming
[ #################### ]   5.455s | Verify file - Done verifying
root@Tudors-MacBook https_client #

Log of it running:

*** Booting nRF Connect SDK v3.0.2-89ba1294ac9b ***
*** Using Zephyr OS v4.0.99-f791c49f492c ***
HTTPS client sample started
Bringing network interface up
Provisioning certificate
CA certificate already exists, sec tag: 42
Connecting to the network
Connected
Network connectivity established and IP address assigned
Looking up google.com
Resolved 142.251.208.110 (AF_INET)
Connecting to google.com:443
connect() failed, err: 113
Disconnected
Network connectivity lost
Disconnected from the network
uart:~$

[Tudor B.]

CA certificate already exists, sec tag: 42

Could it be that by adding the wrong r1.pem file, now when I added the right one, MbedTLS doesn't accept it since it detects that the new one is from google and it already has a google CA inside?

[Håkon Alseth]

Hi,

 

Good catch. That might be the problem.

You can test that theory if you run a "nrfjprog --recover", then flash the .hex that I shared.

  

Kind regards,

Håkon