• State Not Answered
  • Replies 58 replies
  • Subscribers 74 subscribers
  • Views 2458 views
  • Users 0 members are here
Attachments (28) Download All
Nordic Case Info
  • Case ID: 345781
Options

Enabling the TLS layer to get a HTTPS connection going.

Tudor B.

7343.nrf7002dk_nrf5340_cpuapp_ns.conf3124.prj.confHello everyone.

WE're trying to make a https connection with google.com and execute a GET request.

Wifi connection is working; DHCP seems to be working (my personal assumption given the log message we get: "Resolved: [(1, 1, 6, '', ('142.250.201.206', 443))]" which indicates that getaddrinfo() works); but when trying to initiate the socket via TLS, something strange happens: we get the error "OSError: 109".

Inserting some debug prints inside subsys/net/lib/sockets/, we found the culprit to be the function "int zsock_setsockopt_ctx(struct net_context *ctx, int level, int optname, const void *optval, socklen_t optlen)".

The function call that triggers error 109 is:  res = setsockopt(socket->ctx, SOL_TLS, TLS_PEER_VERIFY, &verify, sizeof(verify));

No matter what other option we try to set via setsockopt(), it will fail with the 109 error since the implementation for setsockopt() is somehow set to sockets_inet.c (whose implementation does not recognise SOL_TLS as a valid in its switches) instead of sockets_tls.c (which has handling for SOL_TLS in its switches). My personal hunch is that the config options set in the project are somehow wrong. Can someone please take a look over our .conf files? Maybe we can find the culprit. :)

We can provide any extra code snippets that are necessary for debugging and/ or run any tests. Have a great day and hope to hear from you soon!

Parents
  • 0

    Hi,

     

    I used net/https_client for this exercise.

    You need to download r1.pem from here: https://pki.goog/repository/

     

    Place this in certs/ folder, and make sure that you change the file in CMakeLists.txt, change the domain in kconfig, and add the required configurations in the board .conf file:

    diff --git a/samples/net/https_client/CMakeLists.txt b/samples/net/https_client/CMakeLists.txt
index 2a937786ed..39276fd2e2 100644
--- a/samples/net/https_client/CMakeLists.txt
+++ b/samples/net/https_client/CMakeLists.txt
@@ -14,7 +14,7 @@ set(gen_dir ${CMAKE_CURRENT_BINARY_DIR}/certs)
 zephyr_include_directories(${gen_dir})
 generate_inc_file_for_target(
     app
-    cert/DigiCertGlobalG2.pem
+    cert/r1.pem
     ${gen_dir}/DigiCertGlobalG2.pem.inc
     )
 
diff --git a/samples/net/https_client/Kconfig b/samples/net/https_client/Kconfig
index 90ad33f42e..bb22e82794 100644
--- a/samples/net/https_client/Kconfig
+++ b/samples/net/https_client/Kconfig
@@ -15,7 +15,7 @@ config SAMPLE_TFM_MBEDTLS
 
 config HTTPS_HOSTNAME
        string "HTTPS hostname"
-       default "example.com"
+       default "google.com"
 
 endmenu
 
diff --git a/samples/net/https_client/boards/nrf7002dk_nrf5340_cpuapp_ns.conf b/samples/net/https_client/boards/nrf7002dk_nrf5340_cpuapp_ns.conf
index 9eb362cb16..8366313af8 100644
--- a/samples/net/https_client/boards/nrf7002dk_nrf5340_cpuapp_ns.conf
+++ b/samples/net/https_client/boards/nrf7002dk_nrf5340_cpuapp_ns.conf
@@ -69,3 +69,20 @@ CONFIG_MBEDTLS_TLS_LIBRARY=y
 CONFIG_TFM_PROFILE_TYPE_SMALL=y
 CONFIG_PM_PARTITION_SIZE_TFM_SRAM=0xc000
 CONFIG_PM_PARTITION_SIZE_TFM=0x20000
+
+CONFIG_MBEDTLS_SSL_SERVER_NAME_INDICATION=y
+CONFIG_MBEDTLS_SSL_RENEGOTIATION=y
+CONFIG_MBEDTLS_SSL_MAX_FRAGMENT_LENGTH=y
+CONFIG_MBEDTLS_SSL_SESSION_TICKETS=y
+CONFIG_PSA_WANT_RSA_KEY_SIZE_4096=y
+CONFIG_MBEDTLS_MPI_MAX_SIZE=512
+
+CONFIG_LOG=y
+CONFIG_MBEDTLS_DEBUG=y
+CONFIG_MBEDTLS_SSL_DEBUG_ALL=y
+CONFIG_MBEDTLS_LOG_LEVEL_DBG=y
+CONFIG_MBEDTLS_DEBUG_C=y
+CONFIG_MBEDTLS_DEBUG_LEVEL=4
+# Handle the large influx of prints
+CONFIG_LOG_BUFFER_SIZE=16384
+CONFIG_LOG_BACKEND_UART=y

    I also need to add CONFIG_NET_IPV6=n due to a local network issue at my end.

     

    Kind regards,

    Håkon

  • 0 in reply to Håkon Alseth

    There are many options and suboptions in the link you sent me. Which one is the correct one?

    When attempting to get it working, I got r1.der and then created r1.der.inc. But I'm not sure which option I chose.

  • 0 in reply to Tudor B.

    Hi,

     

    Tudor B. said:
    it could still be done in a single day

    Integrating a unsupported device into a project is not done in a day.

     

    Micropython is a platform that Nordic do not actively use for development. You are ofcourse free to make your decisions towards what you want to integrate into your own product, but please be aware that micropython is not a officially supported platform from us.

    The official recommendation from our side is to use nRF Connect SDK with our devices, and you have this working via a modified https_client sample towards your wanted domain.

     

    I will try to help you to ensure that your desired behavior is running in micropython, but to be able to do that, I must be able to compile and recreate the scenario at my end.

    This is a awfully complex process when I get sent one file after the other, when you could give me a .patch or .diff file to implement your local changes directly into the micropy tree.

    I am still seeing compilation issues and I'm quite certain that I have not placed your files into the correct folder/path, or performed something incorrectly.

     

    Can you please provide patches/diff files for your changes into the micropy repo?

    And please also share the hash that you've based your changes upon.

     

    You are currently using in-place logging, via this config:

    CONFIG_LOG_MODE_IMMEDIATE=y

    I understand why you choose this, but please be aware what this actually means. This means that the log entry occurs in the same context as the log message originated, and if this is logging large message quantities in a high priority thread/irq, it will block until the message is flushed.

    Be very aware that this will, due to its blocking behavior, cause timing related errors to emerge, like this hostname verification error:

    <err> net_sock_tls: TLS handshake error: -0x2700

    Try running without mbedtls debug logs, or adjust the log level:

    CONFIG_MBEDTLS_LOG_LEVEL_ERR=y
CONFIG_MBEDTLS_DEBUG_C=y
CONFIG_MBEDTLS_DEBUG_LEVEL=1

      

    Kind regards,

    Håkon

  • 0 in reply to Håkon Alseth
    Håkon Alseth said:
    Can you please provide patches/diff files for your changes into the micropy repo?

    Hey Hakon.

    I see your point but I think it's gonna be really hard to give you the diff since our repo is a private repo that branched off the main micropython repo ~6-8 months ago. In the meantime both our and the main micropython branches were updated a lot, so it will be hard to tell the actual differences. I ended up getting the micropython repo locally and doing a diff, and this is the diff just for the /ports/zephyr folder:
    zephyr_diff.txt.zip

    It has ~184MB when uncompressed and ~4 million lines. :))

    That's why I wanted to do it simply on my side.

    Håkon Alseth said:
    Try running without mbedtls debug logs, or adjust the log level:

    Interesting point! I disabled CONFIG_LOG_MODE_IMMEDIATE and applied the 3 CONFIG_ changes that you mentioned. Sadly the behaviour seems to be the same, but with less logs:

    00:08:15.788,177] <inf> wifimod: Connection requested
[00:08:15.788,238] <inf> wifimod: ==================
[00:08:15.788,269] <inf> wifimod: State: SCANNING
[00:08:15.788,330] <inf> wifimod: Net If state: 5
[00:08:16.088,470] <inf> wifimod: ==================
[00:08:16.088,500] <inf> wifimod: State: AUTHENTICATING
[00:08:16.088,562] <inf> wifimod: Net If state: 5
[00:08:16.238,159] <dbg> net_sock_packet: zpacket_received_cb: (rx_q[0]): ctx=0x2000f4b8, pkt=0x20063718, st=0, user_data=(nil)
[00:08:16.253,173] <dbg> net_sock_packet: zpacket_received_cb: (rx_q[0]): ctx=0x2000f4b8, pkt=0x200636d8, st=0, user_data=(nil)
[00:08:16.270,080] <inf> wifimod: Wi-Fi connect result: status...
[00:08:16.270,111] <inf> wifimod: Connected
[00:08:16.308,197] <dbg> net_sock: zsock_close_ctx: (rx_q[0]): close: ctx=0x2000f358, fd=3
[00:08:16.308,319] <dbg> net_sock: zsock_close_ctx: (rx_q[0]): close: ctx=0x2000f408, fd=5
[00:08:16.308,807] <dbg> net_sock: zsock_socket_internal: (rx_q[0]): socket: ctx=0x2000f358, fd=3
[00:08:16.309,387] <dbg> net_sock: zsock_socket_internal: (rx_q[0]): socket: ctx=0x2000f408, fd=5
[00:08:16.339,Waiting for IP address...
752] <inf> net_dhcpv4: Received: 192.168.0.250
[00:08:16.339,965] <inf> net_config: IPv4 address: 192.168.0.250
[00:08:16.339,965] <inf> net_config: Lease time: 7200 seconds
[00:08:16.339,996] <inf> net_config: Subnet: 255.255.255.0
[00:08:16.340,026] <inf> net_config: Router: 192.168.0.1
[00:08:16.340,118] <inf> wifimod: Net MGMT: Got IP via DHCP
DHCP IP address: 192.168.0.250
[00:08:16.389,221] <dbg> net_sock_svc: socket_service_thread: (net_socket_service): Received restart event.
Resolving google.com...
Resolved: [(1, 1, 6, '', ('142.251.39.14', 443))]
Success!
Using system DNS resolver...
Querying DNS for google.com (type 1)...
[00:08:17.405,334] <dbg> net_dns_resolve: dns_write: (mp_main): [0] submitting work to server idx 0 for id 62375 hash 59079
semaphore wait...1
[00:08:17.649,932] <dbg> net_sock: zsock_received_cb: (rx_q[0]): ctx=0x2000f358, pkt=0x200636d8, st=0, user_data=(nil)
DNS CALLBACK: status=-100
DNS CALLBACK: status=-103
Giving semaphore on info == NULL
something...1
Trying to initialize socket...
Family: 1, socktype: 1, proto: 259
[00:08:17.652,130] <dbg> net_sock_tls: tls_alloc: (mp_main): Allocated TLS context, 0x2000d3d8
[00:08:17.652,954] <dbg> net_sock: zsock_socket_internal: (mp_main): socket: ctx=0x2000f568, fd=16
Returned value: 15
Done initializing socket!
Performing TLS setup...
TLS setup complete.
Connecting to: ('142.251.39.14', 443)
Trying 1...
1 is done
2 is done
3 is done
[00:08:19.712,554] <dbg> mbedtls: zephyr_mbedtls_debug: WEST_TOPDIR/modules/crypto/mbedtls/library/ssl_tls.c:1331: The SSL configuration is tls12 only.
[00:08:19.717,498] <wrn> mbedtls: WEST_TOPDIR/modules/crypto/mbedtls/library/ssl_tls.c:4661: => handshake
[00:08:19.717,559] <wrn> mbedtls: WEST_TOPDIR/modules/crypto/mbedtls/library/ssl_msg.c:2354: => flush output
[00:08:19.717,590] <wrn> mbedtls: WEST_TOPDIR/modules/crypto/mbedtls/library/ssl_msg.c:2363: <= flush output
--- 57 messages dropped ---
[00:08:19.723,052] <inf> mbedtls: WEST_TOPDIR/modules/crypto/mbedtls/library/ssl_tls12_client.c:0150: client hello, adding ecjpake_kkpp extension
--- 38 messages dropped ---
[00:08:19.830,047] <dbg> mbedtls: zephyr_mbedtls_debug: WEST_TOPDIR/modules/crypto/mbedtls/library/ssl_msg.c:3033: 0040:  c0 09 c0 13 00 33 00 3d 00 35 c0 2a c0 0f c0 26  .....3.=.5.*...&
--- 60 messages dropped ---
[00:08:19.892,761] <dbg> mbedtls: zephyr_mbedtls_debug: WEST_TOPDIR/modules/crypto/mbedtls/library/ssl_msg.c:4073: dumping 'input record from network' (68 bytes)
--- 12 messages dropped ---
[00:08:19.895,996] raceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "sock_test.py", line 68, in step_1
OSError: [Errno 113] ECONNABORTED

  • 0 in reply to Tudor B.

    Hey Håkon,

    We just had a call with Alexander Rawstone, in which we provided him with the local version of the https_client sample project. If you recall from earlier in this thread, I tried setting the host in the https_client  to google.com and adding the certificate that you mentioned but the connection didn't work out. I undid the hostname in the Kconfig for the https_sample, so it's back to example.com. I left some google certs in the project there (mainly I used r1, but I tried others also). So, when I wanted to try google.com as a host, I changed the Kconfig, made a copy of r1.pem and then renamed that copy to DigiCertGlobalG3.pem.

     

  • 0 in reply to Tudor B.

    Hi,

      

    Tudor B. said:
    We just had a call with Alexander Rawstone, in which we provided him with the local version of the https_client sample project. If you recall from earlier in this thread, I tried setting the host in the https_client  to google.com and adding the certificate that you mentioned but the connection didn't work out.

    If you are running NCS v3.0.0, you need to add:

    CONFIG_MBEDTLS_RSA_C=y

    Since you already have the changes in-place, I suspect that this is the only option that you're missing. It shall then print something like this:

    *** Booting nRF Connect SDK v3.0.2-89ba1294ac9b ***
*** Using Zephyr OS v4.0.99-f791c49f492c ***
HTTPS client sample started
Bringing network interface up
Provisioning certificate
CA certificate already exists, sec tag: 42
Connecting to the network
Connected
Network connectivity established and IP address assigned
Looking up google.com
Resolved 216.58.207.238 (AF_INET)
Connecting to google.com:443
Sent 60 bytes
Received 631 bytes

>        HTTP/1.1 301 Moved Permanently

Finished, closing socket.
Disconnected
Network connectivity lost
Disconnected from the network
uart:~$

    I will post the full change, for simplicity.

    Here's a git .patch file showing the changes needed for net/https_client in ncs v3.0.0:

    https_client_ncs3.0_google.patch 

    place this in the path/to/ncs3.0.0/nrf/samples/net/https_client/ folder, and write:

    git apply https_client_ncs3.0_google.patch

     

    If your tree is unmodified, it shall apply cleanly.

     

    Note: I am disabling IPv6 on my end. This is an local network issue with the test-network that I am using, so disabling CONFIG_NET_IPV6 is optional.

     

    Kind regards,

    Håkon

  • 0 in reply to Håkon Alseth

    Sadly it still doesn't work:

    *** Booting nRF Connect SDK v3.0.0-3bfc46578e42 ***
*** Using Zephyr OS v4.0.99-a0e545cb437a ***
HTTPS client sample started
Bringing network interface up
Provisioning certificate
CA certificate already exists, sec tag: 42
Connecting to the network
uart:~$
uart:~$
uart:~$
uart:~$
uart:~$
uart:~$
Connected
Network connectivity established and IP address assigned
Looking up google.com
Resolved 142.251.39.78 (AF_INET)
Connecting to google.com:443
Dead here...56. ret value = -9984
Dead here...66. cert_failed = 1
connect() failed, err: 113
Disconnected
Network connectivity lost
Disconnected from the network

    Edit 1: the two "Dead here..." messages that you see are prints that I included somewhere in ssl_tls.c to see what can potentially cause the TLS handshake to fail. I can comment them but I'm pretty sure the behaviour would be the same. I also want to confirm: the r1.pem is the one that was in that archive, right? More precisely this:

    r1.pem.zip

    Edit 2: Since I have no more ideas, I applied your changes, built and flashed the target and it didn't work, I decided to simply archive and add the sample project here, which will include the build folder:

    1581.https_client.zip

Reply
  • 0 in reply to Håkon Alseth

    Sadly it still doesn't work:

    *** Booting nRF Connect SDK v3.0.0-3bfc46578e42 ***
*** Using Zephyr OS v4.0.99-a0e545cb437a ***
HTTPS client sample started
Bringing network interface up
Provisioning certificate
CA certificate already exists, sec tag: 42
Connecting to the network
uart:~$
uart:~$
uart:~$
uart:~$
uart:~$
uart:~$
Connected
Network connectivity established and IP address assigned
Looking up google.com
Resolved 142.251.39.78 (AF_INET)
Connecting to google.com:443
Dead here...56. ret value = -9984
Dead here...66. cert_failed = 1
connect() failed, err: 113
Disconnected
Network connectivity lost
Disconnected from the network

    Edit 1: the two "Dead here..." messages that you see are prints that I included somewhere in ssl_tls.c to see what can potentially cause the TLS handshake to fail. I can comment them but I'm pretty sure the behaviour would be the same. I also want to confirm: the r1.pem is the one that was in that archive, right? More precisely this:

    r1.pem.zip

    Edit 2: Since I have no more ideas, I applied your changes, built and flashed the target and it didn't work, I decided to simply archive and add the sample project here, which will include the build folder:

    1581.https_client.zip

Children
Related
Terms of service