nrf5340 NCS 2.8 cannot get TLS socket to load CA certif due to net_sock_tls / mbedtls error -0x262e

[BrianW]

Using Zephyr in NCS 2.8 on nrf5340/nrf7002 wifi combo, and trying to get https to work. I'm using the option where the TLS setup is done in the socket layer (enabled in prj.conf)

# use TLS in socket directly

CONFIG_NET_SOCKETS_SOCKOPT_TLS=y

CONFIG_TLS_CREDENTIALS=y

CONFIG_TLS_MAX_CREDENTIALS_NUMBER=10

CONFIG_NET_SOCKETS_TLS_MAX_CREDENTIALS=10

However, I cannot get this to parse a server certificate (.der or .pem formats). Every time it fails when it tries to parse the certificate to load the url, with the following:

[00:00:24.298,767] <inf> app: host [timeapi.io/api/time/current/zone?timeZone=Europe%2FParis&id="app.sysctrl.checkhttp"&value="00:00:00"] port [NULL] path [/api/time/current/zone?timeZone=Europe%2FParis&id="app.sysctrl.checkhttp"&value="00:00:00"] query [?timeZone=E
[00:00:24.322,906] <inf> app: Looking up host [timeapi.io][443], asking for path[/api/time/current/zone?timeZone=Europe%2FParis&id="app.sysctrl.checkhttp"&value="00:00:00"]

[00:00:24.360,046] <inf> app: Resolved 86.105.246.247 (AF_INET)
[00:00:24.366,485] <inf> app: httpmgr: checking certs/ca for certs
[00:00:24.376,586] <inf> app: httpmgr : registered cert [certs/ca/timeapi.io.der]
[00:00:24.387,054] <inf> app: httpmgr : registered cert [certs/ca/www.infrafon.cloud.der]
[00:00:24.395,996] <inf> app: httpmgr: checking certs/cli for certs
[00:00:24.403,930] <inf> app: httpmgr: added 2 TLS certificates
[00:00:24.410,308] <inf> app: httpmgr : TLS setup OK on socket
[00:00:24.416,595] <inf> app: Connecting to timeapi.io:443

[00:00:24.455,474] <err> net_sock_tls: Failed to parse CA certificate, err: -0x262e
[00:00:24.463,867] <wrn> app: Failed to connect socket (-1), [22]
[00:00:24.470,947] <wrn> app: uimgr : http req fails to start to time;timeapi.io/.../zone

Note:

 - I'm using timeapi.io as an easy test URL, not really because this is my target server!

 - at init, I load (once) the certificates from my flash FS by mallocing space for the data and then calling tls_credential_add() to give the buffer to the socket TLS system referenced by a "security tag" id for each one

 - when I create the socket, I setsockopt(TLS_SEC_TAG_LIST) it with the list of all the security tag values I previously added the certificates under so it can reference them.

The problem occurs when I call connect() on the socket, as that's when the TLS layer tries to parse the certificates....

In the debugger; the fail is in 

x509.c : mbedtls_x509_crt_parse()

which calls 

mbedtls_x509_crt_parse_der()

which ends up in 

x509_crt_parse_der_core()

As far as I can tell, its when it calls in line 1168:

mbedtls_x509_get_sig_alg() (in x509.c)

that it gets a return value of  -0x262e. in this function This is the add of 

MBEDTLS_ERR_X509_UNKNOWN_SIG_ALG + the ret from 

mbedtls_oid_get_sig_alg()

This is a function created by lovely macro cin oid.c:

int mbedtls_oid_get_sig_alg(const mbedtls_asn1_buf *oid, mbedtls_md_type_t * md_alg, mbedtls_pk_type_t * pk_alg) {

  const oid_sig_alg_t *data = oid_sig_alg_from_asn1(oid);

  if (data == ((void *)0))

       return -0x002E;

  *(md_alg) = data->md_alg;

  *(pk_alg) = data->pk_alg;

  return 0;

}

(btw this kind of macro *** really makes the debugger hard to follow...)

The oid_sig_alg_from_asn1() function which tries to extract the OID of the certificate's signature algo from the asn.1 and map it to the mbedtls types... is another macro defined function in oid.c:

FN_OID_TYPED_FROM_ASN1(oid_sig_alg_t, sig_alg, oid_sig_alg)

which gives us:

static const oid_sig_alg_t *oid_sig_alg_from_asn1( const mbedtls_asn1_buf *oid) {

  const oid_sig_alg_t *p = (oid_sig_alg);

  const mbedtls_oid_descriptor_t *cur = (const mbedtls_oid_descriptor_t *) p;

  if (p == ((void *)0) || oid == ((void *)0))

    return ((void *)0);

  while (cur->asn1 != ((void *)0)) {

    if (cur->asn1_len == oid->len && memcmp(cur->asn1, oid->p, oid->len) == 0) {

      return p;

    }

    p++;

    cur = (const mbedtls_oid_descriptor_t *) p;

  }

  return ((void *)0);

}

Unpacking this, its searching for the oid given in the certificate in a static array (oid_sig_alg in oic.c) generated with a load of ifdefs depending on which crypto algos are compiled in...

The certificate is using PKCS#1 SHA-256 with RSA (the public key of the certificate of timeapi.io is 2048 size) according to chrome's handy popup...

And indeed, in my oid.c, only sha256 with ECDSA algos are enabled...in fact no RSA algo is enabled (MBEDTLS_RSA_C is not defined)

All this hidden behinf that errno=-22 from connect().... lucky I got that nice error log from the TLS layer eh?

So, it seems I just need to define :

MBEDTLS_RSA_C

However, the saga continues - this define is already set in the mbedtls config file I select (zephyr/modules/mbedtls/config/config-mini-tls1_2.h) in my prj.conf:

CONFIG_NRF_SECURITY=n

CONFIG_MBEDTLS_LIBRARY_NRF_SECURITY=n

CONFIG_CUSTOM_MBEDTLS_CFG_FILE=y

CONFIG_MBEDTLS_CFG_FILE="config-mini-tls1_2.h"

CONFIG_MBEDTLS_PEM_CERTIFICATE_FORMAT=y

However, visual code studio tells me that in the build NRF_SECURITY is 'y', and the MBEDTLS_CFG_FILE is set to 'nrf-config.h'!

The only place I can find this file is in

build/generated/library_nrf_security_psa/nrf-config.h

... and it indeed does NOT define MBEDTLS_RSA_C....

(and why does it all have to be so complicated and wrapped in 27 layers of kconfig files and macros???)

So finally my question : how do I get the nrf wrappers round the zephyr KConfig to accept that I want to use a specific mbedtls config, which includes MBEDTLS_RSA_C?

Also, it seems the sockets TLS code is hardcoded to use mbedtls functions : is there an alternative (which might take up less flash space, eg tinycrypt?

Naively I though I could just do

CONFIG_NRF_SECURITY=n

CONFIG_MBEDTLS=n

CONFIG_TINYCRYPT=y

but no joy (although maybe this is because its forcing mbedtls/nrf_security?)

[BrianW]

BrianW said:

So finally my question : how do I get the nrf wrappers round the zephyr KConfig to accept that I want to use a specific mbedtls config, which includes MBEDTLS_RSA_C?

If I add CONFIG_MBEDTLS_RSA_C=y, then my flash overflows...

prj.conf:

CONFIG_NRF_SECURITY=y

CONFIG_MBEDTLS_LIBRARY_NRF_SECURITY=y

# TLS configuration - don't want everything in mbedtls as far too big

#CONFIG_CUSTOM_MBEDTLS_CFG_FILE=y

#CONFIG_MBEDTLS_CFG_FILE="config-mini-tls1_2.h"

CONFIG_MBEDTLS=y

CONFIG_MBEDTLS_DTLS=n

CONFIG_MBEDTLS_TLS_VERSION_1_2=y

CONFIG_MBEDTLS_RSA_C=y

So, I thought I'd remove EC crypto (as I can use just RSA certificates)? 

CONFIG_MBEDTLS_ECDH_C=n

but KConfig is conflicted about this.

warning: MBEDTLS_PK_WRITE_C (defined at C:/ncs/v2.8.0/nrf\subsys\nrf_security\Kconfig.legacy:875, C:/ncs/v2.8.0/zephyr/modules/mbedtls\Kconfig.tls-generic:470, modules\mbedtls\Kconfig.tls-generic:470) has direct dependencies (MBEDTLS_PK_C && NRF_SECURITY) || (MBEDTLS_BUILTIN && MBEDTLS_CFG_FILE = "config-tls-generic.h" && MBEDTLS) || (MBEDTLS_BUILTIN && MBEDTLS_CFG_FILE = "config-tls-generic.h" && MBEDTLS && 0) with value n, but is currently being y-selected by the following symbols:
- WIFI_NM_WPA_SUPPLICANT_CRYPTO_ALT_LEGACY_NCS (defined at C:/ncs/v2.8.0/zephyr/modules/hostap/Kconfig:215, modules/hostap/Kconfig:215, modules\hostap\Kconfig:215), with value y, direct dependencies <choice WIFI_NM_WPA_SUPPLICANT_CRYPTO_BACKEND> || <choice WIFI_NM_WPA_SUPPLICANT_CRYPTO_BACKEND> || <choice WIFI_NM_WPA_SUPPLICANT_CRYPTO_BACKEND> (value: y), and select condition <choice WIFI_NM_WPA_SUPPLICANT_CRYPTO_BACKEND> (value: y)

Adding

CONFIG_MBEDTLS_ECDSA_C=n

fixed KConfig, but now the compile fails:

C:/ncs/v2.8.0/modules/lib/hostap/src/crypto/crypto_mbedtls_alt.c: In function 'crypto_ec_key_parse_priv':
C:/ncs/v2.8.0/modules/lib/hostap/src/crypto/crypto_mbedtls_alt.c:2354:5: error: unknown type name 'mbedtls_pk_context'; did you mean 'mbedtls_dhm_context'?
 2354 |     mbedtls_pk_context *ctx = os_malloc(sizeof(*ctx));
      |     ^~~~~~~~~~~~~~~~~~
      |     mbedtls_dhm_context
C:/ncs/v2.8.0/modules/lib/hostap/src/crypto/crypto_mbedtls_alt.c:2357:5: warning: implicit declaration of function 'mbedtls_pk_init'; did you mean 'mbedtls_mpi_init'? [-Wimplicit-function-declaration]
 2357 |     mbedtls_pk_init(ctx);
      |     ^~~~~~~~~~~~~~~
      |     mbedtls_mpi_init
C:/ncs/v2.8.0/modules/lib/hostap/src/crypto/crypto_mbedtls_alt.c:2361:9: warning: implicit declaration of function 'mbedtls_pk_parse_key'; did you mean 'mbedtls_ecp_write_key'? [-Wimplicit-function-declaration]
 2361 |     if (mbedtls_pk_parse_key(ctx, der, der_len, NULL, 0, mbedtls_ctr_drbg_random, crypto_mbedtls_ctr_drbg()) == 0)
      |         ^~~~~~~~~~~~~~~~~~~~
      |         mbedtls_ecp_write_key
C:/ncs/v2.8.0/modules/lib/hostap/src/crypto/crypto_mbedtls_alt.c:2365:5: warning: implicit declaration of function 'mbedtls_pk_free'; did you mean 'mbedtls_mpi_free'? [-Wimplicit-function-declaration]
 2365 |     mbedtls_pk_free(ctx);
      |     ^~~~~~~~~~~~~~~
      |     mbedtls_mpi_free
In file included from C:/ncs/v2.8.0/modules/crypto/mbedtls/include/mbedtls/oid.h:17,
                 from C:/ncs/v2.8.0/modules/lib/hostap/src/crypto/crypto_mbedtls_alt.c:2371:
C:/ncs/v2.8.0/modules/crypto/mbedtls/include/mbedtls/pk.h: At top level:
C:/ncs/v2.8.0/modules/crypto/mbedtls/include/mbedtls/pk.h:310:6: warning: conflicting types for 'mbedtls_pk_init'; have 'void(mbedtls_pk_context *)'
  310 | void mbedtls_pk_init(mbedtls_pk_context *ctx);
      |      ^~~~~~~~~~~~~~~
C:/ncs/v2.8.0/modules/lib/hostap/src/crypto/crypto_mbedtls_alt.c:2357:5: note: previous implicit declaration of 'mbedtls_pk_init' with type 'void(mbedtls_pk_context *)'
 2357 |     mbedtls_pk_init(ctx);
      |     ^~~~~~~~~~~~~~~
C:/ncs/v2.8.0/modules/crypto/mbedtls/include/mbedtls/pk.h:323:6: warning: conflicting types for 'mbedtls_pk_free'; have 'void(mbedtls_pk_context *)'
  323 | void mbedtls_pk_free(mbedtls_pk_context *ctx);
      |      ^~~~~~~~~~~~~~~
C:/ncs/v2.8.0/modules/lib/hostap/src/crypto/crypto_mbedtls_alt.c:2365:5: note: previous implicit declaration of 'mbedtls_pk_free' with type 'void(mbedtls_pk_context *)'
 2365 |     mbedtls_pk_free(ctx);
      |     ^~~~~~~~~~~~~~~
C:/ncs/v2.8.0/modules/lib/hostap/src/crypto/crypto_mbedtls_alt.c: In function 'crypto_mbedtls_pk_parse_subpubkey_compressed':
C:/ncs/v2.8.0/modules/lib/hostap/src/crypto/crypto_mbedtls_alt.c:2473:14: warning: implicit declaration of function 'crypto_mbedtls_short_weierstrass_derive_y' [-Wimplicit-function-declaration]
 2473 |           || crypto_mbedtls_short_weierstrass_derive_y(ecp_kp_grp, Y, (*p & 1));
      |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[246/718] Building C object zephyr/lib/libc/common/CMakeFiles/lib__libc__common.dir/source/time/asctime.c.obj
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: 'C:\ncs\toolchains\2d382dcd92\opt\bin\cmake.EXE' --build 'C:\work\dev\if-device-nrf53\cc1-med\build'

How can I get a minimum TLS1.2 capable mbedtls build?

[Amanda Hsieh]

Please check out the HTTPS Client Sample and the Wi-Fi Fundamentals Course. 

[Amanda Hsieh]

Could you try my colleague's suggestion in this post?

[BrianW]

Well, depends which of the suggestions you mean?

CONFIG_MBEDTLS_MPI_MAX_SIZE=512

- my build already has this as the default (maybe due to NCS 2.8?)

CONFIG_MBEDTLS_SSL_IN_CONTENT_LEN=16384
CONFIG_MBEDTLS_SSL_OUT_CONTENT_LEN=16384

CONFIG_NET_SOCKETS_TLS_SET_MAX_FRAGMENT_LENGTH=n

- again, already set to these values by default apparently

Also, it doesn't seem that the issue in that post is in fact resolved anyway?

[Amanda Hsieh]

Could you also add the following configs? and Let me know they can help or not. 

CONFIG_MBEDTLS_HEAP_SIZE=80000
CONFIG_MBEDTLS_SSL_RENEGOTIATION=y
CONFIG_NET_SOCKETS_TLS_MAX_CONTEXTS=6

[BrianW]

Amanda Hsieh said:

CONFIG_MBEDTLS_HEAP_SIZE=80000

No change, I already have

CONFIG_HEAP_MEM_POOL_SIZE=89000

and mbedtls uses the sys heap as far as I understand

I reduced this to 84000 as the increased number of SOCKETS_TLS contexts overflows the RAM otherwise.

[by the way, I have logs to monitor the heap usage every 60s during dev, and check that I don't run out of sysheap free space (which it doesn't)]

Amanda Hsieh said:

CONFIG_MBEDTLS_SSL_RENEGOTIATION=y
CONFIG_NET_SOCKETS_TLS_MAX_CONTEXTS=6

This made no difference to the https tests.

https://eu.httpbin.org/get : ok

 - if I provision the top level amazon certif

https://server.infrafon.club/test : fails

<err> net_sock_tls: TLS handshake error: -0x2700
<wrn> app: httpmgr: Failed to connect socket (-1), [113]

 - this is my server, normally has a self-signed certificate RSA 2048 bits, which I provision in the TLS socket

https://example.com : fails

<err> net_sock_tls: TLS handshake error: -0x2700
<wrn> app: httpmgr: Failed to connect socket (-1), [113]

 - no CA certificate provisioned as they have 4096bit RSA keys and fail when these are enabled, so instead it fails when they are sent during TLS process (I assume)

[BrianW]

My server now serves a self-signed certificate, with a 2048bit RSA key.

Even if I provision this certificate in the TLS socket, the connection fails with the handsshake error -0x2700. This happens both with a generated certificate and also with the traefik auto generated certificate.

2 questions:

- is it not possible to provide TLS with the server's certificate directly? Does it have to have a CA/CA certificate to work?

- the certificates have validaity dates. Does this mean that my device has to have the correct time/date to be able to validate this info?

[Amanda Hsieh]

Hi, 

BrianW said:

Even if I provision this certificate in the TLS socket, the connection fails with the handsshake error -0x2700. This happens both with a generated certificate and also with the traefik auto generated certificate.

Could you try with adding CONFIG_MBEDTLS_SSL_SERVER_NAME_INDICATION=y in the prj.conf? Please see refer to https://academy.nordicsemi.com/courses/wi-fi-fundamentals/lessons/lesson-5-wifi-fundamentals/topic/lesson-5-exercise-2-2/ 

BrianW said:

is it not possible to provide TLS with the server's certificate directly? Does it have to have a CA/CA certificate to work?

You can add with TLS_CREDENTIAL_SERVER_CERTIFICATE. See TLS credentials subsystem. 

BrianW said:

- the certificates have validaity dates. Does this mean that my device has to have the correct time/date to be able to validate this info?

Yes, the device needs to have the correct time and date to properly validate certificate validity periods. See the Sensor Device Threat Model documentation. 

-Amanda H.

[BrianW]

Amanda Hsieh said:

BrianW said:

is it not possible to provide TLS with the server's certificate directly? Does it have to have a CA/CA certificate to work?

You can add with TLS_CREDENTIAL_SERVER_CERTIFICATE. See TLS credentials subsystem. 

This doesn't seem to work for me. Specifically for the test site 

https://eu.httpbin.org

if I load the DER certificate for the root CA of this site (as TLS_CREDENTIAL_CA_CERTIFICATE), it works. If I don't load it, but instead load the server's DER certificate (as TLS_CREDENTIAL_SERVER_CERTIFICATE), then it fails with -0x2700...

(both certificates retrieved from Chrome web browser)

The documentation as to exactly what the TLS_CREDENTIAL_SERVER_CERTIFICATE is expected to do is missing; but when I looked in the code it seemed more like provision of a certificate used to validate TLS operations when running a http server locally (ie server side not client side). Is this correct?

Amanda Hsieh said:

Yes, the device needs to have the correct time and date to properly validate certificate validity periods. See the Sensor Device Threat Model documentation. 

I found this discussion https://github.com/zephyrproject-rtos/zephyr/issues/35401, where it appears that it should be possible to disable the date/time validity check (my device does not have a secure date/time source) by doing

CONFIG_MBEDTLS_HAVE_TIME_DATE=n

However this made no difference to the operation and the kconfig says it was 'n' anyway!

This makes sense, as the eu.httpbin.org CA certificate also has validity dates, but works....

Amanda Hsieh said:

Could you try with adding CONFIG_MBEDTLS_SSL_SERVER_NAME_INDICATION=y in the prj.conf?

YES! this seems to be the magic for my server - fixes it for both 2048bit RSA CA certificates (which I had reconfigured my server to use, and also for 4096bit ones as served by the LetsEncrypt CA server.

I was not aware of the SNI part of TLS, nor indeed that my server using multiple domains on the same host would need it! And hence https to my server failed, but other work...

(obligatory moan :  if the code had a slightly more explicit log  than error -0x2700, or if this error was documented as being "you need SNI enabled" then I might have got here quicker...)