Restrictions on number of CA certifs in a TLS connection? How to calculate memory required?

[BrianW]

On a nrf5340/nrf7002 custom board, using NCS 2.9.0, I am trying to get a reliable https connection to different backend servers.

I'm having trouble with the TLS handshake due to the server certificate validation.

My medtls setup is using nrf_security (which sadly then doesn't let me restrict the crypto options to save some flash, but thats a different problem)

CONFIG_MBEDTLS=y

CONFIG_MBEDTLS_TLS_LIBRARY=y

CONFIG_NRF_SECURITY=y

CONFIG_MBEDTLS_LIBRARY_NRF_SECURITY=y

CONFIG_MBEDTLS_BUILTIN=n

CONFIG_MBEDTLS_DTLS=n

CONFIG_MBEDTLS_TLS_VERSION_1_2=y

CONFIG_MBEDTLS_RSA_C=y

CONFIG_MBEDTLS_ECP_C=y

CONFIG_PSA_WANT_RSA_KEY_SIZE_4096=y

CONFIG_PSA_WANT_RSA_KEY_SIZE_2048=y

CONFIG_PSA_WANT_ECC_SECP_R1_256=y

CONFIG_PSA_WANT_ECC_MONTGOMERY_255=y

I have already dealt with the SNI issue, and enlarged the SSL context and MSI buffers to cope with 4K RSA keys:

CONFIG_MBEDTLS_HAVE_TIME_DATE=n

CONFIG_MBEDTLS_SSL_PROTO_DTLS=n

CONFIG_MBEDTLS_SSL_SERVER_NAME_INDICATION=y

CONFIG_MBEDTLS_SSL_IN_CONTENT_LEN=16384

CONFIG_MBEDTLS_SSL_OUT_CONTENT_LEN=16384

CONFIG_MBEDTLS_SSL_MAX_CONTENT_LEN=16384

CONFIG_MBEDTLS_SSL_RENEGOTIATION=y

CONFIG_MBEDTLS_SSL_MAX_FRAGMENT_LENGTH=y

CONFIG_MBEDTLS_MPI_MAX_SIZE=512

CONFIG_NET_SOCKETS_TLS_SET_MAX_FRAGMENT_LENGTH=n

CONFIG_NET_SOCKETS_TLS_MAX_CONTEXTS=3

NOTE : I do not define the recommended standalone mbedtls heap via

CONFIG_MBEDTLS_ENABLE_HEAP=y

CONFIG_MBEDTLS_HEAP_SIZE=80000

because the limited RAM on nrf5340 is not enough for my device until there is some sharing of the heap.... dedicating 80Kb of 440kB just for mbedtls is not realistic...

(I do create a seperate sys heap of 89000 as required for the wifi  & networking code...)

So mbedtls is using the libc calloc/free() calls (set up by wpa supplient code as it needs mbedtls to work for wifi association!)

I setup the TLS certifcates (in DER format) using  tls_credential_add() at startup (making sure to not free the buffers used), and record the list of tags id.
These are then used each time the code attempts to connect to a https server, via setsockopt() with TLS_PEER_VERIFY (REQUIRED) and TLS_SEC_TAG_LIST. 

The problem is that depending on the number of CA certs added, it seems that the TLS handshake will fail with 0x2700 (X.509 - Certificate verification failed).

In my tests, an RSA 2K + RSA 4K is (mostly) ok, but adding a 3rd RSA 2K will then cause connections using the RSA4K CA to fail...

[NOTE : I don't want to provision RSA 2K intermediate CA certificates as these generally have short valid lifetimes, and for a field deployed IOT device this makes the maintenance complex!]

Even with just the 2 certificates, the RSA 4K connection will sometimes fail with 0x4290 (allocation error in RSA - public key operation failed). 

According to my logs, the libc heap (which is set to 171000) has not failed an allocation request (free before a https request is 59000 for example).

Its 'max used' is 157000 after a successful https get, goes back to having 59000 free, but then the next one fails with a 0x4290!

Question : how can I calculate how much memory is really required by mbedtls, and when? I need to reliably ensure that my TLS connections will  work to different https servers? I do need to leave some memory for my application to use.... 

[Håkon Alseth]

Hi,

 

NOTE : I do not define the recommended standalone mbedtls heap via

CONFIG_MBEDTLS_ENABLE_HEAP=y

CONFIG_MBEDTLS_HEAP_SIZE=80000

This is fine, as it will then use the system heap. This can be useful in scenarios where you are certain that the memory pool is not 

Even with just the 2 certificates, the RSA 4K connection will sometimes fail with 0x4290 (allocation error in RSA - public key operation failed). 

RSA will use quite a lot of memory for the initial connection. This will vary depending on which server you connect to, and the initial handshake process. Since you mention SNI, there will be different physical servers that you are routed to, which will cause a difference in stack usage for the RSA process.

If you have 151k usage for one connection, it might differ on the next. I would recommend that you up the system heap as high as you can, while still observing the heap usage, then see if the connection is more stable.

because the limited RAM on nrf5340 is not enough for my device until there is some sharing of the heap.... dedicating 80Kb of 440kB just for mbedtls is not realistic...

RSA 2k/4k will use quite a lot of stack/heap. 80kB is close to the minimum requirement for RSA.

 

Question : how can I calculate how much memory is really required by mbedtls, and when? I need to reliably ensure that my TLS connections will  work to different https servers? I do need to leave some memory for my application to use.... 

I wish I could tell you a easy way to benchmark this, but unfortunately; there is no direct way other than testing and observing the heap usage.

 

Its 'max used' is 157000 after a successful https get, goes back to having 59000 free, but then the next one fails with a 0x4290!

Could you try to add debug logs to mbedtls to verify what happens here?

CONFIG_MBEDTLS_DEBUG=y
CONFIG_MBEDTLS_SSL_DEBUG_ALL=y
CONFIG_MBEDTLS_LOG_LEVEL_DBG=y
CONFIG_MBEDTLS_DEBUG_C=y
CONFIG_MBEDTLS_DEBUG_LEVEL=4 # could be 3 as well.

 

Since you already have space issues, I would recommend that you log these with CONFIG_LOG_MODE_IMMEDIATE, to avoid having a dedicated logging thread.

This enables in-place logging, to be careful if you have prints in ISRs and other time-critical functions.

 

I would recommend taking one log as "working" and one as "non working" to see if there is any direct differences between the two connections.

 

Kind regards,

Håkon

[BrianW]

Ok, thanks for the info. After playing around with different configs, the problem is definitely a question of memory heap size. By limiting my CA certifications to the minimum required for my basic app, I have managed to get my device management/OTA working from my server.... This will do for my basic product.

However, clearly the nrf5340 is lacking enough RAM/flash to really implement a full wifi/https type of  embedded application.

[Håkon Alseth]

Hi,

 

BrianW said:

After playing around with different configs, the problem is definitely a question of memory heap size.

Unfortunately, this has been my observation when using RSA + SNI in the past as well. It is hard to dial into the lowest possible usage, so you need to add a rather large margin. 

BrianW said:

y limiting my CA certifications to the minimum required for my basic app, I have managed to get my device management/OTA working from my server.... This will do for my basic product.

This is great to hear! Glad you got it running as intended.

BrianW said:

However, clearly the nrf5340 is lacking enough RAM/flash to really implement a full wifi/https type of  embedded application.

I understand that this is quite cumbersome when reaching the resource limit of the nRF. For more flash, you can look into the nRF54LM20, which has just shy of 2MByte, but still has 512kB of RAM.

 

I wish you a wonderful day!

 

Kind regards,

Håkon

[BrianW]

Håkon Alseth said:

I understand that this is quite cumbersome when reaching the resource limit of the nRF. For more flash, you can look into the nRF54LM20, which has just shy of 2MByte, but still has 512kB of RAM.

Honestly I would not consider using any product with <1Mb of RAM if it was intended to do any real wifi/secure connection work. Its a shame there is no external RAM support on the nrf5340 : does the nrf54 support this?

[Håkon Alseth]

Hi,

 

Unfortunately, we do not have any devices that supports external RAM.

 

Kind regards,

Håkon