Certificate based DTLS handshake debugging questions

[Joel H]

Short version:

- Can you enable debug output in the mbedtls used for DTLS in OpenThread, available through the nRF SDK? If so, how?

- Can you at all change the list of supported ECC groups that will be communicated as part of a certificate based DTLS handshake performed with the included OpenThread libraries?

- Are there any recommendations for nrf52840 memory allocation to be used to support (ec) certificate based DTLS handshakes?

Longer version, including background & setup:

We have a project where we are programming a nrf52840 to use OpenThread and nRF SDK for Thread and Zigbee (4.1.0). It connects to and through a OT Border Router running on a raspberry pi, with a zolertia firefly as radio stick. On the border router we run a test server based on the example libcoap server.

Setting up the node to use PSK based DTLS handshake works like a charm, out of the box, with the unmodified nrf-config.h from the SDK. But with certificate based handshake, it fails, also after updating the relevant flags in nrf-config.h. 

Using tcpdump/wireshark we can see that the node fails after receiving the combined ServerHello/Certificate/ServerKeyExchange/CertificateRequest/ServerHelloDone message. Despite having activated the MBEDTLS_SSL_ALL_ALERT_MESSAGES flag, only the generic “error in error” alert message is sent back to the server. 

Some further findings: We can see that the node is advertising nine standard supported ec groups in its ClientHello message, despite that we have tried to undefine most of them in the nrf-config file. As a result, SECP521R1 gets selected by the server side. This group is not supported in our preferred micro-ecc crypto backend, but the same handshake error/behaviour occurs when using the mbedtls backend.

The above was done with the included OpenThread binaries. As a parallel track we have also recompiled the OpenThread libraries (following these instructions -- https://infocenter.nordicsemi.com/index.jsp?topic=%2Fsdk_tz_v3.1.0%2Fthread_ot_libraries_building.html ) --

Doing this we can set the MBEDTLS_SSL_PRESET_DEFAULT which does enforce a more limited set of groups being advertised in the ClientHello message. Also here we fail to activate the mbedtls debugging, despite setting the MBEDTLS_DEBUG_C, the linker doesn’t include the mbedtls debug.c. But we can insert some debugging into the OpenThread part of handshake processing, and it reports failing with MBEDTLS_ERR_ECP_ALLOC_FAILED. This leads to the question if there are special memory setup recommendations to accommodate certificate based handshakes.

[Edvin]

Hello,

Sorry for the late reply. I am trying to reach out to our Thread team. I will keep you updated as soon as I know more. Just wanted to let you know that this ticket isn't "lost".

Best regards,

Edvin

[Joel H]

Thanks for the update!

(We are happy to share any local project settings, but since there are so many config files involved I thought it was better to start a bit more generic, and hopefully get suggestions on where/how to narrow down the debugging.)

[Edvin]

Hello,

I received an update from our Thread team today:

Most likely, the mbedTLS doesn't have sufficient heap size to perform some cryptographic operation. For our examples in the nRF5 SDK, the mbedTLS uses tje mem_manager module for dynamic memory allocation. You can adjust the sizes and amount of memory buckets in the example's sdk_config.h, e.g.: "MEMORY_MANAGER_XLARGE_BLOCK_SIZE".

I think we need more debug details to understand what the memory requirements are. I would recommend to modify the "ot_calloc" function inside the thread_util.c file and put a breakpoint when nrf_calloc results in a NULL pointer. You can then try to understand what the size of the buffer requested by mbedTLS and adjust that in sdk_config.h accordingly. 

Can you look into this, and see if you can see if there seems to be too little memory in ot_calloc?

BR,
Edvin