Random Zephyr hangs due to RNG / Entropy

[krbvroc1]

1205.test.zip

On nRF52840, after power up, I continue to randomly get hangs. I am having trouble figuring it out, but it seems every time I encounter the issue, it is related to RNG. If I attach gdb, it is typically stuck in 'CC_PalWaitInterruptRND'. I thought I could set 'CONFIG_TEST_RANDOM_GENERATOR=y', but CMake complains it is overriden by ENTROPY_HAS_DRIVER... I am not anywhere near enough a CMake expert to figure out if it is even possible to disable the Nordic Entropy to test.

I use openthread which selects a whole string of entropy requirements in the Kconfig tree. I even tried hang modifying some Kconfig source files to no avail.

Here is one such backtrace, but please note that is not just the zephyr coap stack, it also happens when openthread tries to initialize (which means it generates random numbers for its own coap stack). I reduce the occurrence of the board hangs by replacing the sys_rand32_get() with my own call. That stopped the Zephyr coap hangs shown below and the hangs moved into openthread stack which has its own calls to generate random numbers for coap tokens.

#0 0x0002e2ae in CC_PalWaitInterruptRND ()
#1 0x0002eb00 in LLF_RND_WaitRngInterrupt ()
#2 0x0002e8e6 in getTrngSource ()
#3 0x0002ea7a in LLF_RND_GetTrngSource ()
#4 0x0002de14 in mbedtls_hardware_poll ()
#5 0x00057460 in entropy_cc3xx_rng_get_entropy (dev=<optimized out>, buffer=<optimized out>, length=<optimized out>)
at /home/user/NORDIC_CONNECT_SDK/ncs/nrf/drivers/entropy/entropy_cc310.c:67
#6 0x0001ee38 in z_impl_entropy_get_entropy (length=4, buffer=0x20013564 <dfu_task_stack+3972> "\354\062\001 ܛ", dev=<optimized out>)
at /home/user/NORDIC_CONNECT_SDK/ncs/zephyr/include/drivers/entropy.h:77
#7 entropy_get_entropy (length=4, buffer=0x20013564 <dfu_task_stack+3972> "\354\062\001 ܛ", dev=<optimized out>)
at zephyr/include/generated/syscalls/entropy.h:33
#8 z_impl_sys_rand32_get () at /home/user/NORDIC_CONNECT_SDK/ncs/zephyr/subsys/random/rand32_entropy_device.c:33
#9 0x00016e82 in sys_rand32_get () at zephyr/include/generated/syscalls/rand32.h:33
#10 coap_next_token () at /home/user/NORDIC_CONNECT_SDK/ncs/zephyr/subsys/net/lib/coap/coap.c:339

[JONATHAN LL]

Hi,

Looking in to this, will update you.

Can you share your  prj.conf

Regards,
Jonathan

[JONATHAN LL]

It could be related to the issue here : [nrf52840] device hangs when reinitializing cc310 · Issue #2898 · openthread/openthread · GitHub
What version of NCS are you on ?
Regards,
Jonathan

[krbvroc1]

I am currently using 1.5 ... I also had the same problem on the latest 'master' (1.5.99).

A workaround I was given from a Zephyr developer was to override the DTS file, basically changing

'zephyr,entropy = &cryptocell;'

to

'zephyr,entropy = &rng;'

Disabling the CC310 definitely fixes the issue, so there is still a bug in CC310 RNG or a regression that Nordic needs to address.

[JONATHAN LL]

Thanks for the information, we are trying to reproduce this locally.  Could you possibly share a minimal project that still reproduces the issue you are seeing?
Regards,
Jonathan

[krbvroc1]

I just added an attachment to the original post. I am not sure if it reproduce the board hang, BUT...

Running the sample illustrates that calling sys_rand32_get() when the default CC310 cryptocell is used for RNG takes FOREVER. Generating 200 numbers takes over 18 seconds! There is an overlay file included, which comments out the override to the RNG. If you enable the RNG it works fine and generation time is minimal.