Two issues with CC310 HW CryptoCell: Linking and hanging

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

evanperrygrove — Wed, 09 Feb 2022 18:17:23 GMT

I hate to revive a dead thread, but I was having a very similar problem while migrating a project from SDK 15.0 to 15.3. My code was hanging on a DMB instruction somewhere inside SaSi_AesInit. Eventually, the watchdog timed out and the sensor would reboot.

I realized that my sdk_config.h was missing a #define for NRF_CRYPTO_BACKEND_CC310_INTERRUPTS_ENABLED. As soon as I added this (and set it to 1), the problem went away. This appears to be a new macro that was introduced in 15.3 (I did not see it in a quick search of some example code from 15.2).

I also applied jakkra's patch, just to be safe.

Now I'm dealing with a different crash, but the new one doesn't appear to be related to CC310 😉

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

pieterg — Thu, 23 Sep 2021 18:19:59 GMT

OK, the reflashing + timeout while erasing sectors when starting the debugger pointed me into the right direction;

The whole situation seems to be caused by having too many breakpoints.

As soon as I have more than 5 breakpoints, the reflashing at program start occurs, and times out. After that, CRYS_RndInit hangs.

As long as I have less than 6 breakpoints, no reflashing takes place, and CRYS_RndInit does not hang.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

pieterg — Thu, 23 Sep 2021 14:18:38 GMT

Interesting; I just noticed that when erasing the page takes that long, it actually times out:

Timeout while erasing sectors, RAMCode did not respond in time. (PC = 0x00000000, CPSR = 0x00000000, LR = 0x01000003)!

Failed to erase sectors.

Programming failed @ address 0x00041958 (block verification error)

Not sure though whether this is the cause or a result of the actual problem...

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

pieterg — Thu, 23 Sep 2021 14:11:12 GMT

Unfortunately, the problem has come back, nrf_crpyto_rng_init hangs in CRYS_RndInit again.

Even without having nrf_crypto_ecc_public_key_calculate in my code, which does not appear to be causing the problem after all (even though adding/removing it several times reproduced/removed the issue for sure)

I've tried to add sd_nvic_critical_region_enter/exit around nrf_crypto_init and nrf_crypto_rng_init which apparently helped the OP, but that does not help in my case.

When I comment all of my nrf_crypto_ecc code, the problem disappears. Again, none of this code has been executed yet by the time CRYS_RndInit hangs, because this occurs in nrf_crypto_init (which calls nrf_crypto_rng_init internally because of NRF_CRYPTO_RNG_AUTO_INIT_ENABLED), way before anything interesting happens.

I do notice that before CRYS_RndInit is going to hang, when I start the application in the debugger, SEGGER IDE shows "Erasing range 0x41000-0x41FFF" and blocks for about 10s (erasing a single page normally does not take that long).

Whenever erasing (because of reprogramming) a single page takes that long, I already know CRYS_RndInit is going to hang.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

pieterg — Thu, 23 Sep 2021 13:10:28 GMT

I've just had the same problem, nrf_crypto_rng_init hangs in CRYS_RndInit.

Not building SDK as a separate library.

After a couple of hours I found out it occurs as soon as I reference nrf_crypto_ecc_public_key_calculate() in my code.

(nrf_crypto_ecc_public_key_calculate will not work apparently with the cc310 backend, because it is not implemented by that backend, but because of the crypto init hanging, I never got to that point in my code where I even tried to call that function, and find out it was not implemented)

Just linking that function causes CRYS_RndInit to hang.

Now that I've removed the reference to nrf_crypto_ecc_public_key_calculate, nrf_crypto_rng_init works fine again, CRYS_RndInit no longer hangs.

Just posting this for reference, in case someone else gets into the same situation some day.

My problem has been resolved (by avoiding nrf_crypto_ecc_public_key_calculate), so no action needed.

Unless someone perhaps could clarify what happens here, because it puzzles me.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

WirelessCharging — Fri, 12 Jun 2020 10:28:08 GMT

I had the same problem as OP which was solved by his patches. Running nRF52840 with SDK 15.3. Really annoying since it took like 5h to figure it out. I'm using Segger Embedded Studio v4.10. Note that I'm not building the SDK as its separate library hence the upvoted green answer to this thread did not help in my case.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

Einar Thorsrud — Thu, 02 May 2019 11:52:09 GMT

That is probably the explanation, then. I recommend you stop building the SDK as a library, as that has been known to cause several problems. The dummy function probably works because it make the linker aware that the function is being used (it is missing from the section variable where it should have been), and that causes it to include it.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

pellepl — Thu, 02 May 2019 11:30:58 GMT

Ah, yes, you're right. The SDK is built and linked as a static library.

This might be of an esoteric nature, but do you have any clue at all why it then works by adding a call to the dummy function?

Thanks for all help! / Peter

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

Einar Thorsrud — Thu, 02 May 2019 08:29:03 GMT

Hi Peter,

Do you by any chance build the SDK as a library, and then compile your application separately, linking it to the SDK? If so, that could explain this, as section variables require that everything is compiled together, if not they will not be populated.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

pellepl — Tue, 30 Apr 2019 14:03:47 GMT

Hi Einar,

tried with the 6.3 q2 toolchain without any luck.

I also tried removing `static` statement from the functions cc310_backend_init and cc310_backend_uninit without any result.

And also tried adding __attribute__((used)) to the same functions - still they do not show up in the .crypto_data section.

Tried removing gc-sections from linker flags just to try, but unfortunately this made sections overflow and could not get any leads from this.

Edit:

Further investigation indicates that the problem boils down to some issue in the libnrf_cc310_0.9.1.a.

If I add print-gc-sections to our linker, and diff the output with and without jakkras fix, it turns out that following symbols are garbage collected by the linker without the fix:

'.text.SaSi_PalInit'
'.text.SaSi_PalTerminate'
'.rodata.SaSi_PalInit.str1.4'
'.text.SaSi_PalDmaInit'
'.text.SaSi_PalDmaTerminate'
'.text.SaSi_PalMutexCreate'
'.text.SaSi_PalMutexDestroy'
'.rodata.SaSi_PalMutexCreate.str1.4'
'.text.SaSi_HalInit
'.text.SaSi_HalTerminate'

According to docs SaSi_PalInit should be called by SaSi_LibInit. SaSi_LibInit is called from

static ret_code_t cc310_backend_init(void)

which in turn is registered as

CRYPTO_BACKEND_REGISTER(nrf_crypto_backend_info_t const cc310_backend) =
{
.init_fn = cc310_backend_init,
.uninit_fn = cc310_backend_uninit,
};

in your cc310_backend_init.c.

It is weird that SaSi_LibInit is not gc'ed but SaSi_PalInit is (without the fix). Again, it suggests something odd in libnrf_cc310_0.9.1.a.

Cheers / Peter

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

Einar Thorsrud — Tue, 30 Apr 2019 11:55:15 GMT

Hi,

Can you check if you see the same issue with GCC ARM Embedded 6.3 2017-q2-update, which is the tested/supported toolchain for SDK 15.2?

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

jakkra — Tue, 30 Apr 2019 08:16:27 GMT

Hi, we have now upgraded to arm-none-eabi 8.3 and still have the exact same problems. So the issue is not with the compiler version.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

jakkra — Thu, 21 Mar 2019 12:57:40 GMT

Hi, thanks, that is wierd, yes we should use none-eabi. We will update and see if that solves the linking problem.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

Einar Thorsrud — Mon, 18 Mar 2019 14:43:15 GMT

Hi,

I did not notice earlier, but why do you use arm-eabi instead of arm-none-eabi? We only test with arm-none-eabi.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

jakkra — Fri, 15 Mar 2019 10:22:07 GMT

Hi, no problem. The workaround is fine for us right now, but it would be good if we could get rid of it in upcoming SDK releases. I did try with arm-eabi-gcc7.2.0-r3, but it did not compile our application, possibly some build settings need to be changed/updated, as I got some compile issues and didn't have time digging deeper into it.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

Einar Thorsrud — Fri, 15 Mar 2019 09:42:05 GMT

Hi,

I am sorry for the late reply (I have been away for some days). This seems like a linker issue of some sort, but I don't know. I have not had a chance to try to reproduce this yet but will do that early next week. Please let me know if you have any new findings. In any case I will get back to you early next week.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

jakkra — Mon, 11 Mar 2019 11:39:27 GMT

Hi, thanks for looking into this:

- NRF_CRYPTO_RNG_STATIC_MEMORY_BUFFERS_ENABLED 1. I did also try with dynamic but did not make a difference. I have also tested with and without NRF_CRYPTO_RNG_AUTO_INIT_ENABLED 1.

- I managed to find what code makes CRYS_RndInit() hang. Anything obvious you can spot here which causes it? This is part of our initiation.

NVIC_SetPriority(DebugMonitor_IRQn, 7ul);

// IRQ configuration
sd_nvic_SetPriority(RTC2_IRQn, APP_IRQ_PRIORITY_LOW);
sd_nvic_ClearPendingIRQ(RTC2_IRQn);
sd_nvic_EnableIRQ(RTC2_IRQn);

// Start LFCLK clock
nrf_clock_task_trigger(NRF_CLOCK_TASK_LFCLKSTART);

// Configure RTC2 frequency
nrf_rtc_prescaler_set(NRF_RTC2, cbHW_PRESCALER_RTC2);

/* Disable interrupts and events before changing configuration. */
sd_nvic_DisableIRQ(RTC2_IRQn);
sd_nvic_ClearPendingIRQ(RTC2_IRQn);

nrf_rtc_task_trigger(NRF_RTC2, NRF_RTC_TASK_STOP);
nrf_rtc_int_disable(NRF_RTC2, NRF_RTC_INT_COMPARE0_MASK);
nrf_rtc_int_disable(NRF_RTC2, NRF_RTC_INT_TICK_MASK);
nrf_rtc_event_disable(NRF_RTC2, NRF_RTC_INT_COMPARE0_MASK);
nrf_rtc_event_disable(NRF_RTC2, NRF_RTC_INT_TICK_MASK);
sysTickMode = mode;

sd_nvic_ClearPendingIRQ(RTC2_IRQn);
sd_nvic_EnableIRQ(RTC2_IRQn);

nrf_rtc_event_enable(NRF_RTC2, NRF_RTC_INT_TICK_MASK);
nrf_rtc_int_enable(NRF_RTC2, NRF_RTC_INT_TICK_MASK);
nrf_rtc_task_trigger(NRF_RTC2, NRF_RTC_TASK_START);
break;

flowControl = NRF_POWER->GPREGRET >> 3;

NRF_POWER->GPREGRET = 0;

2. Don't have at the moment, but the code above should hopefully reproduce it.

- arm-eabi-gcc.exe (GCC) 6.2.0, GNU ar (GNU Binutils) 2.27. more exact: http://gnutoolchains.com/arm-eabi/ arm-eabi-gcc6.2.0-r4

- Same issue when we build with zero optimization. Problem is that in nrf_crypto_init NRF_CRYPTO_BACKEND_SECTION_ITEM_COUNT is 0 without the patch I showed in the issue.

- I might be able to create an example project some day, but first I think this issue should be tested with the same toolchain we use. I'll also try with the latest arm-eabi-gcc and see if it solves the problem.

RE: Two issues with CC310 HW CryptoCell: Linking and hanging

Einar Thorsrud — Fri, 08 Mar 2019 15:19:29 GMT

Hi,

1. I do not recall seeing this issue before, nor do I immediately understand what is going on. You cannot use CC310 concurrently, and therefore the backed wrapper (cc310_backend_rng.c) use the CC310 mutex. I do not see any reason for using critical sections, though. Therefore, I have a few questions just to understand more about the situation:

How do you handle memory for the RNG module?
Is NRF_CRYPTO_RNG_STATIC_MEMORY_BUFFERS_ENABLED set to 1 in the projects sdk_config.h?
Since using critical sections seem to help - what do you expect is interrupting when initializing the RNG?
Can you provide an example project that demonstrate this?

2. This is also new to me, so I have a few questions here as well:

Which toolchain do you use?
Does it depend on optimization level or other settings?
Can you provide an example project that demonstrate this?