sd_ecb_block_encrypt() execution time

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Wed, 31 May 2023 10:58:05 GMT

[quote user="pengi"]Regarding this workaround, I don't think that's a problem as long as we know what to expect.[/quote]

I have confidence in the workaround and it is not a complex one either. If you do not have anything else then please mark this reply as verified so as to help other community members in the future.

RE: sd_ecb_block_encrypt() execution time

Max Sikström — Wed, 31 May 2023 10:29:48 GMT

That makes sense. Think I've heard about that too.

Regarding this workaround, I don't think that's a problem as long as we know what to expect.

Thanks!

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Wed, 31 May 2023 10:22:23 GMT

Max, Thanks for confirming that the fix works.

[quote user="pengi"]How long should we keep this workaround, and what should we look for in the release notes if it gets fixed?[/quote]

Unfortunately, I do not think we are releasing any baremetal Softdevice(s) versions anymore. The fix for this has been applied to Softdevice Controller (Zephyr based Nordic BLE controller). So you need to maintain this workaround in your application as long as you use nRF5 SDK and not the latest Nordic Connect SDK.

[quote user="pengi"]Will this become a problem if we use it on the s132 softdevice too, or do that have the same issue?[/quote]

This part of the Softdevice code is common for both S132 and S140. So, Yes, you need to maintain this code for both versions of your application.

RE: sd_ecb_block_encrypt() execution time

Max Sikström — Wed, 31 May 2023 10:11:58 GMT

Oh, also one more thing.

Will this become a problem if we use it on the s132 softdevice too, or do that have the same issue?

We are running the same project on both nrf52840 and nrf52832, and we want to keep the codebase as common as possible

RE: sd_ecb_block_encrypt() execution time

Max Sikström — Tue, 30 May 2023 14:28:15 GMT

Hi,

Sorry for the delays. But I have finally managed to test it. I haven't hooked up an oscilloscope yet, but I've tried without the fix and then with the fix in our main system where we got the packet loss due to missing the deadlines.

I can confirm that the fix/workaround works fine. So no issue with missing the deadline we have. Have been sending a couple of hundred packets with 100% PDR it seems now. Before, I might have had one or two of those to squeeze through before.

Thanks for the help :)

One last question then:

How long should we keep this workaround, and what should we look for in the release notes if it gets fixed?

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Wed, 24 May 2023 18:32:09 GMT

I am confident that I tested this well on my end. But this is firmware workaround and we cannot be 100% sure unless you see the same. Good luck with your tests.

RE: sd_ecb_block_encrypt() execution time

Max Sikström — Wed, 24 May 2023 10:03:28 GMT

I'm sorry not. I got some more urgent issues here that I needed to fix. It is still next in the pipeline after those issues though. But will take a day or two longer.

If your tests where deterministic, I feel comfortable that it will work, so I'm planning to go directly to test it in our main system. But for that I need to set up at least two nodes running with correct versions.

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Wed, 24 May 2023 07:33:02 GMT

Did you get a chance to test this Max? Just curious.

RE: sd_ecb_block_encrypt() execution time

Max Sikström — Mon, 22 May 2023 12:55:37 GMT

Thanks. I'll try that tomorrow when I have access to the setup again. Looks promising.

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Mon, 22 May 2023 12:13:26 GMT

I know that there was a fix inside sd_ecb_block_encrypt applied in S140v7.2.0 to solve a power consumption issue which is relevant when we use SEVONPEND bit. The inclusion of this fix seems to have another side affect of wakeup time being longer in many cases. I see that this fix inside the sd_ecb_block_encrypt is removed later in our repo was not being released in nRF5 SDK.

I think you can have a workaround to temporarily disable the SEVONPEND while requesting to block encrypt, something like below.

void ENCRYPT_IRQHandler(
    void)
{
    int i;
    ENCRYPT_GPIO_PORT->OUTSET = 1 << ENCRYPT_GPIO_TRIG_PIN;
    for (i = 0; i < 20; i++) {
        /* Just fill with something */
        memset(&ecb_buf, i, sizeof(nrf_ecb_hal_data_t));

        /* Disable temporarily */
        SCB->SCR &= ~SCB_SCR_SEVONPEND_Msk;

        ENCRYPT_GPIO_PORT->OUTSET = 1 << ENCRYPT_GPIO_ENC_PIN;
        sd_ecb_block_encrypt(&ecb_buf);
        ENCRYPT_GPIO_PORT->OUTCLR = 1 << ENCRYPT_GPIO_ENC_PIN;

        /* Re-enable */
        SCB->SCR |= SCB_SCR_SEVONPEND_Msk;
    }
    ENCRYPT_GPIO_PORT->OUTCLR = 1 << ENCRYPT_GPIO_TRIG_PIN;
}

I have tested this on my end and it seems to work. Can you please test the above changes of code in your vendor\nRF5_SDK_17.0.2_d674dde\examples\peripheral\freertos-nrf-encrypt-timings-main\src\encrypt.c

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Mon, 22 May 2023 11:47:56 GMT

Wow, That was interesting. I see the same thing that you wrote here Max.

You have given me everything that I need to dig into. I will comeback to you when I have more info or workaround for this. Please expect a day or two, while I am at this.

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Mon, 22 May 2023 07:58:39 GMT

Max, Sorry for late reply, we had a long weekend due to some holidays here in Norway.

Thanks for attaching a simplistic project. I will take a look at it today.

RE: sd_ecb_block_encrypt() execution time

Max Sikström — Thu, 18 May 2023 07:38:09 GMT

That's a good theory, I'm not entirely sure that's the case though. Since it's not the latency for the interrupt to start that is the issue, but execution time of the function sd_ecb_block_encrypt() itself. All GPIO pins are controlled from within the interrupt, and all 20 executions of sd_ecb_block_encrypt() behaves the same.

It would be if sd_ecb_block_encrypt() itself puts the CPU and flash to sleep during execution. But then, I would think all executions would be equally fast without debugger. But they are faster in our full system not running FreeRTOS.

But is it possible to force the flash to be always on from software during the execution of that interrupt? If so, that would be something easy to test.

RE: sd_ecb_block_encrypt() execution time

Emil Lenngren — Thu, 18 May 2023 06:52:48 GMT

Are you running your interrupt code from RAM or from flash? Usually after an interrupt triggers, if code from flash starts to run, something like 10 us are needed for starting up the flash. When the debugger is connected, flash is always on so this time is eliminated.

RE: sd_ecb_block_encrypt() execution time

Max Sikström — Wed, 17 May 2023 19:17:40 GMT

Hi,

I have managed to simplify it a lot. I don't now the best way to send it so it persists in this ticket for other people to read in the future. But I have for now created a github repository containing the project.

Since it involves setup of softdevice, and a few other components, it is multiple files.

It can be found here:

https://github.com/pengi/freertos-nrf-encrypt-timings

And the trace without debugger connected can be seen as:

But as soon as I connect with gdb, I get this result, no settings is changed on the oscilloscope:

And it is for me really deterministic.

I am running on a PCA10056 v1.1.0

Ask if I should check any CPU variants or revisions that I missed. But I guess this is where that can be read out:

FICR.DEVICEADDRTYPE              = ffffffff
    DEVICEADDRTYPE                 .......1 - Random
FICR.DEVICEADDR[0]               = 5acc59e0
    DEVICEADDR                     5acc59e0 - 5acc59e0
FICR.DEVICEADDR[1]               = 78b66b84
    DEVICEADDR                     78b66b84 - 78b66b84
FICR.INFO_PART                   = 00052840
    PART                           00052840 - N52840
FICR.INFO_VARIANT                = 41414430
    VARIANT                        41414430 - AAD0
FICR.INFO_PACKAGE                = 00002004
    PACKAGE                        00002004 - QI
FICR.INFO_RAM                    = 00000100
    RAM                            00000100 - K256
FICR.INFO_FLASH                  = 00000400
    FLASH                          00000400 - K1024

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Thu, 11 May 2023 06:15:14 GMT

[quote user="pengi"]I can make an attempt to scale it down again, but that unfortunately has to wait until next week due to some other more urgent issues here.[/quote]

I understand. Thanks for considering to attempt to make a simplistic project. It will make both of our debugging path a bit easier.

RE: sd_ecb_block_encrypt() execution time

Max Sikström — Wed, 10 May 2023 07:26:08 GMT

No worries. This is really a vague problem statement, and the fact I can't really reproduce it either in more than some scenarios that seems unrelated makes it a bit harder.

The issue is that it happens in one setup we have with out full system, but not others, means that the project we have to reproduce it with is our entire system, including our IP. So sending the entire project over is not really possible.

I can make an attempt to scale it down again, but that unfortunately has to wait until next week due to some other more urgent issues here.

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Tue, 09 May 2023 11:25:40 GMT

Max,

I am having a bit of a hard time trying to create a simplistic project to test solely this ecb encrypt function. Instead of me trying to spend so much time on trying to configure my project to get to your, can you please attach your project here. Sorry for making you wait while I attempted to make a simplistic project to test this. But I am not able to use sd_ecb_block_encrypt outside peer manager and I do not want to test the latencies of sd_ecb_block_encrypt under peer manager umbrella as there might be many other things happening under the hood.

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Fri, 05 May 2023 11:03:32 GMT

I am sorry, I did not manage to look at it today, I will have to create a small setup to be able to test this, most likely I will do it early next week. I think you have given clear description and I think I would manage to try to test this time with and without FreeRTOS.

RE: sd_ecb_block_encrypt() execution time

Max Sikström — Thu, 04 May 2023 13:02:37 GMT

This is actually what confuses me a bit too, since it seems like most of the things that differ is totally unrelated. And what I find even more interesting is that just connecting with gdb, and type "mon reset" and "continue" will also "fix" the problem for the exact same executable.

I have unfortunately no clue how to break it down further to isolate the issue, but happy to try to pinpoint it further if there are parts that may differ, for example if it is a clock issue, if I can get a clue of what peripherals or settings to look for, given what that function can do. But since that function to me is a black box, I don't know what peripherals it depends on.

For the different scenarios we have we first need to look at what we are making. We are making a network stack, which has code running in interrupt level, and then a main loop internall. That network stack we are delivering to customers as a library. It is possible to run the main loop as a task in FreeRTOS, or use the main loop and process handling (based on contiki) as base for the program. That means, it is possible to run our library with or without FreeRTOS.

But the part related to encryption is, even though quite deep down in the code, only dependent on this RADIO or TIMER0 interrupt running via the time slotting API waking up a lower priority interrupt via nrf_egu_task_trigger(...) to drop priority, and then do the calls. So I can not see how interrupts of lower priorities, or even thread level code, affects that call. And even just some in the same interrupt.

I was thinking of checking if something else was running and taking up time using a lauterbach utrace, but didn't manage to get that running without also having SWD and debug session going, which means it won't trigger the issue.

So this is actually where I'm lost. I want to isolate it, but I need input and help of what has the possibility to affect the runtime, to know what to look for. Or even if there is any information that I should provide here. And it seems to be affected by so seemingly unrelated parts, I'm not sure I can manage to reproduce it in an isolated environment either.

I get it's hard to give a clean explanation for it with the information I provide, but I'm happy to investigate myself, just that some input or thoughts of where I should continue to look, even if it turns out to be a dead end, would be helpful.

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Thu, 04 May 2023 11:22:31 GMT

[quote user=""]I have measured that the call to sd_ecb_block_encrypt() and a block copy takes 15.8us in every case we have without running FreeRTOS, or when using that version of FreeRTOS while connected to a debugger, but in running state.[/quote]

Can you please define what a "running state" and without FreeRTOS means here? I think this is super interesting as it seems like FreeRTOS might be doing something it was not supposed to do in terms of masking non application specific interrupts. I will anyhow have to reproduce this in a minimalistic way. For that I need to understand what running state and without freertos means here? I am guessing it just means that you have tested with and without starting the freertos scheduler?

RE: sd_ecb_block_encrypt() execution time

Susheel Nuguru — Thu, 04 May 2023 10:58:06 GMT

Oh Wow,

This is super interesting, Just calling sd_ecb_block_encrypt() in FreeRTOS have different execution times even if it is configured to run in high priority. I will have to re-read your description and test few things. Will be back to you with hopefully somemore meaningful info.