Board reboots when a function is executed in TF-M

[Pablosanserr]

Hi,

I'm moving a project from SPM to TF-M. I have imported a static library and the board (nRF5340DK) restarts automatically every time the following mycli/main.c function is executed: signature.

signature(pk, msg, buffer);

More specifically, the problem occurs when executing blst_sign_pk_in_g1 function, which is located in lib/bls_hsm.h.

blst_sign_pk_in_g1(&sig, &hash, secret_keys_store + pk_index*sizeof(blst_scalar));

Other functions from the static library are executed correctly, so I guess the problem is not in the import of the library, but must be related to memory. In case it is of any interest, a few weeks ago I had problems importing such a static library into TF-M. The problem was solved as indicated in the following forum ticket: Problem importing a library into TF-M: Not enough space to import i.

I attach a link to the github project repo, so that the error can be easily reproduced: bls-hsm-2. To reproduce the error, build and execute application mycli. You will see that the board reboots.

Thank you in advice,

Pablo

Useful information about the project:
- I'm using the nRF5340DK development kit.
- nRF SDK v2.0.0
- The project is based on the example: TF-M Secure Partition Sample. When doing the build for this example, the available and used space shown is as follows:

[dejans]

Hi,

I have reproduced your issue. I'll look into it more tomorrow.

Best regards,
Dejan

[dejans]

Hi,

When doing the build for this example, the available and used space shown is as follows:

I guess something is missing. I don't see anything after "follows:". Could you please resend what you wanted to send initially?

Best regards,
Dejan

[dejans]

Hi Pablo,

From my TF-M log, UsageFault Status Register (UFSR) has the value 0x10. You can check various fields of UFSR in the ARM documentation.  

Best regards,
Dejan

[Pablosanserr]

Hi Dejans,

As you can see in the following TF-M log, I also get an UsageFault error. The value of UFSR is 0x10.

I have looked at the meaning of the UFSR fields in the ARM documentation. The hexadecimal value 0x10 is equivalent to bit 2 with value 1. According to the ARM documentation, this is an error with name STKOF: "Stack overflow flag. Sticky flag indicating whether a stack overflow error has occurred".

As I guessed, it seems that there is not enough size in the safe partition stack to run this function. The following issue from the blst repo reports a failure with the same function. In the end, it concludes that it was a stack size problem. It worked for him by allocating more than 20 kB. Do I have a way to see the available stack size or increase it, for the execution of the functions in the secure partition?

Best regards,
Pablo

[dejans]

Hi Pablo,

There is this line in the tfm_secure_peripheral_partition.yml which specifies the size of the stack.

Best regards,
Dejan

[Pablosanserr]

Hi Dejan,

Increasing stack memory didn't work for me, so I used a sample project for you to check it. I don't know what I am doing wrong. I'm using nRF SDK v2.0.0 and nRF5340DK.

tfm_secure_partition_stacksize.zip

CheckMemory is a function that allocate memory using malloc, VLA or alloca. This function is executed from secure partition.

static void CheckMemory(int location, int bytes){
	if(location == 0){
		// HEAP
		printf("Allocating %d bytes using malloc (HEAP)...\r\n", bytes);
		void * m = malloc(bytes);
		printf(" -> HEAP: OK\r\n");
		free(m);
	}else if(location == 1){
		printf("Allocating %d bytes using VLA (STACK)...\r\n", bytes);
		// STACK 1
		char c[bytes];
		printf(" -> STACK 1: OK\r\n");
	}else if(location == 2){
		printf("Allocating %d bytes using alloca (STACK)...\r\n", bytes);
		// STACK 2
		void * c = alloca(bytes);
		printf(" -> STACK 2: OK\r\n");
	}
}

Using CheckMemorySIze with bytes = 4500:

Using CheckMemorySize with bytes = 9000:

As you can see, VLA success if bytes = 4500, but it fails if bytes = 9000. Alloca fails too. VLA and alloca uses stack, so increasing stack size should be enough. Following your recommendation, I increased stacksize from 0x800 to 0x8000 (tfm_dummy_partition.yaml), but it didn't worked for me. It fails again:

P.S: By default, stack size is 0x800 (2048 bytes), but I can use VLA and alloca with 4500. Does it make sense?

Best regards,
Pablo

[dejans]

Hi Pablo,

There are many warnings in your provided sample.

The results of testing your current sample tfm_secure_partition_stacksize are shown below.

Application log



TF-M log



Best regards,
Dejan

[Pablosanserr]

Hi Dejan,

Here you have an updated version of the sample project, with warnings removed. There are only 2 warnings, related with unused variables. These variables are used for testing VLA and alloca. There should be no problem with these warnings.

6874.tfm_secure_partition_stacksize.zip

I think you need to enable the "Enable debug options" option for the build. This should allow you to see the TF-M log that I have shown in my last message.

To reproduce the error, you must modify the size given as a parameter in the CheckMemory function, which is used in dummy_partition/dummy_partition.c. With a value of 9000 there is an error, but with a value of 4500 there is no error.

printf("Let's check memory using malloc, VLA and alloca\r\n");
for(int i = 0; i <= 2; i++){
	CheckMemory(i, 9000);
}

You can also check, as I said in the previous message, that modifying the stack size in dummy_partition/tfm_dummy_partition.yaml has no effect.

"stack_size": "0x8000"

Best regards,
Pablo

[dejans]

Hi Pablo,

I have tested your sample project and managed to reproduce your issue. 
In your code, inside CheckMemory() function, you should be careful how you use array and alloca() function.
The size of the array could potentially be as you have set it. However, there is no guarantee that this number of bytes would be available at run-time. As a result, you may not have enough stack space. Allocation on the stack with function alloca() may produce stack overflow with very large values. Alloca() man page specifies "The alloca() function returns a pointer to the beginning of the allocated space. If the allocation causes stack overflow, program behavior is undefined." Therefore, you may consider using alloca() with fewer bytes. 
I have also modified and tested your sample with char c[10] and void *c=alloca(10) and everything was fine. I would suggest that you use less bytes in your array declaration as well as in the alloca() function.

Best regards,
Dejan

[Pablosanserr]

Hi Dejan,

Thank you for the clarification. However, my problem is not that alloca fails with a large number. As I commented in my previous post, when I increase stack_size the way you told me it doesn't seem to have had an effect.

Using alloca with 4500 bytes and with a stack size of 0x800 works correctly. However, duplicating the number of bytes for alloca (9000 bytes) and greatly increasing the stack_size to 0x8000 does not work.

Best regards,
Pablo

[dejans]

Hi Pablo,

I have made an internal inquiry related to your issue. I'll come back to you when I get new information, hopefully till the end of next week.

Best regards,
Dejan

[dejans]

Hi Pablo,

Your issue might be related to some kind of a bug in your project. I would suggest that you start with simple sample and start developing from there one step at the time. Make sure to test along the way and do not introduce many simultaneous changes into your project during the development.

Best regards,
Dejan