Enabling CONFIG_FPU causes stack overflow

[olegr]

Problem: stack overflow in BT threads when CONFIG_FPU is enabled.

[00:00:00.034,881] <err> os: esf_dump: r0/a1: 0x00000002 r1/a2: 0x200038dc r2/a3: 0xf0f0f0f0
[00:00:00.034,912] <err> os: esf_dump: r3/a4: 0x20004e00 r12/ip: 0x00000014 r14/lr: 0x00027aab
[00:00:00.034,942] <err> os: esf_dump: xpsr: 0x41000000
[00:00:00.034,942] <err> os: esf_dump: s[ 0]: 0x00000000 s[ 1]: 0x00000000 s[ 2]: 0x00000000 s[ 3]: 0x00000000
[00:00:00.034,973] <err> os: esf_dump: s[ 4]: 0x00000000 s[ 5]: 0x00000000 s[ 6]: 0x00000000 s[ 7]: 0x00000000
[00:00:00.034,973] <err> os: esf_dump: s[ 8]: 0x00000000 s[ 9]: 0x00000000 s[10]: 0x00000000 s[11]: 0x00000000
[00:00:00.035,003] <err> os: esf_dump: s[12]: 0x00000000 s[13]: 0x00000000 s[14]: 0x00000000 s[15]: 0x00000000
[00:00:00.035,003] <err> os: esf_dump: fpscr: 0x00000000
[00:00:00.035,034] <err> os: esf_dump: Faulting instruction address (r15/pc): 0x0001f608
[00:00:00.035,064] <err> os: z_fatal_error: >>> ZEPHYR FATAL ERROR 2: Stack overflow on CPU 0
[00:00:00.035,125] <err> os: z_fatal_error: Current thread: 0x20001c68 (BT_LW_WQ)
*** Booting Zephyr OS build v3.2.99-ncs2-rc1 ***l_error_handler: Resetting system

I'm using the basic peripheral_hr sample from the sdk and enabling CONFIG_FPU=y in prj.conf. After that board will crash several times with stack overflow in some BT thread until it manages to come up. Connecting to the peripheral then causes further crashes.

I'm using a PCA10056 devkit and vscode nrf connect extension, SDK and toolchain version 2.3.0-rc1.

My understanding that it should be possible to use FPU on nrf52840, why does this failure happen?

To reproduce the failure, create a new freestanding app using peripheral_hr and add the following to prj.conf:

CONFIG_FPU=y
CONFIG_STACK_SENTINEL=y
CONFIG_THREAD_NAME=y

[olegr]

The problem came down to having CONFIG_STACK_SENTINEL=y. Apparently it is not compatible with CONFIG_FPU. More info in

https://github.com/zephyrproject-rtos/zephyr/issues/32261

I wish KConfig prevented one from enabling CONFIG_STACK_SENTINEL and CONFIG_FPU together.

My reason for enabling CONFIG_STACK_SENTINEL was because I thought MPU (hardware stack protection) was not working. But I now realize that CONFIG_UART_INTERRUPT_DRIVEN=y prevented printing of crash messages.

I only needed UART1 to be interrupt-driven (and UART0 is the debug console and should be async/dma based). That can be done, seeCan UART ASYNC and UART INTERRUPT work together?  and with that I can see MPU crashes on the console and do not need CONFIG_STACK_SENTINEL.

[tesc]

Hi,

Thank you for the detailed report and quick way to reproduce, as well as update with your further findings and links to related Zephyr issues on Github.

olegr said:

I wish KConfig prevented one from enabling CONFIG_STACK_SENTINEL and CONFIG_FPU together.

From what I understand from the github issue, as well as from the possibly related https://github.com/zephyrproject-rtos/zephyr/issues/37608, CONFIG_STACK_SENTINEL and use of MPU is the main conflict, not the use of FPU? It was suggested in the discussion there that the FPU error may be caused by increased stack usage. Has this been ruled out?

Regards,
Terje

[olegr]

tesc said:

It was suggested in the discussion there that the FPU error may be caused by increased stack usage.

That seems reasonable. In my later debugging I've noticed that at least one BT thread (BT ECDH) appears to be using FPU when it is available, so it would need more stack on context switches?

I did check that without CONFIG_FPU=y, the crash does not happen.