MPSL Timer 0 Hard Fault

RE: MPSL Timer 0 Hard Fault

Kenneth — Tue, 28 Sep 2021 09:08:34 GMT

If there are no improvements, then please look at the other case you have open on where you can ship hardware here so we may take a look.

Kenneth

RE: MPSL Timer 0 Hard Fault

csteaderman — Mon, 27 Sep 2021 16:17:47 GMT

Kenneth,

I will try at 500ppm, although I already tried at a higher PPM with no change in behavior. I also tried with internal RC clock with no change in behavior.

I tried updating to ncs 1.6.1 (latest, I thought), with no change in behavior.

RE: MPSL Timer 0 Hard Fault

Kenneth — Mon, 27 Sep 2021 07:28:52 GMT

Hello,

I was out of office the last few days last week. Can you try to set tolerance to 500ppm for test?

Are you able try ncs v1.7.0 for comparison/test?

Kenneth

RE: MPSL Timer 0 Hard Fault

csteaderman — Fri, 24 Sep 2021 14:13:09 GMT

Is there a way to escalate this issue in order to get a resolution? I have exhausted all causes that I can imaging on our custom board and firmware and desperately need assistance to get this resolved. Our development and production timelines are in great risk at this point if I cannot get Bluetooth functional on our boards with the QFAB part. Please let me know my options as soon as possible.

RE: MPSL Timer 0 Hard Fault

csteaderman — Thu, 23 Sep 2021 19:29:24 GMT

Running my application in Segger Ozone, I see that when the fault occurs, the BAR and MMFAR registers have the contents 0xE000EDF8, which appears to be a "Private Peripheral Bus - Internal" address block. What is there, and why does it only fail on the QFAB part?

RE: MPSL Timer 0 Hard Fault

csteaderman — Thu, 23 Sep 2021 16:43:19 GMT

The m_assert_handler is what gets called, and the line number is 2163. Can this be used to narrow in on the specific condition that is being detected within the SoftDevice?

RE: MPSL Timer 0 Hard Fault

csteaderman — Wed, 22 Sep 2021 15:17:18 GMT

Yes, same source and < 20 ppm accuracy.

RE: MPSL Timer 0 Hard Fault

Kenneth — Wed, 22 Sep 2021 13:24:22 GMT

There shouldnt' really be any difference here, are you sure you don't have any code that are disabling interrupts and/or running application interrupts on an invalid interrupt priority, for instance using IRQ_DIRECT_CONNECT()?

Are you using the same LFCLK source and tolerance for both DK and custom board?

RE: MPSL Timer 0 Hard Fault

csteaderman — Tue, 21 Sep 2021 17:03:39 GMT

Do you think it is possible that the same code running on an nRF52832AB part / our custom board might be running slightly slower than the nRF52832AA part / DK and that the MPSL_IRQ_TIMER0_Handler() is picking this up and generating a fault? Do you have a suggestion about how I might confirm this?

RE: MPSL Timer 0 Hard Fault

Kenneth — Tue, 21 Sep 2021 07:06:01 GMT

[quote user="csteaderman"]I select Run, wait 5 seconds or so, pause execution, then Continue and it immediately crashes in the same place as my code, except that mine hits the fault without having to pause and continue.[/quote]

The softdevice have real time requirements with various timers, ppi, task and events. It does not support pausing/breaking program flow from a debugger, this will cause the code to quickly hardfault or assert when continue to run the code. This can also occur if the application prevent the softdevice from executing code for a period of time (e.g. disabling interrupts and/or using very high interrupt priority.)

Best regards,
Kenneth

RE: MPSL Timer 0 Hard Fault

csteaderman — Mon, 20 Sep 2021 23:43:25 GMT

RE: MPSL Timer 0 Hard Fault

csteaderman — Mon, 20 Sep 2021 23:42:55 GMT

No suggestions?

I have determined a way to generate the same MPSL Timer 0 Fault using an out-of-the-box Nrf52DK board and an example project. I loaded the peripheral_hids_mouse (although I believe any BT example project will do) and selected the nRF52DK board (as shown below). I run it in the debugger. It compiles, loads, and stops at the first line of main(). I select Run, wait 5 seconds or so, pause execution, then Continue and it immediately crashes in the same place as my code, except that mine hits the fault without having to pause and continue. Since this area of the program is in an obfuscated library, I cannot debug to determine what is failing. Please reproduce this condition and let me know what is causing the fault (I assume that you have access to the source code for the SoftDevice and can debug what's going on). If you are unable, I am going to need at least some parts of the SoftDevice source code so that I can debug and figure out what is going on.

RE: MPSL Timer 0 Hard Fault

csteaderman — Thu, 16 Sep 2021 11:40:33 GMT

Kenneth,

I’ve cleaned the folder, and also had already done step 3. Still same problem.

RE: MPSL Timer 0 Hard Fault

Kenneth — Thu, 16 Sep 2021 07:39:52 GMT

[quote user="csteaderman"]I will build with --prestine.[/quote]

This parameter can be used if you build with "west" in command line, if you are building with SES you can either delete build folder or re-open the project should do the same thing.

[quote user="csteaderman"]What do you mean by "this approach"?[/quote]

My 3 step procedure I posted initially.

Kenneth

RE: MPSL Timer 0 Hard Fault

csteaderman — Wed, 15 Sep 2021 19:43:55 GMT

--pristine is not a valid compiler. I'm using SES with Nordic Zephyr/nRF Connect SDK.

1> Compiling ‘offsets.c’
1> arm-none-eabi-gcc: error: unrecognized command line option '--pristine'

RE: MPSL Timer 0 Hard Fault

csteaderman — Wed, 15 Sep 2021 19:14:01 GMT

Opened one of the standard projects, changed uP to the AB variant, disabled "Settings". Here is the debug output:

[00:00:07.730,560] [0m<inf> bt_hci_core: HW Platform: Nordic Semiconductor (0x0002)[0m
[00:00:07.730,560] [0m<inf> bt_hci_core: HW Variant: nRF52x (0x0002)[0m
[00:00:07.730,590] [0m<inf> bt_hci_core: Firmware: Standard Bluetooth controller (0x00) Version 217.17378 Build 3224189809[0m
[00:00:07.731,597] [0m<inf> bt_hci_core: Identity: DC:8F:81:FA:69:B1 (random)[0m
[00:00:07.731,628] [0m<inf> bt_hci_core: HCI: version 5.2 (0x0b) revision 0x11c2, manufacturer 0x0059[0m
[00:00:07.731,628] [0m<inf> bt_hci_core: LMP: version 5.2 (0x0b) subver 0x11c2[0m
[00:00:07.740,478] [1;31m<err> mpsl_init: MPSL ASSERT: 112, 2163[0m
[00:01:04.649,291] [1;31m<err> os: ***** HARD FAULT *****[0m
[00:01:04.649,322] [1;31m<err> os: Fault escalation (see below)[0m
[00:01:04.649,322] [1;31m<err> os: r0/a1: 0x00000003 r1/a2: 0x20006850 r2/a3: 0x20001860[0m
[00:01:04.649,322] [1;31m<err> os: r3/a4: 0x00000007 r12/ip: 0x20000f04 r14/lr: 0x00010907[0m
[00:01:04.649,322] [1;31m<err> os: xpsr: 0x41000018[0m
[00:01:04.649,322] [1;31m<err> os: Faulting instruction address (r15/pc): 0x0001b562[0m
[00:01:04.649,353] [1;31m<err> os: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0[0m
[00:01:04.649,353] [1;31m<err> os: Fault during interrupt handling
[0m
[00:01:04.649,353] [1;31m<err> os: Current thread: 0x20001860 (unknown)[0m
[00:01:05.233,612] [1;31m<err> fatal_error: Resetting system[0m

RE: MPSL Timer 0 Hard Fault

csteaderman — Wed, 15 Sep 2021 16:39:57 GMT

I will build with --prestine.

What do you mean by "this approach"?

RE: MPSL Timer 0 Hard Fault

Kenneth — Wed, 15 Sep 2021 14:58:51 GMT

Can you try this approach? Also build with --pristine.

Kenneth

RE: MPSL Timer 0 Hard Fault

csteaderman — Wed, 15 Sep 2021 14:51:49 GMT

Kenneth,

I basically did the same thing. The strange thing is that the same binary works on the DK, but throws exceptions on the custom hardware.

RE: MPSL Timer 0 Hard Fault

Kenneth — Wed, 15 Sep 2021 14:45:55 GMT

(EDIT: I noticed I used the nRF51422 as an example, but this should be same for nRF52832.)

1. Not sure how you have setup your board to use the nRF51822qfab, but I would have simply copied the nrf51dk_nrf51422 board directory to nrf51dk_nrf51422qfab directory.

2. Renamed the 3 files in the nrf51dk_nrf51422qfab directory to nrf51dk_nrf51422qfab, something like:

3. Then finally edit the .dts file to use:

#include <nordic/nrf51822_qfab.dtsi> // instead of nordic/nrf51822_qfac.dtsi

Best regards,
Kenneth