Application won't boot when bootloader and app are using 32kHz RC oscillator. When using MPSL (BLE and IEEE802.15.4 raw) or usb_cdc.

[Maarten de Kruijf]

Hi,

We're having a problem that our application will hang when using the 32 kHz RC oscillator in the mcuboot bootloader and the Zephyr application. We encountered this problem when we updated from NCS 1.5.1 to 1.8.0. 

We have looked at https://devzone.nordicsemi.com/f/nordic-q-a/81418/bug-in-ncs-v1-7-1-with-32khz-rc-oscillator-and-mcuboot-enabled and https://github.com/zephyrproject-rtos/zephyr/issues/40538 but that does not seems to fix our issue or is not included in release 1.8.0

In the supplied files is a markdown file with steps how to reproduce the problem.

What could be a temporary fix? 

What is the prospect of a permanent fix?

Best regards,

Maarten

rc_oscillator_problem_files.zip

[Maarten de Kruijf]

Hi Hung Bui,

I've some extra info,

during debuging the debugger is stuck in: static void lfclk_spinwait(enum nrf_lfclk_start_mode mode);

github.com/.../clock_control_nrf.c

Additionally after some time the non working example seems to sync its clock and run the application when I've build them myself.

This behaviour is very intermittent and for our product the watchdog will reset before the clocks have synced.

I've tried to install the Nordic toolchain manager on Linux but that does not work.

I've also tried the cut xtal traces (SB1 and SB2) with your hex which works but when I build mine and flash it the bootloader won't boot.

As I've setup a clean environment on Linux I'm a bit out of options how to reproduce the working hex.

Maybe if you build example 3 it will reproduce the problem on your side. (see line below)

 

west build -p -b nrf52840dk_nrf52840\
-- -DDTC_OVERLAY_FILE=console.overlay\
-DOVERLAY_CONFIG=radio.conf\
-Dmcuboot_OVERLAY_CONFIG=$(realpath mcuboot.conf)\
-DPICOO_RADIO=ON && west flash

Lastly I want to underscore that the issue is intermittent and a show stopper for updating at this point.

Best regards,

Maarten

[Hung Bui]

Hi Maarten, 

Could you clarify: 

- If you run my hex file, would it work all the time ? 

- If you build the blinky example without mcuboot, would it works with the 32kHz RC configuration ? 

- Have you tested with other DK ? 

My test here with the MPSL\timeslot example showed that the application hang when using: 

CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC=y

CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC_CALIBRATION=y

CONFIG_CLOCK_CONTROL_NRF_CALIBRATION_MAX_SKIP=0

It doesn't hang when I remove CONFIG_CLOCK_CONTROL_NRF_CALIBRATION_MAX_SKIP=0 

It's definitely a bug with our Softdevice controller because if I use CONFIG_BT_LL_SW_SPLIT=y (Zephyr controller) then it worked. 
But if I use 

CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC=y

CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC_CALIBRATION=y

CONFIG_BOOTLOADER_MCUBOOT=y
It also worked fine. So it doesn't seem (at least on my setup) there is an issue with MCUBoot when using 32kHz RC without using CONFIG_CLOCK_CONTROL_NRF_CALIBRATION_MAX_SKIP=0. 

Attached is the project and the hex file that you can try.

timeslot.zip

[Maarten de Kruijf]

Hi Hung,

- When I run the hex file you provided the application works on both devkits I have without any problems.

- When I use the example I provided (blinky) without bootloader the application works fine. The issue occurs when we use a different clock configuration for the bootloader and the application. We currently have a bootloader configured with a low frequency clock source: CONFIG_CLOCK_CONTROL_NRF_K32SRC_SYNTH. But it also doesn't work with the default config (CONFIG_CLOCK_CONTROL_NRF_K32SRC_XTAL). Only when both are set to the _RC oscillator.

- We have three devkits (myself 2 and my college 1) the behaviour is identical on all three. So your example works and when we compile it, we have the boot problem of the application and it works intermittent.

For the second part I've tested your configurations and find the same findings as you do.

The application won't boot when CONFIG_CLOCK_CONTROL_NRF_CALIBRATION_MAX_SKIP=0 is enabled.

As we currently have a bootloader configured with a low frequency clock source: CONFIG_CLOCK_CONTROL_NRF_K32SRC_SYNTH. As we can't replace the bootloader in the field we are stuck with the current configuration. For MPSL we need to use the CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC in the application as the synthesised clock is not supported. 

What would be a viable path to get to a solution, or do we need to wait until this bug is fixed in the softdevice? 

Best regards,

Maarten

[Maarten de Kruijf]

Also I won't be available next week so Martijn (my college) will continue responding on this thread next week.

[Hung Bui]

Hi Maarten, 

Ok so the MCUBoot is fixed to CONFIG_CLOCK_CONTROL_NRF_K32SRC_SYNTH. This is the new information that hasn't included in the original question or the howtoreproduce.md. 

It would be a challenge to have different RC source between the MCUBoot and the application. I'm checking internally to see if there is any solution. 

[Martijn Stommels]

Hi Hung,

Thank you for checking.

I agree switching source isn't ideal, and kind of an edge case. However, since we already deployed thousands of units with the bootloader set to the SYNTH clock, we can't really change this.

This did work perfectly in NCS 1.5, so the hardware does seem to be capable of switching back.

Regards,

Martijn Stommels