Application won't boot when bootloader and app are using 32kHz RC oscillator. When using MPSL (BLE and IEEE802.15.4 raw) or usb_cdc.

[Maarten de Kruijf]

Hi,

We're having a problem that our application will hang when using the 32 kHz RC oscillator in the mcuboot bootloader and the Zephyr application. We encountered this problem when we updated from NCS 1.5.1 to 1.8.0. 

We have looked at https://devzone.nordicsemi.com/f/nordic-q-a/81418/bug-in-ncs-v1-7-1-with-32khz-rc-oscillator-and-mcuboot-enabled and https://github.com/zephyrproject-rtos/zephyr/issues/40538 but that does not seems to fix our issue or is not included in release 1.8.0

In the supplied files is a markdown file with steps how to reproduce the problem.

What could be a temporary fix? 

What is the prospect of a permanent fix?

Best regards,

Maarten

rc_oscillator_problem_files.zip

[Maarten de Kruijf]

Hi Hung Bui,

I've some extra info,

during debuging the debugger is stuck in: static void lfclk_spinwait(enum nrf_lfclk_start_mode mode);

github.com/.../clock_control_nrf.c

Additionally after some time the non working example seems to sync its clock and run the application when I've build them myself.

This behaviour is very intermittent and for our product the watchdog will reset before the clocks have synced.

I've tried to install the Nordic toolchain manager on Linux but that does not work.

I've also tried the cut xtal traces (SB1 and SB2) with your hex which works but when I build mine and flash it the bootloader won't boot.

As I've setup a clean environment on Linux I'm a bit out of options how to reproduce the working hex.

Maybe if you build example 3 it will reproduce the problem on your side. (see line below)

 

west build -p -b nrf52840dk_nrf52840\
-- -DDTC_OVERLAY_FILE=console.overlay\
-DOVERLAY_CONFIG=radio.conf\
-Dmcuboot_OVERLAY_CONFIG=$(realpath mcuboot.conf)\
-DPICOO_RADIO=ON && west flash

Lastly I want to underscore that the issue is intermittent and a show stopper for updating at this point.

Best regards,

Maarten

[Hung Bui]

Hi Maarten, 

Could you clarify: 

- If you run my hex file, would it work all the time ? 

- If you build the blinky example without mcuboot, would it works with the 32kHz RC configuration ? 

- Have you tested with other DK ? 

My test here with the MPSL\timeslot example showed that the application hang when using: 

CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC=y

CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC_CALIBRATION=y

CONFIG_CLOCK_CONTROL_NRF_CALIBRATION_MAX_SKIP=0

It doesn't hang when I remove CONFIG_CLOCK_CONTROL_NRF_CALIBRATION_MAX_SKIP=0 

It's definitely a bug with our Softdevice controller because if I use CONFIG_BT_LL_SW_SPLIT=y (Zephyr controller) then it worked. 
But if I use 

CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC=y

CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC_CALIBRATION=y

CONFIG_BOOTLOADER_MCUBOOT=y
It also worked fine. So it doesn't seem (at least on my setup) there is an issue with MCUBoot when using 32kHz RC without using CONFIG_CLOCK_CONTROL_NRF_CALIBRATION_MAX_SKIP=0. 

Attached is the project and the hex file that you can try.

timeslot.zip

[Maarten de Kruijf]

Also I won't be available next week so Martijn (my college) will continue responding on this thread next week.

[Hung Bui]

Hi Maarten, 

Ok so the MCUBoot is fixed to CONFIG_CLOCK_CONTROL_NRF_K32SRC_SYNTH. This is the new information that hasn't included in the original question or the howtoreproduce.md. 

It would be a challenge to have different RC source between the MCUBoot and the application. I'm checking internally to see if there is any solution. 

[Martijn Stommels]

Hi Hung,

Thank you for checking.

I agree switching source isn't ideal, and kind of an edge case. However, since we already deployed thousands of units with the bootloader set to the SYNTH clock, we can't really change this.

This did work perfectly in NCS 1.5, so the hardware does seem to be capable of switching back.

Regards,

Martijn Stommels

[Maarten de Kruijf]

Hi Hung,

I agree we didn't include the info, we tried to reduce the problem to a simpeler example. 

Do you have any information at this point that could help us? 

Regards,

Maarten

[Hung Bui]

Hi Martijn and Maarten, 

I have reproduced the issue here and have reported it to our team. They are looking into the issue, I will keep you updated. 

[Hung Bui]

Just a quick update: 
We found that the issue was due to MCUboot doesn't clean up the clock before jumping to the application. 
The easy fix is https://github.com/nrfconnect/sdk-mcuboot/commit/7efbda6a58ac619506c0db2fc8ff16bec9620836

But in your case it may not be possible as you can't update the MCUboot. So we will need to find a way to stop the LFCLK and HFCLK before starting the clock in the application. Will let you know when we have a solution. 

[Hung Bui]

Hi guys,

Good news, we have a workaround. 
By adding the following code in main.c the clock stopping function will be executed before the kernel and can stop the clock from MCUBoot after that the application can start and use the clock source of its own. 

static int stop_clocks()
{ 	nrf_clock_int_disable(NRF_CLOCK, 0xFFFFFFFF);
    nrf_clock_task_trigger(NRF_CLOCK, NRF_CLOCK_TASK_LFCLKSTOP);
    nrf_clock_task_trigger(NRF_CLOCK, NRF_CLOCK_TASK_HFCLKSTOP);
	while (nrf_clock_lf_is_running(NRF_CLOCK));
	return 0;
}
SYS_INIT(stop_clocks, PRE_KERNEL_1, 0);

[Maarten de Kruijf]

Hi Hung,

Thanks for all the help so far, I have verified that the work around works for the devkit example. We will go and implement the work around in our application. I needed to adapt the example code to take a const device struct* as parameter: 

static int stop_clocks(const struct device* dev)
{
    nrf_clock_int_disable(NRF_CLOCK, 0xFFFFFFFF);
    nrf_clock_task_trigger(NRF_CLOCK, NRF_CLOCK_TASK_LFCLKSTOP);
    nrf_clock_task_trigger(NRF_CLOCK, NRF_CLOCK_TASK_HFCLKSTOP);
    while (nrf_clock_lf_is_running(NRF_CLOCK))
        ;
    return 0;
}
SYS_INIT(stop_clocks, PRE_KERNEL_1, 0);

If we have any followup questions we will post them.

Regards,
Maarten

[fafeyto]

These solutions didn't work for me. Do you have any other ideas?