Bootloader WDT not Firing

RE: Bootloader WDT not Firing

Chris Ergo — Tue, 25 Oct 2022 01:09:42 GMT

I ran into possibly the same issue in SDK 17.1.0. The problem for me seems to be that nrf_clock_lf_is_running() returns true if the WDT is running, but the LF clock somehow might not be running well enough for the RTC to run. What worked for me is to start the LF clock even if it says it is running (see lines I commented in nrf_bootloader_dfu_timers.c):

/**@brief Function for initializing the timer if it is not already initialized.
 */
static void timer_init(void)
{
    static bool m_timer_initialized;

    if (!m_timer_initialized)
    {
//        if (!nrf_clock_lf_is_running())    * With the WDT running, the LF is running, but somehow not consistently well enough for the RTC, so really start it regardless.
//        {
            nrf_clock_task_trigger(NRF_CLOCK_TASK_LFCLKSTART);
//        }
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_TICK);
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_COMPARE_0);
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_COMPARE_1);
        NRFX_IRQ_PRIORITY_SET(RTC_IRQn, 5);
        NRFX_IRQ_ENABLE(RTC_IRQn);
        nrf_rtc_prescaler_set(RTC_STRUCT, RTC_PRESCALER);
        nrf_rtc_task_trigger(RTC_STRUCT, NRF_RTC_TASK_CLEAR);
        nrf_rtc_task_trigger(RTC_STRUCT, NRF_RTC_TASK_START);
        nrf_rtc_int_enable(RTC_STRUCT, RTC_INTENSET_OVRFLW_Msk);

        m_timer_initialized = true;
    }
}

RE: Bootloader WDT not Firing

Sebastian — Mon, 11 May 2020 12:46:15 GMT

Ok Einar.

Thank you for your help.

RE: Bootloader WDT not Firing

Einar Thorsrud — Mon, 11 May 2020 12:44:41 GMT

Hi,

I am glad to hear you found an effective workaround. I am a bit surprised that changing the configuration of RTC0 has affected the issue for RTC2, but then again I do not understand the root cause of this issue.

RE: Bootloader WDT not Firing

Sebastian — Mon, 11 May 2020 08:19:37 GMT

Good morning Einar,

Well, I think you got it. Although it is not clear to me why this works. I modified the timer_init code to the following:

static void timer_init(void)
{
    static bool m_timer_initialized;
        
    if (!m_timer_initialized)
    {
        /*if (!nrf_clock_lf_is_running())
        {           
            lfclk_config();               //VG 20200211 for init of LFCLK with external Oscillator
        }*/

        NRF_CLOCK->EVENTS_LFCLKSTARTED  = 0;              //20200511 Modified as per Nordic's suggestions. Solves the RTC2 Starting issue
        NRF_CLOCK->TASKS_LFCLKSTART     = 1;
        while (NRF_CLOCK->EVENTS_LFCLKSTARTED == 0) {}
        NRF_RTC0->TASKS_STOP = 0;
                               
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_TICK);
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_COMPARE_0);
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_COMPARE_1);
        NRFX_IRQ_PRIORITY_SET(RTC_IRQn, 5);
        NRFX_IRQ_ENABLE(RTC_IRQn);
        nrf_rtc_prescaler_set(RTC_STRUCT, RTC_PRESCALER);
        nrf_rtc_task_trigger(RTC_STRUCT, NRF_RTC_TASK_CLEAR);
        nrf_rtc_task_trigger(RTC_STRUCT, NRF_RTC_TASK_START);
        nrf_rtc_int_enable(RTC_STRUCT, RTC_INTENSET_OVRFLW_Msk);
                
        m_timer_initialized = true;
    }
}

Implementing Erratum 20.

This WORKS. RTC2 counter initializes normally and as a result the WDT which was operating in the application now is fed normally. The LFCLK source is the RC but that doesn't matter. I checked and in the application, I return LFCLK to XTAL and it works just fine.

Puzzled by this:

The errata requests TASK STOP on RTC0 and this is what I added to the routine. How can a STOP TASK on RTC0 affect RTC2?

Maybe this is because in my application all RTC were being used? RTC0 from the softdevice, RTC1 for application timers and RTC2 for a real time calendar?

Anyway I am happy this is sorted and I can verify your answer above as a solution.

RE: Bootloader WDT not Firing

Einar Thorsrud — Fri, 08 May 2020 15:16:14 GMT

Hi,

I see. So the LF clock source has no impact on this issue. That is good news and expected really since it should not have any impact.

I am puzzled about the fact that your tests show that using a breakpoint and continuing from there "fixes" the issue, but using a delay at the same location does not. If there is a timing issue, it would be the same. But in any case, breaching and continuing from breakpoint impacts the CPU, but not the RTC in any way, other than that potential interrupts are not cleared as soon as they would have been if the CPU were running. Could this be a clue?

Can you stop the RTC before you start it, by implementing the suggested workaround for erratum 20?

Erratum 101 was my bad. It is not relevant for the nRF52840 (but for the nRF52832).

RE: Bootloader WDT not Firing

Sebastian — Fri, 08 May 2020 06:52:21 GMT

Hello Einar,

I tried what you suggested in the following:

/**@brief Function for initializing the timer if it is not already initialized.
 */
static void timer_init(void)
{
    static bool m_timer_initialized;
        
    if (!m_timer_initialized)
    {
        /*if (!nrf_clock_lf_is_running())
        {           
            lfclk_config();               //VG 20200211 for init of LFCLK with external Oscillator
        }*/
                               
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_TICK);
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_COMPARE_0);
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_COMPARE_1);
        NRFX_IRQ_PRIORITY_SET(RTC_IRQn, 5);
        NRFX_IRQ_ENABLE(RTC_IRQn);
        nrf_rtc_prescaler_set(RTC_STRUCT, RTC_PRESCALER);
        nrf_rtc_task_trigger(RTC_STRUCT, NRF_RTC_TASK_CLEAR);
        nrf_rtc_task_trigger(RTC_STRUCT, NRF_RTC_TASK_START);
        nrf_rtc_int_enable(RTC_STRUCT, RTC_INTENSET_OVRFLW_Msk);
                
        m_timer_initialized = true;
    }
}

Indeed after reset, the LFCLKL shoul be running using the internal RC oscillator. So I tried not to initialize the LFCLK but let it run in RC mode.

The result is the same. The LFCLK should be running because I get a WDT reset after 10 seconds. The RTC2 IS NOT running no matter where I place the breakpoint. I can see the interrupts and CC registers initialized but the counter is not measuring hence the WDT causes a reset.

Any ideas are welcome. What is Errata 101? I tried looking in nRF52840 documentation but could not find 101.

Thanks.

RE: Bootloader WDT not Firing

Einar Thorsrud — Thu, 07 May 2020 14:48:58 GMT

Hi,

I need to try to reproduce this on my side and discuss it with my colleagues. At this point, I fail to understand what could be happening here.

Is simply not using the external LF clock when in the bootloader a viable workaround?

In the meantime, I want to mention a point which may not be directly related to this issue, but maybe worth noting. Since the WDT forces the LFCLK on, and the WDT is not reset by a soft reset but the clock configuration is, the LFRC clock is running after reset. This means that when you configure it to run on the external reference, you are violating this statement in the clock chapter in the product specification: "It is not allowed to write to register LFCLKSRC when the LFCLK is running.". Violating this should normally not be a big issue (and it is in fact quite common, though you will trigger erratum 101).

RE: Bootloader WDT not Firing

Sebastian — Wed, 06 May 2020 14:37:30 GMT

Hello Einar,

I know. It is a puzzle and very frustrating. It shouldn't be happening. I know that the LFCLK is running normally in both cases because in the correct situation (where RTC2 is running), the bootloader will reset the MCU after 45 seconds of inacitivity (this is the time I have set) and in the incorrect case (where RTC2 is not running) the MCU resets within 10 seconds (more or less) which is the WDT interval.

Let me answer your questions

1) The interrupts of the RTC are enabled. What is happening is that the RTC2 is not running in the problematic case. I am deducing this by looking at the RTC2 registers. Look below, the case where the RTC2 is running:

Above, I stopped the execution in the wdt_feed to show the result.

Look now below the problematic case:

In the 2 pictures above, the breakpoint was set after the if(!m_timer_initialized) at the end of the timer_init routine. You can see that only the overflow interrupt is set and the counter is not running.

To further show the effect I just paused the debug inside the sleep loop (image 2) where again you can see all interrupts set on RTC2, the CC registers have been filled but the counter is 0. The RTC is not running.

2) I tried delays in various points in the code. I tried small and large delays. None of them works. The effect is always the same. RTC2 is not running

3) To do what you asked, I disabled the application WDT and modified nrf_bootloader_wdt_init routine as follows:

void nrf_bootloader_wdt_init(void)
{
    static bool initialized = false;

    if (initialized)
    {
        return;
    }

    //if (nrf_wdt_started())
    //{
        uint32_t wdt_ticks = 0x50000;//nrf_wdt_reload_value_get();

        NRF_LOG_INFO("WDT enabled CRV:%d ticks", wdt_ticks);

        //wdt_ticks must be reduced to feed the watchdog before the timeout.
        uint32_t reduced_timeout_ticks = MAX((int32_t)wdt_ticks - MAX_FLASH_OP_TIME_TICKS,
                                             NRF_BOOTLOADER_MIN_TIMEOUT_TICKS);

        /* initial watchdog feed */
        wdt_feed();

        NRF_LOG_INFO("Starting a timer (%d ticks) for feeding watchdog.", reduced_timeout_ticks);
        nrf_bootloader_wdt_feed_timer_start(reduced_timeout_ticks, wdt_feed_timer_handler);

        NVIC_EnableIRQ(WDT_IRQn);
    //}
    //else
    //{
    //    NRF_LOG_INFO("WDT is not enabled");
    //}

    initialized = true;
}

This enables the WDT inside the bootloader.

In this case, the RTC2 actually WORKS. I can see the RTC2 counter running no matter if a place a breakpoint or not at any location.

So it seems that something is fishy with the fact that an enabled WDT changes configuration to LFCLK inside the bootloader. It seems to be ok when LFCLK and WDT are both enabled inside the bootloader.

Any ideas?

Thanks.

RE: Bootloader WDT not Firing

Einar Thorsrud — Wed, 06 May 2020 13:46:34 GMT

Hi,

This is still a puzzle to me. The RTC obviously runs of the 32.768 kHz clock, which uses an external oscillator in your case. That should be OK, and no matter how that behaves, the RTC configuration should be unaffected. You could think that the external oscillator was not properly up and running when you configured the RTC, but the order there does not matter.

Some more questions:

How have you verified hat the RTC interrupts are not enabled? Can you show a screenshot or register dump of the RTC registers showing the states when it should be up and running? Preferably both when it works (with your breakpoint present) and without it.
What happens if you replace the breakpoint with a short delay, does that have any effect?
What happens if you don't start the WDT before entering the bootloader (and keep it not running), but modify line 89 in nrf_bootloader_wdt.c to keep the behavior of feeding it, etc as is. Does the regular feeding work as expected in this case? (I wonder if the fact that the WDT forces the LFCLK on could cause problems when you reconfigure it to use an external oscillator, though I do not have any particular reason for thinking that it should).

RE: Bootloader WDT not Firing

Sebastian — Tue, 05 May 2020 15:04:09 GMT

Hello Einar,

Thanks for comming back so quickly. It is odd and what is frustrating is why a breakpoint allows the RTC2 to be triggered.

Indeed I did some changes to the bootloader but I cannot see how these may affect it. Let me explain:

In the file nrf_bootloader_dfu_timers I needed to change the way the LFCLK is initialized because I am using an external oscillator and not a crystal. The routine timer_init has been modified to:

/**@brief Function for initializing the timer if it is not already initialized.
 */
static void timer_init(void)
{
    static bool m_timer_initialized;
        
    if (!m_timer_initialized)
    {
        if (!nrf_clock_lf_is_running())
        {           
            lfclk_config();            
        }
                               
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_TICK);
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_COMPARE_0);
        nrf_rtc_event_clear(RTC_STRUCT, NRF_RTC_EVENT_COMPARE_1);
        NRFX_IRQ_PRIORITY_SET(RTC_IRQn, 5);
        NRFX_IRQ_ENABLE(RTC_IRQn);
        nrf_rtc_prescaler_set(RTC_STRUCT, RTC_PRESCALER);
        nrf_rtc_task_trigger(RTC_STRUCT, NRF_RTC_TASK_CLEAR);
        nrf_rtc_task_trigger(RTC_STRUCT, NRF_RTC_TASK_START);
        nrf_rtc_int_enable(RTC_STRUCT, RTC_INTENSET_OVRFLW_Msk);
                
        m_timer_initialized = true;
    }
}

where the routine lfclk_config is the following:

static void lfclk_config(void)
{      
    bool lfclk_running=	nrfx_clock_lfclk_is_running();
     	                  
         NRF_CLOCK->LFCLKSRC = LFCLKSRC_XTERNAL_OSCILLATOR;  
        
         if(lfclk_running == false){
           	nrfx_clock_lfclk_start();
         }
      
      while(nrfx_clock_lfclk_is_running() == false){
      }
}

Another minor modification is in the nrf_bootloader.c file where I introduced a port initialization in the nrf_dfu_init_user routine as shown below:

/**@brief Weak implementation of nrf_dfu_init
 *
 * @note    This function must be overridden in application if
 *          user-specific initialization is needed.
 */
__WEAK uint32_t nrf_dfu_init_user(void)
{
    NRF_LOG_DEBUG("in weak nrf_dfu_init_user");

    ports_init();                                     

    return NRF_SUCCESS;
}

ports_init only intializes input and output ports for our custom board

Also in the file nrf_dfu_req_handler.c I added a bit of code for toggling an LED inside the dfu_req_handler as shown below:

static void nrf_dfu_req_handler_req(void * p_evt, uint16_t event_length)
{
    nrf_dfu_request_t * p_req = (nrf_dfu_request_t *)(p_evt);
    nrf_dfu_req_handler_req_process(p_req);
    nrf_gpio_pin_toggle(LED_RED);                                                 
}

This toggles an LED.

Finally in main.c I only added toggling of LEDs where needed as shown below:

static void dfu_observer(nrf_dfu_evt_type_t evt_type)
{
    switch (evt_type)
    {
        case NRF_DFU_EVT_DFU_FAILED:
        case NRF_DFU_EVT_DFU_ABORTED:
        case NRF_DFU_EVT_DFU_INITIALIZED:

            //bsp_board_init(BSP_INIT_LEDS);
            //bsp_board_led_on(BSP_BOARD_LED_0);
            //bsp_board_led_on(BSP_BOARD_LED_1);
            //bsp_board_led_off(BSP_BOARD_LED_2);
            nrf_gpio_pin_clear(LED_GRN);
            nrf_gpio_pin_clear(LED_RED);
            break;
        case NRF_DFU_EVT_TRANSPORT_ACTIVATED:
            //bsp_board_led_off(BSP_BOARD_LED_1);
            //bsp_board_led_on(BSP_BOARD_LED_2);            
            nrf_gpio_pin_set(LED_RED);            
            break;
        case NRF_DFU_EVT_DFU_STARTED:            
            break;

        default:
            break;
    }
}

I cannot see how, these modifications could affect the triggering of RTC2.

Meanwhile I tried using RTC1 or RTC0 but the result is the same. I also tried to introduce nrfx driver for the RTC and trigger it with the nrfx driver rather than the RTC HAL and this did not have any effect.

Let me know of your opinion.

RE: Bootloader WDT not Firing

Einar Thorsrud — Tue, 05 May 2020 14:38:59 GMT

Hi,

This is odd. There should not be any other code in the bootloader that touches RTC2. Can you share any changes you have done to the bootloader?