nRF52840 sometimes won't wake up.

[david4255334]

Hi,

I'm using nRF52840 with nRF Connect SDK v2.2.0.

I need my device to go to sleep and wake up after a button press.

I followed "system_off" example.

In my main function I have this snippet:

static int disable_ds_1(const struct device *dev) {
    ARG_UNUSED(dev);

    pm_policy_state_lock_get(PM_STATE_SOFT_OFF, PM_ALL_SUBSTATES);
    return 0;
}

SYS_INIT(disable_ds_1, PRE_KERNEL_2, 0);

Then I initialize GPIO and P0.25 (button line): I set it as input and enable internal pull-up.

Then I configure port event to wake up when the button is pressed (low state):

    /* Configure to generate PORT event (wakeup) on push button press */
    nrf_gpio_cfg_sense_set(NRF_DT_GPIOS_TO_PSEL(DT_ALIAS(push_button), gpios), NRF_GPIO_PIN_SENSE_LOW);

Then, the application counts several minutes, and after that, it automatically shuts down using this code. This code is also executed after long button press and then release, if the user wants to shut down the device without waiting for timeout:

    pm_state_force(0u, &(struct pm_state_info){PM_STATE_SOFT_OFF, 0, 0, 0});

    k_sleep(K_SECONDS(2U));

This code works fine most of the time, but sometimes it won't wake up after pressing the button. The only way to recover from this state is to reset the MCU or power cycle the board.

I was looking for similar issues in DevZone, and I found this topic although it's related to a different Nordic MCU:

nrf52833 failed to wake up from SOFT_OFF - Nordic Q&A - Nordic DevZone - Nordic DevZone (nordicsemi.com)

I don't use Bluetooth mesh, but I do use "regular" BLE functionalities.

When the system starts I enable BLE like that:

// Registering callback here...
err = bt_enable(NULL);
err = bt_le_adv_start(BT_LE_ADV_CONN, ad_, ARRAY_SIZE(ad_), sd_, ARRAY_SIZE(sd_));
smp_bt_register();

Following the ideas from the other ticket mentioned above, I modified the code so that the BLE is disabled before putting the system to sleep:

const int ble_disable_err = bt_disable();

if (ble_disable_err) {
    NVIC_SystemReset();
}

pm_state_force(0u, &(struct pm_state_info){PM_STATE_SOFT_OFF, 0, 0, 0});
k_sleep(K_SECONDS(2U));
NVIC_SystemReset();

I also reset the system in case of unlikely errors when disabling BLE or going to SOFT_OFF - this is better than having the board not waking up.

This worked for me. To test it, I hooked up Arduino to the button line to simulate long press-then-release to put it to sleep and then short press to wake up.

Without disabling BLE, the board got stuck after 30-50 cycles several times. When I added the fix, I kept the test running overnight (around 3000 cycles) and the issue did not occur anymore. This made me think that the problem is fixed for good.

Can anyone explain what is the root cause and if the changes I made make sense? Or my fix only covers a real root cause that Iam not aware of?

[runsiv]

Hi 

I can't say that understand at the moment why disabling BT would solve the problems. 

Would it be possible for you to upload your project? I can set the case to private if you like. 

Do you have any logs from when it hangs? It would be helpful while trying to figure out the root cause

Is your debugger attached when you tried to set it into the power state? The debugger would prevent it from going into power down mode. 

Do you have any current measurements from the tests?

Regards

Runar

[david4255334]

Hi Runar,

thank you for you quick reply.

Unfortunately, I cannot upload the project.

When it comes to logs, when it's stuck, I see nothing int the console. No system logs, and no logs from my application. I've never tweaked the OS logs to a custom setting, but I guess that system error logs would be printed by default.

When performing the tests, my debugger is always disconnected. The board is only powered from USB.

When it comes to current, there is no way to measure the current consumed by nRF only. When we put the board to sleep, we power off most of the power rails, byt there are still some components that are powered on. And the consumption is still a couple of mA even in the sleep mode.

However, I connected the USB power meter (Fnirsi FNB58) between the power source and USB connector of the board to compare the consumption when it goes to sleep correctly and when it hangs.

When it goes to sleep correctly, then I can see consumption around ~7.7 - 7.8 mA.

When it hangs, the consumption is slightly higher: ~8.3 - 9.5 mA.

Another thing is that recently I added the watchdog functionality.

I noticed, that even when the BLE is not disabled before going to sleep, but watchdog is enabled, the overnight test passes and the board does not hang. This looks to me like calling pm_state_force() causes the watchdog reset.

Every time it hangs, I'm sure it entered the function where I call pm_state_force(), because just before putting the board to sleep, I disable other power supplied and this turns off the LED indicating PSU state. This disableOtherPsu() call is literally above pm_state_force(). I did not include this detail in the first post.

disableOtherPsu(); // This calls gpio_pin_set() Zephyr API, and  results in turnign off PSUs and LED
pm_state_force(0u, &(struct pm_state_info){PM_STATE_SOFT_OFF, 0, 0, 0});
k_sleep(K_SECONDS(2U));

What pm_state_force() does, is making the Power Management thread setting the bit in SYSTEMOFF (0x500) register? Do you think that setting that register manually instead of callibg pm_state_force() would help to troubleshoot it? Or is it a bad idea and we should control the power state by accessing the registers directly?

[runsiv]

Hi David

its unfortunate that you can't upload the project, it would make it eaiser for me to help you find the root cause. If I make this case private it would be confidential between you, Nordic Semiconductor and your chosen distributor(non in this case). Do you think you are able to create a small dummy version of the code? 

When you say powered over USB, which usb port are you using? If you are using J2 then the debugger is powered on

Regarding power measurement and how to prepear the DK we have a section on it on our infocenter

The sleep current sounds very high for a sleep in my opinon, but its hard to say for sure when I'm nore sure about what's beeing measured. 

What log options have you enabled?

As for the rest I need to investigate a bit and get back to you.

Thanks for the update

[david4255334]

Hi Runar, 

I'm sorry, I did not make it clear - the code is running on our custom PCB that has nRF52840 on it. We power it from USB, and then the USB power goes through the power supply management chip to the nRF.

I'll find out if I can prepare some small code that can reproduce it. But I would need to see if this can be reproduced on the DK.

I added code that immediately reads RESETREAS register after startup and saves it in a variable for printing it later. I do not clear RESETREAS register.

#include <helpers/nrfx_reset_reason.h>
static uint32_t reason___ = 0;
reason___ = nrfx_reset_reason_get();

// Shutdown code
const int ble_disable_err = bt_disable();
if (ble_disable_err) {
    NVIC_SystemReset();
}
k_sleep(K_MSEC(1000));
PowerSupply_vSys5vDisable();
pm_state_force(0u, &(struct pm_state_info){PM_STATE_SOFT_OFF, 0, 0, 0});
k_sleep(K_SECONDS(2U));
NVIC_SystemReset();

With the code below, after wakeup, RESETREAS is equal to 0x00010000, indicating "Reset due to wake up from System OFF mode when wakeup is triggered from DETECT signal from GPIO". As expected.

Then, I run exactly the same code with only one change - remove first 4 lines that disable BLE. 

// Shutdown code
//const int ble_disable_err = bt_disable();
//if (ble_disable_err) {
//    NVIC_SystemReset();
//}
k_sleep(K_MSEC(1000));
PowerSupply_vSys5vDisable();
pm_state_force(0u, &(struct pm_state_info){PM_STATE_SOFT_OFF, 0, 0, 0});
k_sleep(K_SECONDS(2U));
NVIC_SystemReset();

After many sleep-wakeup cycles, I expect the issue to occur again. I checked the logs with RESETREAS and there was one entry that instead 0x00010000 is showing 0x00010002, that indicates the reset from watchdog, which confirms what I wrote in the previous post.

This is why in the old version of the code, without watchdog configured, the board was bricked. It somehow got stuck and without watchdog nothing could be done.

[runsiv]

Thanks for the good explantion, I have forwarded the information to the dev teams for further input. 

There is one other thing we can check out. From the PS of the 52840 it states the following: 

"Before entering System OFF mode, all on-going EasyDMA transactions need to have completed. See peripheral specific chapters for more information about how to acquire the status of EasyDMA transactions."

The BLE stack uses DMA, so it might be that the stars align and it's powered down during advertisment. I have talked with a few collegues regarding what will happen  if this occure, and there was no clear answare. 

infocenter.nordicsemi.com/index.jsp

Regards

Runar

[david4255334]

Hi Runar,

I finally found some time to prepare a simple project running on nRF52840dk.

I was able to reproduce the wake-up issue in the setup running on the DK. Same situation here - disabling BLE before going to sleep fixes the problem.

Please find below the source code. There are only 3 files, but if you need a zip file, let me know then I can attach it as well.

If there are any new findings or better (proper?) ways to overcome this wake-up issue, then please let me know.

nrf52_wakeup_issue/src/main.c:

// nrf52_wakeup_issue/src/main.c
/* How to reproduce:
Power the DK via J3 micro USB.
After DK goes to sleep for the first time, start Arduino code and immediatelly connect THE_PIN to the Button 4 of DK.
Keep running for a while until DK won't wake up.

I did this test 4 times:
1. won't wake up after 12 min
2. won't wake up after 16 min
3. won't wake up after 3 min
4. won't wake up after 35 min

Arudino Mega board
Code simulating pressing the button after DK goes to sleep

#define THE_PIN 53

void setup() {
    pinMode(THE_PIN, INPUT);
    Serial.begin(9600);
    Serial.println("Start");
    delay(10000);
}
int counter = 0;
void loop() {
    digitalWrite(THE_PIN, LOW);
    pinMode(THE_PIN, OUTPUT);
    Serial.println("Low for 100 ms");
    delay(100);

    // Wait until the board wakes up and goes to sleep again
    pinMode(THE_PIN, INPUT);
    Serial.println("Float for 10 seconds");
    delay(10000);
}

 */
#include <stdint.h>
#include <zephyr/kernel.h>
#include <zephyr/drivers/gpio.h>
#include <zephyr/devicetree.h>
#include <hal/nrf_gpio.h>
#include <zephyr/pm/pm.h>
#include <zephyr/bluetooth/bluetooth.h>
#include <zephyr/settings/settings.h>

static const struct bt_data ad_[] = {
    BT_DATA_BYTES(BT_DATA_FLAGS, (BT_LE_AD_GENERAL | BT_LE_AD_NO_BREDR)),
    BT_DATA(BT_DATA_NAME_COMPLETE, CONFIG_BT_DEVICE_NAME, sizeof(CONFIG_BT_DEVICE_NAME) - 1),
};

static const struct bt_data sd_[] = {
    BT_DATA(BT_DATA_NAME_COMPLETE, CONFIG_BT_DEVICE_NAME, sizeof(CONFIG_BT_DEVICE_NAME) - 1),
};

void main(void) {
    gpio_pin_configure(DEVICE_DT_GET(DT_NODELABEL(gpio0)), 15, GPIO_OUTPUT_HIGH); // LED3 on DK
    gpio_pin_configure(DEVICE_DT_GET(DT_NODELABEL(gpio0)), 25, GPIO_INPUT | GPIO_PULL_UP); // BUTTON4 on DK
    nrf_gpio_cfg_sense_set(25, NRF_GPIO_PIN_SENSE_LOW);
    gpio_pin_set(DEVICE_DT_GET(DT_NODELABEL(gpio0)), 15, 0);

    if (bt_enable(NULL)) {
        printk("bt_enable() failed\n");
        goto err;
    }

    if (bt_le_adv_start(BT_LE_ADV_CONN, ad_, ARRAY_SIZE(ad_), sd_, ARRAY_SIZE(sd_))) {
        printk("bt_le_adv_start() failed\n");
        goto err;
    }

    for (int i = 5; i >= 0; --i) {
        k_sleep(K_MSEC(1000));
        printk("Shutting down the system in %d seconds...\n", i);
    }

    // When bt_disable() is called, there is no wakeup issue.
    // bt_disable();
    
    gpio_pin_set(DEVICE_DT_GET(DT_NODELABEL(gpio0)), 15, 1);
    pm_state_force(0u, &(struct pm_state_info){PM_STATE_SOFT_OFF, 0, 0, 0});

err:
    while (1) {
        k_sleep(K_MSEC(2000));
        printk("Error\n");
    }
}

nrf52_wakeup_issue/CMakeLists.txt:

# nrf52_wakeup_issue/CMakeLists.txt
cmake_minimum_required(VERSION 3.20.0)
# list(APPEND BOARD_ROOT ${CMAKE_CURRENT_SOURCE_DIR})
set_property(GLOBAL PROPERTY CSTD c11)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(wakeup_test)
file(GLOB APPLICATION_SOURCES src/main.c)
target_sources(app PRIVATE ${APPLICATION_SOURCES})
target_include_directories(app PRIVATE src)

nrf52_wakeup_issue/prj.conf:

# nrf52_wakeup_issue/prj.conf
CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC=y
CONFIG_BT=y
CONFIG_NEWLIB_LIBC=y
CONFIG_BT_PERIPHERAL=y
CONFIG_BT_SMP=y
CONFIG_PM_DEVICE=y
CONFIG_NEWLIB_LIBC_MIN_REQUIRED_HEAP_SIZE=2048