Getting low power consumption, with external sensors

[dan_opito]

Hi,

Our application is based on the asset tracker, and has a similar overall process of waking up, taking sensor readings and going back to sleep. This application is used in a number of our products, some of which need to run on small batteries. Over the last week I've been working on getting power consumption down - and I'm currently at 20 uA including the other devices on our board (like high-side switches and external flash memory). 20 uA is awesome!

However, I now need to introduce back the functionality - like measuring external sensors using I2C. I see some sensor drivers, for example the INA219 that I'm using to start with, supports Zephyr's CONFIG_PM_DEVICE option and handles power down. When asked, the INA219 driver will set the device into a power down mode - and I have tested this and found that the INA219 itself uses very little current when in this mode.

How do we go about making this work with the nRF9160? For that option to work, we need the Zephyr Power Manager to be running with a power policy - and I note the CAF Power Manager actually disables that by requiring CONFIG_PM_POLICY_CUSTOM and returning NULL back to the Zephyr Power Manager. Instead, I'm using the residency-based policy (see here: System Power Management — Zephyr Project Documentation) and I've added a couple of power states to my device tree:

/ {
    power-states {
        active: active {
            compatible = "zephyr,power-state";
            power-state-name = "active";
            min-residency-us = < 1000 >;
        };
        idle: idle {
            compatible = "zephyr,power-state";
            power-state-name = "suspend-to-idle";
            min-residency-us = <2000000>;
            exit-latency-us = < 1000 >;
        };
    };

    cpus {
        cpu@0 {
            cpu-power-states = <&active &idle>;
        };
    };
};

When I run the debugger, I can see Zephyr actually runs the function with an initial tick value of 7FFFFFFF (-1, or K_TICKS_FOREVER). Strangely, that doesn't seem to select my first policy (active) and decides to shut down the INA219 on boot - potentially before the initialisation even runs.

Before I spend too much time going down this route, I wanted to ask and see what the Nordic-recommended approach was. How should I be preparing by devices for power down? Is there an event I can capture (like the power down event from CAF), and then somehow ask the external sensors to power down using their PM integration? Does Nordic have any example code where they use a high-current sensor (600 uA is considered high in my application!) and shut it down before sleeping?

I notice there's no nRF91-series example code (in my NCS 2.2.0) showing how those CPU power states are used in the device tree - but I also don't see any using that CAF Power Manager (machine_learning targets the nRF53, and nrf_desktop targets the nRF52).

Searching for power savings, I find a number of references to this article: Measuring PSM idle current on the nRF91 DK - Blog Archives - Blog - Nordic DevZone (nordicsemi.com). This article was really useful in achieving my 20 uA figure (I was able to get even less by removing all the other components on my PCB) - but that relies on disabling a lot of things at compile-time and doesn't explain the power management of external devices.

I'm planning on continuing to investigate these cpu-power-states in Zephyr but I would appreciate any advice you can provide!

Kind regards,

Dan

[ovrebekk]

Hi Dan

Essentially the CAF Power Manager module was introduced at a time when the power management functionality in Zephyr was quite limited, and as you say there is a limited number of applications that use it. 

In general it is better to use standardized Zephyr API's over more specialized API's where possible, and my recommendation would be to use the PM_DEVICE_RUNTIME feature if you want to put I2C sensors into and out of sleep from the application. 

Using the Zephyr power state mechanism doesn't really apply to Nordic devices because of the way power management is handled on a low level. Essentially all you have to do to enable low power in most nRF devices is to enter system on idle mode using WFI or WFE, and make sure not to keep peripherals enabled that don't need to be. 

This was a bit of a short answer to a long question. Please let me know if anything is still unclear and I will do my best to help

Best regards
Torbjørn

[dan_opito]

An answer is an answer, Torbjørn ! You've given me some confidence that this is a beaten track - using the Zephyr power management at least.

What I have right now is a custom policy that will return the PM_STATE_RUNTIME_IDLE state when we want to go to sleep (which is arguable still residency based). The issue I have, even with CONFIG_PM_DEVICE_RUNTIME enabled is the system crashes somewhere (having trouble actually debugging this, but it seems related to a K_FOREVER wait getting killed perhaps by the WDT?).

Does Nordic have any examples using this runtime, and using some mechanism to signal to modules that their drivers need to be powered down?

In my pm_notfifier callbacks, I get this same crash with i2c-based devices (like INA219) but also if I try a simple uart_poll_tx() when we enter and exit the PM_STATE_RUNTIME_IDLE. Seems like this could be an order of execution issue - perhaps the peripheral is already asleep and I'm waiting too long for a semaphore?

I notice in later commits to the Zephyr branch, Nordic seems to have deprecated CONFIG_PM? See here for what I'm referencing: [nrf fromtree] soc: arm: nordic_nrf: drop PM hooks · nrfconnect/sdk-zephyr@15f47a6 (github.com).

I've got a couple of other changes I need to get out early this week, but I'll be following along with any replies and hope to get back to this later this week or early next.

[ovrebekk]

Hi Dan

It is true that CONFIG_PM is being deprecated, since it doesn't really have any use for Nordic devices. All Nordic devices practically only have one sleep mode (not including system OFF mode, which can only be used in very special cases/applications), and this sleep mode will be entered automatically by the idle thread whenever there is nothing to do. 

In other words you only need to enable CONFIG_PM_DEVICE_RUNTIME (and by extension CONFIG_PM_DEVICE) in order to ensure that drivers disable the underlying hardware peripherals when not in use, to ensure low sleep power. 

With PM_DEVICE_RUNTIME enabled the I2C driver should make sure to automatically disable the TWIM module in between transactions, and for the higher level sensor driver there is nothing else needed to be done in order to ensure low sleep current. 

Could you elaborate why you feel you need to use a power policy? 
Using a power policy doesn't really add any value when developing for Nordic devices, for the reasons mentioned above. 

Best regards
Torbjørn

[dan_opito]

I'm pretty confident the CONFIG_PM_DEVICE_RUNTIME works well in the nrfx_twim driver, disabling the peripheral as it should.

To avoid the X-Y problem, here's what I want to achieve. My external sensors, like LEDs, need to be powered off when not being used. The Zephyr features currently handle the CPU itself - and my application needs to know when it can power sensors off and on. I've picked the CONFIG_PM as a possible solution.

Using the INA219 again as my example, it has some built-in averaging that gives better readings so I want to enable that and let the sensor run for a bit. Once that's all done I then take the reading and can put the sensor back to sleep. Using the CONFIG_PM, the module gets a notification (based on the power policy) about when the system enters and exits various modes. When the CPU is going to sleep for a while, the sensor can know to just power down completely. The module using that sensor doesn't even have to know about this, as it's all handled by Zephyr and the driver itself.

Another option is to wait for the APP_EVT_DATA_GET event, power up the sensor and schedule a timer to fire when the INA219 readings are ready. Once I've got that, I pass the reading to the data module and put the INA219 back to sleep. This is not much extra work and also ensures the sensor is only powered on when in use (not every time the CPU is in use). The only API available to the application for this is pm_device_action_run() and needs CONFIG_PM and CONFIG_PM_DEVICE to be accessible. I also requires we either have a NULL power policy or we only use PM_STATE_RUNTIME_IDLE to avoid automated control from the PM suspend loop.

Do you have a recommendation on how I can power my external I2C sensors up and down?

[ovrebekk]

Hi Dan

Thanks for providing more context, it is indeed easy to end up in the XY trap

The problem of power management with sensor drivers in Zephyr is that every driver is different, and some of them don't implement any power management features whatsoever. 

The idea behind the PM system is good on paper, but because it is not implemented across the board it becomes ineffective in practice. 

I believe the best approach is to have the application keep track of when the sensors are needed and when they can be put to sleep, and control this manually, rather than depending on the PM system to do it for you. 

Even with this approach some of the sensor drivers might need to be modified in order to properly support low power operation, as this hasn't necessarily been a priority when the drivers where implemented. 

pm_device_action_run() won't work if the driver doesn't implement any power management. Also, be aware that the pm_device_action_run() function don't do reference counting, so if you need to share a sensor between multiple threads/modules then you would have to handle this separately. 

And if you really want to go the extra mile you can improve the driver and issue a PR to have it included in upstream Zephyr 

Best regards
Torbjørn

[dan_opito]

Thanks, Torbjørn, for your advice on this. I think you're right that the best way forward is having each module handle when they need the sensor and disabling it when they do not. For some sensors this might require introducing delays, but the event architecture is already available to handle this from the data module point of view - so I don't foresee any issues.

Yes, absolutely, we've got some drivers to contribute upstream. This will be another case of minor tweaks we can push back to help out the Zephyr Project (and the downstream NCS port as a result!).

As an added side-benefit of doing it in the modules that use the sensors, any hacks needed to make it work (to circumvent API for example) feels less bad when it's contained in a small part of the code base :). A concrete example would be accessing the I2C bus directly (device->config->bus) to push the device into sleep mode. This was how I prototyped by savings the first time around.

[ovrebekk]

Hi Dan

Delays shouldn't be a problem as long as you handle it in an asynchronous manner. One advantage of using an RTOS is that (seemingly) blocking delays are fine since only the current thread is actually halted, and the system itself will go to sleep, or service other threads, in the mean time. 

Generally the application should know when sensors are needed and when they are not, allowing you to optimize sensor usage in order to get a good compromise between power consumption and performance. 

It is much appreciated if you contribute back upstream the improvements you make, typically that is not something that gets prioritized when time is short ;)

dan_opito said:

As an added side-benefit of doing it in the modules that use the sensors, any hacks needed to make it work (to circumvent API for example) feels less bad when it's contained in a small part of the code base :). A concrete example would be accessing the I2C bus directly (device->config->bus) to push the device into sleep mode.

If you modify the driver directly why would accessing the I2C bus directly be a hack? 

By modifying the drivers directly you should be able to add support for power management at the right level, without having to hack it in at a higher level.  

Best regards
Torbjørn