This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

To identify, adjust sleep period of sample app threads?

Hello Devzone Community,

My name is Ted, and I am continuing my effort to configure a Zephyr based, Nordic aws_iot sample app based firmware to run and support nRF9160 lowest specified power consumption. I am in the process of putting all application threads to sleep, so that Zephyr's idle thread can put the application processor into a low power sleep mode. Nordic Power Profiler waveforms of measured current draw, however, indicate that I've failed in this. Some part of the overall firmware binary appears to be yet running. Lowest average current I achieve is ~190 microamps.

My Devzone account doesn't permit me to upload photos, but here is a link to two captures from Nordic's power profile which I took earlier today.  My apology that I must include an external link here and not the images directly:

wiki.neelanurseries.com/.../20220311

The first one shows a general pattern of current regularly spiking from a few microamps to around three milliamps.  The zoomed capture shows the ten millisecond spikes, a more distinct fast spike and a more blunted longer duration one.

These are taken during a 45 second period when all my application threads are sleeping -- for 45 seconds -- and LTE modem is turned off, and GPIOs to hardware enable points of our board are also turned off, effectively powering down the hardware outside the nRF9160.

From these waveforms I observe that some ARM processor activity appears to be consuming current every ten milliseconds. But though I have put my app threads into a nearly minute long sleep period, and have searched for the sleep periods set on aws_iot sample app threads, I cannot see any firmware configured device or service that is configured to run every ten milliseconds.

This work is based on Nordic's aws_iot sample app, as it appears in ncs v1.6.1. The sample app creates three or four Zephyr threads. Zephyr itself creates a  thread for `int main()` and an idle thread. Work of my own adds a few more threads at run time. These latest threads I can readily put to sleep for minutes or even hours, with a call to k_sleep() in a given thread's effective main function.

I have enabled Zephyr's thread_analyzer feature, and can observe simple reporting at run time of the threads of this firmware. Here is a typical Zephyr output I can see via a serial connection, and using a simple CLI to invoke a handful of commands:

Excerpt 1: Zephyr thread summary report

```1) 'thread_accelerometer' stack size 2048 bytes, stack used 960 bytes, 46%
2) 'thread_dev_work' stack size 3072 bytes, stack used 1504 bytes, 48%
3) 'thread_simple_cli' stack size 4096 bytes, stack used 1768 bytes, 43%
4) 'thread_led' stack size 512 bytes, stack used 136 bytes, 26%
5) 'download_client' stack size 4096 bytes, stack used 104 bytes, 2%
6) 'at_cmd_socket_thread' stack size 1472 bytes, stack used 688 bytes, 46%
7) 'time_thread' stack size 1024 bytes, stack used 728 bytes, 71%
8) 'connection_poll_thread' stack size 3072 bytes, stack used 296 bytes, 9%
9) '0x20015ab0' stack size 1024 bytes, stack used 168 bytes, 16%
10) 'sysworkq' stack size 2048 bytes, stack used 168 bytes, 8%
11) 'idle 00' stack size 320 bytes, stack used 56 bytes, 17%```


Here I modify the report to annotate each thread's origin, which affects how easily or not I can adjust a thread's sleep time:

Excerpt 2:

```1) 'thread_accelerometer' . . . home authored application
2) 'thread_dev_work' . . . home authored application
3) 'thread_simple_cli' . . . home authored application
4) 'thread_led' . . . home authored application
5) 'download_client' . . . aws_iot sample app
6) 'at_cmd_socket_thread' . . . aws_iot sample app
7) 'time_thread' . . . aws_iot sample app
8) 'connection_poll_thread' . . . aws_iot sample app
9) '0x20015ab0' . . . ?, not known
10) 'sysworkq' . . . Zephyr 2.6.0
11) 'idle 00' . . . Zephyr 2.6.0```


So far I have been able to track down in Zephyr's dwt.h library header file that the time thread has a default sleep period of 3600 seconds. Tracing thread 'at_cmd_socket_thread' which is implemented in ncs/nrf/subsys/net/lib/aws_iot/src/aws_iot.c I cannot tell what this thread's effective sleep period is.

Question (1): Can a Devzone team member or community member help me identify where this thread's sleep time is realized?

I have yet to find the implementation details for thread 'download_client', 'connection_poll_thread' and '0x20015ab0'. The last of these threads I have concern for. It is not named, and so I cannot be sure whether it is a Zephyr construct or a Nordic sample app construct. This unnamed thread has shown in the thread analyzer report since I enabled Zephyr's analyzer.


Question (2): is static code analysis the only way to learn "who" in the Zephyr and other source codes is creating the unnamed thread?

Until I can identify the unnamed thread, I won't know whether I can cleanly properly adjust its sleep period. If it is a kernel thread, my app space firmware won't automatically have privileges to make any changes to a kernel thread. I believe there are some extra steps to be able to do so, if that is even possible.

I recognize also that I cannot simply change sleep times of the aws_iot sample app threads, and expect celular level and MQTT client-broker connections to continue to function. I'll need to study and find a way to cleanly close connections before putting such threads to sleep.

Thanks ahead of time to all who can shed some light and help on these deep sleep and nRF9160 power saving questions!

- Ted

  • tedhavelka said:
    Question (1):  is an average current of ~85 microamps indicative of an nRF9180 SICA B1A running with all UARTs and high speed clocks off?

    Yes, if UART RX or HF clocks were running, you should see a power draw of >500uA.

    tedhavelka said:

    Question (3) - When firmware enables PSM mode in the nRF9160 modem, is the "wake period + sleep period" entered immediately?

    Question (4) - When the new PSM times are configured, does the PSM mode begin by keep the modem on for its active period, or does it begin by keeping the modem in power savings mode for its suspended, low power period?  E.g. which part of the power cycle occurs first?

    To send data, or communicate with the network, you need an RRC connection. How long you stay in RRC Connected mode depends on the network. After the RRC connection has been inactive for a while, the device is released from the RRC connection and enters RRC Idle mode. How long you stay here depends on the Active Time, and how often you must listen for incomming data depends on the eDRX settings. After the Active Time finishes, the device enters PSM and turn the radio completely off.

    tedhavelka said:
    Question (2)  is there a clue in the current spikes, appearing at about 25Hz, and intermittently peaking at 4mA and alternately 0.5mA?  Any part of SICA B1A hardware consume power in this way?

    I'll have to come back to you on this.

  • Good morning Didrik,

    Thank you for fast reply to my latest questions on nRF9160 current draw and PSM active and inactive periods.  To clarify my first question from above, can I be confident that I have correctly powered down all high frequency clocks in the 9160 SiP when I measure only 65 microamps to 85 microamps average current draw over a minute's time?

    I'll watch for further help on the question of the roughly 25Hz current use spike activity.

    - Ted

  • tedhavelka said:
    To clarify my first question from above, can I be confident that I have correctly powered down all high frequency clocks in the 9160 SiP when I measure only 65 microamps to 85 microamps average current draw over a minute's time?

    Yes, they are not running. If they had been running the high frequency clocks would have drawn more current.

    tedhavelka said:
    I'll watch for further help on the question of the roughly 25Hz current use spike activity.

    I've talked with one of our current consumption experts, and he's ideas was that it could be either GPIOTE IN event (edge detection on a GPIO pin), or the UART ENABLE base current (with RX disabled).

    Looking at the UART driver implementation, it looks like the driver disables the peripheral properly, so I don't think the current draw comes from there. However, the GPIO edge detection might have been set up by one of the sensor drivers you have included in your project.

    Also, looking at the code of the drivers, it looks like both can create a thread, without giving it a name, if CONFIG_<sensor>_TRIGGER_OWN_THREAD is set. Is either of those two configs set in your project?

    Also, did you meassure the current draw of the .hex file in the current meassurement blog post?

    You can also use the blinky sample with CONFIG_SERIAL=n to verify your current meassurement setup. By running an application with a known current consumption on the nRF9160, we can see check if the extra current consumption comes from somewhere else on the board.

  • Hello Didrik,

    I've reviewed my project's Kconfig symbols summary in the build artifacts directory, `ncs/myproject/build` and find that those two out-of-tree
    Zephyr sensors libraries do not trigger any threads to be called:

    ted@localhost$ grep -nr TRIGGER_OWN_THREAD ./*
    \./build/zephyr/.config.old:1626:# CONFIG_IIS2DH_TRIGGER_OWN_THREAD is not set
    ./build/zephyr/.config.old:1639:# CONFIG_LIS2DH_TRIGGER_OWN_THREAD is not set
    ./build/zephyr/.config:1626:# CONFIG_IIS2DH_TRIGGER_OWN_THREAD is not set
    ./build/zephyr/.config:1639:# CONFIG_LIS2DH_TRIGGER_OWN_THREAD is not set
    
    ted@localhost$

    I read through this file as well to be sure, and to see what related symbols are defined or not defined in the few lines before, between and after these "trigger own thread" symbols.  As I am not yet using hardware interrupt lines from the IIS2DH or other sensor I have not written any ISR for such hardware based interrupts for these sensors.

    Regarding the known current draw .hex file in your current measurements guide current measurements guide, I have not tried running that specific .hex file.  I tried something very similar by returning to Nordic's `ncs/nrf/samples/basic/blinky` Zephyr based app, building and running that on our custom board.  The result I see from `blinky` is an average current draw around 3.0 mA, with a fair amount of noise.  I have to make some adaptation to the blinky app in order to turn off some of our custom peripherals external to the nRF9160.

    Interestingly I see a similar cerca 3.0 mA current draw from my own application, specifically when I disable not only my own threads but also the call to the aws_iot library routine named `nrf_modem_lib_dfu_handler()`.  You may recall that my custom firmware is based partly on Nordic's aws_iot sample app.  Only in these recent low power tests have I started to comment out calls to aws_iot library functions.  When I comment out and no longer call `nrf_modem_lib_dfu_handler()` I lose the AT command parser for the LTE modem, and I appear to lose a bunch other control over the modem's behavior and power consumption.

    So I am not sure whether the pre-built hex file in your guide would help show me whether our board can draw that minimal, deep sleep spec'd current of about 1.4uA.

    In your recent response you mention that VSCode has a gdb plug-in, and would be configured to be thread-aware for Zephyr debugging support.  I have VSCode now installed, but am not able to get gdb running nor showing a menu or button interface for detected threads.  I've also invoked a gdb connection to our nRF9160 firmware at the command line with `west -v debug`.  I can connect to the ARM Cortex-M33 at a Linux command line with this west invocation.  It looks like I must precede my debugging commands to gdb with the token `mon` -- short for monitor? -- in order to do things like halt the firmware and "go" or run again.

    Question (1) can you recommend a good tutorial on the `mon` supported commands for ARM Cortex-M family parts?

    I've done some searching for about an hour, and have some basic gdb experience.  Nonetheless I have not found any tutorial, how-to article or blog or forum posts which go into a practical working detail of how to interact with gdb at command line, and obtain a list of Zephyr registered threads.  Seems like there is a large debugging tool set here, but no instructions or guide on how to get started and make good use of them.

    - Ted

  • tedhavelka said:
    In your recent response you mention that VSCode has a gdb plug-in, and would be configured to be thread-aware for Zephyr debugging support.  I have VSCode now installed, but am not able to get gdb running nor showing a menu or button interface for detected threads. 

    If you haven't seen it already, I would recommend this YouTube playlist on how to install and use VS Code with the nRF Connect SDK: https://www.youtube.com/playlist?list=PLx_tBuQ_KSqEt7NK-H7Lu78lT2OijwIMl

    One of the videos is about debugging, but you might want to look through the other videos as well, to double check that you have set everything up correctly.

    tedhavelka said:

    It looks like I must precede my debugging commands to gdb with the token `mon` -- short for monitor? -- in order to do things like halt the firmware and "go" or run again.

    Question (1) can you recommend a good tutorial on the `mon` supported commands for ARM Cortex-M family parts?

    It's been a while since I used GDB manually myself, but I don't think I used 'mon' for much more than 'mon reset' and 'mon halt'. I don't know about any good tutorials, but I found a list of the monitor commands supported by Segger: https://wiki.segger.com/J-Link_GDB_Server#Supported_remote_.28monitor.29_commands

    For thread aware debugging to work, you must add a plugin that is able to read and parse the information about the threads. E.g. when I start debugging the philosophers sample in VS Code, this is how JLinkGDBServerCL is started:

    Launching gdb-server: "C:\\Program Files (x86)\\SEGGER\\JLink\\JLinkGDBServerCL.exe" -singlerun -nogui -if swd -port 50000 -swoport 50001 -telnetport 50002 -device nRF9160_xxAA -select usb=960060612 -rtos "C:\\Program Files (x86)\\SEGGER\\JLink\\GDBServer\\RTOSPlugin_Zephyr.dll"

    tedhavelka said:
    The result I see from `blinky` is an average current draw around 3.0 mA, with a fair amount of noise.

    The 3mA current draw is a known issue: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/known_issues.html#other-issues

    (Sorry, I should have remembered this).

    The solution is to enable the modem library (CONFIG_NRF_MODEM_LIB=y).

Related