This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

To identify, adjust sleep period of sample app threads?

Hello Devzone Community,

My name is Ted, and I am continuing my effort to configure a Zephyr based, Nordic aws_iot sample app based firmware to run and support nRF9160 lowest specified power consumption. I am in the process of putting all application threads to sleep, so that Zephyr's idle thread can put the application processor into a low power sleep mode. Nordic Power Profiler waveforms of measured current draw, however, indicate that I've failed in this. Some part of the overall firmware binary appears to be yet running. Lowest average current I achieve is ~190 microamps.

My Devzone account doesn't permit me to upload photos, but here is a link to two captures from Nordic's power profile which I took earlier today.  My apology that I must include an external link here and not the images directly:

wiki.neelanurseries.com/.../20220311

The first one shows a general pattern of current regularly spiking from a few microamps to around three milliamps.  The zoomed capture shows the ten millisecond spikes, a more distinct fast spike and a more blunted longer duration one.

These are taken during a 45 second period when all my application threads are sleeping -- for 45 seconds -- and LTE modem is turned off, and GPIOs to hardware enable points of our board are also turned off, effectively powering down the hardware outside the nRF9160.

From these waveforms I observe that some ARM processor activity appears to be consuming current every ten milliseconds. But though I have put my app threads into a nearly minute long sleep period, and have searched for the sleep periods set on aws_iot sample app threads, I cannot see any firmware configured device or service that is configured to run every ten milliseconds.

This work is based on Nordic's aws_iot sample app, as it appears in ncs v1.6.1. The sample app creates three or four Zephyr threads. Zephyr itself creates a  thread for `int main()` and an idle thread. Work of my own adds a few more threads at run time. These latest threads I can readily put to sleep for minutes or even hours, with a call to k_sleep() in a given thread's effective main function.

I have enabled Zephyr's thread_analyzer feature, and can observe simple reporting at run time of the threads of this firmware. Here is a typical Zephyr output I can see via a serial connection, and using a simple CLI to invoke a handful of commands:

Excerpt 1: Zephyr thread summary report

```1) 'thread_accelerometer' stack size 2048 bytes, stack used 960 bytes, 46%
2) 'thread_dev_work' stack size 3072 bytes, stack used 1504 bytes, 48%
3) 'thread_simple_cli' stack size 4096 bytes, stack used 1768 bytes, 43%
4) 'thread_led' stack size 512 bytes, stack used 136 bytes, 26%
5) 'download_client' stack size 4096 bytes, stack used 104 bytes, 2%
6) 'at_cmd_socket_thread' stack size 1472 bytes, stack used 688 bytes, 46%
7) 'time_thread' stack size 1024 bytes, stack used 728 bytes, 71%
8) 'connection_poll_thread' stack size 3072 bytes, stack used 296 bytes, 9%
9) '0x20015ab0' stack size 1024 bytes, stack used 168 bytes, 16%
10) 'sysworkq' stack size 2048 bytes, stack used 168 bytes, 8%
11) 'idle 00' stack size 320 bytes, stack used 56 bytes, 17%```


Here I modify the report to annotate each thread's origin, which affects how easily or not I can adjust a thread's sleep time:

Excerpt 2:

```1) 'thread_accelerometer' . . . home authored application
2) 'thread_dev_work' . . . home authored application
3) 'thread_simple_cli' . . . home authored application
4) 'thread_led' . . . home authored application
5) 'download_client' . . . aws_iot sample app
6) 'at_cmd_socket_thread' . . . aws_iot sample app
7) 'time_thread' . . . aws_iot sample app
8) 'connection_poll_thread' . . . aws_iot sample app
9) '0x20015ab0' . . . ?, not known
10) 'sysworkq' . . . Zephyr 2.6.0
11) 'idle 00' . . . Zephyr 2.6.0```


So far I have been able to track down in Zephyr's dwt.h library header file that the time thread has a default sleep period of 3600 seconds. Tracing thread 'at_cmd_socket_thread' which is implemented in ncs/nrf/subsys/net/lib/aws_iot/src/aws_iot.c I cannot tell what this thread's effective sleep period is.

Question (1): Can a Devzone team member or community member help me identify where this thread's sleep time is realized?

I have yet to find the implementation details for thread 'download_client', 'connection_poll_thread' and '0x20015ab0'. The last of these threads I have concern for. It is not named, and so I cannot be sure whether it is a Zephyr construct or a Nordic sample app construct. This unnamed thread has shown in the thread analyzer report since I enabled Zephyr's analyzer.


Question (2): is static code analysis the only way to learn "who" in the Zephyr and other source codes is creating the unnamed thread?

Until I can identify the unnamed thread, I won't know whether I can cleanly properly adjust its sleep period. If it is a kernel thread, my app space firmware won't automatically have privileges to make any changes to a kernel thread. I believe there are some extra steps to be able to do so, if that is even possible.

I recognize also that I cannot simply change sleep times of the aws_iot sample app threads, and expect celular level and MQTT client-broker connections to continue to function. I'll need to study and find a way to cleanly close connections before putting such threads to sleep.

Thanks ahead of time to all who can shed some light and help on these deep sleep and nRF9160 power saving questions!

- Ted

  • I have not seen this kind of loop before. I have also given the hello_world_low_pwer .zip to a colleague of mine with a Linux computer, and for him the project builds just fine.

    However, he did tell me that he have seen such loops when a project has been moved from one location to another. So you could try to delete the build folder and the .vscode folder and see if that helps.

    However, in the low power projects, and the current meassurement blogpost, there should be compiled versions of the programs as well. So you should be able to use them to meassure the current consumption even though you can't build the projects.

    Or, you could create your own project by applying the same changes:

    1. Enable CONFIG_NRF_MODEM_LIB (which also requires CONFIG_NETWORKING, CONFIG_NET_SOCKETS and CONFIG_HEAP_MEM_POOL_SIZE)

    2. Disable CONFIG_SERIAL in both the application and in any child images (CONFIG_NRF_MODEM_LIB requires that you build the application for the non-secure domain, so you need the SPM to start your application). If you do not, you risk the child image starting the UART, and the application not turning it off.

    3. Ensure that the application doesn't do anything, so that the idle thread puts the device to sleep.

  • Hello Didrik,

    When you hear back findings on the Linux / Ubuntu command line based build, please let me know.  I am actually not using VSCode.  So far `west` and the other `west` orchestrated command line  tools are working well for me. Past experience with integrated development environments has been useful on some points, but always more difficult in terms of understanding compile time settings.  IDEs nearly always seem to hide configuration settings of project `make` and more recently cmake files.

    Thank you also for including `merged.hex` in the example app archives.  Failing to reach a successful build outcome I could still test the specific board for which those pre-compiled binaries are built.

    Regarding the enumerated changes in your reply, yes, I more or less learned those through some trial and error and searching the hello_world / ncs workspace' Kconfig settings.  I am finding `west build -t menuconfig` to be helpful and educational, though I don't know that I will move much beyond scratching the surface of this programming realm.

    Question 1 - Curious, each of your 'hello_world' examples compiles for you with only the prj.conf symbols I find in the zipped archives, correct?  Last night's exercise to revisit and combine elements of two of the example zips you sent provides a good lesson.  But if the zips compile for you, is it that you are building in VSCode, and VSCode holds some project settings outside of your shared, zipped sources?

    Question 2 - does your hello_world `west` manifest file pull in any different set of ncs or third party git repositories, with respect to the west.yml file I posted here a day or two ago?

    Question 3 - I have our nRF9160DK board connect to PPK II as shown in measuring current in Ampere Meter mode.  My first current measurements are around 46 nano-amperes, which seems too low to be real.  In particular, I kept the `while (1) { printk(...); k_msleep(1000); } code block in main() routine, expecting to see a current spike at about one Hertz.  I don't see that.  Is a standard USB-C cable from computer providing 5VDC to the development board?

    When I flash and run the hello_world app, I see the following Zephyr start up messages arrive over the board-powering USB-C cable:

    *** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***
    Flash regions           Domain          Permissions
    00 01 0x00000 0x10000   Secure          rwxl
    02 31 0x10000 0x100000  Non-Secure      rwxl
    
    Non-secure callable region 0 placed in flash region 1 with size 32.
    
    SRAM region             Domain          Permissions
    00 07 0x00000 0x10000   Secure          rwxl
    08 31 0x10000 0x40000   Non-Secure      rwxl
    
    Peripheral              Domain          Status
    00 NRF_P0               Non-Secure      OK
    01 NRF_CLOCK            Non-Secure      OK
    02 NRF_RTC0             Non-Secure      OK
    03 NRF_RTC1             Non-Secure      OK
    04 NRF_NVMC             Non-Secure      OK
    05 NRF_UARTE1           Non-Secure      OK
    06 NRF_UARTE2           Secure          SKIP
    07 NRF_TWIM2            Non-Secure      OK
    08 NRF_SPIM3            Non-Secure      OK
    09 NRF_TIMER0           Non-Secure      OK
    10 NRF_TIMER1           Non-Secure      OK
    11 NRF_TIMER2           Non-Secure      OK
    12 NRF_SAADC            Non-Secure      OK
    13 NRF_PWM0             Non-Secure      OK
    14 NRF_PWM1             Non-Secure      OK
    15 NRF_PWM2             Non-Secure      OK
    16 NRF_PWM3             Non-Secure      OK
    17 NRF_WDT              Non-Secure      OK
    18 NRF_IPC              Non-Secure      OK
    19 NRF_VMC              Non-Secure      OK
    20 NRF_FPU              Non-Secure      OK
    21 NRF_EGU1             Non-Secure      OK
    22 NRF_EGU2             Non-Secure      OK
    23 NRF_DPPIC            Non-Secure      OK
    24 NRF_REGULATORS       Non-Secure      OK
    25 NRF_PDM              Non-Secure      OK
    26 NRF_I2S              Non-Secure      OK
    27 NRF_GPIOTE1          Non-Secure      OK
    
    SPM: NS image at 0x10000
    SPM: NS MSP at 0x200179e8
    SPM: NS reset vector at 0x124dd
    SPM: prepare to jump to Non-Secure image.
    

    But I never see the "hello world" or "printk" messages themselves.  Am I correct that I should see these messages?

    When the 9160 DK is powered, the Linux station enumerates three /dev/ttyACMx devices:  ttyACM0, ttyACM1, ttyACM2.  I see no terminal messages on ACM1 nor ACM2.

    My terminal program is configured for no hardware flow control, no software flow control, and baud of 115200.  I ought to see at least the starting "hello world" message, yes?

    Ah wait, I may need to strike my last question. With UARTs and high speed clocks turned off I will not see any "hello world" message, correct?  As a sanity check then that firmware is actually running, I would want some alternate heartbeat test such as flashing an LED?

    - Ted

  • tedhavelka said:

    Question 1 - Curious, each of your 'hello_world' examples compiles for you with only the prj.conf symbols I find in the zipped archives, correct?  Last night's exercise to revisit and combine elements of two of the example zips you sent provides a good lesson.  But if the zips compile for you, is it that you are building in VSCode, and VSCode holds some project settings outside of your shared, zipped sources?

    Question 2 - does your hello_world `west` manifest file pull in any different set of ncs or third party git repositories, with respect to the west.yml file I posted here a day or two ago?

    I use the Toolchain Manager to manager my NCS installations, and don't keep a sepparate West workspace for each application. So, when I open a terminal through the Toolchain Manager, it will create an environment where West/CMake finds the right version of the toolchain, etc. Perhaps the most important effect for this conversation is that it sets ZEPHYR_BASE to point to my NCS 1.6.0 installation (there shouldn't be any . Similarly, I have configured VS Code to use my 1.6.0 installation (you can see the settings in the .vscode folder).

    But there shouldn't be any "magic" happening. As long as West finds your 1.6.0 installation, I would have expected it to work.

    tedhavelka said:
    Question 3 - I have our nRF9160DK board connect to PPK II as shown in measuring current in Ampere Meter mode.  My first current measurements are around 46 nano-amperes, which seems too low to be real.  In particular, I kept the `while (1) { printk(...); k_msleep(1000); } code block in main() routine, expecting to see a current spike at about one Hertz.  I don't see that.  Is a standard USB-C cable from computer providing 5VDC to the development board?

    It might be that the printk is turned into a no-op when the console is disabled (which it will be because we disabled it's only backend). But I agree that 46 nA is a bit too good. What DK version do you have?

    If your DK didn't come with a jumper on the "nRF current meassurement" header, you need to cut a solder bridge connecting the two pins: https://infocenter.nordicsemi.com/topic/ug_nrf91_dk/UG/nrf91_DK/prepare_board.html

    tedhavelka said:
    Ah wait, I may need to strike my last question. With UARTs and high speed clocks turned off I will not see any "hello world" message, correct?  As a sanity check then that firmware is actually running, I would want some alternate heartbeat test such as flashing an LED?

    Yes, without the UART, you will not see messages over UART Slight smile

    I'll try to be optimistic and provide another sample project. This time the blinky sample, with the same changes as I made to the hello_world sample. When I meassure it on my 1.0.0 DK with a PPK2 in source meter mode, I get a total average of ~3.8 uA, and ~2.7 uA in the idle periods.

    The compiled .hex file is included as well.

    blinky_low_power.zip

  • Hello Didrik,

    On 2022-04-05 Didrik asked:  What DK version do you have?

    A white rectangular sticker on the non-component side of my nRF9160DK reads:

    PCA10090

    1.0.0

    2021.11

    960048580

    Contains FCC ID:  xxxxx . . .

    Contains IC:  24529-NRF9160

    Just downloaded and unzipped your latest, number four nRF9160 low power sample app, "blinky_low_power.zip".  I tried to compile again with `west build -b nrf9160dk_nrf9160ns` and again have that cyclic build error where cmake cannot get past what looks to be Kconfig processing.

    All the same, I'd actually gotten the same blinky app code merged in with my prior modified version of the very first intended low power sample you sent / shared in this Devzone post.  So my "doubting Thomas" streak is sated that I am indeed able to modify the low power app -- something I cannot see at any terminal with UART and high speed clocks suspended -- and I can now share back with you the code which builds on my host and runs on my hardware.

    Question - with PPK II connected to "nRF Current Measurement" jumper, is it true that I am measuring the current consumed only and only by the nRF9160 SiP on the nRF9160DK development kit?

    I thought I had seen mention of this point in an earlier reply of yours, Didrik, but I could not find it reviewing all our exchanges in this Devzone ticket.

    Thank you and will keep you posted on progress I make with 9160DK current measurements on my side.

    - Ted

  • tedhavelka said:
    Question - with PPK II connected to "nRF Current Measurement" jumper, is it true that I am measuring the current consumed only and only by the nRF9160 SiP on the nRF9160DK development kit?

    Yes, the "nRF Current Measurement" header will only give you the current drawn by the nRF9160. The GPIO pins are powered through a sepparate source.

Related