Simple Modem on/off test

[Randall]

I'm working on an application where I will need to initialize the modem, send/receive messages and then turn it down to low power until the next checkin.  I did a real simple loop in a thread that turned on and off the modem and it always ends up locking up in the reinitialization or power down of the modem.  I've tried several variants based on examples I've seen on the forum but nothing seems to work.  Right now I create a new thread and have a loop like this;

while( true )
{
err = k_msgq_get(&mm_msg_que, &recvEvent, K_SECONDS(CHECK_IN_TIMER_INTERVAL_SECONDS));
if ( (err == -EAGAIN) || (err == 0) )
{
printk( "Initializing Modem\n" );
if ( (err = lte_lc_init_and_connect() ) )
{
printk( "Unable to connect to Modem\n" );
k_sleep(5000);
continue;
}
lte_lc_psm_req(true);
printk( "Sending Check In Message\n" );

printk( "Pretending to connect to Cloud\n" );
k_sleep(5000);
dk_set_led_on(2);
k_sleep(5000);
dk_set_led_off(2);

lte_lc_psm_req(false);
printk( "Turning off Modem\n" );
if ( (err = lte_lc_offline() ) )
{
printk( "Failure turning off Modem: %d\n", err );
}
}
k_sleep(1000);
}

Is this the improper way to do it.  I'm thinking that it must be locking up on the blocking send().  This seems like such a simple task, I'm not sure what I'm missing.

[Randall]

I'm not sending any data.  That is the point.  I'm enabling the modem, waiting 10 seconds, then disabling the modem. The CHECK_IN_INTERVAL is another 10 seconds so it isn't connecting every second.  I'm just using this as a cycle test to experiment with turning on and off the modem.  It will be difficult to send the whole project.  This is part of a separate thread that is managing modem communications.  I also have another thread that I'm working on that is handling communications with peripherals.  

[Sigurd]

I have not been able to reproduce this lockup. Here is the code I used to test this:

main.c

/*
 * Copyright (c) 2018 Nordic Semiconductor ASA
 *
 * SPDX-License-Identifier: LicenseRef-BSD-5-Clause-Nordic
 */

#include <zephyr.h>
#include <stdio.h>
#include <uart.h>
#include <string.h>

#include <lte_lc.h>
#include <dk_buttons_and_leds.h>


/**@brief Recoverable BSD library error. */
void bsd_recoverable_error_handler(uint32_t err)
{
	printk("bsdlib recoverable error: %u\n", err);
}

/**@brief Irrecoverable BSD library error. */
void bsd_irrecoverable_error_handler(uint32_t err)
{
	printk("bsdlib irrecoverable error: %u\n", err);

	__ASSERT_NO_MSG(false);
}

void main(void)
{
	printk("The test sample started\n");
	
	bool testing = true;
	
	int err = 0;
	
	while(testing){
			
	printk( "Initializing Modem\n" );
	printk("LTE Link Connecting ...\n");
	
	err = lte_lc_init_and_connect();
	
	if(err == 0)
	{
		printk("LTE Link Connected!\n");
	}
	else{
		printk("LTE link could not be established.\n");
	}
	
	
	lte_lc_psm_req(true);
	
	printk( "Pretending to connect to Cloud\n" );
	k_sleep(5000);
	//dk_set_led_on(2);
	k_sleep(5000);
	//dk_set_led_off(2);
	
	lte_lc_psm_req(false);
	
	printk( "Turning off Modem\n" );
	
	if ( (err = lte_lc_offline() ) )
	{
	printk( "Failure turning off Modem: %d\n", err );
	}
	else{
		printk("Modem turned off\n");
	}
	
	k_sleep(1000);
	
    }

}

prj.conf

#
# Copyright (c) 2019 Nordic Semiconductor ASA
#
# SPDX-License-Identifier: LicenseRef-BSD-5-Clause-Nordic
#
# General config
CONFIG_ASSERT=y
CONFIG_TEST_RANDOM_GENERATOR=y

# Network
CONFIG_NETWORKING=y
CONFIG_NET_SOCKETS=y
CONFIG_NET_SOCKETS_OFFLOAD=y

CONFIG_LOG=y
CONFIG_LOG_IMMEDIATE=y
#CONFIG_AT_CMD_LOG_LEVEL_DBG=y
#CONFIG_LOG_DEFAULT_LEVEL=3
#CONFIG_LOG_OVERRIDE_LEVEL=3
CONFIG_DEBUG=y


# BSD library
CONFIG_BSD_LIBRARY=y

# AT host library
CONFIG_AT_HOST_LIBRARY=y
CONFIG_UART_INTERRUPT_DRIVEN=y

# Stacks and heaps
CONFIG_MAIN_STACK_SIZE=2048
CONFIG_HEAP_MEM_POOL_SIZE=16384

# Disable native network stack to save some memory
CONFIG_NET_IPV4=n
CONFIG_NET_IPV6=n
CONFIG_NET_UDP=n
CONFIG_NET_TCP=n

# LTE link control
CONFIG_LTE_LINK_CONTROL=y
CONFIG_LTE_AUTO_INIT_AND_CONNECT=n

# Library for buttons and LEDs
CONFIG_DK_LIBRARY=y
CONFIG_DK_LIBRARY_INVERT_LEDS=n

I used this command to build and flash the sample: 

nrfjprog -e && west build -b nrf9160_pca10090ns -d build -p && west flash -d build

PS: Modem firmware v1.0.1 was released this week. If you haven't already updated, I recommend to update. You can download the latest modem version from this page. This blog post here is for v1.0.0, but the end of the blog post shows you how to update the modem firmware using the programmer app in nRF Connect for desktop.

[Randall]

How long did you run it?  It takes several loops before it locks up, but usually within a day.  

I am still running modem firmware v1.0.0, I will upgrade and start running a new round of tests.

Thanks

[Randall]

Even with Modem Firmware 1.0.1 it still locks up.  It goes into lte_lc_init_and_connect() after about 30 loops and doesn't come back.  It's doing the same thing you are doing, but in a different thread that is kicked off by the main function.  

[E3V3A]

Sound like a memory leak. Try to check the heap...

[Randall]

I've eliminated the use of the lte_lc module and gone to directly using the at_cmd module.  I'm using the at_cmd_write_with_callback function as well as setting the notification handler to the same funciton that prints out what responses I get from the modem.  I appear to be battling 2 problems.  Sometimes the modem gets stuck and the last feeback message I see is "+CEREG: 2,"4311","049C6410",7".  So the program is in perpetual unregistered mode.   I've also turned on the Debug logging of the AT_CMD and on some other runs I will start getting "[00:28:05.921,997] <err> at_cmd: Failed to send AT command".  So it seems I'm doing something to the modem to lock it up.  Is there a way to programmatically to reset the modem without reseting the rest of the program?

[Sigurd]

Hi,

There are some bugfixes in the latest bsdlib versions. If you are on the ncs v1.0.0 tag, then it will by default use bsdlib v0.3.3. You can change it to use the latest verision by modifying the west.yml file, and point nrfxlib to the latest revision/commit. (There is currently a active PR for this on master branch to move from v0.3.3 to v0.4.0).

- name: nrfxlib
path: nrfxlib
revision: 35120d32233dc354c89d8c62eaa6267093c7e35b

BUT, if you are seeing that you are not able to connect after exactly 30 times, then this is not a bug. This is the GSMA specified Radio Policy Manager (RPM) feature which is blocking the ATTACH command. RPM is used for protecting the network from excessive signalling loads in various scenarios. In this case default value 30 is used for PDP Context Activation Requests (TS.34_8.2.4_REQ_010, See page 43 in this pdf)

[Randall]

So how is this supposed to be done.  I'm trying to put together a system that periodically connects to the cloud sends update messages and then goes back into low power.  I tried doing the same thing but staying in CFUN=1 mode and just putting it in low power after each simulated send.  Then it starts giving me the "[01:10:12.218,963] <err> at_cmd: Failed to send AT command" error.  And I've even updated the nrfxlib per your instructions.  Is there any way to accomplish the goal I have?

[RonanB96]

I'm seeing issues with CFUN=1 hanging after disconnecting and reconnecting, sometimes the first reconnect, sometimes more. I tried to change the nrfxlib revision but this gave me cmake errors with mbedtls not being found. Have you found another way around this

[Randall]

I'm now just leaving CFUN=1 and continuing.  I've discovered that you can set the power saving mode (psm) and things still work.  I'm assuming that it allows the modem to go into power saving mode when not actively communicating.  I haven't tested this all out yet, but that is the track I'm proceeding on.

I had the same problem with mbedtls and had to revert back to v1.0.0

[arthurkoroghlian]

Did you ever receive a solution for this?  We are at this same exact issue at this current moment.

[Randall]

The online/offline issue was what they claimed.  The cell phone providers will shutdown any LTE-M connection from a device that happens more than 30 times in one hour.  The other piece of advice I received is that you don't put the modem in and out of low power.  Once you set the PSM and eDRX parameters, the modem takes care of it itself.  So that is what I've been doing and it seems to work fine.  There is some issue that PSM seems to work only with some carriers, but I've come to rely on eDRX because my modem use frequency is too high for PSM to be of any use.  

Or is it the mbedtls problem that you're talking about?  I've just reverted back to v1.0.0.  I've also upgraded my modem firmware to the latest 1.0.1.  As long as I don't send repeat CPSMS messages.  It works fine.  I've run it for several days and a time and it doesn't crash.