9160/LTE; no return from lte_lc_connect() and keeps resetting

[bluebeam]

Hi,

I'm having trouble with my custom 9160 board. It won't connect to LTE-M network and keeps resetting.

My prototype was developed using the 9160DK. I can connect to LTE-M network and send data to nrf Cloud. The combination is 9160 B1A, mfw 1.3.1, and NCS SDK1.7.1.

Now my custom board is using B0A silicon, with mfw 1.1.1. I have upgrade/downgrade between 1.1.1 and 1.3.1 multiple times. But that doesn't seem to make any difference.

My test code is simplified to like this:

LOG_INF("modem init");

lte_lc_register_handler(lte_lc_evt_handler);

err = lte_lc_init();

LOG_DBG("lte_lc_init(): %d", err);

err = lte_lc_connect();

LOG_DBG("lte_lc_connect(): %d", err);

LOG_INF("modem init done");

The log message looks like this:

00> I: modem init

00> D: Sending AT command to set system mode: AT%XSYSTEMMODE=1,0,1,0

00> D: System mode (4) and preference (0) configured

00> D: lte_lc_init(): 0

00> D: Sending AT command to set system mode: AT%XSYSTEMMODE=1,0,1,0

So it went through lte_lc_init(), but never returned from lte_lc_connect(). It will stays there for various amount of time, from a few seconds to a minute or so, then it resets.

Since this is a custom board, I try to figure out if any hardware issue could cause such problem. I assume lte_lc_connect() will search for the network and try to make a connection. The document says it has a timeout. I assume if it timed out, it should return from lte_lc_connect() with an error code? But it never returned and always resets after a while. Any possible explaination lte_lc_connect() could cause this? Or should I more look at hardware side? How can I get more debug information to troubleshoot this? I already set CONFIG_LTE_LINK_CONTROL_LOG_LEVEL_DBG=y.

The same code runs fine on the 9160DK and get connected in a few seconds.

Thanks!

[Charlie]

Hi Bluebeam,

I add a new board bb91dk_nrf9160 modified from nrf9160dk_nrf9160 to the attached sample.

You can adapt for your custom hardware with an overlay first then add them to the devicetree file.

The device tree configuration can be verified from the following nRF Connet for  VS Code window.

ncs152_udp_bluebeam.zip

Devicetree – Nordic Developer Academy (nordicsemi.com) could be a good starting point if you want to learn more about the devicetree. 

Best regards,

Charlie

[bluebeam]

I think I know what causes the compile problem but I don't understand why. I realize I must remove the line "zephyr_library()" from the CMakeLists.txt file in the board file directory. What is the consequence of removing this line?

With this line removed I have added things like LEDs and beeper etc. They all work. So that seems to prove the board file/device tree is probably ok.

Now I'm back to the original problem - it still resets after making a LTE connection call. I simplified my code quite a bit and the main() is slim down to just a few lines like this. I skipped the error checking code, but I know it runs all the way to the end and all ret value is 0.

From the RTT log, it seems it sends the command: AT%XSYSTEMMODE=1,0,1,0, then nothing happens.


I'm not sure how to get more debug information from that. I already turned on LTE debug log. The last while loop gives periodic VBAT reading. It seems ok at 3.3v. The SIM card works in a 9160DK kit before.

If I replace step async call with the block call: lte_lc_connect(), then it will stuck there until reset.

I suspect maybe hardware issue. But I'd like to rule out if the software is wrong. Are you aware of any situation where it will resets itself after calling lte_lc_connect()? Or do you see if there's anything missing from this sequence?

Thanks! The code of main() and RTT logging is below:

================================================================

static void blink(int n);
static void beep(int n);
static void lte_lc_evt_handler(const struct lte_lc_evt* const evt);
static void cloud_event_handler(const struct cloud_backend* const backend, const struct cloud_event* const evt, void* user_data);
static void lte_get_modem_ver(void);
static void lte_get_hw_ver(void);
static void lte_get_vbat(void);

void main(void)
{
int ret = 0;
int step = 1;

spkr_init();
pwmleds_init();

lte_get_modem_ver();
lte_get_hw_ver();

// 1.
lte_lc_register_handler(lte_lc_evt_handler);

ret = lte_lc_init();
// 2.
ret = lte_lc_connect_async(lte_lc_evt_handler);
//ret = lte_lc_connect();
// 3.
struct cloud_backend* cloud_backend = cloud_get_binding("NRF_CLOUD");

__ASSERT(cloud_backend, "Could not get binding to cloud backend");

// 4.

ret = cloud_init(cloud_backend, cloud_event_handler);

// 5.

ret = cloud_connect(cloud_backend);

while (1){
pwmleds_blink(0, 0, 100);
k_msleep(1000);
lte_get_vbat();
}
}

//------------------------------------------------------------------------------

static void lte_lc_evt_handler(const struct lte_lc_evt* const evt)
{

  // never called
}

//------------------------------------------------------------------------------
static void cloud_event_handler(const struct cloud_backend* const backend, const struct cloud_event* const evt, void* user_data)
{

  // only received event CLOUD_EVT_CONNECTING
}

================================================================

00> *** Booting Zephyr OS build v2.4.99-ncs2-1 ***
00> E: Cannot bind UART_0
00>
00> E: UART could not be initialized: -22
00> D: led initialized
00> I: AT+CGMR: err=0, st=0, rsp=mfw_nrf9160_1.2.8
00>
00> I: AT%HWVERSION: err=0, st=0, rsp=%HWVERSION: nRF9160 SICA B0A
00>
00> D: Sending AT command to set system mode: AT%XSYSTEMMODE=1,0,1,0
00> D: Sending AT command to set system mode: AT%XSYSTEMMODE=1,0,1,0
00> I: AT%XVBAT: err=0, st=0, rsp=%XVBAT: 3371
00>
00> I: AT%XVBAT: err=0, st=0, rsp=%XVBAT: 3378
00>
00> I: AT%XVBAT: err=0, st=0, rsp=%XVBAT: 3375
00>
00> I: AT%XVBAT: err=0, st=0, rsp=%XVBAT: 3367
00>
00> I: AT%XVBAT: err=0, st=0, rsp=%XVBAT: 3386
00>
00> I: AT%XVBAT: err=0, st=0, rsp=%XVBAT: 3378
00>
00> I: AT%XVBAT: err=0, st=0, rsp=%XVBAT: 3375
00>
00> I: AT%XVBAT: err=0, st=0, rsp=%XVBAT: 3371

================================================================

[Charlie]

Hi bluebeam bluebeam,

bluebeam said:

What is the consequence of removing this line?

I am not an expert on this file, but I found a related discussion on the Zephyr GitHub repository, you can add your comments there if the solutions mentioned there does not work.

Improve out-of-tree driver experience · Issue #27048 · zephyrproject-rtos/zephyr (github.com)

bluebeam said:

Are you aware of any situation where it will resets itself after calling lte_lc_connect()? Or do you see if there's anything missing from this sequence?

If you expect to see the events happening during lte connection process, you should add some debug information to your lte_lc_evt_handler. Search lte_handler in NCS code base you will find many samples, here is the one used by the UDP sample. You would be able to find the lte connection stuck on which step.

#if defined(CONFIG_NRF_MODEM_LIB)
static void lte_handler(const struct lte_lc_evt *const evt)
{
	switch (evt->type) {
	case LTE_LC_EVT_NW_REG_STATUS:
		if ((evt->nw_reg_status != LTE_LC_NW_REG_REGISTERED_HOME) &&
		     (evt->nw_reg_status != LTE_LC_NW_REG_REGISTERED_ROAMING)) {
			break;
		}

		printk("Network registration status: %s\n",
			evt->nw_reg_status == LTE_LC_NW_REG_REGISTERED_HOME ?
			"Connected - home network" : "Connected - roaming\n");
		k_sem_give(&lte_connected);
		break;
	case LTE_LC_EVT_PSM_UPDATE:
		printk("PSM parameter update: TAU: %d, Active time: %d\n",
			evt->psm_cfg.tau, evt->psm_cfg.active_time);
		break;
	case LTE_LC_EVT_EDRX_UPDATE: {
		char log_buf[60];
		ssize_t len;

		len = snprintf(log_buf, sizeof(log_buf),
			       "eDRX parameter update: eDRX: %f, PTW: %f\n",
			       evt->edrx_cfg.edrx, evt->edrx_cfg.ptw);
		if (len > 0) {
			printk("%s\n", log_buf);
		}
		break;
	}
	case LTE_LC_EVT_RRC_UPDATE:
		printk("RRC mode: %s\n",
			evt->rrc_mode == LTE_LC_RRC_MODE_CONNECTED ?
			"Connected" : "Idle\n");
		break;
	case LTE_LC_EVT_CELL_UPDATE:
		printk("LTE cell changed: Cell ID: %d, Tracking area: %d\n",
		       evt->cell.id, evt->cell.tac);
		break;
	default:
		break;
	}
}

Best regards,

Charlie

[bluebeam]

Hi Charlie,

I should clarify: I did have some code in lte_lc_evt_handler() which does similar thing as you mentioned. I remove those lines so that the post won't be too long. Sorry for the confusion.

The problem is the event handler function is never called. As the log shows, after sending the system mode command there's no modem activity until reset. So something is wrong. I suspect we need to look at hardware side of things now but I'd like to rule out things that are obvious so we can narrow down. Could you confirm the following:

1. Is the few lines in the main() sufficient for making LTE connection? This is the simplest testing. I assume it's sufficient because the same code runs on 9160DK board and generates a series of responses from modem.

2. Further more, even with hardware issue (RF/antenna etc), the modem should still return with some error messages? Possible bad hardware causes modem to become non-responsive?

3. Is there any known situation that will cause reset after a period of time? It seems some kind of timeout reset because it's fixed duration before reset. Though sometime it's 30s, some other time 90s.

Thanks!

[Charlie]

Hi Bluebeam,

bluebeam said:

1. Is the few lines in the main() sufficient for making LTE connection? This is the simplest testing. I assume it's sufficient because the same code runs on 9160DK board and generates a series of responses from modem.

As long as the codes can work with nRF9160DK, I think the source codes in main.c should have no problem.

bluebeam said:

2. Further more, even with hardware issue (RF/antenna etc), the modem should still return with some error messages? Possible bad hardware causes modem to become non-responsive?

bluebeam said:

3. Is there any known situation that will cause reset after a period of time? It seems some kind of timeout reset because it's fixed duration before reset. Though sometime it's 30s, some other time 90s.

Yes, we need to figure out where the codes are stuck to analyse the potential HW issue, but the good news is that your application still can partly run. I did not see any previous cases reporting similar rest patterns. I just have more suggestions to help you move forward:

1) Try to use debug mode and check where the application is stuck.

2) Did you try to enable more log options and also nrf_modem_fault_handler() to see what it will report?

3) Could you enable modem trace to collect more activities inside the modem?

4) How many kits were tested by far? Did you or your team member request an HW review for this customer board before? We offer this kind of service free of charge, just share your schematic and PCB files with us in a private Devzone case.

Best regards,

Charlie