nrf9160 Modem firmware 1.3.0 intermittent fails to register in LTE network(LTE-m)

[anderkee]

Hi!

I'm using NRF v1.6.1 and Modem firmware v1.3.0 with a nrf9160.
After upgrading to Modem firmware v.1.3.0 we have experienced intermittent failures to register in LTE network(LTE-m mode).
The problem has occurred both at our office and on a customer location far from us.
It's not related to a specific LTE cell it's a common issue.

When this happens we only receive +CEREG: 2 in our application log and no matter how long we wait the code is never able to receive a +CEREG: 5.
This behavior has not ever been seen before. I've been working with nrf9160 since Modem version 1.2.0 and NRF v1.3.0.

A new west flash with software will not recover 9160 to a working state, the chip will remain unable to register to the LTE network.
The only way to recover from this state is to hold down the button and perform a shipment mode deep sleep and connect to a USB cable and wake up the chip.

There seems to be some new problem regarding LTE registration related to the new Modem firmware v1.3.0.
I've seen other issues on this forum with people having problem with this Modem firmware, will there be a fix for this anytime soon?

This intermittent failure has happen a couple of times since upgrading to the latest Modem firmware.
Is a fix to this problem on it's way?
The issue lowers the trust we have for this platform if we can't rely on that the chip is always able to register to the LTE network.

Best regards
Kenneth Andersson

[anderkee]

Hi Håkon!

It seems to be something similar to the reset loop restriction however the device was unable to register not just within 30 minutes but until the customer put the device into shipment mode.

The problem for me is to replicate the problem it's intermittent, we don't have any device that is stuck in this deadlock at the moment.
The customer was able to recover their devices.

I forgot to mention our software is based upon Asset Tracker v1 with our own modificiations.
I use the standard NRF methods lte_lc_system_mode_set, lte_lc_init and set system mode to LTE_LC_SYSTEM_MODE_LTEM_GPS

What I noticed with my own device that were stuck without registration was that the device was not in a reset cycle.

The problem for me is that I haven't been able to force this behavior again.
Thank you for the links, I'll add the subscribe for %MDMEV

Has the reset loop restriction been added in Modem firmware v1.3.0?

Best regards

Kenneth Andersson

[Håkon Alseth]

Hi Kenneth,

 

anderkee said:

It seems to be something similar to the reset loop restriction however the device was unable to register not just within 30 minutes but until the customer put the device into shipment mode.

The problem for me is to replicate the problem it's intermittent, we don't have any device that is stuck in this deadlock at the moment.
The customer was able to recover their devices.

I forgot to mention our software is based upon Asset Tracker v1 with our own modificiations.
I use the standard NRF methods lte_lc_system_mode_set, lte_lc_init and set system mode to LTE_LC_SYSTEM_MODE_LTEM_GPS

It is hard to say what this can be without looking at logs showing the issue.

If you are able to replicate this behavior locally, a modem trace can help us figure out what is happening.

Here's a guide on how to enable and capture modem trace: https://infocenter.nordicsemi.com/index.jsp?topic=%2Fug_trace_collector%2FUG%2Ftrace_collector%2Fintro.html

Could you also attach the full application log?

 

anderkee said:

The problem for me is that I haven't been able to force this behavior again.
Thank you for the links, I'll add the subscribe for %MDMEV

Has the reset loop restriction been added in Modem firmware v1.3.0?

Yes, this is added for the v1.3 series. You can see this in the orange tag near to the AT command documentation:

https://infocenter.nordicsemi.com/topic/ref_at_commands/REF/at_commands/mob_termination_ctrl_status/mdmev.html?cp=2_1_4_38

 

Kind regards,

Håkon

[anderkee]

Hi Håkon!

I've successfully been able to replicate the Reset loop.
I used a simple code that forced the device to crash every time I pressed a button.
After a multiple of times I recieved the %MDMEV: RESET LOOP.
After a bit of struggle I was also able to not just get the output in the log but also in my own code by registering a new at_notif_register_handler.

I left the device and went for meetings and when I came back the device had successfully registered against the LTE network. Unfortunately my log had been broken due to BLE Uart initialization.

However I must ask is the only way to recover from a Reset loop to wait out 30 minutes before connecting again towards the LTE network?

I've read both the links you  added and the AT commands without finding any suggestions:

https://infocenter.nordicsemi.com/pdf/nrf91_at_commands_v1.7.pdf

There are no way to make a graceful modem deinitialization with +CFUN=0 and be able to revert the reset loop?

I'll try to get a complete log today and update this thread with it.

Best regards

Kenneth

[Håkon Alseth]

Hi Kenneth,

 

You should try to catch the assertion/fault that causes the reset to occur.

Do you have the watchdog enabled?

 

Make sure that you have a serial output, that can print any assertion information and log this to file.

Try setting this in prj.conf:

# uncomment this one for blocking assertion
# CONFIG_RESET_ON_FATAL_ERROR=n
CONFIG_SPM_SERVICE_NS_HANDLER_FROM_SPM_FAULT=y
CONFIG_SPM_NS_DEBUG_LOGS_DUMP=y

 

Note the commented configuration as well, which enables blocking assertions/faults. If a hardfault occurrs, you must also need to set the CONFIG_RESET_ON_FATAL_ERROR in your secure side configuration.

This can be done by creating the folder and file child_image/spm.conf in your application folder.

 

If you get assertion information, please attach the full log here so I can look deeper.

 

Kind regards,

Håkon

[anderkee]

Please close this ticket.

Best regards
Kenneth