Modem 30 minute lockup

[Burt]

This is driving me totally nuts. There are inconsistent pieces of information in documents and code. I am getting a modem reset loop problem. The documentation says the timer does not run when the modem has no power. That tells me I have to leave power on in order to get out of this ridiculous mess that cuts me off from any and all LTE-M and NB-IoT networks. The message from some asset tracker code tells me I can get out by doing a power reset, but that does not work.

How in blazes do I get out of this fricking mess????? This is at least one day down the drain.

How about 5 minutes rather than 30 minutes. How about something that simply restores the modem?

Did I put a bug into the code? I can't imagine how, although I do put in printfs.

Thanks.

Burt S.

[Burt]

I eventually was able to get out of the mess. It began when I tried adding this to asset_tracker_v2 prj.conf

CONFIG_LWM2M_CARRIER_PDN_IPV4=y
CONFIG_LWM2M_CARRIER=y
CONFIG_LWM2M_CARRIER_SETTINGS=y
CONFIG_LWM2M_CARRIER_CUSTOM_APN="iot.nb"

in an attempt to do the equivalent of

AT+CGDCONT=1,"IP","iot.nb"

as indicated by the clowns at https://www.twilio.com/docs/iot/wireless/network-configuration#how-do-i-set-the-apn-with-at-commands. Not only did I lock up the modem but I wasn't configuring any "iot-nb" because I don't know how to set auto_startup to 1 in lwm2m_settings.c. It involves a modem key but I have no idea how to create a simple modem key appropriate for the purpose. It seems like this is another complicated topic involving security knowledge. Please help.

I also tried setting

CONFIG_PDN_DEFAULTS_OVERRIDE=y
CONFIG_PDN_DEFAULT_APN="iot.nb"
CONFIG_PDN_DEFAULT_FAM_IPV4=y
CONFIG_PDN_LOG_LEVEL_DBG=y
CONFIG_NVS=y
CONFIG_MODEM_KEY_MGMT=y
CONFIG_SMS=y

and although that did not break anything, it did not solve the problem whereby Twilio SIMs will not connect to T-Mobile. Can you tell me why they cannot just keep things simple like Soracom and the rest of the world?

I guess you have people who know all the tricks and I hope you can help me get back on my feet again. Thanks. Oh, yeah,

Burt

[Burt]

I guess I misread that Twilio web page. I thought they were telling me that I needed to configure the APN somewhere in order to connect/register to the network; like I had to copy the APN that the SIM uses to somewhere else in order to establish an NB-IoT connection. Anyway, I have a ticket open with KORE Wireless/Twilio and they opened a secondary ticket with T-Mobile. I don't think those KORE Wireless folks have any clue about the company Twilio that they purchased; not on a technology level, at least.

[Burt]

trace-2024-07-17T01-04-30.325Z.mtrace

Can you try this trace, Oyvind? I am seeing a MODEM RESET LOOP in the Cellular Monitor Dashboard. Also, I added a line of code

err = nrf_modem_at_printf("AT+CGDCONT=%u, %s, %s", 1, "Non-IP", "iot.nb");

Similar to what I mentioned earlier. I tried various flavors of "IP" "IPV6" "IPV4V6" and the "Non-IP" shown here. None solve a problem but do make a PDN appear in the Dashboard.

[Øyvind]

Hi Burt, thanks for sharing the modem trace. I was able to open but did not find anything out of the ordinary. The modem reset loop happens especially in the field when roaming. Have you read the Modem Reset Loop Restriction white paper? The modem reset loop restriction serves as a safeguard against unexpected issues with the cellular modem and application. Its purpose is to protect networks in case the application or cellular modem does not behave as expected. This protection ensures that the nRF91 device does not congest the network, for example, with excessive signaling (Attach requests), which could occur if the application and modem continuously reset and attempt new connections in a loop.

The modem counts all resets where the modem is not gracefully deinitialized with +CFUN=0. If modem deinitialization with +CFUN=0 is triggered before the predefined limit is reached, the reset counter is set to zero and the counting starts from the beginning. In modem firmware version 1.3.0, the limit is 5. In modem firmware version 1.3.1 and later, the limit is 7.
In product development phase, active reset loop restriction can be removed with the %XFACTORYRESET=0 command. The command should not be used for this purpose in the final product. If %XFACTORYRESET=0 is used, all modem data, including user-configurable data, is reset.

Kind regards,
Øyvind

[Burt]

Thank you very much, Oyvind. I do have a couple of questions.

You say that nothing out of the ordinary shows up in the modem trace. But what I see using the Cell Monitor Dashboard or looking at log messages is that the modem stays in searching, not registered mode, and never gets to registered mode. Any hints from the trace about why that is? The other SIMs we have, from Soracom, do not have that issue; they register.

Question about the %XFACTORYRESET=0 command: I was afraid to try that (as opposed to %XFACTORYRESET=1); I didn't know if I would have to reinstall modem firmware or something more drastic. Can you give me a few words about this? I wish also that the "few words" could be added into the AT Commands Reference for those of us who have limited expertise to understand "all modem data".

Thanks, Oyvind.

Burt

[Burt]

A couple more questions:

Related to %XFACTORYRESET=0, will this erase my loaded certificates put in via %CMNG or will it not touch them? Too bad there are not more details in the AT Commands Reference. Speaking of which, it would be nice if it stated explicitly in that document that reading certificates with %CMNG will not work when something like asset_tracker_v2 is running; it can get confusing when list works but read does not, etc. (It becomes harder and harder to search for or remember "getting started" type documentation when working with AT Commands Reference, if you follow my drift.)

And here is an excellent question: while doing nothing more than switching application firmware back and forth between NB-IoT and LTE-M, and running them, given a SIM from Soracom, and running in NB-IoT mode, we find it often takes about over 4 minutes to connect to the T-Mobile network (this is not the Twilio SIM that refuses completely to connect). But if I issue an AT%XFACTORYRESET=1 properly, and then power reset, the board connects in under 10 seconds. And it continues to connect quickly given a few power resets. Please help me understand what garbage is getting stuck in the modem that needs this cleaning. It's a really cool discovery!

Burt

[Burt]

Oyvind, Hi, I am surprised that you didn't have me issue an AT+CEER command! It is showing an "EMM 6" error, which I understand is a complaint from the network about an unacceptable IMEI.

And, if we went through all the trouble to do a modem trace, wouldn't it know about the EMM 6 problem regardless of whether I manually issued the +CEER command?

The same IMEI is fully accepted when I use a Soracom SIM. So T-Mobile must have no problem, I would think. Is it normal that the home Twilio network would check the IMEI? Any thoughts you have are appreciated.

Burt