nRF9160 modem connection issues on reboot

RE: Invoking button reset does not allow nRF9160 modem connection

Achim Kraus — Sun, 17 Mar 2024 19:23:41 GMT

[quote userid="73166" url="~/f/nordic-q-a/108787/invoking-button-reset-does-not-allow-nrf9160-modem-connection/473278"] Using the %XMONITOR is quite time critical (cannot guarantee it works 100%),[/quote]

In my experience it doesn't work at all. You always get a "%XMONITOR: 2".

RE: Invoking button reset does not allow nRF9160 modem connection

Achim Kraus — Mon, 11 Mar 2024 18:05:57 GMT

Is there a way, that the application knows, if the reject was received non-integrity protected?

RE: Invoking button reset does not allow nRF9160 modem connection

Hakon — Mon, 11 Mar 2024 16:47:17 GMT

Their response;

The forbidden TAI lists (both “forbidden tracking areas for roaming” and “forbidden tracking areas for regional provision of service” specified by 3GPP) are cleared periodically once a day. In case the reject from network was received non-integrity protected, 3GPP allows clearing the forbidden TAI lists after random time between 30 to 60 minutes elapsed from the reject. The other options are power cycle or disconnecting the SIM card. In summary, the modem will recover from the sudden tracking area specific rejects from network, but it will take some time. Anyway, this kind of sudden reject from network doesn’t seem reasonable and it is not normal that the same cell some times rejects the Attach and some times not. The only reason I can come up with is that the network got too many Attach attempts and started to reject them on this tracking area, but this is just a guess as cannot find it specified by 3GPP. Please note that it is not the same thing to hard reset the device than doing power cycle gracefully. In the latter one modem initiates Detach procedure to network in which case the next device power up does not look to network like a new Attach attempt of a device already Attached. The operator should be able to answer why there are sudden Attach rejects with cause #15, if asked by customer. At first I would still try to see whether the problem is reproducible with graceful power cycles.

In case there is a need to recover faster than what 3GPP has specified, application could subscribe +CEREG notifications and wait for tracking area specific reject cause to be received after which the current PLMN can be resolved using %XMONITOR. Using the %XMONITOR is quite time critical (cannot guarantee it works 100%), because modem will drop the cell quite soon for continuing scans after which the cell data is not anymore available. But looking at the provided log, there would have been enough time for resolving the PLMN. After the PLMN selection has been completed (subscribe %MDMEV notifications and wait for %MDMEV: SEARCH STATUS 2), application can check whether tracking area specific cause (e.g. #15) was earlier received from the problematic network 310 410. If it was, application can trigger reboot modem which will clear the forbidden TAI lists and initiate the PLMN selection again as normal.

RE: Invoking button reset does not allow nRF9160 modem connection

Hakon — Mon, 11 Mar 2024 09:40:42 GMT

[quote user="jaredwolff"]Does the modem team have any suggestions on how to handle this condition?[/quote]

I will ask them if they have any suggestion for this.

RE: Invoking button reset does not allow nRF9160 modem connection

Jared — Tue, 05 Mar 2024 13:20:02 GMT

I will agree with Achim here. The modem doesn’t seem to recover after the rejection. It’s very concerning since the MFW is a black box and if it’s on we expect it to work/connect. If I immediately reset the device again it connects so I’m not sure about your thesis about the tower rejecting the connection.

Additionally, it shouldn’t be incumbent on the customer who understands little how the firmware and device works to issue AT commands to get it to reliably connect. This is not acceptable.

Does the modem team have any suggestions on how to handle this condition?

RE: Invoking button reset does not allow nRF9160 modem connection

Achim Kraus — Tue, 05 Mar 2024 10:24:41 GMT

The point will be:

If the modem gets rejected with cause 15, does it do a retry with that network later? Or does it stick to the rejection, as mention in my ticket from last year (see my link above)?

If it sticks, then we just get an other case, where the Nordic assumption/state "15 is permanent" seems to be wrong. Similar to the case I was reporting last year, when some MVNO hiccups showed the same behavior.

And it also shows, that it's very important for reliable applications, to know, which of such "reject codes" or "reject cases" are sticky and need a CFUN=0/CFUN=1 to overcome it.

RE: Invoking button reset does not allow nRF9160 modem connection

Hakon — Tue, 05 Mar 2024 09:31:21 GMT

Some comments from the modem team;

Yes, network 310 410 starts rejecting Attach at one point with reject cause #15 "No suitable cells in tracking area". Earlier the device was happily registered on the same network and cell. Could it be that network just got enough of repeated Attach attempts without Detach in between? Does this happen if customer shuts down the modem gracefully with CFUN=0? Without graceful shut down modem has no chance to store all kind of internal data (cell history etc.) to NVM which can affect to the next start up, too.

RE: Invoking button reset does not allow nRF9160 modem connection

Jared — Mon, 04 Mar 2024 15:10:34 GMT

Sorry about that! Here it is

devzone.nordicsemi.com/.../trace_2D00_2024_2D00_03_2D00_02T02_2D00_19_2D00_51.847Z.mtrace

RE: Invoking button reset does not allow nRF9160 modem connection

Hakon — Mon, 04 Mar 2024 14:33:36 GMT

[quote user="jaredwolff"]nrf9160-no-connect-on-reset-button.pcapng[/quote]

Can you please also include the raw trace file? It should be .bin or mtrace file.

RE: Invoking button reset does not allow nRF9160 modem connection

Achim Kraus — Sat, 02 Mar 2024 17:34:35 GMT

> 302: at+CEREG: 2,"0608","00CC2911",7,0,15

Let's see, what Nordic writes this time on the reject 15 case.

I sticked last year to sporadic and short term MVNO hiccups with "reject 15", which prevents the modem from reconnect without switching off and on (see my ticket %PERIODICSEARCHCONF - relation to EMM reject cause ). Unfortunately, my request for a list of sticky error codes was refused.

RE: Invoking button reset does not allow nRF9160 modem connection

Jared — Sat, 02 Mar 2024 02:33:19 GMT

Here we go. At the beginning we get a successful connection. I removed the packets related to the application.

devzone.nordicsemi.com/.../nrf9160_2D00_no_2D00_connect_2D00_on_2D00_reset_2D00_button.pcapng

Seems like I get a no suitable cell even though I had just connected to it a reset earlier.

Here is the log:

devzone.nordicsemi.com/.../nrf9160_2D00_no_2D00_connect_2D00_on_2D00_reset_2D00_button.log

Invoking a soft reset seems to fix things (kernel reboot warm/cold command). Hitting the reset button may or may not get the device into a good state.

I was thinking it was possibly an initialization issue but I reworked the code without any improvement.

RE: Invoking button reset does not allow nRF9160 modem connection

Jared — Fri, 01 Mar 2024 14:25:29 GMT

[quote userid="73166" url="~/f/nordic-q-a/108787/invoking-button-reset-does-not-allow-nrf9160-modem-connection/471750"]can you reproduce and do modem trace?[/quote]

I knew you were going to ask for that Hakon. ;) I'll see what I can do. That UART is occupied by HCI UART.

RE: Invoking button reset does not allow nRF9160 modem connection

Jared — Fri, 01 Mar 2024 14:24:47 GMT

[quote userid="5203" url="~/f/nordic-q-a/108787/invoking-button-reset-does-not-allow-nrf9160-modem-connection/471556"]By the way: battery or USB powered? [/quote]

USB

RE: Invoking button reset does not allow nRF9160 modem connection

Hakon — Fri, 01 Mar 2024 10:21:05 GMT

Hello,

can you reproduce and do modem trace?

RE: Invoking button reset does not allow nRF9160 modem connection

Achim Kraus — Thu, 29 Feb 2024 14:15:19 GMT

Hm ... maybe you also try to report the network reject codes?

Maybe the network rejects the modem, if it tries to register again too fast. But that should be then the same as switching it off/on also just a couple of seconds after connect.

Anyway, maybe the reject code helps the find the cause.

RE: Invoking button reset does not allow nRF9160 modem connection

Achim Kraus — Thu, 29 Feb 2024 13:59:13 GMT

By the way: battery or USB powered?

RE: Invoking button reset does not allow nRF9160 modem connection

Achim Kraus — Thu, 29 Feb 2024 13:58:13 GMT

[quote userid="3604" url="~/f/nordic-q-a/108787/invoking-button-reset-does-not-allow-nrf9160-modem-connection/471525"]Are you initiating an lte_lc_connect() near boot time?[/quote]

At least, I assume so. I initialize my stack, some peripheral, then the modem and then "lte_lc_connect()" (Edited: it's about 5s uptime, when I "connect").

[quote userid="3604" url="~/f/nordic-q-a/108787/invoking-button-reset-does-not-allow-nrf9160-modem-connection/471525"]the RST button is noisy enough[/quote]

Hm, that may be the cause. I usually don't use the feather's RST button too often, so maybe, if it's more frequently used, it will be noisy. From other HW I'm used to have a internal filter on the reset line. So, yes, an answer from the silicon/hardware engineers is needed.

RE: Invoking button reset does not allow nRF9160 modem connection

Jared — Thu, 29 Feb 2024 12:37:21 GMT

Thanks Achim Kraus

This is something I've seen on and off previously. One of my other customers just "discovered" it again and it threw me for a loop.

Ultimately I'll probably need some feedback from the silicon/hardware engineers at Nordic about a resolution. My board does not have the RC filter on the reset line like the DK. My best guess is that the RST button is noisy enough to put the MFW in a weird state.

Interesting to hear your V5 doesn't have the problem though! Are you initiating an lte_lc_connect() near boot time?

RE: Invoking button reset does not allow nRF9160 modem connection

Achim Kraus — Thu, 29 Feb 2024 08:42:37 GMT

Not sure, what happens on your side. I've tested it right now with a nRF9160-DK and a (your) nRF9160 feather v5, both mfw 1.3.5 and I don't get a difference.

What my app does is saving the "settings" after the first successful connect by a "CFUN=0/CFUN=1" cycle. that speeds up a follow on start, either POR, reset-button or reboot.