nRF9160 fails to attach after several RESETs & attach without power cycle

[Michal Mühlpachr]

Hi,

nRF9160 with SLM NCS v1.9.1 fails to attach after 7th RESET (by RESET pin) & attach without power cycle.

There is not any ERROR state reported by modem, modem fails to attach forever without power cycle.

After nRF9160 power cycle it attaches immediately with same AT sequence.

We need reliable initialization of nRF9160 SLM/Modem without nRF9160 power cycle because we do not have load power in production HW - issue threatens device stability in the field.

I prepared Python script to reproduce issue with nRF9160DK, you can find it together with log at https://gist.github.com/muhlpachr/40a4d8f1b332aee8a25fa7e1ad023751
Modify COM port for AT commands according your PC setup please.

Issue is 100% reproducible.

Attached you can find modem trace.

Thank you.

Regards,
Michal

trace-2022-03-24T23-42-16.760Z.bin

[Øyvind]

Hello Michal, 

Can you please verify that you are not seeing the nRF9160 Modem Reset Loop Restriction?

Kind regards,
Øyvind

[Øyvind]

Please try with more than 60 seconds between each reset. This is most likely due to the reset loop restriction which will count every reset done within 60 seconds.  

[Michal Mühlpachr]

Hello Øyvind,

yes, it is %MDMEV: RESET LOOP, we were not aware of nRF9160 Modem Reset Loop Restriction functionality.

Is 60 seconds delay between RESETs mentioned anywhere in documentation please ?

Why there is not ERROR reported  by modem while in RESET LOOP after +CFUN=1 ?

Thank you.

Regards,
Michal

[Øyvind]

Hi, 

Michal Mühlpachr said:

Why there is not ERROR reported  by modem while in RESET LOOP after +CFUN=1 ?

I assume that this is due to not subscribing to the modem domain event notification.

Michal Mühlpachr said:

Is 60 seconds delay between RESETs mentioned anywhere in documentation please ?

No, looks like this is not documented from what I can see. 

Kind regards,
Øyvind

[Michal Mühlpachr]

It is clear, that without %MDMEV=1 we do not get URC. But it is strange to get OK for +CFUN=1 while in RESET LOOP.

I just tested to try attach 1h after %MDMEV: RESET LOOP URC, but I am still getting %MDMEV: RESET LOOP. It does not behave according https://infocenter.nordicsemi.com/index.jsp?topic=%2Fref_at_commands%2FREF%2Fat_commands%2Fmob_termination_ctrl_status%2Fmdmev_set.html

Modem was powered whole 1h and RESETed by pin with CONFIG_SLM_START_SLEEP=y in SLM at the beginning of 1h and at the beginning of next try. Remaining time was not stored in NVM, because after power cycle, modem attaches immediately. Why 30 minutes timeout do not apply ?

[Øyvind]

Hello, 
can you please provide a new modem trace and log from the updated scenario? 

Michal Mühlpachr said:

But it is strange to get OK for +CFUN=1 while in RESET LOOP.

Note that changing function of modem is not equivalent to network attach. Do you use AT+CFUN=0 to gracefully shut down modem?

Thanks. 
Kind regards,
Øyvind

[Michal Mühlpachr]

Hi,

with modem trace enabled I am not able to reproduce RESET LOOP after 40m or 1h.
I am able to reproduce it only without modem trace. Sequence is same as below.

But I am able to reproduce, that even with 2minutes delay between RESETs I am still getting RESET LOOP in 7th cycle after power cycle. Modem trace is attached.

Regads,

Michaltrace-2022-03-28T21-28-11.914Z.bin

[Øyvind]

Hello, I have forwarded your issue to our nRF91 team. I hope to have an update by end of today or tomorrow.

Kind regards,
Øyvind

[Øyvind]

Hello, 

Our nRF91 team have look at the issue and respond with following clarification:

• When the reset loop restriction has been activated due to maximum number of resets, that is 7 resets, it will be instantly stored to NVM and after that there is no way to get around it.
  The restriction is active and nothing else can be done other than wait (with the device powered) for 30 minutes. While waiting, if the reset occurs again the timer starts again with the last stored remaining value. Note that NVM will be updated with the remaining value always at CFUN=0. If CFUN=0 has not been commanded and the device resets again, then the restriction timer will start with the initial value again at power up, i.e. the restriction will be active for the next 30 minutes.
• The mentioned 60 seconds timer does reset the internal reset counter. So, if there is no resets during 1 minute the counter will be reset and again 7 new resets are needed for activating the actual reset loop restriction. NOTE: This has no effect to possibly already existing reset loop restriction.
• The reset counter is also always reset at CFUN=0, i.e. when application gracefully switches off the modem. NOTE: This has no effect to possibly already existing reset loop restriction.
• After the reset loop restriction is over, modem will automatically initiate Attach procedure if suitable cell is available. Note that if client triggers reset at this point, the old restriction is still at the NVM and will stand up again at the next startup. Using CFUN=0 instead will update the NVM and remove the old stored restriction.
• Modem events need to be subscribed for receiving notification of reset loop restriction (AT%MDMEV=1). The subscription does not persist over resets nor modem de-initialization.