• State Not Answered
  • Replies 8 replies
  • Subscribers 76 subscribers
  • Views 424 views
  • Users 0 members are here
Attachments (0)
Nordic Case Info
  • Case ID: 346627
Options

nRF9151 stuck in disconnected, unable to reconnect to LTE-M

teba99

Hi,

We are testing connection resilience on our nRF9151 product. Generally, LTE-M connectivity (Germany, on Telefonica and/or Telekom) works fine.

However, under certain circumstances the nRF9151 appears unable to reconnect to the network. The only way to get it reconnect is with a full reboot.

One situation is when the SIM is briefly inactive for whatever reason (for example, an outage on the carrier side or because the network assignments change and the SIM is reset). The nRF9151 will loose connectivity immediately (as expected) and will try to connect later, but fails forever with "<err> modem_info_params: Link data not obtained: 5 -5". Only a reboot resolves it, after which the nRF9151 will immediately reconnect (no change on the SIM/network side).

In the same situation, we've also seen the modem actually re-attach to the network (as seen from the signallng logs) but the nRF9151 network stack apparently fails to establish a PDN session repeatedly with "nrf_cloud_transport: Could not connect to nRF Cloud MQTT Broker mqtt.nrfcloud.com, port: 45858. err: -111". Here also after a reboot it immediately reconnects. 

I understand the FPLMN List may come into play here, but I've also tested this by adding a new LTE-M network to the SIM that the device has not seen since the reboot, so it should not be in the FPLMN. But the nRF9151 is still unable to connect even to that new network (until a reboot).

My interpretation is that once the nRF9151 has disconnected it gets into a state where some part of the stack (modem, network stack, link manager) is stuck and fails to reconnect even though it could (SIM is active and suitable networks are available). A reboot resolves that state.

Unfortunately, a reboot is not a good option for us because we would then loose the data queued in RAM.  

Note: We are using the nRF9151 with the latest modem FW and nRF SDK, the application is based on the multi service sample.

Is this a known issue? Can I provide more information to help you analyse it? Do you have recommendations to deal with this situation?

Thanks much!

  • 0 in reply to teba99

    I'm also facing the exact same issue, have you managed to get any work around for this?

    best regards,

  • 0

    Hello Akshay,

    I kindly recommend creating a separate ticket for your issue, as each case is handled independently. Once you've created the new ticket, please share the relevant details there, similar to what I've requested in this conversation.

    Kind regards,
    Abhijith

  • 0 in reply to teba99

    Hi  ,

    I was able to do some more testing and create modem traces.

    I reproduced the problem with the nRF SDK 2.9.0 Modem Shell application on a nRF9161-DK. This means the problem is not related to our hardware or application, but must be somewhere in the nRF software stack.

    I used the Modem Shell unmodified, just built it with CONFIG_MOSH_CLOUD_MQTT=y and the nrf91-modem-trace-uart snippet.

    Here is the trace in the serial window. At ">>>" you see my comments interleaved.

    You can see that after the loss of the connection (poor signal or brief deactivation of SIM) the modem is in auto-connect mode and sees networks, but doesn't try to reconnect. It just 'sits there'. I have experience with other modems (e.g. Quectel) and in such cases the modem would automatically try to reconnect.

    When I 'kick' the nRF9161 (cfun=4, then cfun=1) it immediately reestablishes the connection.

    To me this indicates that after connection loss the modem gets stuck in some state where it fails to try and auto-connect, and that state is only cleared with a modem soft reset.

    I've very similar reports from other developers: https://community.circuitdojo.com/d/224-reconnecting-lte-after-disconnect.

    >>> Reboot:

    Booting TF-M v2.1.1-ncs3

    [Sec Thread] Secure image initializing!

    TF-M isolation level is: 0x00000001

    TF-M Float ABI: Hard

    Lazy stacking enabled

    *** Booting nRF Connect SDK v2.9.0-7787b2649840 ***

    *** Using Zephyr OS v3.7.99-1f8f3dc29142 ***

    Reset reason: PIN reset

    mosh:~$

    MOSH version:       v2.9.0-7787b2649840

    MOSH build id:      custom

    MOSH build variant: dev

    HW version:         nRF9161 LACA ADA

    Modem FW version:   mfw_nrf91x1_2.0.0

    Modem FW UUID:      c234531b-9f14-4135-9092-f4e52d37fb23

    > at AT+CGMR

    at AT+CGMR

    mfw_nrf91x1_2.0.0

    OK

    mosh:~$ > at AT+CFUN=1

    at AT+CFUN=1

    OK

    mosh:~$ > at AT%XCONNSTAT=1

    at AT%XCONNSTAT=1

    OK

    mosh:~$ > at AT%XCONNSTAT?

    at AT%XCONNSTAT?

    %XCONNSTAT: 0,0,0,0,0,0

    OK

    mosh:~$ > at AT+CMEE=1

    at AT+CMEE=1

    OK

    mosh:~$ > at AT%XSIM=1

    at AT%XSIM=1

    OK

    mosh:~$ > at AT+CEREG=5

    at AT+CEREG=5

    OK

    mosh:~$ > at AT%MDMEV=1

    at AT%MDMEV=1

    OK

    mosh:~$ > at AT%CESQ=1

    at AT%CESQ=1

    OK

    mosh:~$ > at AT+CGSN=1

    at AT+CGSN=1

    +CGSN: "352696950002997"

    OK

    mosh:~$ > at AT+CGMM

    at AT+CGMM

    nRF9161-LACA

    OK

    mosh:~$ > at AT+CEMODE?

    at AT+CEMODE?

    +CEMODE: 2

    OK

    mosh:~$ > at AT+CMEE?

    at AT+CMEE?

    +CMEE: 1

    OK

    mosh:~$ > at AT+CNEC?

    at AT+CNEC?

    +CNEC: 16

    OK

    mosh:~$ > at AT+CGEREP?

    at AT+CGEREP?

    +CGEREP: 1,0

    OK

    mosh:~$ at A> at AT+CIND=1,1,1

    T+CIND=1,1,1

    OK

    mosh:~$ > at AT+COPS=3,2

    at AT+COPS=3,2

    OK

    mosh:~$ > at AT+CGDCONT?

    at AT+CGDCONT?

    OK

    mosh:~$ > at AT+CGACT?

    at AT+CGACT?

    OK

    mosh:~$ > at AT+CESQ

    at AT+CESQ

    +CESQ: 99,99,255,255,255,255

    OK

    mosh:~$ > at AT%XSIM?

    at AT%XSIM?

    %XSIM: 0

    OK

    mosh:~$ > at AT+CPIN?

    at AT+CPIN?

    +CME ERROR: 13

    mosh:~$ > at AT+CIMI

    at AT+CIMI

    +CME ERROR: 0

    mosh:~$ > at AT+CNEC=24

    at AT+CNEC=24

    OK

    mosh:~$ > at AT+CEER

    at AT+CEER

    OK

    mosh:~$ > at AT+CGEREP=1

    at AT+CGEREP=1

    OK

    mosh:~$ > at AT%XPOFWARN=1,30

    at AT%XPOFWARN=1,30

    OK

    mosh:~$ > at AT%XVBATLVL=1

    at AT%XVBATLVL=1

    OK

    mosh:~$ > at AT+CGMI

    at AT+CGMI

    Nordic Semiconductor ASA

    OK

    mosh:~$ > at AT+CGMR

    at AT+CGMR

    mfw_nrf91x1_2.0.0

    OK

    mosh:~$ > at AT+CGSN

    at AT+CGSN

    352696950002997

    OK

    mosh:~$ > at AT%XMODEMUUID

    at AT%XMODEMUUID

    %XMODEMUUID: c234531b-9f14-4135-9092-f4e52d37fb23

    OK

    mosh:~$ > at AT%XDATAPRFL?

    at AT%XDATAPRFL?

    %XDATAPRFL: 2

    OK

    mosh:~$ > at AT+CEREG?

    at AT+CEREG?

    +CEREG: 5,0

    OK

    mosh:~$ > at AT+CFUN?

    at AT+CFUN?

    +CFUN: 1

    OK

    mosh:~$ > at AT%CESQ=1

    at AT%CESQ=1

    OK

    mosh:~$ > at AT+CSCON=1

    at AT+CSCON=1

    OK

    mosh:~$ > at AT+CSCON?

    at AT+CSCON?

    +CSCON: 1,0

    OK

    mosh:~$ > at AT+CPAS

    at AT+CPAS

    +CPAS: 0

    OK

    mosh:~$ > at AT+CEDRXRDP

    at AT+CEDRXRDP

    +CEDRXRDP: 0

    OK

    mosh:~$ > at AT%XTIME=1

    at AT%XTIME=1

    OK

    mosh:~$ > at AT%CONEVAL

    at AT%CONEVAL

    %CONEVAL: 1

    OK

    mosh:~$ > at AT%XCBAND=?

    at AT%XCBAND=?

    %XCBAND: (1,2,3,4,5,8,12,13,18,19,20,25,26,28,66,85)

    OK

    mosh:~$ > at AT%XCBAND

    at AT%XCBAND

    %XCBAND: 0

    OK

    mosh:~$ > at AT%HWVERSION

    at AT%HWVERSION

    %HWVERSION: nRF9161 LACA ADA

    OK

    mosh:~$ > at AT%XMODEMTRACE=1,2

    at AT%XMODEMTRACE=1,2

    OK

    mosh:~$ > at AT%XSYSTEMMODE?

    at AT%XSYSTEMMODE?

    %XSYSTEMMODE: 1,0,0,0

    OK

    mosh:~$ > at AT%XICCID

    at AT%XICCID

    %XICCID: 89457300000010597907

    OK

    mosh:~$ > at AT%XEMPR?

    at AT%XEMPR?

    %XEMPR:

    OK

    mosh:~$ at AT+CGDC> at AT+CGDCONT?

    ONT?

    OK

    mosh:~$ > at AT+COPS?

    at AT+COPS?

    +COPS: 0

    OK

    mosh:~$ > at AT%XMONITOR

    at AT%XMONITOR

    %XMONITOR: 0

    OK

    mosh:~$ > at AT+CPINR="SIM PIN"

    at AT+CPINR="SIM PIN"

    +CPINR: "SIM PIN",3

    OK

    mosh:~$ > at AT+CPINR="SIM PIN2"

    at AT+CPINR="SIM PIN2"

    +CPINR: "SIM PIN2",3

    OK

    mosh:~$ > at AT+CPINR="SIM PUK"

    at AT+CPINR="SIM PUK"

    +CPINR: "SIM PUK",10

    OK

    mosh:~$ > at AT+CPINR="SIM PUK2"

    at AT+CPINR="SIM PUK2"

    +CPINR: "SIM PUK2",10

    OK

    mosh:~$ > at AT%XSIM?

    at AT%XSIM?

    %XSIM: 1

    OK

    Network registration status: searching

    Currently active system mode: LTE-M

    LTE cell changed: ID: 45508877, Tracking area: 26552

    RRC mode: Connected

    PDN event: PDP context 0 activated

    PDN event: PDP context 0, PDN type IPv4 only allowed

    Modem domain event: Search done

    Network registration status: Connected - roaming

    PSM parameter update: TAU: 3240, Active time: -1 seconds

    Modem config for system mode: LTE-M

    Modem config for LTE preference: No preference, automatically selected by the modem

    Currently active system mode: LTE-M

    Battery voltage:       5042 mV

    Modem temperature:     27 C

    Device ID:             nrf-352696950002997

    Operator full name:   ""

    Operator short name:  ""

    Operator PLMN:        "26201"

    Current cell id:       45508877 (0x02B6690D)

    Current phy cell id:   337

    Current band:          3

    Current TAC:           26552 (0x67B8)

    Current rsrp:          37: -104dBm

    Current snr:           29: 5dB

    Mobile network time and date: 25/06/27,04:58:30+08

    PDP context info 1:

      CID:                0

      PDN ID:             0

      PDP context active: yes

      PDP type:           IP

      APN:                onomondo

      IPv4 MTU:           1360

      IPv4 address:       100.76.21.49

      IPv6 address:       ::

      IPv4 DNS address:   172.30.8.5, 172.30.8.6

      IPv6 DNS address:   ::, ::

    RRC mode: Idle

    >>> connection established, everything OK. Establish MQTT cloud connection:

    mosh:~$ > cloud connect

    cloud connect

    Endpoint: mqtt.nrfcloud.com

    Connecting to nRF Cloud...

    nRF Cloud event: NRF_CLOUD_EVT_TRANSPORT_CONNECTING

    RRC mode: Connected

    [00:00:45.708,557] <inf> nrf_cloud_info: Device ID: nrf-352696950002997

    [00:00:45.717,956] <inf> nrf_cloud_info: IMEI:      352696950002997

    [00:00:45.810,791] <inf> nrf_cloud_info: UUID:      50363953-3234-40cb-80ae-1e21b10ad7e4

    [00:00:45.816,925] <inf> nrf_cloud_info: Modem FW:  mfw_nrf91x1_2.0.0

    [00:00:45.816,955] <inf> nrf_cloud_info: Protocol:          MQTT

    [00:00:45.816,986] <inf> nrf_cloud_info: Download protocol: HTTPS

    [00:00:45.817,016] <inf> nrf_cloud_info: Sec tag:           16842753

    [00:00:45.817,047] <inf> nrf_cloud_info: Host name:         mqtt.nrfcloud.com

    nRF Cloud event: NRF_CLOUD_EVT_TRANSPORT_CONNECTED

    nRF Cloud event: NRF_CLOUD_EVT_RX_DATA_SHADOW

    nRF Cloud event: NRF_CLOUD_EVT_USER_ASSOCIATED

    [00:00:49.732,299] <inf> nrf_cloud_log: Changing cloud log level from:1 to:3

    [00:00:49.732,330] <inf> nrf_cloud_log: Changing cloud logging enabled to:1

    nRF Cloud event: NRF_CLOUD_EVT_READY

    Connection to nRF Cloud established

    [00:00:51.133,270] <inf> nrf_cloud_info: Team ID:   be46c77d-xxx

    RRC mode: Idle

    >>> Attached network info

    mosh:~$ > at at+cops?

    at at+cops?

    +COPS: 0,2,"26201",7

    OK

    mosh:~$ > at at+cereg?

    at at+cereg?

    +CEREG: 5,5,"67B8","02B6690D",7,,,"11100000","11100000"

    OK

    RRC mode: Connected

    >>> Now: Loss of connectivity:

    nRF Cloud event: NRF_CLOUD_EVT_TRANSPORT_DISCONNECTED, status: 3

    Network registration status: not registered

    Connection to nRF Cloud disconnected

    PDN event: PDP context 0 network detach

    Reconnecting in 10 seconds...

    RRC mode: Idle

    [00:01:53.207,977] <wrn> lte_lc: Registration rejected, EMM cause: 7, Cell ID: 45508877, Tracking area: 26552, LTE mode: 7

    Connecting to nRF Cloud...

    nRF Cloud event: NRF_CLOUD_EVT_TRANSPORT_CONNECTING

    nRF Cloud event: NRF_CLOUD_EVT_TRANSPORT_CONNECT_ERROR, status: -9

    Connecting to nRF Cloud failed

    [00:02:03.257,019] <err> nrf_cloud_transport: Could not connect to nRF Cloud MQTT Broker mqtt.nrfcloud.com, port: 45858. err: -111

    >>> Modem sees networks, is in automatic mode, but does not try to reconnect

    mosh:~$ > at at+cereg?

    at at+cereg?

    +CEREG: 5,0

    OK

    mosh:~$ > at at+cops?

    at at+cops?

    +COPS: 0

    OK

    mosh:~$ at at+cops=?

    > at at+cops=?

    +COPS: (1,"","","26203",7),(1,"","","26201",7),(1,"","","26202",7)

    OK

    Currently active system mode: None

    LTE cell changed: ID: -1, Tracking area: -1

    mosh:~$ at AT+CGDC> at AT+CGDCONT?

    ONT?

    OK

    >>> manually force offline mode, then back to normal mode

    mosh:~$ > at at+cfun=4

    at at+cfun=4

    OK

    mosh:~$ > at at+cfun=1

    at at+cfun=1

    OK

    >>> modem reestablishes connection without further input or actions

    Network registration status: searching

    Currently active system mode: LTE-M

    LTE cell changed: ID: 45508865, Tracking area: 26552

    RRC mode: Connected

    PDN event: PDP context 0 activated

    PDN event: PDP context 0, PDN type IPv4 only allowed

    Modem domain event: Search done

    Network registration status: Connected - roaming

    Modem config for system mode: LTE-M

    Modem config for LTE preference: No preference, automatically selected by the modem

    Currently active system mode: LTE-M

    Battery voltage:       5031 mV

    Modem temperature:     28 C

    Device ID:             nrf-352696950002997

    Operator full name:   ""

    Operator short name:  ""

    Operator PLMN:        "26201"

    Current cell id:       45508865 (0x02B66901)

    Current phy cell id:   379

    Current band:          20

    Current TAC:           26552 (0x67B8)

    Current rsrp:          46: -95dBm

    Current snr:           30: 6dB

    Mobile network time and date: 25/06/27,05:02:49+08

    PDP context info 1:

      CID:                0

      PDN ID:             0

      PDP context active: yes

      PDP type:           IP

      APN:                onomondo

      IPv4 MTU:           1360

      IPv4 address:       100.76.21.49

      IPv6 address:       ::

      IPv4 DNS address:   172.30.8.5, 172.30.8.6

      IPv6 DNS address:   ::, ::

    LTE cell changed: ID: 30601474 (0x01D2F102), PCI 164, Tracking area: 26552 (0x67B8), Band: 3, RSRP: 36 (-105dBm), SNR: 29 (5dB)

    >>> Reestablish MQTT cloud connection

    mosh:~$ > cloud connect

    cloud connect

    Endpoint: mqtt.nrfcloud.com

    Connecting to nRF Cloud...

    nRF Cloud event: NRF_CLOUD_EVT_TRANSPORT_CONNECTING

    nRF Cloud event: NRF_CLOUD_EVT_TRANSPORT_CONNECTED

    nRF Cloud event: NRF_CLOUD_EVT_RX_DATA_SHADOW

    nRF Cloud event: NRF_CLOUD_EVT_USER_ASSOCIATED

    nRF Cloud event: NRF_CLOUD_EVT_READY

    Connection to nRF Cloud established

    [00:05:04.917,236] <inf> nrf_cloud_info: Team ID:   be46c77d-e0a8-4ee0-92cc-85525808fc1b

    RRC mode: Idle

    LTE cell changed: ID: 45508877 (0x02B6690D), PCI 337, Tracking area: 26552 (0x67B8), Band: 3, RSRP: 38 (-103dBm), SNR: 31 (7dB)

    mosh:~$

Related
Terms of service