nRF9151 cell ping-pong prevents eDRX/PSM power savings on Verizon LTE-M

Hardware: nRF9151 DK
Modem firmware: mfw_nrf91x1_2.0.4
Carrier: Verizon (PLMN 311480), Band 13
Antenna: External

Problem
The modem continuously reselects between two cells (same tower, same TAC), causing RRC connections every 20-60 seconds. This prevents PSM from activating and negates eDRX power savings. The 67µA idle baseline confirms eDRX is working between spikes, but the frequent cell reselection causes 40mA spikes lasting ~6 seconds.

Diagnostic Data

AT+CSCON=1 shows CSCON:1 (connected) during every spike.

  CEREG shows the modem alternating between two cells:
  +CEREG: 1,"2C03","00ABEF01",7,,,"00100011","00010111"
  +CEREG: 1,"2C03","00ABEF03",7,,,"00100011","00010111"

  Both cells share TAC 2C03, so intra-TA reselection should not require TAU.

  %XMONITOR output:
  %XMONITOR: 1,"Verizon
  Wireless","VZW","311480","2C03",7,13,"00ABEF01",324,5230,51,22,"1001","11100000","11100000","01011110"

  eDRX is granted:
  +CEDRXRDP: 4,"1001","1001","0001"

  PSM is granted but never activates (T3324=3min, cell reselection resets it):
  T3324: "00100011" (3 min), T3412: "00010111" (230 min)

What I've Tried

  - eDRX only (cycle "1001") — spikes persist
  - PSM (requested T3324=2s, network granted 3min) — never enters PSM
  - AT%XDATAPRFL=1 — no change
  - Disabled eDRX and PSM — same spike pattern
  - NB-IoT instead of LTE-M — worse
  - AT%XRAI=3 — not yet tested on device

Questions

  1. Is there a way to increase cell reselection hysteresis via AT%XEMPR
     or other command to prevent ping-pong between same-tower sectors?
  2. Are there modem firmware improvements planned for this scenario?
  3. Any other recommendations for reducing idle power when the modem is
     at a cell boundary?

Parents
  • Hello,

    Do you have a modem trace showing that issue by chance?

    Best regards,

    Michal

  • Hi Michal,

    After a modem trace, it is clear the spikes are caused by a failing TCP connection and its retransmissions. 

    1. Device attaches, eDRX is granted (+CEDRXP: 4,"1001","1001","0001")
    2. RRC released at t=5.6s — device enters idle, baseline current drops to ~67µA (eDRX working)
    3. At t=105s, the NCS PDN library autonomously creates a second PDN context (CID 1, APN "teal") via AT%XNEWCID? /
      AT+CGDCONT=1,"IP","teal" / AT+CGACT=1,1
    4. TCP SYN to 3.137.58.135:9443 (EC2 instance in us-east-2)
    5. TLS handshake begins but server sends FIN,ACK immediately after Client Key Exchange — connection rejected
    6. TCP retransmissions of the FIN trigger repeated Service Requests → full RRC connection establishment → 40mA spikes
      every 10-30 seconds

    This happens even with our application-level AWS FOTA module disabled (CONFIG_SM_AWS_FOTA=n), because the underlying CONFIG_AWS_FOTA=y, CONFIG_AWS_IOT=y, and CONFIG_FOTA_DOWNLOAD=y remain active in prj.conf.

    1. What NCS library initiates this PDN creation and TCP connection to 3.137.58.135:9443? We do not recognize this server and have nothing configured in us-east-2.

    2. Is this from the aws_fota or fota_download library auto-initializing independently of our application code?

    3. How can we prevent this autonomous connection when no FOTA job is pending?

    Modem trace excerpt (NCS v3.2.0, nRF9151 DK, Verizon LTE-M, Teal SIM):

    "No.","Time","Source","Destination","Protocol","Length","Info"
      "18","14.845734","","","AT","23","Sent AT Command: AT+CEDRXRDP"
      "19","14.845856","","","AT","51","Rcvd AT Command: +CEDRXRDP: 4,""1001"",""1001"",""0001""  OK  "
      "20","68.580017","","","LTE RRC BCCH_BCH","28","MasterInformationBlock (SFN=108)"
      "21","105.726227","","","AT","23","Sent AT Command: AT%XNEWCID?"
      "22","105.726379","","","AT","29","Rcvd AT Command: %XNEWCID: 1  OK  "
      "23","105.726593","","","AT","23","Sent AT Command: AT+CGEREP=1"
      "24","105.726623","","","AT","16","Rcvd AT Command: OK  "
      "25","105.726776","","","AT","36","Sent AT Command: AT+CGDCONT=1,""IP"",""teal"""
      "26","105.726929","","","AT","16","Rcvd AT Command: OK  "
      "27","105.727051","","","AT","24","Sent AT Command: AT+CGACT=1,1"
      "28","105.727173","","","AT","16","Rcvd AT Command: OK  "
      "29","105.727264","","","AT","35","Rcvd AT Command: +CGEV: ME PDN ACT 1,0  "
      "30","105.727386","","","AT","26","Sent AT Command: AT%XGETPDNID=1"
      "31","105.727509","","","AT","31","Rcvd AT Command: %XGETPDNID: 0  OK  "
      "32","105.727631","","","AT","23","Sent AT Command: AT+CGDCONT?"
      "33","105.727844","","","AT","106","Rcvd AT Command: +CGDCONT: 0,""IP"",""teal"",""192.168.3.196"",0,0  +CGDCONT:
      1,""IP"",""teal"",""192.168.3.196"",0,0  OK  "
      "34","105.729065","192.168.3.196","3.137.58.135","TCP","56","54778  >  9443 [SYN] Seq=0 Win=6372 Len=0 MSS=708"
      "50","106.597198","3.137.58.135","192.168.3.196","TCP","52","[TCP ACKed unseen segment] 9443  >  54778 [ACK] Seq=1
      Ack=2090293419 Win=26000 Len=0"
      "51","106.597290","192.168.3.196","3.137.58.135","TCP","52","54778  >  9443 [RST, ACK] Seq=2090293419 Ack=1 Win=58392
      Len=0"
      "52","106.597961","192.168.3.196","3.137.58.135","TCP","56","[TCP Port numbers reused] 54778  >  9443 [SYN] Seq=0
      Win=6372 Len=0 MSS=708"
      "53","106.855164","3.137.58.135","192.168.3.196","TCP","56","9443  >  54778 [SYN, ACK] Seq=0 Ack=1 Win=26883 Len=0
      MSS=1368"
      "55","106.892700","192.168.3.196","3.137.58.135","TLSv1.2","117","Client Hello"
      "57","107.199219","3.137.58.135","192.168.3.196","TLSv1.2","115","Server Hello, Server Hello Done"
      "58","107.295074","192.168.3.196","3.137.58.135","TLSv1.2","153","Client Key Exchange"
      "60","107.612946","192.168.3.196","3.137.58.135","TLSv1.2","143","Change Cipher Spec, Encrypted Handshake Message"
      "62","107.954407","3.137.58.135","192.168.3.196","TCP","52","9443  >  54778 [FIN, ACK] Seq=64 Ack=258 Win=26883 Len=0"
      "64","113.327454","","","LTE RRC DL_DCCH","35","RRCConnectionRelease [cause=other]"

    The server at 3.137.58.135:9443 (EC2 us-east-2) accepts the TCP/TLS handshake but immediately sends FIN,ACK. Subsequent TCP retransmissions trigger Service Requests and full RRC connection establishment every 10-30 seconds —
    causing the 40mA power spikes.

    This connection is initiated autonomously by the NCS stack (not our application code). CONFIG_SM_AWS_FOTA is disabled but CONFIG_AWS_FOTA=y and CONFIG_FOTA_DOWNLOAD=y remain active in the build.

    It is also worth noting that us-east-2 is not a region used by our company, and the Serial Modem firmware has no application logic beyond AT+CFUN=1 and the NCS libraries mentioned. The connection to 3.137.58.135:9443 is not initiated by our application code. This was confirmed by disabling our AWS FOTA module (CONFIG_SM_AWS_FOTA=n) — the connection still occurs because CONFIG_AWS_FOTA=y, CONFIG_AWS_IOT=y, and CONFIG_FOTA_DOWNLOAD=y remain active in the build configuration.

    1. What NCS library initiates this PDN creation and TCP connection to 3.137.58.135:9443? We do not recognize this server and have nothing configured in us-east-2.

    2. Is this from the aws_fota or fota_download library auto-initializing independently of application code?

    3. How can we prevent this autonomous connection when no FOTA job is pending?

    Edit: It appears port 9443 is typically used for GSMA Remote SIM Provisioning (ES9+ / SM-DP+), so I have contacted Teal support to see if this could be a SIM application error. Any workarounds or suggestions are still useful.

Reply
  • Hi Michal,

    After a modem trace, it is clear the spikes are caused by a failing TCP connection and its retransmissions. 

    1. Device attaches, eDRX is granted (+CEDRXP: 4,"1001","1001","0001")
    2. RRC released at t=5.6s — device enters idle, baseline current drops to ~67µA (eDRX working)
    3. At t=105s, the NCS PDN library autonomously creates a second PDN context (CID 1, APN "teal") via AT%XNEWCID? /
      AT+CGDCONT=1,"IP","teal" / AT+CGACT=1,1
    4. TCP SYN to 3.137.58.135:9443 (EC2 instance in us-east-2)
    5. TLS handshake begins but server sends FIN,ACK immediately after Client Key Exchange — connection rejected
    6. TCP retransmissions of the FIN trigger repeated Service Requests → full RRC connection establishment → 40mA spikes
      every 10-30 seconds

    This happens even with our application-level AWS FOTA module disabled (CONFIG_SM_AWS_FOTA=n), because the underlying CONFIG_AWS_FOTA=y, CONFIG_AWS_IOT=y, and CONFIG_FOTA_DOWNLOAD=y remain active in prj.conf.

    1. What NCS library initiates this PDN creation and TCP connection to 3.137.58.135:9443? We do not recognize this server and have nothing configured in us-east-2.

    2. Is this from the aws_fota or fota_download library auto-initializing independently of our application code?

    3. How can we prevent this autonomous connection when no FOTA job is pending?

    Modem trace excerpt (NCS v3.2.0, nRF9151 DK, Verizon LTE-M, Teal SIM):

    "No.","Time","Source","Destination","Protocol","Length","Info"
      "18","14.845734","","","AT","23","Sent AT Command: AT+CEDRXRDP"
      "19","14.845856","","","AT","51","Rcvd AT Command: +CEDRXRDP: 4,""1001"",""1001"",""0001""  OK  "
      "20","68.580017","","","LTE RRC BCCH_BCH","28","MasterInformationBlock (SFN=108)"
      "21","105.726227","","","AT","23","Sent AT Command: AT%XNEWCID?"
      "22","105.726379","","","AT","29","Rcvd AT Command: %XNEWCID: 1  OK  "
      "23","105.726593","","","AT","23","Sent AT Command: AT+CGEREP=1"
      "24","105.726623","","","AT","16","Rcvd AT Command: OK  "
      "25","105.726776","","","AT","36","Sent AT Command: AT+CGDCONT=1,""IP"",""teal"""
      "26","105.726929","","","AT","16","Rcvd AT Command: OK  "
      "27","105.727051","","","AT","24","Sent AT Command: AT+CGACT=1,1"
      "28","105.727173","","","AT","16","Rcvd AT Command: OK  "
      "29","105.727264","","","AT","35","Rcvd AT Command: +CGEV: ME PDN ACT 1,0  "
      "30","105.727386","","","AT","26","Sent AT Command: AT%XGETPDNID=1"
      "31","105.727509","","","AT","31","Rcvd AT Command: %XGETPDNID: 0  OK  "
      "32","105.727631","","","AT","23","Sent AT Command: AT+CGDCONT?"
      "33","105.727844","","","AT","106","Rcvd AT Command: +CGDCONT: 0,""IP"",""teal"",""192.168.3.196"",0,0  +CGDCONT:
      1,""IP"",""teal"",""192.168.3.196"",0,0  OK  "
      "34","105.729065","192.168.3.196","3.137.58.135","TCP","56","54778  >  9443 [SYN] Seq=0 Win=6372 Len=0 MSS=708"
      "50","106.597198","3.137.58.135","192.168.3.196","TCP","52","[TCP ACKed unseen segment] 9443  >  54778 [ACK] Seq=1
      Ack=2090293419 Win=26000 Len=0"
      "51","106.597290","192.168.3.196","3.137.58.135","TCP","52","54778  >  9443 [RST, ACK] Seq=2090293419 Ack=1 Win=58392
      Len=0"
      "52","106.597961","192.168.3.196","3.137.58.135","TCP","56","[TCP Port numbers reused] 54778  >  9443 [SYN] Seq=0
      Win=6372 Len=0 MSS=708"
      "53","106.855164","3.137.58.135","192.168.3.196","TCP","56","9443  >  54778 [SYN, ACK] Seq=0 Ack=1 Win=26883 Len=0
      MSS=1368"
      "55","106.892700","192.168.3.196","3.137.58.135","TLSv1.2","117","Client Hello"
      "57","107.199219","3.137.58.135","192.168.3.196","TLSv1.2","115","Server Hello, Server Hello Done"
      "58","107.295074","192.168.3.196","3.137.58.135","TLSv1.2","153","Client Key Exchange"
      "60","107.612946","192.168.3.196","3.137.58.135","TLSv1.2","143","Change Cipher Spec, Encrypted Handshake Message"
      "62","107.954407","3.137.58.135","192.168.3.196","TCP","52","9443  >  54778 [FIN, ACK] Seq=64 Ack=258 Win=26883 Len=0"
      "64","113.327454","","","LTE RRC DL_DCCH","35","RRCConnectionRelease [cause=other]"

    The server at 3.137.58.135:9443 (EC2 us-east-2) accepts the TCP/TLS handshake but immediately sends FIN,ACK. Subsequent TCP retransmissions trigger Service Requests and full RRC connection establishment every 10-30 seconds —
    causing the 40mA power spikes.

    This connection is initiated autonomously by the NCS stack (not our application code). CONFIG_SM_AWS_FOTA is disabled but CONFIG_AWS_FOTA=y and CONFIG_FOTA_DOWNLOAD=y remain active in the build.

    It is also worth noting that us-east-2 is not a region used by our company, and the Serial Modem firmware has no application logic beyond AT+CFUN=1 and the NCS libraries mentioned. The connection to 3.137.58.135:9443 is not initiated by our application code. This was confirmed by disabling our AWS FOTA module (CONFIG_SM_AWS_FOTA=n) — the connection still occurs because CONFIG_AWS_FOTA=y, CONFIG_AWS_IOT=y, and CONFIG_FOTA_DOWNLOAD=y remain active in the build configuration.

    1. What NCS library initiates this PDN creation and TCP connection to 3.137.58.135:9443? We do not recognize this server and have nothing configured in us-east-2.

    2. Is this from the aws_fota or fota_download library auto-initializing independently of application code?

    3. How can we prevent this autonomous connection when no FOTA job is pending?

    Edit: It appears port 9443 is typically used for GSMA Remote SIM Provisioning (ES9+ / SM-DP+), so I have contacted Teal support to see if this could be a SIM application error. Any workarounds or suggestions are still useful.

Children
  • Thank you for the information.

    I will need some time to dig deeper into it, so I have to ask for some patience now. I hope that is okay.

    Best regards,

    Michal

  • Hi Nathan,

    I'm sorry for the delay, Michal is not in the office this week, So I'll be taking this ticket over.

    nvaniman said:

    1. What NCS library initiates this PDN creation and TCP connection to 3.137.58.135:9443? We do not recognize this server and have nothing configured in us-east-2.

    2. Is this from the aws_fota or fota_download library auto-initializing independently of our application code?

    To be completely honest, I have no idea what is causing that connection here. If you provide us the raw modem traces that you gathered, we might be able to check in a bit more detail what is happening.

    nvaniman said:
    3. How can we prevent this autonomous connection when no FOTA job is pending?

    You need to connect to the FOTA service regularly, because it needs to check if there is a pending FOTA job or not. Usually, the FOTA service does not ping the device to request it to do a FOTA update. It simply raises a flag that indicate the device when it connects to the service that a new update is available.

    nvaniman said:

    Edit: It appears port 9443 is typically used for GSMA Remote SIM Provisioning (ES9+ / SM-DP+), so I have contacted Teal support to see if this could be a SIM application error. Any workarounds or suggestions are still useful.

    Can you maybe try with another SIM card provider to see if this solves / changes the issue? But, please, keep us updated on that.

    Best regards,

    Simon D-M

Related