Unable to connect to AT&T after upgrading MFW to v1.1.0

We had been using an AT&T M1-enabled SIM in one of our development units successfully for quite some time.  After we upgraded that node to MFW v1.1.0, it is no longer able to connect to the cell network.  We tried swapping SIMs around between devices with different firmware versions confirmed that any node still running MFW v1.0.1 is able to connect to the AT&T network using this SIM, and any node running MFW v1.1.0 is not.  I put the SIM in one of our v1.1.0 nodes, loaded up at_client, and turned on as much output as I could.  The relevant error when trying to connect to the network is this:

+CNEC_ESM: 33,0
+CNEC_EMM: 19
+CEREG: 2,"FFFE","FFFFFFFF",7,0,19,"11100000","11100000"
+CEREG: 2,"0411","00F63F11",7,0,19,"11100000","11100000"

It gets the CEREG failure almost immediately and then just sits there doing nothing.  (We waited 15+ minutes)

Again, this failure is using an official AT&T SIM that returns 2 to the AT%XOPERID command.

We are able to connect to the same AT&T cells using v1.1.0 if we put in an MVNO SIM (hologram).

What is the modem doing differently with and without the official AT&T SIM?  Whatever it is doing, it's breaking something in our testing.

  • Hello.  I do not have a solution but I do have a potentially useful additional data point.

    We are also using official AT&T SIMs in southeast USA and modem fw V1.1.0 seems to have broken our ability to connect as well.

    After having problems with our app, we simplified to using the link monitor and stock at_client. 

    What we saw in all cases is modem fw V1.0.1 getting a complete connection in < 10 seconds while V1.1.0 never connects.

    Below is with V1.1.0 (note we get:+CNEC_ESM and +CNEC_EMM messages):

    SPM: NS reset vector at 0xde89
    SPM: prepare to jump to Non-Secure image.
    ***** Booting Zephyr OS build v2.0.99-ncs1 *****
    The AT host sample started
    AT+CFUN=1
    OK
    AT+CFUN?
    +CFUN: 1
    OK
    AT+CGSN=1
    +CGSN: "352656100816191"
    OK
    AT+CGMI
    Nordic Semiconductor ASA
    OK
    AT+CGMM
    nRF9160-SICA
    OK
    AT+CGMR
    mfw_nrf9160_1.1.0
    OK
    AT+CEMODE?
    +CEMODE: 2
    OK
    AT%XCBAND=?
    %XCBAND: (1,2,3,4,5,8,12,13,14,17,18,19,20,25,26,28,66)
    OK
    AT+CMEE?
    +CMEE: 0
    OK
    AT+CMEE=1
    OK
    AT+CNEC?
    +CNEC: 0
    OK
    AT+CNEC=24
    OK
    AT+CGEREP?
    +CGEREP: 0,0
    OK
    AT+CGDCONT?
    OK
    AT+CGACT?
    OK
    AT+CGEREP=1
    OK
    AT+CIND=1,1,1
    OK
    AT+CEREG=2
    OK
    AT+CEREG?
    +CEREG: 2,4,"FFFE","FFFFFFFF",7
    OK
    AT%CESQ=1
    OK
    AT+CESQ
    +CESQ: 99,99,255,255,255,255
    OK
    AT%XSIM=1
    OK
    AT%XSIM?
    %XSIM: 1
    OK
    AT+CPIN?
    +CPIN: READY
    OK
    AT+CPINR="SIM PIN"
    +CPINR: "SIM PIN",3
    OK
    AT+CIMI
    204080813511271
    OK
    AT+CESQ
    +CESQ: 99,99,255,255,255,255
    OK
    AT+CPINR="SIM PIN"
    %CESQ: 43,2,10,1
    +CEREG: 2,"9103","0236CE03",7
    +CPINR: "SIM PIN",3
    OK
    AT+CIMI
    204080813511271
    OK
    +CNEC_ESM: 38,0
    +CNEC_EMM: 19
    %CESQ: 38,1,10,1
    %CESQ: 255,0,255,0
    +CEREG: 2,"FFFE","FFFFFFFF",7
    %CESQ: 255,0,255,0
    AT+CESQ
    +CESQ: 99,99,255,255,255,255
    OK
    AT+CFUN?
    +CFUN: 1
    <snip>
    OK
    AT+CESQ
    +CESQ: 99,99,255,255,255,255
    OK
    AT+CESQ
    +CESQ: 99,99,255,255,255,255
    OK
    %CESQ: 31,1,4,0
    +CEREG: 2,"3319","0ABA3210",7
    +CNEC_EMM: 11
    %CESQ: 255,0,255,0
    +CEREG: 2,"FFFE","FFFFFFFF",7
    %CESQ: 34,1,9,1
    +CEREG: 2,"3319","0ABA3210",7
    AT+CESQ
    +CESQ: 99,99,255,255,9,34
    OK
    +CNEC_ESM: 38,0
    +CNEC_EMM: 19
    %CESQ: 255,0,255,0
    +CEREG: 2,"FFFE","FFFFFFFF",7
    %CESQ: 31,1,7,1
    +CEREG: 2,"3319","0ABA3210",7
    AT+CESQ
    +CESQ: 99,99,255,255,7,31
    OK
    AT+CESQ
    +CESQ: 99,99,255,255,7,31
    OK
    

    Here is with V1.1.0

    SPM: NS reset vector at 0xde89
    SPM: prepare to jump to Non-Secure image.
    ***** Booting Zephyr OS build v2.0.99-ncs1 *****
    The AT host sample started
    AT+CFUN=1
    OK
    AT+CFUN?
    +CFUN: 1
    OK
    AT+CGSN=1
    +CGSN: "352656100816191"
    OK
    AT+CGMI
    Nordic Semiconductor ASA
    OK
    AT+CGMM
    nRF9160-SICA
    OK
    AT+CGMR
    mfw_nrf9160_1.0.1
    OK
    AT+CEMODE?
    +CEMODE: 2
    OK
    AT%XCBAND=?
    %XCBAND: (1,2,3,4,5,8,12,13,14,17,18,19,20,25,26,28,66)
    OK
    AT+CMEE?
    +CMEE: 0
    OK
    AT+CMEE=1
    OK
    AT+CNEC?
    +CNEC: 0
    OK
    AT+CNEC=24
    OK
    AT+CGEREP?
    +CGEREP: 0,0
    OK
    AT+CGDCONT?
    +CGDCONT: 0,"IP","m2m64.com.attz","10.71.183.29",0,0
    OK
    AT+CGACT?
    +CGACT: 0,1
    OK
    AT+CGEREP=1
    OK
    AT+CIND=1,1,1
    OK
    AT+CEREG=2
    OK
    AT+CESQ
    +CESQ: 99,99,255,255,255,34
    OK
    AT+CEREG?
    +CEREG: 2,1,"3319","0ABB1011",7
    OK
    AT+COPS=3,2
    OK
    AT+COPS?
    +COPS: 0,2,"310410",7
    OK
    AT%XCBAND
    %XCBAND: 17
    OK
    AT+CGDCONT?
    +CGDCONT: 0,"IP","m2m64.com.attz","10.71.183.29",0,0
    OK
    AT+CGACT?
    +CGACT: 0,1
    OK
    AT%CESQ=1
    OK
    AT+CESQ
    +CESQ: 99,99,255,255,255,34
    OK
    AT%XSIM=1
    OK
    AT%XSIM?
    %XSIM: 1
    OK
    AT+CPIN?
    +CPIN: READY
    OK
    AT+CPINR="SIM PIN"
    +CPINR: "SIM PIN",3
    OK
    AT+CIMI
    310170833560137
    OK
    

    I can open another ticket if requested but since it seems like it may be the same issue I'm starting here.  Please advise if there is anything we can do to help troubleshoot this.  Thank you in advance for any and all support. 

  • Here is the modem trace when trying to connect with the official AT&T SIM card.

    trace-2019-11-27T14-58-29.822Z.bin

    Here is a timeline of the CEREG notifications we get when trying to connect.

    [00:00:04.069,335] <inf> CEREG: 0,"0411","00F63F11",7,0,0,"11100000","11100000"
    [00:00:04.849,426] <inf> CEREG: 2,"0411","00F63F11",7,0,0,"11100000","11100000"
    [00:00:05.767,028] <inf> CEREG: 2,"0411","00F63F11",7,0,19,"11100000","11100000"
    [00:00:07.032,684] <inf> CEREG: 2,"FFFE","FFFFFFFF",7,0,19,"11100000","11100000"
    [00:00:10.472,930] <inf> CEREG: 2,"3B02","00E6C102",7,0,19,"11100000","11100000"
    [00:00:11.676,757] <inf> CEREG: 2,"3B02","00E6C102",7,0,11,"11100000","11100000"
    [00:00:13.051,727] <inf> CEREG: 2,"FFFE","FFFFFFFF",7,0,11,"11100000","11100000"
    [00:00:14.315,734] <inf> CEREG: 2,"0411","00F63F11",7,0,11,"11100000","11100000"
    [00:00:15.202,514] <inf> CEREG: 2,"0411","00F63F11",7,0,19,"11100000","11100000"
    [00:00:16.484,100] <inf> CEREG: 2,"FFFE","FFFFFFFF",7,0,19,"11100000","11100000"
    [00:00:17.506,408] <inf> CEREG: 2,"0411","00F63F11",7,0,19,"11100000","11100000"

  • Hello.  I have additional data again. 

    I found that I can connect with modem 1.1.0 if I use a hologram SIM but not with an ATT SIM.  It may be useful to note that I've seen the link monitor say the hologram sim roamed successfully on both verizon and ATT.

    Below are modem traces and console output of modem 1.0.1 working with the ATT SIM, 1.1.0 not working with the ATT SIM and 1.1.0 working with the hologram SIM.

    modem101_att_sim_connects_in_10_secs.bin

    modem110_att_sim_no_connect.bin

    modem110_hologram_sim_connects_in_15secs.bin

    modem110_att_sim_no_connect_console_log.txt
    SPM: NS image at 0xc000
    SPM: NS MSP at 0x2002beb8
    SPM: NS reset vector at 0x13155
    SPM: prepare to jump to Non-Secure image.
    [00:00:00.267,974]
    [1;31m<err> i2c_nrfx_twim: Error 195952642 occurred for message 0
    [0m
    ***** Booting Zephyr OS build v2.0.99-ncs1 *****
    *****************
    CX1751 Demo App Started
    *****************
    BME280 bound successfully
    APDS9960 bound successfully
    ACC DT LBL:=MMA8451QFXOS8700 bound successfully
    ACC DT LBL:=VL53L0XVL53L0X bound successfully
    GPS initialized
    Setting up the Battery Monitor succeeded.
    [00:00:00.650,268]
    [0m<dbg> lte_lc.w_lte_lc_connect: Network mode: AT%XSYSTEMMODE=1,0,1,0
    [0m
    %CESQ: 38,1,17,2
    +CEREG: 2,"3319","0ABA3210",7,0,0,"11100000","11100000"
    [00:00:02.294,189]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 2
    [0m
    +CEREG: 2,"3319","0ABA3210",7,0,19,"11100000","11100000"
    [00:00:03.624,267]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 2
    [0m
    %CESQ: 255,0,255,0
    +CEREG: 2,"FFFE","FFFFFFFF",7,0,19,"11100000","11100000"
    [00:00:04.897,949]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 2
    [0m
    %CESQ: 25,1,0,0
    +CEREG: 2,"3319","0ABB1011",7,0,19,"11100000","11100000"
    [00:00:06.126,983]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 2
    [0m
    %CESQ: 12,0,20,2
    %CESQ: 26,1,3,0
    %CESQ: 10,0,22,3
    AT+CFUN?
    +CFUN: 1
    OK
    AT+CGSN=1
    +CGSN: "352656100816191"
    OK
    AT+CGMI
    Nordic Semiconductor ASA
    OK
    AT+CGMM
    nRF9160-SICA
    OK
    AT+CGMR
    mfw_nrf9160_1.1.0
    OK
    AT+CEMODE?
    +CEMODE: 2
    OK
    AT%XCBAND=?
    %XCBAND: (1,2,3,4,5,8,12,13,14,17,18,19,20,25,26,28,66)
    OK
    AT+CMEE?
    +CMEE: 0
    OK
    AT+CMEE=1
    OK
    AT+CNEC?
    +CNEC: 0
    OK
    AT+CNEC=24
    %CESQ: 24,1,4,0
    OK
    AT+CGEREP?
    +CGEREP: 0,0
    OK
    AT+CGDCONT?
    +CGDCONT: 0,"IP","","",0,0
    OK
    AT+CGACT?
    +CGACT: 0,0
    OK
    AT+CGEREP=1
    OK
    AT+CIND=1,1,1
    OK
    AT+CEREG=2
    OK
    AT+CEREG?
    +CEREG: 2,2,"3319","0ABB1011",7
    OK
    AT%CESQ=1
    OK
    AT+CESQ
    +CESQ: 99,99,255,255,4,24
    OK
    AT%XSIM=1
    OK
    AT%XSIM?
    %XSIM: 1
    OK
    AT+CPIN?
    +CPIN: READY
    OK
    AT+CPINR="SIM PIN"
    +CPINR: "SIM PIN",3
    OK
    AT+CIMI
    310170833560137
    OK
    %CESQ: 29,1,4,0
    %CESQ: 12,0,20,2
    %CESQ: 25,1,3,0
    %CESQ: 12,0,20,2
    %CESQ: 25,1,3,0
    %CESQ: 10,0,22,3
    %CESQ: 23,1,4,0
    
    modem_110_hologram_sim_connects_console_log.txt
    SPM: NS MSP at 0x2002beb8
    SPM: NS reset vector at 0x13155
    SPM: prepare to jump to Non-Secure image.
    [00:00:01.638,458]
    [1;31m<err> i2c_nrfx_twim: Error 195952642 occurred for message 0
    [0m
    ***** Booting Zephyr OS build v2.0.99-ncs1 *****
    *****************
    CX1751 Demo App Started
    *****************
    BME280 bound successfully
    APDS9960 bound successfully
    ACC DT LBL:=MMA8451QFXOS8700 bound successfully
    ACC DT LBL:=VL53L0XVL53L0X bound successfully
    GPS initialized
    Setting up the Battery Monitor succeeded.
    [00:00:02.020,690]
    [0m<dbg> lte_lc.w_lte_lc_connect: Network mode: AT%XSYSTEMMODE=1,0,1,0
    [0m
    %CESQ: 38,1,14,2
    +CEREG: 2,"3319","0ABA3210",7,0,0,"11100000","11100000"
    [00:01:33.780,731]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 2
    [0m
    +CEREG: 2,"3319","0ABA3210",7,0,11,"11100000","11100000"
    [00:01:33.886,535]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 2
    [0m
    %CESQ: 38,1,9,1
    %CESQ: 255,0,255,0
    +CEREG: 2,"FFFE","FFFFFFFF",7,0,11,"11100000","11100000"
    [00:01:35.160,308]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 2
    [0m
    %CESQ: 43,2,15,2
    +CEREG: 2,"9103","0237AC02",7,0,11,"11100000","11100000"
    [00:01:36.318,298]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 2
    [0m
    %CESQ: 255,0,255,0
    +CEREG: 2,"FFFE","FFFFFFFF",7,0,11,"11100000","11100000"
    [00:01:38.919,555]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 2
    [0m
    %CESQ: 38,1,15,2
    +CEREG: 2,"3319","0ABA3210",7,0,11,"11100000","11100000"
    [00:01:39.518,463]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 2
    [0m
    %CESQ: 38,1,10,1
    %CESQ: 33,1,10,1
    +CEREG: 5,"3319","0ABA3210",7,,,"11100000","11100000"
    [00:01:42.517,456]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 5
    [0m
    >>APP EVT: LTE Link Connection succeeded!
    AT+COPS=3,2
    OK
    AT+COPS?
    +COPS: 0,2,"310410",7
    OK
    AT%XCBAND
    %XCBAND: 17
    OK
    AT+CGDCONT?
    %CESQ: 33,1,5,0
    +CGDCONT: 0,"IP","hologram","10.184.142.60",0,0
    OK
    AT+CGACT?
    +CGACT: 0,1
    OK
    IPv4 Address found 3.220.217.199
    
    AT+CFUN?
    +CFUN: 1
    OK
    AT+CGSN=1
    +CGSN: "352656100816191"
    OK
    AT+CGMI
    Nordic Semiconductor ASA
    OK
    AT+CGMM
    nRF9160-SICA
    OK
    AT+CGMR
    mfw_nrf9160_1.1.0
    OK
    AT+CEMODE?
    +CEMODE: 2
    OK
    AT%XCBAND=?
    %XCBAND: (1,2,3,4,5,8,12,13,14,17,18,19,20,25,26,28,66)
    OK
    AT+CMEE?
    +CMEE: 1
    OK
    AT+CNEC?
    +CNEC: 24
    OK
    AT+CGEREP?
    +CGEREP: 1,0
    OK
    AT+CIND=1,1,1
    OK
    AT+CEREG=2
    OK
    AT+CEREG?
    +CEREG: 2,5,"3319","0ABA3210",7
    OK
    AT+COPS=3,2
    OK
    AT+COPS?
    +COPS: 0,2,"310410",7
    OK
    AT%XCBAND
    %XCBAND: 17
    OK
    AT+CGDCONT?
    +CGDCONT: 0,"IP","hologram","10.184.142.60",0,0
    OK
    AT+CGACT?
    +CGACT: 0,1
    OK
    AT%CESQ=1
    OK
    AT+CESQ
    +CESQ: 99,99,255,255,15,33
    
    modem101_att_sim_console_log.txt
    SPM: NS MSP at 0x2002beb8
    SPM: NS reset vector at 0x13155
    SPM: prepare to jump to Non-Secure image.
    [00:00:00.263,793]
    [1;31m<err> i2c_nrfx_twim: Error 195952642 occurred for message 0
    [0m
    ***** Booting Zephyr OS build v2.0.99-ncs1 *****
    *****************
    CX1751 Demo App Started
    *****************
    BME280 bound successfully
    APDS9960 bound successfully
    ACC DT LBL:=MMA8451QFXOS8700 bound successfully
    ACC DT LBL:=VL53L0XVL53L0X bound successfully
    GPS initialized
    Setting up the Battery Monitor succeeded.
    [00:00:00.618,408]
    [0m<dbg> lte_lc.w_lte_lc_connect: Network mode: AT%XSYSTEMMODE=1,0,1,0
    [0m
    %CESQ: 26,1
    +CEREG: 2,"3319","0ABB1011",7,0,0,"11100000","11100000"
    [00:00:02.532,379]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 2
    [0m
    +CEREG: 1,"3319","0ABB1011",7,,,"11100000","11100000"
    [00:00:04.841,522]
    [0m<dbg> lte_lc.parse_nw_reg_status: Network registration status: 1
    [0m
    >>APP EVT: LTE Link Connection succeeded!
    IPv4 Address found 3.220.217.199
    client_username: iH3iQ1aPOLlUizn1Zb6C, size: 21
    >>APP EVT: MQTT Broker Connection Succeeded!
    +CEREG: 1,"3319","0ABA3210",7,,,"11100000","11100000"
    

Related