No connection to nRF cloud (Attempts to use AGPS)

[Stas Jis]

We are using nRF9160 (Development Kit), SDK 1.9.1 board with "serial lte modem" firmware.
We are trying to get AGPS coordinates (using an external antenna), but for some reason we stopped receiving confirmations about the connection to the nRF cloud.
In response to the command "AT#XNRFCLOUD=1" we always get only "OK" without (#XNRFCLOUD: 1,0).
Checking the connection to the cloud shows that there is no connection:

AT#XNRFCLOUD?

#XNRFCLOUD: 0,0,16842753,"nrf-352656108600480"

The last time during development we checked the command sequence for getting coordinates using AGPS about a month ago and everything worked fine (was tested on SDK 1.8). But now even on SDK 1.8 we cannot get coordinates using AGPS, only regular GPS works.
AT commands: 

2022-03-31T12:15:16.966Z DEBUG modem << Ready
2022-03-31T12:15:18.043Z DEBUG modem << [00:00:00.467,926] [0m<inf> slm: Serial LTE Modem[0m
2022-03-31T12:15:18.050Z DEBUG modem << [00:00:00.500,030] [0m<inf> slm_at_host: at_host init done[0m
2022-03-31T12:15:25.049Z DEBUG modem >> AT
2022-03-31T12:15:25.059Z DEBUG modem << OK
2022-03-31T12:15:25.873Z DEBUG modem >> AT+CFUN?
2022-03-31T12:15:25.886Z DEBUG modem << +CFUN: 0
2022-03-31T12:15:25.894Z DEBUG modem << OK
2022-03-31T12:15:45.149Z DEBUG modem << *** Booting Zephyr OS build v2.7.99-ncs1-1  ***
2022-03-31T12:15:46.145Z DEBUG modem << Ready
2022-03-31T12:15:47.126Z DEBUG modem << [00:00:01.176,116] [0m<inf> slm: Serial LTE Modem[0m
2022-03-31T12:15:47.135Z DEBUG modem << [00:00:01.208,160] [0m<inf> slm_at_host: at_host init done[0m
2022-03-31T12:16:27.274Z DEBUG modem >> AT
2022-03-31T12:16:27.285Z DEBUG modem << OK
2022-03-31T12:16:28.249Z DEBUG modem >> AT+CFUN?
2022-03-31T12:16:28.265Z DEBUG modem << +CFUN: 0
2022-03-31T12:16:28.272Z DEBUG modem << OK
2022-03-31T12:16:34.122Z DEBUG modem << *** Booting Zephyr OS build v2.7.99-ncs1-1  ***
2022-03-31T12:16:34.439Z DEBUG modem << Ready
2022-03-31T12:16:35.420Z DEBUG modem << [00:00:00.498,748] [0m<inf> slm: Serial LTE Modem[0m
2022-03-31T12:16:35.429Z DEBUG modem << [00:00:00.530,822] [0m<inf> slm_at_host: at_host init done[0m
2022-03-31T12:16:41.915Z DEBUG modem << *** Booting Zephyr OS build v2.7.99-ncs1-1  ***
2022-03-31T12:16:42.651Z DEBUG modem << Ready
2022-03-31T12:16:43.632Z DEBUG modem << [00:00:00.915,985] [0m<inf> slm: Serial LTE Modem[0m
2022-03-31T12:16:43.639Z DEBUG modem << [00:00:00.948,059] [0m<inf> slm_at_host: at_host init done[0m
2022-03-31T12:16:56.482Z DEBUG modem >> AT%XSYSTEMMODE=1,0,1,0
2022-03-31T12:16:56.498Z DEBUG modem << OK
2022-03-31T12:17:01.946Z DEBUG modem >> AT%XMAGPIO=1,0,0,1,1,1574,1577
2022-03-31T12:17:01.960Z DEBUG modem << OK
2022-03-31T12:17:06.921Z DEBUG modem >> AT%XCOEX0
2022-03-31T12:17:06.932Z DEBUG modem << OK
2022-03-31T12:17:11.401Z DEBUG modem >> AT+CEREG=5
2022-03-31T12:17:11.414Z DEBUG modem << OK
2022-03-31T12:17:16.289Z DEBUG modem >> AT+CPSMS=1
2022-03-31T12:17:16.302Z DEBUG modem << OK
2022-03-31T12:17:21.488Z DEBUG modem >> AT+CFUN=1
2022-03-31T12:17:21.534Z DEBUG modem << OK
2022-03-31T12:17:23.561Z DEBUG modem << +CEREG: 2,"026F","00332520",7
2022-03-31T12:17:25.178Z DEBUG modem << +CEREG: 5,"026F","00332520",7,,,"00011110","11100000"
2022-03-31T12:17:28.768Z DEBUG modem >> AT#XNRFCLOUD=1
2022-03-31T12:17:28.776Z DEBUG modem << OK
2022-03-31T12:17:36.432Z DEBUG modem >> AT#XAGPS=1,1
2022-03-31T12:17:36.439Z DEBUG modem << ERROR
2022-03-31T12:17:36.440Z ERROR Error: AT#XAGPS=1,1
 failed
2022-03-31T12:17:42.194Z DEBUG modem >> AT#XNRFCLOUD?
2022-03-31T12:17:42.206Z DEBUG modem << #XNRFCLOUD: 0,0,16842753,"nrf-352656108600480"
2022-03-31T12:17:42.208Z DEBUG modem << OK

prj.conf:

#
# Copyright (c) 2020 Nordic Semiconductor ASA
#
# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
#
# General config
CONFIG_LOG=y
CONFIG_LOG_DEFAULT_LEVEL=3
CONFIG_STACK_SENTINEL=y
CONFIG_NEWLIB_LIBC=y
CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y
CONFIG_RING_BUFFER=y

# Segger RTT
CONFIG_USE_SEGGER_RTT=y
CONFIG_RTT_CONSOLE=y
CONFIG_UART_CONSOLE=y
CONFIG_LOG_BACKEND_RTT=n
CONFIG_LOG_BACKEND_UART=y

# Network
CONFIG_NETWORKING=y
CONFIG_NET_SOCKETS=y
CONFIG_NET_NATIVE=n

# Modem library
CONFIG_NRF_MODEM_LIB=y
# Align the max FD entry to NRF_MODEM_MAX_SOCKET_COUNT(8)
CONFIG_POSIX_MAX_FDS=8
# Enable below for modem trace
#CONFIG_NRF_MODEM_LIB_TRACE_ENABLED=y

# Use GPIO
CONFIG_GPIO=y
CONFIG_GPIO_NRFX=y

# UART interface
CONFIG_SERIAL=y
CONFIG_UART_ASYNC_API=y
CONFIG_NRFX_TIMER2=y

# Stacks and heaps
CONFIG_MAIN_STACK_SIZE=4096
CONFIG_HEAP_MEM_POOL_SIZE=16384
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=4096

# Device power management
CONFIG_PM_DEVICE=y

# FOTA
CONFIG_HTTP_PARSER_URL=y
CONFIG_FOTA_DOWNLOAD=y
CONFIG_FOTA_DOWNLOAD_PROGRESS_EVT=y
CONFIG_DFU_TARGET=y
CONFIG_DOWNLOAD_CLIENT=y
CONFIG_DOWNLOAD_CLIENT_STACK_SIZE=4096
CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_IMG_MANAGER=y

# FLASH
CONFIG_FLASH=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FLASH_MAP=y
CONFIG_MPU_ALLOW_FLASH_WRITE=y
CONFIG_IMG_ERASE_PROGRESSIVELY=y

# Settings
CONFIG_SETTINGS=y
CONFIG_SETTINGS_FCB=y
CONFIG_FCB=y

# nRF Cloud
CONFIG_CLOUD_API=y
CONFIG_NRF_CLOUD_MQTT=y
CONFIG_NRF_CLOUD_FOTA=n
CONFIG_NRF_CLOUD_AGPS=y
CONFIG_NRF_CLOUD_PGPS=y
CONFIG_NRF_CLOUD_CELL_POS=y
CONFIG_NRF_CLOUD_LOG_LEVEL_INF=y
CONFIG_DATE_TIME=y
CONFIG_MODEM_INFO=y
CONFIG_MODEM_INFO_ADD_DATE_TIME=n
CONFIG_AT_MONITOR=y
CONFIG_MODEM_JWT=y
# Use "nrf-<IMEI>" as the default device_id format
# Use "<device_uuid>" as the device_id format by below
#CONFIG_NRF_CLOUD_CLIENT_ID_SRC_INTERNAL_UUID=y

#
# SLM-specific configurations
#
CONFIG_SLM_CUSTOMIZED=n
CONFIG_SLM_LOG_LEVEL_INF=y
CONFIG_SLM_EXTERNAL_XTAL=n
CONFIG_SLM_START_SLEEP=n
#\nUse\nUART_0\n(when\nworking\nwith\nPC\nterminal)
CONFIG_UART_0_NRF_HW_ASYNC_TIMER=2
#\nUse\nUART_2\n(when\nworking\nwith\nexternal\nMCU)
#CONFIG_SLM_CONNECT_UART_2=y
#CONFIG_UART_2_NRF_HW_ASYNC_TIMER=2
#\nnRF\nCloud\nbased\nlocation\nservices
CONFIG_SLM_AGPS=y
CONFIG_SLM_PGPS=y
CONFIG_SLM_CELL_POS=y
#CONFIG_SLM_LOG_LEVEL_DBG=y

[Stas Jis]

justin,
Have you changed or deleted the credentials in the modem sec_tag 16842753?

No, we didn't do anything like that.
I am attaching the results of the AT%CMNG=1 command:

2022-04-05T06:55:56.140Z DEBUG modem >> AT%CMNG=1
2022-04-05T06:55:56.155Z DEBUG modem << %CMNG: 0,0,"0000000000000000000000000000000000000000000000000000000000000000"
2022-04-05T06:55:56.169Z DEBUG modem << %CMNG: 0,1,"0101010101010101010101010101010101010101010101010101010101010101"
2022-04-05T06:55:56.186Z DEBUG modem << %CMNG: 0,2,"0202020202020202020202020202020202020202020202020202020202020202"
2022-04-05T06:55:56.195Z DEBUG modem << %CMNG: 0,6,"0606060606060606060606060606060606060606060606060606060606060606"
2022-04-05T06:55:56.205Z DEBUG modem << %CMNG: 123,0,"0000000000000000000000000000000000000000000000000000000000000000"
2022-04-05T06:55:56.215Z DEBUG modem << %CMNG: 123,1,"0101010101010101010101010101010101010101010101010101010101010101"
2022-04-05T06:55:56.222Z DEBUG modem << %CMNG: 123,2,"0202020202020202020202020202020202020202020202020202020202020202"
2022-04-05T06:55:56.233Z DEBUG modem << %CMNG: 321,0,"0000000000000000000000000000000000000000000000000000000000000000"
2022-04-05T06:55:56.241Z DEBUG modem << %CMNG: 321,1,"0101010101010101010101010101010101010101010101010101010101010101"
2022-04-05T06:55:56.250Z DEBUG modem << %CMNG: 321,2,"0202020202020202020202020202020202020202020202020202020202020202"
2022-04-05T06:55:56.254Z DEBUG modem << %CMNG: 325,0,"0000000000000000000000000000000000000000000000000000000000000000"
2022-04-05T06:55:56.268Z DEBUG modem << %CMNG: 325,1,"0101010101010101010101010101010101010101010101010101010101010101"
2022-04-05T06:55:56.276Z DEBUG modem << %CMNG: 325,2,"0202020202020202020202020202020202020202020202020202020202020202"
2022-04-05T06:55:56.283Z DEBUG modem << %CMNG: 696,0,"0000000000000000000000000000000000000000000000000000000000000000"
2022-04-05T06:55:56.291Z DEBUG modem << %CMNG: 696,1,"0101010101010101010101010101010101010101010101010101010101010101"
2022-04-05T06:55:56.299Z DEBUG modem << %CMNG: 696,2,"0202020202020202020202020202020202020202020202020202020202020202"
2022-04-05T06:55:56.313Z DEBUG modem << %CMNG: 757,0,"0000000000000000000000000000000000000000000000000000000000000000"
2022-04-05T06:55:56.320Z DEBUG modem << %CMNG: 757,1,"0101010101010101010101010101010101010101010101010101010101010101"
2022-04-05T06:55:56.325Z DEBUG modem << %CMNG: 757,2,"0202020202020202020202020202020202020202020202020202020202020202"
2022-04-05T06:55:56.334Z DEBUG modem << %CMNG: 1234,0,"0000000000000000000000000000000000000000000000000000000000000000"
2022-04-05T06:55:56.339Z DEBUG modem << %CMNG: 1234,1,"0101010101010101010101010101010101010101010101010101010101010101"
2022-04-05T06:55:56.346Z DEBUG modem << %CMNG: 1234,2,"0202020202020202020202020202020202020202020202020202020202020202"
2022-04-05T06:55:56.363Z DEBUG modem << %CMNG: 1684,0,"0000000000000000000000000000000000000000000000000000000000000000"
2022-04-05T06:55:56.370Z DEBUG modem << %CMNG: 1684,1,"0101010101010101010101010101010101010101010101010101010101010101"
2022-04-05T06:55:56.379Z DEBUG modem << %CMNG: 1684,2,"0202020202020202020202020202020202020202020202020202020202020202"
2022-04-05T06:55:56.385Z DEBUG modem << %CMNG: 16842753,0,"0000000000000000000000000000000000000000000000000000000000000000"
2022-04-05T06:55:56.389Z DEBUG modem << %CMNG: 16842753,1,"0101010101010101010101010101010101010101010101010101010101010101"
2022-04-05T06:55:56.397Z DEBUG modem << %CMNG: 16842753,2,"0202020202020202020202020202020202020202020202020202020202020202"
2022-04-05T06:55:56.403Z DEBUG modem << %CMNG: 4294967293,10,"0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A"
2022-04-05T06:55:56.417Z DEBUG modem << %CMNG: 4294967292,11,"0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B"
2022-04-05T06:55:56.423Z DEBUG modem << OK

[Stas Jis]

justin,

Have you changed or deleted the credentials in the modem sec_tag 16842753?

The problem may be related to the fact that before that we installed our certificates to communicate with the Amazon MQTT Broker (in this evaluation board)?
Used commands:
AT%CMNG=0,321,0,-----BEGIN CERTIFICATE-----\n"....
AT%CMNG=0,321,1,-----BEGIN CERTIFICATE-----\n" ...
AT%CMNG=0,321,2,-----BEGIN EC PRIVATE KEY-----\n" ...

[justin]

the modem sec tags allow you to have credentials for multiple servers. 
so if you used 321 for AWS IoT and kept 16842753 for nrf cloud, there shouldn't be a problem. 

did you use sec tag 321 when you added your device to nrf cloud?  if so, rebuild your serial LTE modem application with:
CONFIG_NRF_CLOUD_SEC_TAG=321

If that isn't the solution, deleting the device from nrf cloud and re-provisioning with the correct credentials should work.

[Stas Jis]

justin,
did you use sec tag 321 when you added your device to nrf cloud?  if so, rebuild your serial LTE modem application with:

No, I'm having problems registering the device (gives error 40101).
This board was previously registered by our former employee, and we were unable to access his account. Can we somehow transfer the device to my profile?

[Stas Jis]

justin:

How can we return the default certificates for nRF Cloud with the tag (16842753 )?