NB-IoT connection lost while keeping MQTT alive

[men]

Hi,

I'm having issues maintain the connection to a MQTT server. I'm able to establish the connection, keeping this alive and pub/sub some massages, but after some time (~5 min) the client disconnects. After this happens I'm usually unable to immediately establish connection to the server again and I'm unable to make DNS lookup for about 20 sec.

This makes me suspect the connection to the LTE network.

I'm using SDK 0.4.0 and  mfw_nrf9160_0.7.0-29.alpha. I'm connection to Telia Denmark NB-Iot.

Here is a section of my own debugging output, in the square brackets to the left is the time in sec:

[005.220 lte tl2] LTE Link Connecting ...
[018.240 lte tl2] LTE Link Connected!
[018.260 lte tl2] MQTT Link Connecting...
[019.820 lte tl2] IPv4 Address found 0x1f769033
[036.740 lte tl2] OK: mqtt_connect 0
[037.560 lte tl2] [mqtt_evt_handler:235] MQTT client connected!
[037.560 lte tl2] Subscribing to: devices/rsa_test_device_1/messages/devicebound/# len 48
[038.060 lte tl2] [mqtt_evt_handler:282] SUBACK packet id: 1234
...
[414.560 lte tl2] [mqtt_evt_handler:242] MQTT client disconnected -22
[414.570 lte tl2] MQTT Link Connecting...
[414.590 lte tl2] ERROR: getaddrinfo failed 22
[414.590 lte tl2] Broker using 51.144.118.31
[414.610 lte tl3] ERROR: mqtt_connect -114
[414.640 lte tl3] ERROR: mqtt_connect -114
[414.660 lte tl3] ERROR: mqtt_connect -114
[414.690 lte tl3] ERROR: mqtt_connect -114
[414.710 lte tl3] ERROR: mqtt_connect -114
[414.710 lte tl2] FAILED: mqtt_connect -114
[414.730 lte tl2] MQTT Link Connecting...
[414.740 lte tl2] ERROR: getaddrinfo failed 22
[414.750 lte tl2] Broker using 51.144.118.31
[414.770 lte tl3] ERROR: mqtt_connect -114
[414.800 lte tl3] ERROR: mqtt_connect -114
[414.820 lte tl3] ERROR: mqtt_connect -114
[414.850 lte tl3] ERROR: mqtt_connect -114
[414.870 lte tl3] ERROR: mqtt_connect -114
[414.870 lte tl2] FAILED: mqtt_connect -114
...
[433.740 lte tl2] MQTT Link Connecting...
[435.420 lte tl2] IPv4 Address found 0x1f769033
[459.210 lte tl2] OK: mqtt_connect 0

The only thing I'm doing when it fails is keeping the MQTT alive with:  mqtt_live(&client) and mqtt_input(&client);

Do you know if there is any reported issues with NB-Iot and 0.7.0-29.alpha? Or do you have another idea of what is failing?

[jbrzozoski]

That -114 is errno EALREADY.  In my testing w/ LTE-M, the MQTT/socket libraries seemed fragile if I pounded on them too much while there was already an outstanding connection request.  It was easy enough to remember if my device had an outstanding request and hold off if so.  You may want to try the same thing.  Just be sure to keep calling mqtt_input() while waiting for the connection to complete, since it needs to handle connection handshaking.

[men]

Hi jbrzozoski,

Yes I agree, I should be handling the reconnect in a more gentle manner. But the initial problem is that the client gets disconnected from the MQTT server.

I have also tested this with Nordic MQTT sample-code and get the same result, after about 5 min the clients gets disconnected:

The MQTT simple sample started
LTE Link Connecting ...
LTE Link Connected!
IPv4 Address found 0xf11e29c6
[mqtt_evt_handler:166] MQTT client connected!
[mqtt_evt_handler:177] MQTT client disconnected -45
POLLNVAL
Disconnecting MQTT client...
Could not disconnect MQTT client. Error: -57

This is the content of prj.conf:

# General config
CONFIG_TEST_RANDOM_GENERATOR=y

# Networking
CONFIG_NETWORKING=y
CONFIG_NET_SOCKETS_OFFLOAD=y
CONFIG_NET_SOCKETS=y
CONFIG_NET_SOCKETS_POSIX_NAMES=y

# LTE link control
CONFIG_LTE_LINK_CONTROL=y
CONFIG_LTE_AUTO_INIT_AND_CONNECT=n
CONFIG_LTE_NETWORK_MODE_LTE_M=n
CONFIG_LTE_NETWORK_MODE_NBIOT=y
CONFIG_LTE_LEGACY_PCO_MODE=y

# BSD library
CONFIG_BSD_LIBRARY=y

# AT Host
CONFIG_UART_INTERRUPT_DRIVEN=y
CONFIG_AT_HOST_LIBRARY=y

# MQTT
CONFIG_MQTT_LIB=y
CONFIG_MQTT_LIB_TLS=n

# Appliaction
#CONFIG_MQTT_PUB_TOPIC="/my/publish/topic"
#CONFIG_MQTT_SUB_TOPIC="/my/subscribe/topic"
#CONFIG_MQTT_CLIENT_ID="my-client-id"
#CONFIG_MQTT_BROKER_HOSTNAME="iot.eclipse.org"
#CONFIG_MQTT_BROKER_PORT=1883

# Main thread
CONFIG_MAIN_THREAD_PRIORITY=7
CONFIG_MAIN_STACK_SIZE=4096

CONFIG_HEAP_MEM_POOL_SIZE=1024

[Håkon Alseth]

Hi,

 

I agree that there's some cases here that isn't handled well in the application. When getting disconnected, the application will not try to reconnect for instance.

iot.eclipse.org isn't always the most stable broker, as there's alot of traffic going through that broker. Could you try test.mosquitto.org and see if this one also has stability issues?

 

Kind regards,

Håkon

[men]

Hi Håkon,

I've tested with "test.mosquitto.org" but it is the same result, after some time the client gets disconnected.

I suspect the problem leis with the LTE connection. This time I try to get the IP of google.com. At first it is successful and so the connection to "test.mosquitto.org", but when disconnected I'm unable to get IP of google.com:

The MQTT simple sample started
LTE Link Connecting ...
LTE Link Connected!
Testing connection to "google.com"
OK: getaddrinfo success
IPv4 Address found 0x106abb25
[mqtt_evt_handler:166] MQTT client connected!
[mqtt_evt_handler:177] MQTT client disconnected -45
POLLNVAL
Disconnecting MQTT client...
Could not disconnect MQTT client. Error: -57
Testing connection to "google.com"
ERROR: getaddrinfo failed 60

[Håkon Alseth]

Hi,

 

I tested this, by letting it stand there and it disconnected from the broker after a while:

***** Booting Zephyr OS v1.14.99-ncs1 *****
The MQTT simple sample started
LTE Link Connecting ...
LTE Link Connected!
IPv4 Address found 0x106abb25
[mqtt_evt_handler:166] MQTT client connected!
Subscribing to: my/subscribe/topic len 18
[mqtt_evt_handler:216] SUBACK packet id: 1234
[mqtt_evt_handler:172] MQTT client disconnected -45
POLLNVAL
Disconnecting MQTT client...
Could not disconnect MQTT client. Error: -57
IPv4 Address found 0x106abb25

However, it still maintained the network connection, and successfully ran the getaddrinfo() function.

 

I suspect that your device lost the network connection. Could you try with at_client and check your signal strength (last parameter of AT+CESQ)?

It should return:

AT+CESQ
+CESQ: 99,99,255,255,255,46
OK

RSRP = -140 + 46 = -94 dBm signal strength at my end. Not great, but not bad either. 10 to 15 dB away from the receiver sensitivity.

Kind regards,

Håkon

[men]

Hi Håkon,

RSRP on my board is -116 dBm. On the Evalkit RSRP -82 dBm and the connection is not lost. It must be my antenna. :-(

[John]

Can I make a suggestion, I read the iot.eclipse.org will disconnect anyone that has the same Client ID.

Try changing my-client-id to something more obscure, that's potentially unlikely to be used by someone else.

Do the same with the client that is publishing the data as well, but make it different as well.

# Appliaction
#CONFIG_MQTT_PUB_TOPIC="/my/publish/topic"
#CONFIG_MQTT_SUB_TOPIC="/my/subscribe/topic"
#CONFIG_MQTT_CLIENT_ID="my-client-id"
#CONFIG_MQTT_BROKER_HOSTNAME="iot.eclipse.org"
#CONFIG_MQTT_BROKER_PORT=1883

If I'm right you would be dumped the moment someone else comes along and connects to iot.eclipse.org with

#CONFIG_MQTT_CLIENT_ID="my-client-id"

Love t know if I'm correct.

[men]

Hi John,

I dont't think this is the issue, because the same happens with Azure IOT hub where I have a unique client ID.