nRF9160 closes with `RPC_IP_ERR_NOT_CONN` reason during MQTT datamode upload

[ben]

I am using the nRF9160 to upload a large set of data in chunk of 16kB in MQTT datamode. I am using the v2.6.1 SDK and toolchain. During the MQTT upload the connection seems to close due to an RPC_IP_ERR_NOT_CONN error. I cannot find any documentation on this error or trace back to where or why this occurs. This seems to occur at random times during the upload, but seldom occurs during shorter uploads. 

Here is the log 

[00:11:18.353,454] <inf> slm_at_host: Exit datamode
[00:11:20.024,719] <inf> nrf_modem: RPC_IP_CLOSED_NTF fd 0x0
[00:11:20.031,005] <inf> nrf_modem: Close reason RPC_IP_ERR_NOT_CONN
[00:11:20.039,001] <inf> nrf_modem: recv() fd 0x0, buf 0x2001bbdd, len 2, flags 0x40 (non-block)
[00:11:20.048,889] <err> net_mqtt_rx: [CID 0x20018228]: Connection closed.
[00:11:20.056,823] <inf> net_mqtt_sock_tls: Closing socket 0
[00:11:20.063,537] <inf> nrf_modem: close() fd 0x0
[00:11:20.070,312] <inf> nrf_modem: RPC_IP_CLOSE_RES fd 0x0, result RPC_IP_ERR_OK
[00:11:20.079,193] <err> slm_mqtt: ERROR: mqtt_input -128
[00:11:20.085,510] <inf> slm_mqtt: MQTT thread terminated
[00:11:41.184,387] <err> slm_at_host: AT command buffer overflow, 9588 dropped
[00:12:02.831,420] <err> slm_at_host: AT command buffer overflow, 16021 dropped
[00:12:24.480,255] <err> slm_at_host: AT command buffer overflow, 16021 dropped
[00:12:46.127,807] <err> slm_at_host: AT command buffer overflow, 16021 dropped
[00:13:07.776,062] <err> slm_at_host: AT command buffer overflow, 16021 dropped
[00:13:29.423,645] <err> slm_at_host: AT command buffer overflow, 16021 dropped
[00:13:54.020,538] <inf> slm: Power off triggered.
ASSERTION FAIL [!arch_is_in_isr()] @ WEST_TOPDIR/zephyr/kernel/mutex.c:110
        mutexes cannot be used inside ISRs
[00:13:54.036,010] <err> os: r0/a1:  0x00000004  r1/a2:  0x0000006e  r2/a3:  0x00000000
[00:13:54.044,677] <err> os: r3/a4:  0x200184c8 r12/ip:  0x00044257 r14/lr:  0x0003748f
[00:13:54.053,344] <err> os:  xpsr:  0x41000041
[00:13:54.058,532] <err> os: Faulting instruction address (r15/pc): 0x00043b54
[00:13:54.066,436] <err> os: >>> ZEPHYR FATAL ERROR 4: Kernel panic on CPU 0
[00:13:54.074,127] <err> os: Fault during interrupt handling

[00:13:54.080,566] <err> os: Current thread: 0x20017d90 (idle)

[Hakon]

Hello,

can you capture and upload modem trace from this?

[ben]

Hello Hakon,

I am still working on trying to get a trace of this to you. It seems it occurs randomly, so it is hard to capture when we do not disconnect.

In the meantime, do you have any documentation on some of these error messages or could point me to where we may find it? I understand these are from the nRF modem and namely I would like to figure out what these messages, RPC_IP_ERR_NOT_CONN, RPC_IP_RECVFROM_NTF, and RPC_IP_SEND_RES, mean. 

If you have any insight on RPC_IP_ERR_NOT_CONN, this would be most helpful as this seems to be our main issue for our random disconnects.

Thanks

Ben

[Hakon]

There are internally error codes in the modem library, which is not public information unfortunately.

[emdi]

Those errors are only prints inthe log, thus they don't directly affect the program execution as such. What you should check are the actual return values of the socket functions that you are using and the `errno` values they set.

Generally, what that error means is that the TCP peer has closed the connection (they have sent FIN). That is signalled in different ways, depending on which socket function you use.

`recv()` will probably exit with 0, indicating that the peer has closed the connection.