nRF9160 - sent UDP message do not longer reach destination after 4 weeks

[Achim Kraus]

Running a Thingy:91 now for 4 weeks from batter with one message exchange per hour, it stopped working (NB-IoT mode).

The client (Thingy:91) uses "sendto" and that function returns success.

The client's log:

[18:01:38.586,822] <inf> COAP_CLIENT: CoAP request prepared, token 0xfbe72850, 139 bytes
[18:01:38.594,146] <inf> COAP_CLIENT: send_to_peer 175
[18:01:38.618,927] <inf> COAP_CLIENT: LTE modem wakes up
[18:01:40.208,496] <inf> COAP_CLIENT: RRC mode: Connected
[18:01:40.594,757] <inf> COAP_CLIENT: 1/1/729-1621 ms: connected => sent
[18:01:44.595,489] <inf> COAP_CLIENT: CoAP request resend, timeout 6
[18:01:44.596,832] <inf> COAP_CLIENT: resent_to_peer 175
[18:01:51.597,991] <inf> COAP_CLIENT: CoAP request resend, timeout 12
[18:01:51.607,635] <inf> COAP_CLIENT: resent_to_peer 175
[18:02:04.609,924] <inf> COAP_CLIENT: CoAP request resend, timeout 24
[18:02:04.619,537] <inf> COAP_CLIENT: resent_to_peer 175
[18:02:17.364,807] <inf> COAP_CLIENT: RRC mode: Idle after 37156 ms (12745 ms inactivity)
[18:02:25.377,655] <inf> COAP_CLIENT: LTE modem sleeps
[18:02:32.625,030] <inf> COAP_CLIENT: 729/-1ms/-1ms: failure

It starts with passing the message to "sento", which returns success with 175.

Then the modem wakes up and reports RRC connected. Though no response is received, the modem resends the message also with "sendto", which also returns success with 175.

On the server side the ip-capture shows, that no UDP traffic form that modem comes in, but from other clients it is still working.

The status page of the SIM-card provider shows as last event "PDP Context deleted".

For me this look like the modem seems to have trouble with the "PDP Context deleted" but didn't report that in the return code of "sendto".

Any proposal how to overcome that?

(And no, I'm not able to provide a capture of a 4 weeks run ;-)! But I hope, that the Nordic development team does also long term tests and so have already the experience with that.)

[helsing]

Hi Achim, sorry for the delay.

Something to point out is that both sendto (and send) returns success if the data is transferred to the modem. As UDP does not have guaranteed delivery, a successful sendto will not guarantee that the data will reach or have reached the server.

Could this be a data quota issue?

What happens if you reset the device?

[Achim Kraus]

Not sure, but it seems to happens again and again:

I try to provide precise information and I would welcome, if an modem expert of Nordic spends some time in. But it's no problem at all, if this is not possible. I'm only evaluating.

[helsing]

Achim Kraus said:

I try to provide precise information and I would welcome, if an modem expert of Nordic spends some time in.

Sorry for the delay here, I was out of office for a few days. We will try our best to support you and figure this out.

[Achim Kraus]

The kind of support happens again and again.

I provide information, precise as possible, but I get answers like "UDP is unreliable".

Take this other issue:

 https://devzone.nordicsemi.com/f/nordic-q-a/90290/nrf9160-dk---sometimes-busy-for-1-minute-before-entering-psm 

I provide a capture. instead of having this capture analyzed, I get a lot of proposals, what I should do or test. Even proposals, which are clearly fails as the capture contains already the information on that proposal.

What I would really appreciate is to have just either a "not of general interest", that's OK for me, or really an analysis of your modem experts. if you search this forum, you will find the one or other issue with network loss. And maybe the idea to more consequently report them as error is a not too bad one.

[Achim Kraus]

> Does this only happen after 4 weeks, or could it also happen after a shorter time interval?

I detected this only once after 4 weeks. Other devices run also for longer. And other fails with other errors earlier or have been updated before that 4 weeks.

[helsing]

Hi Achim,

I suggest you check out the Memfault module as a tool for investigating further. The module provides "functionality to collect debug information from devices in the form of coredumps, error traces, metrics, logs and more".

In order to investigate furter,

• which hardware version of the Thingy:91 are you using?
• which nRF Connect SDK version?
• which mfw?

[helsing]

Please let us know if you have any concerns regarding how to set up Memfault. A stack trace and core dump would be among the most interesting to see.

It would in any case also be great if you could try provide a modem trace from when the modem crashes.

The status page of the SIM-card provider shows as last event "PDP Context deleted".

Have you asked the provider for any more information regarding "PDP Context deleted". What does this indicate?

Another thing to try(which is less optimal than Memfault and modem trace) is to subscribe to any kind of AT commands(however, this also requires you to be able to receives logs from the device).

[Achim Kraus]

I have no concers, I just don't think that it's the right thing for this case.

So, your modem experts are not sure, what "PDP Context deleted" means?

Was my issue never forwarded to them? After that weeks?

I see two ways to go:

- consider, it's a bug in my application, then we can close it. For prod-mode my app will just reboot.

- consider, the modem should have detected the error and report that as error-code, then it may be of interest for your modem developer. And for me, if the outcome is, that the next firmware version will report that.