Using GPS in single shot mode seems to occasionally cause opened sockets to silently fail.

[JVantol]

I'm trying to figure out a strange issue with GPS. We have a pretty well debugged and stable system that we're adding some features to, mainly GPS. We're on SDK 1.5, using modem firmware 1.2.3. I've also tried 1.3.0-44.prealpha with the same results. We've enabled eDRX, but do not have PSM enabled in this operating mode.

The initial conditions are this:

We're connected to Azure, via MQTT. We send data reports every 20 minutes. When we send normal data reports without gps location, everything works great. Messages are delivered reliably, everything works as expected, as it has for a long while, with a pretty large number of units.

When I enable GPS I get some failures.

Specifically, I enable the GPS for a single fix before sampling the other data and about 1 minute before it's transmitted. What I see is that it usually works fine, but sometimes our messages are constructed as usual, and sent as usual, with no error returned from azure_iot_hub_send, but they are NOT delivered. This goes on for a while, and eventually, the connection drops and is re-made, and then things work fine again. It does not seem to make a difference whether the GPS fixes are successful or not.  I've tested with one unit that cannot see the sky, (no fixes) and one that's got a nice view, which gets a fix every time. Both units exhibit this behavior. We are using QOS 0.

Do you have any ideas what might be going on? Suggestions about the best way to narrow down the cause of this?

We also note that it only seems to be connections that were opened before the GPS fix is taken that are affected. If we run in discontinuous mode (PSM on, open and close MQTT connection for each send, every 4 hours) it works just fine.

We are running these units on ATT.

[Didrik Rokhaug]

Hi,

Can you take a modem trace?

Also, could you enable debug logging in the modem_lib?

CONFIG_LOG=y
CONFIG_NRF_MODEM_LIB_DEBUG_ALLOC=y
CONFIG_NRF_MODEM_LIB_DEBUG_SHM_TX_ALLOC=y
CONFIG_NRF_MODEM_LIB_HEAP_DUMP_PERIODIC=y
CONFIG_NRF_MODEM_LIB_SHM_TX_DUMP_PERIODIC=y
CONFIG_NRF_MODEM_LIB_LOG_LEVEL_DBG=y

Best regards,

Didrik

[JVantol]

Didrik,

I can do this, but I fear that since the problem is so sporadic, It might take a very long time to catch it with a log. I'll work on getting that done, but in the meantime, do you have any other ideas about what might be happening?

Josh

[Didrik Rokhaug]

My suspicion is that the shared memory buffer between the modem and application core gets filled up.

This can happen e.g. if you don't read all the data that comes from the GPS, or if there is a memory leak.

[JVantol]

Didrik,

It turned out this was a red herring. Thanks so much for your advice and information though, it got me pointed in the right direction.