nrf9160: Modem error: 0x4, PC: 0x7ef22

[ClemensG]

Hi Nordic Team,

long story short: I get a

[10:04:59.928,985] <err> nrf_modem: Modem error: 0x4, PC: 0x7ef22

and after that the modem does not allow me to open a socket anymore.

ncs v2.1.2
modem fw: nrf9160_1.3.3 (on SICA B1)

Any ideas of how to prevent this?
Any ideas of how I could detect this and how to act on it to recover and beeing able to send data again?


The long story:

Modem is running in LTE-M + GPS.
Modem is in PSM (no eDRX).

The application (test setup) sends data via UDP every 5 Minutes (thus waking the modem up from PSM).
Each time the aplication opens a new socket, sends one or more udp packets, listens a few seconds for
an answer and closes the socket afterwards.
The sim card allows roaming and I have seen the modem changing cells and networks for time to time
(not shure if it happend during this power up but can look it up if it helps)
Single GPS fixes are done normally when the modem is in PSM.

[10:04:59.832,763] <dbg> llp: llp_transmit: 1 measurement packets sent
[10:04:59.928,985] <err> nrf_modem: Modem error: 0x4, PC: 0x7ef22
[10:04:59.929,107] <err> llp: recvfrom error: 11
[10:05:06.034,271] <dbg> tbws: main: Updating timing (waited 5000 ms)
[10:05:06.034,332] <inf> tbws: Updating time
[10:05:06.046,447] <err> lte_lc: Could not get registration status, error: -1


- "Updating time" is triggering a date_time_update_async(). So the "lte_lc: Could not get registration status, error: -1" might come from there.
- recvfrom error is the errno variable (11: EAGAIN)

After that (5 minutes later, when trying to do the next upload) calling socket() gives:
[10:10:00.277,374] <err> llp: socket: 110

110 is the errno: ESHUTDOWN

After quite some more tries to send (can count them if it helps you):
[10:50:00.262,969] <err> llp: socket: 105

110 is the errno: ENOBUFS


The modem does not seem to recover.

What would be the apropriate way to handle this?
Shutting down the modem and reregister after getting an ESHUTDOWN?
Resetting the system (I'd like to avoid this as I would loose my saved data)?

What pops into my mind: the recvfrom returning EAGAIN in this situation seems to be dangerous.
When I would handle that case I might call recvfrom again, maybe in a loop, and thus locking up.
My current implemntation luckily just bails out an any error. But might be a trap for others.

And: Sorry, no modem traces.

Regards,
Clemens

[ClemensG]

Ah, forgot to mention:

the lte_handler() installed is roughly the same as in the udp sample. So every event handled there should be visible in the log but there is nothing.

Also no special  messages from that handler after the last successful tx+rx (besides: "RRC mode: Idle.")

[ClemensG]

Okay, had a bit time to look into this (from user side).

As outlined here i can define my own modem fault handler. Guess saving data and rebooting is the safest option for me.

Also couldn't reproduce that error, yet.

So in case it still is interesting to Nordic to hunt something down I am happy to help, otherwise I can live with it.

As with most of the stuff I am discovering I mention it here again: I would love to see a general document/manual/best practices writeup of how to get a robust solution. I would not have looked down into the modem_lib documentation as I thought this is "just" a middle layer and I use it through the zephyr interface. Yes, now that I know what to look at I see it's applied in some samples.

[wasan]

Hi Clemens,

I have an issue similar you. Error memory 0x04 appeared in my code. I think there is a problem with the modem's firmware or library.

I'm trying to fix this problem by adding a 200mS delay to the function. It can make sending data less likely to go wrong.

sdk 2.1.2 > nrf > lib > nrf_modem_lib > nrf_modem_os.c 

void *nrf_modem_os_shm_tx_alloc(size_t bytes)
{
    extern uint32_t nrf_modem_lib_shmem_failed_allocs;
    void * const addr = k_heap_alloc(&nrf_modem_lib_shmem_heap, bytes, K_NO_WAIT);

    k_msleep(200);
    if (IS_ENABLED(CONFIG_NRF_MODEM_LIB_MEM_DIAG_ALLOC) && !addr)
    {
        nrf_modem_lib_shmem_failed_allocs++;
    }

    return addr;
}

Note:

The signal strength in my area is about -70 dBm. If your signal is less than -100. I think you should add more than 200mS to the delay.

I hope that the Nordic support team will find out what's going on soon 

Wasan

[ClemensG]

Hmmm.... I am sending one packet (UDP) each minute. I guess adding 200ms delay somewhere might not help. Or is there some kind of a race condition while sending a single packet (haven't looked at the code where you added the delay)?

[Michal]

Hello Clemens,

Sorry for the delay, I will look more into it.

Best regards,

Michal