NRF9161 based LTE-M tracker data drops while moving – can it be optimized?

[marsal]

Hi everyone,

I’m using a custom board (a tracker) that can travel at speeds up to 65 km/h and transmits data regularly every 200 ms. I expect, therefore, that the inactivity timer should not be activated. While the device is in motion, I’m seeing intermittent data gaps ranging up to several seconds, which is unacceptable for our application.

Could this be caused by non-optimal cell-handover settings? I’ve attached a few logs below for reference.

Any suggestions would be greatly appreciated.

Thanks,
Martin

 

Three log fragments examples, related to data send drops:

 

17:31:57.179 +CSCON: 0

17:31:57.230 <inf> main: RRC mode: Idle

17:31:57.276 +CEREG: 1,"04D3","0DBC2351",7,,,"11100000","11100000"

17:31:57.298 <inf> main: Cell update, cell tac: 1235, cell id: 230433617

17:31:57.371 +CSCON: 1

17:31:57.429 <inf> main: RRC mode: Connected

17:31:57.539 +CSCON: 0

17:31:57.568 <inf> main: RRC mode: Idle

17:31:57.907 +CSCON: 1

17:31:57.926 <inf> main: RRC mode: Connected

***

17:37:54.095 +CEREG: 1,"04D3","0DC18750",7,,,"11100000","11100000"

17:37:54.154 [00:54:10.770,263] <inf> main: Cell update, cell tac: 1235, cell id: 230786896

***

17:38:05.373 +CEREG: 1,"04D3","0DDD4350",7,,,"11100000","11100000"

17:38:05.434 <inf> main: Cell update, cell tac: 1235, cell id: 232604496

***

 

Modem firmware: mfw_nrf91x1_2.0.2

 

Current 9161 modem initiation sequence is:

 

AT+CFUN=0

AT%XSYSTEMMODE=1,0,0,0

AT+CPSMS=0

AT+CEDRXS=0

AT%RAI=0

AT+CSCON=1

AT+CEREG=5

AT+CFUN=1

[Achim Kraus]

marsal said:

I’m seeing intermittent data gaps ranging up to several seconds, which is unacceptable for our application.

marsal said:

5 Hz to an MQTT broker

You chose MQTT. That usually queues the messages and transmit them, when transmission is possible. So, there may be a short gap in the time-stream, but not the data-stream, that's just delayed. If you add the "sensor-time" to the data, you will also be able to have the time-stream available with delays.

I guess, you need to adapt your application to that common practice.

[marsal]

Thanks, well, my interval_between_two_subsequent_successful_mqtt_sends is between 210ms and 300ms in case that there are no cell updates, which is OK for me. When cell update comes, it raises to e.g. 1.5 sec. So the cell update seems really be the case.

Er, I am struggling just now to catch the modem trace... Not sure if UART or RTT approach is better for me. 2.6.3.

[marsal]

Thanks, dejans,
one question regarding the current setup: are you please aware of an implementation with LTE-M NRF9161 where the handover goes smoothly? Which carrier provider?
Thx!

[marsal]

This is really a puzzle for us. We did run other bunch of tests today on both O2 and Tmobile network in the area where is clearly no issue with the signal strength/quality (always better than -90dBm). We were observing dropouts up to 15s, typically related to cell update, but not always. So if I understand well, both Tmobile and O2 networks in the Czech Republic are currently not suitable to run LTE-M devices which are moving for a businss case which needs uninterrupted data delivery?

[marsal]

I'd like just say thanks again with the hope that DECT NR+ will be able to save our business case.

I am not a network expert but looking at the modem trace just confirms for me what you had stated - there seems really to be no on connect handover, all "handovers" are initiated by the modem (even when the transition is "good" or "bad"):



Just FYI, the modem trace is here: https://e.pcloud.link/publink/show?code=XZEMTtZRiah3HaWN7Q0Pifo1kt09HcMUDTX, I have used a tiny app for testing:

/*
 * Copyright (c) 2018 Nordic Semiconductor ASA
 *
 * SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
 */

#include <modem/nrf_modem_lib.h>
#include <nrf_modem_at.h>
#include <stdio.h>
#include <string.h>
#include <zephyr/drivers/clock_control.h>
#include <zephyr/drivers/clock_control/nrf_clock_control.h>
#include <zephyr/drivers/uart.h>
#include <zephyr/kernel.h>
#include <zephyr/logging/log.h>
LOG_MODULE_REGISTER(main, CONFIG_LOG_DEFAULT_LEVEL);

#include <zephyr/net/net_context.h>
#include <zephyr/net/net_core.h>
#include <zephyr/net/net_if.h>
#include <zephyr/net/net_ip.h>
#include <zephyr/net/socket.h>


/* To strictly comply with UART timing, enable external XTAL oscillator */
void enable_xtal(void) {
    struct onoff_manager *clk_mgr;
    static struct onoff_client cli = {};

    clk_mgr = z_nrf_clock_control_get_onoff(CLOCK_CONTROL_NRF_SUBSYS_HF);
    sys_notify_init_spinwait(&cli.notify);
    (void)onoff_request(clk_mgr, &cli);
}

char *msend;
char atb[500];


int main(void) {
    int err;

    printk("The AT host sample started\n");

    err = nrf_modem_lib_init();
    if (err) {
        printk("Modem library initialization failed, error: %d\n", err);
        return -1;
    }
    enable_xtal();
    printk("Ready\n");


    msend = "AT+CFUN=0";
    LOG_INF("Initialization sequence of modem, %s", msend);
    err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
    LOG_INF("at response (%d): %s", err, atb);

    msend = "AT+CEMODE=0";
    LOG_INF("Initialization sequence of modem, %s", msend);
    err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
    LOG_INF("at response (%d): %s", err, atb);

    // syntax: if %, this character must be doubled for nrf_modem_at_cmd!
    // disable GNSS, LTE-M only
    msend = "AT%XSYSTEMMODE=1,0,0,0";
    LOG_INF("Initialization sequence of modem, %s", msend);
    err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
    LOG_INF("at response (%d): %s", err, atb);

    msend = "AT+CPSMS=0";
    LOG_INF("Initialization sequence of modem, %s", msend);
    err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
    LOG_INF("at response (%d): %s", err, atb);

    msend = "AT+CEDRXS=0";
    LOG_INF("Initialization sequence of modem, %s", msend);
    err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
    LOG_INF("at response (%d): %s", err, atb);

    // syntax: if %, this character must be doubled for nrf_modem_at_cmd!
    //  disable RAI
    msend = "AT%RAI=0";
    LOG_INF("Initialization sequence of modem, %s", msend);
    err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
    LOG_INF("at response (%d): %s", err, atb);

    // if %, this character must be doubled for nrf_modem_at_cmd!
    //  runtime bandlock mask
    //  CAN VERIFY AT%%XBANDLOCK?
    msend = "AT%XBANDLOCK=2,\"10000000000000000000\"";
    LOG_INF("Initialization sequence of modem, %s", msend);
    err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
    LOG_INF("at response (%d): %s", err, atb);

    msend = "AT+CSCON=1";
    LOG_INF("Initialization sequence of modem, %s", msend);
    err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
    LOG_INF("at response (%d): %s", err, atb);

    msend = "AT+CEREG=5";
    LOG_INF("Initialization sequence of modem, %s", msend);
    err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
    LOG_INF("at response (%d): %s", err, atb);

    msend = "AT+CFUN=1";
    LOG_INF("Initialization sequence of modem, %s", msend);
    err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
    LOG_INF("at response (%d): %s", err, atb);


reset_socket:

    int sock;
    struct sockaddr_in server_addr;
    const uint8_t keepalive_payload[1] = {0x42};

    /* (Re)create UDP socket ------------------------------------------------ */
    sock = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
    if (sock < 0) {
        LOG_ERR("socket() failed: %d", errno);
        k_msleep(500);
        goto reset_socket;
    }

    memset(&server_addr, 0, sizeof(server_addr));
    server_addr.sin_family = AF_INET;
    server_addr.sin_port = htons(12345);

    /* 8.8.8.8 -> binary ---------------------------------------------------- */
    inet_pton(AF_INET, "8.8.8.8", &server_addr.sin_addr);

    LOG_INF("Socket ready, entering keep-alive loop");

    /* Keep-alive loop ------------------------------------------------------ */

#define INTEVALRESET 20

    int intervalreset = INTEVALRESET;  // how many loops to try to reconnect
    for (;;) {
        ssize_t sent = sendto(sock, keepalive_payload, sizeof(keepalive_payload), 0, (struct sockaddr *)&server_addr,
                              sizeof(server_addr));

        if (sent < 0) {
            int err = errno;
            LOG_WRN("sendto() failed: %d, resetting socket", err);
            close(sock);
            k_msleep(100);
            goto reset_socket;
        } else {
            LOG_INF("Keep-alive sent (%zd bytes)", sent);
        }

        // tick wait loop
        k_msleep(500);

        // reconnect trial?
        intervalreset -= 1;
        if (intervalreset <= 0) {
            intervalreset = INTEVALRESET;

            msend = "AT+CGATT=0";
            LOG_INF("Initialization sequence of modem, %s", msend);
            err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
            LOG_INF("at response (%d): %s", err, atb);

            k_msleep(500);

            msend = "AT+CGATT=1";
            LOG_INF("Initialization sequence of modem, %s", msend);
            err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
            LOG_INF("at response (%d): %s", err, atb);

            /*

                        msend = "AT+CFUN=4";
                        LOG_INF("Initialization sequence of modem, %s", msend);
                        err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
                        LOG_INF("at response (%d): %s", err, atb);

                        k_msleep(1);

                        msend = "AT+CFUN=1";
                        LOG_INF("Initialization sequence of modem, %s", msend);
                        err = nrf_modem_at_cmd(atb, sizeof(atb), "%s", msend);
                        LOG_INF("at response (%d): %s", err, atb);

            */
        }
    }
}

We are negotiating with the local O2 MNO, we will see...

Now, I am trying the only option we possibly have, to try to force reconnect when we observe the low quality signal. Our best option found so far is something like (shown in the code): 

"AT+CGATT=0";

wait 500ms

"AT+CGATT=1";

but this situation is not optimal, indeed...

Thanks again!

[Achim Kraus]

marsal said:

but this situation is not optimal, indeed...

Such gaps are more or less the nature of radio transmissions, regardless of cellular or other radio technologies.

In some more expensive implementations, a second antenna/receiver is used to minimize the gap. If that pays-off depends on the use-case.

As I wrote above, for the very most use-cases there is a common practice about that, even if this requires some adaptions in the application logic.

[marsal]

Thanks, Achim, well, if the modem is connected to a cell with strong signal, it behaves perfectly. If there is cell update, our problems begin and without on-connect handover enabled by MNO they will persist (yes, you are true, we can expect issues even if it were enabled, but not so systemic).

[dejans]

Hi,

marsal said:

Thanks, here is the idea - this means a rework of the current setup, of course:

10 to 20 trackers in one race  + 1 gateway. Distance between trackers and gateway can be up to 1km, direct line of sight. If not enough for the data transfer, an "intermediate" node is expected to be positioned somewhere in the middle between the gateway and the more distant place of the race.

The key here is good hardware radiated performance. Otherwise, NR+ 1.9 GHz should be able to handle distances relatively easy.

marsal said:

Trackers (using our current hardware, btw ;-),  ):
a) Sending to the gateway each 200ms data about position/speed/direction (approx 100 bytes each)
b) Receiving from gateway each 1 second GNSS correction RTK data (up to 1KB each second, same for each tracker)
Idea: trackers typically communicate with gateway, if the gateway is not reachable for a tracker and vice versa, they send/receive via another tracker (mesh).

NR+ should be able to handle such traffic per tracker node, NR+ sink node connected to Cellular/LTE gateway would be the focus point performance wise.

marsal said:

Gateway (our current hardware + LTE modem rework):
a) receiving position/speed/direction data from trackers, sending it to an MQTT server via LTE (I guess not via LTE-M for this case)
b) receiving RTK corrective data from another server and sending them to trackers via DECT NR+
c) Some other communication (AGNSS, some controls)

RTK correction data would be more of a broadcast type than point-to-point per each tracker node.

marsal said:

one question regarding the current setup: are you please aware of an implementation with LTE-M NRF9161 where the handover goes smoothly? Which carrier provider?

Yes, any cellular provider which supports handover for LTE-M1 which would for example be just about every network in US or Nordics. This feature has been enabled for several years already, and it should not be a big task for any carrier to enable it.

marsal said:

So, any hints and DECT NR+ firmware to test this warmly welcomed.

I recommend that you contact your regional sales manager to discuss the possibilities of going forward with NR+.

Best regards,
Dejan

[marsal]

Thanks again, dejans, we hope our O2 negotiations will be fruitful. But as our project lasts 9 months already without usable outcome, even a quick "smoke" mesh test could heal the business position for us.