nRF9160 PPP stability

RE: nRF9160 PPP stability

dejans — Tue, 02 Jul 2024 14:13:13 GMT

Hi Bernhard,

Thank you for additional information.

I am looking at one of your previous comments.

[quote user="Bernhard"]

1. longer time without an error

2. a lot of these:

[/quote]

How long does your application work as expected, without errors?
When do the errors

00> ppp_modem_dl_data_thread_handler: recv() from modem failed, -1, errno: -115

appear? How often do they appear? How did the device recover when they started appearing?

[quote user="Bernhard"]3. until:[/quote]

What happens after you receive FATAL ERROR? How did the device recover?

How often do errors mentioned in 2. and 3. occur? Do you see recurrent behavior?

Which LTE network do you connect to?

Best regards,
Dejan

RE: nRF9160 PPP stability

Bernhard — Mon, 01 Jul 2024 18:59:30 GMT

Hello Dejan,

output of the tool is:

C:/ncs/v2.6.0/zephyr/lib/os/printk.c:227

which is the first 'if' statement in:

static int str_out(int c, struct str_context *ctx)
{
	if (ctx->str == NULL || ctx->count >= ctx->max) {
		ctx->count++;
		return c;
	}

	if (ctx->count == ctx->max - 1) {
		ctx->str[ctx->count++] = '\0';
	} else {
		ctx->str[ctx->count++] = c;
	}

	return c;
}

RE: nRF9160 PPP stability

dejans — Mon, 01 Jul 2024 11:43:56 GMT

Hi,

You could try to run addr2line tool on the faulting instruction address to get the line in the code which corresponds to the faulting address.
On Linux, addr2line can be run as
/toolchains/<toolchain_version>/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-addr2line -e build/zephyr/zephyr.elf 0x0004e2e2

Best regards,
Dejan

RE: nRF9160 PPP stability

Bernhard — Sun, 30 Jun 2024 18:08:43 GMT

Hello Dejans,

[quote userid="111786" url="~/f/nordic-q-a/112547/nrf9160-ppp-stability/491339"]If you want to use RTT backend instead of UART, you need to enable RTT support by adding overlay-rtt.conf file to your build command.[/quote]

I have done that and RTT logging is working, but only after the 'Connect' in RTT Viewer, the application is not starting up on it's own.

[quote userid="111786" url="~/f/nordic-q-a/112547/nrf9160-ppp-stability/491339"]Do you connect to LTE-M or NB-IoT network?[/quote]

LTE-M

[quote userid="111786" url="~/f/nordic-q-a/112547/nrf9160-ppp-stability/491339"]What happens after connection gets lost? Could you describe stability issue in more detail?[/quote]

For example I got following error messages:

1. longer time without an error

2. a lot of these:

00> ppp_modem_dl_data_thread_handler: recv() from modem failed, -1, errno: -115
00> ppp_modem_dl_data_thread_handler: recv() from modem failed, -1, errno: -115
00> ppp_modem_dl_data_thread_handler: recv() from modem failed, -1, errno: -115

3. until:

00> [01:16:49.348,419] <wrn> net_ppp: 66 written to RX buf, but after that only 498 space left. Disabling RX for now.
00> . Disabling RX for now.
00>  leftrtt:~$ [01:47:14.208,435] <wrn> at_monitor: No heap space for incoming notification: %CESQ: 23,1,0,0
00> 
00> rtt:~$ [01:47:14.220,275] <err> os: r0/a1:  0x00000004  r1/a2:  0x0000004f  r2/a3:  0x00000008
00> rtt:~$ [01:47:14.220,306] <err> os: r3/a4:  0x0003187d r12/ip:  0x00000000 r14/lr:  0x000387cf
00> rtt:~$ [01:47:14.220,336] <err> os:  xpsr:  0x4100003a
00> rtt:~$ [01:47:14.220,336] <err> os: s[ 0]:  0x000388b1  s[ 1]:  0x2000dea8  s[ 2]:  0x2000dd74  s[ 3]:  0x2000dd6c
00> rtt:~$ [01:47:14.220,367] <err> os: s[ 4]:  0x00000002  s[ 5]:  0x2001c8a6  s[ 6]:  0x00000000  s[ 7]:  0x00042bc1
00> rtt:~$ [01:47:14.220,367] <err> os: s[ 8]:  0x00042b51  s[ 9]:  0x00000001  s[10]:  0x2000dd74  s[11]:  0x000421e1
00> rtt:~$ [01:47:14.220,397] <err> os: s[12]:  0x2001c41c  s[13]:  0x2001c33c  s[14]:  0x00000001  s[15]:  0x00000000
00> rtt:~$ [01:47:14.220,428] <err> os: fpscr:  0x20011e08
00> rtt:~$ [01:47:14.220,458] <err> os: r4/v1:  0x2000dec0  r5/v2:  0x00000000  r6/v3:  0x20027f28
00> rtt:~$ [01:47:14.220,458] <err> os: r7/v4:  0x20027f28  r8/v5:  0x00000000  r9/v6:  0x20011500
00> rtt:~$ [01:47:14.220,489] <err> os: r10/v7: 0x00000000  r11/v8: 0x00000000    psp:  0x20022588
00> rtt:~$ [01:47:14.220,489] <err> os: EXC_RETURN: 0x0
00> rtt:~$ [01:47:14.220,489] <err> os: Faulting instruction address (r15/pc): 0x0004e2e2
00> rtt:~$ [01:47:14.220,550] <err> os: >>> ZEPHYR FATAL ERROR 4: Kernel panic on CPU 0
00> rtt:~$ [01:47:14.220,550] <err> os: Fault during interrupt handling
00> 
00> rtt:~$ [01:47:14.220,611] <err> os: Current thread: 0x20011e08 (ppp_modem_dl_data_thread)
00> rtt:~$ [01:47:14.314,971] <err> os: Halting system

RE: nRF9160 PPP stability

dejans — Fri, 28 Jun 2024 12:55:20 GMT

Hi,

[quote user=""]Also we had the problem that the application is not starting up with RTT logging, only after RTT Viewer was connected. (?)[/quote]

If you want to use RTT backend instead of UART, you need to enable RTT support by adding overlay-rtt.conf file to your build command.

[quote user=""]This is working in principle, but we are facing problems with long term stability, the main application is running for a while but after a few hours the connection may get lost.[/quote]

What happens after connection gets lost? Could you describe stability issue in more detail?
Do you connect to LTE-M or NB-IoT network?

Best regards,
Dejan