Why is my PPP connection failing using the modem_shell on the nRF9160dk to talk to a Dart-6UL over uart? Also what are these errors I'm seeing on the MoSH even when a PPP connection is successful?

[MichelleV]

I'm trying to connect the nrf9160dk from its uart0 running the modem_shell sample code from SDK v2.2.0 to our Dart6ul on its tty4 uart running at 921600 baud using a PPP connection. I have been able to use the modem_shell on the nrf9160 and ubuntu laptop successfully. But I'm getting errors in both configurations that I don't understand completely. When trying to replace the ubuntu laptop (ttyACM0) with the Dart6UL (ttymxc4), I use the same command (changing the port) to initiate the PPP connection from the unix device:

 pppd -detach /dev/ttymxc4 921600 noauth crtscts noccp novj nodeflate nobsdcomp local debug +ipv6 ipv6cp-use-ipaddr usepeerdns noipdefault defaultroute ipv6cp-restart 5 ipcp-restart 5 lcp-echo-interval 0

I have been able to successfully run the Serial LTE Modem example on my 9160 and Dart-6ul with these same devicetree settings to use the ttymxc4 and uart0.

When running the mosh, I see these same errors on the MoSH terminal (regardless of ubuntu or dart-6ul):

[00:00:04.657,104] <wrn> net_if: iface 0x200127c4 is down

[00:00:34.670,959] <err> net_sock: invalid access on sock -1 by thread 0x20012ad0

[00:00:36.702,850] <err> net_ppp: uart_rx_enable() failed, err -16

And the bottom two repeat. I'm attaching photos of the ubuntu and the dart-6ul outputs from MoSH prompt. In addition the Dart-6ul is not completing the setup of the PPP connection. I see the PPP start but then it fails out and eventually gives me a fork error. I'm attaching my net-connect and net-chat file running on the Dart-6ul. And a log file that the net-connect kicks off.

Any help getting these understood and me pointed in the right direction would be greatly appreciated!

9160dk_mosh_errors.jpg shows the startup of the LTE connection and the issues identified above.

dart_6ul_fail1.jpg shows that the PPP did initiate but it doesn't appear as a network interface and then displays the fork error.

Fullscreen net-connect.txt Download

#!/bin/bash
exec > ppp_connect_log.txt 2>&1

/usr/sbin/pppd -detach /dev/ttymxc4 921600 noauth crtscts noccp novj nodeflate nobsdcomp local debug +ipv6 ipv6cp-use-ipaddr usepeerdns noipdefault defaultroute ipv6cp-restart 5 ipcp-restart 5 lcp-echo-interval 0

Fullscreen net-chat.txt Download

ABORT 'NO CARRIER'
ABORT 'NO DIALTONE'
ABORT 'ERROR'
'' ATZ
OK 'AT+CGDCONT=1,"IP","hologram"'
OK 'ATDT*99***1#'
CONNECT \d\c

Fullscreen ppp_connect_log.txt Download

Script /etc/ppp/net-connect finished (pid 511), status = 0x8
Connect script failed

ce temporarily unavailable
Connect script failed

[Didrik Rokhaug]

Hi,

When running the mosh, I see these same errors on the MoSH terminal (regardless of ubuntu or dart-6ul):

I'll check with the developers, but I think you can ignore them.

see the PPP start but then it fails out and eventually gives me a fork error.

This seems to be a problem with your Linux setup, not the nRF9160 or the modem_shell.

One thing you should check is how many child processes ppp can create, and consider increasing the limit.

Best regards,

Didrik

[MichelleV]

I had also started a ticket with variscite on the linux side. Of course, they thought it was more of a nRF9160 problem with the uart setup. And sent me the following link: https://devzone.nordicsemi.com/f/nordic-q-a/101569/error-when-using-date_time_ntp-in-ncs-2-4-0

Both you and variscite have suggested increasing the child processes ppp can create. I'm a little confused because I have no idea how many would be expected and they way that they were popping up looks to me like they would just continue until the new limit is reached. If you don't have a number of max processes I should expect, can you point me to somewhere with this kind of information?

[Didrik Rokhaug]

MichelleV said:

Both you and variscite have suggested increasing the child processes ppp can create. I'm a little confused because I have no idea how many would be expected and they way that they were popping up looks to me like they would just continue until the new limit is reached. If you don't have a number of max processes I should expect, can you point me to somewhere with this kind of information?

I would start by looking at what the current limit is. If the limit is low, you can try to increase it, but if it seems like it should be big enough, then there might be something else going on.

However, I don't know what would be reasonable values for "low" or "big enough" in this case.

[MichelleV]

I reached out to variscite and they said they would help me align my kernel to work with the 9160 in this but that I needed to ask you what the kernel configuration requirements are. Can you help me with that?

[Didrik Rokhaug]

You can find Zephyr's documentation for their PPP interface here: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/2.4.0/zephyr/connectivity/networking/api/ppp.html

We use Zephyr's PPP implementation, so you might alos try to ask them if they have any suggestions.

The developers of the Modem Shell confirm that the error messages in the log can be ignored. The "invalid access" messages appeared due to some timing changes in Zephyr, which causes the PPP socket to be polled after it is closed.

The UART error comes from the PPP driver's usage of the UART.

But the application is able to recover from both errors, so they shouldn't cause any problems.

[MichelleV]

Thank you! But do they have any idea why it is creating so many pppd child processes I should expect with debian and the modem_shell? 

[MichelleV]

I have one more question. As I have been going through this with you, I have started to question if our intended design will work as expected. The idea is to use the ppp connection from the dart-6ul to the 9160 module via a single uart. We intended to control the device and get the data from the single uart connection. However, I have needed to use two tty connections to interface with the 9160, one for the console and the other for the modem.

Is it required to have two individual uart interfaces for the 9160 to dart-6ul interface?

[MichelleV]

An additional question, I was just looking into enabling modem trace but there is a line in the overlay-ppp.conf (line 118) that states "UART traced cannot be enabled". Does that mean I can't get a modem trace while using the PPP configuration?

[Didrik Rokhaug]

MichelleV said:

An additional question, I was just looking into enabling modem trace but there is a line in the overlay-ppp.conf (line 118) that states "UART traced cannot be enabled". Does that mean I can't get a modem trace while using the PPP configuration?

At the moment (this will change in the next NCS release), the UART backend for modem traces is hard-coded to use UART1. It can be changed to use any UART, you just have to manually edit the source file, rather than doing it with the device tree.

In ppp.overlay, UART0 is chosen for PPP, and UART1 is chosen for the shell. As this conflicts with the UART used for modem traces, modem tracing is disabled.

However, you can use other UARTs than the ones chosen in the overlay file, or you can change which UART is used for modem traces. Another limitation is that on the nRF9160 DK, only two UARTs are routed to the interface chip, so you would have to use an external UART-to-USB converter to get the trace.

A third option is to send the modem trace over RTT instead, by enabling CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_RTT. It isn't as stable as the UART backend, and we don't have any tools for receiving the trace (so you would have to use the tools from Segger), which is why it isn't our preferred option, but it is a possible solution.

MichelleV said:

Is it required to have two individual uart interfaces for the 9160 to dart-6ul interface?

If you need to control the modem? Yes, I believe so.

The modem will try to re-establish network connection if the connection is lost, and the modem shell can be configured to restart if it crashes, so you might be able to get by without any extra control of the modem in some use cases.

However, in general, yes, you would need to use two UARTs.

If the limit is in the number of UARTs on the host processor, you might also be able to work around it with a multiplexer or switches.

[MichelleV]

Didrik Rokhaug said:

MichelleV said:

Is it required to have two individual uart interfaces for the 9160 to dart-6ul interface?

If you need to control the modem? Yes, I believe so.

What kind of control do mean here? The modem_shell should handle the modem operation, correct?