Ping Instability on Serial LTE Modem

[Oguzhan Turk]

I'm using firmware 1.3.6 and sdk 2.7 for Nordic nRF9160. For the internet connection ppp is being used.

Here are the results:

modem shell:

time wget https://leil.de/di/files/more/testdaten/1mb.test
{}2024-11-06 13:43:04{}  https://leil.de/di/files/more/testdaten/1mb.test
Resolving leil.de... 85.13.146.170
Connecting to leil.de|85.13.146.170|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048577 (1.0M) [application/octet-stream]
Saving to: '1mb.test'

1mb.test                                        100%[=====================================================================================================>]   1.00M  5.25KB/s    in 3m 8s   

2024-11-06 13:46:16 (5.46 KB/s) - '1mb.test' saved [1048577/1048577]

real    3m11.698s
user    0m0.104s
sys    0m0.094s

ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=110 time=415.513 ms
64 bytes from 8.8.8.8: seq=1 ttl=110 time=288.383 ms
64 bytes from 8.8.8.8: seq=2 ttl=110 time=215.223 ms
64 bytes from 8.8.8.8: seq=3 ttl=110 time=260.121 ms
64 bytes from 8.8.8.8: seq=4 ttl=110 time=219.662 ms
64 bytes from 8.8.8.8: seq=5 ttl=110 time=253.860 ms
64 bytes from 8.8.8.8: seq=6 ttl=110 time=219.204 ms
64 bytes from 8.8.8.8: seq=7 ttl=110 time=261.940 ms
64 bytes from 8.8.8.8: seq=8 ttl=110 time=221.751 ms
64 bytes from 8.8.8.8: seq=9 ttl=110 time=281.535 ms
64 bytes from 8.8.8.8: seq=10 ttl=110 time=217.329 ms
64 bytes from 8.8.8.8: seq=11 ttl=110 time=205.198 ms
64 bytes from 8.8.8.8: seq=12 ttl=110 time=212.157 ms
64 bytes from 8.8.8.8: seq=13 ttl=110 time=250.918 ms
64 bytes from 8.8.8.8: seq=14 ttl=110 time=211.623 ms
64 bytes from 8.8.8.8: seq=15 ttl=110 time=249.445 ms
64 bytes from 8.8.8.8: seq=16 ttl=110 time=209.267 ms
64 bytes from 8.8.8.8: seq=17 ttl=110 time=251.017 ms
64 bytes from 8.8.8.8: seq=18 ttl=110 time=210.638 ms
64 bytes from 8.8.8.8: seq=19 ttl=110 time=247.540 ms
64 bytes from 8.8.8.8: seq=20 ttl=110 time=208.451 ms
64 bytes from 8.8.8.8: seq=21 ttl=110 time=295.088 ms
64 bytes from 8.8.8.8: seq=22 ttl=110 time=205.773 ms
64 bytes from 8.8.8.8: seq=23 ttl=110 time=248.615 ms
64 bytes from 8.8.8.8: seq=24 ttl=110 time=209.308 ms
64 bytes from 8.8.8.8: seq=25 ttl=110 time=246.882 ms
64 bytes from 8.8.8.8: seq=26 ttl=110 time=213.419 ms
64 bytes from 8.8.8.8: seq=27 ttl=110 time=263.234 ms
64 bytes from 8.8.8.8: seq=28 ttl=110 time=216.908 ms
64 bytes from 8.8.8.8: seq=29 ttl=110 time=258.720 ms
64 bytes from 8.8.8.8: seq=30 ttl=110 time=240.915 ms
64 bytes from 8.8.8.8: seq=31 ttl=110 time=247.380 ms
64 bytes from 8.8.8.8: seq=32 ttl=110 time=207.153 ms
64 bytes from 8.8.8.8: seq=33 ttl=110 time=264.260 ms
64 bytes from 8.8.8.8: seq=34 ttl=110 time=211.652 ms
64 bytes from 8.8.8.8: seq=35 ttl=110 time=244.559 ms
64 bytes from 8.8.8.8: seq=36 ttl=110 time=204.369 ms
64 bytes from 8.8.8.8: seq=37 ttl=110 time=246.127 ms
64 bytes from 8.8.8.8: seq=38 ttl=110 time=205.897 ms
64 bytes from 8.8.8.8: seq=39 ttl=110 time=249.985 ms
64 bytes from 8.8.8.8: seq=40 ttl=110 time=215.934 ms
64 bytes from 8.8.8.8: seq=41 ttl=110 time=259.636 ms
^C
— 8.8.8.8 ping statistics —
42 packets transmitted, 42 packets received, 0% packet loss
round-trip min/avg/max = 204.369/239.680/415.513 ms

slm:

time wget https://leil.de/di/files/more/testdaten/1mb.test
-2024-11-06 14:05:39-  https://leil.de/di/files/more/testdaten/1mb.test
Resolving leil.de... 85.13.146.170
Connecting to leil.de|85.13.146.170|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048577 (1.0M) [application/octet-stream]
Saving to: '1mb.test'

1mb.test                                        100%[=====================================================================================================>]   1.00M  5.99KB/s    in 3m 21s  

2024-11-06 14:09:22 (5.10 KB/s) - '1mb.test' saved [1048577/1048577]

real    3m42.271s
user    0m0.114s
sys    0m0.080s

ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=57 time=1090.205 ms
64 bytes from 8.8.8.8: seq=1 ttl=57 time=2096.799 ms
64 bytes from 8.8.8.8: seq=2 ttl=57 time=1109.679 ms
64 bytes from 8.8.8.8: seq=3 ttl=57 time=1123.361 ms
64 bytes from 8.8.8.8: seq=4 ttl=57 time=1085.234 ms
64 bytes from 8.8.8.8: seq=5 ttl=57 time=1129.830 ms
64 bytes from 8.8.8.8: seq=6 ttl=57 time=2148.470 ms
64 bytes from 8.8.8.8: seq=7 ttl=57 time=1160.427 ms
64 bytes from 8.8.8.8: seq=8 ttl=57 time=2070.497 ms
64 bytes from 8.8.8.8: seq=9 ttl=57 time=1082.377 ms
64 bytes from 8.8.8.8: seq=10 ttl=57 time=1103.330 ms
64 bytes from 8.8.8.8: seq=11 ttl=57 time=1136.550 ms
64 bytes from 8.8.8.8: seq=12 ttl=57 time=2153.073 ms
64 bytes from 8.8.8.8: seq=13 ttl=57 time=1164.889 ms
64 bytes from 8.8.8.8: seq=14 ttl=57 time=2057.927 ms
64 bytes from 8.8.8.8: seq=15 ttl=57 time=1077.550 ms
64 bytes from 8.8.8.8: seq=16 ttl=57 time=1098.615 ms
64 bytes from 8.8.8.8: seq=17 ttl=57 time=2103.559 ms
64 bytes from 8.8.8.8: seq=18 ttl=57 time=1114.472 ms
64 bytes from 8.8.8.8: seq=19 ttl=57 time=2099.151 ms
64 bytes from 8.8.8.8: seq=20 ttl=57 time=1119.168 ms
64 bytes from 8.8.8.8: seq=21 ttl=57 time=2100.020 ms
64 bytes from 8.8.8.8: seq=22 ttl=57 time=1111.763 ms
64 bytes from 8.8.8.8: seq=23 ttl=57 time=1125.321 ms
64 bytes from 8.8.8.8: seq=24 ttl=57 time=1092.266 ms
64 bytes from 8.8.8.8: seq=25 ttl=57 time=105.238 ms
64 bytes from 8.8.8.8: seq=26 ttl=57 time=1090.017 ms
64 bytes from 8.8.8.8: seq=27 ttl=57 time=1049.417 ms
64 bytes from 8.8.8.8: seq=28 ttl=57 time=2067.123 ms
64 bytes from 8.8.8.8: seq=29 ttl=57 time=1078.016 ms
64 bytes from 8.8.8.8: seq=30 ttl=57 time=1134.484 ms
64 bytes from 8.8.8.8: seq=31 ttl=57 time=1070.799 ms
64 bytes from 8.8.8.8: seq=32 ttl=57 time=118.682 ms
64 bytes from 8.8.8.8: seq=33 ttl=57 time=1132.258 ms
64 bytes from 8.8.8.8: seq=34 ttl=57 time=1101.850 ms
64 bytes from 8.8.8.8: seq=35 ttl=57 time=2103.199 ms
64 bytes from 8.8.8.8: seq=36 ttl=57 time=1115.130 ms
64 bytes from 8.8.8.8: seq=37 ttl=57 time=1133.586 ms
64 bytes from 8.8.8.8: seq=38 ttl=57 time=2073.434 ms
64 bytes from 8.8.8.8: seq=39 ttl=57 time=1085.863 ms
64 bytes from 8.8.8.8: seq=40 ttl=57 time=2064.546 ms
64 bytes from 8.8.8.8: seq=41 ttl=57 time=1077.322 ms
^C
— 8.8.8.8 ping statistics —
43 packets transmitted, 42 packets received, 2% packet loss
round-trip min/avg/max = 105.238/1341.797/2153.073 ms

Downloading speed are more or less same but but ping results are dramatically worse on SLM. How can it be? What might be the reason of ping instability on SLM?

[Øyvind]

Hello, 

I've forwarded your question internally to those who develop the SLM. 

Will get back to you by end of the day.

Kind regards,
Øyvind

[Øyvind]

Hello again,

The developers agree that this is suspicious, but with limited information on how they can reproduce it is difficult to tell exactly what causes thiis.

Are you able to provide more information on the setup of each application. E.g. are both using LTE-M? How do you configure each sample before starting this test? Do you have more logs to share from each from startup? Are you able to collect modem traces as well during each test? What SIM card are you using? 

Thanks!

Kind regards,
Øyvind

[Oguzhan Turk]

I'm in Germany and using O2 sim card. It was LTE-M connection. I didn't change anything but modified the overlay file for ppp connection.

I also tried Serial LTE Modem of 2.6 SDK with 1.3.6 firmware and same modified ppp overlay file and then it worked perfectly:

ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=113 time=165.667 ms
64 bytes from 8.8.8.8: seq=1 ttl=113 time=191.232 ms
64 bytes from 8.8.8.8: seq=2 ttl=113 time=165.185 ms
64 bytes from 8.8.8.8: seq=3 ttl=113 time=128.151 ms
64 bytes from 8.8.8.8: seq=4 ttl=113 time=159.620 ms
64 bytes from 8.8.8.8: seq=5 ttl=113 time=119.200 ms
64 bytes from 8.8.8.8: seq=6 ttl=113 time=163.098 ms
64 bytes from 8.8.8.8: seq=7 ttl=113 time=204.357 ms
64 bytes from 8.8.8.8: seq=8 ttl=113 time=181.489 ms
64 bytes from 8.8.8.8: seq=9 ttl=113 time=197.444 ms
64 bytes from 8.8.8.8: seq=10 ttl=113 time=178.525 ms
64 bytes from 8.8.8.8: seq=11 ttl=113 time=190.697 ms
64 bytes from 8.8.8.8: seq=12 ttl=113 time=150.017 ms
64 bytes from 8.8.8.8: seq=13 ttl=113 time=189.505 ms
64 bytes from 8.8.8.8: seq=14 ttl=113 time=163.905 ms
64 bytes from 8.8.8.8: seq=15 ttl=113 time=188.434 ms
64 bytes from 8.8.8.8: seq=16 ttl=113 time=148.029 ms
64 bytes from 8.8.8.8: seq=17 ttl=113 time=184.078 ms
64 bytes from 8.8.8.8: seq=18 ttl=113 time=143.896 ms
64 bytes from 8.8.8.8: seq=19 ttl=113 time=184.349 ms
64 bytes from 8.8.8.8: seq=20 ttl=113 time=152.094 ms
64 bytes from 8.8.8.8: seq=21 ttl=113 time=195.506 ms
64 bytes from 8.8.8.8: seq=22 ttl=113 time=154.951 ms
64 bytes from 8.8.8.8: seq=23 ttl=113 time=194.785 ms
64 bytes from 8.8.8.8: seq=24 ttl=113 time=149.357 ms
64 bytes from 8.8.8.8: seq=25 ttl=113 time=192.802 ms
64 bytes from 8.8.8.8: seq=26 ttl=113 time=152.842 ms
64 bytes from 8.8.8.8: seq=27 ttl=113 time=192.650 ms
64 bytes from 8.8.8.8: seq=28 ttl=113 time=152.106 ms
64 bytes from 8.8.8.8: seq=29 ttl=113 time=190.597 ms
64 bytes from 8.8.8.8: seq=30 ttl=113 time=134.619 ms
64 bytes from 8.8.8.8: seq=31 ttl=113 time=175.662 ms
64 bytes from 8.8.8.8: seq=32 ttl=113 time=135.141 ms
64 bytes from 8.8.8.8: seq=33 ttl=113 time=171.926 ms
64 bytes from 8.8.8.8: seq=34 ttl=113 time=132.017 ms
64 bytes from 8.8.8.8: seq=35 ttl=113 time=222.372 ms
^C
--- 8.8.8.8 ping statistics ---
36 packets transmitted, 36 packets received, 0% packet loss
round-trip min/avg/max = 119.200/169.341/222.372 ms

So, I think, something was broken for Serial LTE Modem on transition from 2.6 SDK to 2.7 SDK.

[Oguzhan Turk]

I detected the problem. Sorry, It was not related with 2.6 or 2.7 SDK.

Here was the working solution's build command on the output tab of VS code:

west build --build-dir build . --pristine --board nrf9160dk/nrf9160/ns -- -DNCS_TOOLCHAIN_VERSION=NONE -DEXTRA_CONF_FILE=overlay-ppp.conf -DEXTRA_DTC_OVERLAY_FILE=overlay-ppp-without-cmux.overlay -DBOARD_ROOT=.

And here was the not working solutions's:

west build --build-dir build . --pristine --board nrf9160dk/nrf9160/ns -- -DNCS_TOOLCHAIN_VERSION=NONE -DCONF_FILE=prj.conf -DEXTRA_CONF_FILE=overlay-ppp.conf -DEXTRA_DTC_OVERLAY_FILE=overlay-ppp-without-cmux.overlay -DBOARD_ROOT=.

AS you can see the difference is:

-DCONF_FILE="prj.conf"

Btw: I modified the paths on commands ("build", ".", "prj.conf" etc.) above with relative path. Normally, VS Code was generating them with exact path with my folder names.

If I select "prj.conf" for "Base configuration files (Kconfig fragments)" it was not working (unstable ping), otherwise works fine. On the other firmware application "modem shell" it was not a problem with selecting but it is for slm. So, question is why?

[Øyvind]

Hello, our developers inform me that you should not select prj.conf as this is picked up by the build system automatically, and by selecting this through -DCONF_FILE="prj.conf" it messes with the build configuration. 

More information can be found here: https://docs.nordicsemi.com/bundle/ncs-latest/page/zephyr/build/kconfig/setting.html#the_initial_configuration