Hello,
We are currently certifying our module for Verizon in the US. As such, we are updating our software to work with MFW v1.3.0 and the corresponding NCS version v1.7.0. The upgrade went relatively smoothing and things are generally working as we'd expect. Our device will connect either over the CATM1 or NBIOT network (configurable) and then connect to an MQTT broker and send data up. However, we have encountered an issue and require some assistance in troubleshooting it.
When we connect specifically over Vodafone (AT&T in my area) using CATM1 (using SYSTEMMODE set to both CATM1/NBIOT supported w/ LTE Preference = M1 Preferred), we will see our device successfully register to roaming. It will then successfully open the TCP connection to the MQTT broker and send an MQTT Connect message. However we never hear back from the server with the CONNACK. Our device currently waits ~20 seconds before it gives up after not receiving a CONNACK. Now running the EXACT same code and just swapping out the LTEPreference to "NB Preferred", we can successfully get a CONNACK and send/receive MQTT data. When using the SAME MFW version (v1.3.0) and the older NCS version (v1.4.2) we can successfully connect over MQTT using CATM1. This tells me its specifically something with the NCS version, likely something in the nrfxlib Modem Library. I'm not sure if its necessarily a bug, but it could be some subtle way in which the modem library interacts with the Vodafone network while on CAT M1.
I've got some modem trace pcap files below that may help diagnose this issue. In both tests, my device (IP 10.49.0.225) is connecting to server w/ IP 192.168.4.13. You will need to filter the MQTT broker server pcap w/ "ip.src_host == 10.49.0.225 or ip.dst_host == 10.49.0.225".
I've run two tests:
- Connect over NBIOT and successfully use MQTT
- Connect over M1 and FAIL to use MQTT
You can see in the NBIOT pcap files they connection and CONNACK happen as expected. However with the M1 logs, you can see a couple things:
- The TCP SYN Handshake occur successfully.
- The NRF9160 will send the MQTT Connect packet and it looks like i'd expect (screenshot 1).
- Server logs (screenshot 2) looks like the TCP Payload is all zeros. The other difference I could note is that when its received the "Differentiated Services Field" is set differently on the server side and looks invalid (or unknown).
It ultimately seems like the NRF Modem Library version in conjunction with the Vodafone CATM1 network is causing us problems. However I'm not sure how or why this could be happening. I was hoping your team might have some insight on the issue? Any insight or recommendations would be much appreciated!
SCREENSHOT 1
SCREENSHOT 2
