MQTT publisher doesn't connect to broker

[PabloAg]

I'm trying to make MQTT publisher example work using a BMD-340 evaluation board and nRF5 SDK for Thread and Zigbee 4.1.0. I'm using a Raspberry Pi 3B with a Raspberry Pi OS based on linux kernel version 5.4.

I followed this guide to connect BLE devices to the Raspberry Pi, and this one to distribute a global IPv6 prefix.

pi@raspberrypi:~ $ sudo uname -a
Linux raspberrypi 5.4.79-v7+ #1373 SMP Mon Nov 23 13:22:33 GMT 2020 armv7l GNU/Linux
pi@raspberrypi:~ $ sudo su
root@raspberrypi:/home/pi# mount -t debugfs none /sys/kernel/debug
mount: /sys/kernel/debug: none already mounted on /sys/fs/bpf.
root@raspberrypi:/home/pi# modprobe bluetooth_6lowpan
root@raspberrypi:/home/pi# echo 1 > /sys/kernel/debug/bluetooth/6lowpan_enable
root@raspberrypi:/home/pi# lsmod | grep bluetooth_6lowpan
bluetooth_6lowpan      20480  0
6lowpan                28672  8 nhc_udp,nhc_routing,nhc_mobility,bluetooth_6lowpan,nhc_fragment,nhc_dest,nhc_hop,nhc_ipv6
bluetooth             360448  25 bluetooth_6lowpan,hci_uart,bnep,btbcm
ipv6                  458752  39 bluetooth_6lowpan,6lowpan
root@raspberrypi:/home/pi# hciconfig
hci0:   Type: Primary  Bus: UART
        BD Address: B8:27:EB:53:CF:57  ACL MTU: 1021:8  SCO MTU: 64:1
        UP RUNNING
        RX bytes:1434 acl:0 sco:0 events:84 errors:0
        TX bytes:2518 acl:0 sco:0 commands:84 errors:0

root@raspberrypi:/home/pi# hciconfig hci0 reset
root@raspberrypi:/home/pi# hcitool lescan
LE Scan ...
00:70:DF:39:D5:E8 MQTT_Pub
root@raspberrypi:/home/pi# echo "connect 00:70:DF:39:D5:E8 1" > /sys/kernel/debug/bluetooth/6lowpan_control
root@raspberrypi:/home/pi# ifconfig bt0 add 2001:db8::1/64
root@raspberrypi:/home/pi# systemctl restart radvd
root@raspberrypi:/home/pi# ifconfig bt0
bt0: flags=4161<UP,RUNNING,MULTICAST>  mtu 1280
        inet6 fe80::b827:ebff:fe53:cf57  prefixlen 64  scopeid 0x20<link>
        inet6 2001:db8::1  prefixlen 64  scopeid 0x0<global>
        unspec B8-27-EB-53-CF-57-00-00-00-00-00-00-00-00-00-00  txqueuelen 1000  (UNSPEC)
        RX packets 1  bytes 20 (20.0 B)
        RX errors 0  dropped 2  overruns 0  frame 0
        TX packets 18  bytes 757 (757.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

root@raspberrypi:/home/pi# ping6 -I bt0 fe80::70:DFFF:FE39:D5E8
ping6: Warning: source address might be selected on device other than bt0.
PING fe80::70:DFFF:FE39:D5E8(fe80::70:dfff:fe39:d5e8) from :: bt0: 56 data bytes
64 bytes from fe80::70:dfff:fe39:d5e8%bt0: icmp_seq=1 ttl=255 time=81.4 ms
64 bytes from fe80::70:dfff:fe39:d5e8%bt0: icmp_seq=2 ttl=255 time=54.7 ms
64 bytes from fe80::70:dfff:fe39:d5e8%bt0: icmp_seq=3 ttl=255 time=76.6 ms
64 bytes from fe80::70:dfff:fe39:d5e8%bt0: icmp_seq=4 ttl=255 time=98.6 ms
64 bytes from fe80::70:dfff:fe39:d5e8%bt0: icmp_seq=5 ttl=255 time=71.9 ms

--- fe80::70:DFFF:FE39:D5E8 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 11ms
rtt min/avg/max/mdev = 54.716/76.646/98.626/14.212 ms

From the firmware side, I set BLE_6LOWPAN_LEGACY_MODE define in sdk_config.h to 0 and made the corresponding changes to use the example in port 1883 without TLS.

When pressing button 1, MQTT conexion doesn't establish.

I've been trying to detect where is the problem so I used tshark to check raspberry tcp packets and a BLE sniffer. I detected two different behaviours when trying to connect to the broker.

• The first behaviour is the following:

root@raspberrypi:/home/pi# tshark -i bt0 -Y "tcp.port == 1883"
Running as user "root" and group "root". This could be dangerous.
Capturing on 'bt0'
    2 5.396557114 2001:db8::70:dfff:fe39:d5e8 → 2001:db8::1  TCP 80 49153 → 1883 [SYN] Seq=0 Win=3584 Len=0 MSS=896
    5 5.493086729  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 1883 → 49153 [SYN, ACK] Seq=0 Ack=1 Win=64660 Len=0 MSS=1220
    6 5.590976755 2001:db8::70:dfff:fe39:d5e8 → 2001:db8::1  MQTT 104 Connect Command
    7 5.591140713  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 76 1883 → 49153 [ACK] Seq=1 Ack=29 Win=64632 Len=0
    8 5.592257062  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 MQTT 80 Connect Ack
    9 6.185166321  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Retransmission] 1883 → 49153 [PSH, ACK] Seq=1 Ack=29 Win=64632 Len=4
   12 7.445175284  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Retransmission] 1883 → 49153 [PSH, ACK] Seq=1 Ack=29 Win=64632 Len=4
   13 9.845215725  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Retransmission] 1883 → 49153 [PSH, ACK] Seq=1 Ack=29 Win=64632 Len=4
   14 14.645218527  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Retransmission] 1883 → 49153 [PSH, ACK] Seq=1 Ack=29 Win=64632 Len=4
   16 24.245211216  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Retransmission] 1883 → 49153 [PSH, ACK] Seq=1 Ack=29 Win=64632 Len=4

• The other one:

root@raspberrypi:/home/pi# tshark -i bt0 -Y "tcp.port == 1883"
Running as user "root" and group "root". This could be dangerous.
Capturing on 'bt0'
    2 3.662276285 2001:db8::70:dfff:fe39:d5e8 → 2001:db8::1  TCP 80 49153 → 1883 [SYN] Seq=0 Win=3584 Len=0 MSS=896
    3 3.662432014  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 76 [TCP ACKed unseen segment] 1883 → 49153 [ACK] Seq=1 Ack=29 Win=64632 Len=0
    4 3.758546820 2001:db8::70:dfff:fe39:d5e8 → 2001:db8::1  TCP 76 49153 → 1883 [RST, ACK] Seq=29 Ack=1 Win=3584 Len=0
    5 3.808513439 2001:db8::70:dfff:fe39:d5e8 → 2001:db8::1  TCP 80 [TCP Retransmission] 49153 → 1883 [SYN] Seq=0 Win=3584 Len=0 MSS=896
    6 3.808643855  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Previous segment not captured] [TCP Port numbers reused] 1883 → 49153 [SYN, ACK] Seq=1548073366 Ack=1 Win=64660 Len=0 MSS=1220
    7 3.906312825 2001:db8::70:dfff:fe39:d5e8 → 2001:db8::1  TCP 104 [TCP ACKed unseen segment] [TCP Retransmission] 49153 → 1883 [PSH, ACK] Seq=1 Ack=1548073367 Win=3584 Len=28
    8 3.906442096  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 76 1883 → 49153 [ACK] Seq=1548073367 Ack=29 Win=64632 Len=0
    9 3.907529281  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 MQTT 80 Connect Ack
   10 4.216925325  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Retransmission] 1883 → 49153 [PSH, ACK] Seq=1548073367 Ack=29 Win=64632 Len=4
   11 4.826932482  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Retransmission] 1883 → 49153 [PSH, ACK] Seq=1548073367 Ack=29 Win=64632 Len=4
   12 6.066934618  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Retransmission] 1883 → 49153 [PSH, ACK] Seq=1548073367 Ack=29 Win=64632 Len=4
   13 8.546978780  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Retransmission] 1883 → 49153 [PSH, ACK] Seq=1548073367 Ack=29 Win=64632 Len=4
   14 13.426975260  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Retransmission] 1883 → 49153 [PSH, ACK] Seq=1548073367 Ack=29 Win=64632 Len=4
   16 23.506978611  2001:db8::1 → 2001:db8::70:dfff:fe39:d5e8 TCP 80 [TCP Retransmission] 1883 → 49153 [PSH, ACK] Seq=1548073367 Ack=29 Win=64632 Len=4

When looking into the broker's log it seems that it tries to send a CONNACK message, but it never reaches the publisher

1611066958: New connection from 2001:db8::70:dfff:fe39:d5e8 on port 1883.
1611066958: New client connected from 2001:db8::70:dfff:fe39:d5e8 as nrfPublisher (p1, c1, k60).
1611066958: No will message specified.
1611066958: Sending CONNACK to nrfPublisher (0, 0)

I've been trying to make this work for a while now and I don't know what else to do, so any help is appreciated.

[Stian Røed Hafskjold]

Hi, I tried on this on a RPi too and I see the same thing (your "first behavior"). When I capture packets directly on the bt0 interface I see that the MQTT broker is trying to retransmit the CONNACK, but on the BLE sniffer the packets are never sent from the RPi.

Here is a screenshot of the bt0 interface:

Packet number 7 is the last packet that you can see on the BLE sniffer. It is just an ACK, no data. Then there is another packet (number 8), sent from the broker as well, which has the same sequence number, and same ACK number, but it also contains the CONNACK (4 bytes). Then you can see all the retransmissions of this packet that never arrived.

Since the CONNACK packet is never sent over air, I think there must be something wrong on the RPi side. I will have to investigate further to figure out what is wrong.

[Ros]

Hi,

Did you enable ipv6 forwarding? Tis is required to send global unique addresses, Nevertheless, this is not solved problem at my side.

sudo echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
# OR set ipv6 forwading in /etc/sysctl.conf
# Run radvd daemon.
sudo service radvd restart

I have successfully connected two raspberry pi, using 6lowpan, but cannot send data one to another even-though ping6 works fine at both end. I checked the routing table

route -A inet6

and seems everything fine.

[PabloAg]

Hi! Yes, i did enable IPv6 forwarding as said here.

[Ros]

Have you solved the problem?

[PabloAg]

Hi, I haven´t