nRF9160, Sending messages via MQTT larger than 256 bytes

RE: nRF9160, Sending messages via MQTT larger than 256 bytes

Stas Jis — Sun, 13 Mar 2022 08:34:37 GMT

Didrik,
We tried to send JSON to SDK version 1.9, everything worked, JSON was completely received. Thanks for the help.

RE: nRF9160, Sending messages via MQTT larger than 256 bytes

Stas Jis — Sun, 13 Mar 2022 07:38:01 GMT

Didrik,
JSON we send to "Data mode":

AT#XMQTTCON=1,"999876","","","a34k7wa09ujucc-ats.iot.us-east-1.amazonaws.com",8883,321
OK
#XMQTTEVT: 0,0
AT#XMQTTPUB="Rigel/ToServer/v2/AR_TEST","",1

{"deviceId":"","deviceType":"Collar","requestTime":0,"carrier":"CELLULAR","rssi":-70,"messages":[{"messageId":536875008,"messageCategory":1,"messageType":9,"recordedTime":0,"firmwareVersion":"collar-fw-0.1.3","bootVersion":"collar-boot-0.0.1","attachmentStatus":"Unknown","battery":0,"chargeStatus":0,"operationMode":0,"debugMode":0,"sos":0,"safeZone":0}]}
#XMQTTEVT: 3,0
#XMQTTEVT: 9,0

RE: nRF9160, Sending messages via MQTT larger than 256 bytes

Didrik Rokhaug — Fri, 11 Mar 2022 14:10:30 GMT

Could you try with NCS v1.9.0?

And how are you sending the JSON data to the nRF9160?

When using NCS v1.9.0, I am able to send 575 bytes (which matches MQTT_MESSAGE_BUFFER_LEN = NET_IPV4_MTU = 576 pretty well. The last byte is probably a newline which isn't displayed by my desktop MQTT client), when I paste it all into minicom at once. However, if I write manually into minicom, the device sends each letter by itself.

RE: nRF9160, Sending messages via MQTT larger than 256 bytes

Stas Jis — Thu, 10 Mar 2022 07:29:01 GMT

Didrik,
I am attaching logs from RTT Viewer:

00>                                       41 54 2b 43 45 52 45 47  3d 35                   |AT+CEREG =5      
00> [00:13:28.292,175] <dbg> slm_at_host.uart_callback: RX_DISABLED
00> [00:13:28.298,309] <dbg> slm_at_host: TX
00>                                       0d 0a                                            |..               
00> [00:13:28.298,370] <dbg> slm_at_host: TX
00>                                       4f 4b 0d 0a                                      |OK..             
00> [00:13:38.572,906] <dbg> slm_at_host: RX
00>                                       41 54 2b 43 46 55 4e 3d  31                      |AT+CFUN= 1       
00> [00:13:38.573,394] <dbg> slm_at_host.uart_callback: RX_DISABLED
00> [00:13:38.609,130] <dbg> slm_at_host: TX
00>                                       0d 0a                                            |..               
00> [00:13:38.609,802] <dbg> slm_at_host: TX
00>                                       4f 4b 0d 0a                                      |OK..             
00> [00:13:39.660,369] <dbg> slm_at_host: TX
00>                                       0d 0a                                            |..               
00> [00:13:39.660,430] <dbg> slm_at_host: TX
00>                                       2b 43 45 52 45 47 3a 20  32 2c 22 30 32 36 46 22 |+CEREG:  2,"026F"
00>                                       2c 22 30 30 33 33 32 35  32 31 22 2c 37 0d 0a    |,"003325 21",7.. 
00> [00:13:42.765,625] <dbg> slm_at_host: TX
00>                                       0d 0a                                            |..               
00> [00:13:42.765,686] <dbg> slm_at_host: TX
00>                                       2b 43 45 52 45 47 3a 20  35 2c 22 30 32 36 46 22 |+CEREG:  5,"026F"
00>                                       2c 22 30 30 33 33 32 35  32 31 22 2c 37 2c 2c 2c |,"003325 21",7,,,
00>                                       22 30 30 30 31 31 31 31  30 22 2c 22 31 31 31 30 |"0001111 0","1110
00>                                       30 30 30 30 22 0d 0a                             |0000"..          
00> [00:13:50.564,025] <dbg> slm_at_host: RX
00>                                       41 54 23 58 4d 51 54 54  43 4f 4e 3d 31 2c 22 39 |AT#XMQTT CON=1,"9
00>                                       39 39 38 37 36 22 2c 22  22 2c 22 22 2c 22 61 33 |99876"," ","","a3
00>                                       34 6b 37 77 61 30 39 75  6a 75 63 63 2d 61 74 73 |4k7wa09u jucc-ats
00>                                       2e 69 6f 74 2e 75 73 2d  65 61 73 74 2d 31 2e 61 |.iot.us- east-1.a
00>                                       6d 61 7a 6f 6e 61 77 73  2e 63 6f 6d 22 2c 38 38 |mazonaws .com",88
00>                                       38 33 2c 33 32 31                                |83,321           
00> [00:13:50.564,514] <dbg> slm_at_host.uart_callback: RX_DISABLED
00> [00:13:56.083,831] <dbg> slm_at_host: TX
00>                                       0d 0a 4f 4b 0d 0a                                |..OK..           
00> [00:13:56.369,232] <dbg> slm_at_host: TX
00>                                       0d 0a 23 58 4d 51 54 54  45 56 54 3a 20 30 2c 30 |..#XMQTT EVT: 0,0
00>                                       0d 0a                                            |..               
00> [00:14:00.742,980] <dbg> slm_at_host: TX
00>                                       0d 0a                                            |..               
00> [00:14:00.743,041] <dbg> slm_at_host: TX
00>                                       2b 43 45 52 45 47 3a 20  35 2c 22 30 32 36 46 22 |+CEREG:  5,"026F"
00>                                       2c 22 30 30 33 33 32 35  32 30 22 2c 37 2c 2c 2c |,"003325 20",7,,,
00>                                       22 30 30 30 31 31 31 31  30 22 2c 22 31 31 31 30 |"0001111 0","1110
00>                                       30 30 30 30 22 0d 0a                             |0000"..          
00> [00:14:56.583,587] <dbg> slm_mqtt.mqtt_evt_handler: PINGRESP packet
00> [00:14:56.583,648] <dbg> slm_at_host: TX
00>                                       0d 0a 23 58 4d 51 54 54  45 56 54 3a 20 39 2c 30 |..#XMQTT EVT: 9,0
00>                                       0d 0a                                            |..               
00> [00:15:03.542,541] <dbg> slm_at_host: TX
00>                                       0d 0a                                            |..               
00> [00:15:03.542,602] <dbg> slm_at_host: TX
00>                                       2b 43 45 52 45 47 3a 20  35 2c 22 30 32 36 46 22 |+CEREG:  5,"026F"
00>                                       2c 22 30 30 33 33 32 35  32 31 22 2c 37 2c 2c 2c |,"003325 21",7,,,
00>                                       22 30 30 30 31 31 31 31  30 22 2c 22 31 31 31 30 |"0001111 0","1110
00>                                       30 30 30 30 22 0d 0a                             |0000"..          
00> [00:15:04.019,592] <dbg> slm_at_host: RX
00>                                       41 54 23 58 4d 51 54 54  50 55 42 3d 22 52 69 67 |AT#XMQTT PUB="Rig
00>                                       65 6c 2f 54 6f 53 65 72  76 65 72 2f 76 32 2f 41 |el/ToSer ver/v2/A
00>                                       52 5f 54 45 53 54 22 2c  22 22 2c 31             |R_TEST", "",1    
00> [00:15:04.020,080] <dbg> slm_at_host.uart_callback: RX_DISABLED
00> [00:15:04.020,355] <inf> slm_at_host: Enter datamode
00> [00:15:26.313,659] <dbg> slm_at_host.uart_callback: RX_RDY 256
00> [00:15:26.313,781] <inf> slm_at_host: Raw send 256
00> [00:15:26.313,781] <dbg> slm_at_host: RX-DATAMODE
00>                                       7b 22 64 65 76 69 63 65  49 64 22 3a 22 22 2c 22 |{"device Id":"","
00> [00:15:26.321,624] <inf> slm_mqtt: datamode send: 0
00> [00:15:26.324,859] <dbg> slm_at_host.uart_callback: RX_RDY 102
00> [00:15:26.324,920] <inf> slm_at_host: Raw send 102
00> [00:15:26.324,920] <dbg> slm_at_host: RX-DATAMODE
00>                                       61 74 75 73 22 3a 22 55  6e 6b 6e 6f 77 6e 22 2c |atus":"U nknown",
00> [00:15:26.326,446] <inf> slm_mqtt: datamode send: 0
00> [00:15:27.400,177] <dbg> slm_mqtt.mqtt_evt_handler: PUBACK packet id: 1
00> [00:15:27.400,238] <dbg> slm_at_host: TX
00>                                       0d 0a 23 58 4d 51 54 54  45 56 54 3a 20 33 2c 30 |..#XMQTT EVT: 3,0
00>                                       0d 0a                                            |..               
00> [00:15:49.309,478] <dbg> slm_at_host.uart_callback: RX_RDY 154
00> [00:15:49.309,539] <inf> slm_at_host: Raw send 154
00> [00:15:49.309,570] <dbg> slm_at_host: RX-DATAMODE
00>                                       7b 22 64 65 76 69 63 65  49 64 22 3a 22 22 2c 22 |{"device Id":"","
00> [00:15:49.317,321] <inf> slm_mqtt: datamode send: 0
00> [00:15:49.330,718] <dbg> slm_at_host.uart_callback: RX_RDY 204
00> [00:15:49.330,780] <inf> slm_at_host: Raw send 204
00> [00:15:49.330,780] <dbg> slm_at_host: RX-DATAMODE
00>                                       39 2c 22 72 65 63 6f 72  64 65 64 54 69 6d 65 22 |9,"recor dedTime"
00> [00:15:49.332,427] <inf> slm_mqtt: datamode send: 0
00> [00:15:50.129,058] <dbg> slm_mqtt.mqtt_evt_handler: PUBACK packet id: 1
00> [00:15:50.129,119] <dbg> slm_at_host: TX
00>                                       0d 0a 23 58 4d 51 54 54  45 56 54 3a 20 33 2c 30 |..#XMQTT EVT: 3,0
00>                                       0d 0a                                            |..

RE: nRF9160, Sending messages via MQTT larger than 256 bytes

Didrik Rokhaug — Wed, 09 Mar 2022 16:55:35 GMT

Hi,

The SLM should also output log over RTT. Could you provide that log as well, so that we can see more of what the application is doing, and where the split happens?

Best regards,

Didrik

RE: nRF9160, Sending messages via MQTT larger than 256 bytes

Stas Jis — Wed, 09 Mar 2022 16:13:34 GMT

Markus,

I changed the value to "#define MQTT_MESSAGE_BUFFER_LEN 2048" and flash the board.
However, this had a different effect, the messages still do not arrive in their entirety, but only in chunks (of an even smaller size), with data loss.
Original JSON:

{"deviceId":"","deviceType":"Collar","requestTime":0,"carrier":"CELLULAR","rssi":-70,"messages":[{"messageId":536875008,"messageCategory":1,"messageType":9,"recordedTime":0,"firmwareVersion":"collar-fw-0.1.3","bootVersion":"collar-boot-0.0.1","attachmentStatus":"Unknown","battery":0,"chargeStatus":0,"operationMode":0,"debugMode":0,"sos":0,"safeZone":0}]}

Result on the server:

1 Message:
{"deviceId":"","deviceType":"Collar","requestTime":0,"carrier":"CELLULAR","rssi":-70,"messages":[{"messageId":536875008,

2 Message
1","attachmentStatus":"Unknown","battery":0,"chargeStatus":0,"operationMode":0,"debugMode":0,"sos":0,"safeZone":0}]}

RE: nRF9160, Sending messages via MQTT larger than 256 bytes

Markus Tacker (he/him) — Wed, 09 Mar 2022 14:04:57 GMT

Looking at the source, `MQTT_MESSAGE_BUFFER_LEN` needs to be set to a larger value.

#define MQTT_MESSAGE_BUFFER_LEN	2000

RE: nRF9160, Sending messages via MQTT larger than 256 bytes

Stas Jis — Wed, 09 Mar 2022 13:58:27 GMT

Markus,
We are working with SDK 1.8 where this configuration is not defined.

#define MQTT_MAX_TOPIC_LEN	128
#define MQTT_MAX_CID_LEN	64
#define MQTT_MESSAGE_BUFFER_LEN	NET_IPV4_MTU

Need to upgrade to a different SDK version? Or just adding "MQTT_MAX_PUB_LEN" fixes the situation.

RE: nRF9160, Sending messages via MQTT larger than 256 bytes

Markus Tacker (he/him) — Wed, 09 Mar 2022 13:11:45 GMT

There is a hardcoded limit in the source code. You can increase that to suit your needs.