This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

nRF9160, Sending messages via MQTT larger than 256 bytes

We use nRF9160 on our own board.
The firmware "Serial lte modem" is installed on the chip with connection to an external MCU.
We need to publish the JSON messages larger than 256 bytes (up to 2000 bytes) to the broker.
We transmit data in "Data Mode" and receive confirmation to publish messages.
For the test we are sending 356 bytes, here is the AT command sequence:

2022-03-09T08:49:53.339Z INFO Modem port is opened
2022-03-09T08:49:54.456Z DEBUG modem >> AT
2022-03-09T08:49:54.471Z DEBUG modem << OK
2022-03-09T08:49:55.439Z DEBUG modem >> AT+CFUN?
2022-03-09T08:49:55.453Z DEBUG modem << +CFUN: 0
2022-03-09T08:49:55.462Z DEBUG modem << OK
2022-03-09T08:49:58.061Z INFO Updating CA certificate...
2022-03-09T08:49:58.063Z DEBUG modem >> AT%CMNG=0,321,0,"-----BEGIN CERTIFICATE-----

2022-03-09T08:49:58.343Z DEBUG modem >> -----END CERTIFICATE-----"
2022-03-09T08:49:58.499Z DEBUG modem << OK
2022-03-09T08:49:58.500Z INFO Updating client certificate...
2022-03-09T08:49:58.502Z DEBUG modem >> AT%CMNG=0,321,1,"-----BEGIN CERTIFICATE-----

2022-03-09T08:49:58.702Z DEBUG modem >> z6s4x54=
2022-03-09T08:49:58.715Z DEBUG modem >> -----END CERTIFICATE-----"
2022-03-09T08:50:00.457Z DEBUG modem << OK
2022-03-09T08:50:00.459Z INFO Updating private key...
2022-03-09T08:50:00.460Z DEBUG modem >> AT%CMNG=0,321,2,"-----BEGIN EC PRIVATE KEY-----

2022-03-09T08:50:00.512Z DEBUG modem >> -----END EC PRIVATE KEY-----"
2022-03-09T08:50:00.711Z DEBUG modem << OK
2022-03-09T08:50:00.712Z INFO Certificate update completed
2022-03-09T08:50:21.135Z DEBUG modem >> AT+CEREG=5
2022-03-09T08:50:21.148Z DEBUG modem << OK
2022-03-09T08:50:24.911Z DEBUG modem >> AT+CFUN=1
2022-03-09T08:50:24.952Z DEBUG modem << OK
2022-03-09T08:50:25.923Z DEBUG modem << +CEREG: 2,"026F","00332521",7
2022-03-09T08:50:28.862Z DEBUG modem << +CEREG: 5,"026F","00332521",7,,,"00011110","11100000"
2022-03-09T08:50:35.864Z DEBUG modem >> AT#XMQTTCON=1,"999876","","","a34k7wa09ujucc-ats.iot.us-east-1.amazonaws.com",8883,321
2022-03-09T08:50:36.869Z ERROR Error: 'AT#XMQTTCON=1,"999876","","","a34k7wa09ujucc-ats.iot.us-east-1.amazonaws.com",8883,321
' timed out
2022-03-09T08:50:38.506Z DEBUG modem << OK
2022-03-09T08:50:38.810Z DEBUG modem << #XMQTTEVT: 0,0
2022-03-09T08:50:57.167Z DEBUG modem >> AT#XMQTTPUB="Rigel/ToServer/v2/AR_TEST","",1
2022-03-09T08:51:08.472Z DEBUG modem >> {"deviceId":"","deviceType":"Collar","requestTime":0,"carrier":"CELLULAR","rssi":-70,"messages":[{"messageId":536875008,"messageCategory":1,"messageType":9,"recordedTime":0,"firmwareVersion":"collar-fw-0.1.3","bootVersion":"collar-boot-0.0.1","attachmentStatus":"Unknown","battery":0,"chargeStatus":0,"operationMode":0,"debugMode":0,"sos":0,"safeZone":0}]}
2022-03-09T08:51:09.258Z DEBUG modem << #XMQTTEVT: 3,0
2022-03-09T08:52:08.971Z DEBUG modem << #XMQTTEVT: 9,0

But from the server side, we always receive messages with a maximum length of 256 bytes (truncated):
{"deviceId":"","deviceType":"Collar","requestTime":0,"carrier":"CELLULAR","rssi":-70,"messages":[{"messageId":536875008,"messageCategory":1,"messageType":9,"recordedTime":0,"firmwareVersion":"collar-fw-0.1.3","bootVersion":"collar-boot-0.0.1","attachmentSt

There are no problems when transmitting a message via UART, we record everything that we transmit via "Logic" App, and clearly saw that the message via UART was completely gone.
Plus, we also saw the same problem when we subscribe to the topic. If we wait to receive messages larger than 256 bytes, it comes truncated and after we receive event for a read error and a disconnect.
2022-03-08T15:29:20.381Z DEBUG modem >> AT#XMQTTSUB="Rigel/ToServer/v2/AR_TEST",0
2022-03-08T15:29:21.134Z DEBUG modem << OK
2022-03-08T15:29:21.612Z DEBUG modem << #XMQTTEVT: 7,0
2022-03-08T15:29:24.096Z DEBUG modem << Rigel/ToServer/v2/AR_TEST
2022-03-08T15:29:31.789Z DEBUG modem << {"deviceId":"","deviceType":"Collar","requestTime":0,"carrier":"CELLULAR","rssi":-70,"messages":[{"messageId":536875008,"messageCategory":1,"messageType":9,"recordedTime":0,"firmwareVersion":"collar-fw-0.1.3","bootVersion":"collar-boot-0.0.1","attachmentSt#XMQTTEVT: 2,0#XMQTTMSG: 29,256Rigel/ToServer/v2/Operational{"deviceId":"","deviceType":"Collar","requestTime":0,"carrier":"CELLULAR","rssi":-70,"messages":[{"messageId":536875008,"messageCategory":1,"messageType":9,"recordedTime":0,"firmwareVersion":"collar-fw-0.1.3","bootVersion":"collar-boot-0.0.1","attachmentSt
2022-03-08T15:29:44.526Z DEBUG modem << #XMQTTEVT: 2,0
2022-03-08T15:29:44.689Z DEBUG modem << #XMQTTEVT: 2,-122
2022-03-08T15:29:44.700Z DEBUG modem << #XMQTTEVT: 1,-113

Parents
  • There is a hardcoded limit in the source code. You can increase that to suit your needs.

  • Markus,

    I changed the value to "#define MQTT_MESSAGE_BUFFER_LEN 2048" and flash the board.
    However, this had a different effect, the messages still do not arrive in their entirety, but only in chunks (of an even smaller size), with data loss.
    Original JSON:

    {"deviceId":"","deviceType":"Collar","requestTime":0,"carrier":"CELLULAR","rssi":-70,"messages":[{"messageId":536875008,"messageCategory":1,"messageType":9,"recordedTime":0,"firmwareVersion":"collar-fw-0.1.3","bootVersion":"collar-boot-0.0.1","attachmentStatus":"Unknown","battery":0,"chargeStatus":0,"operationMode":0,"debugMode":0,"sos":0,"safeZone":0}]}


    Result on the server:
    1 Message:
    {"deviceId":"","deviceType":"Collar","requestTime":0,"carrier":"CELLULAR","rssi":-70,"messages":[{"messageId":536875008,
    
    2 Message
    1","attachmentStatus":"Unknown","battery":0,"chargeStatus":0,"operationMode":0,"debugMode":0,"sos":0,"safeZone":0}]}


  • Hi,

    The SLM should also output log over RTT. Could you provide that log as well, so that we can see more of what the application is doing, and where the split happens?

    Best regards,

    Didrik

Reply Children
  • Didrik,
    I am attaching logs from RTT Viewer:

    00>                                       41 54 2b 43 45 52 45 47  3d 35                   |AT+CEREG =5      
    00> [00:13:28.292,175] <dbg> slm_at_host.uart_callback: RX_DISABLED
    00> [00:13:28.298,309] <dbg> slm_at_host: TX
    00>                                       0d 0a                                            |..               
    00> [00:13:28.298,370] <dbg> slm_at_host: TX
    00>                                       4f 4b 0d 0a                                      |OK..             
    00> [00:13:38.572,906] <dbg> slm_at_host: RX
    00>                                       41 54 2b 43 46 55 4e 3d  31                      |AT+CFUN= 1       
    00> [00:13:38.573,394] <dbg> slm_at_host.uart_callback: RX_DISABLED
    00> [00:13:38.609,130] <dbg> slm_at_host: TX
    00>                                       0d 0a                                            |..               
    00> [00:13:38.609,802] <dbg> slm_at_host: TX
    00>                                       4f 4b 0d 0a                                      |OK..             
    00> [00:13:39.660,369] <dbg> slm_at_host: TX
    00>                                       0d 0a                                            |..               
    00> [00:13:39.660,430] <dbg> slm_at_host: TX
    00>                                       2b 43 45 52 45 47 3a 20  32 2c 22 30 32 36 46 22 |+CEREG:  2,"026F"
    00>                                       2c 22 30 30 33 33 32 35  32 31 22 2c 37 0d 0a    |,"003325 21",7.. 
    00> [00:13:42.765,625] <dbg> slm_at_host: TX
    00>                                       0d 0a                                            |..               
    00> [00:13:42.765,686] <dbg> slm_at_host: TX
    00>                                       2b 43 45 52 45 47 3a 20  35 2c 22 30 32 36 46 22 |+CEREG:  5,"026F"
    00>                                       2c 22 30 30 33 33 32 35  32 31 22 2c 37 2c 2c 2c |,"003325 21",7,,,
    00>                                       22 30 30 30 31 31 31 31  30 22 2c 22 31 31 31 30 |"0001111 0","1110
    00>                                       30 30 30 30 22 0d 0a                             |0000"..          
    00> [00:13:50.564,025] <dbg> slm_at_host: RX
    00>                                       41 54 23 58 4d 51 54 54  43 4f 4e 3d 31 2c 22 39 |AT#XMQTT CON=1,"9
    00>                                       39 39 38 37 36 22 2c 22  22 2c 22 22 2c 22 61 33 |99876"," ","","a3
    00>                                       34 6b 37 77 61 30 39 75  6a 75 63 63 2d 61 74 73 |4k7wa09u jucc-ats
    00>                                       2e 69 6f 74 2e 75 73 2d  65 61 73 74 2d 31 2e 61 |.iot.us- east-1.a
    00>                                       6d 61 7a 6f 6e 61 77 73  2e 63 6f 6d 22 2c 38 38 |mazonaws .com",88
    00>                                       38 33 2c 33 32 31                                |83,321           
    00> [00:13:50.564,514] <dbg> slm_at_host.uart_callback: RX_DISABLED
    00> [00:13:56.083,831] <dbg> slm_at_host: TX
    00>                                       0d 0a 4f 4b 0d 0a                                |..OK..           
    00> [00:13:56.369,232] <dbg> slm_at_host: TX
    00>                                       0d 0a 23 58 4d 51 54 54  45 56 54 3a 20 30 2c 30 |..#XMQTT EVT: 0,0
    00>                                       0d 0a                                            |..               
    00> [00:14:00.742,980] <dbg> slm_at_host: TX
    00>                                       0d 0a                                            |..               
    00> [00:14:00.743,041] <dbg> slm_at_host: TX
    00>                                       2b 43 45 52 45 47 3a 20  35 2c 22 30 32 36 46 22 |+CEREG:  5,"026F"
    00>                                       2c 22 30 30 33 33 32 35  32 30 22 2c 37 2c 2c 2c |,"003325 20",7,,,
    00>                                       22 30 30 30 31 31 31 31  30 22 2c 22 31 31 31 30 |"0001111 0","1110
    00>                                       30 30 30 30 22 0d 0a                             |0000"..          
    00> [00:14:56.583,587] <dbg> slm_mqtt.mqtt_evt_handler: PINGRESP packet
    00> [00:14:56.583,648] <dbg> slm_at_host: TX
    00>                                       0d 0a 23 58 4d 51 54 54  45 56 54 3a 20 39 2c 30 |..#XMQTT EVT: 9,0
    00>                                       0d 0a                                            |..               
    00> [00:15:03.542,541] <dbg> slm_at_host: TX
    00>                                       0d 0a                                            |..               
    00> [00:15:03.542,602] <dbg> slm_at_host: TX
    00>                                       2b 43 45 52 45 47 3a 20  35 2c 22 30 32 36 46 22 |+CEREG:  5,"026F"
    00>                                       2c 22 30 30 33 33 32 35  32 31 22 2c 37 2c 2c 2c |,"003325 21",7,,,
    00>                                       22 30 30 30 31 31 31 31  30 22 2c 22 31 31 31 30 |"0001111 0","1110
    00>                                       30 30 30 30 22 0d 0a                             |0000"..          
    00> [00:15:04.019,592] <dbg> slm_at_host: RX
    00>                                       41 54 23 58 4d 51 54 54  50 55 42 3d 22 52 69 67 |AT#XMQTT PUB="Rig
    00>                                       65 6c 2f 54 6f 53 65 72  76 65 72 2f 76 32 2f 41 |el/ToSer ver/v2/A
    00>                                       52 5f 54 45 53 54 22 2c  22 22 2c 31             |R_TEST", "",1    
    00> [00:15:04.020,080] <dbg> slm_at_host.uart_callback: RX_DISABLED
    00> [00:15:04.020,355] <inf> slm_at_host: Enter datamode
    00> [00:15:26.313,659] <dbg> slm_at_host.uart_callback: RX_RDY 256
    00> [00:15:26.313,781] <inf> slm_at_host: Raw send 256
    00> [00:15:26.313,781] <dbg> slm_at_host: RX-DATAMODE
    00>                                       7b 22 64 65 76 69 63 65  49 64 22 3a 22 22 2c 22 |{"device Id":"","
    00> [00:15:26.321,624] <inf> slm_mqtt: datamode send: 0
    00> [00:15:26.324,859] <dbg> slm_at_host.uart_callback: RX_RDY 102
    00> [00:15:26.324,920] <inf> slm_at_host: Raw send 102
    00> [00:15:26.324,920] <dbg> slm_at_host: RX-DATAMODE
    00>                                       61 74 75 73 22 3a 22 55  6e 6b 6e 6f 77 6e 22 2c |atus":"U nknown",
    00> [00:15:26.326,446] <inf> slm_mqtt: datamode send: 0
    00> [00:15:27.400,177] <dbg> slm_mqtt.mqtt_evt_handler: PUBACK packet id: 1
    00> [00:15:27.400,238] <dbg> slm_at_host: TX
    00>                                       0d 0a 23 58 4d 51 54 54  45 56 54 3a 20 33 2c 30 |..#XMQTT EVT: 3,0
    00>                                       0d 0a                                            |..               
    00> [00:15:49.309,478] <dbg> slm_at_host.uart_callback: RX_RDY 154
    00> [00:15:49.309,539] <inf> slm_at_host: Raw send 154
    00> [00:15:49.309,570] <dbg> slm_at_host: RX-DATAMODE
    00>                                       7b 22 64 65 76 69 63 65  49 64 22 3a 22 22 2c 22 |{"device Id":"","
    00> [00:15:49.317,321] <inf> slm_mqtt: datamode send: 0
    00> [00:15:49.330,718] <dbg> slm_at_host.uart_callback: RX_RDY 204
    00> [00:15:49.330,780] <inf> slm_at_host: Raw send 204
    00> [00:15:49.330,780] <dbg> slm_at_host: RX-DATAMODE
    00>                                       39 2c 22 72 65 63 6f 72  64 65 64 54 69 6d 65 22 |9,"recor dedTime"
    00> [00:15:49.332,427] <inf> slm_mqtt: datamode send: 0
    00> [00:15:50.129,058] <dbg> slm_mqtt.mqtt_evt_handler: PUBACK packet id: 1
    00> [00:15:50.129,119] <dbg> slm_at_host: TX
    00>                                       0d 0a 23 58 4d 51 54 54  45 56 54 3a 20 33 2c 30 |..#XMQTT EVT: 3,0
    00>                                       0d 0a                                            |..   

  • Could you try with NCS v1.9.0?

    And how are you sending the JSON data to the nRF9160?

    When using NCS v1.9.0, I am able to send 575 bytes (which matches MQTT_MESSAGE_BUFFER_LEN = NET_IPV4_MTU = 576 pretty well. The last byte is probably a newline which isn't displayed by my desktop MQTT client), when I paste it all into minicom at once. However, if I write manually into minicom, the device sends each letter by itself.

  • Didrik,
    We tried to send JSON to SDK version 1.9, everything worked, JSON was completely received. Thanks for the help.

Related