Sending A large binary payload via MQTT

[Randall]

Hi, I'm using the serial_lte_modem sample application and I've modified the AT#XMQTTPUB command so that I can specify a slot number from which the device can grab a bank of binary data and send it up to an aws server.  For testing, I just have a predefined bank of binary data and I use the slot number to determine how much of the buffer is sent;

AT#XMQTTPUB=<topic>,<msg>[,<qos>[,<retain>]].  So in my implementation, msg is required, it can be a string, then that message is published, it can be "" or 0, in which case the unit goes into datamode, or it can be a number, which refers to a slot.  In the future, I'm going to be adding AT# functions to populate these slots with binary data, but for now, I have a fixed set of binary data and I use the slot number to indicate how many bytes of that data I will send (just for testing purposes);

if (strlen(pub_msg) == 0)
{
if ( binSlot == 0 )
{
/* Publish payload in data mode */
err = enter_datamode(mqtt_datamode_callback);
}
else
{
err = do_mqtt_publish(pub_bin, binSlot );
}
} else {
err = do_mqtt_publish(pub_msg, msg_sz);
}

OK, I've enabled CONFIG_NRF91_SOCKET_SEND_SPLIT_LARGE_BLOCKS so that I get around the 2k byte limit of TLS.  but I'm still having problems going over 6047 bytes.  Up to that point, the data goes over fine.  Once I got 6048 and beyond I get;

AT#XMQTTPUB="<topic obscured>",6048,0

#XMQTTEVT: 1,-95

out the debug port i get; [00:00:48.454,010] <err> slm_mqtt: POLLNVAL, which I assume because it's still trying to do transmissions on a closed socket.

To facilitate the large transmission size I've added 

#define MQTT_TX_BUFFER_LEN (64*1024)
#define MQTT_MESSAGE_BUFFER_LEN NET_IPV4_MTU

to create a special buffer for the MQTT Transmission;

static uint8_t tx_buffer[MQTT_TX_BUFFER_LEN];

I feel like there are other buffers I need to increase to avoid the EOPNOTSUPP, but I can't seem to find it.

I'm using SDK 1.8.0

[Albrecht Markus Schellenberger]

Hello again Randall,

I have gotten some feedback from our developers as well:

“Please be aware that the modem IP stack has a buffer limit of 4248 byte (MSS 708, max 6 packets). To publish large payload, MQTT Publish should have QoS 1 at least so as to get PUBACK, i.e. some basic uplink flow control. Otherwise there is possibility to overflow the TCP transmission buffer on modem side. My recommendation is to add QoS 1 in MQTT Pub request to monitor the traffic status first. TCP connection could vary for each uplink packet based on network condition.”

I hope this will help you.

Regards,

//Markus

[Randall]

Well, I recompiled my serial_lte_modem project with CONFIG_NRF_MODEM_LIB_TRACE_ENABLED=y, then I used the AT%XMODEMTRACE=1,1 command to enable tracing on the application and AT+CFUN=0 which is my understanding that it should save it to NV RAM.  In the Trace Collector V2 app I chose the debug UART, but nothing seemed to come over.  All the .bin files generated have 0 bytes in them.  So I'm not sure what else I'm supposed to do to get a trace.

[Albrecht Markus Schellenberger]

Hello Randall,

Randall said:

So I'm not sure what else I'm supposed to do to get a trace.

it should be enough to set:

CONFIG_NRF_MODEM_LIB_TRACE_ENABLED=y

I have tested this with the Serial LTE modem application on my nRF9160 DK an it works as expected. Which COM port did you select for recording?

Regards,

Markus