Sending A large binary payload via MQTT

[Randall]

Hi, I'm using the serial_lte_modem sample application and I've modified the AT#XMQTTPUB command so that I can specify a slot number from which the device can grab a bank of binary data and send it up to an aws server.  For testing, I just have a predefined bank of binary data and I use the slot number to determine how much of the buffer is sent;

AT#XMQTTPUB=<topic>,<msg>[,<qos>[,<retain>]].  So in my implementation, msg is required, it can be a string, then that message is published, it can be "" or 0, in which case the unit goes into datamode, or it can be a number, which refers to a slot.  In the future, I'm going to be adding AT# functions to populate these slots with binary data, but for now, I have a fixed set of binary data and I use the slot number to indicate how many bytes of that data I will send (just for testing purposes);

if (strlen(pub_msg) == 0)
{
if ( binSlot == 0 )
{
/* Publish payload in data mode */
err = enter_datamode(mqtt_datamode_callback);
}
else
{
err = do_mqtt_publish(pub_bin, binSlot );
}
} else {
err = do_mqtt_publish(pub_msg, msg_sz);
}

OK, I've enabled CONFIG_NRF91_SOCKET_SEND_SPLIT_LARGE_BLOCKS so that I get around the 2k byte limit of TLS.  but I'm still having problems going over 6047 bytes.  Up to that point, the data goes over fine.  Once I got 6048 and beyond I get;

AT#XMQTTPUB="<topic obscured>",6048,0

#XMQTTEVT: 1,-95

out the debug port i get; [00:00:48.454,010] <err> slm_mqtt: POLLNVAL, which I assume because it's still trying to do transmissions on a closed socket.

To facilitate the large transmission size I've added 

#define MQTT_TX_BUFFER_LEN (64*1024)
#define MQTT_MESSAGE_BUFFER_LEN NET_IPV4_MTU

to create a special buffer for the MQTT Transmission;

static uint8_t tx_buffer[MQTT_TX_BUFFER_LEN];

I feel like there are other buffers I need to increase to avoid the EOPNOTSUPP, but I can't seem to find it.

I'm using SDK 1.8.0

[Randall]

Update:

I've discovered that if I add the commented out k_sleep shown below;

offset = 0;
flags |= NRF_MSG_WAITALL;
while (offset < msg->msg_iov[i].iov_len) {
for( retries = 0; retries < 5; retries++ )
{
ret = nrf91_socket_offload_sendto(obj,
(((uint8_t *) msg->msg_iov[i].iov_base) + offset),
(msg->msg_iov[i].iov_len - offset), flags,
msg->msg_name, msg->msg_namelen);
if ( ret >= 0 ) break;
k_sleep(K_MSEC( 500 + (retries * 500) ) );
}
if (ret < 0) {
printf( "Transmission Failed\n" );
return ret;
}
printf( "offset = %lu, len = %lu, ret = %lu\n", offset, len, ret );
offset += ret;
len += ret;
//if ( offset < msg->msg_iov[i].iov_len ) k_sleep(K_MSEC(500) );
}

it works.  But this seems rather inefficient.  So I added the flags |= NRF_MSG_WAITALL.  This is much more efficient but occasionally fails in 2 different ways.  1 returns failed with errno set to EOPNOTSUPP or it simply locks up and seems to never return (at least not during the time period I was willing to wait).  Now I could make this work with some extrra higher level error handling.   But I was hoping that there was a better way.

[Randall]

OK.  Made a small mistake.  It turns out that the flag change didn't help anything.  I forgot I still had the retries in there.  So it was actually them making it work, with the 2 failure conditions.

[Randall]

OK, I should probably make it clear what I'm asking for here.  With the extra 500 MSec at the end of the loop, the system seems to work pretty reliably, but is slow (takes about 15 seconds up upload  64k).  Without the 500 Msec but only the retires, it sometimes works faster but fails about 50% of the time, so that isn't acceptable.  I was hoping there was some way to check the socket buffer, and wait until it's empty, then only to the SendTo when it's completely empty.  Is that possible?  Or is there any way to retain the reliability but have it send faster?

[Albrecht Markus Schellenberger]

Hello Randall,

Randall said:

Is that possible?  Or is there any way to retain the reliability but have it send faster?

Thanks for the updates on your progress!

I’m not aware of any way to speed this up. Please note that TLS transmission is limited by hardware to 2kB, so you are already operating with a workaround.

But I’ll check with our developers if they have any input or comments to share.

Regards,

Markus

[Albrecht Markus Schellenberger]

Hello Randall,

could you provide me with a modem trace + full application log from device boot, showing the described issue you are facing? You can preferably use the Trace Collector V2 preview for that.

Thanks & regards,

Markus