Using serial_lte_modem pass through mode

[Randall]

I need to send bulk data to an MQTT topic on a server hosted by AWS.  I'm trying to use datamode option the serial_lte_modem sample but I'm running in to lots of problems.

I connect successfully to the broker via the XMQTTCON command.  Then I send the;

AT#XMQTTPUB="<topic-obscured>","",0

When I follow this with bulk data, even with small chunks like 32-bytes, I get things like;

[00:01:13.392,059] <inf> slm_at_host: Enter datamode
[00:01:13.480,804] <wrn> slm_at_host: data buffer full (480)
[00:01:13.481,384] <inf> slm_at_host: Raw send 32
[00:01:13.490,692] <inf> slm_mqtt: datamode send: 0
[00:01:13.676,086] <wrn> slm_at_host: data buffer full (480)
[00:01:13.676,666] <inf> slm_at_host: Raw send 32
[00:01:13.677,581] <inf> slm_mqtt: datamode send: 0
[00:01:14.673,645] <wrn> slm_at_host: data buffer full (509)
[00:01:14.674,255] <inf> slm_at_host: Raw send 3
[00:01:14.675,201] <inf> slm_mqtt: datamode send: 0

After that the interface is pretty messed up.  I can't even send the termination sequence to get out of it. 

when I look at the code I see in raw_rx_handler

/* Second, save data to buffer */
ret = ring_buf_put(&data_rb, data, datalen);
if (ret != datalen) {
LOG_ERR("enqueue data error (%d, %d)", datalen, ret);
uart_rx_disable(uart_dev);
return -1;
}
ret = ring_buf_space_get(&data_rb);
if (ret < UART_RX_LEN ) {
LOG_WRN("data buffer full (%d)", ret);
uart_rx_disable(uart_dev);
return -1;
}

it seems like right after data is but in the ring buffer, it has to read empty or the uart_rx_disable is called.  I tried to modify this, but it didn't really help.  

So I guess I want to know if there might be a problem here, or if I'm potentially using datamode wrong and need to modify.

[Randall]

Yeah, I guess UART_RX_LEN was changed from 256 to 512.  Not sure why that makes a difference.

I'm working on increasing the other buffer sizes so that I can get a very large MQTT publish (~60 K) to go.

Based on some other posts I saw, I just moved the the stack to TLS natively rather than from the modem because of the 2K byte limit.  But I'm still getting errors.  

I'm actually working on a different approach now. Since the system system to quadruple any MQTT buffers it uses,  I've modified the AT#XMQTTPUB command so that if the msg parameter is an integer, it's going to indicate the a storage buffer to be sent up as raw data.  I'm going to add other commands to build up these storage locations in flash.  So while I'm still curious about datamode, I've moved on to a different approach based on other learnings .  But rather than convert this ticket to another question.  Should we close this and I can start asking questions about my other approach because I still appear to be hitting some buffer limits that I can't find.

[Håkon Alseth]

Hi,

 

It sounds like you are pushing more data than the LTE link can handle.

It could be beneficial to adjust these configs on your end:

CONFIG_NRF_MODEM_LIB_HEAP_SIZE=4096

CONFIG_NRF_MODEM_LIB_SENDMSG_BUF_SIZE=2048

 

If you are sending data in bursts, these will allow you to buffer up a bit more.

Randall said:

I'm actually working on a different approach now. Since the system system to quadruple any MQTT buffers it uses,  I've modified the AT#XMQTTPUB command so that if the msg parameter is an integer, it's going to indicate the a storage buffer to be sent up as raw data.

Have you added any retry logic here? If you are sending data faster than the network can handle, you'll need to retry the send procedure.

 

Kind regards,

Håkon