nRF9160 send larger MQTT payloads than 2048 bytes to save battery power (need to send 200KB)

[Johan]

Hi,

I want to optimize the transmission efficiency and thinking of sending larger packets than 2048 bytes of MQTT would help.

We have limited battery power, and looking at the MQTT traffic towards AWS I see that this is happening:

1) MQTT-packet is sent.

2) Waiting for response (several seconds)

3) Receiving an ACK.

4) goto 1 (is the fastest way I can do it now, send the next packet of max 2048 bytes).

If I can change the max mqtt packet size to for instance 16K, that would mean an improvement I'm guessing.

Changing it though, makes the example code crash, going somewhere into non program space. It doesn't seem to allow changing of that size, even though it's available in the menu <Project -> configure nRF SDK...etc -> searching for MQTT -> finding maximum MQTT PACKET SIZE set to 2048>

I tried 4096, 8192, 16384.

I get for bigger sizes, crash in the code.

For smaller increases like 4096, I get an error from the socket_write instead, so it's then in the modem software I'm guessing.

I really want to send bigger packages, to improve battery life.

Who can help with this? 

IS this bugs or crash by design, or fail by design, even though the MQTT PACKET SIZE is allowed to be changed by design?

Johan

[Martin Lesund]

Hi Johan,

It is a bug.

The TLS output buffer has a size of 2kB, and there is a bug that you do not get a notification that the messages are too big. (this will be fixed)

So the solution is to have shorter messages.

Best Regards,

Martin  L,.

[Johan]

Ok, looking forward to that bugfix.

Will you enable us to send larger packets later? Is there a limitation somewhere? Is there power to save to send more in a bigger chunk, than to send them separately?

So, how to get up speed when needing to send a large data buffer?

I'm thinking to split the big chunk of 300KB (example size), into like you suggest, smaller parts

* send part 1

* send part 2

...

* send part n

immediately after eachother.

There is no need to wait for the ACK for each part before sending the next I'm guessing.

[Johan]

I tried this and result was:

Trial 1: sent 2 packets of 1990 bytes.
Packet 1, sent ok.
Packet 2, sent ok.
Response came in, for packet 1, then for packet 2.
OK!

--------------

Trial 2: Sent 10 packets of 1990 bytes

Packet 1, sent ok.
Packet 2, sent ok.
PAcket 3, ---> mqtt.c:client_disconnect() -> MQTT_EVT_DISCONNECT (-95) 
Errno lookup gives this.

5

EIO

I/O error

Packet 4..10 - same failure.
Errno lookup gives this.

128

EKEYREVOKED

Key has been revoked

So there is no support for sending multiple packets in sequence, more than 2? Bug again or limitation of some kind?

I need to bring sending speed here, and optimize for transmission. Otherwise I get 300bytes/sec, by waiting for each ACK before I can send the next packet.

[Martin Lesund]

Hi Johan,

I would highly recommend that you test out the newest NCS v1.0.0-rc3 to see if that changes the behavior since there have been several fixes implemented.

cd ncs/nrf
git checkout master
git pull
git checkout v1.0.0-rc3
west update

Then rebuild and try your solution.

Also make sure to have the newest modem firmware: https://www.nordicsemi.com/Products/Low-power-cellular-IoT/nRF9160/Download#infotabs

Best regards,

Martin L.

[Johan]

I flashed the new modem firmware you sent me via email.
I tried it with first the v0.4.0 and it worked with 2048 buffer size like before (asset_tracker example, modified for our own cloud with our own keys)

I tried with buffer size 4096 and 8192. They both seem to go into writing - via the modem without an error,
but there is no response at all from the network. SUBACK is what I'm expecting via the MQTT-drivers.
Checking the cloud, there is no data uploaded either, so still only 2048 bytes works.

Using instead v1.0.0-rc3, and the standard unmodified asset_tracker, I get 

```

***** Booting Zephyr OS v1.14.99-ncs2-rc4 *****

Flash region     Domain     Per***** Hardware exception *****

Current thread ID = 0x20001870

Faulting instruction address = 0x30100

Fatal fault in ISR! Spinning...

```

Note how I'm building and uploading.
1. Building from menu, "Build Solution".
2. Menu "Target -> Connect J-Link".
3. Menu "Target-> Download file -> Download Intel Hex File -> and choosing the merged.hex

This is from how you told me earlier.

Perhaps something has changed again, so this way of flashing and running is wrong?

Johan