MQTT Quality of Service 1 with the nRF Cloud library

[oddgrd]

Hey! I have been using MQTT with QoS 0 to transmit my sensor readings to nRF Cloud. However, I see data loss regularly, and I expect that is happening between my device and nRF Cloud, since the device is mobile and connects over LTE.

I'd like to upgrade to QoS 1, to see if it resolves my data loss. However, I had some questions about how QoS is handled in the nRF Cloud library, as I have been digging into the source code, but I am a bit new to both nRF devices and C.

1. If I simply send my message with MQTT_QOS_1_AT_LEAST_ONCE, does the library handle checking for PUBACKs with matching IDs and retrying messages that are not eventually ACKed?
2. Either way, is it possible for me to access these PUBACKs from the nRF cloud library events? I see that there is a  `NRF_CLOUD_EVT_SENSOR_DATA_ACK` nrf_cloud_evt type, and the old asset tracker v2 (which is deprecated, but it's the only one I find that shows how to do this) example extracts a message id from that, and then uses the nRF Quality of Service library to handle tracking acknowledgements in the application itself.
   1. The reason I say either way is that I'd like to protect against the case where my device transmits a message, but then reboots before the PUBACK is received. And to do that I need to keep track of whether a message has been acknowledged in non-volatile memory.

I am using the 2.8.0 version of the nRF Connect SDK. Here is my transmission code:

int cloud_transmit_measurement_block(const uint8_t *data, size_t size, const uint8_t *iv) {

    const int ret = encode_measurement_block(json_buffer, sizeof(json_buffer), data, size, iv);

    if (ret < 0) {

        LOG_ERR("Could not encode measurement block, err: %d", ret);

        return ret;

    }

    struct nrf_cloud_tx_data msg = {

        .data =

            {

                .ptr = json_buffer,

                .len = ret,

            },

        .id = sys_rand32_get(),

        .qos = MQTT_QOS_1_AT_LEAST_ONCE,

        .topic_type = NRF_CLOUD_TOPIC_MESSAGE,

    };

    return cloud_transmit(&msg);

}


int cloud_transmit(const struct nrf_cloud_tx_data *msg) {

    const int ret = nrf_cloud_send(msg);

    if (ret == -EACCES || ret == -ENOTCONN) {

        LOG_WRN("Attempted to transmit to cloud without being connected");

        return -ENOTCONN;

    } else if (ret < 0) {

        LOG_ERR("Unhandled error, err: %d", ret);

    }

    return ret;

}


If anyone can point me in the right direction here, it would be greatly appreciated. I am happy to provide more context if needed. Thanks!


[Benjamin]

Hi,

These libraries do not automatically retransmit messages if no PUBACK is received. You need to handle this in the application. The MQTT helper library offer the on_puback callback to handle incoming PUBACKs. You will need to send again after some time.

You can also use MQTT Asynchronous Events that are notified to the application from the module through a callback registered in the application. See MQTT_EVT_PUBACK.

Regards,
Benjamin

[oddgrd]

Thank you, Benjamin! I did see that if I used the Zephyr MQTT library directly I would have access to these events and more control, but I was hoping that I could continue using the nRF Cloud library, as it significantly reduces the complexity of my application. And it seems to me that the MQTT helper library expects an MQTT client, and as I understand it I won't have access to the MQTT client directly when I use the nRF Cloud library.

Do you have a sample application using QoS-1 with the nRF Cloud library, which also listens for PUBACK and retries transmission? I linked one in my post, the `asset_tracker_v2` application, but it seemed quite convoluted, and it is deprecated in the newer versions of the SDK. I also noticed that you don't do it in the same way in the modern version of the example, the multi-service sample, so I assumed that it is not a recommended approach. However, in this sample you also set QoS to 1, but as far as I can tell, there is no logic to handle PUBACKs. To me that seems equivalent to setting QoS 0 when it comes to delivery guarantee, so I assume I am missing something. Is that correct?

Sorry for all the questions, but I've been digging into this for a while, and I struggled to find clear documentation for handling the various MQTT Quality of service settings on the application side. Perhaps the answer is just that I have to live with dropping the nRF cloud library, and drop down a level to the Zephyr MQTT library, as you suggested. I just want to make sure that is what I need to do before I get started, because it's a quite significant change in my firmware.

Edit: looking closer at the MQTT helper library, maybe I don't need to use the Zephyr MQTT library after all, but rather keep using the nRF cloud library for transmissions, and then just use the helper library to hook into PUBACKs. I'll see if that works, thanks!