Receiving MQTT Shadow Data before NRF_CLOUD_EVT_READY

[egood]

Hello,

I'm currently debugging an issue we are having with MQTT device shadow interaction on the nrf9160. The issue presents when we push a large amount of config JSON to the device shadow. This typically happens when we are setting up a new device for the first time and need to set all the parameters in the "config" object via the REST API. The issue is when this large delta occurs the device will try and sync it via MQTT and it gets stuck in a connect/disconnect loop. 

I'm currently looking at once specific case that I can get to repeat multiple times. Where the following events occur in sequence:

1. NRF_CLOUD_EVT_TRANSPORT_CONNECTED occurs 

2. NRF_CLOUD_EVT_RX_DATA_SHADOW occurs with the large delta (in my case about 1.3Kb)

3.  NRF_CLOUD_EVT_USER_ASSOCIATED occurs

4.  NRF_CLOUD_EVT_TRANSPORT_DISCONNECTED occurs for reason  -> NRF_CLOUD_DISCONNECT_MISC

5. The LTE connection is reset (not a reboot) and it attempts to connect again and goes through the same sequence.

Now the device never gets a chance to send the shadow delta back up to the cloud because our firmware waits until NRF_CLOUD_EVT_READY before we send any data back to nrfCloud. And because we don't acknowledge the shadow delta it gets sent again the next time we connect, resulting in an endless loop.  

The question that comes to mind is why we are getting shadow data before receiving, NRF_CLOUD_EVT_READY? The documentation here seems to imply this shouldn't happen?

Any insight on how to debug this further would be greatly appreciated.

Thanks!

[Charlie]

Hi Eric,

According to our nRF Cloud developers, AWS IoT does not support TLS MFLN now, and they limit their TLS records to 8192 bytes, so splitting the JSON shadow from nRF Cloud side is not an option now. There are two potential options that may solve your request.

1. One solution is that you can use REST for the shadow operation. There is a way to filter the server response: https://docs.nrfcloud.com/APIs/REST/Tutorials/Transforms.html

2. Generate a new custom shadow topic

The device subscribes to "trimmed" shadow according to https://docs.nrfcloud.com/APIs/MQTT/Topics.html#shadow-topics.

The following items are sent to the trimmed shadow topics:

desired.pairing

desired.nrfcloud_mqtt_topic_prefix

desired.control

reported.control

reported.config

so by default, the entire shadow is not sent to the device (if using nrf_cloud fw library) when it first connects. The config section is user/application defined. nRF Cloud just provides a UI card to edit it.  Current Assert Tracker V2 is the only app/sample that uses it. Putting most/all of your config data in the config section should be appropriate but in your case, due to its size, perhaps using a section that is not in the config section would be helpful.  You can create some other section that is not part of the trimmed shadow. This suggestion would prevent the issue from occurring when the device is doing the inital mqtt connection handshaking (since that is when the trimmed shadow is sent to the device). When you update custom defined section via REST, AWS will issue a delta event to the device, then the device would be able to get detla update that contains your custom section with settings

Please consider the above suggestions and feel free to ask if you need more help.

Best regards,

Charlie