Sending commands from nRF Cloud to LTE Device

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Fri, 31 Jan 2025 19:01:40 GMT

Lincoln Thomas Thanks, this section is very helpful.

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Fri, 31 Jan 2025 19:01:11 GMT

I am finally getting back to this.

I can confirm that {'t':'state.desired.config'} returns the pending desired changes. This allows us to receive and process these desired changes which the device previously missed because it was offline.

However, to me this seems like a "workaround" because the behavior of shadow/get/accepted/trim appears to be inconsistent with the general semantics of the shadow desired (and delta) functionality.

I would expect that pending changes in the desired:config section are present in the shadow until the device has processed them -- either by reporting the config changes back or by overwriting the desired:config section (if the device is ignoring the changes).

However, the fact that the desired:config section is always trimmed away -- regardless of whether the device has processed the section or not -- is counterintuitive.

This makes device code unnecessarily complex. If the pending changes in the desired:config section would be present until the device processed them, then the handling of online and offline shadow updates could be the same. Maybe I am misunderstanding something.

Thanks again for all the help.

Best,
-- Terrence

RE: Sending commands from nRF Cloud to LTE Device

Lincoln Thomas — Fri, 20 Dec 2024 22:31:39 GMT

The documentation page https://docs.nordicsemi.com/bundle/nrf-cloud/page/APIs/MQTT/Topics.html has been updated to clarify and expand the transforms feature. See the table including {deviceId}/shadow/get/accepted/tf and the examples below that table.

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Wed, 20 Nov 2024 17:14:58 GMT

Tuomas P Thanks, will look into this

RE: Sending commands from nRF Cloud to LTE Device

Tuomas P — Wed, 20 Nov 2024 09:05:22 GMT

Hello teba99

You are right, the nRF Cloud trims the config from the shadow.

There is now a BETA feature in production for you to try out. Unfortunately I have not been able to test this yet, so only "copy and pasting" instructions for you.

There are now new topics:

subscribe "{deviceId}/shadow/get/accepted/tf" to receive the transformed subset of the device’s full shadow (response to `.../get/tf`). See (3)

Publish to "{deviceId}/shadow/get/tf" to request a subset of the device’s full shadow, defined by a transform.

Transformation example

The required 't' property defines the transform string or JSON object.

The optional 'l' property defines the payload size limit (500 bytes in this example):

```
{'t':'state.desired.config', 'l': 500}
```

The transformation is used in REST side and you can check https://docs.nordicsemi.com/bundle/nrf-cloud/page/APIs/REST/Tutorials/Transforms.html

This should enable you to get only the parts you need with the get/tf.

RE: Sending commands from nRF Cloud to LTE Device

Tuomas P — Mon, 11 Nov 2024 19:05:21 GMT

Hello,

I'll report this to the RnD.

I tried to simulate the device being offline just by stopping the simulator after the device had been associated. At least the device did not get update/delta but received correct info with get/accepted.

I'll check from our testing engineers that do we have this kind of test case in our setup. I suspecting that no or there is similar wrong approach as what I used.

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Mon, 11 Nov 2024 18:32:06 GMT

Tuomas P

I think I finally have some insight. I think the nRF Cloud incorrectly trims away parts of the desired section, whereas AWS doesn't.

Here is what I did:

1. The previous reported state of the device is {"cmd": "sendReport": "7"}

2. The device is offline

3. I send the desired update to the server {"cmd": "sendReport": "8"}

4. When the device comes online it gets the shadow via ${deviceId}/shadow/get/accepted.

The shadow shows config {"cmd": "sendReport": "7"} but is missing desired {"cmd": "sendReport": "8"}:

************** MESSAGE RECEIVED ***********

TOPIC: devsim4/shadow/get/accepted

MESSAGE: {

"desired": {

"pairing": {

"state": "paired",

"topics": {

"d2c": "...",

"c2d": "..."

}

},

"nrfcloud_mqtt_topic_prefix": "..."

},

"reported": {

"control": {

"alertsEn": true,

"logLvl": 3

}

},

"config": {

"cmd": {

"sendReport": "7"

}

}

}

*******************************************

4. I restart the device and it now it gets the shadow via $aws/things/${deviceId}/shadow/get/accepted.

The shadow now contains the desired {"cmd": "sendReport": "8"}:

************** MESSAGE RECEIVED ***********

TOPIC: $aws/things/devsim4/shadow/get/accepted

MESSAGE: {

"state": {

"desired": {

"nrfcloud_mqtt_topic_prefix": "...",

"pairing": {

"state": "paired",

"topics": {

"d2c": "...",

"c2d": "..."

}

},

"config": {

"cmd": {

"sendReport": "8"

}

}

},

"reported": {

"connection": {

"status": "connected",

"keepalive": 30

},

"control": {

"alertsEn": true,

"logLvl": 3

},

"config": {

"cmd": {

"sendReport": "7"

}

},

I did nothing else, the device did not report any changes and I did not resend any commands.

It looks like the nRF cloud ${deviceId}/shadow/get/accepted trims the desired delta, regardless of whether the device received it (i.e. was offline) or reported (cleared) it.

This does not look right. Why is the config section of desired trimmed away by nRF Cloud? What is the correct way to do this?

Thanks,
-- Terrence

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Mon, 11 Nov 2024 16:33:16 GMT

Hi Tuomas P

Thanks for the reply. Question: was your device/simulator offline when you sent this:

[quote userid="117557" url="~/f/nordic-q-a/115595/sending-commands-from-nrf-cloud-to-lte-device/509866"]NOTE! Update of the config CMD in server side[/quote]

My experiments show what you describe, but only when the device is online, not when it is offline.

When the device is offline while the config CMD is updated on the server side and then the device comes online then the updates to config CMD seem to be lost - they are not sent as further delta updates nor included in the full shadow.

[quote userid="117557" url="~/f/nordic-q-a/115595/sending-commands-from-nrf-cloud-to-lte-device/509866"]

Can it be that your client already reports back the command and shadow send bending delta again?

[/quote]

I considered that, but the device is definitely not reporting back because I just tested commenting out the reporting code -- same behavior.

This continues to confuse me. My expectation is that -- as long as the device has not reported (cleared) the delta -- then the delta will remain present in the desired section to any shadow message to the device.

Best,
-- Terrence

RE: Sending commands from nRF Cloud to LTE Device

Tuomas P — Mon, 11 Nov 2024 12:04:30 GMT

Hello,

Can it be that your client already reports back the command and shadow send bending delta again?

I am seeing with my implementation that the nRF Cloud send new notification on /update/delta after I reported the previous command. Please see log below

************** MESSAGE SENT ***********
TOPIC: $aws/things/nrfsim-608892161600763900000/shadow/get
MESSAGE: [
""
]
***************************************

************** MESSAGE RECEIVED ***********
TOPIC: nrfsim-608892161600763900000/shadow/get/accepted
MESSAGE: {
"desired": {
"pairing": {
"state": "paired",
"topics": {
"d2c": "prod/2e1ed31b-0c2e-45fb-918e-bdc66a68b002/m/d/nrfsim-608892161600763900000/d2c",
"c2d": "prod/2e1ed31b-0c2e-45fb-918e-bdc66a68b002/m/d/nrfsim-608892161600763900000/+/r"
}
},
"nrfcloud_mqtt_topic_prefix": "prod/2e1ed31b-0c2e-45fb-918e-bdc66a68b002/"
},
"reported": {
"control": {
"alertsEn": true,
"logLvl": 3
}
},
"config": {
"activeMode": true,
"locationTimeout": 300,
"activeWaitTime": 300,
"movementResolution": 120,
"movementTimeout": 3600,
"accThreshAct": 4,
"accThreshInact": 4,
"accTimeoutInact": 60,
"cmd": "command_100",
"nod": []
}
}
*******************************************

************** MESSAGE SENT ***********
TOPIC: $aws/things/nrfsim-608892161600763900000/shadow/update
MESSAGE: {
"state": {
"reported": {
"config": {
"cmd": "command_100"
}
}
}
}
***************************************

NOTE! Extra delta received as it was not reported(cleared) by device
************** MESSAGE RECEIVED ***********
TOPIC: nrfsim-608892161600763900000/shadow/update/delta
MESSAGE: {
"state": {
"nrfcloud_mqtt_topic_prefix": "prod/2e1ed31b-0c2e-45fb-918e-bdc66a68b002/",
"pairing": {
"state": "paired",
"topics": {
"d2c": "prod/2e1ed31b-0c2e-45fb-918e-bdc66a68b002/m/d/nrfsim-608892161600763900000/d2c",
"c2d": "prod/2e1ed31b-0c2e-45fb-918e-bdc66a68b002/m/d/nrfsim-608892161600763900000/+/r"
}
}
}
}
*******************************************

NOTE! Update of the config CMD in server side

************** MESSAGE RECEIVED ***********
TOPIC: nrfsim-608892161600763900000/shadow/update/delta
MESSAGE: {
"state": {
"nrfcloud_mqtt_topic_prefix": "prod/2e1ed31b-0c2e-45fb-918e-bdc66a68b002/",
"pairing": {
"state": "paired",
"topics": {
"d2c": "prod/2e1ed31b-0c2e-45fb-918e-bdc66a68b002/m/d/nrfsim-608892161600763900000/d2c",
"c2d": "prod/2e1ed31b-0c2e-45fb-918e-bdc66a68b002/m/d/nrfsim-608892161600763900000/+/r"
}
},
"config": {
"cmd": "command_101"
}
}
}
*******************************************

************** MESSAGE SENT ***********
TOPIC: $aws/things/nrfsim-608892161600763900000/shadow/update
MESSAGE: {
"state": {
"reported": {
"config": {
"cmd": "command_101"
}
}
}
}
***************************************

NOTE! Extra delta received as it was not reported(cleared) by device

In other words, you cannot compare the device's received delta only to the "desired" in server. In server side you need to check both "reported" and "desired". If both contains the same value ("cmd":"command_101") it is not going to be as delta.

For example, after sending a new command to device my shadow looks like this:

But device receives only:

as the config part does not have delta

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Thu, 07 Nov 2024 13:29:11 GMT

Hi Tuomas P ,

what you describe is what I would expect to see, but it's not what I am seeing.

When the device comes online and sends an empty message to $aws/things/${deviceId}/shadow/get it gets a shadow back, but that shadow does not contain the pending delta, even though the delta is pending in the cloud:

i.e. a cloud API FetchDevice shows

 "state": {

   "desired": {

     "nrfcloud_mqtt_topic_prefix": "prod/...",

     "pairing": {

       "state": "paired",

       "topics": {

         "d2c": "prod/.../d2c",

         "c2d": "prod/.../m/d/devsim4/+/r"

       }

     },

     "config": {

       "cmd": "command26"

     }

   },

but the shadow received via ${deviceId}/shadow/get/accepted shows the following (note that the pending delta "config" is missing:

  "desired": {

    "nrfcloud_mqtt_topic_prefix": "prod/.../"

    "pairing": {

      "state": "paired",

      "topics": {

        "d2c": "prod/.../d2c",

        "c2d": "prod/.../+/r"

      }

    },

  },

Here is the relevant code in the device simulator, this is executed in the onconnect callback:

 // subscribe to shadow gets `${deviceId}/shadow/get/accepted`

 await device.subscribe(device.topics.shadow.accepted);
 device.registerListener(device.topics.shadow.accepted, (param: { topic: string; payload: object}) => 
 {
 log.info(`shadow accepted: ${JSON.stringify(param.payload)}`);
 processCmd(param);
 });


 // subscribe to shadow rejected `${deviceId}/shadow/get/rejected`

 await device.subscribe(device.topics.shadow.rejected);
 device.registerListener(device.topics.shadow.rejected, (param: { topic: string; payload: object}) => 
 {
 log.info(`shadow rejected: ${JSON.stringify(param.payload)}`);
 });

 // subscribe to shadow updates `${deviceId}/shadow/update/delta`

 await device.subscribe(device.topics.shadow.delta);
 device.registerListener(device.topics.shadow.delta, (param: { topic: string; payload: object}) => 
 {
 log.info(`shadow delta: ${JSON.stringify(param.payload)}`);
 processCmd(param);
 }); 


 // since we just connected, get the current shadow from the cloud `$aws/things/${deviceId}/shadow/get`

 await device.publish(device.topics.shadow.get, {}); // payload is irrelevant

Thanks,
-- Terrence

RE: Sending commands from nRF Cloud to LTE Device

Tuomas P — Thu, 07 Nov 2024 07:29:42 GMT

Hello,

So, there are two separate topics to subscribe. Please see https://docs.nordicsemi.com/bundle/nrf-cloud/page/APIs/MQTT/Topics.html#shadow-topics

`${deviceId}/shadow/get/accepted` - nRF Cloud will send delta only when you send empty message on $aws/things/${deviceId}/shadow/get.

`${deviceId}/shadow/update/delta` - nRf Cloud will send event automatically when change happens

SO, you need to subscribe both. And when device is connecting again after offline period you send a empty message to the $aws/things/${deviceId}/shadow/get. This will trigger nRF Cloud to send the delta to the "get/accepted".

Justin's coment that cloud would be sending even on "update/delta" is incorrect in the use case you described. Yes could will be sending the message if there is connection (device is online and keeping the connection alive). But if device is sleeping, the connection will close after 30 seconds.

RE: Sending commands from nRF Cloud to LTE Device

Tuomas P — Fri, 01 Nov 2024 07:08:42 GMT

Hello, Justin is not able to contribute to cloud support anymore. I'll take over. Apologies for the delay.

It is clear that our documentation needs more info on this topic. Issue reported

The notify of shadow update while device was offline does not seem to work as Justin describe. I'll try to reproduce the issue with simulator v2

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Mon, 28 Oct 2024 18:48:03 GMT

Hi justin ,

"state" is necessary when interacting with the shadow via MQTT because that is what AWS requires.
it is not needed in the REST API since that is a custom nrf cloud API.

Got it, thanks.

the device should subscribe to the /update/delta topic and it should receive delta events until they are processed.

Unfortunately, the device simulator doesn't get any delta events after connecting.

Here's the delta event when the device simulator is connected:

************** MESSAGE RECEIVED ***********

TOPIC: <deviceid>/shadow/update/delta

MESSAGE: {

"state": {

"nrfcloud_mqtt_topic_prefix": "prod/.../",

"pairing": {

"state": "paired",

"topics": {

"d2c": "prod/...",

"c2d": "prod/..."

}

},

"config": {

"cmd": "command"

}

*******************************************

Then I stop the device simulator and send a new "cmd: command1" update via the cloud API. Then I start the device simulator. It subscribes (again) to /update/delta and fetches the full shadow:

************** MESSAGE RECEIVED ***********

TOPIC: <deviceid>/shadow/get/accepted

MESSAGE: {

"desired": {

"pairing": {

"state": "paired",

"topics": {

"d2c": "prod/...",

"c2d": "prod/..."

}

},

"nrfcloud_mqtt_topic_prefix": "prod/..."

},

"reported": {

"control": {

"alertsEn": true,

"logLvl": 3

}

},

"config": {

"activeMode": true,

"locationTimeout": 300,

"activeWaitTime": 300,

"movementResolution": 120,

"movementTimeout": 3600,

"accThreshAct": 4,

"accThreshInact": 4,

"accTimeoutInact": 60,

"nod": []

}

*******************************************

The "cmd: command1" is not present. And even if the device simulator continues to be connected it never receives a delta update of the "cmd: command1".

Thanks,
-- Terrence

RE: Sending commands from nRF Cloud to LTE Device

justin — Mon, 28 Oct 2024 18:17:47 GMT

[quote userid="120541" url="~/f/nordic-q-a/115595/sending-commands-from-nrf-cloud-to-lte-device/508236"]why "state" is necessary when sending a device update via MQTT but is not allowed when using the API (UpdateDevice)?[/quote]

"state" is necessary when interacting with the shadow via MQTT because that is what AWS requires.
it is not needed in the REST API since that is a custom nrf cloud API.

the device should subscribe to the /update/delta topic and it should receive delta events until they are processed.

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Mon, 28 Oct 2024 17:54:13 GMT

Hi justin ,

Ah, thanks -- using "state" works as expected ... now the cmd is processed and appears in "reported: config: cmd" in the full shadow. I think that addresses my first problem. For my information, can you explain why "state" is necessary when sending a device update via MQTT but is not allowed when using the API (UpdateDevice)?

However, the problem I still have is: When the device is online, it receives the delta update in the "desired: config: cmd" section (and can now process it, as above). But when the device is offline and then comes online, how should it retrieve the pending delta update?

Thanks,
-- Terrence

RE: Sending commands from nRF Cloud to LTE Device

justin — Mon, 28 Oct 2024 16:21:05 GMT

In the meantime, I am now trying to at least have the device handle the delta message it receives while it is connected. Upon receiving the delta message (as per above) the device now publishes to $aws/things/<devceid>/shadow/update the following:

Try the following:

{
  "state": {
    "reported": {
      "config": {
        "cmd": "command"
      }
    }
  }
}

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Mon, 28 Oct 2024 16:18:24 GMT

Hi Tuomas P ,

Thanks for the answer. Yes, I've added the necessary code to the device simulator. I subscribe to the appropriate topics to handle full and delta "desired" shadows, I handle the cmd, and I publish the device shadow "reported" update to the appropriate topic, as described above.

To remove the device simulator from the equation I will try and replicate the problem with a desktop- or web-based MQTT client.

Thanks,
-- Terrence

RE: Sending commands from nRF Cloud to LTE Device

Tuomas P — Mon, 28 Oct 2024 16:03:01 GMT

This issue has been notified to simulator and backend teams.

Do I understand correctly that you use the simulator V2 and have modified it.

The simulator does not seem to have the report back functionality even a dummy one. It inits shadow with semi hardcoded JSON and subscribes shadow topics. But it does not seem to do anything based on received events.

The device side would need to report the cmd being hendled.

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Wed, 23 Oct 2024 15:06:18 GMT

Hi dejans ,

Despite more experiments, I have not yet been able to retrieve the full shadow (including the cmd section) yet. Any advice is appreciated.

{

"reported": {

"config": {

"cmd": "command"

}

}

My expectation is that the "cmd" in "desired: config" in the shadow moves to "reported: config" or that "cmd" now is in both "desired" and "reported", indicating that the device has processed the shadow change.

But that is not what I am seeing. Instead, when I retrieve the full shadow via the cloud API the "cmd" is still present in the desired section, now also in the "delta" section as well, but not in "reported". It is as if the cloud hasn't processed the update message from the device and it is still pending in "delta".

I have spent several hours on this topic now, and what I am seeing does not match the documentation or my understanding of AWS device shadow. https://docs.nordicsemi.com/bundle/nrf-cloud/page/Devices/Properties/Shadows.html#configuring-a-device provides a round-trip example, but this is not what I am seeing.

I would be very grateful if you could help clear this up.

Thanks!

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Tue, 22 Oct 2024 16:10:48 GMT

Hi dejans ,

Thanks for the quick reply. You wrote:

In this case, reported config section is sent on trimmed shadow topic. The delta for "cmd" change in desired will come in on the delta topic

That is not what I am seeing. As I wrote above, when the device reconnects and retrieves the (trimmed) shadow, it does not contain the delta for "cmd" in desired.

Please compare the two screenshorts:

1. shadow in the cloud

2. device log

Note that I am using the device-simulator-v2, not a real device. But the device-simulator is suscribed to the correct deviceId shadow topic, so it shouldn't matter.

What might be wrong?

Thanks,

-- Terrence

RE: Sending commands from nRF Cloud to LTE Device

dejans — Tue, 22 Oct 2024 15:20:50 GMT

Hi teba99 ,

nRF Cloud library does not actually request the full shadow because it can be too large for modem to receive it. Topic ${deviceId}/shadow/get/accepted receives only trimmed shadow which contains only essential shadow data. You can find more information about shadow topics in the documentation. The reason that you do not receive your "cmd" section is due to trimmed shadow being received.

[quote user="teba99"]We also tried a shadow update with section
{
"desired": {
"config": {
"cmd": "command"
}
}
}[/quote]

This is fine. In this case, reported config section is sent on trimmed shadow topic. The delta for "cmd" change in desired will come in on the delta topic.

Best regards,
Dejan

RE: Sending commands from nRF Cloud to LTE Device

teba99 — Mon, 21 Oct 2024 09:06:38 GMT

Hi Dejan,

I tried this, it doesn't seem to work as expected ... or maybe we are misunderstanding.

From the cloud, we use the API devices/${deviceid}/state to send
{
"desired": {
"cmd": "command"
}
}

If the device is online then it receives the delta right away, as we expect:

TOPIC: shadow/update/delta
MESSAGE: {
"state": {
"nrfcloud_mqtt_topic_prefix": "prod/<tenant>/",
"pairing": {
"state": "paired",
"topics": {
"d2c": "prod/<tenant>/m/d/<deviceid>/d2c",
}
},
"cmd": "command"
}
}

If the device is offline then our understanding is the device would need to explicitly get the shadow via topic shadow/get/accepted when reconnecting . However, this is what the device gets (note that it does not contain the desired { "cmd": "command"} ):

TOPIC: shadow/get/accepted
MESSAGE: {
"desired": {
"pairing": {
"state": "paired",
"topics": {
"d2c": "prod/<tenant>/m/d/<deviceid>/d2c",
}
},
"nrfcloud_mqtt_topic_prefix": "prod/<tenant>/",
},
"reported": {
"control": {
"alertsEn": true,
"logLvl": 3
}
},
"config": {
"activeMode": true,
...
}
}

We also tried a shadow update with section
{
"desired": {
"config": {
"cmd": "command"
}
}
}

but get the same result: the delta contains the update, but the full shadow does not.

Could you please clarify where we would add the command and how the device gets that shadow change after it comes online?

Thanks!

RE: Sending commands from nRF Cloud to LTE Device

ragruslan — Mon, 21 Oct 2024 08:27:27 GMT

Thank you!

RE: Sending commands from nRF Cloud to LTE Device

dejans — Fri, 18 Oct 2024 14:10:34 GMT

Hi,

C2D topic is best used as a return path for data requested by the device on a D2C channel rather than asynchronous commands for the device. If device is online, it will not receive messages on C2D or any other MQTT topic. For issuing commands by any means you wish, shadow would be appropriate to use.

We have an example using "config" section in multi-service sample. Using a section for commands would be the same process. You could create "cmd" section in your shadow. You can use built-in AWS shadow delta mechanism and/or newly added transform MQTT topic, with nrf_shadow_transform_request().

Best regards,
Dejan

RE: Sending commands from nRF Cloud to LTE Device

dejans — Thu, 17 Oct 2024 16:14:15 GMT

Hi,

I will look into your questions and come back to you possibly in the first half of next week.

Best regards,
Dejan