Very high response times of reliable unicast messages.

RE: Very high response times of reliable unicast messages.

tesc — Thu, 29 Nov 2018 13:13:34 GMT

Hi,

Revisiting this one, after a (way too long) silence from my side.

I do have a couple of suggestions.

The first one is to update to a newer release of the nRF5 SDK for Mesh. We have experienced that especially with the proxy feature enabled you may experience packet loss because of the parameters chosen for advertising and connections. If I recall correctly then default settings were changed slightly for the current release (v2.2.0) and from what I understand there will be further improvements for next release.

The time between retransmits depend on TTL. A shorter TTL setting means shorter time between retransmits. The values that you have experienced for the delay correspond to a relatively high TTL setting.

You could also experiment with the ACCESS_RELIABLE_INTERVAL_DEFAULT setting, which is also proportional to the time between retransmissions. There is also the ACCESS_RELIABLE_BACK_OFF_FACTOR which makes each consecutive retransmission delay more than the previous, with that factor. Defaults to 2, which means you see a doubling in delay for each retransmission.

And of course, if you have found out anything in your end then please feel free to share your findings.

Regards,
Terje

RE: Very high response times of reliable unicast messages.

tesc — Mon, 03 Sep 2018 15:15:02 GMT

Hi,

Thank you. That it is in fact a doubling is very interesting, and it supports the hypothesis. I will try once more to reproduce, and check to see if changing parameters may improve the situation.

Regards,
Terje

RE: Very high response times of reliable unicast messages.

Armin — Fri, 31 Aug 2018 07:32:50 GMT

Regarding advertising interval I did not change that parameter against what is preset in your examples. I even don't know where this is defined, so if I shall increase it as you recommended please let me know in which file it is exactly defined so that I can find it.

Yes, the projects are those which are mentioned by you before. Look:

For reproduction you should take the original Nordic examples and override the sources in:

\nRF5_SDK_for_Mesh\examples\light_switch\proxy_client\include
\nRF5_SDK_for_Mesh\examples\light_switch\proxy_client\src
\nRF5_SDK_for_Mesh\models\generic_on_off\include
\nRF5_SDK_for_Mesh\models\generic_on_off\src
\nRF5_SDK_for_Mesh\examples\light_switch\include
\nRF5_SDK_for_Mesh\examples\light_switch\thingy_provisioning_demo_generic_OnOff_BLINK\include
\nRF5_SDK_for_Mesh\examples\light_switch\thingy_provisioning_demo_generic_OnOff_BLINK\src

Regarding the delays I made a mistake in my description: it is actually always a double amount of the previous number starting with 5 as you mentioned above, i.e. 5, 10, 20, etc. and not just a multiply of 5. Sorry for that. In fact, if I understood it right, this confirms your hypothesis of lost acknowledges.

However, the question remains why there are messages lost. Maybe the behavior improves when the advertising interval time is increased. So either you test it on your own or you help me finding the code where the parameter is defined so I can test it. But this may take some days as I'm currently quite busy with other work.

RE: Very high response times of reliable unicast messages.

tesc — Mon, 27 Aug 2018 15:34:08 GMT

Hi,

Regarding the delays that you see, you mentioned they all are a multiple of 5 seconds. Can you confirm that you see all of the multiples of 5, and that it is not a series where each number is the double amount of the previous number, as in 5, 10, 20, etc?

One hypothesis is that while the message is sent and received, the acknowledgement is lost, which means the message is resent until an acknowledgement is correctly received (or a timeout). But the behavior for resending messages is that the time between retries is doubled every time, and that does not correspond to delays increasing linearly.

Regards,
Terje

RE: Very high response times of reliable unicast messages.

tesc — Fri, 24 Aug 2018 17:07:11 GMT

Hi,

We are very sorry for the delays over the past days. I have been looking at this issue today.

I think that your reported 10 ms advertising interval is way too short for letting the Mesh stack run properly. Did you test increasing it?

It would be great if we could build and reproduce this issue locally, but it seems many of the files that are needed to build the project are still missing... Usually we ask for a minimal example showing the erroneous behavior. It may be an option to strip some of the unneeded functionality away from the project instead. Or, we would need all the included c and h files for building the projects.

If I have understood things correctly, the projects are:

For nRF52840 DK: examples\light_switch\proxy_client\light_switch_proxy_client_nrf52840_xxAA_s140_6_0_0.emProject

For the Thingy:52s: examples\light_switch\thingy_provisioning_demo_generic_OnOff_BLINK\light_switch_proxy_server_nrf52832_xxAA_s132_6_0_0.emProject

Can you confirm or correct that?

Regards,
Terje

RE: Very high response times of reliable unicast messages.

Armin — Thu, 23 Aug 2018 11:46:58 GMT

Yes, according to the RELEASE_NOTES.md we use

BLE Mesh v2.0.1

RE: Very high response times of reliable unicast messages.

Bjørn Kvaale — Thu, 23 Aug 2018 11:15:43 GMT

Hi Armin,

Very sorry for the delayed response. Hung has been on vacation for the past two weeks & it seems we forgot to follow up on this case. I am very sorry about that. Hung will be back on Monday, but I will try my best to help you out & hopefully we can figure out the issue before he is back. Are you using mesh sdk v2.0.1?

Kind Regards,

Bjørn

RE: Very high response times of reliable unicast messages.

Armin — Thu, 23 Aug 2018 08:40:49 GMT

Did you make some progress in your investigations for this issue?

Regarding your questions:

1. I use the generic on off model provided in the thingy example.

2. I use that model on the server as well.

I tried to attach the whole affected SDK as you requested. However, although I stripped off all binaries, builds and documentation the .zip still has 53 MB. Obviously this is too big for your upload infrastructure as uploading fails with an error as follows:

Alternatively here is at least the .zip with the whole light switch projects:

/cfs-file/__key/communityserver-discussions-components-files/4/light_5F00_switch.zip

RE: Very high response times of reliable unicast messages.

Hung Bui — Fri, 27 Jul 2018 12:21:48 GMT

Hi Armin,

I can see that you use generic_on_off instead of simple_on_off as in our example. Did you use the generic on off model provided in our thingy example or your own implementation of the model ? You use that model on the server as well ? I would need to zip your whole light switch projects (because there is some definition in the common header files) or better just zip the whole SDK. You may want to delete the compiled binary to reduce the size of the .zip.

RE: Very high response times of reliable unicast messages.

Armin — Thu, 26 Jul 2018 10:32:48 GMT

devzone.nordicsemi.com/.../proxy_5F00_client.zip

RE: Very high response times of reliable unicast messages.

Hung Bui — Thu, 26 Jul 2018 09:16:09 GMT

(Sorry for the delay in the response, we are on the peak of summer vacation in Norway)

10ms is very short interval. This will largely affect the mesh operation. It must be something wrong here. We will do some investigation.

Yes you can post a full file, by using Insert -> Insert Image/video/file , then you can attach the file. We prefer to have the full project. Just in case any setting in the compiler option was wrong.

RE: Very high response times of reliable unicast messages.

Armin — Wed, 25 Jul 2018 14:47:41 GMT

According to nRFConnect App advertising intervals are significantly different between client and server:

Server (Thingy:52): 500ms
Client (nRF52840 Preview DK): 10ms

Is it possible to attach a complete sourcecode file here? I won't prefer pasting my complete main.c here.

RE: Very high response times of reliable unicast messages.

Hung Bui — Wed, 25 Jul 2018 14:04:04 GMT

yes, please attach the code.

You can use a phone to see the advertising interval (nRFConnect app)

RE: Very high response times of reliable unicast messages.

Armin — Wed, 25 Jul 2018 09:11:43 GMT

It makes no difference whether the Smartphone is connected or not.

You mention you have to check the advertising interval. Please tell me exactly which parameter I shall provide to you in case.

I'm sending multiple reliable unicast messages to multiple nodes. After each send I wait for the reply, wait an additional time (200ms) and then send the next.

I could attach the source here if it helps.

RE: Very high response times of reliable unicast messages.

Hung Bui — Wed, 25 Jul 2018 08:09:17 GMT

Hi Armin,

I guess the proxy feature could be the cause here. When you see the issue, was it with the phone connected or not ? If it's connected to the phone, we need to check the actual connection interval.

If it's not connected to the phone, we need to check the advertising interval of the client and also the advertising interval of other server nodes.

We want to test and reproduce the issue here. How did you test ? Did you simply just send reliable message to one single node or to multiple nodes (with multiple messages) ? You mentioned you can see it once in about 20 trials ?

RE: Very high response times of reliable unicast messages.

Armin — Tue, 24 Jul 2018 14:04:08 GMT

The behavior is independent from the number of nodes. I tested with 3, 5, 20 and 30 nodes.

The messages are sent from the light switch client node, which runs on the nRF52840 Preview DK with project light_switch_proxy_client_nrf5284_xxAA_s140_6.0.0. By that it is also the proxy for commissioning via Smartphone. All server nodes running on Thingy:52 with project light_switch_proxy_server_nrf52832_xxAA_s132_6.0.0, so all servers are also proxy.

The TTL value is the default set in nrf_mesh_config_app.h #define ACCESS_DEFAULT_TTL (SERVER_NODE_COUNT) whereas SERVER_NODE_COUNT is set in light_switch_example_common.h as #define SERVER_NODE_COUNT (30)

The reliable messages are sent to a unicast address. Group addresses can only be used with unreliable messages!

Yes, the packet arrives at the servers always very fast. An LED is switched on there when the message arrives and I see almost no visual delay between the time the message is sent which is indicated by a log output at JLink Viewer and the LED lights on. By that the high delay comes actually from the response!

RE: Very high response times of reliable unicast messages.

Hung Bui — Tue, 24 Jul 2018 13:19:25 GMT

Hi Armin,

No I don't think it's a normal behavior. Do you see the same problem when you test with unmodified version of the light switch examples ?

Do you see the same problem if you have less number of nodes ?

Did you run anything else beside mesh on the node ? Running BLE or proxy on the node would affect the mesh performance.

The mechanism to resend notification is similar to trickle algorithm the interval is multiple by ACCESS_RELIABLE_BACK_OFF_FACTOR by each retry. But your 5-10-15 seconds observation is quite strange, wouldn't match with what we have.

Which TTL value did you set ? The timeout is calculated by the TTL value.

Did you send the reliable message to a unicast address or to a group address ?

Also do you see the packet arrive on the peer device before that 5 -10 .. seconds ? We need to find if it was the original packet missing or the response packet missing.