Odd/Unexpected Behavior from NRFMesh

[NickN]

Greetings.

We've been developing a product that uses BLE Mesh. We are currently using the Waveshare nRF52840 Bluetooth 5.0 Evaluation Kit for prototyping purposes. As far as I know, we are using the latest SDK and our firmware is based off the "sensor" project example.

I am seeing two unexpected behaviors with nRFMesh. I would welcome any feedback you all might have.

Issue #1

I currently have seven nodes provisioned and reporting to a gateway. Five are definitely within range of the gateway. One is definitely out of range. One is right on the border (sometimes it seems to report directly and other times the message seems to get relayed).

I am provisioning the mesh using an Android phone and nRFMesh. We have two custom element models defined in the node server profile (a server model and a setup model). The server model is set to publish to a group address every 10s (the publish period is set to 10s). Within "Publication Settings" for the element model that is publishing messages, I set the "Publish TTL" value manually to 15.

Here's what's odd: when I come back to review the mesh settings at a later time, I often find that TTL has reset to a value of 1. Please note that I am connected to the mesh within nRFMesh when I check this value. However, I am still receiving messages at the gateway, so the data must be being relayed. i.e. the TTL cannot be 1.

Equally confusing, the default TTL on the node configuration page of the app still shows a value of 15.  

Is this normal behavior? It seems odd that the value is reset to 1 when it clearly cannot be 1.

Issue #2

Let's say I want to update the Publication Settings for the nodes in my mesh. I connect to the mesh within the nRFMesh app and then change the parameters for each provisioned node.

This works fine for nodes that are definitely within range of wherever I am at the time. I can connect to any of the five nodes in range and make changes to any of those five without reconnecting/changing which node I connect to within nRFMesh. 

Here's what's odd: For the two nodes that are out of range, I have to move to be within their range and connect to the node I want to change within nRFMesh. 

Since this is a mesh and data is being relayed, I would expect to be able to change the settings of any provisioned node in the mesh, regardless of its location (as long as it is connected to another node).

 

My concern with both issues is that we may have something set incorrectly, or some underlying bug we need to squash. All suggestions/comments are most welcome.

 

[daviddedwin]

#1 For the TTL issue, use the nRF Sniffer and take a look at the messages that are actually on the network. It maybe an Android app issue.
However posting the nRFSniffer wireshark capture file will be useful.
Additionally post the Android app version and the Android OS version that you are using.

#2 Take a look at the relay settings for the nodes, ideally all nodes should not be relaying as that will choke the network however you should look at the nRFMesh setting for Relay and also in the code as the Relay capability can be set both ways.

Actions requested:
- nRFSniffer Wireshark trace
- Android app version
- Android OS version
- Post a map of the network 
- Post the JSON file that you have exported from the nRFMesh

[NickN]

Thanks for the quick reply.

- nRFSniffer Wireshark trace

Work in progress. I've set up nRFSniffer with an nRF Dongle, but I'm new to wireshark so it will take me a short while to figure out how to get useful data out of it. Working with my developer to figure this out.

- Android app version

3.1.5

- Android OS version

11, running on a Samsung S21 Ultra

- Post a map of the network 

- Post the JSON file that you have exported from the nRFMesh

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "http://www.bluetooth.com/specifications/assigned-numbers/mesh-profile/cdb-schema.json#",
  "version": "1.0.0",
  "meshUUID": "4076B36A-9332-495F-8AA8-A270EEEA5EB9",
  "meshName": "nRF Mesh Network",
  "timestamp": "2021-05-02T15:25:54-04",
  "partial": false,
  "netKeys": [
    {
      "name": "Network Key 1",
      "index": 0,
      "key": "759738F46882F31FEC9887E4A23BDE99",
      "phase": 0,
      "minSecurity": "insecure",
      "timestamp": "2021-05-02T14:30:21-04"
    }
  ],
  "appKeys": [
    {
      "name": "Application Key 1",
      "index": 0,
      "boundNetKey": 0,
      "key": "DA7979E5B38D4525F9B4FD1AFF4A5294"
    },
    {
      "name": "Application Key 2",
      "index": 1,
      "boundNetKey": 0,
      "key": "381ED8CC5E001DCE617E3615A584739B"
    },
    {
      "name": "Application Key 3",
      "index": 2,
      "boundNetKey": 0,
      "key": "E61A3AE2D95C1E7FE5AB9CEFE3B5C95B"
    }
  ],
  "provisioners": [
    {
      "provisionerName": "nRF Mesh Provisioner",
      "UUID": "B48C3296-6F72-405A-AB48-B1A3F0C7CAAC",
      "allocatedUnicastRange": [
        {
          "lowAddress": "0001",
          "highAddress": "199A"
        }
      ],
      "allocatedGroupRange": [
        {
          "lowAddress": "C000",
          "highAddress": "CC9A"
        }
      ],
      "allocatedSceneRange": [
        {
          "firstScene": "0001",
          "lastScene": "3333"
        }
      ]
    }
  ],
  "nodes": [
    {
      "UUID": "B48C3296-6F72-405A-AB48-B1A3F0C7CAAC",
      "name": "nRF Mesh Provisioner",
      "deviceKey": "B2A5F637F59705B97762FE493C46A7FB",
      "unicastAddress": "0001",
      "security": "insecure",
      "configComplete": true,
      "features": {
        "friend": 2,
        "lowPower": 2,
        "proxy": 2,
        "relay": 2
      },
      "defaultTTL": 5,
      "netKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "appKeys": [
        {
          "index": 0,
          "updated": false
        },
        {
          "index": 1,
          "updated": false
        },
        {
          "index": 2,
          "updated": false
        }
      ],
      "elements": [
        {
          "name": "Element: 0x0001",
          "index": 0,
          "location": "0000",
          "models": [
            {
              "modelId": "0001",
              "bind": [],
              "subscribe": []
            }
          ]
        }
      ],
      "excluded": false
    },
    {
      "UUID": "3A7AD8F9-4C5F-45E0-98AC-8580EBF9835C",
      "name": "Gateway 002 AC:D8",
      "deviceKey": "9CDB9E1D4405621574830EF009537448",
      "unicastAddress": "0002",
      "security": "insecure",
      "configComplete": false,
      "cid": "0059",
      "pid": "0000",
      "vid": "0000",
      "crpl": "0028",
      "features": {
        "friend": 2,
        "lowPower": 2,
        "proxy": 1,
        "relay": 1
      },
      "defaultTTL": 40,
      "networkTransmit": {
        "count": 2,
        "interval": 20
      },
      "netKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "appKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "elements": [
        {
          "name": "Element: 0x0002",
          "index": 0,
          "location": "0000",
          "models": [
            {
              "modelId": "0000",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "0002",
              "bind": [],
              "subscribe": []
            }
          ]
        },
        {
          "name": "Element: 0x0003",
          "index": 1,
          "location": "0000",
          "models": [
            {
              "modelId": "1102",
              "bind": [
                0
              ],
              "subscribe": [
                "C000"
              ],
              "publish": {
                "address": "C010",
                "index": 0,
                "ttl": 1,
                "period": {
                  "numberOfSteps": 0,
                  "resolution": 100
                },
                "retransmit": {
                  "count": 3,
                  "interval": 100
                },
                "credentials": 0
              }
            }
          ]
        },
        {
          "name": "Element: 0x0004",
          "index": 2,
          "location": "0000",
          "models": [
            {
              "modelId": "1102",
              "bind": [],
              "subscribe": []
            }
          ]
        }
      ],
      "excluded": false
    },
    {
      "UUID": "962CDE69-8BC4-413F-ACDE-86BB662C7975",
      "name": "0005 DE:EC",
      "deviceKey": "3D6B0477DA08D1F1745D06256D428154",
      "unicastAddress": "0005",
      "security": "insecure",
      "configComplete": false,
      "cid": "0059",
      "pid": "0000",
      "vid": "0000",
      "crpl": "0028",
      "features": {
        "friend": 2,
        "lowPower": 2,
        "proxy": 1,
        "relay": 1
      },
      "defaultTTL": 15,
      "networkTransmit": {
        "count": 2,
        "interval": 20
      },
      "netKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "appKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "elements": [
        {
          "name": "Element: 0x0005",
          "index": 0,
          "location": "0000",
          "models": [
            {
              "modelId": "0000",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "0002",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "1100",
              "bind": [
                0
              ],
              "subscribe": [],
              "publish": {
                "address": "C000",
                "index": 0,
                "ttl": 1,
                "period": {
                  "numberOfSteps": 10,
                  "resolution": 1000
                },
                "retransmit": {
                  "count": 1,
                  "interval": 100
                },
                "credentials": 0
              }
            },
            {
              "modelId": "1101",
              "bind": [
                0
              ],
              "subscribe": [
                "C010"
              ]
            }
          ]
        }
      ],
      "excluded": false
    },
    {
      "UUID": "F737B90F-FED8-42AA-87E9-A0CD3AA7E3B1",
      "name": "0006 E9:07",
      "deviceKey": "2327B46BD9E6BC62FFD6186FAC1B5CDA",
      "unicastAddress": "0006",
      "security": "insecure",
      "configComplete": false,
      "cid": "0059",
      "pid": "0000",
      "vid": "0000",
      "crpl": "0028",
      "features": {
        "friend": 2,
        "lowPower": 2,
        "proxy": 1,
        "relay": 1
      },
      "defaultTTL": 15,
      "networkTransmit": {
        "count": 2,
        "interval": 20
      },
      "netKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "appKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "elements": [
        {
          "name": "Element: 0x0006",
          "index": 0,
          "location": "0000",
          "models": [
            {
              "modelId": "0000",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "0002",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "1100",
              "bind": [
                0
              ],
              "subscribe": [],
              "publish": {
                "address": "C000",
                "index": 0,
                "ttl": 1,
                "period": {
                  "numberOfSteps": 10,
                  "resolution": 1000
                },
                "retransmit": {
                  "count": 1,
                  "interval": 100
                },
                "credentials": 0
              }
            },
            {
              "modelId": "1101",
              "bind": [
                0
              ],
              "subscribe": [
                "C010"
              ]
            }
          ]
        }
      ],
      "excluded": false
    },
    {
      "UUID": "38BC8408-FD73-43B4-9877-EFD9E04035EF",
      "name": "0007 77:18",
      "deviceKey": "B82AC438F5FCA510CB45C86A08F869BA",
      "unicastAddress": "0007",
      "security": "insecure",
      "configComplete": false,
      "cid": "0059",
      "pid": "0000",
      "vid": "0000",
      "crpl": "0028",
      "features": {
        "friend": 2,
        "lowPower": 2,
        "proxy": 1,
        "relay": 1
      },
      "defaultTTL": 15,
      "networkTransmit": {
        "count": 2,
        "interval": 20
      },
      "netKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "appKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "elements": [
        {
          "name": "Element: 0x0007",
          "index": 0,
          "location": "0000",
          "models": [
            {
              "modelId": "0000",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "0002",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "1100",
              "bind": [
                0
              ],
              "subscribe": [],
              "publish": {
                "address": "C000",
                "index": 0,
                "ttl": 1,
                "period": {
                  "numberOfSteps": 10,
                  "resolution": 1000
                },
                "retransmit": {
                  "count": 1,
                  "interval": 100
                },
                "credentials": 0
              }
            },
            {
              "modelId": "1101",
              "bind": [
                0
              ],
              "subscribe": [
                "C010"
              ]
            }
          ]
        }
      ],
      "excluded": false
    },
    {
      "UUID": "B7774E19-91DA-4B32-8340-59D78553445D",
      "name": "0008 40:03",
      "deviceKey": "434F0060A3D1399D4B93E77558E5962D",
      "unicastAddress": "0008",
      "security": "insecure",
      "configComplete": false,
      "cid": "0059",
      "pid": "0000",
      "vid": "0000",
      "crpl": "0028",
      "features": {
        "friend": 2,
        "lowPower": 2,
        "proxy": 1,
        "relay": 1
      },
      "defaultTTL": 15,
      "networkTransmit": {
        "count": 2,
        "interval": 20
      },
      "netKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "appKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "elements": [
        {
          "name": "Element: 0x0008",
          "index": 0,
          "location": "0000",
          "models": [
            {
              "modelId": "0000",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "0002",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "1100",
              "bind": [
                0
              ],
              "subscribe": [],
              "publish": {
                "address": "C000",
                "index": 0,
                "ttl": 1,
                "period": {
                  "numberOfSteps": 10,
                  "resolution": 1000
                },
                "retransmit": {
                  "count": 1,
                  "interval": 100
                },
                "credentials": 0
              }
            },
            {
              "modelId": "1101",
              "bind": [
                0
              ],
              "subscribe": [
                "C010"
              ]
            }
          ]
        }
      ],
      "excluded": false
    },
    {
      "UUID": "70B2CBD9-C85F-49B1-8F1B-C7F38859FC68",
      "name": "0009 1B:CF",
      "deviceKey": "A98420C24D35056627F15CE54130276C",
      "unicastAddress": "0009",
      "security": "insecure",
      "configComplete": false,
      "cid": "0059",
      "pid": "0000",
      "vid": "0000",
      "crpl": "0028",
      "features": {
        "friend": 2,
        "lowPower": 2,
        "proxy": 1,
        "relay": 1
      },
      "defaultTTL": 15,
      "networkTransmit": {
        "count": 2,
        "interval": 20
      },
      "netKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "appKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "elements": [
        {
          "name": "Element: 0x0009",
          "index": 0,
          "location": "0000",
          "models": [
            {
              "modelId": "0000",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "0002",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "1100",
              "bind": [
                0
              ],
              "subscribe": [],
              "publish": {
                "address": "C000",
                "index": 0,
                "ttl": 1,
                "period": {
                  "numberOfSteps": 10,
                  "resolution": 1000
                },
                "retransmit": {
                  "count": 1,
                  "interval": 100
                },
                "credentials": 0
              }
            },
            {
              "modelId": "1101",
              "bind": [
                0
              ],
              "subscribe": [
                "C010"
              ]
            }
          ]
        }
      ],
      "excluded": false
    },
    {
      "UUID": "7398B249-7CE5-4BA3-BCD5-E242882A1C11",
      "name": "Dead",
      "deviceKey": "E8D823F3C8C0411AAA614309D4F6CD0A",
      "unicastAddress": "000A",
      "security": "insecure",
      "configComplete": false,
      "cid": "0059",
      "pid": "0000",
      "vid": "0000",
      "crpl": "0028",
      "features": {
        "friend": 2,
        "lowPower": 2,
        "proxy": 1,
        "relay": 1
      },
      "defaultTTL": 15,
      "networkTransmit": {
        "count": 2,
        "interval": 20
      },
      "netKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "appKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "elements": [
        {
          "name": "Element: 0x000A",
          "index": 0,
          "location": "0000",
          "models": [
            {
              "modelId": "0000",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "0002",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "1100",
              "bind": [
                0
              ],
              "subscribe": [],
              "publish": {
                "address": "C000",
                "index": 0,
                "ttl": 1,
                "period": {
                  "numberOfSteps": 50,
                  "resolution": 100
                },
                "retransmit": {
                  "count": 1,
                  "interval": 100
                },
                "credentials": 0
              }
            },
            {
              "modelId": "1101",
              "bind": [
                0
              ],
              "subscribe": [
                "C010"
              ]
            }
          ]
        }
      ],
      "excluded": false
    },
    {
      "UUID": "03B4796F-5D05-4357-8249-3E3266F0BE0B",
      "name": "000C 49:42",
      "deviceKey": "AC4E9ACA299C91DB37DDDC578D5FCF2F",
      "unicastAddress": "000C",
      "security": "insecure",
      "configComplete": false,
      "cid": "0059",
      "pid": "0000",
      "vid": "0000",
      "crpl": "0028",
      "features": {
        "friend": 2,
        "lowPower": 2,
        "proxy": 1,
        "relay": 1
      },
      "defaultTTL": 15,
      "networkTransmit": {
        "count": 2,
        "interval": 20
      },
      "netKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "appKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "elements": [
        {
          "name": "Element: 0x000C",
          "index": 0,
          "location": "0000",
          "models": [
            {
              "modelId": "0000",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "0002",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "1100",
              "bind": [
                0
              ],
              "subscribe": [],
              "publish": {
                "address": "C000",
                "index": 0,
                "ttl": 1,
                "period": {
                  "numberOfSteps": 10,
                  "resolution": 1000
                },
                "retransmit": {
                  "count": 1,
                  "interval": 100
                },
                "credentials": 0
              }
            },
            {
              "modelId": "1101",
              "bind": [
                0
              ],
              "subscribe": [
                "C010"
              ]
            }
          ]
        }
      ],
      "excluded": false
    },
    {
      "UUID": "AB89D50B-7FDE-4E41-84A7-75BAE822FA1A",
      "name": "000D A7:44 Good",
      "deviceKey": "FAAAF943E35E15541453E2CD304D537F",
      "unicastAddress": "000D",
      "security": "insecure",
      "configComplete": false,
      "cid": "0059",
      "pid": "0000",
      "vid": "0000",
      "crpl": "0028",
      "features": {
        "friend": 2,
        "lowPower": 2,
        "proxy": 1,
        "relay": 1
      },
      "defaultTTL": 15,
      "networkTransmit": {
        "count": 2,
        "interval": 20
      },
      "netKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "appKeys": [
        {
          "index": 0,
          "updated": false
        }
      ],
      "elements": [
        {
          "name": "Element: 0x000D",
          "index": 0,
          "location": "0000",
          "models": [
            {
              "modelId": "0000",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "0002",
              "bind": [],
              "subscribe": []
            },
            {
              "modelId": "1100",
              "bind": [
                0
              ],
              "subscribe": [],
              "publish": {
                "address": "C000",
                "index": 0,
                "ttl": 1,
                "period": {
                  "numberOfSteps": 10,
                  "resolution": 1000
                },
                "retransmit": {
                  "count": 1,
                  "interval": 100
                },
                "credentials": 0
              }
            },
            {
              "modelId": "1101",
              "bind": [
                0
              ],
              "subscribe": [
                "C010"
              ]
            }
          ]
        }
      ],
      "excluded": false
    }
  ],
  "groups": [
    {
      "name": "SensorData",
      "address": "C000",
      "parentAddress": "0000"
    },
    {
      "name": "TimeSync",
      "address": "C010",
      "parentAddress": "0000"
    }
  ],
  "scenes": [],
  "networkExclusions": []
}

[daviddedwin]

Thank you, this starts to show some things.

1. The yellow node and and the red node where the connectivity is poor will need to be sniffed to see if they are actually getting relayed, they are in the edge of the range.
 - Followup questions and actions
   What is your transmit power, set this to the maximum that it possible.
   What is your antenna type and external antenna is preferred here.

2. The TTL numbers for "defaultTTL" are all over the place, for such a small network try setting it to TTL=5 but set the retransmit to at least 3 for all node transmissions. A re-transmit setting of 1 will not work and will cause packet losses.

3. What is the bandwidth on this network, can you comment on the message rates between nodes and gateways.

My first hypothesis is that the relaying is not working well due to the low retransmit settings so that will be the main thing to check.

My second hypothesis is that there are proxies running on all the nodes, please provide the advertising interval and the connection interval being used on the proxy, this can also choke the relaying capacity and show symptoms like the ones you are seeing.