Bluetooth: Mesh Demo example automatic provisioning

[akif]

Hello,

We are working on the Bluetooth: Mesh Demo example on nRF-52 DK using nRF Connect SDK 2.3.0. The example doesn't requires provisioning which is what I want. I run the application and;

nRF Mesh app scan (Connect or + Add Node button) doesn't show any node. I cant see any nodes in the network.

nRF Connect app finds BLE mesh device but as you below image, the device advertise with different MAC every 5 secs.

Only one mesh device exist in the environment.

Questions:

1- How I can see same network mesh devices in nRF mesh app? 

2-  Why same mesh device changes its MAC at every advertisement?

3- My final goal is have multiple beacons and multiple relays and one gateway node. They will have hardcoded individual node addresses. Is it possible create this mesh structure using this example as a reference?

nRF Mesh app settings: Network settings

Unicast Address: 0002

Network Key: 0123456789ABCDEF0123456789ABCDEF

App Key: 0123456789ABCDEF0123456789ABCDEF

nRF Connect app, same device but advertise with different MAC:

I didn't change the code except prj.conf for nRF-52 DK:

CONFIG_MAIN_STACK_SIZE=1024
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048

CONFIG_BT=y
CONFIG_BT_TINYCRYPT_ECC=y
CONFIG_BT_OBSERVER=y
CONFIG_BT_BROADCASTER=y

CONFIG_BT_MESH=y
CONFIG_BT_MESH_RELAY=y
CONFIG_BT_MESH_SUBNET_COUNT=1
CONFIG_BT_MESH_APP_KEY_COUNT=1
CONFIG_BT_MESH_MODEL_GROUP_COUNT=2
CONFIG_BT_MESH_ADV_BUF_COUNT=10
CONFIG_BT_MESH_LABEL_COUNT=0
CONFIG_BT_MESH_PB_ADV=n
CONFIG_BT_MESH_CFG_CLI=y
CONFIG_BT_MESH_LOOPBACK_BUFS=8

CONFIG_BT_SETTINGS=y
CONFIG_FLASH=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FLASH_MAP=y
CONFIG_NVS=y
CONFIG_SETTINGS=y
CONFIG_BT_MESH_RPL_STORE_TIMEOUT=600

Terminal output:

[Hieu]

Hi Akif,

I don't think that sample can run out of the box with the nRF52 DK. I cannot run the sample successfully on NCS v2.3.0. What NCS version are you using, and what modifications have you made to it?

1- How I can see same network mesh devices in nRF mesh app? 

Update*: My answer below for Akif's question #1 is off the mark. Please refer to my later reply for explanation on why the nRF Mesh app, or any similar app, cannot see the list of nodes provisioned automatically.

The nRF Mesh app is still running on the phone and is not a BT Mesh device. Even with the same network and application key, it needs a Proxy Node to join the network. You need to configure one of the nodes as a proxy for the app to connect to.

The Add Node button is to provision nodes. As the demo nodes provision themselves, this is not necessary.

The Connect button is to connect to a Proxy node, which I explained above.

You will also need to configure the nRF Mesh app and the boards to have the same Network Key and Application Key. Have you done this?

2-  Why same mesh device changes its MAC at every advertisement?

The device is advertising using random address to ensure security.

3- My final goal is have multiple beacons and multiple relays and one gateway node. They will have hardcoded individual node addresses. Is it possible create this mesh structure using this example as a reference?

Is this for a demo or is this for production? The sample saves you the provisioning step by provisioning themselves using hard-coded keys. The address for each device also has to be provided at build with 

This is not secure and can pose many problems. It is noted in the sample's documentation:

This sample is a Bluetooth mesh application intended for demonstration purposes only. The application provisions and configures itself (i.e. no external provisioner needed) with hard-coded network and application key values. The local unicast address can be set using a NODE_ADDR build variable (e.g. NODE_ADDR=0x0001 for unicast address 0x0001), or by manually editing the value in the board.h file.

Because of the hard-coded values, the application is not suitable for production use, but is quite convenient for quick demonstrations of mesh functionality.

Hieu

[akif]

Sorry but I am not sure you have read my post clearly.

1- The only modification I did at prj.conf and clearly mentioned and shared in the original post "I didn't change the code except prj.conf for nRF-52 DK". Interesting that you cant compile, only proj.conf needs to be changed. I have mentioned I used NCS v2.30 in the original post. Please can you try again?

Yes, I set the nrf Mesh App. Please see nRF Mesh app settings in the original post  Why still I cant see the device on nRF Mesh App but I see on nRF Connect App?

2- Thanks. It would be helpful if you can share a reference document about . I just want to understand the idea.

3-  Yes, it is for production. Thank you for sharing a part of .readme in the post. I read that part. If we build every node with individual Node Address, I think we can use this project as reference. Isn't it?
Why it is not secure and can pose many problems? What causes problems, please can you explain in detail?


After my original post, I have removed buzzer and display functions (because they are for only BBC Microbit board ) on the Bluetooth: Mesh Demo. When I press the button the device should send a messege. But, bt_mesh_model_send returns -22 which is -EINVAL (Invalid argument). Please see below code:

	/* Bind to Health model */
	bt_mesh_model_msg_init(&msg, OP_VENDOR_BUTTON);

	if ( (ret = bt_mesh_model_send(&vnd_models[0], &ctx, &msg, NULL, NULL)) != 0) {
		printk("Unable to send Vendor Button message %d\n",ret);
	}

If I look the bt_mesh_model_send it returns EINVAL if "Model not bound to AppKey". Please see below code:

int bt_mesh_model_send(struct bt_mesh_model *model, struct bt_mesh_msg_ctx *ctx,
		       struct net_buf_simple *msg,
		       const struct bt_mesh_send_cb *cb, void *cb_data)
{
	if (!bt_mesh_model_has_key(model, ctx->app_idx)) {
		LOG_ERR("Model not bound to AppKey 0x%04x", ctx->app_idx);
		return -EINVAL;
	}

	return bt_mesh_access_send(ctx, msg, bt_mesh_model_elem(model)->addr, cb, cb_data);
}

I am using Mesh Demo example original values for app key and other constants and variables. I didn't change anything. What do you think about this issue?

[Hieu]

Hello Akif,

akif said:

Sorry but I am not sure you have read my post clearly.

My apologies. I did miss your mention of the SDK version and the change you made in main stack size. With the increased stack size, the sample has now run for me.

akif said:

Why still I cant see the device on nRF Mesh App but I see on nRF Connect App?

The nRF Connect App scan shows all device within your phone/device scanning range. However, it has no information about the mesh network. If you press on the mesh advertisements, you will see that the data are all encrypted. 

Regarding the nRF Mesh App, I am sorry again. Connecting to a Proxy Node let the app "join" the network, but it does not show a full list of nodes.

The nRF Mesh App lists the nodes in the network which it has knowledge of. However, as the provisioning process happened independent of the app, it does not know which nodes have joined the network.

In order to give the nRF Mesh App that knowledge, you need to import a Configuration Database JSON file with information of all the nodes.

A way to create this file is to export the current network of the nRF Mesh App into a JSON file, and update that file.

On Android, you can find the options to export and import the database by expanding the "kebab menu" in the Settings tab.
They should also be at a similar location on the iOS version of the app, but I don't have an iOS device to verify this.

The format of the JSON file is specified in the Mesh Configuration Database Profile specification. Here is the link to v1.0 of the spec: https://www.bluetooth.com/specifications/specs/mesh-configuration-database-profile-1-0/.

Please note that to let the phone app join the network and see the nodes' statuses live, you still need a Proxy node. This can be done by adding the following Kconfig:

CONFIG_BT_PERIPHERAL=y

CONFIG_BT_MESH_GATT_PROXY=y

CONFIG_BT_MESH_GATT_PROXY_ENABLED=y

akif said:

Thanks. It would be helpful if you can share a reference document about [the random addresses]. I just want to understand the idea.

You probably know this already, but the addresses you see in the nRF Connect app are not Bluetooth Mesh Addresses, but the Bluetooth Device Addresses.

You can find an introduction to address types in BLE in this video from Ellisys: Intro to Bluetooth Addresses and Privacy | Bluetooth Technology Website.

If you prefer a written explanation, they are also discussed in this DevZone thread:  GAP Address types .

The general idea is that the random addresses help prevent a node from being easily tracked and vulnerable to attacks.

The source and destination of a mesh address is encrypted in the packets and can be decrypted by a node with the correct key.

akif said:

3-  Yes, it is for production. Thank you for sharing a part of .readme in the post. I read that part. If we build every node with individual Node Address, I think we can use this project as reference. Isn't it?
Why it is not secure and can pose many problems? What causes problems, please can you explain in detail?

In normal provisioning use cases, the Network Key and Application Keys are generated new and completely random.

With the hardcoded keys, all of your products share one key. If this key is ever compromised, all of your products, at all locations across the world, are compromised.

As the key is hardcoded, your mesh network also cannot perform the Key Refresh procedure. 

As your code contain the key, the privacy and security of your customers are also compromised because anyone with access to the source code know the key to their network. 

Because the key is the same for all customers, all of their devices are theoretically in the same network. If two customers' network physically overlap, then they would "merge" and nodes of a customer would start working in the other customer's network.

Of course, you can build the firmware with a new key for each customer, but I cannot imagine a good way to manage that, and that is certainly a lot more work.

There are also several other downsides with this setup as is:

- Setting up a mobile app to see all nodes can be cumbersome, as described above.
- Each node needs its own firmware build with its node address hardcoded in.
- Only one Application Key, allowing only one Application.

Please note that the above is not an exhaustive list of all possible issues.

---

Regarding your recent modification, you are also adding button logic for the nRF52 DK, right? The micro:bit code should not compile at all when you build for the nrf52dk_nrf52832 board, and even if it compiles, there are no "buttona" and "buttonb" as refered to by the code.

Please see that "Model not bound to app key" is not the only possible scenario, as bt_mesh_model_send() also returns the error codes from bt_mesh_access_send().

I couldn't get to debugging this yet, but I will arrange for it within the next two days.

Should you choose to not use the mesh_demo sample as a reference anymore, please let me know and we could skip that debugging.

[akif]

No problem. Thank you for all information.

Hieu said:

Connecting to a Proxy Node let the app "join" the network, but it does not show a full list of nodes.

Can we iterate (with changing configuration?) a Proxy Node application from nRF from Mesh Demo application (with adding GATT), or do you suggest any other example application for Proxy Node?

Thank you for explanation on Mesh Configuration Database Profile and random addresses, really helpful, I will dig into.

Thank you for downsides of auto provisioning. But in the manual provisioning -as I know- if a node resets, the node needs to be added to network again manually, isn't it? Looks like, the auto provisioning eliminates this problem.

---

Hieu said:

Regarding your recent modification, you are also adding button logic for the nRF52 DK, right? The micro:bit code should not compile at all when you build for the nrf52dk_nrf52832 board, and even if it compiles, there are no "buttona" and "buttonb" as refered to by the code.

Yes, I am using the button logic, I have assigned sw0 for "buttona" and sw1 for "buttonb" on nRF52 DK. Also I have tried on nrf52840dk. And also with micro:bit v2 board. I get "Model not bound to app key" error on every board.

Hieu said:

Please see that "Model not bound to app key" is not the only possible scenario, as bt_mesh_model_send() also returns the error codes from bt_mesh_access_send().

According to my debugging the bt_mesh_model_send() -> bt_mesh_model_has_key() causes return EINVAL. It would be nice if you can do this debugging, you too. Thanks

[Hieu]

akif said:

Can we iterate (with changing configuration?) a Proxy Node application from nRF from Mesh Demo application (with adding GATT), or do you suggest any other example application for Proxy Node?

Yes, you can just add the Kconfig I listed in my previous reply and it should setup the Proxy feature on the node.

akif said:

But in the manual provisioning -as I know- if a node resets, the node needs to be added to network again manually, isn't it? Looks like, the auto provisioning eliminates this problem.

It depends on what kind of reset you are talking about.

If you mean removing the node out of the network, then yes. 
After removing the node, it is just a normal unprovisioned node, so the difference is simply the exact same as in the case of adding that node to the mesh for the first time.
However, I don't think this case is common in normal use cases.

If you are talking about updating the firmware on the node, then as long as the mesh configurations in non-volatile storage is untouched, the node will rejoin the network just fine, no re-provisioning necessary.

What kind of "reset" scenario are you expecting?

akif said:

According to my debugging the bt_mesh_model_send() -> bt_mesh_model_has_key() causes return EINVAL. It would be nice if you can do this debugging, you too. Thanks

I found the root cause for this.

The way the sample bind an app key to its model is by using its own Configuration Client to write to its own Configuration Server.

Let's look at the following log:

*** Booting Zephyr OS build v3.2.99-ncs2 ***
[00:00:00.000,549] <inf> main: Initializing...

[00:00:00.000,610] <inf> main: Unicast address: 0x0005

[00:00:00.007,843] <inf> fs_nvs: 8 Sectors of 4096 bytes
[00:00:00.007,843] <inf> fs_nvs: alloc wra: 0, fe8
[00:00:00.007,843] <inf> fs_nvs: data wra: 0, 0
[00:00:00.007,965] <inf> bt_sdc_hci_driver: SoftDevice Controller build revision:
                                            d8 0c 2d 2f 36 ae e2 5c  80 26 80 4c 3f 4d 16 53 |..-/6..\ .&.L?M.S
                                            50 96 c7 73                                      |P..s
[00:00:00.010,894] <inf> bt_hci_core: No ID address. App must call settings_load()
[00:00:00.010,894] <inf> main: Bluetooth initialized

[00:00:00.010,986] <inf> main: Mesh initialized

[00:00:00.011,016] <inf> main: Loading stored settings

[00:00:00.011,383] <inf> bt_mesh_main: Primary Element: 0x0005
[00:00:00.011,383] <dbg> bt_mesh_main: bt_mesh_provision: net_idx 0x0000 flags 0x00 iv_index 0x0000
[00:00:00.024,383] <inf> main: Provisioning completed

[00:00:00.024,383] <inf> main: configure1...

[00:00:00.024,688] <inf> main: bt_mesh_cfg_cli_app_key_add 0
[00:00:03.024,749] <inf> main: configure2...

[00:00:03.024,932] <inf> main: bt_mesh_cfg_cli_mod_app_bind_vnd 0
[00:00:03.025,085] <inf> main: bt_mesh_cfg_cli_mod_app_bind 0
[00:00:03.025,238] <inf> main: bt_mesh_cfg_cli_mod_sub_add_vnd 0
[00:00:03.025,238] <inf> main: Configuration complete

[00:00:03.025,390] <inf> main: Subscribing to heartbeat messages

[00:00:03.190,002] <dbg> bt_mesh_cfg_srv: get_model: Company 0x05f1 ID 0x0000 addr 0x0005
[00:00:03.190,032] <dbg> bt_mesh_cfg_srv: mod_bind: model 0x200005f0 key_idx 0x000
[00:00:03.190,032] <dbg> bt_mesh_cfg_srv: mod_app_bind: status 0x03
[00:00:03.190,460] <dbg> bt_mesh_cfg_srv: get_model: ID 0x0002 addr 0x0005
[00:00:03.190,490] <dbg> bt_mesh_cfg_srv: mod_bind: model 0x20000668 key_idx 0x000
[00:00:03.190,643] <dbg> bt_mesh_cfg_srv: mod_app_bind: status 0x03
[00:00:03.190,979] <dbg> bt_mesh_cfg_srv: mod_sub_add: elem_addr 0x0005, sub_addr 0xc000
[00:00:03.191,009] <dbg> bt_mesh_cfg_srv: get_model: Company 0x05f1 ID 0x0000 addr 0x0005
[00:00:03.191,040] <dbg> bt_mesh_cfg_srv: send_mod_sub_status: status 0x00 elem_addr 0x0005 sub_addr 0xc000
[00:00:03.191,497] <dbg> bt_mesh_cfg_srv: heartbeat_sub_set: src 0x0005
[00:00:03.191,497] <dbg> bt_mesh_cfg_srv: heartbeat_sub_set: sub_src 0x000f sub_dst 0xc000 period 0x10
[00:00:03.191,528] <dbg> bt_mesh_cfg_srv: hb_sub_send_status: src 0x0005
[00:00:03.191,925] <dbg> bt_mesh_cfg_cli: mod_app_status: net_idx 0x0000 app_idx 0xfffe src 0x0005 len 9: 0305000000f1050000
[00:00:03.192,169] <dbg> bt_mesh_cfg_cli: mod_app_status: net_idx 0x0000 app_idx 0xfffe src 0x0005 len 7: 03050000000200
[00:00:03.192,352] <dbg> bt_mesh_cfg_cli: mod_sub_status: net_idx 0x0000 app_idx 0xfffe src 0x0005 len 9: 00050000c0f1050000
[00:00:03.192,565] <dbg> bt_mesh_cfg_cli: hb_sub_status: net_idx 0x0000 app_idx 0xfffe src 0x0005 len 9: 000f0000c010007f00
[00:00:03.192,932] <dbg> bt_mesh_cfg_srv: app_key_add: AppIdx 0x0000 NetIdx 0x0000
[00:00:03.194,763] <dbg> bt_mesh_cfg_cli: app_key_status: net_idx 0x0000 app_idx 0xfffe src 0x0005 len 4: 00000000

You can see that the code tries to, via the Configuration Client, tell its own Configuration Server to do the following operations:

- Add an Application Key
- Bind the Application Key to its Health Server
- Bind the Application Key to its Vendor Model

However, in the Configuration Server logs, we see that the Application Key adding operation is run after the binding operations.

Therefore, the Application Key is not available when the key binding operations run, and the binding operations fail.

Having said all of that, I haven't found a solution for this. I tried changing the order of the Configuration Client function calls, or adding a sleep wait in between, but those don't work. The Application Key adding still happens last.

Ideally, we need to setup a callback that is triggered when the Application Key adding is done, and only then perform the other operations.  

I am out of office for the rest of the week, so I will have to stop here for a few days. Do you think you can try to discover how to achieve that callback on your own?

---

In case you are wondering about the difference with your log, mine is indeed from the same mesh_demo, but I made the following changes which affect the log:

- Replaced printk with logging with timestamps
- Enable logging on the Configuration Server (cfg_srv) and Configuration Client (cfg_cli) modules
- Separated the configure() function into configuration1() with the app key adding, and configuration2() with the rest (to test swapping the order)

[Hieu]

P.s: A very crude workaround comes to mind. You can modify the code so that:

• On initialization when the device is not yet provision, add the app key to the node. The model features won't work yet in this boot.  
• On initialization where the device has been provisioned, run the key binding. From this point on, the features should work normally.
• Thus, to get things run, after flashing the device and let it run for the first time, simply reset it and things should then work.

It is rather crude, and I would not recommend it for production. However, for testing purpose, I think it is acceptable.

As far as this demo setup go, ideally, we should find some way to setup the callback like I described in my last reply.