How to access all advertisement data in scan filter match callback?

[wkpolara]

I'm using nRF Connect v2.0.2 and running in Zephyr.

When scanning by filtering for UUID, I find the matching devices, and the scan filter match callback gets called.  However, when I try to parse the advertising data, I can only see one type.

Example: If I filter on UUID, the advertising data parse only contains UUID type

BT_SCAN_CB_INIT(scan_cbhs, scan_filter_match, NULL, NULL, NULL);

start_scanning_function()
{
    struct bt_scan_init_param scan_init = {
		.connect_if_match = 0,
		.scan_param = BT_LE_SCAN_ACTIVE,   // can this be passive?
		.conn_param = BT_LE_CONN_PARAM_DEFAULT
	};

	bt_scan_init(&scan_init);
	bt_scan_cb_register(&scan_cbhs);

    bt_scan_filter_add(BT_SCAN_FILTER_TYPE_UUID, BT_UUID_DECLARE_128(BLE_UUID128_ENCODE(K_BLE_UUID_FILTER_VAL)));

	bt_scan_filter_enable(BT_SCAN_UUID_FILTER, false);
   
    bt_scan_start(BT_SCAN_TYPE_SCAN_ACTIVE);
}

static void scan_filter_match(struct bt_scan_device_info *device_info,
			      struct bt_scan_filter_match *filter_match,
			      bool connectable)
{
    bt_data_parse(device_info->adv_data, find_adv_data, NULL);
}

bool find_adv_data(struct bt_data *data, void *user_data)
{
    printk("Data type is %u \n", data->type);
    
    return true; // keep parsing
}

In the code example above, the only print is
Parse type 7
which is I think is BT_DATA_UUID128_ALL, but not other calls to the parser callback

Similarly if I filter on device name instead of UUID, then I only get
Parse type 1
which I think is BT_DATA_FLAGS and that's it.

If I scan and filter based on UUID, how do I access the advertised device name, manufacturer data, or other data from the device's advertisement?

Thanks

[Vidar Berg]

Hello,

I did not get around to testing this here today, but I suspect what happens is that the advertising buffer is overwritten while you are parsing it. With legacy advertising, you usually have to place the 128-bit vendor UUIDs in the scan response packet due to the limited packet size (31 octets), which means the payload will become split across 2 advertisement packets.

Similarly if I filter on device name instead of UUID, then I only get
Parse type 1
which I think is BT_DATA_FLAGS and that's it.

When using this filter, could you try to start scanning with bt_scan_start(BT_SCAN_TYPE_SCAN_PASSIVE) instead of bt_scan_start(BT_SCAN_TYPE_SCAN_ACTIVE) to see if you are able to parse the remaining fields? With passive scanning, you will not receive the scan response packet with the 128-bit UUID.

Either way, I will test this tomorrow and update the ticket.

Best regards,

Vidar

[wkpolara]

Fullscreen 44230.main.c Download

/*
 * Copyright (c) 2019 Nordic Semiconductor ASA
 *
 * SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
 */

/** @file
 *  @brief Nordic Battery Service Client sample
 */

#include <zephyr/types.h>
#include <stddef.h>
#include <inttypes.h>
#include <errno.h>
#include <zephyr/kernel.h>
#include <zephyr/sys/printk.h>

#include <zephyr/bluetooth/bluetooth.h>
#include <zephyr/bluetooth/hci.h>
#include <zephyr/bluetooth/conn.h>
#include <zephyr/bluetooth/uuid.h>
#include <zephyr/bluetooth/gatt.h>
#include <bluetooth/gatt_dm.h>
#include <bluetooth/scan.h>
#include <dk_buttons_and_leds.h>

#include <zephyr/settings/settings.h>

/**
 * Button to read the battery value
 */
#define KEY_READVAL_MASK DK_BTN1_MSK


#define K_BLE_UUID_WPC_VAL 	        BT_UUID_128_ENCODE(0xAD014001,0x3BCB,0xFEA6,0x7746,0xC3084AD1DF97)


static struct bt_conn *default_conn;


bool find_in_adv_data(struct bt_data *data, void *user_data)
{
    ARG_UNUSED(user_data);

    printk("Parse type %u \n",data->type );

    return true;
}

static void scan_filter_match(struct bt_scan_device_info *device_info,
			      struct bt_scan_filter_match *filter_match,
			      bool connectable)
{
	char addr[BT_ADDR_LE_STR_LEN];

	bt_addr_le_to_str(device_info->recv_info->addr, addr, sizeof(addr));

    bt_data_parse(device_info->adv_data, find_in_adv_data, NULL);

	printk("Filters matched. Address: %s connectable: %s\n",
		addr, connectable ? "yes" : "no");
}

static void scan_connecting_error(struct bt_scan_device_info *device_info)
{
	printk("Connecting failed\n");
}

static void scan_connecting(struct bt_scan_device_info *device_info,
			    struct bt_conn *conn)
{
	default_conn = bt_conn_ref(conn);
}

static void scan_filter_no_match(struct bt_scan_device_info *device_info,
				 bool connectable)
{
	int err;
	struct bt_conn *conn;
	char addr[BT_ADDR_LE_STR_LEN];

	if (device_info->recv_info->adv_type == BT_GAP_ADV_TYPE_ADV_DIRECT_IND) {
		bt_addr_le_to_str(device_info->recv_info->addr, addr,
				  sizeof(addr));
		printk("Direct advertising received from %s\n", addr);
		bt_scan_stop();

		err = bt_conn_le_create(device_info->recv_info->addr,
					BT_CONN_LE_CREATE_CONN,
					device_info->conn_param, &conn);

		if (!err) {
			default_conn = bt_conn_ref(conn);
			bt_conn_unref(conn);
		}
	}
}

BT_SCAN_CB_INIT(scan_cb, scan_filter_match, scan_filter_no_match,
		scan_connecting_error, scan_connecting);

static void connected(struct bt_conn *conn, uint8_t conn_err)
{
	char addr[BT_ADDR_LE_STR_LEN];

	bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr));


	printk("Connected: %s\n", addr);

}

static void disconnected(struct bt_conn *conn, uint8_t reason)
{
	char addr[BT_ADDR_LE_STR_LEN];

	bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr));

	printk("Disconnected: %s (reason %u)\n", addr, reason);

}


BT_CONN_CB_DEFINE(conn_callbacks) = {
	.connected = connected,
	.disconnected = disconnected,
};

static void scan_init(void)
{
	int err;

	struct bt_scan_init_param scan_init = {
		.connect_if_match = 1,
		//.scan_param = BT_LE_SCAN_PASSIVE,
        .scan_param = BT_LE_SCAN_ACTIVE,
		.conn_param = BT_LE_CONN_PARAM_DEFAULT
	};

	bt_scan_init(&scan_init);
	bt_scan_cb_register(&scan_cb);

	err = bt_scan_filter_add(BT_SCAN_FILTER_TYPE_UUID, BT_UUID_DECLARE_128(K_BLE_UUID_WPC_VAL));
	if (err) {
		printk("Scanning filters cannot be set (err %d)\n", err);

		return;
	}

	err = bt_scan_filter_enable(BT_SCAN_UUID_FILTER, false);
	if (err) {
		printk("Filters cannot be turned on (err %d)\n", err);
	}
}



void main(void)
{
	int err;

	printk("Starting Bluetooth Central BAS example\n");


	err = bt_enable(NULL);
	if (err) {
		printk("Bluetooth init failed (err %d)\n", err);
		return;
	}

	printk("Bluetooth initialized\n");

	if (IS_ENABLED(CONFIG_SETTINGS)) {
		settings_load();
	}

	scan_init();

	err = bt_scan_start(BT_SCAN_TYPE_SCAN_ACTIVE);
    //err = bt_scan_start(BT_SCAN_TYPE_SCAN_PASSIVE);
	if (err) {
		printk("Scanning failed to start (err %d)\n", err);
		return;
	}

	printk("Scanning successfully started\n");
}

Hi Vidar

I tried starting the scan with BT_SCAN_TYPE_SCAN_PASSIVE, but did not get any scan results.  (I think this is because the devices I'm scanning for do not include the UUID in their legacy advertisements.)

I've modified and simplified one of the sample projects to show the issue, and am attaching the main.c and prj.conf for it.  This code builds with the latest SDK v2.2.0.

You'll want to change the value of K_BLE_UUID_WPC_VAL to use an advertised UUID from a peripheral to test with.

I ran this on a ubx_bmd340eval_nrf52840 board.

Regards,
Walt

[wkpolara]

Fullscreen 3617.prj.conf.txt Download

#
# Copyright (c) 2019 Nordic Semiconductor ASA
#
# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
#
CONFIG_NCS_SAMPLES_DEFAULTS=y

CONFIG_BT=y
CONFIG_BT_DEBUG_LOG=y
CONFIG_BT_CENTRAL=y
CONFIG_BT_GATT_CLIENT=y
CONFIG_BT_GATT_DM=y
CONFIG_HEAP_MEM_POOL_SIZE=1024

CONFIG_BT_SCAN=y
CONFIG_BT_SCAN_FILTER_ENABLE=y
CONFIG_BT_SCAN_UUID_CNT=1

CONFIG_BT_SETTINGS=y
CONFIG_FLASH=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FLASH_MAP=y
CONFIG_NVS=y
CONFIG_SETTINGS=y

CONFIG_DK_LIBRARY=y

[Vidar Berg]

Hi Walt,

It looks like my assumption was wrong - the problem is not that the advertisement report buffer is overwritten during parsing, but rather that the scanner module's 'match_all' option fails when we are looking for matching fields in separate advertising packets.

  

Here is a RTT log showing a modified version of the central_uart sample parsing both the advertising and scan response packet before initiating the connection:

I copied my changes over to your main.c. Please try this and see if it works for you.

Modified main.c

Fullscreen 12787.main.c Download

/*
 * Copyright (c) 2019 Nordic Semiconductor ASA
 *
 * SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
 */

/** @file
 *  @brief Nordic Battery Service Client sample
 */

#include <zephyr/types.h>
#include <stddef.h>
#include <inttypes.h>
#include <errno.h>
#include <zephyr/kernel.h>
#include <zephyr/sys/printk.h>

#include <zephyr/bluetooth/bluetooth.h>
#include <zephyr/bluetooth/hci.h>
#include <zephyr/bluetooth/conn.h>
#include <zephyr/bluetooth/uuid.h>
#include <zephyr/bluetooth/gatt.h>
#include <bluetooth/gatt_dm.h>
#include <bluetooth/scan.h>
#include <dk_buttons_and_leds.h>

#include <zephyr/settings/settings.h>

/**
 * Button to read the battery value
 */
#define KEY_READVAL_MASK DK_BTN1_MSK


#define K_BLE_UUID_WPC_VAL 	        BT_UUID_128_ENCODE(0xAD014001,0x3BCB,0xFEA6,0x7746,0xC3084AD1DF97)


static struct bt_conn *default_conn;


bool find_in_adv_data(struct bt_data *data, void *user_data)
{
    ARG_UNUSED(user_data);

    printk("Parse type %u \n",data->type );

    return true;
}

static void scan_filter_match(struct bt_scan_device_info *device_info,
			      struct bt_scan_filter_match *filter_match,
			      bool connectable)
{
	int err; 
	char addr[BT_ADDR_LE_STR_LEN];
	struct bt_conn_le_create_param *conn_params;
	static uint32_t matched_filters;

	bt_addr_le_to_str(device_info->recv_info->addr, addr, sizeof(addr));

    bt_data_parse(device_info->adv_data, find_in_adv_data, NULL);

	printk("Filters matched. Address: %s connectable: %s\n",
		addr, connectable ? "yes" : "no");

	if (filter_match->name.match) {
		matched_filters |= BT_SCAN_NAME_FILTER;
	} else if (filter_match->uuid.match) {
		matched_filters |= BT_SCAN_UUID_FILTER;
	}

	/* Initiate connection if we have had a match on both filters */
	//TODO: check if address filters where matched for the same device? 
	if ((matched_filters & BT_SCAN_NAME_FILTER) &&
	    (matched_filters & BT_SCAN_UUID_FILTER)) {
		
		matched_filters = 0;

		err = bt_scan_stop();
		if (err) {
			LOG_INF("Stop LE scan failed (err %d)", err);
		}

		conn_params = BT_CONN_LE_CREATE_PARAM(BT_CONN_LE_OPT_NO_1M,
						      BT_GAP_SCAN_FAST_INTERVAL,
						      BT_GAP_SCAN_FAST_INTERVAL);

		err = bt_conn_le_create(device_info->recv_info->addr, conn_params,
					BT_LE_CONN_PARAM_DEFAULT, &default_conn);
		if (err) {
			LOG_INF("Create conn failed (err %d)", err);

			err = bt_scan_start(BT_SCAN_TYPE_SCAN_ACTIVE);
			if (err) {
				LOG_INF("Scanning failed to start (err %d)", err);
				return;
			}
		}

		LOG_INF("Connection pending");
	}
}

static void scan_connecting_error(struct bt_scan_device_info *device_info)
{
	printk("Connecting failed\n");
}

static void scan_connecting(struct bt_scan_device_info *device_info,
			    struct bt_conn *conn)
{
	default_conn = bt_conn_ref(conn);
}

static void scan_filter_no_match(struct bt_scan_device_info *device_info,
				 bool connectable)
{
	int err;
	struct bt_conn *conn;
	char addr[BT_ADDR_LE_STR_LEN];

	if (device_info->recv_info->adv_type == BT_GAP_ADV_TYPE_ADV_DIRECT_IND) {
		bt_addr_le_to_str(device_info->recv_info->addr, addr,
				  sizeof(addr));
		printk("Direct advertising received from %s\n", addr);
		bt_scan_stop();

		err = bt_conn_le_create(device_info->recv_info->addr,
					BT_CONN_LE_CREATE_CONN,
					device_info->conn_param, &conn);

		if (!err) {
			default_conn = bt_conn_ref(conn);
			bt_conn_unref(conn);
		}
	}
}

BT_SCAN_CB_INIT(scan_cb, scan_filter_match, scan_filter_no_match,
		scan_connecting_error, scan_connecting);

static void connected(struct bt_conn *conn, uint8_t conn_err)
{
	char addr[BT_ADDR_LE_STR_LEN];

	bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr));


	printk("Connected: %s\n", addr);

}

static void disconnected(struct bt_conn *conn, uint8_t reason)
{
	char addr[BT_ADDR_LE_STR_LEN];

	bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr));

	printk("Disconnected: %s (reason %u)\n", addr, reason);

}


BT_CONN_CB_DEFINE(conn_callbacks) = {
	.connected = connected,
	.disconnected = disconnected,
};

static void scan_init(void)
{
	int err;

	struct bt_scan_init_param scan_init = {
		.connect_if_match = 0,
		//.scan_param = BT_LE_SCAN_PASSIVE,
        .scan_param = BT_LE_SCAN_ACTIVE,
		.conn_param = BT_LE_CONN_PARAM_DEFAULT
	};

	bt_scan_init(&scan_init);
	bt_scan_cb_register(&scan_cb);

	err = bt_scan_filter_add(BT_SCAN_FILTER_TYPE_UUID, BT_UUID_DECLARE_128(K_BLE_UUID_WPC_VAL));
	if (err) {
		printk("Scanning filters cannot be set (err %d)\n", err);

		return;
	}

	err = bt_scan_filter_add(BT_SCAN_FILTER_TYPE_UUID, BT_UUID_DECLARE_128(K_BLE_UUID_WPC_VAL));
	if (err) {
		printk("Scanning filters cannot be set (err %d)\n", err);

		return;
	}

	err = bt_scan_filter_add(BT_SCAN_FILTER_TYPE_NAME, <insert name of target device here>);
	if (err) {
		LOG_ERR("Scanning filters cannot be set (err %d)", err);
		return err;
	}

	err = bt_scan_filter_enable(BT_SCAN_UUID_FILTER | BT_SCAN_NAME_FILTER, false);
	if (err) {
		printk("Filters cannot be turned on (err %d)\n", err);
	}
}



void main(void)
{
	int err;

	printk("Starting Bluetooth Central BAS example\n");


	err = bt_enable(NULL);
	if (err) {
		printk("Bluetooth init failed (err %d)\n", err);
		return;
	}

	printk("Bluetooth initialized\n");

	if (IS_ENABLED(CONFIG_SETTINGS)) {
		settings_load();
	}

	scan_init();

	err = bt_scan_start(BT_SCAN_TYPE_SCAN_ACTIVE);
    //err = bt_scan_start(BT_SCAN_TYPE_SCAN_PASSIVE);
	if (err) {
		printk("Scanning failed to start (err %d)\n", err);
		return;
	}

	printk("Scanning successfully started\n");
}

Changes

Fullscreen 8836.diff.diff Download

diff --git a/main.c b/main.c
index 1deb208..d693f4e 100644
--- a/main.c
+++ b/main.c
@@ -51,7 +51,10 @@ static void scan_filter_match(struct bt_scan_device_info *device_info,
 			      struct bt_scan_filter_match *filter_match,
 			      bool connectable)
 {
+	int err; 
 	char addr[BT_ADDR_LE_STR_LEN];
+	struct bt_conn_le_create_param *conn_params;
+	static uint32_t matched_filters;
 
 	bt_addr_le_to_str(device_info->recv_info->addr, addr, sizeof(addr));
 
@@ -59,6 +62,43 @@ static void scan_filter_match(struct bt_scan_device_info *device_info,
 
 	printk("Filters matched. Address: %s connectable: %s\n",
 		addr, connectable ? "yes" : "no");
+
+	if (filter_match->name.match) {
+		matched_filters |= BT_SCAN_NAME_FILTER;
+	} else if (filter_match->uuid.match) {
+		matched_filters |= BT_SCAN_UUID_FILTER;
+	}
+
+	/* Initiate connection if we have had a match on both filters */
+	//TODO: check if address filters where matched for the same device? 
+	if ((matched_filters & BT_SCAN_NAME_FILTER) &&
+	    (matched_filters & BT_SCAN_UUID_FILTER)) {
+		
+		matched_filters = 0;
+
+		err = bt_scan_stop();
+		if (err) {
+			LOG_INF("Stop LE scan failed (err %d)", err);
+		}
+
+		conn_params = BT_CONN_LE_CREATE_PARAM(BT_CONN_LE_OPT_NO_1M,
+						      BT_GAP_SCAN_FAST_INTERVAL,
+						      BT_GAP_SCAN_FAST_INTERVAL);
+
+		err = bt_conn_le_create(device_info->recv_info->addr, conn_params,
+					BT_LE_CONN_PARAM_DEFAULT, &default_conn);
+		if (err) {
+			LOG_INF("Create conn failed (err %d)", err);
+
+			err = bt_scan_start(BT_SCAN_TYPE_SCAN_ACTIVE);
+			if (err) {
+				LOG_INF("Scanning failed to start (err %d)", err);
+				return;
+			}
+		}
+
+		LOG_INF("Connection pending");
+	}
 }
 
 static void scan_connecting_error(struct bt_scan_device_info *device_info)
@@ -131,7 +171,7 @@ static void scan_init(void)
 	int err;
 
 	struct bt_scan_init_param scan_init = {
-		.connect_if_match = 1,
+		.connect_if_match = 0,
 		//.scan_param = BT_LE_SCAN_PASSIVE,
         .scan_param = BT_LE_SCAN_ACTIVE,
 		.conn_param = BT_LE_CONN_PARAM_DEFAULT
@@ -147,7 +187,20 @@ static void scan_init(void)
 		return;
 	}
 
-	err = bt_scan_filter_enable(BT_SCAN_UUID_FILTER, false);
+	err = bt_scan_filter_add(BT_SCAN_FILTER_TYPE_UUID, BT_UUID_DECLARE_128(K_BLE_UUID_WPC_VAL));
+	if (err) {
+		printk("Scanning filters cannot be set (err %d)\n", err);
+
+		return;
+	}
+
+	err = bt_scan_filter_add(BT_SCAN_FILTER_TYPE_NAME, <insert name of target device here>);
+	if (err) {
+		LOG_ERR("Scanning filters cannot be set (err %d)", err);
+		return err;
+	}
+
+	err = bt_scan_filter_enable(BT_SCAN_UUID_FILTER | BT_SCAN_NAME_FILTER, false);
 	if (err) {
 		printk("Filters cannot be turned on (err %d)\n", err);
 	}

[wkpolara]

Hi Vidar,

Thanks very much for your response! 

The problem is NOT that we want to match both the name and UUID, which is what the modified code code does.  I testing using a name filter instead of the UUID filter, to see if that allowed seeing the other parts of the advertisement, but it had the same issue - only the BT_DATA_FLAGS could be parsed.

What is needed is to be able to inspect the other advertising fields on a UUID match.  The UUID service is the only advertising element that is consistent among the possible peripherals devices.  The name is based on a serial number so cannot match a specific filter, and the advertisement BT_DATA_MANUFACTURER_DATA is needed, but is also not consistent so cannot be matched to a filter.  

When the UUID filter is matched, how can the other portions of the advertisement be obtained for inspection, such as the BT_DATA_MANUFACTURER_DATA?  For example given this demo code, when the UUID is matched, how could it print out the Device Name and the Manufacturer Data (as hex) ?

Best,

Walt

[Vidar Berg]

Hi Walt,

Thanks for the clarification. The reason I used the name filter was to ensure the scan_filter_match() callback would be invoked for both the advertisement and scan response packet, but I realize now that it will not work for you.

Just for reference, this is the data I had in my advertisement and scan response packet:

From what I can tell by looking at the code, it will not be trivial to support your use case with our current implementation of the Scanning module. The only solution I can think of is to use the scan_filter_match() callback for parsing the packet with the UUID and scan_filter_no_match() for the one not containing the UUID. But this will require your application to keep track of the advertising addresses to know what packets to parse in scan_filter_no_match(). 

Another alternative may be to have the app handle the filtering from the scan_filter_no_match() callback (i.e. not enable the scanner module filters.

I can try to make a basic proof-of-concept of what I had in mind if it helps? 

Best regards,

Vidar

[wkpolara]

Vidar,

Thanks for your timely responses.  It looks like we have an answer, though it may not be the one we expected, which is that on a scan filter match it is not possible to see the full contents of the advertising message.

We have already been looking at just using the scan_filter_no_match() and doing the matching manually so the other advertisement fields can be inspected.  Based on your response, we'll move forward with that approach.  We appreciate your offer, but don't need a concept example.

Regards,

Walt