Issue with maximum size of ext adv data, CONFIG_BT_CTLR_ADV_DATA_LEN_MAX

[Mark G.]

Hi all,

I have an issue with achieving bigger sizes of ext adv data as 31 bytes.

My current BLE code is pretty much the same as the nrf_dm sample code:

https://github.com/nrfconnect/sdk-nrf/blob/main/samples/bluetooth/nrf_dm/src/main.c

I just want to add more data into the "sd[]" array / extended advertising data. But it seems the maximum size is 31 bytes. Although I set:

CONFIG_BT_EXT_ADV=y
CONFIG_BT_CTLR_ADV_DATA_LEN_MAX=192 

in my configuration. Following things I figured out:

The bitfield flag "BT_ADV_EXT_ADV" seems to not be set. Thus in adv.c, hci_set_adv_ext_complete() the total_data_len gets truncated to 31 bytes. The code will internally return that the advertising payload is too large. However when I force setting the flag, then the program will generate an error later on in bt_hci_cmd_send_sync with "W: opcode 0x2038 status 0x12" 

What is the correct way of increasing my advertising size?

[Hung Bui]

Hi Mark, 

If you want to do extended advertising (longer payload) you would need to include BT_LE_ADV_OPT_EXT_ADV into the adv_param_conn. 

When you do extended advertising you will have to choose to have your payload in either advertising data or scan response data. It doesn't work the same as when you do legacy advertising. Please take a look at the description of the bt_le_ext_adv_set_data() function: 

 * When both @ref BT_LE_ADV_OPT_EXT_ADV and @ref BT_LE_ADV_OPT_SCANNABLE are

 * enabled then advertising data is ignored.

 * When @ref BT_LE_ADV_OPT_SCANNABLE is not enabled then scan response data is

 * ignored.

I would strongly suggest to take a look at peripheral_hr_coded  to see how extended advertising is done. 

[Mark G.]

Hi Hung Bui,

ok that is interesting. I am not sure if in this way I can realize my use case. I would appreciate your guidance.

As mentioned I build on top of the nrf_dm example which relies on advertising and the additional scan responses. So in This case. is it possible at all to increase payload size for either of the advertising or the scan response?

This is the current implementation:

#define DEVICE_NAME CONFIG_BT_DEVICE_NAME
#define DEVICE_NAME_LEN (sizeof(DEVICE_NAME) - 1)

#define SUPPORT_DM_CODE 0xFF55AA5A

struct adv_mfg_data
{
    uint16_t company_code;    /* Company Identifier Code. */
    uint32_t support_dm_code; /* To identify the device that supports distance measurement. */
    uint32_t rng_seed;        /* Random seed used for generating hopping patterns. */
} __packed;

static adv_mfg_data mfg_data;
bt_le_adv_param adv_param_conn = BT_LE_ADV_PARAM_INIT(BT_LE_ADV_OPT_CONN | BT_LE_ADV_OPT_NOTIFY_SCAN_REQ,
                                                      BT_GAP_ADV_FAST_INT_MIN_2, BT_GAP_ADV_FAST_INT_MAX_2, NULL);

bt_le_adv_param* adv_param = &adv_param_conn;

// double posData[3] = {0, 1, 2};
float posData[1] = {0};
static const bt_data ad[] = {
    BT_DATA_BYTES(BT_DATA_FLAGS, (BT_LE_AD_GENERAL | BT_LE_AD_NO_BREDR)),
    BT_DATA(BT_DATA_NAME_COMPLETE, DEVICE_NAME, DEVICE_NAME_LEN),
    //BT_DATA(BT_DATA_INDOOR_POS, posData, sizeof(posData)),
};

static const bt_data sd[] = {
    BT_DATA(BT_DATA_MANUFACTURER_DATA, reinterpret_cast<unsigned char*>(&mfg_data), sizeof(mfg_data)),
    BT_DATA_BYTES(BT_DATA_UUID128_ALL, BT_UUID_DDFS_VAL)};

static bt_le_scan_param scan_param = {
    .type = BT_LE_SCAN_TYPE_ACTIVE,
    .options = BT_LE_SCAN_OPT_NONE,
    .interval = BT_GAP_SCAN_FAST_INTERVAL,
    .window = BT_GAP_SCAN_FAST_WINDOW,
    .timeout = 0,
};

static bt_scan_init_param scan_init = {.scan_param = &scan_param, .connect_if_match = false, .conn_param = nullptr};

static uint32_t scanner_random_share;

static bt_le_ext_adv* adv;
static void adv_work_handle(k_work* item);
static K_WORK_DEFINE(adv_work, adv_work_handle);
static void adv_update_data();
static uint32_t scanner_addr_to_random_share(const bt_addr_t* p_scanner_addr);
static uint32_t get_id_addr_random_share();

static bt_scan_manufacturer_data scan_mfg_data = {
    .data = reinterpret_cast<unsigned char*>(&mfg_data),
    .data_len = sizeof(mfg_data.company_code) + sizeof(mfg_data.support_dm_code),
};

static bool data_cb(bt_data* data, void* user_data)
{
    dm_request req{};

    switch (data->type)
    {
    case BT_DATA_MANUFACTURER_DATA:
        if (sizeof(adv_mfg_data) == data->data_len)
        {
            auto recv_mfg_data = (adv_mfg_data*)data->data;

            bt_addr_le_copy(&req.bt_addr, static_cast<const bt_addr_le_t*>(user_data));
            req.role = DM_ROLE_INITIATOR;
            req.ranging_mode = peer_ranging_mode_get();

            /* We need to make sure that we only initiate a ranging to a single peer.
             * A scan response that is received by this device can be received by
             * multiple other devices which can all start a ranging at the same time
             * as a consequence. To prevent this, we need to make sure that we set a
             * per-peer random as the random seed. This helps the ranging library to
             * avoid interference from other devices trying to range at the same time.
             *
             * This means that the initiator and the reflector need to set the same
             * value for the random seed.
             */
            req.rng_seed = sys_le32_to_cpu(recv_mfg_data->rng_seed) + scanner_random_share;
            req.start_delay_us = 0;
            req.extra_window_time_us = 0;

            dm_request_add(&req);
        }
        return true;
    case BT_DATA_INDOOR_POS:
        // parse position data
        Debug::log("Received pos data\n");
        return true;
    default:
        return true;
    }
}

static uint32_t get_id_addr_random_share()
{
    bt_addr_le_t addrs[CONFIG_BT_ID_MAX];
    size_t count = CONFIG_BT_ID_MAX;

    bt_id_get(addrs, &count);

    __ASSERT(count == 1, "The sample assumes a single ID addr");

    return scanner_addr_to_random_share(&addrs[0].a);
}

static uint32_t scanner_addr_to_random_share(const bt_addr_t* p_scanner_addr)
{
    return (p_scanner_addr->val[0] | p_scanner_addr->val[1] << 8 | p_scanner_addr->val[2] << 16 |
            p_scanner_addr->val[3] << 24) +
           (p_scanner_addr->val[4] | p_scanner_addr->val[5] << 8);
}

static void scan_filter_match(bt_scan_device_info* device_info, bt_scan_filter_match* filter_match, bool connectable)
{
    bt_addr_le_t addr;

    bt_addr_le_copy(&addr, device_info->recv_info->addr);
    peer_supported_add(device_info->recv_info->addr);
    bt_data_parse(device_info->adv_data, data_cb, &addr);
}

BT_SCAN_CB_INIT(scan_cb, scan_filter_match, NULL, NULL, NULL);

static void adv_scanned_cb(bt_le_ext_adv* adv, bt_le_ext_adv_scanned_info* info)
{
    dm_request req{};
    if (peer_supported_test(info->addr))
    {
        bt_addr_le_copy(&req.bt_addr, info->addr);
        req.role = DM_ROLE_REFLECTOR;
        req.ranging_mode = peer_ranging_mode_get();

        /* We need to make sure that we only initiate a ranging to a single peer.
         * A scan response from this device can be received by multiple peers which can
         * all start a ranging at the same time as a consequence. To prevent this,
         * we need to make sure that we set a per-peer random as the random seed.
         * This helps the ranging library to avoid interference from other devices
         * trying to range at the same time.
         *
         * This means that the initiator and the reflector need to set the same value
         * for the random seed.
         */
        req.rng_seed = peer_rng_seed_get() + scanner_addr_to_random_share(&info->addr->a);
        req.start_delay_us = 0;
        req.extra_window_time_us = 0;

        dm_request_add(&req);
        adv_update_data();
    }
}

const static bt_le_ext_adv_cb adv_cb = {
    .scanned = adv_scanned_cb,
};

static void adv_update_data()
{
    if (!adv)
    {
        return;
    }
    mfg_data.rng_seed = peer_rng_seed_prepare();
    int err = bt_le_ext_adv_set_data(adv, ad, ARRAY_SIZE(ad), sd, ARRAY_SIZE(sd));
    if (err)
    {
        printk("Failed setting adv data (err %d)\n", err);
    }
}

static int adv_start()
{
    int err;
    bt_le_ext_adv_start_param ext_adv_start_param = {0};

    if (adv)
    {
        err = bt_le_ext_adv_stop(adv);
        if (err)
        {
            printk("Failed to stop extended advertising  (err %d)\n", err);
            return err;
        }
        err = bt_le_ext_adv_delete(adv);
        if (err)
        {
            printk("Failed to delete advertising set  (err %d)\n", err);
            return err;
        }
    }

    err = bt_le_ext_adv_create(adv_param, &adv_cb, &adv);
    if (err)
    {
        printk("Failed to create advertising set (err %d)\n", err);
        return err;
    }

    err = bt_le_ext_adv_set_data(adv, ad, ARRAY_SIZE(ad), sd, ARRAY_SIZE(sd));
    if (err)
    {
        printk("Failed setting adv data (err %d)\n", err);
        return err;
    }

    err = bt_le_ext_adv_start(adv, &ext_adv_start_param);
    if (err)
    {
        printk("Failed to start extended advertising  (err %d)\n", err);
        return err;
    }

    return err;
}

static int scan_start()
{
    bt_scan_init(&scan_init);
    bt_scan_cb_register(&scan_cb);

    int err = bt_scan_filter_add(BT_SCAN_FILTER_TYPE_MANUFACTURER_DATA, &scan_mfg_data);
    if (err)
    {
        printk("Scanning filters cannot be set (err %d)\n", err);
        return err;
    }

    err = bt_scan_filter_enable(BT_SCAN_MANUFACTURER_DATA_FILTER, false);
    if (err)
    {
        printk("Filters cannot be turned on (err %d)\n", err);
        return err;
    }

    scanner_random_share = get_id_addr_random_share();

    err = bt_scan_start(BT_SCAN_TYPE_SCAN_ACTIVE);
    if (err)
    {
        printk("Scanning failed to start (err %d)\n", err);
        return err;
    }

    return err;
}

static void adv_work_handle(k_work* item)
{
    adv_start();
}

static int bt_sync_init()
{
    /* Synchronisation is based on advertising and scanning modes.
     * It occurs when SCAN_REQ and SCAN_RESP packets are exchanged.
     */
    printk("DM Bluetooth LE Synchronization initialization\n");

    mfg_data.company_code = sys_cpu_to_le16(CONFIG_BT_COMPANY_ID_NORDIC);
    mfg_data.support_dm_code = sys_cpu_to_le32(SUPPORT_DM_CODE);
    mfg_data.rng_seed = sys_cpu_to_le32(peer_rng_seed_prepare());

    int err = adv_start();
    if (err)
    {
        printk("Failed to start advertising (err %d)\n", err);
        return err;
    }

    err = scan_start();
    if (err)
    {
        printk("Failed to start scanning (err %d)\n", err);
    }

    return err;
}

static void data_ready(dm_result* result)
{
    if (result->status)
    {
        peer_update(result);
    }
}

static dm_cb dm_cb = {
    .data_ready = data_ready,
};

void Bluetooth::init()
{
    dm_init_param init_param{};
    printk("Starting Distance Measurement sample\n");

    int err = peer_init();
    if (err)
    {
        printk("Peer init failed (err %d)\n", err);
        return;
    }

    init_param.cb = &dm_cb;

    err = dm_init(&init_param);
    if (err)
    {
        printk("Distance measurement init failed (err %d)\n", err);
        return;
    }

    err = service_ddfs_init();
    if (err)
    {
        printk("DDF Service init failed (err %d)\n", err);
        return;
    }

    err = bt_enable(NULL);
    if (err)
    {
        printk("Bluetooth init failed (err %d)\n", err);
        return;
    }

    err = bt_sync_init();
    if (err)
    {
        printk("Synchronisation init failed (err %d)\n", err);
    }
}

[Hung Bui]

Hi Mark, 
I don't think the dm library has any requirement on the advertising packets. This mean you can choose to do what you want with the advertising. 

What's the issue doing extended advertising with nrf_dm  ? 

If you proprietary data can not fit into the 31 bytes payload of either advertising packet or scan request packet then I don't think there is any other option. 

[Mark G.]

Thanks. I think I am almost there. I added 
BT_LE_ADV_OPT_EXT_ADV | BT_LE_ADV_OPT_SCANNABLE, and moved all my payload data to the scan response.

The payload is accepted and I can properly see all data in e.g. nrfConnect on my phone. 

One issue remains, the following code is used on two devices that then should initiate distance measurement. It is not working, because "adv_scanned_cb" is not being triggered when the other device is scanning for the first device. I dont understand why. The callback is correctly triggered when third party devices surrounding me are scanning.

the code (I removed err handling prints and only left relevant parts):

struct adv_mfg_data
{
    uint16_t company_code;    /* Company Identifier Code. */
    uint32_t support_dm_code; /* To identify the device that supports distance measurement. */
    uint32_t rng_seed;        /* Random seed used for generating hopping patterns. */
} __packed;

static adv_mfg_data mfg_data;
bt_le_adv_param adv_param_conn =
    BT_LE_ADV_PARAM_INIT(BT_LE_ADV_OPT_NOTIFY_SCAN_REQ | BT_LE_ADV_OPT_EXT_ADV | BT_LE_ADV_OPT_SCANNABLE,
                         BT_GAP_ADV_FAST_INT_MIN_2, BT_GAP_ADV_FAST_INT_MAX_2, NULL);

bt_le_adv_param* adv_param = &adv_param_conn;

double posData[3] = {0, 1, 2};
static const bt_data sd[] = {
    BT_DATA_BYTES(BT_DATA_FLAGS, (BT_LE_AD_GENERAL | BT_LE_AD_NO_BREDR)),
    BT_DATA(BT_DATA_NAME_COMPLETE, DEVICE_NAME, DEVICE_NAME_LEN), BT_DATA(BT_DATA_INDOOR_POS, posData, sizeof(posData)),
    BT_DATA(BT_DATA_MANUFACTURER_DATA, reinterpret_cast<unsigned char*>(&mfg_data), sizeof(mfg_data)),
    BT_DATA_BYTES(BT_DATA_UUID128_ALL, BT_UUID_DDFS_VAL)};

static bt_le_scan_param scan_param = {
    .type = BT_LE_SCAN_TYPE_ACTIVE,
    .options = BT_LE_SCAN_OPT_NONE,
    .interval = BT_GAP_SCAN_FAST_INTERVAL,
    .window = BT_GAP_SCAN_FAST_WINDOW,
    .timeout = 0,
};

static bt_scan_init_param scan_init = {.scan_param = &scan_param, .connect_if_match = false, .conn_param = nullptr};

static uint32_t scanner_random_share;

static bt_le_ext_adv* adv;
static void adv_update_data();
static uint32_t scanner_addr_to_random_share(const bt_addr_t* p_scanner_addr);
static uint32_t get_id_addr_random_share();

static bt_scan_manufacturer_data scan_mfg_data = {
    .data = reinterpret_cast<unsigned char*>(&mfg_data),
    .data_len = sizeof(mfg_data.company_code) + sizeof(mfg_data.support_dm_code),
};

static bool data_cb(bt_data* data, void* user_data)
{
    dm_request req{};
    switch (data->type)
    {
    case BT_DATA_MANUFACTURER_DATA:
        if (sizeof(adv_mfg_data) == data->data_len)
        {
            printf("mf - data\n");

            dm_request_add(&req);
        }
        return true;
    default:
        return true;
    }
    return true;
}

static void scan_filter_match(bt_scan_device_info* device_info, bt_scan_filter_match* filter_match, bool connectable)
{
    bt_addr_le_t addr;

    bt_addr_le_copy(&addr, device_info->recv_info->addr);
    char str[BT_ADDR_STR_LEN];
    bt_addr_to_str(&device_info->recv_info->addr->a, str, sizeof(str));

    printf("supported peer added: %s\n", str);

    peer_supported_add(device_info->recv_info->addr);
    bt_data_parse(device_info->adv_data, data_cb, &addr);
}

BT_SCAN_CB_INIT(scan_cb, scan_filter_match, NULL, NULL, NULL);

static void adv_scanned_cb(bt_le_ext_adv* adv, bt_le_ext_adv_scanned_info* info)
{
    dm_request req{};

    char str[BT_ADDR_STR_LEN];
    bt_addr_to_str(&info->addr->a, str, sizeof(str));
    printf("supported peer check: %s\n", str);
    if (peer_supported_test(info->addr))
    {
        printf("supported peer\n");

        bt_addr_le_copy(&req.bt_addr, info->addr);
        req.role = DM_ROLE_REFLECTOR;
        req.ranging_mode = peer_ranging_mode_get();

        req.rng_seed = peer_rng_seed_get() + scanner_addr_to_random_share(&info->addr->a);
        req.start_delay_us = 0;
        req.extra_window_time_us = 0;

        dm_request_add(&req);
        adv_update_data();
    }
}

const static bt_le_ext_adv_cb adv_cb = {
    .scanned = adv_scanned_cb,
};

static void adv_update_data()
{
    if (!adv)
    {
        return;
    }
    mfg_data.rng_seed = peer_rng_seed_prepare();
    int err = bt_le_ext_adv_set_data(adv, nullptr, 0, sd, ARRAY_SIZE(sd));
}

static int adv_start()
{
    int err;
    bt_le_ext_adv_start_param ext_adv_start_param = {0};

    if (adv)
    {
        err = bt_le_ext_adv_stop(adv);
        err = bt_le_ext_adv_delete(adv);
    }

    err = bt_le_ext_adv_create(adv_param, &adv_cb, &adv);

    err = bt_le_ext_adv_set_data(adv, nullptr, 0, sd, ARRAY_SIZE(sd));

    err = bt_le_ext_adv_start(adv, &ext_adv_start_param);

    return err;
}

static int scan_start()
{
    bt_scan_init(&scan_init);
    bt_scan_cb_register(&scan_cb);

    int err = bt_scan_filter_add(BT_SCAN_FILTER_TYPE_MANUFACTURER_DATA, &scan_mfg_data);

    err = bt_scan_filter_enable(BT_SCAN_MANUFACTURER_DATA_FILTER, false);

    scanner_random_share = get_id_addr_random_share();

    err = bt_scan_start(BT_SCAN_TYPE_SCAN_ACTIVE);
    return err;
}

static int bt_sync_init()
{
    mfg_data.company_code = sys_cpu_to_le16(CONFIG_BT_COMPANY_ID_NORDIC);
    mfg_data.support_dm_code = sys_cpu_to_le32(SUPPORT_DM_CODE);
    mfg_data.rng_seed = sys_cpu_to_le32(peer_rng_seed_prepare());

    int err = adv_start();

    err = scan_start();
    return err;
}

static void data_ready(dm_result* result)
{
    if (result->status)
    {
        peer_update(result);
    }
}

static dm_cb dm_cb = {
    .data_ready = data_ready,
};

void Bluetooth::init()
{
    dm_init_param init_param{};
    peer_init();
    init_param.cb = &dm_cb;
    dm_init(&init_param);
    service_ddfs_init();
    bt_enable(nullptr);
    bt_sync_init();
}

Output on one device looks like this, in this case the other device is "25:59:FC:B5:EE:C8", since it got added it means that it has been scanned and theexpected manufacturing data has been found. Meaning there must also have been a scan response being sent, since only those contain the manufacturing data? So I do not understand why "adv_scanned_cb" is not being called for this device.

mf - data
supported peer check: F0:4A:77:DB:52:F3
supported peer check: F0:4A:77:DB:52:F3
supported peer added: 25:59:FC:B5:EE:C8
mf - data
supported peer added: 25:59:FC:B5:EE:C8
mf - data
supported peer check: F0:4A:77:DB:52:F3
supported peer added: 25:59:FC:B5:EE:C8
mf - data
supported peer check: F0:4A:77:DB:52:F3
supported peer added: 25:59:FC:B5:EE:C8
mf - data
supported peer check: F0:4A:77:DB:52:F3
supported peer added: 25:59:FC:B5:EE:C8
mf - data

[Hung Bui]

Hi Mark, 
Please correct me if I'm wrong. 
You have 2 boards running nrf_dm. 

On one board you can see the adv_scanned_cb called and the peer is added all work as expected. But on the other board, you don't see it  ?

Even though both boards are identical and do exactly the same extended advertising and scanning at the same time ? 

[Mark G.]

Hi Hung Bui,

"You have 2 boards running nrf_dm."

-> correct. Both with the same code shown above.

"On one board you can see the adv_scanned_cb called and the peer is added all work as expected. But on the other board, you don't see it  ?"

-> wrong. What I meant is that the "adv_scanned_cb" is never called when scanned by the "other nrf_dm" device. But it gets called when I e.g. scan with my phone using nrfConnect. 

[Hung Bui]

Could you explain what you meant by: Output on one device looks like this, in this case the other device is "25:59:FC:B5:EE:C8", since it got added it means that it has been scanned and theexpected manufacturing data has been found. 

Have you tried to debug a bit more, maybe remove the scan filter , etc ?