Trigger iOS Shutter via BLE on iOS 16

RE: Trigger iOS Shutter via BLE on iOS 16

florianbuerger — Tue, 01 Nov 2022 09:55:25 GMT

Hey Sigurd, apologies for my delayed response. Thank you for your help, and adjusting the sample. We have a working version now for iOS 16. It doesn't work anymore on iOS 15, but we decided to accept that and don't invest any more time in it. The number of iOS 15 users for our app is quite low, and only will get lower over time. The important part was that iOS 16 works.

RE: Trigger iOS Shutter via BLE on iOS 16

Sigurd — Wed, 26 Oct 2022 11:11:56 GMT

Hi,

[quote user="sigurdon"]Small update: I will get hold of a iPhone with iOS16 and try to reproduce the issue here.[/quote]

We have 1 sample in the nRF Connect SDK that implements HID Volume control: https://github.com/nrfconnect/sdk-nrf/tree/main/samples/bluetooth/peripheral_fast_pair ,

The HID report map looks like this: https://github.com/nrfconnect/sdk-nrf/blob/v2.1.0/samples/bluetooth/peripheral_fast_pair/src/hids_helper.c#L58

I removed some of the fast-pairing stuff, and was left with this:

devzone.nordicsemi.com/.../peripheral_5F00_hids_5F00_volume.zip

Button 2 and 4 on a nRF52-DK can be used to control the volume.

The sample was built using NCS v2.1.0, and tested on a iPhone with iOS 16.1, and the HID volume control worked fine.

RE: Trigger iOS Shutter via BLE on iOS 16

Sigurd — Tue, 25 Oct 2022 20:16:59 GMT

Small update: I will get hold of a iPhone with iOS16 and try to reproduce the issue here.

RE: Trigger iOS Shutter via BLE on iOS 16

florianbuerger — Mon, 24 Oct 2022 08:47:05 GMT

All the above testing was done with the 16.1 release candidate, which should be the same version as the public release today.

RE: Trigger iOS Shutter via BLE on iOS 16

Sigurd — Mon, 24 Oct 2022 07:54:07 GMT

Hi,

iOS v16.1.0 is being released today. Could you check if this is issue is fixed in v16.1.0?

RE: Trigger iOS Shutter via BLE on iOS 16

florianbuerger — Fri, 21 Oct 2022 15:15:48 GMT

That results in iOS 15 not working at all, iOS 16 receives a volume down command, but then never receives the release command.

RE: Trigger iOS Shutter via BLE on iOS 16

Sigurd — Fri, 21 Oct 2022 13:39:51 GMT

Hi,

[quote userid="120430" url="~/f/nordic-q-a/92798/trigger-ios-shutter-via-ble-on-ios-16/391915"] I've attached the updated HID report map.[/quote]

Could you try to set MEDIA_KEYS_ID to 0 ?

More context for this suggestion can be found in this PR: https://github.com/nrfconnect/sdk-nrf/pull/8772

RE: Trigger iOS Shutter via BLE on iOS 16

florianbuerger — Fri, 21 Oct 2022 13:33:35 GMT

Okay, I can confirm that we adhere to the suggested intervals. Our accessory connects fine, and never gets disconnected by iOS.

We experimented with the report descriptor, and found a version that works on iOS 16. But, the same version does not work on iOS 15. Instead of receiving a single volume down command, iOS 15 treats it as a continuous volume up command. It seems the key release event is never sent.

I was unable to capture a sniffer trace, for some reason the Wireshark plugin didn't work. I tried on macOS first, with no luck. I installed Windows 10 on an Intel MacBook, the plugin worked once, but then it stopped. We do have logs from running our firmware on the Nordic Dev Board. I've attached the updated HID report map.

#include <hid.h>

#include <zephyr/types.h>
#include <stddef.h>
#include <string.h>
#include <errno.h>
#include <sys/byteorder.h>
#include <zephyr.h>

#include <bluetooth/bluetooth.h>
#include <bluetooth/hci.h>
#include <bluetooth/conn.h>
#include <bluetooth/uuid.h>
#include <bluetooth/gatt.h>

#include <logging/log.h>
LOG_MODULE_REGISTER(hid);

// BLE HID Descriptor Report ID
#define MEDIA_KEYS_ID 0x01

#define KEY_VOLUME_UP (0xe9)
#define KEY_VOLUME_DOWN (0xea)
#define KEY_RELEASE (0x00)

#define KEY_DELAY K_MSEC(20)

#define STACKSIZE 1024
#define PRIORITY 7

enum {
	HIDS_REMOTE_WAKE = BIT(0),
	HIDS_NORMALLY_CONNECTABLE = BIT(1),
};

struct hids_info {
	uint16_t version;
	uint8_t code;
	uint8_t flags;
} __packed;

struct hids_report {
	uint8_t id;
	uint8_t type;
} __packed;

static struct hids_info info = {
	.version = 0x0101,
	.code = 33,
	.flags = HIDS_NORMALLY_CONNECTABLE,
};

enum {
	HIDS_INPUT = 0x01,
	HIDS_OUTPUT = 0x02,
	HIDS_FEATURE = 0x03,
};

enum {
	HIDS_BOOT_PROTOCOL = 0x0,
	HIDS_REPORT_PROTOCOL = 0x1,
};

static struct hids_report input = {
	.id = 0x01,
	.type = HIDS_INPUT,
};

static uint8_t protocol = HIDS_REPORT_PROTOCOL;
static uint8_t send_input;
static uint8_t ctrl_point;

static uint8_t report_map[] = {
0x05, 0x0C,                     // Usage Page (Consumer)
        0x09, 0x01,                     // Usage (Consumer Control)
        0xA1, 0x01,                     // Collection (Application)
        0x85, MEDIA_KEYS_ID,            //     Report Id (1)
        0x15, 0x00,                     //     Logical minimum (0)
        0x25, 0x01,                     //     Logical maximum (1)
        0x75, 0x01,                     //     Report Size (1)
        0x95, 0x01,                     //     Report Count (1)

        0x09, KEY_VOLUME_DOWN,          //     Usage (Volume Down)
        0x81, 0x02,                     //     Input (Data,Value,Relative,Bit Field)
        0x09, KEY_VOLUME_UP,            //     Usage (Volume Up)
        0x81, 0x02,                     //     Input (Data,Value,Relative,Bit Field)

        0xC0                            // End Collection	
};


static ssize_t read_info(struct bt_conn *conn, const struct bt_gatt_attr *attr, 
	void *buf, uint16_t len, uint16_t offset)
{
	return bt_gatt_attr_read(conn, attr, buf, len, offset, attr->user_data, sizeof(struct hids_info));
}

static ssize_t read_report_map(struct bt_conn *conn, const struct bt_gatt_attr *attr, 
	void *buf, uint16_t len, uint16_t offset)
{
	return bt_gatt_attr_read(conn, attr, buf, len, offset, report_map, sizeof(report_map));
}

static ssize_t read_report(struct bt_conn *conn, const struct bt_gatt_attr *attr, 
	void *buf, uint16_t len, uint16_t offset)
{
	return bt_gatt_attr_read(conn, attr, buf, len, offset, attr->user_data, sizeof(struct hids_report));
}

static ssize_t read_protocol(struct bt_conn *conn, const struct bt_gatt_attr *attr, 
	void *buf, uint16_t len, uint16_t offset)
{
	return bt_gatt_attr_read(conn, attr, buf, len, offset, attr->user_data, sizeof(protocol));
}

static void input_ccc_changed(const struct bt_gatt_attr *attr, uint16_t value)
{
	send_input = (value == BT_GATT_CCC_NOTIFY) ? 1 : 0;
}

static ssize_t read_input_report(struct bt_conn *conn, 
	const struct bt_gatt_attr *attr, void *buf, uint16_t len, uint16_t offset)
{
	return bt_gatt_attr_read(conn, attr, buf, len, offset, NULL, 0);
}

static ssize_t write_ctrl_point(struct bt_conn *conn, 
	const struct bt_gatt_attr *attr, const void *buf, uint16_t len, 
	uint16_t offset, uint8_t flags)
{
	uint8_t *value = attr->user_data;

	if (offset + len > sizeof(ctrl_point)) 
    {
		return BT_GATT_ERR(BT_ATT_ERR_INVALID_OFFSET);
	}

	memcpy(value + offset, buf, len);

	return len;
}

/* HID Service Declaration */
BT_GATT_SERVICE_DEFINE(hid_service,
	BT_GATT_PRIMARY_SERVICE(BT_UUID_HIDS),
	BT_GATT_CHARACTERISTIC(BT_UUID_HIDS_INFO, 
		BT_GATT_CHRC_READ, BT_GATT_PERM_READ, read_info, NULL, &info),
	BT_GATT_CHARACTERISTIC(BT_UUID_HIDS_REPORT_MAP, 
		BT_GATT_CHRC_READ, BT_GATT_PERM_READ, read_report_map, NULL, NULL),
	BT_GATT_CHARACTERISTIC(BT_UUID_HIDS_REPORT, 
		BT_GATT_CHRC_READ | BT_GATT_CHRC_NOTIFY, BT_GATT_PERM_READ_ENCRYPT, 
		read_input_report, NULL, NULL),
	BT_GATT_CCC(input_ccc_changed, 
		BT_GATT_PERM_READ_ENCRYPT | BT_GATT_PERM_WRITE_ENCRYPT),
	BT_GATT_DESCRIPTOR(BT_UUID_HIDS_REPORT_REF, 
		BT_GATT_PERM_READ, read_report, NULL, &input),
    BT_GATT_CHARACTERISTIC(BT_UUID_HIDS_PROTOCOL_MODE, 
		BT_GATT_CHRC_READ, BT_GATT_PERM_READ, read_protocol, NULL, &protocol),
	BT_GATT_CHARACTERISTIC(BT_UUID_HIDS_CTRL_POINT, 
		BT_GATT_CHRC_WRITE_WITHOUT_RESP, BT_GATT_PERM_WRITE, NULL, 
		write_ctrl_point, &ctrl_point),
);

struct key_report 
{
	int reportId;
    int keycode;
};

K_MSGQ_DEFINE(key_report_queue, sizeof(struct key_report), 8, 4);

static void 
send_key(int reportId, int keycode)
{
	struct key_report report = { reportId, keycode };

	LOG_INF("Sending key report to HID thread {%x, %x}", report.reportId, report.keycode);
	
	int rc = k_msgq_put(&key_report_queue, &report, K_NO_WAIT);

	if (rc != 0)
	{
		LOG_ERR("Failed to send key report");
	}
}

void 
hid_thread_handler(void)
{
	struct key_report report = { 0 };

	for (;;)
	{
		k_sleep(KEY_DELAY);

		k_msgq_get(&key_report_queue, &report, K_FOREVER);

		LOG_INF("[HID Thread] Received key report {%x, %x}", report.reportId, report.keycode);

		uint8_t raw_cc_report[] = { report.reportId, report.keycode };
		LOG_HEXDUMP_INF(raw_cc_report, sizeof(raw_cc_report), "Hex dump");
		bt_gatt_notify(NULL, hid_service.attrs + 5, raw_cc_report, sizeof(raw_cc_report));
	}
}

K_THREAD_DEFINE(hid_thread, STACKSIZE, hid_thread_handler, NULL, NULL, NULL, PRIORITY, 0, 0);

void
hid_send_press_event(void)
{
    LOG_INF("Sending HID press event");

	if (send_input == 0)
	{
		LOG_INF("iOS is not listening for the events");
		return;
	}

    send_key(MEDIA_KEYS_ID, KEY_VOLUME_DOWN);
}

void
hid_send_release_event(void)
{
	LOG_INF("Sending HID release event");

	if (send_input == 0)
	{
		LOG_INF("iOS is not listening for the events");
		return;
	}

    send_key(0, KEY_RELEASE);
}

RE: Trigger iOS Shutter via BLE on iOS 16

Sigurd — Thu, 13 Oct 2022 07:46:36 GMT

Ok, let me know once you know more. Should the connection parameters turn out to be inline with the guidelines, then the next step in debugging this would be to take a sniffer trace: https://www.nordicsemi.com/Products/Development-tools/nrf-sniffer-for-bluetooth-le

RE: Trigger iOS Shutter via BLE on iOS 16

florianbuerger — Thu, 13 Oct 2022 07:38:20 GMT

Hey Sigurd, thanks for getting back to me. I’ll double-check with our engineer, but the device works flawlessly with using our app—which then doesn’t rely on HID, we are using our services/characteristics. But there are no disconnects.

RE: Trigger iOS Shutter via BLE on iOS 16

Sigurd — Wed, 12 Oct 2022 14:00:10 GMT

Hi,

Can you check and make sure that all connection parameters match the requirements from the Apple accessory design guidelines?
https://developer.apple.com/accessories/Accessory-Design-Guidelines.pdf

[quote userid="120430" url="~/f/nordic-q-a/92798/trigger-ios-shutter-via-ble-on-ios-16"]I wonder if anyone has encountered this issue, or understand what is happening?[/quote]

We have seen reports that there have been some changes in iOS v16 on how these guidelines are being enforced. See e.g. this case: RE: Getting continuous disconnects with iOS 16