HID Finger Swipe Functionality (iPhone)

[matt_wilson]

I’m very familiar with the HID keyboard and mouse profile and have built products using it but need direction on how to replicate the functionality of a up/down/left/right finger swipe on a iPhone. Any guidance will be valuable.  Thanks 

[matt_wilson]

What I'm trying to accomplish is exactly what this remote is doing but not sure what type of Bluetooth HID profile it's working with. What it's doing is performing up/down/left/right scroll as if it was a finger swipe.  I think it is acting like a digitizer but it shows up as a keyboard profile and not a mouse.  There is another thread that talks about HID digitizer and has an example project to go with it but its base is on top of mouse. So I'm little confused on what HID profile to focus on.  I do have the packet logs captured from this remote.  Is there a way to capture the report profile from a device?

www.aliexpress.com/.../4000061114207.html

[ovrebekk]

Hi Matthew

If you have access to the device in question you should be able to connect to it from the nRF Connect mobile app, and read out the HID report descriptor from there. 

Then you can try to replicate that HID descriptor in your own peripheral implementation. 

Best regards
Torbjørn

[matt_wilson]

Hi Overbekk,

Thanks for the reply.  I'm in the process of rapidly growing my knowledge of Bluetooth LE.  I have done what you suggested and not sure where I should pull the feature report from in the nRF Connect App on my iPhone.  I have attached two screen shots to show what the app is revealing for the device.  My goal is to get and recreate the feature report like the example below.  At what point is the feature report sent to the iPhone during the connection and paring?  I have the packet logs that I pulled using the Mac OS X PacketLogger for Bluetooth.  The logs have the connection and paring information and also the info for each button press.  I have also uploaded them in case they help in any way.  As you know they can be viewed through Wireshark but wanted to add this note for others.  I'm going to try and see what the logs tell me but if you have any further advice or information please share.  Thanks...

0x05, 0x0D,        // Usage Page (Digitizer)
   0x09, 0x04,        // Usage (Touch Screen)
   0xA1, 0x01,        // Collection (Application)
      ....,.....,        //   etc....
      0x95, 0x01,        //   Report Count (1)
      0xB1, 0x02,        //   Feature (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position,Non-volatile)
   0xC0               // End Collection




0310.RemotePacketLogs.pklg

[ovrebekk]

Hi Matthew

Using the ble_app_hids_mouse example in the SDK you should be able to set up something similar to your remote device, as long as you copy the HID Report descriptor. One possible issue is that iOS in general doesn't support HID mice, only keyboards. 

The log you sent was very useful, and shows the report descriptor being transferred (packets 100, 103, 106, ... in the log). The descriptor appears to identify the device as a mouse, so I am surprised this would work, but I guess you have verified that the device can be used to control the phone?

The screenshots you sent don't appear to show the HID service at all, only the standard Generic Access, Generic Attribute and Battery services. Since the HID service is handled by the OS directly it is probably made inaccessible by the system. 

An alternative is to use a Nordic devkit or dongle connected to your PC/Mac, and use the nRF Connect for Desktop application to connect to it. Then you should be able to see all the services. 

Best regards
Torbjørn

[matt_wilson]

Hi Ovrebekk,

You are correct about the iPhone not supporting HID mouse unless enable now in the Accessibility options.  But when I connect the remote to the computer it shows up as a keyboard and not a mouse.  Now I think you recognize my struggle and need the report descriptor.

If I understand you correct.  I will use nRF Connect on my Mac that is connected to my nRF52832 devkit and have the remote connect and pair to the devkit so I can extract the info?  I'm guessing there is a sample project that allows for me to host as a master on the devkit?

I will review the packet logs and try to figure out what you are suggesting.  Thanks...

[ovrebekk]

Hi Matthew

Did you check packets 100, 103, 106 etc like I mentioned?

Packet 100 starts with 0x05, 0x0D, 0x09, 0x02, 0xA1, 0x01 .., which is clearly the start of the HID descriptor. It is quite similar to the descriptor you shared below, except the usage is 0x02 (mouse) instead of 0x04 (touch screen). 

Looking at the log again the entire descriptor seems to be sent across the following packets: 100, 103, 106, 109, 112 and 115.

Best regards
Torbjørn

[matt_wilson]

Hi Ovrebekk,

I received a new remote in the mail today that was better quality and is performing the same function I'm trying to duplicate.  This remote has a nRF51822 instead of some cheap China IC.  I took everything that you have educated me on and derived to the following information.  I was able to extract the report and also attempted to fill it out with comments.  It looks like it is presenting itself as a Digitizer and performing a single finger touch swipe effect.  I was able to capture the data for each button press.  I'm assuming the data is representing the finger press and the sliding across the screen.  I'm thinking I need to take the ble_app_hids_keyboard example and add this report then try sending the raw data. I say keyboard because the remote shows up as keyboard on the computer and I plan to use this with my current product that is a keyboard.  Also, I'm having trouble designing the packet structure from this report just for the digitizer.  I haven't quite wrapped my mind around how it works. Do you have any recommendations?  This has been a great practice to learn in depth how all this works.  Your feedback has been great and really appreciate it.

My attempt to start a packet structure.

typedef PACKED_STRUCT
{

uint8_t report_id:
uint8_t tip_switch : 1;
uint8_t range : 1; // Not sure if this is right
uint16_t x;
uint16_t y;
???? width; // Not sure if this is needed
???? height; // Not sure if this is needed
} digitizer_report_t;

Packet 1: 050D 0901 A101 8501 0922 A102 0942 1500 2501 7501 9501
Packet 2: 8102 0932 8102 9506 8103 0501 26E8 0375 1095 0155 0065
Packet 3: 0009 3035 0046 E803 8102 0931 46E8 0381 02C0 050D 0948
Packet 4: 0949 9502 8102 C005 0C09 01A1 0185 0209 E909 EA09 E209
Packet 5: B509 B60A 2402 09CD 0930 0A23 0209 4015 0125 0C75 1095
Packet 6: 0181 00C0 0501 0906 A101 8503 0507 19E0 29E7 1500 2501
Packet 7: 7501 9508 8102 7508 9501 1500 25F4 0507 1900 29F4 8100
Packet 8: C0

www.usb.org/.../hut1_12v2.pdf

0x05, 0x0D, // USAGE_PAGE (Digitizer) 
0x09, 0x01, // USAGE (Digitizer)
0xA1, 0x01, // Collection (Application)
0x85, 0x01, // REPORT_ID 1
0x09, 0x22, // USAGE (Finger)
0xA1, 0x02, // Collection (Logical)
0x09, 0x42, // USAGE (Tip Switch)
0x15, 0x00, // LOGICAL_MINIMUM (0)
0x25, 0x01, // LOGICAL_MAXIMUM (1)
0x75, 0x01, // REPORT_SIZE (1)
0x95, 0x01, // REPORT_COUNT (1)
0x81, 0x02, // INPUT (Data,Var,Abs)
0x09, 0x32, // USAGE (In Range)
0x81, 0x02, // INPUT (Data,Var,Abs)
0x95, 0x06, // REPORT_COUNT (6)
0x81, 0x03, // INPUT (Cnst,Ary,Abs)
0x05, 0x01, // USAGE_PAGE (Generic Desktop)
0x26, 0xE8, 0x03, // LOGICAL_MAXIMUM (0x03E8 = 1000)
0x75, 0x10, // REPORT_SIZE (16)
0x95, 0x01, // REPORT_COUNT (1)
0x55, 0x00, // UNIT_EXPONENT (0)
0x65, 0x00, // UNIT(Inch, EngLinear)
0x09, 0x30, // USAGE (X)
0x35, 0x00, // PHYSICAL_MINIMUM (0)
0x46, 0xE8, 0x03, // PHYSICAL_MAXIMUM (0x03E8 = 1000)
0x81, 0x02, // INPUT (Data,Var,Abs)
0x09, 0x31, // USAGE (Y)
0x46, 0xE8, 0x03, // PHYSICAL_MAXIMUM (0x03E8 = 1000)
0x81, 0x02, // INPUT (Data,Var,Abs)
0xC0, // END_COLLECTION
0x05, 0x0D, // USAGE_PAGE (Digitizer)
0x09, 0x48, // USAGE (Width)
0x09, 0x49, // USAGE (Height)
0x95, 0x02, // REPORT_COUNT (2)
0x81, 0x02, // INPUT (Data,Var,Abs)
0xC0, // END_COLLECTION

0x05, 0x0C, // USAGE_PAGE (Consumer Devices)
0x09, 0x01, // USAGE_(Consumer Control)
0xA1, 0x01, // COLLECTION (Application)
0x85, 0x02, // REPORT_ID 2
0x09, 0xE9, // USAGE (Volume Increment)
0x09, 0xEA, // USAGE (Volume Decrement)
0x09, 0xE2, // USAGE (Mute)
0x09, 0xB5, // USAGE (Scan Next Track)
0x09, 0xB6, // USAGE (Scan Previous Track)
0x0A, 0x24, 0x02, // USAGE (WWW Back)
0x09, 0xCD, // USAGE (Play/Pause)
0x09, 0x30, // USAGE (Power)
0x0A, 0x23, 0x02, // USAGE (WWW Home)
0x09, 0x40, // USAGE (Menu)
0x15, 0x01, // LOGICAL_MINIMUM (1)
0x25, 0x0C, // LOGICAL_MAXIMUM (12)
0x75, 0x10, // REPORT_SIZE (16)
0x95, 0x01, // REPORT_COUNT (1)
0x81, 0x00, // INPUT (Data,Ary,Abs)
0xC0, // END_COLLECTION

0x05, 0x01, // USAGE_PAGE (Generic Desktop)
0x09, 0x06, // USAGE (Keyboard)
0xA1, 0x01, // Collection (Application)
0x85, 0x03, // REPORT_ID 3
0x05, 0x07, // USAGE_PAGE (Keyboard)
0x19, 0xE0, // USAGE_MINIMUM (Keyboard LeftControl)
0x29, 0xE7, // USAGE_MAXIMUM (Keyboard Right GUI)
0x15, 0x00, // LOGICAL_MINIMUM (0)
0x25, 0x01, // LOGICAL_MAXIMUM (1)
0x75, 0x01, // REPORT_SIZE (1)
0x95, 0x08, // REPORT_COUNT (8)
0x81, 0x02, // INPUT (Data,Var,Abs)
0x75, 0x08, // REPORT_SIZE (8)
0x95, 0x01, // REPORT_COUNT (1)
0x15, 0x00, // LOGICAL_MINIMUM (0)
0x25, 0xF4, // LOGICAL_MAXIMUM (244)
0x05, 0x07, // USAGE_PAGE (Keyboard)
0x19, 0x00, // USAGE_MINIMUM (Reserved (no event indicated))
0x29, 0xF4, // USAGE_MAXIMUM (????)
0x81, 0x00, // INPUT (Data,Ary,Abs)
0xC0 // END_COLLECTION

## UP Button Values ## 
Packet 1: 07F4 0120 038E 038E 03 
Packet 2: 07F4 0102 038E 038E 03 
Packet 3: 07F4 01E4 028E 038E 03 
Packet 4: 07F4 01C6 028E 038E 03 
Packet 5: 07F4 01A8 028E 038E 03 
Packet 6: 07F4 018A 028E 038E 03 
Packet 7: 07F4 016C 028E 038E 03 
Packet 8: 07F4 014E 028E 038E 03 
Packet 9: 07F4 0130 028E 038E 03 
Packet 10: 07F4 0112 028E 038E 03 
Packet 11: 07F4 01F4 018E 038E 03 
Packet 12: 07F4 01D6 018E 038E 03 
Packet 13: 07F4 01B8 018E 038E 03 
Packet 14: 07F4 019A 018E 038E 03 
Packet 15: 07F4 017C 018E 038E 03 
Packet 16: 07F4 015E 018E 038E 03 
Packet 17: 07F4 0140 018E 038E 03 
Packet 18: 00F4 0140 018E 038E 03

## Down Button Values ## 
Packet 1: 07F4 012C 018E 038E 03 
Packet 2: 07F4 014A 018E 038E 03 
Packet 3: 07F4 0168 018E 038E 03 
Packet 4: 07F4 0186 018E 038E 03 
Packet 5: 07F4 01A4 018E 038E 03 
Packet 6: 07F4 01C2 018E 038E 03
Packet 7: 07F4 01E0 018E 038E 03 
Packet 8: 07F4 01FE 018E 038E 03 
Packet 9: 07F4 011C 028E 038E 03 
Packet 10: 07F4 013A 028E 038E 03 
Packet 11: 07F4 0158 028E 038E 03 
Packet 12: 07F4 0176 028E 038E 03 
Packet 13: 07F4 0194 028E 038E 03 
Packet 14: 07F4 01B2 028E 038E 03 
Packet 15: 07F4 01D0 028E 038E 03 
Packet 16: 07F4 01EE 028E 038E 03 
Packet 17: 07F4 010C 038E 038E 03 
Packet 18: 00F4 010C 038E 038E 03

## Left Button Values ## 
Packet 1: 0720 032C 018E 038E 03 
Packet 2: 07BC 022C 018E 038E 03 
Packet 3: 0758 022C 018E 038E 03 
Packet 4: 07F4 012C 018E 038E 03 
Packet 5: 0790 012C 018E 038E 03 
Packet 6: 072C 012C 018E 038E 03 
Packet 7: 002C 012C 018E 038E 03 
Packet 8: 002C 012C 018E 038E 03

## Right Button Values ##
Packet 1: 07C8 002C 018E 038E 03 
Packet 2: 072C 012C 018E 038E 03 
Packet 3: 0790 012C 018E 038E 03 
Packet 4: 07F4 012C 018E 038E 03 
Packet 5: 0758 022C 018E 038E 03 
Packet 6: 07BC 022C 018E 038E 03 
Packet 7: 00BC 022C 018E 038E 03 
Packet 8: 002C 012C 018E 038E 03