try to integrate Mult-touch and Keyboard functionalities in a single program, but it only can send Mult-touch information normally, failed to send Key value.

[huangqiang2007]

I try to integrate Mult-touch(touch screen) and Keyboard feature in a single program, when I run this program, the phone side(Android8.0 based) can only receive Mult-touch information, it always can't get the key value at all.

my question is:

- is it possible to integrate Mult-touch and Keyboard feature in a single program based on a single 51822 board?

- if it's feasible， could you please to give some suggestions to implement it?

 

some developing detail is as follows:

Softdevice: s130_nrf51_2.0.1_softdevice.hex

SDK: nRF5_SDK_12.2.0_f012efa

HW: nRF51 DK

report_map_data[ ] = {

// ------ MULT_TOUCH_SUPPORT-----

0x05, 0x0D, // USAGE_PAGE(Digitizers)
0x09, 0x04, // USAGE (Touch Screen)
0xA1, 0x01, // COLLECTION(Application)

// define the maximum amount of fingers that the device supports
0x09, 0x55, // USAGE(Contact Count Maximum)
0x25, 0x02, // LOGICAL_MAXIMUM (2) 
0xB1, 0x02, // FEATURE (Data,Var,Abs)

0x85, 0x01, // Report Id 1

// define the actual amount of fingers that are concurrently touching the screen
0x09, 0x54, // USAGE (Contact count)
0x95, 0x01, // REPORT_COUNT(1)
0x75, 0x08, // REPORT_SIZE (8)
0x81, 0x02, // INPUT (Data,Var,Abs)

// declare a finger collection
0x09, 0x22, // USAGE (Finger)
0xA1, 0x02, // COLLECTION (Logical)

// declare an identifier for the finger
0x09, 0x51, // USAGE (Contact Identifier) //touch ID
0x75, 0x08, // REPORT_SIZE (8)
0x95, 0x01, // REPORT_COUNT (1)
0x81, 0x02, // INPUT (Data,Var,Abs)

// declare Tip Switch and In Range
0x09, 0x42, // USAGE (Tip Switch)
0x09, 0x32, // USAGE (In Range)
0x15, 0x00, // LOGICAL_MINIMUM (0)
0x25, 0x01, // LOGICAL_MAXIMUM (1)
0x75, 0x01, // REPORT_SIZE (1)
0x95, 0x02, // REPORT_COUNT(2)
0x81, 0x02, // INPUT (Data,Var,Abs)

// declare the remaining 6 bits of the first data byte as constant -> the driver will ignore them
0x95, 0x06, // REPORT_COUNT (6)
0x81, 0x03, // INPUT (Cnst,Ary,Abs)

// define absolute X and Y coordinates of 16 bit each (percent values multiplied with 100)
0x05, 0x01, // USAGE_PAGE (Generic Desktop)
0x09, 0x30, // Usage (X)
0x09, 0x31, // Usage (Y)
0x16, 0x00, 0x00, // Logical Minimum (0)
0x26, 0x10, 0x27, // Logical Maximum (10000)
0x36, 0x00, 0x00, // Physical Minimum (0)
0x46, 0x10, 0x27, // Physical Maximum (10000)
0x66, 0x00, 0x00, // UNIT (None)
0x75, 0x10, // Report Size (16),
0x95, 0x02, // Report Count (2),
0x81, 0x02, // Input (Data,Var,Abs)
0xC0, // END_COLLECTION

// declare a finger collection
0xA1, 0x02, // COLLECTION (Logical)
0x05, 0x0D, // USAGE_PAGE(Digitizers)

// declare an identifier for the finger
0x09, 0x51, // USAGE (Contact Identifier) // touch ID
0x75, 0x08, // REPORT_SIZE (8)
0x95, 0x01, // REPORT_COUNT (1)
0x81, 0x02, // INPUT (Data,Var,Abs)

// declare Tip Switch and In Range
0x09, 0x42, // USAGE (Tip Switch)
0x09, 0x32, // USAGE (In Range)
0x15, 0x00, // LOGICAL_MINIMUM (0)
0x25, 0x01, // LOGICAL_MAXIMUM (1)
0x75, 0x01, // REPORT_SIZE (1)
0x95, 0x02, // REPORT_COUNT(2)
0x81, 0x02, // INPUT (Data,Var,Abs)

// declare the remaining 6 bits of the first data byte as constant -> the driver will ignore them
0x95, 0x06, // REPORT_COUNT (6)
0x81, 0x03, // INPUT (Cnst,Ary,Abs)

// define absolute X and Y coordinates of 16 bit each (percent values multiplied with 100)
0x05, 0x01, // USAGE_PAGE (Generic Desktop)
0x09, 0x30, // Usage (X)
0x09, 0x31, // Usage (Y)
0x16, 0x00, 0x00, // Logical Minimum (0)
0x26, 0x10, 0x27, // Logical Maximum (10000)
0x36, 0x00, 0x00, // Physical Minimum (0)
0x46, 0x10, 0x27, // Physical Maximum (10000)
0x66, 0x00, 0x00, // UNIT (None)
0x75, 0x10, // Report Size (16),
0x95, 0x02, // Report Count (2),
0x81, 0x02, // Input (Data,Var,Abs)
0xC0, // END_COLLECTION
0xC0, // END_COLLECTION

// With this declaration a data packet must be sent as:
// byte 1 -> "contact count" (always == 1)
// byte 2 -> "contact identifier" (any value)
// byte 3 -> "Tip Switch" state (bit 0 = Tip Switch up/down, bit 1 = In Range)
// byte 4,5 -> absolute X coordinate (0...10000)
// byte 6,7 -> absolute Y coordinate (0...10000)

// byte 8 -> "contact identifier" (any value)
// byte 9 -> "Tip Switch" state (bit 0 = Tip Switch up/down, bit 1 = In Range)
// byte 10, 11 -> absolute X coordinate (0...10000)
// byte 12,13 -> absolute Y coordinate (0...10000)

// ------ KEYBOARD_SUPPORT-----
0x05, 0x01, // Usage Page (Generic Desktop)
0x09, 0x06, // Usage (Keyboard)
0xA1, 0x01, // Collection (Application)

0x85, 0x02, // Report ID 2

0x05, 0x07, // Usage Page (Key Codes)
0x19, 0xe0, // Usage Minimum (224)
0x29, 0xe7, // Usage Maximum (231)
0x15, 0x00, // Logical Minimum (0)
0x25, 0x01, // Logical Maximum (1)
0x75, 0x01, // Report Size (1)
0x95, 0x08, // Report Count (8)
0x81, 0x02, // Input (Data, Variable, Absolute)

0x95, 0x01, // Report Count (1)
0x75, 0x08, // Report Size (8)
0x81, 0x01, // Input (Constant) reserved byte(1)

0x95, 0x05, // Report Count (5)
0x75, 0x01, // Report Size (1)
0x05, 0x08, // Usage Page (Page# for LEDs)
0x19, 0x01, // Usage Minimum (1)
0x29, 0x05, // Usage Maximum (5)
0x91, 0x02, // Output (Data, Variable, Absolute), Led report
0x95, 0x01, // Report Count (1)
0x75, 0x03, // Report Size (3)
0x91, 0x01, // Output (Data, Variable, Absolute), Led report padding

0x95, 0x06, // Report Count (6)
0x75, 0x08, // Report Size (8)
0x15, 0x00, // Logical Minimum (0)
0x25, 0x65, // Logical Maximum (101)
0x05, 0x07, // Usage Page (Key codes)
0x19, 0x00, // Usage Minimum (0)
0x29, 0x65, // Usage Maximum (101)
0x81, 0x00, // Input (Data, Array) Key array(6 bytes)

0x09, 0x05, // Usage (Vendor Defined)
0x15, 0x00, // Logical Minimum (0)
0x26, 0xFF, 0x00, // Logical Maximum (255)
0x75, 0x08, // Report Count (2)
0x95, 0x02, // Report Size (8 bit)
0xB1, 0x02, // Feature (Data, Variable, Absolute)

0xC0 // End Collection (Application)
};

I have verified report map data of Mult-touch and Keyboard separately, they are functional.

1. send Mult-touch information with the below code

static void mouse_moveto_2xy(uint8_t value,uint16_t x_delta,uint16_t y_delta,
uint16_t x2_delta,uint16_t y2_delta)
{
     // With this declaration a data packet must be sent as:
     // byte 1 -> "contact count" (always == 1)
     // byte 2 -> "contact identifier" (any value)
     // byte 3 -> "Tip Switch" state (bit 0 = Tip Switch up/down, bit 1 = In Range)
     // byte 4,5 -> absolute X coordinate (0...10000)
     // byte 6,7 -> absolute Y coordinate (0...10000)
     uint8_t buffer[INPUT_REP_TOUCH_LEN];
     uint32_t err_code;
     APP_ERROR_CHECK_BOOL(INPUT_REP_TOUCH_LEN == 13);

     x_delta = MIN(x_delta, 0x2710);
     y_delta = MIN(y_delta, 0x2710);

     buffer[0] = CONTACT_COUNT;
     buffer[1] = 1;
     buffer[2] = value;
     buffer[3] = x_delta & 0x00ff;
     buffer[4] = (x_delta>>8)&0x00ff;
     buffer[5] = y_delta & 0x00ff;
     buffer[6] = (y_delta>>8)&0x00ff;

    buffer[7] = 2;
    buffer[8] = value;
    buffer[9] = x2_delta & 0x00ff;
    buffer[10] = (x2_delta>>8)&0x00ff;
    buffer[11] = y2_delta & 0x00ff;
    buffer[12] = (y2_delta>>8)&0x00ff;

     err_code = ble_hids_inp_rep_send(&m_hids,
          INPUT_REP_TOUCH_INDEX,
          INPUT_REP_TOUCH_LEN,
          buffer);
     if ((err_code != NRF_SUCCESS) &&
          (err_code != NRF_ERROR_INVALID_STATE) &&
          (err_code != BLE_ERROR_NO_TX_PACKETS) &&
          (err_code != BLE_ERROR_GATTS_SYS_ATTR_MISSING))
     {
          APP_ERROR_HANDLER(err_code);
     }
}

static void touch_emulation(bsp_event_t event)
{
     static uint16_t touch_num = 8000;
     static uint16_t touch_num2 = 2000;

     switch (event)
     {
     case BSP_EVENT_KEY_1:
          if (m_conn_handle != BLE_CONN_HANDLE_INVALID) {
               mouse_moveto_2xy(1,touch_num,touch_num, touch_num2, touch_num2);
               touch_num -= 100;
               if(touch_num < 1000)
                    touch_num = 8000;

               touch_num2 += 100;
               if(touch_num2 > 8000)
                    touch_num2 = 1000;
     }
     break;

     case BSP_EVENT_KEY_3:
          if (m_conn_handle != BLE_CONN_HANDLE_INVALID) {
               mouse_moveto_2xy(1,touch_num2,touch_num2, touch_num,touch_num);
               touch_num -= 100;
          if(touch_num < 1000)
               touch_num = 8000;

          touch_num2 += 100;
          if(touch_num2 > 8000)
               touch_num2 = 1000;
     }
     break;

     default:
          // mouse_moveto_2xy(0,5000,5000);
          // touch_num=5000;
     break;
     }
}

2. send Key value with the below code

static void keys_send(uint8_t key_pattern_len, uint8_t * p_key_pattern)
{
      uint32_t err_code;
      uint16_t actual_len;

      err_code = send_key_scan_press_release(&m_hids,
            p_key_pattern,
            key_pattern_len,
            0,
            &actual_len);
      // An additional notification is needed for release of all keys, therefore check
      // is for actual_len <= key_pattern_len and not actual_len < key_pattern_len.
      if ((err_code == BLE_ERROR_NO_TX_PACKETS) && (actual_len <= key_pattern_len))
      {
            // Buffer enqueue routine return value is not intentionally checked.
            // Rationale: Its better to have a a few keys missing than have a system
            // reset. Recommendation is to work out most optimal value for
            // MAX_BUFFER_ENTRIES to minimize chances of buffer queue full condition
            UNUSED_VARIABLE(buffer_enqueue(&m_hids, p_key_pattern, key_pattern_len, actual_len));
      }

      if ((err_code != NRF_SUCCESS) &&
            (err_code != NRF_ERROR_INVALID_STATE) &&
            (err_code != BLE_ERROR_NO_TX_PACKETS) &&
            (err_code != BLE_ERROR_GATTS_SYS_ATTR_MISSING)
      )
      {
            APP_ERROR_HANDLER(err_code);
      }
}

[Håkon Alseth]

Hi,

The function "send_key_scan_press_release" uses index INPUT_REPORT_KEYS_INDEX, which needs to be concise with your configuration (in your specific case; should be set to 1).

An easier way to send keyboard data is to send it directly without using the added buffering logic in the example, for instance like this:
// Send "hi"
uint8_t temp_buf[INPUT_REPORT_KEYS_MAX_LEN] = {0x00, 0x00, 0xb, 0xc, 0x2c, 0x0, 0x0, 0x0};
ble_hids_inp_rep_send(&m_hids, INPUT_REPORT_KEYS_INDEX, INPUT_REPORT_KEYS_MAX_LEN, (uint8_t*)temp_buf, m_conn_handle);

Could you check your index values and report them here if it still does not work?

Kind regards,
Håkon

[huangqiang2007]

Hi Hakon Alseth,

I confirmed that the INPUT_REPORT_KEYS_INDEX is 1. Actually I have configured input and output report in hids_init( ) as follows:

{

...

ble_hids_inp_rep_init_t input_report_array[2];
ble_hids_outp_rep_init_t output_report_array[1];

p_input_report = &input_report_array[INPUT_REP_TOUCH_INDEX];
p_input_report->max_len = INPUT_REP_TOUCH_LEN;
p_input_report->rep_ref.report_id = INPUT_REP_TOUCH_ID;
p_input_report->rep_ref.report_type = BLE_HIDS_REP_TYPE_INPUT;

BLE_GAP_CONN_SEC_MODE_SET_ENC_NO_MITM(&p_input_report->security_mode.cccd_write_perm);
BLE_GAP_CONN_SEC_MODE_SET_ENC_NO_MITM(&p_input_report->security_mode.read_perm);
BLE_GAP_CONN_SEC_MODE_SET_ENC_NO_MITM(&p_input_report->security_mode.write_perm);

p_input_report = &input_report_array[INPUT_REPORT_KEYS_INDEX ];
p_input_report->max_len = INPUT_REPORT_KEYS_MAX_LEN;
p_input_report->rep_ref.report_id = INPUT_REPORT_KEYS_ID;
p_input_report->rep_ref.report_type = BLE_HIDS_REP_TYPE_INPUT;

BLE_GAP_CONN_SEC_MODE_SET_ENC_NO_MITM(&p_input_report->security_mode.cccd_write_perm);
BLE_GAP_CONN_SEC_MODE_SET_ENC_NO_MITM(&p_input_report->security_mode.read_perm);
BLE_GAP_CONN_SEC_MODE_SET_ENC_NO_MITM(&p_input_report->security_mode.write_perm);

p_output_report = &output_report_array[OUTPUT_REPORT_INDEX];
p_output_report->max_len = OUTPUT_REPORT_MAX_LEN;
p_output_report->rep_ref.report_id = OUTPUT_REP_REF_ID;
p_output_report->rep_ref.report_type = BLE_HIDS_REP_TYPE_OUTPUT;

BLE_GAP_CONN_SEC_MODE_SET_ENC_NO_MITM(&p_output_report->security_mode.read_perm);
BLE_GAP_CONN_SEC_MODE_SET_ENC_NO_MITM(&p_output_report->security_mode.write_perm);

hid_info_flags = HID_INFO_FLAG_REMOTE_WAKE_MSK | HID_INFO_FLAG_NORMALLY_CONNECTABLE_MSK;

memset(&hids_init_obj, 0, sizeof(hids_init_obj));

hids_init_obj.evt_handler = on_hids_evt;
hids_init_obj.error_handler = service_error_handler;
hids_init_obj.is_kb = false;
hids_init_obj.is_mouse = false;
hids_init_obj.inp_rep_count = 2;
hids_init_obj.p_inp_rep_array = input_report_array;
hids_init_obj.outp_rep_count = 1;
hids_init_obj.p_outp_rep_array = output_report_array;

hids_init_obj.feature_rep_count = 0;
hids_init_obj.p_feature_rep_array = NULL;

hids_init_obj.rep_map.data_len = sizeof(report_map_data);
hids_init_obj.rep_map.p_data = report_map_data;

hids_init_obj.hid_information.bcd_hid = BASE_USB_HID_SPEC_VERSION;
hids_init_obj.hid_information.b_country_code = 0;
hids_init_obj.hid_information.flags = hid_info_flags;
hids_init_obj.included_services_count = 0;
hids_init_obj.p_included_services_array = NULL;

...

}

some macro defines as follows:

#define INPUT_REP_TOUCH_INDEX  0
#define INPUT_REP_TOUCH_LEN     13
#define INPUT_REP_TOUCH_ID         1 

#define INPUT_REPORT_KEYS_INDEX          1

#define OUTPUT_REPORT_MAX_LEN            8

#define INPUT_REPORT_KEYS_ID                 2

#define OUTPUT_REPORT_INDEX                  0

#define OUTPUT_REPORT_MAX_LEN            1

#define OUTPUT_REP_REF_ID                        0

In addition, I have set the appearance to BLE_APPEARANCE_GENERIC_HID. If I keep the appearance as BLE_APPEARANCE_HID_KEYBOARD, it can't send any information to phone side.

Best Regards,

QiangHuang

[Håkon Alseth]

Hi,

 

if "hids_init_obj.is_kbd" is not set true, the ble_hids library does not populate the boot characteristics (or the protocol mode characteristic). Could you try setting this to true and see if this has any effect?

 

Best regards,

Håkon

[huangqiang2007]

Hi Hakon,

Actually I have tried to set hids_init_obj.is_kbd to true and kept appearance as  BLE_APPEARANCE_HID_KEYBOARD, but it still failed to send key value and can send touch value normally. 

Best Regatds,

QiangHuang

[huangqiang2007]

Hi Hakon,

Actually I have tried to set hids_init_obj.is_kbd to true and kept appearance as  BLE_APPEARANCE_HID_KEYBOARD, but it still failed to send key value and can send touch value normally. 

Best Regatds,

QiangHuang

[huangqiang2007]

Hi Hakon,

Actually I have tried to set hids_init_obj.is_kbd to true and kept appearance as  BLE_APPEARANCE_HID_KEYBOARD, but it still failed to send key value and can send touch value normally. 

Best Regatds,

QiangHuang

[huangqiang2007]

Hi Hakon,

please ignore the identical repeated reply due to my network issue.

Thanks,

QiangHuang