Queued Write without Response questions

RE: Queued Write without Response questions

Vidar Berg — Thu, 25 Jul 2019 12:15:35 GMT

All packets are acked at the link layer, but write with response will also need to be acknowledged by the application layer running on top.

RE: Queued Write without Response questions

AKYR — Thu, 25 Jul 2019 12:08:27 GMT

I can't have access to the receiving device implementation so only supositions here :) But it is using an Uart protocol tu proceed Write without Response datas.

But ins't acking the transmition of data the point of Write with Response ? Since I'm using Write without Response I thought the packets weren't acknowledged ?

RE: Queued Write without Response questions

Vidar Berg — Thu, 25 Jul 2019 11:52:07 GMT

Glad to hear it was of some help:)

[quote userid="80049" url="~/f/nordic-q-a/50193/queued-write-without-response-questions/200465"]In fact the problem wasn't in an overlap on the process FIFO function but that it was too fast causing the other side device to lose packets ...[/quote]

I'm not that familiar with your implementation, but just want to point out that packets that are successfully added to the output buffer (hvx function returns NRF_SUCCESS), will be re-transmitted on-air until they're acked by the peer device. And the link will be considered lost if a packet is not acked withing the supervision timeout. My point is that you may want to find what causes the packet loss on the receiving device.

[quote userid="80049" url="~/f/nordic-q-a/50193/queued-write-without-response-questions/200465"]The clean implementation is to call the send function when a BLE_GATTC_EVT_WRITE_CMD_TX_COMPLETE arrive to tell that a buffer is available. What I'm afraid of is that two events like this may overlap. [/quote]

They should not overlap as long as you are not using a preemptive scheduler for SD event processing. The ble callback is called in an interrupt context in most of our examples.

RE: Queued Write without Response questions

AKYR — Thu, 25 Jul 2019 10:16:31 GMT

Thanks Vidar your remarks made me realise something :)

In fact the problem wasn't in an overlap on the process FIFO function but that it was too fast causing the other side device to lose packets ...

I'm indeed using another BLE device as peer.

I tried sending continuously send from your main loop and ignore the return value. This way it took me 43 msec to send 1KB of data.

When I wait for at least 4 buffers to be released, it takes 86 msecs to send 1KB of data.

The clean implementation is to call the send function when a BLE_GATTC_EVT_WRITE_CMD_TX_COMPLETE arrive to tell that a buffer is available. What I'm afraid of is that two events like this may overlap.

Are successive BLE_GATTC_EVT_WRITE_CMD_TX_COMPLETE and other BLE events stacking in some sort of FIFO to be dispatch one after the other (Which means i can synchronously execute my function to avoid overlaps). Or does the SoftDevice implementation could send an event while the execution of another one isn't terminated yet ?

What I mean is, is this case possible ?

RE: Queued Write without Response questions

Vidar Berg — Thu, 25 Jul 2019 08:45:07 GMT

There should be plenty of time to prepare the buffers for the next connection event when you get the tx complete signal. So I don't think you will be able to further improve the throughput by changing the buffer handling. An easy way to confirm this would be to continuously call the send from your main loop and ignore the return value to see how much data you can push through.

Are you testing with Nordic nRF devices on both sides? The connection event length can have a big impact on how many packets you can send per event.

Below is a sniffer trace from a test I did with coded PHY. Notice the number of packets between the connection intervals:

RE: Queued Write without Response questions

AKYR — Thu, 25 Jul 2019 08:36:42 GMT

Yes this was my first implementation that worked.

But I need it to go faster to match the speed of a previous device.

Working on the 4 buffers already upgraded the speed by ~250%.

But waiting on the 4 buffers to be released to write again 4 times can be optimize and it is more or less what I'm lacking to match the speed I need :)

When you write do the 4 consecutive writes at the beggining you have a buffer release each 2-3 seconds whereas when working on one buffer only there is a release every 6-7 seconds.

RE: Queued Write without Response questions

Vidar Berg — Thu, 25 Jul 2019 08:23:21 GMT

Hello Aloïs,

Did you consider to just process the buffer once when you insert a new data, and then each time you get the BLE_GATTC_EVT_WRITE_CMD_TX_COMPLETE event? This should ensure sufficient utilization of the Softdevice's transmit buffer.

[quote user="AKYR"]I would like to optimize this by doing a sd_ble_gattc_write whenever a buffer is available instead of waiting for the pool to be available. [/quote]

BLE_GATTC_EVT_WRITE_CMD_TX_COMPLETE would signalize that you have buffer space available.

Best regards,

Vidar

RE: Queued Write without Response questions

AKYR — Wed, 24 Jul 2019 14:48:01 GMT

Hello Vidar and thanks for your answer,

I have made this work by waiting for the write_cmd_tx_queue_size number of buffers to be released before looping on sd_ble_gattc_write again.

I would like to optimize this by doing a sd_ble_gattc_write whenever a buffer is available instead of waiting for the pool to be available.

To do so I need to protect the calls to the function processing the buffers. I tried using sd_mutex functions but i got undefined behavior. Looks like the thread managing the BLE event is forever locked...

If you have some suggestion, here is the sample of the code that works:

#define MINIMUM_AVAILABLE_WRITE_CMD_TX_BUFFER 4
uint8_t m_available_write_cmd_tx_buffer;

static void on_write_cmd_tx_rsp(const ble_gattc_evt_t *const p_ble_gattc_evt)
{
  ret_code_t err_code;
  const ble_gattc_evt_write_cmd_tx_complete_t *p_write_cmd_tx_rsp = &(p_ble_gattc_evt->params.write_cmd_tx_complete);
  m_available_write_cmd_tx_buffer += p_write_cmd_tx_rsp->count;
  if (m_available_write_cmd_tx_buffer >= MINIMUM_AVAILABLE_WRITE_CMD_TX_BUFFER)
  {
    err_code = process_tx_fifo_send_bytes();
    APP_ERROR_CHECK(err_code);
  }
}

static uint32_t process_tx_fifo_send_bytes()
{
  uint32_t nbReadableBytes = 0;
  uint32_t length = 0;
  static uint8_t data_array[BLE_MLDP_MAX_DATA_LEN];
  ret_code_t err_code;
  bool exitMainLoop = false;

  // Get nbReadableBytes availables in the FIFO
  err_code = app_fifo_read(&m_mldp_tx_fifo, NULL, &nbReadableBytes);
  if ((err_code != NRF_ERROR_NOT_FOUND) && (err_code != NRF_SUCCESS))
  {
    APP_ERROR_CHECK(err_code);
  }

  if ((nbReadableBytes == 0) || err_code == NRF_ERROR_NOT_FOUND)
  {
    // No data to send in TX FIFO
    return NRF_SUCCESS;
  }

  while (m_available_write_cmd_tx_buffer && nbReadableBytes)
  {
    length = fmin(nbReadableBytes, BLE_MLDP_MAX_DATA_LEN);

    err_code = app_fifo_read(&m_mldp_tx_fifo, data_array, &length);
    if ((err_code != NRF_ERROR_NOT_FOUND) && (err_code != NRF_SUCCESS))
    {
      APP_ERROR_CHECK(err_code);
    }

    do
    {
      err_code = ble_mldps_c_string_send(&m_ble_mldps_c, data_array, length);
      if ((err_code != NRF_SUCCESS) && (err_code != NRF_ERROR_BUSY))
      {
        NRF_LOG_ERROR("Failed sending MLDP message. Error 0x%x. ", err_code);
        APP_ERROR_CHECK(err_code);
      }
    } while (err_code == NRF_ERROR_BUSY);

    if (err_code == NRF_SUCCESS)
    {
      m_available_write_cmd_tx_buffer--;
    }
    else if (err_code == NRF_ERROR_RESOURCES)
    {
      // Should reeval m_available_write_cmd_tx_buffer probably but will see this later
      break;
    }
    else
    {
      APP_ERROR_CHECK(err_code);
    }
    nbReadableBytes -= length;
  }
  return NRF_SUCCESS;
}

static void ble_evt_handler(ble_evt_t const *p_ble_evt, void *p_context)
{
  ret_code_t err_code;
  ble_gap_evt_t const *p_gap_evt = &p_ble_evt->evt.gap_evt;

  pm_handler_secure_on_connection(p_ble_evt);

  switch (p_ble_evt->header.evt_id)
  {
  case BLE_GATTC_EVT_WRITE_CMD_TX_COMPLETE:
  {
    on_write_cmd_tx_rsp(&(p_ble_evt->evt.gattc_evt));
    APP_ERROR_CHECK(err_code);
  }
  break;

  default:
    break;
  }
}

int main(void)
{
  //Inits the FIFO with 1K bytes data
  init_fifo(&m_mldp_tx_fifo);
  //Inits write_cmd_tx_queue_size with the value MINIMUM_AVAILABLE_WRITE_CMD_TX_BUFFER
  ble_stack_init();
  //First call, the next will be from callback from BLE_GATTC_EVT_WRITE_CMD_TX_COMPLETE event
  process_tx_fifo_send_bytes();
}

And here is what I modified to try to make work:

static void on_write_cmd_tx_rsp(const ble_gattc_evt_t *const p_ble_gattc_evt)
{
    ret_code_t err_code;
    const ble_gattc_evt_write_cmd_tx_complete_t *p_write_cmd_tx_rsp = &(p_ble_gattc_evt->params.write_cmd_tx_complete);
    // mutex inited in main function
    while (sd_mutex_acquire(&m_mutex) == NRF_ERROR_SOC_MUTEX_ALREADY_TAKEN)
    {
        nrf_delay_us(1000);
    }
    m_available_write_cmd_tx_buffer += p_write_cmd_tx_rsp->count;
    /*if (m_available_write_cmd_tx_buffer >= MINIMUM_AVAILABLE_WRITE_CMD_TX_BUFFER)
    {*/
        err_code = process_tx_fifo_send_bytes();
        APP_ERROR_CHECK(err_code);
        sd_mutex_release(&m_mutex);
    //}
}

My guess is there is something wrong with the way I'm waiting ...

Do you have an advice on how to do this correctly ?

Thanks in advance,

Aloïs KYROU

RE: Queued Write without Response questions

Vidar Berg — Wed, 24 Jul 2019 11:56:30 GMT

Hi Aloïs,

The ble_gattc_conn_cfg_t::write_cmd_tx_queue_size value is "The guaranteed minimum number of Write without Response 
that can be queued for transmission". To increase the default size you can try to add the following code to ble_stack_init:
static void ble_stack_init(void)
{
    ret_code_t err_code;

    err_code = nrf_sdh_enable_request();
    APP_ERROR_CHECK(err_code);

    // Configure the BLE stack using the default settings.
    // Fetch the start address of the application RAM.
    uint32_t ram_start = 0;
    err_code = nrf_sdh_ble_default_cfg_set(APP_BLE_CONN_CFG_TAG, &ram_start);
    APP_ERROR_CHECK(err_code);


    ble_cfg_t ble_cfg;

    memset(&ble_cfg, 0, sizeof(ble_cfg));
    ble_cfg.conn_cfg.conn_cfg_tag = APP_BLE_CONN_CFG_TAG;
    ble_cfg.conn_cfg.params.gattc_conn_cfg.write_cmd_tx_queue_size = 7;

    err_code = sd_ble_cfg_set(BLE_CONN_CFG_GATTC, &ble_cfg, ram_start);
    APP_ERROR_CHECK(err_code);
    
    ...


Note: you may need to increase the app RAM base to leave more RAM to the Softdevice - See debug log for details. 

Best regards,
Vidar