BLE Fragmentation data loss

RE: BLE Fragmentation data loss

Vidar Berg — Tue, 04 Mar 2025 12:59:13 GMT

Bare hyggelig 🙂 Good luck with your project!

RE: BLE Fragmentation data loss

Tbnilsen — Tue, 04 Mar 2025 12:28:43 GMT

Thanks Vidar! You have been most helpful. I’ll pass this thread onto Particle support. In fairness to them their documentation does state that their BLE implementation supports maximum 244 byte packets. So technically I’m violating that. I have been able to modify a copy of their latest OS and at least allow my application to receive these fragmented packets. Ha det bra!

Terje Berg Nilsen.

RE: BLE Fragmentation data loss

Vidar Berg — Tue, 04 Mar 2025 06:45:28 GMT

Hallo,

Yes, with a MTU of 247 bytes, you can receive a read response with up to 246 bytes of payload. Regarding fragmentation, it seems like it must be handled at the application layer on both devices. The maximum attribute value length in a bluetooth characteristic is 512 bytes, which makes it impossible to receive the full 779 bytes of data in one read, even with a long read.

I also had a look at the hal_ble_gatt_client_read() implementation and see that it only issues "ATT_READ_REQ" requests (i.e. it does not perform multiple read request with offsets). The response to this request is ATT_READ_RSP (notice that there is only 1 byte overhead in this response):

ttps://github.com/particle-iot/device-os/blob/507ab1517a8720af4e3bd20a6c6cc66f81cfc1f1/hal/src/nRF52840/ble_hal.cpp#L3239

RE: BLE Fragmentation data loss

Tbnilsen — Mon, 03 Mar 2025 18:46:15 GMT

Vidar,

When you say the payload for a read response can be up to MTU - 1, does that mean if the negotiated MTU is 247 and the peripheral is sending a response large enough to require fragmentation it's possible to get data fragments (payload) of 246 or less? (I.e. 245,244)? This would certainly explain my problem since the Particle OS layer does not allow packets greater than MTU - 3.

RE: BLE Fragmentation data loss

Vidar Berg — Mon, 03 Mar 2025 16:29:29 GMT

Hi,

Unfortunately, we don't have an offline version of the SD API documentation available. I think it's difficult to speculate what the root cause of this issue without some debugging or code review. This is why I initially suggested capturing sniffer trace to analyze the packet exchange on-air. This allows you to verify if the read requests and responses are as expected. If they are, you would know that the bytes are being lost at the application level.

RE: BLE Fragmentation data loss

Tbnilsen — Mon, 03 Mar 2025 13:44:03 GMT

Hallo Vidar,

any chance I can get a copy of the nRF52840 SoftDevice API doc?

Also, do you have an opinion on how such a discrepancy could arise (other than a direct "it was set wrongly")? Is there something in the fragmentation algorithm that could explain this?

RE: BLE Fragmentation data loss

Vidar Berg — Mon, 03 Mar 2025 09:39:27 GMT

Thank you for reporting back and providing an explanation to the issue you observed. As you may know, the default att MTU for BLE devices is always 23 bytes. The longer MTU in this case is negotiated between the devices after the connection is established. The payload for a read response can be up to MTU - 1 byte ATT header while for a write or read request it is MTU - 3 bytes.

Softdevice API documentation for future reference

- sd_ble_gattc_read()

- GATTC Characteristic or Descriptor Value Read

RE: BLE Fragmentation data loss

Tbnilsen — Sat, 01 Mar 2025 12:00:40 GMT

Vidar,

I was able to change Particle's OS to allow getValue reads of more than 244 bytes each call. After I did that, each call to getValue returned 245 bytes (I asked for the length my buffer can hold (1024) each call). Looping through, I now get the full message length (in this test case the message was 798 bytes and I got 245+245+245+63).

I've observed that limiting read requests (ultimately) to SoftDevice, any value short of 245 causes loss of data. For example reading 244 each time loses 1 byte each full buffer return. Reading only 200 each call loses 45, etc.

Does SoftDevice have a bug? Or is this a documented (or implied) feature that must be adhered to for proper fragmented BLE coms. I have also informed Particle about this. Not sure who repairs this.

EDIT: Is this maybe an MTU issue? I tried finding the API for SoftDevice so I can read up on its use, but I was not successful.

EDIT2: It is an MTU issue. Apparently, Particle believes it has set it for a payload of 244 bytes each fragment, when SoftDevice has it as 245. So 1 byte is always lost per full fragment regardless of what the MTU is set to in Particle.

I consider this a Particle issue and not a SoftDevice one. Thanks for your time Vidar. (Tusen takk).

RE: BLE Fragmentation data loss

Tbnilsen — Sat, 01 Mar 2025 00:32:37 GMT

Sorry, looks like there’s another layer in the Particle OS before SoftDevice. I’ll be looking into that further.

That said, I’d really like to get some insight into how a layer such as Particle’s OS should conduct itself when dealing with a fragmented packet being received from a peripheral device.

RE: BLE Fragmentation data loss

Tbnilsen — Fri, 28 Feb 2025 21:12:55 GMT

Here's their code (Particle). I call this member function getValue() four times in rapid succession asking for 244 bytes of data. I get 3 full packets of 244 bytes, and one final packet of 44 bytes for a total of 776 bytes. The message is supposed to be 779 bytes long. When I examine that data I see that the 245th, 490th, and 735th bytes are missing. (BLE_MAX_ATTR_PACKET_SIZE is 244).

ssize_t BleCharacteristic::getValue(uint8_t* buf, size_t len) const {
    if (buf == nullptr || len == 0) {
        return SYSTEM_ERROR_INVALID_ARGUMENT;
    }
    len = std::min(len, (size_t)BLE_MAX_ATTR_VALUE_PACKET_SIZE);
    if (impl()->connHandle() != BLE_INVALID_CONN_HANDLE){
        return hal_ble_gatt_client_read(impl()->connHandle(), impl()->attrHandles().value_handle, buf, len, nullptr);
    }
    else if (impl()->isLocal()) {
        return hal_ble_gatt_server_get_characteristic_value(impl()->attrHandles().value_handle, buf, len, nullptr);
    }
    return SYSTEM_ERROR_INVALID_STATE;
}

I don't know if hal_ble_gatt_client_read is your SoftDevice layer being called, but I think so. Is there something special that needs to happen for fragmented packages?

RE: BLE Fragmentation data loss

Tbnilsen — Fri, 28 Feb 2025 20:59:13 GMT

Hi Vidar,

I've actually done that - the data is moving across BLE correctly. As stated, I used a PC to verify that. I'm now leaning towards the Particle OS layer that interacts with your stack. Their source is open and see that they call hal_ble_gatt_client_read() which I believe is your stack. They limit each call to that to 244 byte buffer. I think they didn't implement the requirements on their end for receiving fragmented packets.

Do you have any documentation for me where I can read up on how to use your stack and specifically on fragmentation?

RE: BLE Fragmentation data loss

Vidar Berg — Fri, 28 Feb 2025 18:12:41 GMT

Hello,

The easiest way to troubleshoot this would likely be by capturing a bluetooth sniffer trace and compare the packet exchange when it works and when it doesn't. Our nRF Sniffer for Bluetooth LE is a low cost option, but it requires that you have a nRF52840 Dongle or one of the other supported boards laying around. Is the code to poll the sensor provided by Particle or is it something you have written. Either way, it would be nice to see the code showing how it is implemented if possible. I'm not aware of any known bugs in the stack that could explain the data loss.

Best regards,

Vidar