Scan Beacons loss too many packets

RE: Scan Beacons loss too many packets

ReedLi — Wed, 09 Jan 2019 01:02:08 GMT

Thanks for your time and patience.

RE: Scan Beacons loss too many packets

natersoz — Tue, 08 Jan 2019 17:27:20 GMT

I will try to replicate your results this weekend (I have to do other work during the week). I'll let you know what I find. I think I understand how you are counting packets and loss now.

RE: Scan Beacons loss too many packets

Kenneth — Tue, 08 Jan 2019 08:08:34 GMT

An advertisement packet is up to 47bytes including everything, this equal 376us. If we presume you have 100 beacons advertise every 500ms, and that you in a perfect world is able to align 99 of the beacons ideally after each other they will take approx 37ms. The risk of a beacon to collide with the 99 others then are 37/500ms = 7.4%. However since you are not in an ideal world it is likely all 100 will collide with each other, so I don't think 17% packet loss is unexpected here.

RE: Scan Beacons loss too many packets

ReedLi — Tue, 08 Jan 2019 03:18:48 GMT

Hi, when receiving BLE_GAP_EVT_ADV_REPORT event, advertising packet number add 1 and i would get peer addr/rssi/dlen/data of the struct ble_gap_evt_adv_report_t.

About channel hopping, beacons would advertise on both 3 channels(37,38,39), and scanner would also scan on both 3 channels.So it wouldn't be 33%.

RE: Scan Beacons loss too many packets

ReedLi — Tue, 08 Jan 2019 02:10:16 GMT

Hi, some collisions and channel switching may cause packet loss, i just don't know if 17% packet loss rate is reasonable and how to improve it.

RE: Scan Beacons loss too many packets

natersoz — Mon, 07 Jan 2019 14:54:18 GMT

Digging deeper :)

What part of the struct ble_gap_evt_adv_report_t are you using for your 'advertising packet number'?

I would expect that with channel hopping enabled, a scanner would only see 1/3 (33%) of the advertised packets. So I am surprised that you are not seeing a much larger packet loss number.

RE: Scan Beacons loss too many packets

Kenneth — Mon, 07 Jan 2019 13:33:57 GMT

I assume there may be some packet loss when switching scan channel, this will occur every scan interval. If you have 100 devices, then it is likely that there will be some collisions, where two advertisers advertise at the same time, this may prevent either or both packets to be received at some intervals.

RE: Scan Beacons loss too many packets

ReedLi — Mon, 07 Jan 2019 08:47:07 GMT

Thanks for your thoughts.Here are replies to thoughts:

Advertisement and scan in 3 channels
There is no printf(), i turn off it.

I can only get adv packets when receive BLE_GAP_EVT_ADV_REPORT Event in ble_evt_dispatch. And after receiving adv packets i only count the number, no flash and printf() operations.

RE: Scan Beacons loss too many packets

natersoz — Mon, 07 Jan 2019 05:29:20 GMT

Update: I realized that the method I am counting packets and you are counting packets are different.

I changed the python program to match what I think you are doing to count packets received and lost and updated the python code. It is checked in here:

https://github.com/natersoz/nrf/blob/master/scripts/packet_loss.py

It is checked in with 1.5 msec of software latency, which is still very high. 17% packet loss.

RE: Scan Beacons loss too many packets

natersoz — Mon, 07 Jan 2019 05:07:37 GMT

Sorry - more questions:

Are you limiting your advertising and scanning to only one frequency?

Let me assume that you are.

At 31 bytes of advertising data, this is a fully loaded advertising packet.

I'm not an expert on the PHY/Link layer aspects of BLE, but that should be 376 usec of duration. Citing Robin Hayden, Bluetooth LE: The Developer's Handbook, 3.1.1 Physical Layer.

This seems right: For a full Link Layer packet there are 37 bytes, 376 bits. At a 1 Mbps symbol rate, 1 bit / symbol that is a Link Layer duration of 376 usec.

Here is how I would model this:

Assuming you're on a single PHY layer frequency (otherwise, I don't know how this could be lossless. I would expect a hopping scanner and a single hopping beacon to experience a 2/3 transmission loss (67%)).
There is an expected arrival time of the advertising interval / the beacon count.

I think this can be modeled as a Poisson distribution:

https://en.wikipedia.org/wiki/Poisson_distribution#Probability_of_events_for_a_Poisson_distribution

The interval is 367 usec + software latency (let's assume ero for now).

Poisson's lambda λ = 367 usec / 5 msec = 0.0734

For zero time of software latency, I get an expected packet loss of 7%:

Beacon Count = 100
Advertising Interval = 500 msec
Link Layer Packet Length = 47 bytes, 376 bits
Link Layer Packet Period = 376 usec
P( 0, 0.075) = 9.276e-01
P( 1, 0.075) = 6.975e-02
P( 2, 0.075) = 2.623e-03
P( 3, 0.075) = 6.574e-05
P( 4, 0.075) = 1.236e-06
P( 5, 0.075) = 1.859e-08
P( 6, 0.075) = 2.330e-10
P( 7, 0.075) = 2.503e-12
P( 8, 0.075) = 2.353e-14
P( 9, 0.075) = 1.966e-16

P packet none = 9.2756e-01
P packet none = 6.9752e-02
P packet lost = 2.6897e-03

Using this python 3.6 script:

''' Calculate advertising packet loss based on the Poission distribution '''
import math

# Link Layer packet lengths.
PREAMBLE_LENGTH         =  1
ACCESS_ADDRESS_LENGTH   =  4
HEADER_LENGTH           =  1
LENGTH_LENGTH           =  1
PAYLOAD_LENGTH          = 37    # 6 + Advertising data bytes
CRC_LENGTH              =  3

# Link Layer
LINK_LAYER_PACKET_LENGTH = (PREAMBLE_LENGTH +
                            ACCESS_ADDRESS_LENGTH +
                            HEADER_LENGTH +
                            LENGTH_LENGTH +
                            PAYLOAD_LENGTH +
                            CRC_LENGTH)

# For a 1 Mbps data rate the symbol period:
BLE_SYMBOL_PERIOD = 1.0E-6

# The Data Packet duration, this is the period of one symbol multiplied
# by the number of bits (bytes * 8) in a packet.
LINK_LAYER_PACKET_PERIOD = BLE_SYMBOL_PERIOD * LINK_LAYER_PACKET_LENGTH * 8

BEACON_COUNT                = 100
BEACON_ADVERTISING_INTERVAL = 500E-3
SOFTWARE_LATENCY            = 0.0  #3.0E-3

def poisson(k, lamb):
    return math.exp(-lamb) * math.pow(lamb, k) / math.factorial(k)

def main():
    print("Beacon Count             = {}".format(BEACON_COUNT))
    print("Advertising Interval     = {:.0f} msec".format(BEACON_ADVERTISING_INTERVAL * 1000))
    print("Link Layer Packet Length = {} bytes, {} bits".format(LINK_LAYER_PACKET_LENGTH, LINK_LAYER_PACKET_LENGTH*8))
    print("Link Layer Packet Period = {:.0f} usec".format(LINK_LAYER_PACKET_PERIOD * 1.0E6))

    p_packet_none = 0.0     # The probability that no packet is received or lost.
    p_packet_recv = 0.0     # The probability that a packet was received.
    p_packet_loss = 0.0     # The probability that a packet was lost

    lamb = (LINK_LAYER_PACKET_PERIOD + SOFTWARE_LATENCY) / (BEACON_ADVERTISING_INTERVAL / BEACON_COUNT)
    for k in range(0, 10):
        p = poisson(k, lamb)
        if k == 0:
            p_packet_none = p
        elif k == 1:
            p_packet_recv = p
        elif k > 1:
            p_packet_loss += p
        print("P({:3d}, {:.3f}) = {:.3e}".format(k, lamb, p))

    print("")
    print("P packet none = {:.4e}".format(p_packet_none))
    print("P packet none = {:.4e}".format(p_packet_recv))
    print("P packet lost = {:.4e}".format(p_packet_loss))

if __name__ == '__main__':
    main()

Here are some thoughts:

Are you using just 1 frequency for BLE advertisements? This is a fundamental assumption I have made and if it is incorrect then none of this makes any sense.
If you are then the most likely point of loss is that the BLE central which is receiving the advertisements is blocking and getting in the way of reception? Increasing the SOFTWARE_LATENCY value to 3.0E-3 (a very large value) will get you in the ballpark. Is your accounting on the Central side using a blocking printf()?

This is a very, very interesting setup that you have and I would like to know how your results turn out.

Thanks!

RE: Scan Beacons loss too many packets

ReedLi — Mon, 07 Jan 2019 01:05:38 GMT

Hi, i test 1 advertiser just now, and i get nearly zero packet loss.

RE: Scan Beacons loss too many packets

natersoz — Sun, 06 Jan 2019 19:07:28 GMT

That sounds like a pretty cool setup!

If you reduce your beacon count to just 1 advertiser then do you get nearly zero packet loss?