Best approach for collecting data from hundreds of sensors

[gweppler]

Hello, 

I'm looking for some general guidance on the following application:

• 1 nRF52840 based gateway to cloud, on power supply
• a few hundred nRF52832 sensor nodes, all battery operated, all within reach of the GW
• sensor nodes are asleep most of the time (events wake them up, then they send data while a certain condition is met, then go back to sleep after a while).
• Updates from sensor nodes to gw are 10-15 bytes each, at intervals of 1s-30s.
• Acks from GW to nodes "required"/very nice to have but lack of acks is not necessarily a show stopper as very few lost updates wouldn't be a massive problem. Just not a perferred starting point for us. 

We have this running fine with GATT connections for up to 20 sensors until we hit the softdevice's concurrent connection limit in the GW.

Questions

1) What approaches would be best suited to scale this beyond 20 sensor nodes? We are currently considering the following options: 

a) BLE/GATT/connection based as per our status quo, but adding some logic on top that builds the lists of sensor nodes to connect to next. First tests seem to indicate the connecting/disconnecting will take a couple of seconds so this maybe to slow to reach above goals. Is there a best practice approach of how to build the scheduler on top of the 20-connections-limit so that the number of devices can increase significantly? 

b) Advertisements only / Beacon. No acks... 

c) mesh. All sensor nodes would be LPNs. All of them are within RF reach of the GW. Would this work with mesh? 

d) ESB. 

Which of these make most sense for above scenario?

2) Are there other alternatives that we should consider? 

3) Any other hints that could help us with the above?

Thanks much in advance!

gj

[Edvin]

Hello,

1) That really depends on the number of nodes, and how often they send whether this is possible or not. 

100 nodes, 10B per 30 seconds = 33Bps = 0.26kbps

100 nodes, 15B per 1 second = 1.5kBps = 12kbps

The issue with mesh networks is that the throughput capacity goes down the more nodes you have. It is really not designed for gathering data from several hundred sensors.

A standard BLE connection has much higher throughput, but it is as you say, the connection also takes time.

ESB is possible. It has ACKs, and may be a possiblility for your case. I would recommend to have some sort of "connection" here as well, so that not everyone is broadcasting the sensor data at all times, as it would increase the overall traffic, increasing the chance of packet loss. So basically that every node listens for a specific message, meaning that it is that node's turn to send the payload data.

BLE connections and disconnection is also a possibility. In order to speed up the connection/disconnection time, you can decrease the connection interval on your devices (the initial connection intervals). Shortening this will allow you to be able to send data quite quick.

So I think BLE or ESB is the way to go. It is difficult to say what would work best. As mentioned, in ESB, I would look into having some sort of communication protocol, so that not everyone is sending the sensor data at will, to reduce the chance for packet loss. In BLE I would recommend to keep the number of connections (far) below 20. Maybe only 2-3, or maybe even only 1 at the time. This way the central can use the radio 100% for scanning whenever it isn't connected, meaning that if it is looking for one particular node that it hasn't been connected to in a while, it will increase the chance of picking up the advertising packet for that node. 

Whether you use BLE or ESB, I would recommend that the sensor nodes go quiet for as long as possible after connection and sensor dumping, leaving the air traffic to the nodes that hasn't been connected in a while. 

So, in BLE, things are a bit more random, while in ESB, you can create your own protocol. Maybe the gateway can broadcast a message with a node ID telling everyone that this node should send the data now.

It is an interesting problem. I think the throughput is too high for a mesh network of that size. Connecting and disconnecting with that number of nodes is on the edge of what is possible in BLE. The ESB protocol that I imagine you would have to implement is maybe a bit more work, but it may be the best shot if it is not possible with BLE. You can do some testing on how fast you can receive the payload (10-15 Bytes) from a single node using BLE and ESB. 

I also recommend you read through this guide on advertising (even though you go for ESB), as it gives some perspective on scanning and broadcasting. Note that it is the time you actively use the radio that affects the current consumption. So advertising is cheap, while scanning is expensive, power wise.

[gweppler]

Thank you for this 5 star answer- very helpful! We'll digest & get back.