nRF52 sending large bulk data

RE: nRF52 sending large bulk data

endnode — Sat, 12 Nov 2016 18:19:47 GMT

The post you mention is solving the basic problem "how to cut longer message into 20-byte fragments, stack them to BLE Stack - aka Nordic Soft Device - to go out as Tx packets, manage overflow of internal Soft Device buffers and continue once some buffer is empty until all data go out". Now it's question if you need some "session" management on top of it (e.g. which side can talk when, how you find out that whole "long message" has arrived and you can process it etc,) In principle you can define per message or per fragment but there are countless ways how to define all the details...

RE: nRF52 sending large bulk data

endnode — Sat, 12 Nov 2016 18:19:36 GMT

Well there is this "Nordic UART Service" aka "NUS" available in nRF5 SDK examples and it is one example how to solve "ATT MTU fragmentation and reassembly" problem. You probably can just use it out of the box without big investigation of details.

RE: nRF52 sending large bulk data

kmorris — Sat, 12 Nov 2016 00:15:35 GMT

I just implemented mine following this example devzone.nordicsemi.com/.../ Jack - have you tried this one?

I don't update my 'last synchronized' pointer until it all goes through, that way it ends up starting over if it gets interrupted. (I may change that and have it start where it left off, idk yet) I will probably also implement a way for the app/client to ask for prior data, just in case something goes awry on the app side.

RE: nRF52 sending large bulk data

endnode — Fri, 11 Nov 2016 23:45:06 GMT

(continuation...) define some "ACK" messages (they can even contain number of bytes received or hash - but that would be probably too heavy). In the end pretty anything will do the job, it's just a question how difficult for implementation you make it, how efficient you want it (in terms of computation power/time and overhead inside GATT data which lowers your effective bandwidth) and what error/recovery mechanisms you need for session management...

RE: nRF52 sending large bulk data

endnode — Fri, 11 Nov 2016 23:42:50 GMT

TCP/IP, like srsly???:) I assume you will be using some "self-consistent" protocol on top of GATT anyway (e.g. there will be some length indicators or even integrity checks based on some CRC/hash/crypto signature scheme). But if you are sure that data formatting is completely free and up to you, then simply use some light-weight protocol used in other embedded systems. E.g. Host Control Protocol (HCP) or something even simpler: dedicate part of each "fragment" to "header" and indicate if this is last fragment or not. If there will be any issue it will anyway break the link so you don't need to care about anything else then case where receiver would be processing incomplete data. If your use case has no natural "session" management (e.g. that one side would be sending commands and then waiting for response) then you can (to be continued...)

RE: nRF52 sending large bulk data

jackbauerctu — Fri, 11 Nov 2016 23:33:14 GMT

I was also thinking about how to do implement this.

Perhaps I can try to mimic the windowing mechanism from TCP/IP? Maybe the initial window will be 10 items. Then you would get DATA OK for each of those items. Once you get the DATA OK for item 0, then move the window up.

You can even go further and dynamically adjust the window size when the BLE connection isn't that stable too.

Am I over thinking this? Or maybe the simple send one item and expect DATA OK; then send the next item?

RE: nRF52 sending large bulk data

kmorris — Thu, 10 Nov 2016 17:10:03 GMT

Thanks. That makes sense.

RE: nRF52 sending large bulk data

endnode — Thu, 10 Nov 2016 12:04:48 GMT

Hi kmorris,

I guess there is slight misunderstanding of how BT LE Logical Link layer works (and why we all love BLE - including you;): LL makes sure that there is automatic packet re-transmission so a) you don't have any problem with missing packets in the middle and b) if connection fails then you know (at least on LL level) which was the last packet transferred. If you read BT SIG Core specification then unacknowledged or acknowledged WRITE/READ/NOTIFY/INDICATE methods are simply because implementation specifics can cause overflow or other issues on GATT and APP layers so even some PDUs are delivered by LL layer the data might be discarded (and the way to ensure 100% certainty is to use proper "with Response" types of ATT/GATT methods). However in reality this is very unlikely and it's much easier to implement final ACK on top of GATT/APP protocol (your app will just say "DATA OK").

RE: nRF52 sending large bulk data

Petter Myhre — Thu, 10 Nov 2016 11:10:01 GMT

Notifications are acknowledged at the link layer, so you will not lose any.

RE: nRF52 sending large bulk data

kmorris — Wed, 09 Nov 2016 23:33:21 GMT

Hi Jack & Jan -

As a follow-up, I understand when transferring large amounts of data it has always been recommended to use notifications to optimize throughput - because you don't have to get a response. However, what if you DO care that all the data gets received? Is it still better to send it all quickly through notifications and then have the client request a re-send of any missing data? Say I have a 20-byte datapoint for every minute and I've been storing them for the last 30 minutes or so, and I send all 30 of them as notifications, if the client doesn't receive datapoints 11-15, I could request a re-send? Or, is there an example where the acknowledgement is received per packet sent? And would you be able to point me to it, since most examples (and what I have used previously) was the notifications?

RE: nRF52 sending large bulk data

endnode — Sat, 05 Nov 2016 23:15:36 GMT

Hi Jack,

Short answer is YES.

Long answer follows:

If you want to use BT LE and at the same time being interoperable with platforms such as Android, iOS, Windows etc. then you have almost no other choice then to deploy application on top of GATT layer (so it will be using GATT Server or GATT Client role).
Then to optimize throughput you want to use WRITE without response method from GATT Client side and VALUE NOTIFICATION (HVX) from GATT Server side.
And in case that your data could be in principal larger then (ATT_MTU - 3) size you will need to develop some kind of fragmentation/reassembly protocol on top of these GATT methods (note that default ATT_MTU size is 23 bytes).
If you want to see some practical examples and throughput numbers then please download latest S13x SoftDevice Specification v3.0 and refer to chapter 18 (there aren't all possible combinations but these values illustrate both "theoretical" and "practical" limits very well).

Cheers Jan