Which is the best way of sending the data read from a sensor?

RE: Which is the best way of sending the data read from a sensor?

Einar Thorsrud — Fri, 30 Aug 2024 07:22:27 GMT

Hi Adrian,

I am not familiar with the adxl355, but how fast can you read data from it? There is probably a maximum sampling rate etc. That should be specified in the data sheet. I would suggest that you first look into what you can actually achieve theoretically. And then start testing without bluetooth this time, just logging (not very often) how much data you have recieved, and ignore the data.

RE: Which is the best way of sending the data read from a sensor?

Adrian1504 — Thu, 29 Aug 2024 15:27:10 GMT

Hi Einar,

Thank you for your answer.

I tried what you suggested, and yes, now I receive more. So I am receiving 25920 int 32 which I think makes it .8 Mbps. Since I think now the problem is the reading of the accelerometer data, I changed my program to read the FIFO, instead of reading register each time it was accessed. I am now getting .5 Mbps, which is good, but I'd like to increase it to .8 that was amazing. Would you know of any way I could make this better?

Maybe making 2 buffers and sending one and then the other? Increasing the frequency of SPI?

I will try them, but if you have any advice, I would be really interested in reading it.

Best regards,
Adrian Lopez.

RE: Which is the best way of sending the data read from a sensor?

Einar Thorsrud — Mon, 26 Aug 2024 14:28:28 GMT

Hi Adrian,

[quote user="Adrian1504"]Here, what do you mean?[/quote]

I was refering to the more data field in the packets in the sniffer trace. When that is not set, that indicate that the nRF does not have more data to send, and the connection event will be closed. So it seems you are not providing data to the BLE stack fast enoug (for higher throughput).

Looking at your code I see you send data in your ble_notify_thread() and this waits for a semaphore buffer_sem given in simulate_data. What throuhgput do you get if you skip this part and just send the same dummy data over and over (no acellerometer data and no semaphore)? The way the code looks now I cannot see if the limiting factor here is getting the accelerometer data or sending the data over BLE.

RE: Which is the best way of sending the data read from a sensor?

Adrian1504 — Mon, 26 Aug 2024 11:02:58 GMT

Hi Einar,

Thank you so much for your reply.

Here, what do you mean?

[quote userid="7377" url="~/f/nordic-q-a/108551/which-is-the-best-way-of-sending-the-data-read-from-a-sensor/499324"]I also see that ther more data field is often not set, which is also a potential problem[/quote]

I adjusted the buffer sizes and also the event length.

I tried changing the connection interval to 30ms, and it still does not work. Maybe the way I am setting my BLE connection is not right?

For the prj:

CONFIG_BT_USER_DATA_LEN_UPDATE=y
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_BT_BUF_ACL_RX_SIZE=502
CONFIG_BT_BUF_ACL_TX_SIZE=502
CONFIG_BT_L2CAP_TX_MTU=498

CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048
CONFIG_MAIN_STACK_SIZE=2048
CONFIG_BT_RX_STACK_SIZE=2048

And in the main, I see that in the sample you attaches, they are using bt_le_conn_param, I am using this, Should I change this?:

static struct bt_le_adv_param *adv_param = BT_LE_ADV_PARAM(
	(BT_LE_ADV_OPT_CONNECTABLE |
	 BT_LE_ADV_OPT_USE_IDENTITY), /* Connectable advertising and use identity address */
	BT_GAP_ADV_FAST_INT_MIN_2, /* 0x30 units, 48 units, 30ms */
	BT_GAP_ADV_FAST_INT_MAX_2, /* 0x60 units, 96 units, 60ms */
	NULL);

I attach my code in case it is more useful.

devzone.nordicsemi.com/.../BLESPIZip.zip

Thank you so much

Adrian Lopez

RE: Which is the best way of sending the data read from a sensor?

Einar Thorsrud — Thu, 22 Aug 2024 08:31:10 GMT

Hi Adrian,

From what I understand you want to maximize throughput. Generally, to maximize thourhput with BLE from a GATT server to a client this is important:

2 Mbps PHY
Use data length extension to send as long link laye rpackets as possible
Use along event length (equal to the connection interval).
If using an event length equal to the connection inteval and the link has little packet loss, increasing the connection interval can then increase the thorughput due to overhead when changing connection interval.
Make sure various buffers are large enough and always provide data to the stack so that it has data to send.

Looking at the sniffer trace, 1 and 2 from the above list is good. The connection inteval is 15 ms which is perhaps a bit low. But at the same time, I also see that ther more data field is often not set, which is also a potential problem.

So I suggest that you start with adjusting buffer sizes and event length as shown in the prj.conf from the throughput sample and perhaps also adjust the connection interval. Do you get better throhgput then? If not, make sure you attempt to send data fast enough.

RE: Which is the best way of sending the data read from a sensor?

Einar Thorsrud — Wed, 21 Aug 2024 14:06:57 GMT

Hi Adrian,

I will look into this and get back to you tomorrow.

Br,

Einar

RE: Which is the best way of sending the data read from a sensor?

Adrian1504 — Mon, 19 Aug 2024 13:07:50 GMT

Now I figured out what to send, an apology.
devzone.nordicsemi.com/.../sniffer1.zip

Best regards,

Adrian Lopez

RE: Which is the best way of sending the data read from a sensor?

Adrian1504 — Mon, 19 Aug 2024 13:03:16 GMT

Hi Karl,

I am really sorry for the delay. I had to focus also on the reading of the sensor, the app for displaying the data and got stuck for a really long time. Now I had the chance to return and have a look at this issue again.

I see that now I am obtaining again .02 MB/s. I tried making 2 different threads, one that reads the values from the sensor ands stacks them in an array, and other one that sends the data. It is also not working.

Having a look at the sniffer tool that you mentioned, I receive some things which I don't know if it is helpful to know, or how to interpret them. I see for example, that the updates in parameters to 2M and data length are being made.

But when I subscribe to the notifications, I see that the master is also taking quite some time to send something and it is empty, is this normal?

Also, I don't see any warnings or errors.

I attach some screenshots of the wireshark. If there is a better way to share the logs, please let me know.

This first image is before any connection event

This second image is when the connection happened

This third image is before I subscribe

This fourth image is during the subscription

If there is any other information that might be useful, please let me know.

Best regards,

Adrian Lopez

RE: Which is the best way of sending the data read from a sensor?

Karl Ylvisaker — Tue, 02 Apr 2024 10:45:25 GMT

Hello again, Adrian

Thank you for your patience with this - I have been out of office due to the public easter holiday last week, but now I am back.

Please make sure that you have followed all the steps in the Installing the sniffer guide - especially the part on Installing the sniffer capture tool. Please do this, and let me know if the issue persists!

Best regards,
Karl

RE: Which is the best way of sending the data read from a sensor?

Adrian1504 — Mon, 25 Mar 2024 16:57:01 GMT

Hello Karl,

Sorry for the late response. I have been trying to install it, but I ran into this problem:
"The term 'nrf_sniffer_ble.bat' is not recognized as
the name of a cmdlet, function, script file, or operable program."

I have already checked and I have python installed. Do you know what is the problem?

RE: Which is the best way of sending the data read from a sensor?

Karl Ylvisaker — Thu, 14 Mar 2024 13:09:47 GMT

Hello Adrian.

Thank you for clarifying - I am glad to hear that you are having progress on the throughput!

The next step in figuring out where the bottleneck in the communication is would be to gather a sniffer trace, so that we can verify that the application always have data ready to send, and whether there is anything hampering the data from being sent.
It could for instance here be that the application is not supplying the data fast enough due to other tasks taking time away from the data generation.
Have you had a look at my suggestion for sniffer in my previous comment?

Best regards,
Karl

RE: Which is the best way of sending the data read from a sensor?

Adrian1504 — Thu, 14 Mar 2024 12:01:58 GMT

Hi Karl,

A small update, I changed some things in my code and was able to achieve now .9 Mbps, so I guess the way I am queueing the data and sending it is still not correct. I would be happy to be able to get it up to 1.2. Of course, I will leave some of my functions here:

static void simulate_dataX(void)
{
	sensor_valueX++;
	if (sensor_valueX == 9) {
		sensor_valueX = 0;
	}
}

static void simulate_data(void){
	simulate_dataX();
	simulate_dataY();
	simulate_dataZ();
}

int my_lbs_send_sensor_notify(void)
{
	if (!notify_sensor_enabled) {
		return -EACCES;
	}

	return bt_gatt_notify(NULL, &accelerometer_svc.attrs[1], 
			      &buffer,
			      sizeof(buffer));
}

static void pack_data(uint8_t x, uint8_t y, uint8_t z){
	buffer[buffer_index].x = x;
    buffer[buffer_index].y = y;
    buffer[buffer_index].z = z;
    buffer_index++;

	if (buffer_index == BUFFER_SIZE){
		int err = my_lbs_send_sensor_notify();
		if (err) {
			LOG_ERR("Notification failed:  %d", err);
		}
		buffer_index = 0;
	}

}

void send_data_thread(void)
{
	while(1){
		if(notify_sensor_enabled){ //Without this if I was only achieveing .02 Mbps
			simulate_data();
			pack_data(sensor_valueX,sensor_valueY,sensor_valueZ);
		}

	}
		
}

K_THREAD_DEFINE(send_data_thread_id, STACKSIZE, send_data_thread, NULL, NULL,
		NULL, PRIORITY, 0, 0);

Here is the prj:

# nothing here
CONFIG_LOG=y

CONFIG_BT=y
CONFIG_BT_PERIPHERAL=y
CONFIG_BT_DEVICE_NAME="BLE Device"
CONFIG_BT_DEVICE_APPEARANCE=0
#CONFIG_BT_BAS=y

CONFIG_BT_GATT_CLIENT=y

CONFIG_DEBUG_OPTIMIZATIONS=y
CONFIG_DEBUG_THREAD_INFO=y
CONFIG_FPU=y

CONFIG_BT_PERIPHERAL_PREF_MIN_INT=30
CONFIG_BT_PERIPHERAL_PREF_MAX_INT=40
CONFIG_BT_PERIPHERAL_PREF_LATENCY=0
CONFIG_BT_PERIPHERAL_PREF_TIMEOUT=1000
CONFIG_BT_GAP_AUTO_UPDATE_CONN_PARAMS=y

CONFIG_BT_USER_PHY_UPDATE=y

CONFIG_BT_USER_DATA_LEN_UPDATE=y
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_BT_BUF_ACL_RX_SIZE=251
CONFIG_BT_BUF_ACL_TX_SIZE=251
CONFIG_BT_L2CAP_TX_MTU=247

CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048
CONFIG_MAIN_STACK_SIZE=2048
CONFIG_BT_RX_STACK_SIZE=2048

If you need any other part of the code, let me know. Also, I am not familiar to this tool, but I will have a look at it! Thank you!

Best regards,

Adrian Lopez Vudoyra

RE: Which is the best way of sending the data read from a sensor?

Karl Ylvisaker — Wed, 13 Mar 2024 12:01:20 GMT

Hello Adrian Lopez,

[quote user="Adrian1504"]

When I connect my device it prints this notifications in the nrf Connect app:

PHY updated (TX: LE 2M, RX: LE 2M)
Connection parameters updated (interval: 11.25ms, latency: 0, timeout: 10000ms)

And in the terminal I am getting this messages:

MTU exchange successful, new MTU: 244 bytes
Data length updated. Length 251/27 bytes, time 2120/2120 us

So I think that the central is accepting the changes successfully and is setting the same parameters as in the link you provided me.

[/quote]

Thank you for clarifying - it is good to have it confirmed that you are using a 11.25 ms connection interval.

It seems that you have successfully changed your connection parameters, and so that should not be what is causing the issue with the throughput either. Could you detail how you are generating and queueing the data for sending again? It would be great to see the relevant code as well, if possible.

If the throughput is still an issue we will need to capture a sniffer trace of the communication between the devices. Are you familiar with the nRF Sniffer tool?

Best regards,
Karl

RE: Which is the best way of sending the data read from a sensor?

Adrian1504 — Thu, 07 Mar 2024 13:27:56 GMT

Hi Karl,

[quote userid="87869" url="~/f/nordic-q-a/108551/which-is-the-best-way-of-sending-the-data-read-from-a-sensor/472107"]What does your central's device log say when you connect to your peripheral device?
Does it log the used connection parameters that it uses?[/quote]

When I connect my device it prints this notifications in the nrf Connect app:

PHY updated (TX: LE 2M, RX: LE 2M)
Connection parameters updated (interval: 11.25ms, latency: 0, timeout: 10000ms)

And in the terminal I am getting this messages:

MTU exchange successful, new MTU: 244 bytes
Data length updated. Length 251/27 bytes, time 2120/2120 us

So I think that the central is accepting the changes successfully and is setting the same parameters as in the link you provided me.

I will leave here a stackoverflow link that I created in case you can have a look at some of the code I have.

stackoverflow.com/.../nrf52840-not-achieving-sending-2-mbps

Thank you so much for your time.

Adrian Lopez.

RE: Which is the best way of sending the data read from a sensor?

Karl Ylvisaker — Mon, 04 Mar 2024 17:21:23 GMT

Hello Adrian Lopez,

[quote user="Adrian1504"]No, here I meant that there was a sleeping function after the notification for delaying the notification process. But now I deleted it. I also created a buffer of size 80 and each position has a 3 variable struct. Now I am able to get 11 buffers per second (That is 2640 bytes per second which is nothing if we convert it to Mbps). To be able to achieve this, because I was getting only 2 buffers per second, in the prj.conf I changed the following parameters: [/quote]

This way it does not actually matter when you queue the notification, since the check in essence is 'is there a notification ready for sending' when a connection event is upcoming.

[quote user="Adrian1504"]

CONFIG_BT_PERIPHERAL_PREF_MIN_INT=10

CONFIG_BT_PERIPHERAL_PREF_MAX_INT=10

Why am i getting so little?
Would it be useful for me to send my code?

[/quote]

Those kconfig options only sets the peripheral's preferences, while it is still the central that decides which connection interval that will actually be used. What does your central's device log say when you connect to your peripheral device?
Does it log the used connection parameters that it uses?

The peripheral may send a connection parameter update request, to suggest to use its preferred connection parameters, but the central may reject this update request and instead use whichever connection parameters it would like. The peripheral may then either accept this, or terminate the connection.

I think the key to your throughput in this case is the connection parameters. Please change these on your central device, or make sure that the central device will accept any connection parameter update request from the peripheral, and then use one of the connection parameter configurations shown here to achieve higher throughput.

Best regards,
Karl

RE: Which is the best way of sending the data read from a sensor?

Adrian1504 — Mon, 04 Mar 2024 13:02:58 GMT

Hi Karl,

[quote userid="87869" url="~/f/nordic-q-a/108551/which-is-the-best-way-of-sending-the-data-read-from-a-sensor/471614"]By this I assume you mean the connection interval?[/quote]

No, here I meant that there was a sleeping function after the notification for delaying the notification process. But now I deleted it. I also created a buffer of size 80 and each position has a 3 variable struct. Now I am able to get 11 buffers per second (That is 2640 bytes per second which is nothing if we convert it to Mbps). To be able to achieve this, because I was getting only 2 buffers per second, in the prj.conf I changed the following parameters:

CONFIG_BT_PERIPHERAL_PREF_MIN_INT=10

CONFIG_BT_PERIPHERAL_PREF_MAX_INT=10

Why am i getting so little?
Would it be useful for me to send my code?

Best regards,

Adrian Lopez

RE: Which is the best way of sending the data read from a sensor?

Karl Ylvisaker — Thu, 29 Feb 2024 15:58:26 GMT

Hello Adrian Lopez,

[quote user="Adrian1504"]Sorry for so many questions, but thank you for your help![/quote]

No need to apologize, I am happy to help! :)

[quote user="Adrian1504"]The value that I changed here was the Notify interval. [/quote]

By this I assume you mean the connection interval?
It would be great if you could be explicit about the changes you made, so that I can know with certainty which value or parameter was changed, since it seems to me that the change was unsuccessful.

[quote user="Adrian1504"]In the function mentioned (ble_write thread), there is no notify_interval, this is not necessary for notifications? I also saw that the function being used is bt_nus_send(), here I could also use the bt_gatt_notify function? Or this function(bt_nus_send) is specific for sending buffers?[/quote]

You are correct that there is no notify_interval directly, and this is because the peripheral will send the notification in the next possible connection event - there is one connection event each time the connection interval has elapsed.
So for example, if you queue a notification for sending every 500 ms, and your connection have a connection interval of 250 ms, then your peripheral will send one empty packet in the first connection event (because it did not have any notification to send), and then send queued notification in the next connection event, then it will revert to sending an empty packet again to maintain the connection until a new notification is queued for sending.

You are also correct that this particular example uses the bt_nus_send function to send the notification, because the example is made to showcase the use of the Nordic UART service in particular - but if you look into the implementation of this service you will see that it also uses the bt_gatt_notify functionality behind the scenes :)

[quote user="Adrian1504"]And last, I guess that instead of the line 638, instead of using k_fifo_get, I should put the function which with I will read the information of my sensor? or it should be the function that will put the information in the buffer?[/quote]

This depends on the requirements to your application - such as whether it would be acceptable or perhaps important to drop older sensor measurements, or whether it is important that all measurements make it across the link.
In your case, I would recommend using a FIFO buffer in the same way that the NUS does, since this lets you gather data for sending independently of the BLE notification queueing.

Best regards,
Karl

RE: Which is the best way of sending the data read from a sensor?

Adrian1504 — Thu, 29 Feb 2024 13:45:03 GMT

Hi Karl,

Again, thank you for your detailed and helpful answer.

[quote userid="87869" url="~/f/nordic-q-a/108551/which-is-the-best-way-of-sending-the-data-read-from-a-sensor/470758"]Which value did you change here exactly?[/quote]

The value that I changed here was the Notify interval.

[quote userid="87869" url="~/f/nordic-q-a/108551/which-is-the-best-way-of-sending-the-data-read-from-a-sensor/470758"] can take a look at the Peripheral Nordic UART example[/quote]

I went through the code that you mentioned, and it looks like it can be really helpful, I just have some doubts:

In the function mentioned (ble_write thread), there is no notify_interval, this is not necessary for notifications? I also saw that the function being used is bt_nus_send(), here I could also use the bt_gatt_notify function? Or this function(bt_nus_send) is specific for sending buffers?

And last, I guess that instead of the line 638, instead of using k_fifo_get, I should put the function which with I will read the information of my sensor? or it should be the function that will put the information in the buffer?

Sorry for so many questions, but thank you for your help!

Adrian Lopez

RE: Which is the best way of sending the data read from a sensor?

Karl Ylvisaker — Mon, 26 Feb 2024 13:36:09 GMT

Hello,

[quote user="Adrian1504"]Thank you for the answer.[/quote]

No problem at all, I am happy to help! :)

[quote user="Adrian1504"]I would like to receive more often, I tried lowering the notify interval, which was set to 500. And I still get the same amount of numbers. [/quote]

Which value did you change here exactly?
Please note that the peripheral device may only have a preference for the connection parameters, it does not actually set it. It is the central that decides which connection parameters that will be used.

[quote user="Adrian1504"]

I am not understanding how I can should send with the bt_gatt_notify(). Should it be: bt_gatt_nottify(NULL,<characteristic declaration index>, &<array>,sizeof(array)); ?

Or should it be one gatt_notify() per each array that I have?

[/quote]

If you would like to see a good example of how you can regularly output your sensor data over the BLE link you can take a look at the Peripheral Nordic UART example, which forwards everything it receives over BLE out on its UART, and visa versa. In the ble_write_thread you can see how it checks the FIFO for new data, and then queues it for transfer over BLE is there is any.

Best regards,
Karl

RE: Which is the best way of sending the data read from a sensor?

Adrian1504 — Mon, 26 Feb 2024 11:51:35 GMT

Dear Karl,

Thank you for the answer.

I saw that in fact I was getting all the data, but I am getting 3 int_8 per second. I would like to receive more often, I tried lowering the notify interval, which was set to 500. And I still get the same amount of numbers.

I need to read my sensor which throws 3 values per reading (x,y,z), the frequency of reading is still not defined, but I am thinking on doing it via I2C.

Should I do an array of 3 arrays for each of the values?

I am not understanding how I can should send with the bt_gatt_notify(). Should it be: bt_gatt_nottify(NULL,<characteristic declaration index>, &<array>,sizeof(array)); ?

Or should it be one gatt_notify() per each array that I have?

Thank you!

RE: Which is the best way of sending the data read from a sensor?

Karl Ylvisaker — Mon, 26 Feb 2024 11:15:57 GMT

Hello,

[quote user="Adrian1504"]I am still not reading any sensor, for now it is just dummy values, and supposedly they should be increasing in 1 unit each time the loop comes. But sometimes it is being increased in more than one unit.
I think maybe sending each time the sensor is being "read" is not the best. Is it possible to send maybe an array? Or which is the best way to send the most amount of the "read" sensor?[/quote]

Thank you for clarifying. I assume then that this means that you are sending the dummy data over your BLE link, and the main question is why some of the numbers in the sequence is missing.
In essence, BLE is a lossless protocol - no data is ever lost in the link - since every packet is either acknowledged by the peer. If a packet it not acknowledged by the peer in its next packet it will be retransmitted until acknowledged, or until the link is terminated.
Therefore, the data must be lost before it is queued for sending.
Could you be that your call to queue the data for sending failed, for instance due to a full buffer?
This could lead to some missing samples in the sequence.
If you share your relevant 'loop' code then I can take a look.

[quote user="Adrian1504"]I think maybe sending each time the sensor is being "read" is not the best. Is it possible to send maybe an array? Or which is the best way to send the most amount of the "read" sensor?[/quote]

You can send an array in a single packet, yes - it just depends on the size of the data and frequency.
How often are you reading the sensor data, how large as the samples, and what latency requirements do you have for your application?

Best regards,
Karl

RE: Which is the best way of sending the data read from a sensor?

Adrian1504 — Sat, 24 Feb 2024 13:01:06 GMT

Hi Karl,

I am still not reading any sensor, for now it is just dummy values, and supposedly they should be increasing in 1 unit each time the loop comes. But sometimes it is being increased in more than one unit.
I think maybe sending each time the sensor is being "read" is not the best. Is it possible to send maybe an array? Or which is the best way to send the most amount of the "read" sensor?

Thank you!

RE: Which is the best way of sending the data read from a sensor?

Karl Ylvisaker — Fri, 23 Feb 2024 10:16:07 GMT

Hello,

[quote user=""]I need to read the data from an acceleration sensor. I tried with a custom program of mine to simulate some data, but it is delaying too much.[/quote]

Could you elaborate on what you are doing here, and which delay you are referring to here?
Are you reading out the sensor data over I2C for instance, and then sending it over BLE, where the mentioned delay applies to the BLE communication?

If you are just starting out with the nRF Connect SDK then I highly recommend taking a look through the nRF Connect SDK Fundemental course to quickly familiarize with the SDK! :)

Best regards,
Karl