UART FIFO Flush: How to determine that UART TX is completed

RE: UART FIFO Flush: How to determine that UART TX is completed

ovrebekk — Fri, 02 Aug 2019 08:19:53 GMT

It seems like the buffers are full then. When the NRF_ERROR_NO_MEM error occurs you should stop sending bytes to the UART, and buffer them locally instead.

Once the APP_UART_TX_EMPTY event occurs you know that the UART has emptied it's internal buffers and are ready to receive more data.

Best regards
Torbjørn

RE: UART FIFO Flush: How to determine that UART TX is completed

Comodo — Thu, 01 Aug 2019 13:00:28 GMT

Hi I have tried that with a buffer of 100.000Bytes size and fed it with BLE data but it is always the same:

-> NRF_ERROR_NO_MEM (4)

I have also tried to buffer 20 Packets (20 x 244Bytes) and start then with the UART but no effect.

RE: UART FIFO Flush: How to determine that UART TX is completed

ovrebekk — Thu, 01 Aug 2019 07:33:24 GMT

If you have room to buffer the entire image in RAM then you should be able to simply use this buffer. Once the BLE data starts coming in you store it in the buffer, and once the buffer has got enough data to send a block over the UART you send a block from the buffer over the UART.

The incoming data from BLE will always go to the buffer first, and it doesn't matter what the UART is doing.

If the UART is quicker than the BLE link then they will stay more or less in sync, but if the UART is slower then the buffer will store the data while the UART is working.

[quote user="Comodo"]Regarding LOG_INFO or debug information; does it has any impact on UART speed? [/quote]

Which backend are you using?
The RTT backend should have very little impact on system performance.

Best regards
Torbjørn

RE: UART FIFO Flush: How to determine that UART TX is completed

Comodo — Wed, 31 Jul 2019 16:12:49 GMT

Yeah I have tried to send small heaps (244Bytes, each BLE Packet) on every "on_HVX" interrupt. I have faced crashes at some point (for example packetnr. 55 out of 365 BLE Packets). It was necessary to slow down the BLE notifications (by 4 ms for each packet). The reason for crash I guess was that the interrupt came before UART wasn't done with a packet transfer. BLE 2MBit is (as expected) a bit faster than UART can handle. But unfortunately reducing the speed to BLE 1MBit was not the solution. I had to build some (4ms) delays between notifications on Android side. In sdk_config I have:

#define NRF_BLE_SCAN_MIN_CONNECTION_INTERVAL 7.5

#define NRF_BLE_SCAN_MAX_CONNECTION_INTERVAL 30

#define NRF_SDH_BLE_GATT_MAX_MTU_SIZE 247

#define NRF_SDH_BLE_GAP_EVENT_LENGTH 400

and

config of the PHY:

ble_gap_phys_t const phys =
{
.rx_phys = BLE_GAP_PHY_1MBPS,
.tx_phys = BLE_GAP_PHY_1MBPS,
};

has no impact since the delay of 4 ms is necessary.

The part of the code for on_hvx:

static void on_hvx(ble_timeandsound_c_t * p_ble_timeandsound_c, ble_evt_t const * p_ble_evt)
{

....

if(pcounter <= NumOfExpPackets)
{
u16counter++;
ble_tas_uart_senddata(p_ble_evt->evt.gattc_evt.params.hvx.data, m_ble_max_data_len, counter);
//NRF_LOG_INFO(" PACKET = %d", pcounter);
}else
{
u16counter = 0;
}

....

p_ble_timeandsound_c->evt_handler(p_ble_timeandsound_c, &ble_timeandsound_c_evt);

}

So there is no delay in on_hvx handle and m_ble_max_data_len = 244.

Regarding LOG_INFO or debug information; does it has any impact on UART speed?

RE: UART FIFO Flush: How to determine that UART TX is completed

ovrebekk — Wed, 31 Jul 2019 13:09:18 GMT

If I understand you correctly you spend 2.3 seconds transferring the entire image over Bluetooth, and then 1.0 seconds afterwards to send it over UART?

If so then you would probably realize the biggest savings if you are able to send the data over the UART during BLE activity, so that you don't have to wait until the end of the BLE transmission before starting UART TX. In this case you should be able to shave off almost the entire 1 second spent during UART activity, since it appears the BLE communication is slower than the UART communication in your example.

I realize you did the UART transfer after the BLE transfer to solve some other issue, but tackling this issue will probably be easier than to move everything to USB.

It is technically possible to configure the USB interface in the nRF52840, and handover information from the BLE stack to USB, but I doubt it will be significantly faster compared to the UART running at 1Mbaud.

Best regards
Torbjørn

RE: UART FIFO Flush: How to determine that UART TX is completed

Comodo — Tue, 30 Jul 2019 17:47:32 GMT

So our final conclusion: I can transfer 80KB of data from Android device->Nordic->UART->PC(Matlab) in around 4,4 seconds which is still too long (1000ms of sound data, 2300ms BLE 1Mbits, 1000ms UART). The bottleneck is the UART (1Mbits) ofcourse in case we use 2Mbits. My collegue asked me today why do I am not using the USBD interface for the data transfer. I checked out the examples and I am not sure if its possible to use it with Matlab directly. Have someone any experience with BLE->USB?

RE: UART FIFO Flush: How to determine that UART TX is completed

ovrebekk — Tue, 18 Jun 2019 06:52:48 GMT

Ok, just let me know when you have more questions.

RE: UART FIFO Flush: How to determine that UART TX is completed

Comodo — Mon, 17 Jun 2019 12:39:10 GMT

Thanks for the support. I will get back to you after I got some new results.

RE: UART FIFO Flush: How to determine that UART TX is completed

ovrebekk — Mon, 17 Jun 2019 12:08:49 GMT

Hi Hüseyin

[quote user="Comodo"]Why unsubscribe?[/quote]

A client would typically unsubscribe to data updates if the application controlling it doesn't need the data, or if it is unable to receive them (like in your case).

By being able to only subscribe to updates on the characteristics that are currently needed you can get a more power efficient system, since you are not wasting time and energy sending data that the client doesn't need.

[quote user="Comodo"]I am not responsible for the Android implementation, but it was wrong definition, I meant "acknowledge" that the BLE notification packet on nRF52 side have been received. But consumes time (packets between 200- 400).[/quote]

Maybe you are referring to sending an indication instead of a notification, which requires the client to send an explicit acknowledge on every received packet.

This will definitely slow down communication, and is not recommended if you are trying to optimize throughput and/or power consumption.

[quote user="Comodo"]Somehow singlebytes cause a traceble crash in nRF SW. [/quote]

Hm, that shouldn't really happen, but from an efficiency standpoint it is definitely better to send data in larger chunks. Otherwise you won't be able to take advantage of the easyDMA controller to seamlessly get data from the RAM during longer sequences.

Best regards
Torbjørn

RE: UART FIFO Flush: How to determine that UART TX is completed

Comodo — Fri, 14 Jun 2019 12:42:08 GMT

So first of all thx for the answer, I wasn't able the work on that project in the past 2 weeks. Now I am back on track again:

[quote userid="2116" url="~/f/nordic-q-a/47566/uart-fifo-flush-how-to-determine-that-uart-tx-is-completed/191131"]By sending data over the UART as heap I assume you mean you are forwarding a larger buffer to the UART driver, rather than just single bytes?[/quote]

Yes, as I mentioned before, at every notification 244Bytes will be received and if I try to send this over UART right after, I lose some BLE notifications and packets. So I have created a large buffer to process them later.

[quote userid="2116" url="~/f/nordic-q-a/47566/uart-fifo-flush-how-to-determine-that-uart-tx-is-completed/191131"]Also, the client on the nRF52 side should always be able to unsubscribe or subscribe to notification updates, doing otherwise would be in violation of the Bluetooth specification.[/quote]

Why unsubscribe?

[quote userid="2116" url="~/f/nordic-q-a/47566/uart-fifo-flush-how-to-determine-that-uart-tx-is-completed/191131"]I am not quite sure what you mean by the notification response. There is an onNotificationSent callback issued after the server on the Android side sends a notification, which you should check to ensure the notifications are successfully sent. [/quote]

I am not responsible for the Android implementation, but it was wrong definition, I meant "acknowledge" that the BLE notification packet on nRF52 side have been received. But consumes time (packets between 200- 400).

[quote userid="2116" url="~/f/nordic-q-a/47566/uart-fifo-flush-how-to-determine-that-uart-tx-is-completed/191131"]An alternative to controlling the data flow by enabling/disabling notifications is to use a separate characteristic to inform the server about how much buffer space you have on the nRF52 side. Then you can update this dynamically, and the Android app can make sure not to send any more data than the buffers in the nRF52 can handle. [/quote]

I can try this one. Good idea.

Somehow singlebytes cause a traceble crash in nRF SW.

RE: UART FIFO Flush: How to determine that UART TX is completed

ovrebekk — Wed, 05 Jun 2019 13:30:19 GMT

I am not quite sure what you mean by the notification response. There is an onNotificationSent callback issued after the server on the Android side sends a notification, which you should check to ensure the notifications are successfully sent.

Also, the client on the nRF52 side should always be able to unsubscribe or subscribe to notification updates, doing otherwise would be in violation of the Bluetooth specification.

An alternative to controlling the data flow by enabling/disabling notifications is to use a separate characteristic to inform the server about how much buffer space you have on the nRF52 side. Then you can update this dynamically, and the Android app can make sure not to send any more data than the buffers in the nRF52 can handle.

By sending data over the UART as heap I assume you mean you are forwarding a larger buffer to the UART driver, rather than just single bytes?

Best regards
Torbjørn

RE: UART FIFO Flush: How to determine that UART TX is completed

Comodo — Tue, 04 Jun 2019 11:58:20 GMT

Hi, at the moment we have different data sizes depending on setting 2KB to 190KB. We are using two Pixel 3 phones (BLE5 capable) to cross-check the implementation. My collegue who is implementing the android side mentioned that he can set MTU up to 512. Transfering data between two phones is very quicker (ca. 1000ms). The android BLE implementation has no notification response to keep the speed; so it is not possible to stop notification (2 ms per package). Right now I have a workaround, I store all RX BLE data in to a Buffer (which is limited to 90KB) and send it over uart as heaps, which takes 6 seconds.

I am not sure if your proposal works more faster or ever implementable (2ms between packages, it should be longer to react at right time).

Right now I am working on another workaround to speed up the RX-Buffer -> UART (by using bigger heaps, 4K-16K).

RE: UART FIFO Flush: How to determine that UART TX is completed

ovrebekk — Mon, 27 May 2019 14:27:19 GMT

Are you able to disable notifications on the characteristic temporarily to avoid the client on the Android side sending more data until the UART has had time to flush the buffers?

Then you could disable the notifications once you have received a certain amount of data, wait for the APP_UART_TX_EMPTY event to occur, and enable notifications again.

This probably requires the addition of some code in the app to verify if notifications are enabled before uploading data to the Bluetooth stack.

Best regards
Torbjørn

RE: UART FIFO Flush: How to determine that UART TX is completed

Comodo — Wed, 22 May 2019 18:06:16 GMT

Hi still got this problem

I fill the Tx buffer and wait until it has been sent but that doesnt happen it stacks there.

RE: UART FIFO Flush: How to determine that UART TX is completed

Andy — Tue, 21 May 2019 13:27:09 GMT

Nevermind, just saw the API.

RE: UART FIFO Flush: How to determine that UART TX is completed

Andy — Tue, 21 May 2019 13:24:43 GMT

Ah, if you're using the app_uart library then yes. But then I don't understand your problem with flushing the FIFO. I've never used that library, but I assume it flushes the buffers by itself.

RE: UART FIFO Flush: How to determine that UART TX is completed

Comodo — Tue, 21 May 2019 13:17:13 GMT

You mean "APP_UART_TX_EMPTY" ?

RE: UART FIFO Flush: How to determine that UART TX is completed

Andy — Tue, 21 May 2019 12:04:50 GMT

You should get NRFX_UART_EVT_TX_DONE on the event handler.