UART FIFO Flush: How to determine that UART TX is completed

[Comodo]

Hi everyone,

in our project we transmit from an android app to Nordic BLE big amount of data (up to 180.000 bytes). I do it by using hvx notifications (MTU 247) and each time I get the notification I try to transmit the data over UART. But we have realized that the uart TX takes too much time and causes packet loss. So I have decieded to store the BLE data and send it after the BLE notifications are completed. Now I face the problem that the UART FIFO (2048) will be full and causes system fault. I need to find out a way where I can determine when the TX transmition is completed and I can flush the TX buffer. I ve searched the uart driver but could find the right function for that. Can anyone help me regarding that?

Thx and Regards

[Andy]

You should get NRFX_UART_EVT_TX_DONE on the event handler.

[Comodo]

You mean "APP_UART_TX_EMPTY" ?

[Comodo]

Hi still got this problem

I fill the Tx buffer and wait until it has been sent but that doesnt happen it stacks there. 

[ovrebekk]

Hi 

Are you able to disable notifications on the characteristic temporarily to avoid the client on the Android side sending more data until the UART has had time to flush the buffers?

Then you could disable the notifications once you have received a certain amount of data, wait for the APP_UART_TX_EMPTY event to occur, and enable notifications again. 

This probably requires the addition of some code in the app to verify if notifications are enabled before uploading data to the Bluetooth stack. 

Best regards
Torbjørn

[Comodo]

Hi, at the moment we have different data sizes depending on setting 2KB to 190KB. We are using two Pixel 3 phones (BLE5 capable) to cross-check the implementation. My collegue who is implementing the android side mentioned that he can set MTU up to 512.  Transfering data between two phones is very quicker (ca. 1000ms). The android BLE implementation has no notification response to keep the speed; so it is not possible to stop notification (2 ms per package). Right now I have a workaround, I store all RX BLE data in to a Buffer (which is limited to 90KB) and send it over uart as heaps, which takes 6 seconds. 

I am not sure if your proposal works more faster or ever implementable (2ms between packages, it should be longer to react at right time). 

Right now I am working on another workaround to speed up the RX-Buffer -> UART (by using bigger heaps, 4K-16K).

[ovrebekk]

Hi

I am not quite sure what you mean by the notification response. There is an onNotificationSent callback issued after the server on the Android side sends a notification, which you should check to ensure the notifications are successfully sent. 

Also, the client on the nRF52 side should always be able to unsubscribe or subscribe to notification updates, doing otherwise would be in violation of the Bluetooth specification. 

An alternative to controlling the data flow by enabling/disabling notifications is to use a separate characteristic to inform the server about how much buffer space you have on the nRF52 side. Then you can update this dynamically, and the Android app can make sure not to send any more data than the buffers in the nRF52 can handle. 

By sending data over the UART as heap I assume you mean you are forwarding a larger buffer to the UART driver, rather than just single bytes?

Best regards
Torbjørn

[Comodo]

So first of all thx for the answer, I wasn't able the work on that project in the past 2 weeks. Now I am back on track again:

ovrebekk said:

By sending data over the UART as heap I assume you mean you are forwarding a larger buffer to the UART driver, rather than just single bytes?

Yes, as I mentioned before, at every notification 244Bytes will be received and if I try to send this over UART right after, I lose some BLE notifications and packets. So I have created a large buffer to process them later.

ovrebekk said:

Also, the client on the nRF52 side should always be able to unsubscribe or subscribe to notification updates, doing otherwise would be in violation of the Bluetooth specification.

Why unsubscribe?

ovrebekk said:

I am not quite sure what you mean by the notification response. There is an onNotificationSent callback issued after the server on the Android side sends a notification, which you should check to ensure the notifications are successfully sent. 

I am not responsible for the Android implementation, but it was wrong definition, I meant "acknowledge" that the BLE notification packet on nRF52 side have been received. But consumes time (packets between 200- 400).

ovrebekk said:

An alternative to controlling the data flow by enabling/disabling notifications is to use a separate characteristic to inform the server about how much buffer space you have on the nRF52 side. Then you can update this dynamically, and the Android app can make sure not to send any more data than the buffers in the nRF52 can handle. 

I can try this one. Good idea.

ovrebekk said:

By sending data over the UART as heap I assume you mean you are forwarding a larger buffer to the UART driver, rather than just single bytes?

Somehow singlebytes cause a traceble crash in nRF SW. 

[ovrebekk]

Hi Hüseyin 

Comodo said:

Why unsubscribe?

 A client would typically unsubscribe to data updates if the application controlling it doesn't need the data, or if it is unable to receive them (like in your case). 

By being able to only subscribe to updates on the characteristics that are currently needed you can get a more power efficient system, since you are not wasting time and energy sending data that the client doesn't need. 

Comodo said:

I am not responsible for the Android implementation, but it was wrong definition, I meant "acknowledge" that the BLE notification packet on nRF52 side have been received. But consumes time (packets between 200- 400).

 Maybe you are referring to sending an indication instead of a notification, which requires the client to send an explicit acknowledge on every received packet. 

This will definitely slow down communication, and is not recommended if you are trying to optimize throughput and/or power consumption.  

Comodo said:

Somehow singlebytes cause a traceble crash in nRF SW. 

Hm, that shouldn't really happen, but from an efficiency standpoint it is definitely better to send data in larger chunks. Otherwise you won't be able to take advantage of the easyDMA controller to seamlessly get data from the RAM during longer sequences. 

Best regards
Torbjørn

[Comodo]

Thanks for the support. I will get back to you after I got some new results.

[ovrebekk]

Ok, just let me know when you have more questions. 

[Comodo]

So our final conclusion: I can transfer 80KB of data from Android device->Nordic->UART->PC(Matlab) in around 4,4 seconds which is still too long (1000ms of sound data, 2300ms BLE 1Mbits, 1000ms UART). The bottleneck is the UART (1Mbits) ofcourse in case we use 2Mbits. My collegue asked me today why do I am not using the USBD interface for the data transfer. I checked out the examples and I am not sure if its possible to use it with Matlab directly. Have someone any experience with BLE->USB?

[ovrebekk]

Hi

If I understand you correctly you spend 2.3 seconds transferring the entire image over Bluetooth, and then 1.0 seconds afterwards to send it over UART?

If so then you would probably realize the biggest savings if you are able to send the data over the UART during BLE activity, so that you don't have to wait until the end of the BLE transmission before starting UART TX. In this case you should be able to shave off almost the entire 1 second spent during UART activity, since it appears the BLE communication is slower than the UART communication in your example. 

I realize you did the UART transfer after the BLE transfer to solve some other issue, but tackling this issue will probably be easier than to move everything to USB. 

It is technically possible to configure the USB interface in the nRF52840, and handover information from the BLE stack to USB, but I doubt it will be significantly faster compared to the UART running at 1Mbaud. 

Best regards
Torbjørn