UART max 80 bytes on receive is possible

RE: UART max 80 bytes on receive is possible

mbozic — Fri, 19 Feb 2021 12:21:09 GMT

Hi Edvin, Henry,

I've managed to completely resolve the issue.

I've confirmed that I can receive 255 bytes in one burst on 1Mega baud with flow control. Without flow control around 250000 is pretty much stable so our only option would be to lower the baud rate for regular UART.

Libuarte has no problems with 255 bytes at 576000, so I'm sticking with it! Many thanks to one of your engineers who provided the implementation on some other thread.

Biggest issue I had was RTT not printing all bytes it received with NRF_LOG_HEXDUMP due small NRF_LOG_MSGPOOL_ELEMENT_COUNT, and that took me to wrong direction. Lesson learned! :)

No overrun errors with libuarte! No overrun errors with flow control up to 1Mega baud. Using my own reading from uart, implemented years ago.

Thanks for support guys!

RE: UART max 80 bytes on receive is possible

Edvin — Fri, 19 Feb 2021 11:48:32 GMT

So how is it going? Do you still get the overrun error? If so, I believe you are not handling the data fast enough in your event handler. Is it still the default uart event handler from the ble_app_uart? Or did you change it?

RE: UART max 80 bytes on receive is possible

mbozic — Fri, 19 Feb 2021 08:41:45 GMT

Hi Henry,

don't worry, I am forcing the 0xFF in the packet so I can easily find the end of the packet to validate it. :) It's easier to spot 0xFF and then 0x00 while sending a lot of bytes like 250+.

Edit:

Don't worry about COBS, it's the code we had for years, its tested and verified. Mine is N+3 currently because I add additional zero at the beginning of packet too for testing. You can modify COBS to your needs as long as encode follows decode and I'm not even doing a decode here, I'm just sending plain data. I could test Rx with any data since my nRF receiver does not do decoding it is only receiving data. So my problem lies in receiving not decoding.

Thanks for caring! :)

RE: UART max 80 bytes on receive is possible

Henry_Chou — Fri, 19 Feb 2021 08:35:45 GMT

Hey! It's me. mbozic:

I have a question. It's no matter about the UART RX issue.

I think that you have better to check your cobs encode. Because your encode data the last two bytes are always ff 00.

But if your original data contents are not zeo the COBS encode the last second byte will not be ff. Because ff means after 255 bytes is zero. But the later byte is zero. It's quite strange. Isn't it?

COBS encode algorithm is n to n+2. But yours is n to n+3.

RE: UART max 80 bytes on receive is possible

mbozic — Fri, 19 Feb 2021 08:01:54 GMT

Okay, first I've checked the hardware, for the 10th time, could not found errors.
Then I compiled and run ble_app_uart, getting the same results!
Then I went ballistic...Placed logging all over the place and eventually I figured all data are received, only NRF_LOG_HEXDUMP was printing first 80 bytes!
Changed NRF_LOG_MSGPOOL_ELEMENT_COUNT from 8 to 64 and everything prints! I can receive up to 256 bytes with no problems at baudrates of 115200 and 250000. Bloody hell...

After that it was time to see why UART gave comm errors (overrun) at 576k baudrate, so I connected oscilloscope and another FTDI to se what I am transmitting and receiving. Transmitted data is valid, so is the received data. But, nRF does not receive more than 33 bytes at 576k.

Not sure if image will be visible correctly but I always get overrun error on UART if I send more than 33 bytes.

I would assume this is because I don't use flow control?

I will try with flow control just to verify the issue and I will try libuarte again to se if that library helps. Main problem is we can't have flow control. So I would guess our only option would be to lower the baudrate.

RE: UART max 80 bytes on receive is possible

mbozic — Thu, 18 Feb 2021 10:45:50 GMT

Hi Edvin,

process_uart() function is called in main while loop.

// Enter main loop.
while (1)
{
    NRF_LOG_PROCESS();
    process_uart();
}

I'll test with ble_app_uart.

I'm also evaluating our custom hardware where this is being run. I also doubt as Henry it's something in nRF SW or HW, it's probably something with our side of HW. I'll know more in few days.

Thanks for info!

RE: UART max 80 bytes on receive is possible

Edvin — Thu, 18 Feb 2021 10:31:10 GMT

Hello,

Please note that we have an "insert" -> "code" for inserting code snippets. It is easier to read if the code snippets are long, because it maintains the formatting.

My concern is:

        case APP_UART_DATA_READY:
        {

        } break;

which does nothing.In itself, this should be fine as long as your fifo buffer is large enough (larger than 80 bytes), but I don't know how you set up your project.

What I believe is confusing you is the app_uart_fifo and the uart peripheral driver. These are not the same thing. The app_uart_fifo is the library that pulls data from the uart peripheral. Do you use app_uart_fifo.c in your project? If so, look at what is generating the APP_UART_DATA_READY event inside app_uart_fifo.c:

static void uart_event_handler(nrf_drv_uart_event_t * p_event, void* p_context)
{
    app_uart_evt_t app_uart_event;
    uint32_t err_code;

    switch (p_event->type)
    {
        case NRF_DRV_UART_EVT_RX_DONE:
            // If 0, then this is a RXTO event with no new bytes.
            if(p_event->data.rxtx.bytes == 0)
            {
               // A new start RX is needed to continue to receive data
               (void)nrf_drv_uart_rx(&app_uart_inst, rx_buffer, 1);
               break;
            }

            // Write received byte to FIFO.
            err_code = app_fifo_put(&m_rx_fifo, p_event->data.rxtx.p_data[0]);
            if (err_code != NRF_SUCCESS)
            {
                app_uart_event.evt_type          = APP_UART_FIFO_ERROR;
                app_uart_event.data.error_code   = err_code;
                m_event_handler(&app_uart_event);
            }
            // Notify that there are data available.
            else if (FIFO_LENGTH(m_rx_fifo) != 0)
            {
                app_uart_event.evt_type = APP_UART_DATA_READY;
                m_event_handler(&app_uart_event);
            }

            // Start new RX if size in buffer.
            if (FIFO_LENGTH(m_rx_fifo) <= m_rx_fifo.buf_size_mask)
            {
                (void)nrf_drv_uart_rx(&app_uart_inst, rx_buffer, 1);
            }
            else
            {
                // Overflow in RX FIFO.
                m_rx_ovf = true;
            }

            break;

        case NRF_DRV_UART_EVT_ERROR:
            app_uart_event.evt_type                 = APP_UART_COMMUNICATION_ERROR;
            app_uart_event.data.error_communication = p_event->data.error.error_mask;
            (void)nrf_drv_uart_rx(&app_uart_inst, rx_buffer, 1);
            m_event_handler(&app_uart_event);
            break;

        case NRF_DRV_UART_EVT_TX_DONE:
            // Get next byte from FIFO.
            if (app_fifo_get(&m_tx_fifo, tx_buffer) == NRF_SUCCESS)
            {
                (void)nrf_drv_uart_tx(&app_uart_inst, tx_buffer, 1);
            }
            else
            {
                // Last byte from FIFO transmitted, notify the application.
                app_uart_event.evt_type = APP_UART_TX_EMPTY;
                m_event_handler(&app_uart_event);
            }
            break;

        default:
            break;
    }
}

Does that function look exactly like this in your case? And is it being used?

And for context, you say that the 81st byte is missing. Where do you read your bytes?

The function process_uart() with the empty APP_UART_DATA_READY: event handler. Where is it called from?

Just a shot in the dark: Have you tried the ble_app_uart example from the SDK? Do you see the same behavior there if you don't modify anything?

BR,

Edvin

RE: UART max 80 bytes on receive is possible

mbozic — Thu, 18 Feb 2021 09:35:15 GMT

"If it's the COBS encode, I think it will never be zero until end of fifo."

Yes, it will never be zero until end of packet(not fifo!), but packet can be 40 bytes and everything works, go to 60 bytes, everything works, give it 80 bytes, works. Now give it 81 bytes, and 81st is lost!

Yes, it's a overrun issue but why? I can see all the data in oscilloscope, I'm using FTDI cable to send data, everything is there, valid. FIFO size is 256 bytes for both Rx and Tx.

Still looking...

RE: UART max 80 bytes on receive is possible

Henry_Chou — Thu, 18 Feb 2021 09:10:24 GMT

If it's the COBS encode, I think it will never be zero until end of fifo. I agree the suggestion of Edvin.

You may check the 0x480 (ERRORSRC) in your handle function. Maybe it's the overrun issue (TX signal issue) in this case.

RE: UART max 80 bytes on receive is possible

mbozic — Thu, 18 Feb 2021 08:55:34 GMT

This is COBS, you put N bytes in Encode and get out N+3 bytes, last byte being 0 which marks the end of the packet. You can adapt COBS to your needs but 0 always marks start and/or end of the packet, in my case it marks the end.

I've checked encoding function, I've checked the data in debug and with the oscilloscope and it's valid. Wasted few days on this actually.

I will also check apply_old_config, maybe I've missed something!

RE: UART max 80 bytes on receive is possible

Henry_Chou — Thu, 18 Feb 2021 08:39:43 GMT

Okay~mbozic

I don't think that the bug is the HW/FW bug of SDK16 or SOC. Because I had done UART RX more than 128 bytes before(115200bps). I wonder that the possible case is the RX data format from your side. From your uart_process() function in while loop, it get the uart rx data fifo untile the data is zero.

            if (uartWorker.byteIndex < UART_BUFFER_SIZE)
            {
                uartWorker.buffer[uartWorker.byteIndex] = uartWorker.byte;
                uartWorker.byteIndex++;
            }

            if (uartWorker.byte == 0)
            {

Here. But if the 81th byte is zero and over 82 bytes are valid. Your data fifo will be 80 bytes only. So another way, you may check the TX data from your test pattern.

RE: UART max 80 bytes on receive is possible

mbozic — Thu, 18 Feb 2021 08:28:43 GMT

Hi,

uart_process() function only transfers receiving the data into while loop. According to my testing handling it in while loop makes the receiving data more stable, basically without errors in receiving. At least till now...

I already tried handling it similar like you mentioned, behaves the same. I will try again, perhaps I missed something the first time I tried.

I believe there is an error on my side, obviously, but what confuses me that I also get the same behavior using libuarte and with libuarte I'm handling the data in libuarte event handler (uart_event_handler(void * context, nrf_libuarte_async_evt_t * p_evt)) with same results.

So any way I go I get to this 80 bytes max in a single burst which makes this UART unusable for our use case. I wish I defined 80 somewhere but I didn't. Very weird behavior.

RE: UART max 80 bytes on receive is possible

Henry_Chou — Thu, 18 Feb 2021 08:02:08 GMT

What's your process_uart() purpose?

Why don't you receive the data fifo from uart_event_handle()?

It's must be something wrong, when you get uart data in process_uart() .

Normally it's handle in event (interrupt).

You may try the following example.

void uart_event_handle(app_uart_evt_t * p_event)
{
static uint8_t data_array[256];
static uint8_t uartWorker.byteIndex = 0;

switch (p_event->evt_type)
{
/**@snippet [Handling data from UART] */
case APP_UART_DATA_READY:
//UNUSED_VARIABLE(app_uart_get(&data_array[uartWorker.byteIndex]));
app_uart_get(&data_array[uartWorker.byteIndex]);
uartWorker.byteIndex++;

if ((data_array[uartWorker.byteIndex - 1] == '\n') || (uartWorker.byteIndex >=UART_BUFFER_SIZE ))
{
data_array[uartWorker.byteIndex]=0;
uartWorker.byteIndex = 0;
memcpy(uartWorker.buffer,data_array,sizeof(data_array));
}
break;
/**@snippet [Handling data from UART] */
case APP_UART_COMMUNICATION_ERROR:
APP_ERROR_HANDLER(p_event->data.error_communication);
break;

case APP_UART_FIFO_ERROR:
APP_ERROR_HANDLER(p_event->data.error_code);
break;

default:
break;
}
}

Good luck!

RE: UART max 80 bytes on receive is possible

mbozic — Thu, 18 Feb 2021 06:39:40 GMT

Hi Edvin,

thanks for replaying.

This is the uart init function:

void UART_Initialize(void)
{
    uint32_t err_code;
    app_uart_comm_params_t const comm_params =
    {
        .rx_pin_no    = UART_RX_PIN,
        .tx_pin_no    = UART_TX_PIN,
        .flow_control = APP_UART_FLOW_CONTROL_DISABLED,
        .use_parity   = false,
        .baud_rate    = UART_BAUDRATE_BAUDRATE_Baud250000
    };

    APP_UART_FIFO_INIT(&comm_params, UART_BUFFER_SIZE, UART_BUFFER_SIZE, uart_event_handle, UART_PRIORITY, err_code);
    APP_ERROR_CHECK(err_code);
}

UART_BUFFER_SIZE = 256

UART_PRIORITY = APP_IRQ_PRIORITY_HIGHEST // Tried every possible option

#define UART_EASY_DMA_SUPPORT 1

#define UART_LEGACY_SUPPORT 0

This is how I process Rx data, this function is called in main.c, in the main while(1) loop(UART_BUFFER_SIZE = 512):

static void process_uart(void)
{
    if (!uartWorker.enabled)
    {
        return;
    }

    if (uartWorker.bleBufferAvailable)
    {
        if (app_uart_get(&uartWorker.byte) == NRF_SUCCESS)
        {
            BOARD_LedRxOn();
            //NRF_LOG_INFO("Rx: %d", uartWorker.byte);

            if (uartWorker.byteIndex < UART_BUFFER_SIZE)
            {
                uartWorker.buffer[uartWorker.byteIndex] = uartWorker.byte;
                uartWorker.byteIndex++;
            }

            if (uartWorker.byte == 0)
            {

#ifdef PRINT_UART_COMM
                NRF_LOG_INFO("-> BLE(1): %d", uartWorker.byteIndex);
                NRF_LOG_HEXDUMP_INFO(uartWorker.buffer, uartWorker.byteIndex);
#endif
                if (dongleInfo.verified)
                {
                    uartWorker.bleBufferAvailable = ble_rws_try_to_send(&m_rws, uartWorker.buffer, &uartWorker.byteIndex);
                    if (uartWorker.bleBufferAvailable)
                    {
                        NRF_LOG_INFO("-> BLE(1) SENT");
                        uartWorker.byteIndex = 0;
                    }
                }
                else
                {
                    uartWorker.byteIndex = 0;
                }
            }
        }
    }

    if (uartWorker.txComplete)
    {

#ifdef PRINT_UART_COMM
        NRF_LOG_INFO("-> BLE(2)");
        NRF_LOG_HEXDUMP_INFO(uartWorker.buffer, uartWorker.byteIndex);
#endif

        if (dongleInfo.verified)
        {
            uartWorker.txComplete = 0;
            uartWorker.bleBufferAvailable = ble_rws_try_to_send(&m_rws, uartWorker.buffer, &uartWorker.byteIndex);
            if (uartWorker.bleBufferAvailable)
            {
                NRF_LOG_INFO("-> BLE(2) SENT");
                uartWorker.byteIndex = 0;
            }
        }
    }
}

I don't use NRF_DRV_UART_ET_RX_DONE. My uart event handling function does not do much:

void uart_event_handle(app_uart_evt_t * p_event)
{
    switch (p_event->evt_type)
    {
        case APP_UART_DATA_READY:
        {

        } break;

        case APP_UART_COMMUNICATION_ERROR:
        {
            NRF_LOG_ERROR("APP_UART_COMMUNICATION_ERROR");// | Code: 0x%x", p_event->data.error_code);
            uartWorker.byteIndex = 0;
        } break;

        case APP_UART_FIFO_ERROR:
        {
            NRF_LOG_ERROR("APP_UART_FIFO_ERROR");
            uartWorker.byteIndex = 0;
        } break;

        default:
        {

        } break;
    }
}

Pretty much no matter how I configure the UART it's the same behaviour.

BAUDRATE: 115200 & 250000 -> Maximum 80 bytes is received correctly and printed in:

NRF_LOG_INFO("-> BLE(1): %d", uartWorker.byteIndex);

NRF_LOG_HEXDUMP_INFO(uartWorker.buffer, uartWorker.byteIndex);

If I send 81 bytes, 81st is lost, send 82, last 2 are lost.

For baudrate 576000 this is 34. So if I send 36 bytes, I loose last 2.

Please do take in account I've also implemented libuarte, the only difference is that now I can receive 80 bytes on baudrate of 576000.

You should be able to get this behavior from default ble_app_uart example since my UART implementation is the same as it is there. I can always send you my entire project.

This is the code we're using for a looong time, problem occurred when our custom device sent more than 34 bytes on UART and nRF didn't get it. I've developed a small UART app for windows so now we're testing with that.

Please try with ble_app_uart, if there are no problems I will send you my entire project. I'm still looking for an error in my project and debugging, but I can't find nothing obvious especially since libuarte is also behaving the same.

Thanks!

RE: UART max 80 bytes on receive is possible

Edvin — Wed, 17 Feb 2021 14:46:24 GMT

[quote user=""]Is 80 bytes some theoretical limit or is it defined somewhere in the sdk?[/quote]

No. It shouldn't be. As you may already have checked, the APP_UART_COMMUNICATION_ERROR with code 1 means OVERRUN:

(screenshot from infocenter)

How have you set up your UART? Do you have an event handler that handles the incoming bytes? Do you use the app_uart_fifo.c? If so, do you call nrf_drv_uart_rx() in the NRF_DRV_UART_ET_RX_DONE event?

Remember that the UART FIFO is a separate buffer, reading from the UART peripheral.

Is there some way for me to reproduce this on an nRF52DK? How do you send your data?

BR,
Edvin

RE: UART max 80 bytes on receive is possible

mbozic — Tue, 16 Feb 2021 14:07:15 GMT

Using nRF52832 and SDK16. Edited!