UART package length problems nRF VS Code SDK2.0.0

Hi everyone,

at the moment we are trying to upgrade our nRF9160 software form the SDK v1.8.0 to the newer version SDK2.0.0.
The main issue of the migration is our UART implementation. In the last version the UART implementation worked like a charm and we were able to transfer large packages without any issues, but with the new version we are having difficulties with the larger packages.

We are sending certificates and keys to the nRF9160 to be able to establish a secure MQTT connection. The length of these certificates and keys vary form approximately 1200 to 1800 bytes. Smaller packages all approximately 1 to 500 bytes are valid transactions and show a "/r/n" at their end, but with the larger data transfers the received length does not correspond with the original package length. For example a certificate command is 1736 bytes long and the received length is 1721.

In the old SDK v1.8.0 we got another transmission with the rest of the package, but in the new SDK version the rest of the transmission is just missing.

The following code is our UART implementation. After every successfully received and processed command the function "uart_reinit" is triggered to be able to receive again. 

#include <zephyr/zephyr.h>
#include <zephyr/drivers/uart.h>
#include <zephyr/logging/log.h>
#include <stdlib.h>
#include <general_definitions.h>
#include <uart_communication.h>
#include <led_config.h>
#include <device_uart_interface.h>
#include <scheduler.h>
#include <mqtt.h>

static K_MEM_SLAB_DEFINE(uart_slab, BUF_SIZE, 2, 4);
uint8_t *buf;
char uart_receive_buf[UART_RECEIVE_BUFFER_SIZE];
uint32_t uart_receive_index;
uint32_t uart_last_receive_time;
uint32_t previous_uart_length = 0;

LOG_MODULE_REGISTER(uart_communication,LOG_LEVEL_INF);

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// uart callback
void uart_callback(const struct device *dev,
                   struct uart_event *evt,
                   void *user_data)
{
	struct device *uart = user_data;
	int err;
    uint32_t first_bracket_offset = 0;

	switch (evt->type) {
	case UART_TX_DONE:
		LOG_INF("UART_TX_DONE");
		break;

	case UART_TX_ABORTED:
		LOG_INF("UART_TX_ABORTED");
		break;

	case UART_RX_RDY:
            // Save it to uart receive buffer
            LOG_INF("UART_RX_RDY");
            //LOG_INF("UART: Content buffer address: 0x%x", evt->data.rx.buf);
            //LOG_INF("UART: Content buffer: %s", evt->data.rx.buf);
            //LOG_INF("UART: Offset: %d", evt->data.rx.offset);
            //LOG_INF("UART: Length: %d", evt->data.rx.len);
            
            
            if ((evt->data.rx.buf[previous_uart_length + evt->data.rx.len - 2] == '\r') &&
                (evt->data.rx.buf[previous_uart_length + evt->data.rx.len - 1] == '\n')) 
            {
                if(k_sem_take(&uart_sem, K_NO_WAIT) == 0)
                {
                    uart_rx_disable(uart);

                    while(first_bracket_offset < 4096 && evt->data.rx.buf[previous_uart_length + first_bracket_offset] != 0x7B)
                    {
                        first_bracket_offset++;
                    }
                    if(evt->data.rx.buf[previous_uart_length + first_bracket_offset] == 0x7B && evt->data.rx.len > 10)
                    {
                        // LOG_INF("uart_receive_buf with saftey_index: %s", evt->data.rx.buf);
                        // memcpy(uart_receive_buf, evt->data.rx.buf + first_bracket_offset, evt->data.rx.len);
                        memcpy(uart_receive_buf, evt->data.rx.buf, evt->data.rx.len + previous_uart_length);
                        
                        scheduler_dip_parse.data = uart_receive_buf;
                        scheduler_dip_parse.cmd_len = evt->data.rx.len;
                        previous_uart_length = 0;

                        memset(mqtt_recv_operation,0x0,MQTT_RECV_OPERATION_LENGTH);
                        k_work_submit_to_queue(&oem_workqueue, &scheduler_dip_parse.work);
                    }
                    else
                    {
                        uart_rx_enable(uart, buf, BUF_SIZE, 100);
                        k_sem_give(&uart_sem);
                    }
                }
            }
            else
            {
                previous_uart_length = previous_uart_length + evt->data.rx.len;
            }
                
		break;

	case UART_RX_BUF_REQUEST:
	{
        LOG_INF("UART_RX_BUF_REQUEST");
		uint8_t *buf;

		err = k_mem_slab_alloc(&uart_slab, (void **)&buf, K_NO_WAIT);
		__ASSERT(err == 0, "Failed to allocate slab");

		err = uart_rx_buf_rsp(uart, buf, BUF_SIZE);
		__ASSERT(err == 0, "Failed to provide new buffer");

		break;
	}

	case UART_RX_BUF_RELEASED:
        LOG_INF("UART_RX_BUF_RELEASED");
		k_mem_slab_free(&uart_slab, (void **)&evt->data.rx_buf.buf);
		break;

	case UART_RX_DISABLED:
                LOG_INF("UART_RX_DISABLED");
		break;

	case UART_RX_STOPPED:
                LOG_INF("UART_RX_STOPPED");
		break;
	}
}

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// init uart
void uart_init(void)
{
	int err = 0;

    uart = device_get_binding("UART_0");
	__ASSERT(uart, "Failed to get the device");

	err = k_mem_slab_alloc(&uart_slab, (void **)&buf, K_NO_WAIT);
	__ASSERT(err == 0, "Failed to alloc slab");

	err = uart_callback_set(uart, uart_callback, (void *)uart);   // set callback
	__ASSERT(err == 0, "Failed to set callback");

	err = uart_rx_enable(uart, buf, BUF_SIZE, 100);    // enable receiving
	__ASSERT(err == 0, "Failed to enable RX"); 

    previous_uart_length = 0;

    if(err == 0)
    {
        uart_initialization_status = true;
    }
}

void uart_reinit(void)
{
    int err = 0;

    memset(uart_receive_buf,0x0,UART_RECEIVE_BUFFER_SIZE);
    uart_receive_index = 0;

    k_sem_give(&uart_sem);
    
    previous_uart_length = 0;

    if(!mqtt_output)
    {
        err = uart_rx_enable(uart, buf, BUF_SIZE, 100);    // enable receiving
        __ASSERT(err == 0, "Failed to enable RX");
    }
    k_sleep(K_MSEC(10)); 
}
 

I looked through the different SDK release notes from 1.8.0 up to 2.0.0. I did not find any major changes in the UART section and also my attempts to fix the problem did not work. I hope somebody is able to help me with this problem.

Best regards,

Andreas

  • For example a certificate command is 1736 bytes long and the received length is 1721.

    Can you show me how your application is configuring the UART? Is it using hardware flow control? This kind of symptoms seems very likely when there is no hardware flow control and when the receive buffers overflow, the application most likely does not have a handshake mechanism to ask the peer to resend the overflowed bytes.

  • Hi Susheel Nugru,

    Thanks for your help.
    Yes you are right we are not using hardware flow control. Due to pin restrictions we probably also will not be able to implement hardware flow control in our normal application. Our UART Configuration looks like this:

    # Uart Config
    CONFIG_SERIAL=y
    CONFIG_NRFX_UARTE0=y
    CONFIG_UART_ASYNC_API=y

    I will implement the hardware flow control in a new test application to verify if this would solve our problem. The variable "BUF_SIZE" in our UART communication has a value of 4096.

  • Andreas, as you may already know, sometimes it is not just enough to have a large buffer size in the application buffers. The bottle necks could be  post processing of the received data that might be slower than the speed with which the data is coming into RX buffers.

    One easy way to test if this is the case is to decrease the uart speed (baudrate) on both ends and see if you the do not miss any data. If you do not miss any data on slower baudrate, then we can confirm our theory.

  • Hi,

    the change in baudrate from 115200 to 9600 solved the problem with the missing bytes of the certificates. Thanks seems your theory was right.

    Unfortunately another problem has been created with the change. 60% of the time the receiving works fine. The only difference is that now each byte is recognized via the UART_RX_RDY event and the command is assembled byte by byte.

    This in its own is no problem, but the other 30% of the time the UART_RX_RDY event reports that it has only received 1 byte, but actually the whole message was already received and is present in the
    "evt->data.rx.buf".

    I tried to go down again with the baudrate, but this also did not solve this problem. DO you have any suggestions on how to takle this particular problem?

    Best regards,

    Andreas

  • AscherA said:
    but the other 30% of the time the UART_RX_RDY event reports that it has only received 1 byte, but actually the whole message was already received and is present in the
    "evt->data.rx.buf".

    Hmm, interesting, Do you think that the UART driver is only reporting UART_RX_RDY event with 1 byte size buffer and not reporting UART_RX_RDY event for the other bytes? 

    or 

    Is it possible that the UART driver is using the same buffer for received data to generate future event aswell and your application is just sniffing the evt->data.rx.buf a bit early to know that the meaninful data is already there?

Related