Help with Ring Buffer Implementation

[Luis Santos]

Hello Nordic Support Team,

I am trying to implement a ring buffer of structs.

Even though my consumer runs much more frequently than my producer, the buffer becomes full just a few moments after starting my code.

I suspect the issue might be related to data alignment, but I am unable to identify the problem.

Could someone please help me with this issue?

Thank you in advance.

typedef struct
{
    uint16_t addr;
    uint8_t valid_addr;
    filterSrvSensorType_t analyzing_type;
    emaFilterContext_t ema_context;
    temaFilterContext_t tema_context;
    kalmanFilterContext_t kalman_context;
    filterSrvSensorContext_t ble_sensor_context;
    filterSrvSensorContext_t lora_sensor_context;
} filterSrvContext_t;

#define FILTER_SRV_REFRESH_INTERVAL 50

#define FILTER_SRV_RING_BUFFER_AMOUNT 64 // Number of filterSrvContext_t
#define FILTER_SRV_BUFFER_SIZE_BYTES (FILTER_SRV_RING_BUFFER_AMOUNT * ((sizeof(filterSrvContext_t) + 3) / 4))

struct ring_buf filterSrvRingBuffer;
static uint8_t filterSrvBufferStorage[FILTER_SRV_BUFFER_SIZE_BYTES];

filterSrvContext_t filterSrvContext[FILTER_SRV_MAX_CONTEXTS];

uint8_t filterSrvMsg[I2C_SRV_TX_BUF_LEN - 5] = {0};
uint8_t filterSrvMsgSize = 0;

uint8_t filterSrvProducerItem(filterSrvContext_t *filter_context)
{
    __aligned(4) static uint32_t tmp_buf_static[(sizeof(filterSrvContext_t) + 3) / 4];
    memcpy(tmp_buf_static, filter_context, sizeof(filterSrvContext_t));

    int ret = ring_buf_item_put(&filterSrvRingBuffer,
                                filter_context->analyzing_type,
                                0,
                                tmp_buf_static,
                                (sizeof(filterSrvContext_t) + 3) / 4);

     LOG_INF("Producing %s Distance: %.2f",
             (filter_context->analyzing_type == FILTER_SRV_SENSOR_TYPE_BLE) ? "BLE " : "LORA",
             (filter_context->analyzing_type == FILTER_SRV_SENSOR_TYPE_BLE) ? filter_context->ble_sensor_context.distance_value
                                                                            : filter_context->lora_sensor_context.distance_value);

    if (ret != 0)
    {
        LOG_ERR("Ring buffer is full");
        return FILTER_SRV_RING_IS_FULL;
    }
    else
    {
        return FILTER_SRV_RING_IS_NOT_FULL;
    }
}

uint8_t filterSrvConsumerItem(filterSrvContext_t *filter_context)
{
    uint16_t type;
    uint8_t value;
    uint8_t size32 = (sizeof(filterSrvContext_t) + 3) / 4;
    uint32_t tmp_buf[size32];                             

    int ret = ring_buf_item_get(&filterSrvRingBuffer,
                                &type,
                                &value,
                                tmp_buf,
                                &size32);

    if (ret == 0)
    {
        memcpy(filter_context, tmp_buf, sizeof(filterSrvContext_t));

         LOG_INF("Consuming %s Distance: %.2f",
                 (filter_context->analyzing_type == FILTER_SRV_SENSOR_TYPE_BLE) ? "BLE " : "LORA",
                 (filter_context->analyzing_type == FILTER_SRV_SENSOR_TYPE_BLE) ? filter_context->ble_sensor_context.distance_value
                                                                                : filter_context->lora_sensor_context.distance_value);

        return FILTER_SRV_RING_IS_NOT_EMPTY;
    }
    else
    {
        return FILTER_SRV_RING_IS_EMPTY;
    }
}


void filterSrvThread(void)
{
    while (1)
    {
        filterSrvContext_t aux = {0};
        while (filterSrvConsumerItem(&aux) != FILTER_SRV_RING_IS_EMPTY)
        {
            filterSrvAddSensorData(&aux);
        }

        k_sleep(K_MSEC(FILTER_SRV_REFRESH_INTERVAL));
    }
}

[hmolesworth]

At first sight it looks like put and get applies to a single element queue filterSrvRingBuffer instead of filterSrvBufferStorage; that single-element queue would fill up pretty quickly. There is also (perhaps benign) alignment issue between bytes and struct; I would suggest something like:

// Align of size of largest struct field, probably 32-bit but use 64-bit:
static uint8_t filterSrvBufferStorage[FILTER_SRV_BUFFER_SIZE_BYTES] __attribute__((section(".bss"), aligned(8), used));

I would personally order the struct and then compact to reduce storage and get/put overhead, although that can slow access to individual fields within the struct outside of the queue:

#pragma pack(1) //ensure byte by byte struct packing
typedef struct
{
    uint16_t addr;
    uint8_t valid_addr;        /// <<== move all known odd-byte fields to end of struct
    filterSrvSensorType_t analyzing_type;
    emaFilterContext_t ema_context;
    temaFilterContext_t tema_context;
    kalmanFilterContext_t kalman_context;
    filterSrvSensorContext_t ble_sensor_context;
    filterSrvSensorContext_t lora_sensor_context;
} filterSrvContext_t;
#pragma pack()

The underlying queue uses 32-bit values so best to ensure the struct has overall n*4 bytes.

[Elfving]

Hello Luis, 

I'll have to get back to you on this later this week, though I see you already have some pointers here. Another thing to make sure is that you are feeding the right data item to ring_buf_item_put(), and that you are not mixing up item and data mode. You can see more here.

Regards,

Elfving

[Elfving]

Hi again,

since you haven't replied I assume you figured it out?

Regards,

Elfving

[Luis Santos]

Sorry for the late response, Elfving.

I think the issue was related to the length of my struct. Once I decreased the struct length, I stopped having those problems.

Thanks a lot for your support!

[Elfving]

Ah okay. Thanks for letting me know!  Even though that wasn't on my list of suggestions I hope you'll let us know if there is anything else you need help with in the future.

Regards,

Elfving