nrf_log_frontend_dequeue must be atomically protected against re-entry from interrupt context

RE: nrf_log_frontend_dequeue must be atomically protected against re-entry from interrupt context

hmolesworth — Thu, 20 Feb 2020 16:41:16 GMT

Ah, quite so - thanks!

RE: nrf_log_frontend_dequeue must be atomically protected against re-entry from interrupt context

Anthony Ambuehl — Thu, 20 Feb 2020 16:33:20 GMT

the return type for nrf_log_frontend_dequeue is bool so in your busy check should look like this:

if (nrf_atomic_flag_set_fetch(&m_log_data.log_is_busy)==1)
{
return false;
}

also you should use the atomic library to clear the variable:

UNUSED_RETURN_VALUE(nrf_atomic_flag_clear(&m_log_data.log_is_busy));

RE: nrf_log_frontend_dequeue must be atomically protected against re-entry from interrupt context

hmolesworth — Thu, 20 Feb 2020 16:28:22 GMT

Good find. I have the same issue, and following you suggestion have implemented the following, which if you have time to review I would be most grateful.

typedef struct
{
    uint32_t                  wr_idx;          // Current write index (never reset)
    uint32_t                  rd_idx;          // Current read index  (never_reset)
    uint32_t                  mask;            // Size of buffer (must be power of 2) presented as mask
    uint32_t                  buffer[NRF_LOG_BUF_WORDS];
    nrf_log_timestamp_func_t  timestamp_func;  // A pointer to function that returns timestamp
    nrf_log_backend_t const * p_backend_head;
    nrf_atomic_flag_t         log_skipping;
    nrf_atomic_flag_t         log_skipped;
    nrf_atomic_flag_t         log_is_busy;     // This flag replaces the blocking use of critical region
    nrf_atomic_u32_t          log_dropped_cnt;
    bool                      autoflush;
} log_data_t;

ret_code_t nrf_log_init(nrf_log_timestamp_func_t timestamp_func, uint32_t timestamp_freq)
{
    (void)NRF_LOG_ITEM_DATA_CONST(app);

    if (NRF_LOG_USES_TIMESTAMP && (timestamp_func == NULL))
    {
        return NRF_ERROR_INVALID_PARAM;
    }

    m_log_data.mask         = NRF_LOG_BUF_WORDS - 1;
    m_log_data.wr_idx       = 0;
    m_log_data.rd_idx       = 0;
    m_log_data.log_skipped  = 0;
    m_log_data.log_skipping = 0;
    m_log_data.log_is_busy  = 0;
    m_log_data.autoflush    = NRF_LOG_DEFERRED ? false : true;

bool nrf_log_frontend_dequeue(void)
{
    if (buffer_is_empty())
    {
        return false;
    }
    // Note also add atomic flag set before this __DSB() and after this function exits
    // See https://devzone.nordicsemi.com/f/nordic-q-a/39188/nrf_log_frontend_dequeue-must-be-atomically-protected-against-re-entry-from-interrupt-context
    if (nrf_atomic_flag_set_fetch(&m_log_data.log_is_busy))
    {
        return NRF_ERROR_BUSY;
    }
...
    // Skip any in progress packets.
    do {
        if (invalid_packets_omit(p_header, &rd_idx) && (m_log_data.log_skipped == 0))
        {
            //Check if end of data is not reached.
            if (rd_idx >= m_log_data.wr_idx)
            {
                m_log_data.rd_idx     = m_log_data.wr_idx;
                // Ensure that next invocation occurs after this flag is cleared.
                m_log_data.log_is_busy = 0;
                return false;
            }
...
    // Ensure that next invocation occurs after this flag is cleared.
    m_log_data.log_is_busy = 0;
    return buffer_is_empty() ? false : true;
}

RE: nrf_log_frontend_dequeue must be atomically protected against re-entry from interrupt context

Kenneth — Mon, 27 Jan 2020 15:30:03 GMT

I have updated the internal jira with your suggestions. Hopefully it the meantime others may find his case and your suggestion.

Best regards,
Kenneth

RE: nrf_log_frontend_dequeue must be atomically protected against re-entry from interrupt context

Anthony Ambuehl — Mon, 06 Jan 2020 22:10:36 GMT

I noticed what appears as an attempt at fixing this issue in the nrf_log_frontend.c. The change in recent SDK adds a CRITICAL_REGION around the NRF_LOG_FLUSH() called by autoflush. This may address the issue however it will block interrupts for an extended time while waiting for logs to transfer.

I recommend not using this workaround. Instead, I recommend adding an atomic flag set_fetch before the DSB call in nrf_log_frontend_dequeue() and atomic flag clear before returning. This change would allow ISR's to continue normal operation.

I have been using this fix for over a year and it has not shown any issues.

RE: nrf_log_frontend_dequeue must be atomically protected against re-entry from interrupt context

awneil — Wed, 10 Oct 2018 16:54:56 GMT

Thanks.

RE: nrf_log_frontend_dequeue must be atomically protected against re-entry from interrupt context

Anthony Ambuehl — Wed, 10 Oct 2018 16:42:35 GMT

The symptom is log messages that are sometimes repeated. They may be a single log repeated or a series of logs that are repeated in a sequence.

for example

log 1

log 2

log 3

log 1

log 2

log 3

log 1

log 2

log 3

RE: nrf_log_frontend_dequeue must be atomically protected against re-entry from interrupt context

haakonsh — Mon, 08 Oct 2018 11:20:39 GMT

I'll notify the SDK team. It might be some time before we get a reply.

RE: nrf_log_frontend_dequeue must be atomically protected against re-entry from interrupt context

awneil — Thu, 04 Oct 2018 20:01:13 GMT

Hmmm - what would be the symptoms of this going wrong?