Observer under FreeRTOS: Events stop coming after a few minutes.

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Arie_tondo — Sun, 26 Nov 2023 17:40:03 GMT

I also had similar problem and it solved it

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Kosmine — Thu, 05 Sep 2019 13:04:45 GMT

Also had this problem. Lucky I stumbled on this, because it seems to have fixed my problem.

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

thin_ice — Sun, 31 Mar 2019 20:03:53 GMT

stupid question... if SD keeps raising interrupts by SD_EVT_IRQHandler, then why would the scanner stop working forever as described in the problem statement which seemed to imply that the softdevice_task was unable to wake up ?

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Susheel Nuguru — Fri, 02 Nov 2018 15:35:27 GMT

Hi Siddhesh,

The changes in this post are for SDK 15 and not for prior versions.
I have created an internal ticket (NRFFOSDK-12391) for this fix to be more tested and included in future release.

Unfortunately, Technical Support cannot control as to which release this fix gets included in..

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Siddhesh — Tue, 30 Oct 2018 18:26:47 GMT

Hi Susheel,
Will this change will be implemented in the next SDK version i.e. 15.3

Is the change in port_cmsis_systick.c according to the following post valid ?
https://devzone.nordicsemi.com/f/nordic-q-a/24733/nrf52840-sdk-14-0-0-and-tickless-idle-freertos---some-suggestions

Any timeline on the release of SDK 15.3 ?

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

CheMax — Tue, 30 Oct 2018 12:23:21 GMT

Hi Susheel,

Thanks. I struggled with this problem for 4 days and already wanted to ask a question, but I found this post. You saved me!

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Norbert — Fri, 26 Oct 2018 08:51:36 GMT

Hi Susheel,

Excellent catch!

I am testing the fix in my non-stripped application. It has been running now without fault for an hour with ~40 advertisers nearby while the application w/o fix was failing after 2 minutes at most. So I am convinced that the bug is fixed. To be sure, I will keep the app running for the weekend.

This is something to go into the next release of the SDK.

Great work!

Thanks a lot,

Norbert

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Susheel Nuguru — Wed, 24 Oct 2018 19:27:17 GMT

Awesome ..

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

eric-appion — Wed, 24 Oct 2018 18:58:19 GMT

I had a related problem, reported in a private case, and this code change does fix it. Thanks Susheel!

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Susheel Nuguru — Wed, 24 Oct 2018 10:35:09 GMT

found a race condition in nrf_sdh_freertos.c file

softdevice task() tries to pull events from SD and then suspends the tasks unconditionally until the SD_EVT_IRQHandler resumes it.

But, if the below happens

nrf_sdh_evts_poll();

##### Interrupted by SD_EVT_IRQHandler which does a TaskResume##

### but when the context comes back to softdevice_task() it still suspends itself####

vTaskSuspend(NULL);

###at this point we have lost the chance to enable the scanner again###

SOLUTION

replace the task controls to TaskNotification as below

void SD_EVT_IRQHandler(void)
{
     BaseType_t xYieldRequired;

     //xYieldRequired = xTaskResumeFromISR( m_softdevice_task );
     vTaskNotifyGiveFromISR(m_softdevice_task, &xYieldRequired);

     if( xYieldRequired == pdTRUE )
     {
         portYIELD_FROM_ISR(xYieldRequired);
     }
}


/* This function gets events from the SoftDevice and processes them. */
static void softdevice_task(void * pvParameter)
{
    NRF_LOG_DEBUG("Enter softdevice_task.");

    if (m_task_hook != NULL)
    {
        m_task_hook(pvParameter);
    }

    while (true)
    {
        nrf_sdh_evts_poll(); // Let the handlers run first, in case the EVENT occured before creating this task.
        //vTaskSuspend(NULL);

      (void) ulTaskNotifyTake(pdTRUE,         /* Clear the notification value before exiting (equivalent to the binary semaphore). */
                      portMAX_DELAY); /* Block indefinitely (INCLUDE_vTaskSuspend has to be enabled).*/

    }
}

I am pretty sure this will fix the problem.

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Norbert — Fri, 19 Oct 2018 09:02:33 GMT

PM posted. Thanks!

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Susheel Nuguru — Thu, 18 Oct 2018 09:49:35 GMT

you can attach your project link or send me a private message of the link. I can then review the architecture of your tasks and their priorities.

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Norbert — Wed, 17 Oct 2018 15:31:31 GMT

Aryan,

I would be happy to provide you the stripped-down test case, if you could spare the time to look into it. Just let me know how I can push a project to you.

Thanks!

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Susheel Nuguru — Wed, 17 Oct 2018 15:27:50 GMT

Norbert,

The fact that your solution worked makes me believe stronger that this was a deadlock issue. You serialized the access using a binary semaphore and the problem went away, I cannot say with 100% certainty without understanding your different priority tasks and when they suspend and resume.

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Norbert — Wed, 17 Oct 2018 14:43:46 GMT

Not sure. As I said, I have stripped down the project to only one task (the one running the SD), so there cannot be a deadlock between tasks.

My current workaround is to communicate the event to a lower-priority task using a binary semaphore and treat the event there. This takes only ~20us in the event handler and the problem goes away.

However this does not satisfy me as I think there is still something wrong at the base.

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Susheel Nuguru — Wed, 17 Oct 2018 14:34:52 GMT

[quote user="ngrunert"]So the probability of getting two reports coming in very close together is pretty high.[/quote]

This should not be a problem for the SD and I do not think is the reason for what you see.

I think that you are seeing a deadlock somewhere in one of your threads. Hard to say where and why, but if there are any conditions where you are waiting for some flags set in another thread, then make sure that you have scoped out a possibility of having a deadlock there since these checks are not normally atomic.

RE: Observer under FreeRTOS: Events stop coming after a few minutes.

Norbert — Mon, 15 Oct 2018 11:59:13 GMT

Forgot to mention: I have a lot (>20) of advertisers running in the vicinity to stress my system. So the probability of getting two reports coming in very close together is pretty high.