Watchdog feed while using blocking APIs

RE: Watchdog feed while using blocking APIs

Ankit_chauhan — Mon, 29 Jul 2024 10:11:32 GMT

Hi Charlie,

Thanks for your reply.

RE: Watchdog feed while using blocking APIs

Charlie — Fri, 26 Jul 2024 07:46:58 GMT

Hi Ankit,

Task watchdog is not directly maintained by Nordic, you can try to create an issue report here in its original Zephyr repository:

New Issue · zephyrproject-rtos/zephyr (github.com)

Task Watchdog — Zephyr Project Documentation

Best regards,

Charlie

RE: Watchdog feed while using blocking APIs

Ankit_chauhan — Tue, 23 Jul 2024 12:43:30 GMT

Hello,

In my last reply, as I mentioned I implemented the task watchdog for multi-thread application using the channel watchdog concept which allows me to set a distinct time in every watchdog channel. I have tested this approach and it is working as expected.

However, I came across another issue with this approach. In task watchdog, I have defined a callback function that will be called in case of feed failure. Currently, I am not restarting the system, just indicating the system failure and continuing the operation.

I am using the MCU boot to support the OTA using the nRF Connect App. When I performed the DFU, my sensor got restarted due to the watchdog (probably failed for feed). But the strange thing is that as I mentioned, I am not resetting the device when I am in task_watchdog callback function. Also from the logs, I can say that even the task watchdog callback is not even called. Below is my callback function snippet.

static void task_wdt_callback(int channel_id, void *user_data)
{
	/*
	 * If the issue could be resolved, call task_wdt_feed(channel_id) here
	 * to continue operation.
	 *
	 * Otherwise we can perform some cleanup and reset the device.
	 */
	printk("\n********************************************\n");
	printk("********************************************\n");
	printk("Task watchdog triggered with channel %d\n", channel_id);
	
	task_wdt_feed(channel_id);

	printk("Feeding the channel id : %d...\n", channel_id);
	printk("********************************************\n");
	printk("********************************************\n");
	// sys_reboot(SYS_REBOOT_COLD);
}

Below is the error I got from the RTT logs,

00> I: Reset Reason, RESETREAS=0x2
00> I: Reset Causes: 
00> I:  Watchdog 0

I have checked the task_wdt_init() which initializes the wdt_timeout_cfg structure with flag value WDT_FLAG_RESET_SOC. It has the option of WDT_FLAG_RESET_NONE and I tried to change that flag but the wdt_install_timeout() returns 134 which is a "not supported" error. I have also tried enabling the CONFIG_WDT_DISABLE_AT_BOOT but it did not help.

So bottom of the line, I do not want to restart my system in case of watchdog failure, I can showcase the error via LED or other medium but the watchdog should not affect the OTA.

Regards,

Ankit.

RE: Watchdog feed while using blocking APIs

Ankit_chauhan — Wed, 03 Jul 2024 12:46:56 GMT

Hi Charlie,

Thanks for the update.

Sorry, I was using the Zephyr sample not the nRF SDK sample. Here is the link https://github.com/zephyrproject-rtos/zephyr/tree/main/samples/subsys/task_wdt.

For the second query for watchdog control in blocking API, I am thinking of controlling the watchdog start and stop in between the usage of those blocked threads.

My system is designed like that whenever the sensor sample collection is requested then only I am enabling the sensor threads otherwise all are in a suspended state. So I am planning to enable the task watchdog when the sensor threads are resumed. Similarly will stop the task watchdog when the threads are suspended using the task_wdt_add and task_wdt_delete APIs respectively.

In between these states, even the threads which are having the blocking APIs will run repetitively. As you said I have calculated the time of a thread that stays blocking and I have set that time in the task watchdog time out and that should work.

Let me try this approach and will let you know if I find any issues.

Thanks once again for your support.

Regards,

Ankit.

RE: Watchdog feed while using blocking APIs

Charlie — Wed, 03 Jul 2024 12:10:02 GMT

Hi Ankit,

Thanks for the feedback. May I know which "nRF SDK sample" you refer to? Do you mean Task watchdog — Zephyr Project documentation (nRF Connect SDK) (nordicsemi.com). To be honest, I do not have much experience about Task Watchdog, it is one subsystem originally from Zephyr project. Good to hear it works as you expected, but I need to draw your attention that it used a kernel timer as its backend instead of a separate watchdog peripheral.

[quote user="Ankit_chauhan"]However, I am unsure how to handle the other sensor threads that have the ZBUS and Semaphore blocking API with this approach.[/quote]

When it comes to this question, have you evaluated what is the maximum time the thread could be blocking? As I shared before, this is critical for picking up the proper watchdog timeout to detect a real loop death, and it is related to your application logic about how to get and process sensor data.

Best regards,

Charlie

RE: Watchdog feed while using blocking APIs

Ankit_chauhan — Wed, 03 Jul 2024 05:18:25 GMT

Hi Charlie,

Thanks for your reply.

After spending some time finding the solution to this issue, I came across an nRF sample of task_wdt in the nRF SDK sample. I have implemented that into my application using the multi-channel watchdog concept where each channel is configured with common task_wdt_add API.

On the main thread and a sensor thread, I am using task_wdt_feed which feeds the watchdog with the respective channel ID.

In some instances, I have seen that it is working because if I am not feeding either thread with task_wdt_feed, the task_wdt_callback will be called and the system gets restarted with the channel ID of the thread which is expected.

However, I am unsure how to handle the other sensor threads that have the ZBUS and Semaphore blocking API with this approach.

Regards,

Ankit.

RE: Watchdog feed while using blocking APIs

Charlie — Tue, 02 Jul 2024 14:22:57 GMT

Hi Ankit,

It would be better to have a main thread to run the app logic in a loop.

If you have to use WDT in sensor threads or there is no main loop, you need to pick up a proper watchdog triggering time for it.

This will depend on your application, as sensor threads may block but only allow them to block for a period of time that your application can tolerate, otherwise the application will reset to restart.

Let me know if you need more advice.

Best regards,

Charlie