nRF52833 + external 32 kHz TCXO: soft reset hangs

[alexdr5398]

I’m working with a custom board based on the nRF52833. It uses an external 32 kHz TCXO (SiT1552) as the LFCLK source.

• In my main Zephyr app I set: CONFIG_CLOCK_CONTROL_NRF_K32SRC_EXT_FULL_SWING=y  and in mcuboot i set CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC=y. This boots fine if I power cycle the board, but then when the app does a software reboot (sys_reboot(...) or NVIC_SystemReset()), the system hangs indefinitely. Power-cycling causes it to again boot into the primary image normally. 

• If I switch mcuboot to also use EXT_FULL_SWING, then after reset I see the UART log:

  mcuboot: Jumping to the first image slot
  

  but then nothing. I investigated with a debugger and it the app never seems to reach main() or even any SYS_INIT routines. Whenever I break, it's just in the zephyr kernel's idle thread. 

So, to summarize:

• App works with TCXO only if mcuboot uses RC, but any software reboots causes hang.

• If both use TCXO, app never boots.

• If both use RC, then every thing works normally (but i of course want to use the external tcxo). 

Versions:

zephyr: nrfconnect sdk-zephyr v3.5.99
nrf: nrfconnect sdk-nrf v2.6.1

Any help on this is greatly appreciated. 

Thanks

EDIT: I forgot to mention that I have indeed verified that the program is getting past the `lfclk_spinwait` function, so it does seem to be detecting that the clock has started. I placed a breakpoint at `clock_control_nrf.c:540` after the `!nrf_clock_is_running` loop, and it does indeed reach that point. 

[Vidar Berg]

Hello,

I don't have a TCXO to test with here so I can't easily reproduce this, but I assume the reason it hangs is that the LF clock started event is not triggered causing the program to never exit clock_control_nrf.c->lfclk_spinwait() function. You may be able to confirm this by looking at the call stacks.

If I switch mcuboot to also use EXT_FULL_SWING, then after reset I see the UART log:

As a test,  please try and see if you get the same result if you build the app with CONFIG_SYSTEM_CLOCK_NO_WAIT=y.

Best regards,

Vidar

[alexdr5398]

Small update: 

I have confirmed that I do hit breakpoints in PRE_KERNEL_X SYS_INIT's and in POST_KERNEL SYS_INIT with priority 0, *but not* priority 99. 

Currently narrowing down exactly which sys_init is causing the issue. 

[alexdr5398]

I have confirmed that this breakpoint is hit:

static int test_init() {
	return 0;
}
SYS_INIT(test_init, POST_KERNEL, 79);

but this one never is:

static int test_init() {
	return 0;
}
SYS_INIT(test_init, POST_KERNEL, 80);

[Vidar Berg]

Interesting. You can view the list of init functions here if you're using VS code: 

alexdr5398 said:

CONFIG_SYSTEM_CLOCK_NO_WAIT=y had no effect, i'm assuming this is just skipping that spinwait function?

Correct.

[alexdr5398]

I'm not using vscode unfortunately. do you what command line tool gives access to that info?

I can maybe try to get vscode working for this project. 

[Vidar Berg]

I don't know how they are making this list, I would have to ask them. I don't immediately see any ways to check this manually. However, you can try to to search for this priority in the .config file for your app(build/<app name>/zephyr/.config)

[alexdr5398]

I managed to find the sys init that caused the issue. It's in a custom driver init that hangs when it calls 'k_sleep'. 

When I put `k_msleep` in my test_init function, it hangs there too. I believe that zephyr uses the LFCLK for its scheduling, so it would seem that the LFCLK is not actually started? 

[Vidar Berg]

Yes, the scheduler relies on the lfclk, so I agree, this indicates that the clock is not running. Not sure what could be causing this yet.

[alexdr5398]

EDIT: this whole reply is inaccurate, see next reply

Okay I managed to get it to work reliably by swapping from k_sleep -> k_busy_wait in another driver init and by adding this function that explicitly starts the LF clock:

static int start_clock() {
	int err;
	struct onoff_manager *clk_mgr;
	struct onoff_client clk_cli;

	clk_mgr = z_nrf_clock_control_get_onoff(CLOCK_CONTROL_NRF_SUBSYS_LF);
	if (!clk_mgr) {
		LOG_ERR("Unable to get the Clock manager");
		return -ENXIO;
	}

	sys_notify_init_spinwait(&clk_cli.notify);

	err = onoff_request(clk_mgr, &clk_cli);
	if (err < 0) {
		LOG_ERR("Clock request failed: %d", err);
		return err;
	}

	int res;
	do {
		err = sys_notify_fetch_result(&clk_cli.notify, &res);
		// printk("err %d res %d\n", err, res);
		if (!err && res) {
			// LOG_ERR("Clock could not be started: %d", res);
			return res;
		}
	} while (err);

	return 0;
}
SYS_INIT(start_clock, POST_KERNEL, 0);

The strange part is that before updating that driver to use k_busy_wait, if I changed the log level in the module with the `start_clock` sys_init from LOG_LEVEL_WRN to LOG_LEVEL_DBG, it did boot normally.  

I really don't like how unreliable this seems, especially sihce this is a device that we will definitely not be able to update in the field if they don't boot. 

Is there a way to simply use the internal RC for both mcuboot and the primary image, but then switch to the external TCXO at runtime? that way if it doesn't work (or if the component breaks/falls off), we can always fall back to the internal RC.