nrf54l15 with ncs sdk 3.4.0 : MPSL deadlock

Hi,

We are using nrf54l15 soc in our product and using NCS SDK 3.4.0.   The LFCLK is set to SYNTH mode where it derives the clock from HFCLK.  We have a crystal driving the HFXO.

Couple of boards consistently fail to boot up and when we put the JTAG and then halt and go, it boots up.  There are good number of boards that come up fine too.  So this seems some corner condition/data race. 

When I halted the CPU using JTAG and looked at the PC and cross checked with the disassembled code, it was stuck somewhere in MPSL code . It's only a guess as the call stack shows some mangled function names which evenutally point to mpsl_init() ..  If it can help I can share the disassembly with our code portions removed and limiting only to the stack trace upto mpsl_init.

I dumped the CLOCK registers and it seems like this happens only when there is an intermittent XO TUNE Error.   We consistently see the below contents . 

J-Link>mem32 0x4010E110 3
4010E110 = 00000001 00000001 00000000

XOTUNED = 00000001 

XOTUNEERROR = 00000001 

XOTUNEFAILED = 00000000

J-Link>mem32 0x4010E108 1
4010E108 = 00000001

LFCLKSTARTED = 00000001

Hovering around the code for mpsl_init , I saw the below highlighted note in "ncs\v3.4.0\nrfxlib\mpsl\include\mpsl.h" and tried setting CONFIG_SYSTEM_CLOCK_NO_WAIT=y, and when I do so, this issue disappears and the board boots up fine !! . Looks like something is going on here with respect to clocks. 

/** @brief MPSL initialization
*
* @param[in] p_clock_config Clock configuration.
If NULL the LF clock will be configured as an RC source with rc_ctiv =
@ref MPSL_RECOMMENDED_RC_CTIV, .rc_temp_ctiv =
@ref MPSL_RECOMMENDED_RC_TEMP_CTIV, and .accuracy_ppm = @ref MPSL_DEFAULT_CLOCK_ACCURACY_PPM.
The parameter is not used when external clock driver is registered @ref mpsl_clock_ctrl_source_register().
* @param[in] low_prio_irq IRQ to pend when low priority processing should be executed. The application
* shall call @ref mpsl_low_priority_process after this IRQ has occurred.
* @param[in] p_assert_handler Pointer to MPSL assert handler.
*
* @note If `CONFIG_SYSTEM_CLOCK_NO_WAIT` is set to 0,
* never modify the SEVONPEND flag in the SCR register,
* while this function is executing.
* Doing so might lead to a deadlock.
*
* @note For nRF54h SoC series the function always waits for LFCLK to be ready. The LFCLK is handled by system controller
* so response must arrive from other domain. That shall be done in non-blocking context. To do not change
* requirements for other MPSL APIs delayed wait for LFCLK is not allowed for the nRF54h SoC series.
*
* @note If only Front End Module functionality is needed, @ref mpsl_fem_init can be called instead.

*
* @retval 0 MPSL is successfully initialized.
* @retval -NRF_EPERM MPSL is already initialized.
* @retval -NRF_EINVAL Invalid parameters supplied.
*/
int32_t mpsl_init(mpsl_clock_lfclk_cfg_t const * p_clock_config, IRQn_Type low_prio_irq, mpsl_assert_handler_t p_assert_handler);

With that context, please help with the following questions.

1) Does this point to any issue in the code ?  It's possible that the crystal we are using could cause a temporary XOTUNEERROR but re tune seems to get it into place.  So getting a XOTUNEERROR must be tolerated.

2) Is it okay to use CONFIG_SYSTEM_CLOCK_NO_WAIT=y ? Does it have any other side effect. 

Regards

Ramakrishnan.

  • Hi,

     

    Couple of boards consistently fail to boot up

    Since this issue is only occurring on some boards, I would recommend that you check the soldering around your HFXO crystal.

    Given that this configuration works:

    tried setting CONFIG_SYSTEM_CLOCK_NO_WAIT=y, and when I do so, this issue disappears and the board boots up fine !! . Looks like something is going on here with respect to clocks. 

    It indicates that some of your boards have issues with the crystal. Could you try to rework these boards, with specific focus on the xtal + XL1/XL2 lines?

     

    Kind regards,

    Håkon

  • Hi,

    Thanks for the response.

    I'd like to clarify one point about the failing boards - when they are in the stuck state, if I connect J-link debugger and then do halt and resume ( using segger J-Link commander application, issue 'h' to halt and then 'g )  they come out of the stuck state. If it was a hard failure due to crystal, that is unlikely to happen and the board must be stuck for ever, isn't it ?. So I'm suspecting if the halt+resume is bring out the code from some blocking waiting state ( like a wfe ) which is not happening in a regular boot up scenario.

    Having said that, I'll check on the soldering as well one more time, esp around xl1/xl2.  We had checked it earlier. 

    Meanwhile, can you pls share your thoughts on the above point ?

    Regards

    Ramakrishnan

Related