Using die temp interrupt causes BLE cannot be initialized

Question

Hi, I'm working on the DK for nrf52832. I have a working code for BLE, and another working code for die temperature using interrupt. Now I want to merge them and trigger the die temp in an app_timer in main. Then the callback of temperature will be processed later in main. Below is my die_temp.c

#include "../include/die_temp.h"

static void (*on_report_callback)(int32_t temp) = NULL;

void TEMP_IRQHandler()
{
    NRF_TEMP->TASKS_STOP = 1;
    NRF_TEMP->EVENTS_DATARDY = 0;

    /**@note Workaround for PAN_028 rev2.0A anomaly 29 - TEMP: Stop task clears the TEMP register. */
    int32_t temp = (nrf_temp_read() / 4);

    /**@note Workaround for PAN_028 rev2.0A anomaly 30 - TEMP: Temp module analog front end does not power down when DATARDY event occurs. */
    NRF_TEMP->TASKS_STOP = 1; /** Stop the temperature measurement. */

    if (on_report_callback) on_report_callback(temp);
}


void die_temp_set_report_callback(void (*callback)(int32_t temp)){
    on_report_callback = callback;
}

void die_temp_init(void)
{
    nrf_temp_init();
    sd_nvic_SetPriority(TEMP_IRQn, APP_IRQ_PRIORITY_LOWEST);
    sd_nvic_EnableIRQ(TEMP_IRQn);
    NRF_TEMP->INTENSET = TEMP_INTENSET_DATARDY_Enabled << TEMP_INTENSET_DATARDY_Pos;
}

inline void die_temp_start(void){
    NRF_TEMP->TASKS_START = 1; /** Start the temperature measurement. */
}

After I add this library code into my BLE code, the BLE cannot be initialized anymore. I traced back and error originated from

nrf_sdh_enable_request() fails to enable soft device with error code 4097. I don't understand why. Can you please help?

Thank you