Identifying a High-Frequency ISR in SystemView

I started analyzing my application using SystemView to try to resolve a bug. I think I have configured everything correctly. However, when I start the analysis, I notice that an ISR is being called periodically every 61 microseconds, significantly occupying the CPU. Initially, I thought it was the Zephyr RTOS tick, but since it occurs every 120 microseconds, I ruled out that possibility. Could it be related to the BLE stack?

  1. How can I determine which ISR it is? Is it possible to assign names to ISRs when tracing?
  2. Why don't I see the Zephyr RTOS tick ISR?

Am I perhaps acquiring the data incorrectly?

UPDATE: I also noticed that if I enable tracing

CONFIG_TRACING=y
CONFIG_SEGGER_SYSTEMVIEW=y

the function bt_scan_start() in my application returns the error -EAGAIN due to the BT_DEV_READY flag never being set. Even if I put a loop to wait for it to be set

while ( bt_is_ready() == false )
{
    k_sleep( K_MSEC( 100 ) );
}

ret = bt_scan_start( BT_SCAN_TYPE_SCAN_ACTIVE );

if ( ret != 0 )
{
    LOG_WRN( "bt_scan_start (err %d)", ret );
	return SYSTEM_ERROR;
}

I never exit the loop. By running some tests, I discovered that it's as if BLE is never activated. In the BLE initialization, I have:

void Ble_Connection_Ready( int err )
{
    if ( err )
    {
        return;
    }

    k_sem_give( &bluetoothReady );
}

[...]

if ( bt_enable( Ble_Connection_Ready ) != 0 )
{
	return SYSTEM_ERROR;
}

if ( k_sem_take( &bluetoothReady, K_SECONDS( 10 ) ) != 0 )
{
    return SYSTEM_ERROR;
}

And the k_sem_take() always fails because BLE is never enabled cause the Ble_Connection_Ready( int err ) returns error -105 (ENOBUFS). However, if I disable tracing, everything works as expected. Below is my prj.conf.

################################################
### SYSTEM CONFIGURATION #######################
################################################
CONFIG_LOG=y
CONFIG_LOG_MAX_LEVEL=2
CONFIG_USE_SEGGER_RTT=y
CONFIG_RTT_CONSOLE=y
CONFIG_HEAP_MEM_POOL_SIZE=2048
CONFIG_FPU=y
CONFIG_RING_BUFFER=y
CONFIG_ZCBOR=y
CONFIG_ZCBOR_STOP_ON_ERROR=y
# CONFIG_DISABLE_FLASH_PATCH=y
CONFIG_EVENTS=y
CONFIG_KERNEL_BIN_NAME="cruise_dongle"
CONFIG_TRACING=y
CONFIG_SEGGER_SYSTEMVIEW=y

################################################
### SOC'S PERIPHERALS CONFIGURATION ############
################################################
CONFIG_GPIO=y
CONFIG_PWM=y
CONFIG_LED=y
CONFIG_SERIAL=y
CONFIG_UART_ASYNC_API=y
CONFIG_UART_0_NRF_HW_ASYNC=y
CONFIG_UART_0_NRF_HW_ASYNC_TIMER=1

################################################
### BLE CONFIGURATION ##########################
################################################
CONFIG_BT_MAX_CONN=8
CONFIG_BT=y
CONFIG_BT_CENTRAL=y
CONFIG_BT_SMP=y
CONFIG_BT_DFU_SMP=y
CONFIG_BT_GATT_CLIENT=y
CONFIG_BT_SCAN=y
CONFIG_BT_SCAN_FILTER_ENABLE=y
CONFIG_BT_SCAN_CONN_ATTEMPTS_FILTER=y
CONFIG_BT_SCAN_CONN_ATTEMPTS_FILTER_LEN=6
CONFIG_BT_SCAN_CONN_ATTEMPTS_COUNT=2
CONFIG_BT_SCAN_UUID_CNT=3
CONFIG_BT_GATT_DM=y
CONFIG_BT_BUF_ACL_RX_SIZE=502
CONFIG_BT_ATT_PREPARE_COUNT=2
# CONFIG_BT_L2CAP_TX_BUF_COUNT=10
CONFIG_BT_L2CAP_TX_MTU=498
# CONFIG_BT_L2CAP_DYNAMIC_CHANNEL=y
CONFIG_BT_CONN_TX_MAX=10
CONFIG_BT_BUF_ACL_TX_COUNT=10
CONFIG_BT_BUF_ACL_TX_SIZE=502
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_BT_CTLR_PHY_2M=y
CONFIG_BT_CTLR_RX_BUFFERS=2
CONFIG_BT_CTLR_SDC_MAX_CONN_EVENT_LEN_DEFAULT=4000000

CONFIG_BT_USER_DATA_LEN_UPDATE=y
CONFIG_BT_USER_PHY_UPDATE=y

################################################
### NVS MEMORY CONFIGURATION ###################
################################################
CONFIG_NVS=y

################################################
### FLASH MEMORY SUPPORT CONFIGURATION #########
################################################
CONFIG_FLASH=y
CONFIG_MPU_ALLOW_FLASH_WRITE=y
CONFIG_FLASH_PAGE_LAYOUT=y

  • Hi Andrea,

    It is hard to immediately say what is causing this, though this interrupt definitely could be from the softdevice. One question is if you are you seeing this on just your application, on all samples, or maybe just the BLE ones.

    How can I determine which ISR it is? Is it possible to assign names to ISRs when tracing?

    There should be some info on how to do this here.

    Why don't I see the Zephyr RTOS tick ISR?

    The blog mentions that you need to explicitly enable logging for specific interrupts, eg. sysview_irq_log( RTC1_IRQn, true );

    Regards,

    Elfving

  • Thanks for the links you sent me. I'll check them and see if I can solve the issue. Thanks. By the way, do you know anything about the ENOBUFS error that the Ble_Connection_Ready() function returns when I use it in bt_enable()? The problem only appears if I enable tracing.

    void Ble_Connection_Ready( int err )
    {
        if ( err )
        {
            LOG_WRN( "Ble_Connection_Ready returned err %d", err );
            return;
        }
    
        k_sem_give( &bluetoothReady );
    }
    
    [...]
    
    if ( bt_enable( Ble_Connection_Ready ) != 0 )
    {
    	return SYSTEM_ERROR;
    }
    
    if ( k_sem_take( &bluetoothReady, K_SECONDS( 5 ) ) != 0 )
    {
        return SYSTEM_ERROR;
    }

    You can find my configuration prj.conf in the previous post.

    Or would it be better to open another topic?

  • No problem Slight smile

    Andrea Verdecchia said:
    By the way, do you know anything about the ENOBUFS error that the Ble_Connection_Ready() function returns when I use it in bt_enable()? The problem only appears if I enable tracing.

    Must have missed that question in the OP. It is out of buffer space. Tracing requires a bit more resources, so it does make sense that it can affect things. Does increasing the buffer size help? Use CONFIG_BT_BUF_CMD_TX_COUNT.

    This does look a bit similar to this recent issue..., What NCS version are you using?

    Regards,

    Elfving

  • Yes, exactly. After replying to you, I went to look for the source of the error and arrived at the function bt_hci_cmd_send_sync (specifically net_buf_alloc). At this point, after searching online, I added these configurations

    CONFIG_BT_BUF_EVT_RX_COUNT=10
    CONFIG_BT_BUF_EVT_RX_SIZE=255
    CONFIG_BT_BUF_CMD_TX_COUNT=10
    CONFIG_BT_BUF_CMD_TX_SIZE=255
    CONFIG_BT_RX_STACK_SIZE=2048
    CONFIG_BT_HCI_TX_STACK_SIZE=2048

    The issue related to net_buf_alloc seems to be resolved, but now a new error appears in the logs:

    *** Booting My Application v0.0.10-31b40401242e ***
    00> *** Using nRF Connect SDK v2.9.0-7787b2649840 ***
    00> *** Using Zephyr OS v3.7.99-1f8f3dc29142 ***
    00> [00:00:00.023,101] <wrn> bt_hci_core: Vendor HCI extensions not available
    00> [00:00:00.023,132] <wrn> bt_id: Failed to read public address
    00> [00:00:00.023,223] <wrn> bt_id: Read Static Addresses command not available
    00> [00:00:00.023,742] <err> bt_id: Unable to set random address
    00> [00:00:00.023,742] <wrn> ble_connection: Ble_Connection_Ready returned err -105

    I don’t have the device with me right now, but I will run more tests tomorrow. Thanks in the meantime for confirming the possible solution with CONFIG_BT_BUF_CMD_TX_COUNT. If you have any suggestions regarding the new error, they are more than welcome.

    Does the fix you sent me apply to my case? I'm working with NCS 2.9.0.

  • Andrea Verdecchia said:
    Does the fix you sent me apply to my case? I'm working with NCS 2.9.0

    I don't think it's included. You could cherry pick that if you want to, but if the buffer modification works I guess it's fine. 

    Andrea Verdecchia said:
    If you have any suggestions regarding the new error, they are more than welcome.

    I'm not 100% sure here. But just to start with "Vendor HCI extensions not available", as that also might've affected everything else, do you know what HCI extensions in your app it could be referring to?

    Regards,

    Elfving

Related