disconnecting while operations are in progress never gives BLE_GAP_EVT_DISCONNECTED event

[Paul Bradford]

2020-01-24-092119EST-ProductStoppedGettingEventsFromNordicDK.txtImprivataTestNordicEventsNotReceived.zipCalls_to_pc_ble_driver.cpp0285.2020-02-24-TestProgramUploadedToNordicSupport.zipFeb25TestProgramUploadedToNordicSupport.zipImprivata_bgTestApp.zipbgSDKTestAppMay4.zip2020-05-05-035347-NordicDK_USB840M_200505_ClockInternal_2in1.hex.txt.txtbgSDKTestAppMay6.zipI’m developing an application based on pc-ble-driver to talk to an nRF52840-based dongle (from Fanstel).

I’m having trouble disconnecting cleanly when a connection has operations in progress.  For example, I call ‘sd_ble_gattc_write’, which returns NRF_SUCCESS, but I don’t receive event BLE_GATTC_EVT_WRITE_RSP (after waiting 60 seconds), so I decide to disconnect. When this happens, sd_ble_gap_disconnect returns NRF_SUCCESS, but I do not receive BLE_GAP_EVT_DISCONNECTED even after waiting 30 seconds. The connection supervision timeout is 4 seconds.  What could cause the disconnect to not generate any BLE_GAP_EVT_DISCONNECTED event?

What I’m trying to accomplish here: if a connection is not responsive, I want to end that connection, without disturbing other connections I have open.

Thanks,

Paul Bradford

[Paul Bradford]

I uploaded a modified test program. The change is to have a command-line option to say how often to do the challenge/response, which is what triggers the notifications from the phone, which is what we suspect is causing this problem.   With no option, this is done every 5 minutes. I've been testing with new command-line option -f5 which means to do it every 5 seconds.

[Jørgen Holmefjord]

LeoChen: I did manage to reproduce once with today with the USB840_Clock_ACCURACY_2in1.hex file you provided on a Nordic Dongle. Unfortunately, it does not provide much information about the fatal error. I had modified this in the connectivity hex that I build with queue size of 64 and logging enabled. Can you replace this file in your SDK? It will give the location of the error and the error code on the log. 

Fullscreen ser_conn_error_handling.c Download

/**
 * Copyright (c) 2014 - 2019, Nordic Semiconductor ASA
 *
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without modification,
 * are permitted provided that the following conditions are met:
 *
 * 1. Redistributions of source code must retain the above copyright notice, this
 *    list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form, except as embedded into a Nordic
 *    Semiconductor ASA integrated circuit in a product or a software update for
 *    such product, must reproduce the above copyright notice, this list of
 *    conditions and the following disclaimer in the documentation and/or other
 *    materials provided with the distribution.
 *
 * 3. Neither the name of Nordic Semiconductor ASA nor the names of its
 *    contributors may be used to endorse or promote products derived from this
 *    software without specific prior written permission.
 *
 * 4. This software, with or without modification, must only be used with a
 *    Nordic Semiconductor ASA integrated circuit.
 *
 * 5. Any software provided in binary form under this license must not be reverse
 *    engineered, decompiled, modified and/or disassembled.
 *
 * THIS SOFTWARE IS PROVIDED BY NORDIC SEMICONDUCTOR ASA "AS IS" AND ANY EXPRESS
 * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 * OF MERCHANTABILITY, NONINFRINGEMENT, AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL NORDIC SEMICONDUCTOR ASA OR CONTRIBUTORS BE
 * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
 * GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
 * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 *
 */
#include "nrf_assert.h"
#include "app_error.h"
#include "ser_config.h"
#include "app_util_platform.h"
#include "nrf_strerror.h"

#if defined(SOFTDEVICE_PRESENT) && SOFTDEVICE_PRESENT
#include "nrf_sdm.h"
#endif

#include "boards.h"

/** @file
 *
 * @defgroup ser_conn_error_handling Errors handling for the Connectivity Chip.
 * @{
 * @ingroup sdk_lib_serialization
 *
 * @brief   A module to handle the Connectivity Chip errors and warnings.
 *
 * @details This file contains definitions of functions used for handling the Connectivity Chip
 *          errors and warnings.
 */

/**@brief Function for handling the Connectivity Chip errors and warnings.
 *
 * @details This function will be called if the ASSERT macro in the connectivity application fails.
 *          Function declaration can be found in the app_error.h file.
 *
 * @param[in] error_code  Error code supplied to the handler.
 * @param[in] line_num    Line number where the handler is called.
 * @param[in] p_file_name Pointer to the file name.
 */

#include "app_util_platform.h"
#include "nrf_soc.h"
#include "nrf_log.h"
#include "nrf_log_ctrl.h"

// uint32_t m_error_code;
// uint32_t m_error_line_num;
// const uint8_t *m_p_error_file_name;

/*lint -save -e14 */
void app_error_fault_handler(uint32_t id, uint32_t pc, uint32_t info)
{
    // disable INTs
    CRITICAL_REGION_ENTER();

    NRF_LOG_ERROR("Fatal error");
    switch (id)
    {
#if defined(SOFTDEVICE_PRESENT) && SOFTDEVICE_PRESENT
        case NRF_FAULT_ID_SD_ASSERT:
            NRF_LOG_ERROR("SOFTDEVICE: ASSERTION FAILED");
            break;
        case NRF_FAULT_ID_APP_MEMACC:
            NRF_LOG_ERROR("SOFTDEVICE: INVALID MEMORY ACCESS");
            break;
#endif
        case NRF_FAULT_ID_SDK_ASSERT:
        {
            assert_info_t * p_info = (assert_info_t *)info;
            NRF_LOG_ERROR("ASSERTION FAILED at %s:%u",
                          p_info->p_file_name,
                          p_info->line_num);
            break;
        }
        case NRF_FAULT_ID_SDK_ERROR:
        {
            error_info_t * p_info = (error_info_t *)info;
            NRF_LOG_ERROR("ERROR %u [%s] at %s:%u\r\nPC at: 0x%08x",
                          p_info->err_code,
                          nrf_strerror_get(p_info->err_code),
                          p_info->p_file_name,
                          p_info->line_num,
                          pc);
             NRF_LOG_ERROR("End of error report");
            break;
        }
        default:
            NRF_LOG_ERROR("UNKNOWN FAULT at 0x%08X", pc);
            break;
    }
    NRF_LOG_FINAL_FLUSH();

    #if LEDS_NUMBER > 0

    /* Light a LED on error or warning. */
    // nrf_gpio_cfg_output(SER_CONN_ASSERT_LED_PIN);
    // nrf_gpio_pin_set(SER_CONN_ASSERT_LED_PIN);

    #endif

    // m_p_error_file_name = p_file_name;
    // m_error_code = error_code;
    // m_error_line_num = line_num;

    /* Do not reset when warning. */
    if (SER_WARNING_CODE != id)
    {
        /* This call can be used for debug purposes during application development.
        * @note CAUTION: Activating code below will write the stack to flash on an error.
        * This function should NOT be used in a final product.
        * It is intended STRICTLY for development/debugging purposes.
        * The flash write will happen EVEN if the radio is active, thus interrupting any communication.
        * Use with care. Un-comment the line below to use. */

        /* ble_debug_assert_handler(error_code, line_num, p_file_name); */

#ifndef DEBUG
        /* Reset the chip. Should be used in the release version. */
        NVIC_SystemReset();
#else   /* Debug version. */
        /* To be able to see function parameters in a debugger. */
        uint32_t temp = 1;
        while (temp);
#endif

    }

    CRITICAL_REGION_EXIT();
}
/*lint -restore */


/**@brief Callback function for asserts in the SoftDevice.
 *
 * @details This function will be called if the ASSERT macro in the SoftDevice fails. Function
 *          declaration can be found in the nrf_assert.h file.
 *
 * @param[in] line_num    Line number of the failing ASSERT call.
 * @param[in] p_file_name File name of the failing ASSERT call.
 */
void assert_nrf_callback(uint16_t line_num, const uint8_t * p_file_name)
{
    app_error_handler(SER_SD_ERROR_CODE, line_num, p_file_name);
}
/** @} */

So far, I'm not able to reproduce the issue with the connectivity FW I'm building. Can you upload the SDK directory that you used, along with the path to the exact project used to build it?

@paul-bradford: I'm not sure this issue you are now seeing is the same as the original problem. They may look similar to the application, but I never saw the fatal error with my previous testing. I'm not sure if notifications are the cause of this anymore, but we need to find the origin of the fatal error in order to determine what is happening. Hopefully, you should be able to detect that quite fast with the above change I requested from Leo.

[LeoChen]

Hi Jorgen,

The following link is source project that I am using.

https://www.dropbox.com/s/t3pcunczygxxr7k/nRF5_SDK_15.3.0_59ac345_PC_Connect_200505.rar?dl=0

I did compare with the source patched project and can't see any thing wrong.

Would you please also give me the entire SDK that you are using to compare.

The Global setting

Macro:SDK=E:\Keil_v5\ARM\Device\Nordic\nRF5_SDK_15.3.0_59ac345_PC_Connect_200505

The Project path

~\nRF5_SDK_15.3.0_59ac345_PC_Connect_200505\examples\connectivity\ble_connectivity\pca10056\ser_s140_usb_hci\ses

The HEX files that  replace error handling (External Clock does not work with Fanstel USB840M).

https://www.dropbox.com/s/z6u7f227h2sogl7/USB840M0505.rar?dl=0

How long you will see the error happen?

I programmed 3 dongles and running over 4 hours.

I can't see that error on my Win 7  desktop.

Regards,

Leo

[Paul Bradford]

I tested the firmware Leo referenced. With the dev-dongle, I tried both USB840M_200505_ClockInternal.zip and USB840M_200505_ClockExternal.zip and both failed with the "serial port write operation" error after a few minutes.

With the Nordic DK I tried USB840M_200505_ClockInternal_2in1.hex and got "serial port write operation" error after 30 minutes. I attached the firmware logging file 2020-05-05-035347-NordicDK_USB840M_200505_ClockInternal_2in1.hex.txt

The firmware logging ends with 

[00:00:02.638,154] <debug> app: event:BLE_GATTC_EVT_HVX

[00:00:02.643,318] <debug> sphy_hci: TX request (34 bytes)

[00:00:02.648,884] <debug> sphy_hci: Started TX packet (payload 34).

[00:00:02.655,796] <error> app: Fatal error

[00:00:02.659,891] <error> app: ERROR 4 [NRF_ERROR_NO_MEM] at :0

PC at: 0x00000000

[00:00:02.667,605] <error> app: End of error report

[00:00:02.674,529] <info> app: BLE Clock Internal,UART log,Size 64,CRITICAL_REGION,

[00:00:02.682,869] <info> app: USB power detected

[00:00:02.688,683] <info> app: USB ready

I am now testing the Nordic DK with USB840M_200505_ClockExternal_2in1.hex

[Jørgen Holmefjord]

I reproduced the issue with the hex-files from Leo as well. How exactly did you change the scheduler queue size? As far as I can see, this is still 16 in the latest project you uploaded (nRF5_SDK_15.3.0_59ac345_PC_Connect_200505.rar):

// In ser_conn_handlers.h:
/** Maximum number of events in the application scheduler queue. */
#ifdef S112
#define SER_CONN_SCHED_QUEUE_SIZE             8u
#else
#define SER_CONN_SCHED_QUEUE_SIZE             16u
#endif

// In main.c:
APP_SCHED_INIT(SER_CONN_SCHED_MAX_EVENT_DATA_SIZE, SER_CONN_SCHED_QUEUE_SIZE);

And when debugging, it is set to 16 when app_scheduler is initialized:

I created a clean patched SDK, with the appropriate fixes and a few changes. Can you please try building using this? These are the changes (can also be seen in attached patch-file:changes.patch)

• Added Critical region in main.c
• Increased scheduler size to 64
• Removed serialization error handler, use SDK default which will output error info
• Added DEBUG and DEBUG_NRF preprocessor defines to Release build configuration, to catch errors and asserts.
• Added separate build configurations for different LF clocks: Release_LFXO (external crystal) and Release_LFRC (internal RC oscillator).
• Enabled UART logging, and set the log level to only output error logs (debug logs adds heavy CPU load).
• Changed UART log TX pin to P0.31, to allow reading logs from nRF52840 Dongle. When running on DK, a wire must be connected between P0.31 and P0.06 to output logs. P0.31 on the dongle can be connected to P0.06 on an empty DK to read logs.

nRF5_SDK_15.3.0_59ac345_patched_addedfixes.zip

Precompiled hex-files:

• External LF crystal - 200505_ble_connectivity_s140_usb_hci_pca10056_lfxo_mergedsoftdevice.hex
• Internal RC oscillator - 200505_ble_connectivity_s140_usb_hci_pca10056_lfrc_mergedsoftdevice.hex

I have not yet received the Fanstel Dongles, so I'm not sure if these will work with that. You may need to do some changes to support the bootloader etc.