Recover from a sd_softdevice_disable()

[brianreinhold]

When writing to flash it is MUCH easier to perform the write if SoftDevice is disabled. If it is not, one has to wait for an event to continue writing. ON the Nordic platforms, that is very complicated as one already has code for fragmented notifications handling asynchronous waits. The only 'wait' one can perform that is not a CPU busy loop is through sd_app_evt_wait().

Thus I simply disable the softdevice and the write is synchronous; it returns when done. However, now I need to recover so I can start advertising again.

My work flow is:

• Advertise
• Get connected
• Get disconnected
• disable soft device
• write to flash
• ? (what do I need to do here)
• Advertise

My cop-out approach has been after the write to flash to simply call NVIC_SystemReset() and have the whole application start from scratch. That approach has worked fine on the DKs but as soon as I invoke that NVIC_SystemReset() on an nRF52840 dongle, it no longer works. The app simply dies and one needs to power cycle the dongle.

Since NVIC_SystemReset() does not work, how do I recover from an sd_softdevice_disable? What does that disable? The documentation suggests EVERYTHING. Have I lost my service tables? Do I have to re-initialize the BLE stack? How many of the startup 'init' methods do I need to call?

[ovrebekk]

Hi

When writing to flash it is MUCH easier to perform the write if SoftDevice is disabled. If it is not, one has to wait for an event to continue writing. ON the Nordic platforms, that is very complicated as one already has code for fragmented notifications handling asynchronous waits.

Both the SoftDevice and the SDK is event based, and to handle this you could for instance set up some kind of state machine to handle larger operations that require multiple 'call some function -> wait for an event' type operations.

Then this can run in parallel with whatever else the application is doing, and once the state machine gets to the end of its operation it can signal this to the rest of the system. 

The only 'wait' one can perform that is not a CPU busy loop is through sd_app_evt_wait().

If you mean a blocking wait then I guess this is true. Unless some RTOS is used on top of the SoftDevice there is no concept of multi-threading, and blocking wait calls are not recommended. 

That approach has worked fine on the DKs but as soon as I invoke that NVIC_SystemReset() on an nRF52840 dongle, it no longer works.

I assume the factory USB bootloader is present on the dongle?

I have to check this with the DFU experts, but it is possible that the bootloader is causing some issue here. 

Since NVIC_SystemReset() does not work, how do I recover from an sd_softdevice_disable? What does that disable? The documentation suggests EVERYTHING. Have I lost my service tables? Do I have to re-initialize the BLE stack? How many of the startup 'init' methods do I need to call?

Disabling the SoftDevice disables everything, yes. All peripherals are released back to the application, and all the memory previously used by the SoftDevice is also available (except for the 8 first bytes of memory in the SoftDevice region). 

In other words you have to call all the same functions, starting with the sd_softdevice_enable(..) call, and all the subsequent GAP/GATT initialization functions. 

In other words I would strongly suggest trying to handle the flash writes through the SoftDevice instead. As long as the SoftDevice is not doing anything else you should be able to finish the writes almost as quick, unless you issue a lot of small writes. 

Best regards
Torbjørn

[brianreinhold]

I have come a lot further on this. The reason that the NVIC_SystemReset() was not working had nothing to do with that method. The problem is the flash write itself. On the dongle, the flash write kills the application and the dongle dies. It needs a power cycle to restart. If I remove the flash write, everything works the same on the Dongle as the DK.

The flash write is here (note that it works fine on the DK and I have tested it by writing data to the flash and reading it back right after. Well, at least on the DK it works fine - the code is below.

void saveKeysToFlash(ble_gap_sec_keyset_t* keys,
    unsigned char **saveDataBuffer, unsigned short int *saveDataLength,
    bool* cccdSet, unsigned short* noOfCccds)
{
    int i;
//        char buffer[1024];
    unsigned char *keysDataBuffer = NULL;
    ret_code_t err_code;

    uint32_t pg_size = NRF_FICR->CODEPAGESIZE;
    uint32_t pg_num  = NRF_FICR->CODESIZE - 1;  // Use last page in flash
    uint32_t *addr;
    NRF_LOG_DEBUG("Saving data to flash.");

    int size = sizeof(nameKey) +
               sizeof(ble_gap_enc_key_t) +      // should be a fixed length even if no pairing was done
               sizeof(ble_gap_id_key_t) +       // should be a fixed length even if no pairing was done so no IRK
               sizeof(ble_gap_enc_key_t) +
               sizeof(ble_gap_id_key_t) +
               sizeof(unsigned short) +         // Number of CCCDs
               sizeof(bool) * (*noOfCccds) +    // CCCD set
               sizeof(unsigned short) +         // Savedata length
               ((*saveDataBuffer != NULL) ? *saveDataLength * sizeof(unsigned char) : 0);
    // The writes are done in 4-byte hunks so we have to even out the length
    size = 4 + ((size >> 2) << 2);  // Here we round up to the nearest 4-byte size so it is evenly divisible by 4
    NRF_LOG_DEBUG("Size of data to write %u.", size);
    keysDataBuffer = calloc(1, size);
    
    // Now load all the data we want to save into this buffer
    uint8_t *ptr = keysDataBuffer;
    memcpy(ptr, nameKey, sizeof(nameKey));          // Load identifier
    ptr = ptr + sizeof(nameKey);
    memcpy(ptr, keys->keys_own.p_enc_key, sizeof(ble_gap_enc_key_t)); // load our LTK
    ptr = ptr + sizeof(ble_gap_enc_key_t);
    memcpy(ptr, keys->keys_own.p_id_key, sizeof(ble_gap_id_key_t));   // load our LTK id
    ptr = ptr + sizeof(ble_gap_id_key_t);
    memcpy(ptr, keys->keys_peer.p_enc_key, sizeof(ble_gap_enc_key_t)); // load peer LTK
    ptr = ptr + sizeof(ble_gap_enc_key_t);
    memcpy(ptr, keys->keys_peer.p_id_key, sizeof(ble_gap_id_key_t));   // load peer LTK id
    ptr = ptr + sizeof(ble_gap_id_key_t);
    memcpy(ptr, noOfCccds, sizeof(unsigned short));             // Load number of CCCDs
    ptr = ptr + sizeof(unsigned short);
    memcpy(ptr, cccdSet, sizeof(unsigned char) * (*noOfCccds)); // Load the cccdSet[]
    ptr = ptr + sizeof(unsigned char) * (*noOfCccds);
    memcpy(ptr, saveDataLength, sizeof(unsigned short));        // Load saveDataLength
    ptr = ptr + sizeof(unsigned short);
    if (*saveDataBuffer != NULL)
    {
        memcpy(ptr, *saveDataBuffer, *saveDataLength * sizeof(unsigned char));   // Load the saveDataBuffer
        ptr = ptr + *saveDataLength * sizeof(unsigned char);
    }

//    memset(buffer, 0 , 1024);
//    NRF_LOG_DEBUG("Full set of data being written %s", (uint32_t)byteToHex(keysDataBuffer, buffer, " ", size));
//    NRF_LOG_FLUSH();

    // Now we have to write the data in hunks into flash
    // Each page is 1024 bytes, and a write is in 4-byte hunks
    // So we will find the number of 1024 byte pages to write,
    // and then the number of 4-byte hunks left over.

    // Disable soft device so we don't have to deal with events which would make this write PAINFULLY complicated to implement
    //sd_softdevice_disable();
    nrf_sdh_disable_request();
    // Buffer to write to flash. Need to write it in four-byte hunks
    uint32_t *ptr32 = (uint32_t *)keysDataBuffer;
    while (true)
    {
        // Where to write
        addr = (uint32_t *)(pg_size * pg_num);
        // Erase page:
        while(true)
        {
            err_code = sd_flash_page_erase(pg_num);
            if (err_code == NRF_SUCCESS)
            {
                break;
            }
            if (err_code != NRF_ERROR_BUSY)
            {
                NRF_LOG_DEBUG("Erasing data returned error %u.", err_code);
                APP_ERROR_CHECK(err_code);      // Program dies
            }
        }

        i = (size >= pg_size) ? (pg_size >> 2) : // Do as many write that will fill the page, eg if pg size = 1024, then 256 writes
                                (size >> 2);     // All fits in one page, so divide size by four
        NRF_LOG_DEBUG("Number of 4-byte hunks to write %u.", i);
        while(true)
        {
            err_code = sd_flash_write(addr,     // Pointer to start of flash location to be written.
                                      ptr32,    // Pointer to buffer with data to be written.
                                      i);       // Number of 32-bit words to write. Maximum size is the number of words in one
                                                // flash page. See the device's Product Specification for details (pg size I assume).
            if (err_code == NRF_SUCCESS)
            {
                break;
            }
            if (err_code != NRF_ERROR_BUSY)
            {
                NRF_LOG_DEBUG("Writing data returned error %u.", err_code);
                APP_ERROR_CHECK(err_code);      // Program dies
            }
        }
        size = size - pg_size;  // Subtract a page size from the total size
        if (size <= 0)          // if zero or less, all data has been written
        {
            NRF_LOG_DEBUG("Flash written");
            break;
        }
        pg_num++;   // TODO we are on last page, so have to hope everything fits in one page. Still have not figured out how to get
                    // to the start of the first free page available.
        ptr32 = (uint32_t *)(keysDataBuffer + pg_size);
    }
    free(keysDataBuffer);
}

The flash write is to save pairing/bonding info. I happen to have a test app  (RGT Cycle) that connects and if it receives no measurements it disconnects only to reconnect if I advertise. One the DK, I go through endless cycles of this.

On the dongle, the app dies in the flash write (which happens on the disconnect).

I have gone to the point of returning in the above code just before the while loop that does the writes. Works okay on both platforms if I return before attempting any flash operations. 

If I try just the first erase operation, the app dies on the dongle. So its pretty clear its the first attempt to do the erase that kills the app. That is as far as I have come.

In the end if I have to do a flash write, I will call the NVIC_SystemReset(), and if I don't, I will skip the reset and just start advertising. Works great on the DK but if the flash write is needed the dongle dies.

That is where I am at now. A lot of background is here: 

devzone.nordicsemi.com/.../sd_softdevice_is_enabled-returns-false-but-then-nrf_sdh_enable_request-returns-error-8-softdevice-already-enabled

Wondering if I should simply open up a new issue about the flash writes on the dongle vs the DK

[ovrebekk]

Hi 

I read the other case, and the possibility that you are writing to the same page that the bootloader is using for its settings could explain the issues you are seeing on the dongle. 

I suggest you continue the discussion there, since that case has more of the background. 

Best regards
Torbjørn

[brianreinhold]

I ended up posting a separate issue dedicated to just this. And of course I wrote to the same areas. I followed the example provided in the SDK. There is no indication anywhere in the documentation that I stumbled upon to suggest that the pages alluded to in the example would cause problems. There needs to be clear documentation somewhere in the nRF52840 Dongle 'how-to' that there is this critical difference between the DK and the dongle. It has caused me problems for months...but it took a long time to finally identify the problem as being the flash write. It generated issues that on face value appeared to have nothing to do with the flash write (since once cannot debug the dongle).

PS. I still have not validated yet that it is the problem. I will drop down 3 pages and see if that works.

[ovrebekk]

Hi 

I agree this could be described better in the dongle documentation. 

Since the chip is exactly the same any difference has to be down to the internal flash content, or to the difference in external hardware (missing LED's, buttons, external memory chip, debugger etc). 

Hopefully moving the flash content will solve your issue

Best regards
Torbjørn

[brianreinhold]

Yes it did. And I guess the answer to the actual posted question (should I ever need to do it) is that I basically have to start from scratch. In other words, the best option after the disabled soft device call is to do a system reset and reinitialize everything, the bt stack, the services and characteristics, advertisement, etc.

[ovrebekk]

Hi

All the SoftDevice init will need to be repeated, yes, but a full system reset is not really necessary. 

I would recommend designing some kind of Bluetooth handler/manager module that takes care of all the SoftDevice init steps, allowing you to initialize the SoftDevice from a single call to this module in the rest of your code. 

This will speed up the process of disabling, writing to flash, re-enabling the SoftDevice compared to doing a full system reset. 

Best regards
Torbjørn

[brianreinhold]

I was thinking about doing that but basically realized that would be about 90% of my main function.

int main(void)
{
    ret_code_t err_code;
    unsigned char cccds[2];
    bool result;
    m_adv_handle = 0;
    queue = initializeQueue(10);
    if (queue == NULL)
    {
        NRF_LOG_DEBUG("Could not allocate memory for the message queue. Quitting");
        return 0;
    }
    ghs_abort = false;
    send_live_data = false;
    first_cont_sent = false;
    numberOfStoredMsmtGroups = 0;
    global_send.continuous_stage = CONT_NONE;

    // Initialize.
    err_code = NRF_LOG_INIT(NULL);
    APP_ERROR_CHECK(err_code);
    NRF_LOG_DEFAULT_BACKENDS_INIT();
    NRF_LOG_DEBUG("Main start GHS");

    // Allocate memory for the security keys
    allocateMemoryForSecurityKeys(&keys);
    sec_params_init();
    timers_init();

    memset(&m_sccd_ghs_cp_handle, 0, sizeof(m_sccd_ghs_cp_handle));
    memset(&m_sccd_ghs_response_handle, 0, sizeof(m_sccd_ghs_response_handle));

    memset(cccds, 0, noOfCccds);
    loadKeysFromFlash(&keys, &saveDataBuffer, &saveDataLength, cccds, &noOfCccds);
    memcpy(cccdSet, cccds, noOfCccds);  // destination, source, length


    buttons_leds_init();

    gpiote_init_new();

//================================= All stuff fof softdevice init
    // This also provides the callback method to receive events.
    ble_stack_init();
    sd_mutex_new(&p_mutex);
    sd_mutex_new(&q_mutex);

    // Set device address
    ble_gap_addr_t addrStruct;
    addrStruct.addr_type = BLE_GAP_ADDR_TYPE_PUBLIC;
    memcpy(addrStruct.addr, getBtAddress(), 6);  // destination, source, length
    err_code = sd_ble_gap_addr_set(&addrStruct);
    if (err_code != NRF_SUCCESS)
    {
        NRF_LOG_DEBUG("Could not set the Bluetooth Address. Error code %d", err_code);
        APP_ERROR_CHECK(err_code);
    }
    configureSpecializations();

    // Sets max and min connection intervals, slave latency, and the GAP characteristic entries
    // for the friendly name and appearance.
    gap_params_init();
    // Creates the advertisement and scan response data arrays
    err_code = advertisement_ghs_set(&m_adv_handle);
    if (err_code != NRF_SUCCESS)
    {
        NRF_LOG_DEBUG("Could not configure the advertisements");
        APP_ERROR_CHECK(err_code);
    }
    // ===================================== Create the GHS service
    err_code = createPrimaryService(&m_sccd_ghs_service_handle, BTLE_SCCD_GHS_SERVICE);
    if (err_code != NRF_SUCCESS)
    {
        NRF_LOG_DEBUG("Could not create GHS service");
        APP_ERROR_CHECK(err_code);
    }
    // Create the GHS control point characteristic
    err_code = createStandardCharacteristic(m_sccd_ghs_service_handle,
        &m_sccd_ghs_cp_handle,
        BTLE_SCCD_GHS_CP_CHAR,
        true,   // Has CCCD
        true,   // This will cause indicate instead of notify if CCCD is set to true
        0,
        NULL,
        false,
        (ble_gap_conn_sec_mode_t) {1, 1},   // [CCCD write is open]
        (ble_gap_conn_sec_mode_t) {0, 0},   // Reading characteristic value is forbidden
        (ble_gap_conn_sec_mode_t) {1, 1});  // [Writing characteristic is open]
    if (err_code != NRF_SUCCESS)
    {
        NRF_LOG_DEBUG("Could not create GHS control point characteristic");
        APP_ERROR_CHECK(err_code);
    }
    // ===================================== Create the GHS response characteristic
    err_code = createStandardCharacteristic(m_sccd_ghs_service_handle,
        &m_sccd_ghs_response_handle,
        BTLE_SCCD_GHS_RESPONSE_CHAR,
        true,   // Has CCCD
        false,  // This will cause indicate instead of notify if CCCD is set to true
        0,
        NULL,
        false,
        (ble_gap_conn_sec_mode_t)
        {
            1, 1
        },
        // CCCD write is open
        (ble_gap_conn_sec_mode_t)
        {
            0, 0
        },
            // Reading characteristic value is forbidden
        (ble_gap_conn_sec_mode_t)
        {
            1, 1
        });  // Writing characteristic is open
    if (err_code != NRF_SUCCESS)
    {
        NRF_LOG_DEBUG("Could not create GHS response characteristic");
        APP_ERROR_CHECK(err_code);
    }
    // Not sure about this yet. I think it allows one to put up a fight to argue 
    // over connection parameters and sets up responses to connection parameter
    // change requests.
    //conn_params_init();
//=============================== End of SoftDevice init???
    // Start execution.
    err_code = app_timer_start(m_app_dummy_timer_id, CMD_SENSOR_TIME, NULL);
    APP_ERROR_CHECK(err_code);
    elapsedTimeStart = app_timer_cnt_get();
    #if (USES_TIMESTAMP == 1)
        if (sTimeInfo->timeFlagsSupport == 1)
        {
            sGhsTime->epoch = (unsigned long long)elapsedTimeStart;
        }
        else if(sTimeInfo->timeFlagsSupport > 1)
        {
            epoch = GHS_TIME_EPOCH_DEFAULT;
            sGhsTime->epoch = (unsigned long long)elapsedTimeStart + epoch;
        }
    #endif
    NRF_LOG_INFO("GHS Start at time %u!\r", getTicks());
    #if (USE_DK == 0)
        bring_up_adver();
    #endif
    // Since we don't have continuous power we cannot maintain a real time clock between power cycles
    // So each power cycle will be treated as a factory reset where we start a new wall clock time
    // of a facory reset value. We will then create stored measurements with a button push and use our
    // tick counter to add to this factory reset time
    //=============================================================================== Start GHS Pulse OX
    // Enter main loop.
    main_loop();
}

If I understand how much that would be it looks like all the code between my doft device init start comment and soft device init end comment. I suppose there is a lot of stuff that happens on a system reset that I don't see so even though 90 % of my main function would be the softdevice init and setup of all the BTLE stuff it would still be more efficient.