DFU approach from cloud to ESP32 to nRF52840

[JasonGeo]

Hi,

My setup include a nRF52840-DK board that has an ESP32 WiFi module connected to it via UART. The idea is to source the new firmware from a server located in the cloud.

GCC toolchain,

Running FreeRTOS,

SDK 15.2.0

I am getting really confused after reading all the information on DFU and how I should be doing it. Simon also recently posted an article on DFU using Connect SDK https://devzone.nordicsemi.com/guides/nrf-connect-sdk-guides/b/software/posts/ncs-dfu

However, we are not using NCS at the moment.

Questions that I asked myself or still confused about are:

1. Should I be using background DFU or should I be modifying the bootloader to perform the DFU?
2. Where do I get the bin file from? I have a .zip package from the build output. Is that the bin file to use?
3. What is an init packet? Is that the .dat file that comes in the .zip package? How do you trigger the init packet? Via application or bootloader?
4. What is this bootloader settings file?
5. If I am using dual bank update, how do I specify the flash address for the second bank where the updated firmware gets written to?
6. How to let the bootloader knows that there is a new firmware in the second bank for validation and swap over? 
7. What transport should I be looking at? I assume serial - UART?

My initial approach is to write my own in a simplified manner.

The application will issue HTTP GET to the server and retrieve data chunks of the binary until the whole bin file is received. As it receives the chunks, it writes to a flash location.

After the whole bin file is received, the CPU reboots and the application copies the new firmware, overriding the old firmware in place.

Will that "upset" the bootloader when it reboots next time seeing that the firmware signature is different from the one being overwritten?

This is not foolproof as there is no validation and checks for data corruption. It's also not the "proper" way that Nordic perform DFU.

I hope someone can help with those questions and shed some light on the best method to approach this (given my setup, NRF52-ESP32-cloud).

What are the articles that I should focus reading up on. 

[Hung Bui]

Hi Jason, 

If you are working with nRF5SDK then the article from Simon is not relevant as it's applied for NCS SDK. 

Could you let me know how you are setting up the ESP and the NRF52 in your application ? Where the main application running on ? Are you using the ESP32 as the connectivity chip or the main application running on it? This would decide how the background DFU should work in your case. 

You mentioned "I have a .zip package from the build output". I'm not aware of a toolchain that can build the .zip output, could you elaborate more how your build system consist of ? Usually we use nrfutil.exe tool to generate .zip file. 

Regarding your question about bootloader setting you can have a look here. I have a blog post here that covers a little bit about bootloader settting and also give you some explanation on how the bootloader works. You can also find the link to a background DFU example at the end of the blog. 

What we do in the background DFU approach is to receive the image to bank 1 (swap bank , defined in the bootloader setting) and after the image is fully received, write to the bootloader setting about the progress and then reset to the bootloader to do post validation and the actual image swap. 

You can also find the Background DFU example in Thread and Zigbee SDK where we receive the image via Thread/Zigbee and then reset to bootloader to do the swap. 

[JasonGeo]

Hi Hung, thanks for your support.

The ESP32 connects to the nrf52840 DK board via UART and the nrf52 issues AT commands to it to send and receive data to the cloud. The main application runs on the nrf52 and ESP32 is purely for connectivity. 

The build system we are using is CMake with GCC and at the end of the CMakeList file, there are rules to make DFU archives using the nrfutil.

In the light of this, can you recommend the best approach to undertake for our system? Will it be background DFU? If so, I will focus on making background DFU happen.

From your suggestion, it seems like we do not need to modify the bootloader code, only the bootloader settings need to be modified? Where can I find the example code for a suitable bootloader?

Thanks for the links, I will have a good read about them.

Hope to hear from you soon.

[Hung Bui]

Hi Jason, 

You are right. In your case the nRF52 application should do the main job of getting the image and storing it into bank1 memory (background DFU). 

And it's correct that you don't need to modify the bootloader for the task as it should be compatible except for declaring NRF_DFU_IN_APP=1 in sdk_config.h
What you need to do in the application is to write to bootloader setting when you are done receiving the image. 

I haven't looked into the background DFU example for awhile, but my suggestion is to look at these functions: 

- nrf_dfu_req_handler_on_req() inside that on_data_obj_execute_request_sched() is important, it's where the image trunks are received and once it's fully received, nrf_dfu_validation_post_data_execute() -> postvalidate() will be called. 
When you reset the bootloader the bootloader will do this: 

It will start the progress of swapping the image.

In the application, settings need to be written nrf_dfu_settings_write_and_backup() inside on_data_obj_execute_request_sched() 

[JasonGeo]

Thanks Hung, that should get me off the ground.

Couple of questions.

1. How do I set or find where Bank1 address is allocated in Flash memory? Because the app needs to point to the starting address to start writing to it and also to be sure it's at least after main application size in flash.

2. Maybe because I have 2 sdk_config.h files, one used for the main app and the other for the bootloader project, however I can't find the symbol NRF_DFU_IN_APP=1 in both configs.

3. Do I set the following to 1 in both sdk_config.h files for app and bootloader?

NRF_DFU_SETTINGS_ALLOW_UPDATE_FROM_APP

NRF_DFU_SETTINGS_IN_APP

NRF_DFU_FORCE_DUAL_BANK_APP_UPDATES

They are currently set to zero.

4. On my copy of on_data_obj_execute_request_sched(), where is the part that's receiving the image chunks and writing to flash? I am having trouble finding those operations. Also it mentioned about scheduler, is the function using back the same FreeRTOS scheduler I have running for the application or is it a specific one provided by Nordic SDK?

 

5. In my copy of nrf_bootloader.c module, I don't see the lines you highlighted in the picture. Maybe I have a different version from yours? I've attached a copy of mine.

Fullscreen 4442.nrf_bootloader.c Download

/**
 * Copyright (c) 2016 - 2018, Nordic Semiconductor ASA
 *
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without modification,
 * are permitted provided that the following conditions are met:
 *
 * 1. Redistributions of source code must retain the above copyright notice, this
 *    list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form, except as embedded into a Nordic
 *    Semiconductor ASA integrated circuit in a product or a software update for
 *    such product, must reproduce the above copyright notice, this list of
 *    conditions and the following disclaimer in the documentation and/or other
 *    materials provided with the distribution.
 *
 * 3. Neither the name of Nordic Semiconductor ASA nor the names of its
 *    contributors may be used to endorse or promote products derived from this
 *    software without specific prior written permission.
 *
 * 4. This software, with or without modification, must only be used with a
 *    Nordic Semiconductor ASA integrated circuit.
 *
 * 5. Any software provided in binary form under this license must not be reverse
 *    engineered, decompiled, modified and/or disassembled.
 *
 * THIS SOFTWARE IS PROVIDED BY NORDIC SEMICONDUCTOR ASA "AS IS" AND ANY EXPRESS
 * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 * OF MERCHANTABILITY, NONINFRINGEMENT, AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL NORDIC SEMICONDUCTOR ASA OR CONTRIBUTORS BE
 * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
 * GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
 * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 *
 */
#include "nrf_bootloader.h"

#include "compiler_abstraction.h"
#include "nrf.h"
#include "boards.h"
#include "sdk_config.h"
#include "nrf_power.h"
#include "nrf_delay.h"
#include "nrf_log.h"
#include "nrf_log_ctrl.h"
#include "nrf_dfu.h"
#include "nrf_error.h"
#include "nrf_dfu_settings.h"
#include "nrf_dfu_utils.h"
#include "nrf_bootloader_wdt.h"
#include "nrf_bootloader_info.h"
#include "nrf_bootloader_app_start.h"
#include "nrf_bootloader_fw_activation.h"
#include "nrf_bootloader_dfu_timers.h"
#include "app_scheduler.h"

static nrf_dfu_observer_t m_user_observer; //<! Observer callback set by the user.
static volatile bool m_flash_write_done;

#define SCHED_QUEUE_SIZE      32          /**< Maximum number of events in the scheduler queue. */
#define SCHED_EVENT_DATA_SIZE NRF_DFU_SCHED_EVENT_DATA_SIZE /**< Maximum app_scheduler event size. */

#if !(defined(NRF_BL_DFU_ENTER_METHOD_BUTTON)    && \
      defined(NRF_BL_DFU_ENTER_METHOD_PINRESET)  && \
      defined(NRF_BL_DFU_ENTER_METHOD_GPREGRET)  && \
      defined(NRF_BL_DFU_ENTER_METHOD_BUTTONLESS))
    #error Configuration file is missing flags. Update sdk_config.h.
#endif

STATIC_ASSERT((NRF_BL_DFU_INACTIVITY_TIMEOUT_MS >= 100) || (NRF_BL_DFU_INACTIVITY_TIMEOUT_MS == 0),
             "NRF_BL_DFU_INACTIVITY_TIMEOUT_MS must be 100 ms or more, or 0 to indicate that it is disabled.");

#if defined(NRF_LOG_BACKEND_FLASH_START_PAGE)
STATIC_ASSERT(NRF_LOG_BACKEND_FLASH_START_PAGE != 0, 
    "If nrf_log flash backend is used it cannot use space after code because it would collide with settings page.");
#endif

/**@brief Weak implemenation of nrf_dfu_init
 *
 * @note   This function will be overridden if nrf_dfu.c is
 *         compiled and linked with the project
 */
 #if (__LINT__ != 1)
__WEAK uint32_t nrf_dfu_init(nrf_dfu_observer_t observer)
{
    NRF_LOG_DEBUG("in weak nrf_dfu_init");
    return NRF_SUCCESS;
}
#endif


/**@brief Weak implementation of nrf_dfu_init
 *
 * @note    This function must be overridden in application if
 *          user-specific initialization is needed.
 */
__WEAK uint32_t nrf_dfu_init_user(void)
{
    NRF_LOG_DEBUG("in weak nrf_dfu_init_user");
    return NRF_SUCCESS;
}


static void flash_write_callback(void * p_context)
{
    UNUSED_PARAMETER(p_context);
    m_flash_write_done = true;
}


static void reset_after_flash_write(void * p_context)
{
    UNUSED_PARAMETER(p_context);

    NRF_LOG_FINAL_FLUSH();

#if NRF_MODULE_ENABLED(NRF_LOG_BACKEND_RTT)
    // To allow the buffer to be flushed by the host.
    nrf_delay_ms(100);
#endif

    NVIC_SystemReset();
}


static void bootloader_reset(void)
{
    NRF_LOG_DEBUG("Resetting bootloader.");

    m_flash_write_done = false;
    nrf_dfu_settings_backup(reset_after_flash_write);
}


static void inactivity_timeout(void)
{
    NRF_LOG_INFO("Inactivity timeout.");
    bootloader_reset();
}


/**@brief Function for handling DFU events.
 */
static void dfu_observer(nrf_dfu_evt_type_t evt_type)
{
    switch (evt_type)
    {
        case NRF_DFU_EVT_DFU_STARTED:
        case NRF_DFU_EVT_OBJECT_RECEIVED:
            nrf_bootloader_dfu_inactivity_timer_restart(
                        NRF_BOOTLOADER_MS_TO_TICKS(NRF_BL_DFU_INACTIVITY_TIMEOUT_MS),
                        inactivity_timeout);
            break;
        case NRF_DFU_EVT_DFU_COMPLETED:
        case NRF_DFU_EVT_DFU_ABORTED:
            bootloader_reset();
            break;
        default:
            break;
    }

    if (m_user_observer)
    {
        m_user_observer(evt_type);
    }
}


/**@brief Function for initializing the event scheduler.
 */
static void scheduler_init(void)
{
    APP_SCHED_INIT(SCHED_EVENT_DATA_SIZE, SCHED_QUEUE_SIZE);
}


/**@brief Suspend the CPU until an interrupt occurs.
 */
static void wait_for_event(void)
{
#ifdef BLE_STACK_SUPPORT_REQD
    (void)sd_app_evt_wait();
#else
    // Wait for an event.
    __WFE();
    // Clear the internal event register.
    __SEV();
    __WFE();
#endif
}


/**@brief Continually sleep and process tasks whenever woken.
 */
static void loop_forever(void)
{
    while (true)
    {
        //feed the watchdog if enabled.
        nrf_bootloader_wdt_feed();

        app_sched_execute();

        if (!NRF_LOG_PROCESS())
        {
            wait_for_event();
        }
    }
}

/**@brief Function for initializing button used to enter DFU mode.
 */
static void dfu_enter_button_init(void)
{
    nrf_gpio_cfg_sense_input(NRF_BL_DFU_ENTER_METHOD_BUTTON_PIN,
                             BUTTON_PULL,
                             NRF_GPIO_PIN_SENSE_LOW);
}


static bool crc_on_valid_app_required(void)
{
    bool ret = true;
    if (NRF_BL_APP_CRC_CHECK_SKIPPED_ON_SYSTEMOFF_RESET &&
        (nrf_power_resetreas_get() & NRF_POWER_RESETREAS_OFF_MASK))
    {
        nrf_power_resetreas_clear(NRF_POWER_RESETREAS_OFF_MASK);
        ret = false;
    }
    else if (NRF_BL_APP_CRC_CHECK_SKIPPED_ON_GPREGRET2 &&
            (nrf_power_gpregret2_get() & BOOTLOADER_DFU_SKIP_CRC))
    {
        nrf_power_gpregret2_set(nrf_power_gpregret2_get() & ~BOOTLOADER_DFU_SKIP_CRC);
        ret = false;
    }
    else
    {
    }

    return ret;
}


/**@brief Function for clearing all DFU enter flags that
 *        preserve state during reset.
 *
 * @details This is used to make sure that each of these flags
 *          is checked only once after reset.
 */
static void dfu_enter_flags_clear(void)
{
    if (NRF_BL_DFU_ENTER_METHOD_PINRESET &&
       (NRF_POWER->RESETREAS & POWER_RESETREAS_RESETPIN_Msk))
    {
        // Clear RESETPIN flag.
        NRF_POWER->RESETREAS |= POWER_RESETREAS_RESETPIN_Msk;
    }

    if (NRF_BL_DFU_ENTER_METHOD_GPREGRET &&
       (nrf_power_gpregret_get() & BOOTLOADER_DFU_START))
    {
        // Clear DFU mark in GPREGRET register.
        nrf_power_gpregret_set(nrf_power_gpregret_get() & ~BOOTLOADER_DFU_START);
    }

    if (NRF_BL_DFU_ENTER_METHOD_BUTTONLESS &&
       (s_dfu_settings.enter_buttonless_dfu == 1))
    {
        // Clear DFU flag in flash settings.
        s_dfu_settings.enter_buttonless_dfu = 0;
        APP_ERROR_CHECK(nrf_dfu_settings_write(NULL));
    }
}


/**@brief Function for checking whether to enter DFU mode or not.
 */
static bool dfu_enter_check(void)
{
    if (!nrf_dfu_app_is_valid(crc_on_valid_app_required()))
    {
        NRF_LOG_DEBUG("DFU mode because app is not valid.");
        return true;
    }

    if (NRF_BL_DFU_ENTER_METHOD_BUTTON &&
       (nrf_gpio_pin_read(NRF_BL_DFU_ENTER_METHOD_BUTTON_PIN) == 0))
    {
        NRF_LOG_DEBUG("DFU mode requested via button.");
        return true;
    }

    if (NRF_BL_DFU_ENTER_METHOD_PINRESET &&
       (NRF_POWER->RESETREAS & POWER_RESETREAS_RESETPIN_Msk))
    {
        NRF_LOG_DEBUG("DFU mode requested via pin-reset.");
        return true;
    }

    if (NRF_BL_DFU_ENTER_METHOD_GPREGRET &&
       (nrf_power_gpregret_get() & BOOTLOADER_DFU_START))
    {
        NRF_LOG_DEBUG("DFU mode requested via GPREGRET.");
        return true;
    }

    if (NRF_BL_DFU_ENTER_METHOD_BUTTONLESS &&
       (s_dfu_settings.enter_buttonless_dfu == 1))
    {
        NRF_LOG_DEBUG("DFU mode requested via bootloader settings.");
        return true;
    }

    return false;
}


ret_code_t nrf_bootloader_init(nrf_dfu_observer_t observer)
{
    NRF_LOG_DEBUG("In nrf_bootloader_init");

    ret_code_t                            ret_val;
    nrf_bootloader_fw_activation_result_t activation_result;
    uint32_t                              initial_timeout;
    bool                                  dfu_enter = false;

    m_user_observer = observer;

    if (NRF_BL_DFU_ENTER_METHOD_BUTTON)
    {
        dfu_enter_button_init();
    }

    ret_val = nrf_dfu_settings_init(false);
    if (ret_val != NRF_SUCCESS)
    {
        return NRF_ERROR_INTERNAL;
    }

    // Check if an update needs to be activated and activate it.
    activation_result = nrf_bootloader_fw_activate();

    switch (activation_result)
    {
        case ACTIVATION_NONE:
            initial_timeout = NRF_BOOTLOADER_MS_TO_TICKS(NRF_BL_DFU_INACTIVITY_TIMEOUT_MS);
            dfu_enter       = dfu_enter_check();
            break;

        case ACTIVATION_SUCCESS_EXPECT_ADDITIONAL_UPDATE:
            initial_timeout = NRF_BOOTLOADER_MS_TO_TICKS(NRF_BL_DFU_CONTINUATION_TIMEOUT_MS);
            dfu_enter       = true;
            break;

        case ACTIVATION_SUCCESS:
            bootloader_reset();
            NRF_LOG_ERROR("Should never come here: After bootloader_reset()");
            return NRF_ERROR_INTERNAL; // Should not reach this.

        case ACTIVATION_ERROR:
        default:
            return NRF_ERROR_INTERNAL;
    }

    if (dfu_enter)
    {
        nrf_bootloader_wdt_init();

        scheduler_init();

        // Clear all DFU stop flags.
        dfu_enter_flags_clear();

        // Call user-defined init function if implemented
        ret_val = nrf_dfu_init_user();
        if (ret_val != NRF_SUCCESS)
        {
            return NRF_ERROR_INTERNAL;
        }

        nrf_bootloader_dfu_inactivity_timer_restart(initial_timeout, inactivity_timeout);

        ret_val = nrf_dfu_init(dfu_observer);
        if (ret_val != NRF_SUCCESS)
        {
            return NRF_ERROR_INTERNAL;
        }

        NRF_LOG_DEBUG("Enter main loop");
        loop_forever(); // This function will never return.
        NRF_LOG_ERROR("Should never come here: After looping forever.");
    }
    else
    {
        // Erase additional data like peer data or advertisement name
        ret_val = nrf_dfu_settings_additional_erase();
        if (ret_val != NRF_SUCCESS)
        {
            return NRF_ERROR_INTERNAL;
        }

        m_flash_write_done = false;
        nrf_dfu_settings_backup(flash_write_callback);
        ASSERT(m_flash_write_done);

        nrf_bootloader_app_start();
        NRF_LOG_ERROR("Should never come here: After nrf_bootloader_app_start()");
    }

    // Should not be reached.
    return NRF_ERROR_INTERNAL;
}

[Hung Bui]

Hi Jason, 

I think we need to align the SDK version that you are using. There were some change between SDK v15.2 and SDK v15.3
I was looking at SDK v15.3 . And I would suggest to port from v15.2 to SDK v15.3 , Here is what I have in on_data_obj_execute_request_sched()

It's important that you find a working example of background DFU and can actually do some background DFU to test. After we can use it as the golden sample to port it to your application. 

I'm not sure if I have pointed you to the example by my coworker Vidar in my blog here.  Could you please try to test if it works for you ? 

[JasonGeo]

Thanks Hung, I will see if I can update my SDK version.

Can you help with question 1 please?