OFF CHIP OTA UPDATE using external flash

[Priyesh Shahi]

I am working on this project off chip ota update on nRF5_SDK  using ecample/thread/dfu

by now, i have done changes in these functions on_data_obj_create_request() and on_data_obj_write_request(). initially these functions were erasing and writing data in internal flash which I changed to qspi functions like nrf_drv_qspi_erase() and nrf_drv_qspi_write(). and it works perfectly.

Later i changed functions for hash verification which is also working but i am not sure about what should be the next step so that bootloader changes the earlier application with the new firmware image.

[tesc]

Hi,

Unfortunately our experts on this matter are out-of-office today. We expect to get back to you beginning of next week. If you have any progress in the mean time then please update this ticket with your findings.

Regards,
Terje

[Priyesh Shahi]

I have made three changes in code 

1: in flash erase i have changed it to erase external flash using QSPI.

static void on_data_obj_create_request(nrf_dfu_request_t * p_req, nrf_dfu_response_t * p_res)
{
    NRF_LOG_DEBUG("Handle NRF_DFU_OP_OBJECT_CREATE (data)");

    if (!nrf_dfu_validation_init_cmd_present())
    {
        /* Can't accept data because DFU isn't initialized by init command. */
        NRF_LOG_ERROR("Cannot create data object without valid init command");
        p_res->result = NRF_DFU_RES_CODE_OPERATION_NOT_PERMITTED;
        return;
    }

    if (p_req->create.object_size == 0)
    {
        NRF_LOG_ERROR("Object size cannot be 0.")
        p_res->result = NRF_DFU_RES_CODE_INVALID_PARAMETER;
        return;
    }

    if (  ((p_req->create.object_size & (CODE_PAGE_SIZE - 1)) != 0)
        && (s_dfu_settings.progress.firmware_image_offset_last + p_req->create.object_size != m_firmware_size_req))
    {
        NRF_LOG_ERROR("Object size must be page aligned");
        ota_error_code = 1;
        p_res->result = NRF_DFU_RES_CODE_INVALID_PARAMETER;
        return;
    }

    if (p_req->create.object_size > DATA_OBJECT_MAX_SIZE)
    {
        /* It is impossible to handle the command because the size is too large */
        NRF_LOG_ERROR("Invalid size for object (too large)");
        p_res->result = NRF_DFU_RES_CODE_INSUFFICIENT_RESOURCES;
        return;
    }

    if ((s_dfu_settings.progress.firmware_image_offset_last + p_req->create.object_size) >
        m_firmware_size_req)
    {
        NRF_LOG_ERROR("Creating the object with size 0x%08x would overflow firmware size. "
                      "Offset is 0x%08x and firmware size is 0x%08x.",
                      p_req->create.object_size,
                      s_dfu_settings.progress.firmware_image_offset_last,
                      m_firmware_size_req);

        p_res->result = NRF_DFU_RES_CODE_OPERATION_NOT_PERMITTED;
        ota_error_code = 2;
        return;
    }

    s_dfu_settings.progress.data_object_size      = p_req->create.object_size;
    s_dfu_settings.progress.firmware_image_crc    = s_dfu_settings.progress.firmware_image_crc_last;
    s_dfu_settings.progress.firmware_image_offset = s_dfu_settings.progress.firmware_image_offset_last;
    s_dfu_settings.write_offset                   = s_dfu_settings.progress.firmware_image_offset_last;

    /* Erase the page we're at. */
    if (nrf_dfu_flash_erase((m_firmware_start_addr + s_dfu_settings.progress.firmware_image_offset),
                            CEIL_DIV(p_req->create.object_size, CODE_PAGE_SIZE), NULL) != NRF_SUCCESS)
    {
        NRF_LOG_ERROR("Erase operation failed");
        p_res->result = NRF_DFU_RES_CODE_INVALID_OBJECT;
        return;
    }

    NRF_LOG_DEBUG("Creating object with size: %d. Offset: 0x%08x, CRC: 0x%08x",
                 s_dfu_settings.progress.data_object_size,
                 s_dfu_settings.progress.firmware_image_offset,
                 s_dfu_settings.progress.firmware_image_crc);
    
   NRF_LOG_INFO("Creating object with size: %d.  Offset: 0x%08x, CRC: 0x%08x\r\n",
                 s_dfu_settings.progress.data_object_size,
                 s_dfu_settings.progress.firmware_image_offset,
                 s_dfu_settings.progress.firmware_image_crc);


      /* check if flash is enabled */
        if(is_flash_enable() == false)
        {
          NRF_LOG_INFO("Flash in sleep state. Reinintialising\r\n");

          Enable_Flash();
        }

        if(nrf_drv_qspi_erase(NRF_QSPI_ERASE_LEN_4KB, FIRMWARE_FLASH_ADDR + (s_dfu_settings.progress.firmware_image_offset)) != NRFX_SUCCESS)
        {
          NRF_LOG_INFO("Failed to rrase data blocks in Ext Flash.\r\n");
        }
        else
        {
          NRF_LOG_INFO("Succsessfully Erased Ext Flash block at: 0x%X\r\n\r\n",FIRMWARE_FLASH_ADDR + (s_dfu_settings.progress.firmware_image_offset));
          WAIT_FOR_PERIPH();
          nrf_delay_ms(500);
        }


}

 

2: in flash store I have changed to write the code in external flash using QSPI.

static void on_data_obj_write_request(nrf_dfu_request_t * p_req, nrf_dfu_response_t * p_res)
{
    NRF_LOG_DEBUG("Handle NRF_DFU_OP_OBJECT_WRITE (data)");

    if (!nrf_dfu_validation_init_cmd_present())
    {
        /* Can't accept data because DFU isn't initialized by init command. */
        p_res->result = NRF_DFU_RES_CODE_OPERATION_NOT_PERMITTED;
        return;
    }

    uint32_t const data_object_offset = s_dfu_settings.progress.firmware_image_offset -
                                        s_dfu_settings.progress.firmware_image_offset_last;

    if ((p_req->write.len + data_object_offset) > s_dfu_settings.progress.data_object_size)
    {
        /* Can't accept data because too much data has been received. */
        NRF_LOG_ERROR("Write request too long");
        p_res->result = NRF_DFU_RES_CODE_INVALID_PARAMETER;
        return;
    }



    uint32_t const write_addr = m_firmware_start_addr + s_dfu_settings.write_offset;
    /* CRC must be calculated before handing off the data to fstorage because the data is
     * freed on write completion.
     */
    uint32_t const next_crc =
        crc32_compute(p_req->write.p_data, p_req->write.len, &s_dfu_settings.progress.firmware_image_crc);

    ASSERT(p_req->callback.write);

    ret_code_t ret = NRF_ERROR_BASE_NUM;
      //ret_code_t ret =
      //  nrf_dfu_flash_store(write_addr, p_req->write.p_data, p_req->write.len, p_req->callback.write);

        
        /* check if flash is enabled */
        if(is_flash_enable() == false)
        {
          NRF_LOG_INFO("Flash in sleep state. Reinintialising\r\n");

          Enable_Flash();
        }
        
        
        if(nrf_drv_qspi_write(p_req->write.p_data, p_req->write.len,FIRMWARE_FLASH_ADDR + s_dfu_settings.write_offset) != NRFX_SUCCESS)
        {
          NRF_LOG_INFO("\r\nFailed storing data blocks in Ext Flash.\r\n");
        }
        else
        {
          NRF_LOG_INFO("Succsessfully storing data block at: 0x%X\r\n\r\n",FIRMWARE_FLASH_ADDR + s_dfu_settings.write_offset);
          WAIT_FOR_PERIPH();
          p_req->callback.write((void*)p_req->write.p_data);
          ret = NRF_SUCCESS;
        }

    if (ret != NRF_SUCCESS)
    {
        /* When nrf_dfu_flash_store() fails because there is no space in the queue,
         * stop processing the request so that the peer can detect a CRC error
         * and retransmit this object. Remember to manually free the buffer !
         */
        p_req->callback.write((void*)p_req->write.p_data);
        return;
    }

    /* Update the CRC of the firmware image. */
    s_dfu_settings.write_offset                   += p_req->write.len;
    s_dfu_settings.progress.firmware_image_offset += p_req->write.len;
    s_dfu_settings.progress.firmware_image_crc     = next_crc;

    /* This is only used when the PRN is triggered and the 'write' message
     * is answered with a CRC message and these field are copied into the response.
     */
    p_res->write.crc    = s_dfu_settings.progress.firmware_image_crc;
    p_res->write.offset = s_dfu_settings.progress.firmware_image_offset;
}

3: in calculate hash i have changed it to copy data from QSPI and update the hash accordingly.

ret_code_t nrf_crypto_hash_calculate(nrf_crypto_hash_context_t    * const p_context,
                                     nrf_crypto_hash_info_t       const * p_info,
                                     uint8_t                      const * p_data,
                                     size_t                               data_size,
                                     uint8_t                            * p_digest,
                                     size_t                       * const p_digest_size)
{
    ret_code_t                      ret_val;
    nrf_crypto_hash_context_t     * p_ctx  = (nrf_crypto_hash_context_t *)p_context;
    void                          * p_allocated_context = NULL;

// Internal allocation of context not available for CC310_BL in order to save code size.
#if defined(NRF_CRYPTO_BACKEND_CC310_BL_HASH_SHA256_ENABLED) && (NRF_CRYPTO_BACKEND_CC310_BL_HASH_SHA256_ENABLED == 1)
    
    // Do nothing
    
#elif defined(NRF_CRYPTO_BACKEND_CC310_BL_HASH_SHA256_ENABLED) && (NRF_CRYPTO_BACKEND_CC310_BL_HASH_SHA256_ENABLED == 0)
    
    // Validate input. Only validate input parameters that are used locally, others are validated
    // in the init, update and/or finalize functions.
    VERIFY_TRUE(p_info != NULL, NRF_ERROR_CRYPTO_INPUT_NULL);

    // Allocate context if needed (not provided by the user).
    if (p_context == NULL)
    {
        p_allocated_context = NRF_CRYPTO_ALLOC(p_info->context_size);
        if (p_allocated_context == NULL)
        {
            return NRF_ERROR_CRYPTO_ALLOC_FAILED;
        }
        p_ctx = (nrf_crypto_hash_context_t *)p_allocated_context;
    }
    
#else

    #warning NRF_CRYPTO_BACKEND_CC310_BL_HASH_SHA256_ENABLED define not found in sdk_config.h (Is the sdk_config.h valid?).
    
#endif // NRF_CRYPTO_BACKEND_CC310_BL_HASH_SHA256_ENABLED

    ret_val = nrf_crypto_hash_init(p_ctx, p_info);
    NRF_CRYPTO_VERIFY_SUCCESS_DEALLOCATE(ret_val, p_allocated_context);


    if(validation_state == 0)
    {
        ret_val = nrf_crypto_hash_update(p_ctx, p_data, data_size);
        NRF_CRYPTO_VERIFY_SUCCESS_DEALLOCATE(ret_val, p_allocated_context);
        validation_state = 1;
    }
    else
    {
        uint16_t firmware_blocks = data_size / FIRMWARE_BLOCK_SIZE;
        uint8_t rest_block = data_size % FIRMWARE_BLOCK_SIZE;


        printf("data_size: %d", data_size);
        printf("firmware_blocks: %d", firmware_blocks);
        printf("rest_block: %d", rest_block);


        for(uint16_t i=0; i <= firmware_blocks; i++)
        {
          /* check if flash is enabled */
            if(is_flash_enable() == false)
            {
                printf("Flash in sleep state. Reinintialising\r\n");

                Enable_Flash();
            }
        
            uint8_t m_buffer_rx[FIRMWARE_BLOCK_SIZE];

            if(i == firmware_blocks)
            {
                if(nrf_drv_qspi_read(m_buffer_rx, rest_block,FIRMWARE_FLASH_ADDR + (FIRMWARE_BLOCK_SIZE * i)) != NRFX_SUCCESS)
                {
                   printf("\r\nFailed read data blocks in Ext Flash.\r\n");
                }
                else
                {
                   printf("Succsessfully read rest_block data block from: 0x%X\r\n\r\n",FIRMWARE_FLASH_ADDR + (FIRMWARE_BLOCK_SIZE * i));
                   WAIT_FOR_PERIPH();
                   ret_val = nrf_crypto_hash_update(p_ctx, m_buffer_rx, rest_block);
                   NRF_CRYPTO_VERIFY_SUCCESS_DEALLOCATE(ret_val, p_allocated_context);
                }
            }
            else
            {
                if(nrf_drv_qspi_read(m_buffer_rx, FIRMWARE_BLOCK_SIZE,FIRMWARE_FLASH_ADDR + (FIRMWARE_BLOCK_SIZE * i)) != NRFX_SUCCESS)
                {
                    printf("\r\nFailed read data blocks in Ext Flash.\r\n");
                }
                else
                {
                    printf("Succsessfully read data block %d from: 0x%X\r\n\r\n", i ,FIRMWARE_FLASH_ADDR + (FIRMWARE_BLOCK_SIZE * i));
                    WAIT_FOR_PERIPH();
                    ret_val = nrf_crypto_hash_update(p_ctx, m_buffer_rx, FIRMWARE_BLOCK_SIZE);
                    NRF_CRYPTO_VERIFY_SUCCESS_DEALLOCATE(ret_val, p_allocated_context);
                }
            }            
        }
        validation_state = 0;
    }

    ret_val = nrf_crypto_hash_finalize(p_ctx, p_digest, p_digest_size);
    NRF_CRYPTO_VERIFY_SUCCESS_DEALLOCATE(ret_val, p_allocated_context);

#if !NRF_MODULE_ENABLED(NRF_CRYPTO_BACKEND_CC310_BL_HASH_SHA256)
    // Free context if allocated internally
    if (p_allocated_context != NULL)
    {
        NRF_CRYPTO_FREE(p_allocated_context);
    }
#endif // !NRF_MODULE_ENABLED(NRF_CRYPTO_BACKEND_CC310_BL_HASH_SHA256)

    return NRF_SUCCESS;
}

#endif // NRF_MODULE_ENABLED(NRF_CRYPTO_HASH)

#endif // NRF_MODULE_ENABLED(NRF_CRYPTO)

and after that few more changes to get notified that hash has been verified and also to find out flow of the code 

1

#if NRF_DFU_IN_APP
        res.result = nrf_dfu_validation_post_data_execute(m_firmware_start_addr, m_firmware_size_req);
        printf("Post Validation Error Code: %d\r\n",res.result );

        #else
        res.result = nrf_dfu_validation_activation_prepare(m_firmware_start_addr, m_firmware_size_req);
        #endif

2 and in this this change i didn't find the output after this  if (!is_trusted). may be this piece of code is not executing or something

nrf_dfu_result_t postvalidate(uint32_t data_addr, uint32_t data_len, bool is_trusted)
{
    nrf_dfu_result_t           ret_val = NRF_DFU_RES_CODE_SUCCESS;
    dfu_init_command_t const * p_init  = mp_init;

    if (!fw_hash_ok(p_init, data_addr, data_len))
    {
        ret_val = EXT_ERR(NRF_DFU_EXT_ERROR_VERIFICATION_FAILED);
    }
    else
    {
        printf("postvalidate Hash ok\r\n");
      
        if (p_init->type == DFU_FW_TYPE_APPLICATION)
        {
            if (!postvalidate_app(p_init, data_addr, data_len, is_trusted))
            {
                ret_val = NRF_DFU_RES_CODE_INVALID_OBJECT;
                printf("postvalidate fun error: postvalidate_app %d\r\n",ret_val);
            }
            else
            {
              printf("postvalidate fun success\r\n");
            }
        }
#if NRF_DFU_SUPPORTS_EXTERNAL_APP
        else if (p_init->type == DFU_FW_TYPE_EXTERNAL_APPLICATION)
        {
            if (!is_trusted)
            {
                // This function must be implemented externally
                ret_val = nrf_dfu_validation_post_external_app_execute(p_init, is_trusted);
            }
            else
            {
                s_dfu_settings.bank_1.bank_code = NRF_DFU_BANK_VALID_EXT_APP;
            }
        }
#endif // NRF_DFU_SUPPORTS_EXTERNAL_APP
        else
        {
            bool with_sd = p_init->type & DFU_FW_TYPE_SOFTDEVICE;
            bool with_bl = p_init->type & DFU_FW_TYPE_BOOTLOADER;

            if (!postvalidate_sd_bl(p_init, with_sd, with_bl, data_addr, data_len, is_trusted))
            {
                ret_val = NRF_DFU_RES_CODE_INVALID_OBJECT;
                if (is_trusted && with_sd && !DFU_REQUIRES_SOFTDEVICE &&
                    (data_addr == nrf_dfu_softdevice_start_address()))
                {
                    nrf_dfu_softdevice_invalidate();
                }
            }
        }
    }

    if (!is_trusted)
    {
        if (ret_val == NRF_DFU_RES_CODE_SUCCESS)
        {
                printf("\r\nSet DFU setting current bank.\r\n");

            s_dfu_settings.bank_current = NRF_DFU_CURRENT_BANK_1;
        }
        else
        {
            printf("nrf_dfu_settings_progress_reset if !is_trusted\r\n");

            nrf_dfu_settings_progress_reset();
        }
    }
    else
    {
        if (ret_val == NRF_DFU_RES_CODE_SUCCESS)
        {
        printf("Mark the update as complete and valid.\r\n");
            // Mark the update as complete and valid.
            s_dfu_settings.bank_1.image_crc  = crc32_compute((uint8_t *)data_addr, data_len, NULL);
            s_dfu_settings.bank_1.image_size = data_len;
        }
        else
        {
            nrf_dfu_bank_invalidate(&s_dfu_settings.bank_1);
                    printf("Mark the update as Invalid. : %d\r\n",ret_val);

        }

        printf("nrf_dfu_settings_progress_reset if is_trusted\r\n");

        nrf_dfu_settings_progress_reset();
        s_dfu_settings.progress.update_start_address = data_addr;
    }

    return ret_val;
}

3

static bool nrf_dfu_validation_hash_ok(uint8_t const * p_hash, uint32_t src_addr, uint32_t data_len, bool little_endian)
{
    ret_code_t err_code;
    bool       result   = true;
    uint8_t    hash_be[NRF_CRYPTO_HASH_SIZE_SHA256];
    size_t     hash_len = NRF_CRYPTO_HASH_SIZE_SHA256;

    nrf_crypto_hash_context_t hash_context = {0};

    crypto_init();

    if (little_endian)
    {
        // Convert to hash to big-endian format for use in nrf_crypto.
        nrf_crypto_internal_swap_endian(hash_be,
                                        p_hash,
                                        NRF_CRYPTO_HASH_SIZE_SHA256);
        p_hash = hash_be;
    }

    NRF_LOG_DEBUG("Hash verification. start address: 0x%x, size: 0x%x",
                  src_addr,
                  data_len);

   NRF_LOG_INFO("Hash verification. start address: 0x%x, size: 0x%x",
                  src_addr,
                  data_len);


    err_code = nrf_crypto_hash_calculate(&hash_context,
                                         &g_nrf_crypto_hash_sha256_info,
                                         (uint8_t*)src_addr,
                                         data_len,
                                         m_fw_hash,
                                         &hash_len);

    if (err_code != NRF_SUCCESS)
    {
        NRF_LOG_ERROR("Could not run hash verification (err_code 0x%x).", err_code);
        result = false;
    }
    else if (memcmp(m_fw_hash, p_hash, NRF_CRYPTO_HASH_SIZE_SHA256) != 0)
    {
        NRF_LOG_WARNING("Hash verification failed.");
        NRF_LOG_INFO("Hash verification failed.");
        NRF_LOG_DEBUG("Expected FW hash:")
        NRF_LOG_INFO("Expected FW hash:")
        NRF_LOG_HEXDUMP_DEBUG(p_hash, NRF_CRYPTO_HASH_SIZE_SHA256);
        NRF_LOG_DEBUG("Actual FW hash:")
        NRF_LOG_INFO("Actual FW hash:")
        NRF_LOG_HEXDUMP_DEBUG(m_fw_hash, NRF_CRYPTO_HASH_SIZE_SHA256);
        NRF_LOG_FLUSH();

        result = false;
    }

    return result;
}

[Vidar Berg]

Hello,

Does your bootloader not fall back to the old application when the CRCs don't match?

[Priyesh Shahi]

Yes, it does. However, The firmware version changes, and when I attempt to restart my device, it fails within the app_is_valid() function while executing the boot_validate() function. I've been investigating the cause and noticed that during the execution of this code segment:

// Postvalidate if DFU has signaled that update is ready.
if (s_dfu_settings.bank_current == NRF_DFU_CURRENT_BANK_1)
{
postvalidate();
}
#endif

The firmware version and s_dfu_settings.boot_validation_app are updated within the postvalidate_app() function.

memcpy(&s_dfu_settings.boot_validation_app, &boot_validation, sizeof(boot_validation));

s_dfu_settings.app_version = p_init->fw_version;

// The is_trusted argument specifies whether the function should have side effects.
static bool postvalidate_app(dfu_init_command_t const * p_init, uint32_t src_addr, uint32_t data_len, bool is_trusted)
{
    boot_validation_t boot_validation;

    ASSERT(p_init->type == DFU_FW_TYPE_APPLICATION);

    if (!boot_validation_extract(&boot_validation, p_init, 0, src_addr, data_len, VALIDATE_CRC))
    {
        return false;
    }
#if !NRF_DFU_IN_APP
    else if (NRF_BL_APP_SIGNATURE_CHECK_REQUIRED &&
             (boot_validation.type != VALIDATE_ECDSA_P256_SHA256))
    {
        NRF_LOG_WARNING("The boot validation of the app must be a signature check.");
        return false;
    }
#endif

    if (!is_trusted)
    {
        return true;
    }

    memcpy(&s_dfu_settings.boot_validation_app, &boot_validation, sizeof(boot_validation));

    s_dfu_settings.bank_1.bank_code = NRF_DFU_BANK_VALID_APP;

    NRF_LOG_DEBUG("Invalidating old application in bank 0.");
    s_dfu_settings.bank_0.bank_code = NRF_DFU_BANK_INVALID;

    if (!DFU_REQUIRES_SOFTDEVICE && !update_requires_softdevice(p_init))
    {
         // App does not need SD, so it should be placed where SD is.
         nrf_dfu_softdevice_invalidate();
    }

    if (!NRF_DFU_DEBUG ||
                (NRF_DFU_DEBUG && (p_init->has_is_debug == false || p_init->is_debug == false)))
    {
        s_dfu_settings.app_version = p_init->fw_version;
    }

    return true;
}

[Vidar Berg]

This suggests that the bank0.crc value in bt settings may have been updated (assuming you're using CRC validation (default)). The bootloader will verify the compute the CRC of the application to verify the integrity of the app image before booting. You also need to revert the app version number.

[Priyesh Shahi]

Should I save these values in external flash and reset them when encountering the same error? Or is there another method to retrieve the previous settings?

[Vidar Berg]

It seems it would be easier to back up the original settings to RAM. However, it's a bit surprising that you encounter occasional CRC errors in your copy routine. The image in bank 1 is validated by the post-validate function before the activation takes place.

[Priyesh Shahi]

Is there a way to erase the trigger settings or the settings saved for the currently downloading firmware? Because once the above error occurs, my firmware process works. However, alternatively, if it works fine this time, then the next time there might be an error. Or, if there's an error this time, it might work the next time.

I think some type of setting or data is stored in the system that isn't erased when something fails

[Priyesh Shahi]

The occasional CRC failure occurred due to the 'rest_block' variable stemming from the 'FIRMWARE_BLOCK_SIZE,' which is set to 4096. Therefore, whenever the 'rest_block' size exceeded 256, this error occurred.

// CRC verification on external image

uint16_t firmware_blocks = image_size / FIRMWARE_BLOCK_SIZE;

uint8_t rest_block = image_size % FIRMWARE_BLOCK_SIZE;

uint32_t computed_crc = 0;

As this problem has been solved, my task is now completed. Thank you so much!