DFU approach from cloud to ESP32 to nRF52840

[JasonGeo]

Hi,

My setup include a nRF52840-DK board that has an ESP32 WiFi module connected to it via UART. The idea is to source the new firmware from a server located in the cloud.

GCC toolchain,

Running FreeRTOS,

SDK 15.2.0

I am getting really confused after reading all the information on DFU and how I should be doing it. Simon also recently posted an article on DFU using Connect SDK https://devzone.nordicsemi.com/guides/nrf-connect-sdk-guides/b/software/posts/ncs-dfu

However, we are not using NCS at the moment.

Questions that I asked myself or still confused about are:

1. Should I be using background DFU or should I be modifying the bootloader to perform the DFU?
2. Where do I get the bin file from? I have a .zip package from the build output. Is that the bin file to use?
3. What is an init packet? Is that the .dat file that comes in the .zip package? How do you trigger the init packet? Via application or bootloader?
4. What is this bootloader settings file?
5. If I am using dual bank update, how do I specify the flash address for the second bank where the updated firmware gets written to?
6. How to let the bootloader knows that there is a new firmware in the second bank for validation and swap over? 
7. What transport should I be looking at? I assume serial - UART?

My initial approach is to write my own in a simplified manner.

The application will issue HTTP GET to the server and retrieve data chunks of the binary until the whole bin file is received. As it receives the chunks, it writes to a flash location.

After the whole bin file is received, the CPU reboots and the application copies the new firmware, overriding the old firmware in place.

Will that "upset" the bootloader when it reboots next time seeing that the firmware signature is different from the one being overwritten?

This is not foolproof as there is no validation and checks for data corruption. It's also not the "proper" way that Nordic perform DFU.

I hope someone can help with those questions and shed some light on the best method to approach this (given my setup, NRF52-ESP32-cloud).

What are the articles that I should focus reading up on. 

[Hung Bui]

Hi Jason, 

If you are working with nRF5SDK then the article from Simon is not relevant as it's applied for NCS SDK. 

Could you let me know how you are setting up the ESP and the NRF52 in your application ? Where the main application running on ? Are you using the ESP32 as the connectivity chip or the main application running on it? This would decide how the background DFU should work in your case. 

You mentioned "I have a .zip package from the build output". I'm not aware of a toolchain that can build the .zip output, could you elaborate more how your build system consist of ? Usually we use nrfutil.exe tool to generate .zip file. 

Regarding your question about bootloader setting you can have a look here. I have a blog post here that covers a little bit about bootloader settting and also give you some explanation on how the bootloader works. You can also find the link to a background DFU example at the end of the blog. 

What we do in the background DFU approach is to receive the image to bank 1 (swap bank , defined in the bootloader setting) and after the image is fully received, write to the bootloader setting about the progress and then reset to the bootloader to do post validation and the actual image swap. 

You can also find the Background DFU example in Thread and Zigbee SDK where we receive the image via Thread/Zigbee and then reset to bootloader to do the swap. 

[JasonGeo]

Hi Hung, thanks for your support.

The ESP32 connects to the nrf52840 DK board via UART and the nrf52 issues AT commands to it to send and receive data to the cloud. The main application runs on the nrf52 and ESP32 is purely for connectivity. 

The build system we are using is CMake with GCC and at the end of the CMakeList file, there are rules to make DFU archives using the nrfutil.

In the light of this, can you recommend the best approach to undertake for our system? Will it be background DFU? If so, I will focus on making background DFU happen.

From your suggestion, it seems like we do not need to modify the bootloader code, only the bootloader settings need to be modified? Where can I find the example code for a suitable bootloader?

Thanks for the links, I will have a good read about them.

Hope to hear from you soon.

[Hung Bui]

Hi Jason, 

I think we need to align the SDK version that you are using. There were some change between SDK v15.2 and SDK v15.3
I was looking at SDK v15.3 . And I would suggest to port from v15.2 to SDK v15.3 , Here is what I have in on_data_obj_execute_request_sched()

It's important that you find a working example of background DFU and can actually do some background DFU to test. After we can use it as the golden sample to port it to your application. 

I'm not sure if I have pointed you to the example by my coworker Vidar in my blog here.  Could you please try to test if it works for you ? 

[JasonGeo]

Thanks Hung, I will see if I can update my SDK version.

Can you help with question 1 please?

[Hung Bui]

Hi Jason, 

Please have a look at the function nrf_dfu_bank1_start_addr() in nrf_dfu_utils.c . It's used in this function update_data_addr_get() 

Basically Bank1 starts right after Bank0 . So it dynamically changes depending on the size of the application which placed at bank0. This is only applied for dual bank update.  

[JasonGeo]

Hi,

I am still working on the DFU, progress has been slow.

I am trying to access the global variable s_dfu_settings, that has been loaded into memory by my bootloader app hex, in my application code (a separate hex)

I tried to print the value in the settings e.g. app_version or bank_0.image_size and the result is always zero. I did things like

extern "C" nrf_dfu_settings_t s_dfu_settings;

 in my app code but didn't help. I am puzzled why I can't see s_dfu_settings in my app code.

However, when I do direct pointer access to the memory address from app, I can access the settings and values.

The bootloader code is built with nrf_dfu_settings.c included in the makefile. Do I have to include nrf_dfu_settings.c this same file in my application build? I did the inclusion but still can't "see" the global variable s_dfu_settings.

Also do i need to include  bootloader_settings_page (r) : ORIGIN = 0x000FF000, LENGTH = 0x1000 in my linker script for my application since it's included in the linker script for the bootloader.

Hope I am making sense here.

[Hung Bui]

Hi Jason, 

The bootloader and the application are 2 separated image. So you can't use the normal "extern" to get the visibility of the variable(s) from bootloader to application. 

There is a way to share data and function between the 2 images is to use the SVCI interface. You can have a look at the ble_app_buttonless example , where we use SVCI to either send bond information or advertising name to the bootloader via this interface. 

But anyway, for bootloader setting, it's easier that you access flash directly from the application than trying to communicate with bootloader via SVCI on this. What you need to do is to read the bootloader setting from flash and write new value to flash after you finished receiving image. Then you reset to the bootloader. The bootloader will read the bootloader setting in flash and continue accordingly. 

This also applied for the linker script, they are 2 different applications and not related to each other when you compiling. 

[JasonGeo]

Thanks Hung for the detailed explaination. It helps in my understanding and goes to show there is still lots to learn!

I think I am a step more closer to my understanding and implementation of background DFU now.

Still trying to get my head over something.

It's not possible for me to implement a DFU master on the cloud server to send the nrf52 application the bytes chunks while observing the serial DFU
protocol.

My application has to act like the DFU Master as well as the slave at the same time because the server will only do
whatever my application request from it. So I was thinking maybe after the app receiving a chunk of data from the server, the app then put on the DFU Master
"role" and "package" the data chunk with a serial DFU opcode and then call another local function which will decode the OP code in a serial DFU opcode switch/case block and
eventaully pass the data to nrf_dfu_req_handler_on_req() to be written to flash?

Will background DFU still work if I did not implement the full serial DFU opcodes processing e.g. can I skip SERIAL_DFU_OP_CODE_CREATE_OBJECT, SERIAL_DFU_OP_CODE_GET_SERIAL_MTU etc.
and just implement SERIAL_DFU_OP_CODE_WRITE_OBJECT and probably SERIAL_DFU_OP_CODE_CALCULATE_CRC, SERIAL_DFU_OP_CODE_EXECUTE_OBJECT?

Or I don't even need to implement this serial protocol and just call nrf_dfu_req_handler_on_req() directly each time I receive a data chunk?

Why do we need to send the init packet first and then execute the init packet. Also after every data chunk we need to execute object?

I am also having some build issue at the moment after pulling in the dfu modules from the bootloader for the application.

It's saying
undefined reference to `nrf_cc310_bl_ecdsa_verify_secp256r1'
undefined reference to `nrf_cc310_bl_hash_sha256_init'
undefined reference to `nrf_cc310_bl_hash_sha256_update'

but I've already included all the source modules for crypto and the libraries as well.

[JasonGeo]

Hi Hung,

Do you have some time to go over the questions? 

I've got the build issue sorted out. It has to do with the linking order of the libraries. So don't need to worry about the last part of the ticket.

Would appreciate if you can help with the rest of the queries.

[Hung Bui]

Hi Jason, 

I'm sorry for the late response. It was Easter vacation here in Norway last week. 
Regarding your question, no you don't need to implement any serial opcode. In fact you just need to find a way to receive your image and store it to the correct location (slot 1) and then call the function nrf_dfu_validation_pst_data_execute() just as in the code that I pointed to in previous replies.

The most important part is that you write to the bootloader setting correctly and trigger a reset to get to the bootloader. The bootloader will then swap the image when it read the bootloader setting. 

So it doesn't matter how you receive the image and that you don't have the DFU master on the cloud to send the image chunks the same way as the DFU serial protocol. 

You can read more about the init package here. I would suggest to test with the background DFU example from Vidar that I pointed to as well. 

[JasonGeo]

Happy Easter and thank you for your reply.

Ok, will give what you suggested a try. Am more relief now that I don't really need to follow the protocol but only the steps you mentioned!

Will revert back when I am succesful with the process.

[Bosemani_Raja]

Hai JasonGeo 

 Currently I'm also planning for this implementation.

After reading this discussion I'm bit relief.

Do you have any points or examples?

If you share this will be very helpful!

Best regards

Bose