Custom DFU transport (sdk 12)

RE: Custom DFU transport (sdk 12)

Ray — Wed, 28 Dec 2016 19:18:43 GMT

Interesting thought, and thanks for continuing to brainstorm about this.

Yesterday I did some analysis and realized that because the .BIN is basically uncompressable, there isn't much on-air savings for sticking with ZIP. And so I deconstruct the ZIP on the service, discard the manifest, and send the .DAT and .BIN down to the device uncompressed. This will make it easier for the bootloader.

I am now traveling but I will indeed try to find the time to look at and understand the existing code. Your thoughts sound fascinating. Thanks again.

RE: Custom DFU transport (sdk 12)

Hung Bui — Wed, 28 Dec 2016 16:31:06 GMT

Hi again Ray,

I had a discussion with one of our coworkers, Vidar. It seems that it's actually possible to reuse most of the secure bootloader. As you already noticed we tried to create the DFU with transport agnostic in mind. We only have one type of transport medium for now (BLE). But we think it's possible to replace the transport layer with your own custom transport. In your transport layer, you can instead of waiting for command from "DFU master", the NRF5 actually read the flash and check if there is a new image or not, if there is, it continue to read flash to get the init packet, ( the signature, CRC, size) and continue to call NRF_DFU_OBJECT_OP_EXECUTE so the upper layer can check the packet as it is done now with the nrf_ble_dfu.c and dfu_reg_handling.c (upper layer).

We are working on the UART transport layer for the secure bootloader now, most likely it will come with SDK v13 production, but I don't know when.

RE: Custom DFU transport (sdk 12)

Hung Bui — Tue, 27 Dec 2016 15:23:50 GMT

I understand Ray. Since you have some constrains as you described, you will need to implement almost your own bootloader. You can re-use what we got to store data to swap page, check CRC, then replace old image with new one. That's it. Other than that you may need to implement your own protocol for receiving the init packet, the image, etc. I would suggest you to have a look at the UART bootloader we have as the base for your development.

Regarding the .zip, we usually don't do un-zip on the nRF5, but on the phone (DFU Master) instead. You should choose to do un-zip and hex to bin converting on the device with less constrain on power consumption. Maybe on your 3G module ? What you store in the flash driver is simply the binary image (and the init data if needed) that the nRF5 should flash into its ROM.

RE: Custom DFU transport (sdk 12)

Ray — Tue, 27 Dec 2016 12:16:14 GMT

Hi. Thanks for your time.

My situation, in greater detail, is:

the developer does a 'git push' of updated builds to a special repo on our 'base service. From this moment on, everything happens in an unattended manner.
Each of a widely distributed set of devices poll the service once nightly to check if an update is pending for the firmware of its specific configuration. If such an update is available, the download is performed by our nRF app.
the download is done over 3G cellular. This is all run by a tiny nonprofit, and so it is important that the firmware is small. And so we are happy that nrfutil generates a ZIP file, and this is why we distribute the application only. The firmware was delivered 100% in the clear, and we don't trust link level encryption anyway, and so we love the new digitally signed images.
The download is done over FTP and this ZIP file is buffered entirely within the 3G cellular module's own 2GB flash drive (a SIMCOM 5320, FYI).
after download, each of these solar-powered devices will wait until it has sufficient battery life to ensure a complete DFU. It may be immediate, or it may be days. When it is ready, the nRF app reboots the device.
Now back in the bootloader, we initialize serial and use it to talk with the 3G module to look for the new firmware ZIP file. If it isn't present, we just start the app. If the ZIP file is present, we will begin the DFU process. In essence this implements a buttonless DFU. (By the way, I really like your use of _WEAK methods to allow me to override things like the "should start" method.)
Unfortunately, the only transfer method from the 3G module's Flash drive and the nRF is to initiate a stream of bytes from the zip file, from beginning to end, with no random access or flow control. As such, I have configured it to transfer very slowly (9600bps), and it uses app_sched of course to allow the bootloader to process it in 512-byte chunks.

All the code above is completed and tested. As you can see, there is no "controller" or "master" of the DFU process; the device must update itself given nothing more than a ZIP file.

And so given that I need to decode this zip file on the fly, and verify its integrity and digital signature, and to do the DFU, all inside the bootloader, I'm trying to come up with a plan as to what code I can borrow/transliterate and what I need to write.

Thanks again for taking the time to read and assist.

RE: Custom DFU transport (sdk 12)

Hung Bui — Tue, 27 Dec 2016 10:28:47 GMT

Hi Ray,

UART Bootloader is available since SDK v6. However with the introduce of Secure Bootloader in SDK v12, we removed UART as it's not been updated to secure bootloader yet.

So if you need UART bootloader, I would suggest to use the Legacy bootloader from SDK v11.

However, in your case, the DFU master side is simply a storage device with AT command ? Then you may need to modify the bootloader a little bit as the storage device may not be able to handle the DFU starting sequence. I don't think you really need to "reversed engineering" the DFU master, but simply follow what we described in our documentation to make your DFU master or to modify the bootloader to cope with your setup. The documentation for UART bootloader is here.