Custom DFU transport (sdk 12)

[Ray]

Hello. I am at the point in my project (using sdk v12) where I need to get DFU implemented.

In my case, DFU must occur through an alternative transport that is none of the 'usual suspects' of BLE, serial, etc.

Having struggled to get DFU implemented at all in the past, I am extremely impressed by how you've generalized the DFU transport mechanism. I don't understand it all yet, but the modularity is really slick.

I'm also incredibly impressed with the new secure code signing mechanisms. Such an improvement, and really the right thing to do for the industry.

That said, the actual DFU mechanisms are still pretty much a black box to me at this point, and given the complexity I think it's best that I treat it like a black box as much as I can - focusing on transport and not on the logic performed to actually unpack the image and do the DFU.

As a starting point, I've got DFU in my project implemented to the point where it (now, finally) works flawlessly with BLE dfu from my iphone.

For my custom DFU, the state is that:

1. I've got an external storage device attached to the UART that is addressable with AT commands

2. The only AT command relevant here is that I can ask the device to stream the file contents into the nrf52 over the uart. The file contents is streamed inward in variable sized chunks, from beginning-of-file to end-of-file. It is sequential fetch only; no random seek allowed.

3. Before I ever get into the bootloader to begin the DFU, the ZIP file for the image has been downloaded into that device. The bootloader will go into buttonless DFU mode if and only if the AT commands indicate that the file is present on that device.

And so, ideally, my task will be to "simply" (!) enhance the bootloader to have a custom transport that does as little logically as possible other than to read and feed the file into someone else's already-debugged logic to process it and to "do what the phone does", ultimately generating and calling nrf_dfu_req_handler_on_req's to process what's in the zip file and do the dfu.

Given that I am about to embark upon the process of reverse engineering the IOS DFU library to see what the heck it's doing and how it works, it occurs to me that I should just stop and ask you:

Do you have a different alternate transport written yet (such as UART), even if pre-release, that I can use as a starting point rather than to do this extremely complicated reverse engineering?

Your assistance would be greatly appreciated because this task is looking to be a bit daunting at this point.

Thanks much.

[Ray]

Hi. Thanks for your time.

My situation, in greater detail, is:

• the developer does a 'git push' of updated builds to a special repo on our 'base service. From this moment on, everything happens in an unattended manner.

• Each of a widely distributed set of devices poll the service once nightly to check if an update is pending for the firmware of its specific configuration. If such an update is available, the download is performed by our nRF app.

• the download is done over 3G cellular. This is all run by a tiny nonprofit, and so it is important that the firmware is small. And so we are happy that nrfutil generates a ZIP file, and this is why we distribute the application only. The firmware was delivered 100% in the clear, and we don't trust link level encryption anyway, and so we love the new digitally signed images.

• The download is done over FTP and this ZIP file is buffered entirely within the 3G cellular module's own 2GB flash drive (a SIMCOM 5320, FYI).

• after download, each of these solar-powered devices will wait until it has sufficient battery life to ensure a complete DFU. It may be immediate, or it may be days. When it is ready, the nRF app reboots the device.

• Now back in the bootloader, we initialize serial and use it to talk with the 3G module to look for the new firmware ZIP file. If it isn't present, we just start the app. If the ZIP file is present, we will begin the DFU process. In essence this implements a buttonless DFU. (By the way, I really like your use of _WEAK methods to allow me to override things like the "should start" method.)

• Unfortunately, the only transfer method from the 3G module's Flash drive and the nRF is to initiate a stream of bytes from the zip file, from beginning to end, with no random access or flow control. As such, I have configured it to transfer very slowly (9600bps), and it uses app_sched of course to allow the bootloader to process it in 512-byte chunks.

All the code above is completed and tested. As you can see, there is no "controller" or "master" of the DFU process; the device must update itself given nothing more than a ZIP file.

And so given that I need to decode this zip file on the fly, and verify its integrity and digital signature, and to do the DFU, all inside the bootloader, I'm trying to come up with a plan as to what code I can borrow/transliterate and what I need to write.

Thanks again for taking the time to read and assist.

[Ray]

Interesting thought, and thanks for continuing to brainstorm about this.

Yesterday I did some analysis and realized that because the .BIN is basically uncompressable, there isn't much on-air savings for sticking with ZIP. And so I deconstruct the ZIP on the service, discard the manifest, and send the .DAT and .BIN down to the device uncompressed. This will make it easier for the bootloader.

I am now traveling but I will indeed try to find the time to look at and understand the existing code. Your thoughts sound fascinating. Thanks again.