How to implement the Nordic OTA protocol in your own phone App
This document is meant to be useful if you are implementing DFU updates within in your own smartphone app, and want to do it yourself, rather than rely on trying to turn nrf toolbox into a library that your app calls. In a previous app we did just that, but it was a nightmare, and the nrf toolbox has since changed its structure a lot, making it even harder to do this now.
Our app is a cross-platform javascript app, and in the past we had built our own bootloaders and DFU protocols, but this time I wanted to use the nordic DFU bootloader, so I set up our own implementation of the app-side of the DFU scheme. If there is a lot of clamoring for it, i may turn what I’ve created into an open source library, but that is probably more work than I’m willing to do. This document instead outlines the main steps and resources you will need. Just a quick warning, unless you can get some debugging going on the bootloader side, it’s going to be really hard to do this properly.
- SDK version 12.3 (this is important because the protocol changes at 12 and up!). Download it here- https://developer.nordicsemi.com/nRF5_SDK/
- Softdevice version S130 V2.0.1, but you can use other versions of S130 v2 and up, I think
- Hardware was a PCA10028 board with an NRF51422 on it. This will work for NRF51822 and NRF 52 series as well.
- All programming was done with Jlink
- I used the armgcc compiler on a Mac
- I used KDS for the actual debugging of the bootloader as it has some nice stuff built into it for debugging. If you don’t already have the means to build and debug nrf code, don’t go down this path.
How DFU OTA works This article does a pretty good job of explaining what a bootloader is as well - https://devzone.nordicsemi.com/tutorials/9/, so I’ll keep my explanation short.
This article breaks down some more details about the bootloader - https://devzone.nordicsemi.com/blogs/683/a-quick-presentation-on-how-nrf51-dfu-works/
I think both of these articles refer to pre-SDK 12 though, so they may be outdated. I only skimmed them so I can’t say for sure…
If you don’t know what a bootloader is, it is basically a piece of code that gets called on startup, and that is separate from your main firmware (called the App in the DFU world). In this case we are using the bootloader as a small piece of code that can facilitate connecting to a phone and then replacing your main App memory space with new App memory, sent from your phone. To use Nordic’s DFU bootloader out of the box, you actually need to send 2 files down - and “init” file and the actual firmware data file. The init file exists because we need to have a scheme for error checking so that if the application memory region gets corrupted somehow, we know about it and know not to try to jump into the corrupted application (thus bricking your device!). You will need to generate a new init file every time you change your firmware, as that file includes information about your application file (like length and CRC error check number)
For the NRF series of softdevices (which are what actually run the show on startup) you need to put your bootloader near the back of the chip’s memory space. How the softdevice knows where to look for the bootloader is beyond me, but it makes it happen, so don’t worry about that part.
Your app will need to have some mechanism of being kicked into DFU mode and setting the appropriate bits. I already had a UART characteristic with acks, so instead of having a dedicated DFU characteristic (which eats up RAM), I write a distinct character sequence into my pre-existing UART channel to kick things over. I grabbed the code from the mbed DFU Characteristic to actually set the appropriate bits for transitioning us into DFU mode.
Once the bootloader is running, you will see a device called DfuTarg, that has 1 service with 2 characteristics - one is the “Control Point” that is write/notify and the other is the “Data Point” that is write only. You will send and receive various settings and status information through the control point, and you will send down all large chunks of data as series of 20-byte packets via the Data Point.
General Steps and main things to keep track of :
Before you go down this path, make sure you’re using the correct version of the softevices that math your bootloader. They made big changes with SDK 12. In the past I used S110 V7 and the corresponding off the shelf bootloader. This time I used S130 V2.0.1 (SDK V12.3) and modified the bootloader a little bit (more on that below). 12.3 worked for me, so just use that if you’re going to follow this tutorial.
- You will need to generate an Init file. The documentation surrounding this is pretty weak IMO, so I’ve provided more below.
- You will need to put a softdevice and bootloader on your chip. You can get a pre-compiled softdevice from the SDK when you download it. The pre-compiled bootloader is less easy to get, so I think you basically need to build it yourself. The quickest way to do that will be to head over to [SDK_ROOT path]/examples/dfu/bootloader_secure/pca10028/armgcc and run “make”. This will fail the first time most likely, and you’ll need to do some google searching on the errors to get over the various roadblocks. As I recall the main roadblocks were setting up armgcc and some other library.
- This is a secure bootloader, so you need to generate a unique key, as well as set up the bootloader to do this. Creating your key and setting it up in the bootloader is pretty easy, just follow the steps here - https://github.com/NordicSemiconductor/pc-nrfutil . I downloaded a pre-built version and ran it on my pc. From what I see it looks to be a bit annoying to build it yourself. The command I used for generating my pem file was “nrfutil keys generate private.pem” . Then run “nrfutil keys display --key pk --format code private.pem“ to display your security code, then copy-paste the output into the dfu_piblic_key.c file in the dfu code.
- Using the key that you just generated, and your application that you’d like to load over FOTA, you will need to generate a zip file using nrfutil. This part is a bit tricky, but here’s a vanilla command to get you started. Read the documentation if you’re straying from what I’m up to. Please note that —sd-req can have at most 4 things. Also there were claims that using 0xFFFE would put it into some sort of permissable mode, but that didn’t work for me (until I modified the bootloader!). nrfutil pkg generate --hw-version 51 --sd-req 0xFFFE,0x81,0x87,0x88 --application-version 0--application MyMagicApp.hex --key-file private.pem app_dfu_package.zip Once you have your .zip file, open it up to get at the .dat and .bin files within. The dat file is your init file (you can change the extension to .bin if you want) and the .bin file is your application, converted to bin format. These files are what you’ll be loading over
- Have your application send the init file and firmware files over to the bootloader. As this takes more than 1 paragraph to explain, I’ll break it down below. Do note though that the firmware file is actually sent as a number of Data Objects. Each Data Object is of maximum size 4,096 Bytes (0x1000), so unless you’re amazing and writing clean code, you’ll probably need to break your firmware up into multiple data objects.
Actually sending data to the bootloader The part you’ve been waiting for! A semi-handy block diagram is at - https://infocenter.nordicsemi.com/index.jsp?topic=%2Fcom.nordic.infocenter.sdk5.v12.0.0%2Flib_bootloader_dfu_process.html But the really useful part is at - http://developer.nordicsemi.com/nRF51_SDK/nRF51_SDK_v8.x.x/doc/8.0.0/s110/html/a00103.html#ota_intro_section In case the above links break (as they often do) head to the SDK V12 documentation in the infocenter, then head to Libraries->Bootloader_modules->DfuTransport->BLE
From careful reading of the above documents, here’s the simplest implementation of the above, in pesudo code form. CmdPntSend performs writes, and DataPntSend performs writeWithoutResponse (writes won’t work). You also need to sign up for notification on on the command point characteristic. I call these receives CmdPntRcv, and then respond to those as listed below. Note that you should always send a buffer of the correct size. Sending a buffer of size 20 that has only 5 valid bits will result in errors! The below numbers are all hex values btw
//****** Send the init file first :
CmdPntSend ( 06, 01 ) //tell the bootloader to select the init packet
CmdPntRcv( 60, 06, 01 …. ) //we should receive this if all is well. The … represents some other information that is for your error checking that I will ignore for this
CmdPntSend ( 01, 01, [ 4 bytes, init packet length, LSB first ] ) //prep things for the init packet. It’s length should be < 256 bytes or you’ll get an error (and your packet is probably invalid)
CmdPntRcv( 60 01 01 ) // that worked! If not, check your error codes
//we have not turned on the PRN stuff, so send the packet until you are done, and don’t expect a response.
while (have init packet data to send)
DataPntSend ( next chunk of init packet ) //send the init packet broken into 20 byte chunks, LSB. Make sure the last packet is of the correct length
CmdPntSend( 3 ) //tell it to calculate the CRC
CmdPntRcv( 60 , 3, 1 , … ) //all is well, I’m ignoring the error checking
CmdPntSend ( 4 ) //execute the command
CmdPntRcv( 60 04 01 ) //init packet worked
//now we’re going to set up the Packet Reciept Notification, or PRN. This will send us an Ack from the bootloader on the CmdPnt for every X packets that we specify.
//I recommend using ~20 packets per ack for good phones, down to 5 for older androids (or even new androids honestly)
CmdPntSend( 2 , XX , XX ) //send in the number of packets per ack, 6 bits, LSB
CmdPntRcv (60 , 2 , 1 ) //nice work
//****** Send the firmware data file
CmdPntSend( 06 , 02 ) //select the firmware file
CmdPntRcv( 60 06 01/// ) //have at it
//Now we need to loop over our firmware file, breaking it into distinct data objects that we send.
Firmware Bytes Left = firmware.length
while ( Firmware Bytes Left > 0 ){
if( Firmware Bytes Left > 0x1000 ) {
Firmware Bytes Left -= 0x1000
SendDataObject ( 0x1000 )
}else{
//we’re sending the last packet!
SendDataObject( Firmware Bytes Left )
Firmware Bytes Left = 0;
CmdPntSend ( 3 ) //check the CRC
CmdPntRcv(60 03 01 … ) //Here’s the CRC, as if you’ll really check it
CmdPntSend ( 4 ) //execute the code we just sent over
CmdPntRcv(60 04 01 ) //all done, we’re gonna run your main application code now!
}
}
//** this is our function that handles sending the whole data object SendDataObject( num bytes to send ) {
CmdPntSend( 01 02 xx xx xx xx ) // create a data object of “num bytes to send” length, LSB first
CmdPntRcv (06 01 01 ) //go for it
while (have data left to send in Data Object)
for “number of packets per ack”
DataPntSend( next 20 bytes of firmware data)
CmdPntRcv ( 60 03 01 … ) //got those bytes, with extra information to ignore
Bootloader firmware gotchas and details : Once in DFU mode, if you are using a WDT in your main code (like I am) you’ll need to insert the below into nrf_dfu.c -> wait_for_event() : // Transport is waiting for event? while(true) { // Can't be emptied like this because of lack of static variables app_sched_execute(); NRF_WDT->RR[0] = WDT_RR_RR_Reload; }
Android is bad at handling disconnect/reconnects. One way around this is to turn off the phone bluetooth programmatically, then turn it back on once you’ve kicked your device into bootloader mode. This is a bit of a quagmire though, so just a heads up!
Actually very important! You probably want to comment out the step that checks for the button pin being low . If you don’t want to build/modify the bootloader, you should definitely make sure that you don’t have to worry about pin 7 being low on startup. Otherwise you’re going to have a bad time - that once screwed me pretty good.
