nrfutil behavior incompatible with sdkv11 dfu_init_template.c

[Ray]

This is my first DFU project, and so this may be super-obvious to all of you.

I am writing this here to help others, because it appears that there are others out there right now struggling to get DFU working.

1. The SDKV11 dfu_init_template.c is written assuming that a crc is used to validate the image, with a suggestion that there is lots of flexibility to do other things.
2. When using GCC as the dev environment, one is naturally led to use the PC-NRFUTIL python utility on github.
3. One immediately learns that the "master" branch isn't compatible with SDK11. It's clearly a work in progress for what's going on in SDKV12. By doing research, the suggestion is that the "0_5_1" branch was the thing to use. (I looked at the other branches, and I don't see an alternative; perhaps I'm wrong.)
4. If you build your image using that version of NRFUTIL, you immediately find that the manifest does not have a CRC in it. (You'll also find that the init packet is so large that it requires bumping the buffer size in both dfu_single_bank.c and dfu_init_template.c from 64 to 128. But I digress.)
5. When you get your DFU code working and get to dfu_init_postvalidate(), you find that the CRC in the init packet simply isn't there. It looks as though it's replaced with the ext_packet_id.

I don't see any discussion on this forum that points to a version of dfu_init_template.c that can be used to verify the firmware_hash rather than crc. Perhaps I didn't look hard enough.

For now, I disabled the CRC check, and my DFU (finally) works perfectly.

I don't know how long it will be until SDKV12, but I'm looking forward to seeing the new methodology.

Thank you to those of you out there who gave helpful comments while trying to get DFU working; I do very much appreciate it.

[Øyvind Karlsen]

Hi,

Seems like you've pretty much figured this out yourself. I have not encountered the issues in 4, but it looks like you've found something that works for you.

The .dat file is the one that is actually being sent to the device, a .json is bundeled as it is useful for interfacing with Android and applications as it is more readable than the .dat file.

The application needs to know beforehand the format of the DFU packet, if it differs you will see odd behaviour, such as the CRC issues you've faced.

For future reference there's some additional information in the readme for nrfutil.

For those who are struggling with nrfutil you can apply this workaround.

First off we will download the 0.5 branch of nrfutil (Turns out that the live branch is for a future SDK 12) direct link to zip here.

After unzipping we need to install it, first we need to fix the dependencies, opening the requirements.txt file we find that we need to install the following (these should all have builds available through pip for "any" platform, if they don't you will have to manually install):

pip install pyserial
pip install enum34
pip install click
pip install ecdsa
pip install behave
pip install py2exe

After installing the dependencies you should be able to run the installer from the unzip location, eg. "[MY UNZIP LOCATION]\pc-nrfutil-0_5_1"

python setup.py install

The neat trick here is that the python script will run "uncompiled" so we can run main without wrappers, navigate to the nordicsemi folder and run main.py

cd nordicsemi\
python __main__.py

For all intents and purposes this main.py is the same as nrfutil.

Finally there's a bug in the most recent SDK where the init files for some of the precompiled .zip files contain wrong values for SoftDevice version. As a sanity check I recommend running through the setting up DFU tutorial, there I also supply a working .zip.

Best regards,

Øyvind

[zhouyu]

I'm struggling to do dfu. What's the nrfutil argument --sd-req for s132. Thanks in advance.

[zhouyu]

I got it. Leaving it empty is OK. My shell is like this."nrfutil dfu genpkg --application app.hex --application-version 0xffffffff --dev-type 0xffff --dev-revision 0xffff app.zip". Thank you all. You saved my life.

[Ray]

Ok, Øyvind, now I feel just plain dumb.

First, thank you for telling me to re-read the readme for nrfutil. Why? Because at the very, very, very end of the file it refers to the repo of a secure bootloader that contains the SHA source code and sample bootloader that uses it. I had been looking for that; that's what I was referring to when in my note above I said "Perhaps I didn't look hard enough."

MORE IMPORTANTLY, thank you for your comments, which caused me to poke around and read the Python source code for NRFUTIL. Why did this help? Because it made me realize that I made a truly dumb mistake.

You see, when I had first tried NRFUTIL and didn't know what I was doing, I had created a PEM keyfile, and had added this to the NRFUTIL command line in my makefile: "--key-file $(APPNAME).pem"

Of course, as I learned more about NRFUTIL, I realized that keyfiles were only needed for secure init, and I just forgot about it and focused on using CRC.

When looking at the source code for NRFUTIL this morning, I realized that if the --key-file parameter is found on the command line, it a) jams the version to 0.8 rather than 0.5, b) includes the stuff in the init packet corresponding to the new secure/hashed format, and c) eliminates the CRC from the output under the assumption that you're no longer using the old way of validating the image.

The moment I removed the spurious --key-file argument from nrfutil, the CRC appeared in the output (and in the manifest.json), and the CRC check in the postvalidate() now works perfectly!

And so, the reality is that the title of this topic "nrfutil behavior incompatible with sdkv11 dfu_init_template.c" is flat-out wrong. NRFUTIL 0_5_1 behavior IS fully compatible with the dfu_init_template.c so long as you don't stupidly use the --key-file parameter on the nrfutil command line.

Thank you again for your help. This has been quite a learning process, with a positive conclusion.

If there IS one takeaway for you folks at Nordic, though, it is that I think you can probably see that DFU has enough moving parts (in understanding memory layouts and how it works, in getting the tooling loaded, in writing a buttonless bootloader, in creating the composite softdevice+app+bootloader image, in creating the download image, etc) that this it is probably worth the investment on your part in creating not one but a suite of multiple DFU examples for different build environments and with different security requirements.

Thank you again.