DFU protocol details for a custom DFU controller?

[scla]

Hi,

I'm trying to create a Node application (using Noble) that acts as a DFU controller so that I can automate DFU'ing hundreds of devices (nRF52 using SDK 15.0.0). I've been referencing this documentation. I have several questions that I can't seem to find the answers to.

First, I start with the SD BL init packet. The responses I get from the DFU Control Point are exactly similar to the sequence flow chart. I compare the CRC and image size and confirm they are correct. I am rather confused why this works, since in the sequence diagram it states that I can only send up to 20 bytes at the time. I instead send 137 bytes at once. Also, for whatever reason the "Response PRN success" (btw, what does PRN mean?) never occurs. After sending the data to the data point, I send <3> to the data control point, and I get back the CRC and length which are correct. I then assume that this process has worked (has it?) and move to the SD BL image.

Now I begin the same process as above, but when I send the create command (<01, 02, C8>, where 0xC8 = 200 bytes), I get the response <60, 01, 03>. I cannot seem to find anywhere in the documentation what 03 means. I then begin sending to the data point and when this is finished, I send the CRC command and get the response  <Buffer 60 03 01 00 00 00 00 00 00 00 00>. This looks clearly wrong as the CRC and length are both 0.

To reiterate in a better format:

SD BL INIT PACKET:

CMD SENT (hex)       | RESPONSE (hex)
<06 01>              | <60 06 01 00 00 00 00 0 00 00 00 00 00 00 00>
<01 01 89 00 00 00>  | <60 01 01>
<0x89 bytes> to DP   | NOTHING
<3>                  | <60 03 01 89 00 00 00 2a 9c fe 44>
<4>                  | <60 4 1>


SD BL IMAGE

CMD SENT (hex)       | RESPONSE (hex)
<06 02>              | <60 06 01 00 10 00 00 00 00 00 00 00 00 00 00>
<01 02 C8 00 00 00>  | <60 01 03>
<0xC8 bytes> to DP   | NOTHING
 ...
<0xC8 bytes> to DP   | NOTHING
<3>                  | <60 03 01 00 00 00 00 00 00 00 00>
(Stop since CRC failed.)

Is there further documentation that I'm missing here? Am I misunderstanding something in the linked documentation?

Thanks

[Einar Thorsrud]

Hi,

I am rather confused why this works, since in the sequence diagram it states that I can only send up to 20 bytes at the time.

It seems the diagram hs not been updated, and is the same as for older SDK versions, before the bootloader.

Also, for whatever reason the "Response PRN success" (btw, what does PRN mean?) never occurs.

 PRN is Packet Receipt Notification. You can see more description in the serial transport layer documentation (though the concept is there regardless of transport):

Before the actual DFU process can start, the DFU controller must set the Packet Receipt Notification (PRN) value and obtain the maximum transmission unit (MTU). In most cases, the PRN can be set to 0 to disable checksums being sent back to the controller during transfers. However, if the transport layer is unreliable, set the PRN to a value other than 0.

Now I begin the same process as above, but when I send the create command (<01, 02, C8>, where 0xC8 = 200 bytes), I get the response <60, 01, 03>. I cannot seem to find anywhere in the documentation what 03 means.

3 is NRF_DFU_RES_CODE_INVALID_PARAMETER. Have you done anything to increase the MTU before this? From the documentation: "If no larger Att MTU size is negotiated with the DFU Service, the expected maximum size of a write is 20 Bytes. "

[scla]

Hi Einar, thanks for the reply.

I think I've sorted through most of the issues I was having, which mostly came from not properly understanding data objects. Now I have everything set up like the diagram. Also, I do not set the MTU, but I think I will return to this issue at a later time. Right now I'm sending 230 bytes each iteration which is fast enough for development.

One large issue that I'm running into is creating the last data object. The target gives me a max data object size of 4096 bytes. For the SD + BL init + bin files, everything seems to send correctly until after I send the last data object, CRC command, and execute. CRC checks out fine, but after sending execute I get the response <60, 04, 0b, 0c> which seems to be a validation error. With a smartphone using nRF Connect, I see this same data object being sent with a response of <60, 04, 01> in the logs. Edit: I checked the CRC's for every data object sent over my controller with the smartphone DFU controller. They are exactly the same!

I feel like I'm missing something very simple here for the last data object. Data objects and their sizes seem to be the same as what is in the smartphone logs, however. Any ideas?

Edit:

Just tried importing and using pc-nrf-dfu-js instead of my own custom controller and have the same issue as in the below link. It looks like the hash in the init packet is not matching with the application file hash.

https://github.com/NordicSemiconductor/pc-nrf-dfu-js/issues/44?_pjax=%23js-repo-pjax-container

The DFU package was made with nrfutil and works over a smartphone, so I do not think the package is at fault here. Strange that both my custom controller and pc-nrf-dfu-js fail. Any ideas?

[Einar Thorsrud]

Hi,

As you write,  the response <60, 04, 0b, 0c> means you have a verification failure (NRF_DFU_EXT_ERROR_VERIFICATION_FAILED). This only happens if validation fails, and since we know the bootloader itself with public key etc is OK (it works if you use another DFU master, and validation can only faile for one of two reasons: either the data is corrupt in some way or the signature is corrupt in some way.

scla said:

Edit: I checked the CRC's for every data object sent over my controller with the smartphone DFU controller. They are exactly the same!

Does this apply to all CRC's? And none are missing on the failing case? If so, this indicates that the data is correct, since you have the same CRC values using both DFU masters, even though only one of the DFU masters is working.

scla said:

Just tried importing and using pc-nrf-dfu-js instead of my own custom controller and have the same issue as in the below link. It looks like the hash in the init packet is not matching with the application file hash.

Can you upload the two hashes here? Could it be something as simple as the endianness being swapped?

It could be interesting to see full flash dumps of the device after upgrade in both cases so that we can diff the dumps and easily see what the differences are. You can dump the flash content using for instance "nrfjprog --memrd 0 --n 0x80000 > full_flash.txt" (adjust the size if you have a part with other than 512 kB flash).

[m.reynal]

Hello,

 

I write a controller in cpp to do dfu over ble.

I have the same issue (not a 100% bug) : all crc and offset are ok until the end. But I have a « hash failure » on the last execute command.

I think my controller is sending packets too fast, so last packets are not written yet in flash (queue usage not empty) when execute command is handled by the target. (see target’s logs).

I set PRN to have last packet of each object waiting for response with CRC and offset (see in logs « PRN reached). CRC and Offset are ok, so controller sends execute command… then hash failure

Maybe I’m missing something before the last execute command, something to ‘synchronize’ controller and target (now I have a awfull hack to wait 100ms before sending last execute command?

 

Thanks.

Fullscreen dfu ota - hash failure - target logs.txt.txt Download

108139] nrf_ble_dfu.c:618 PRN reached 
[108141] nrf_ble_dfu.c:345 Sending CRC: [0x60, 0x03, 0x01, 0:x0003a2d0, CRC:0x9b467ed5]
[108145] nrf_dfu_flash.c:76 Flash write success: addr=0005d180, pending 1
[108164] nrf_dfu_flash.c:129 nrf_fstorage_write(addr=0005d280, len=0x80 bytes), queue usage: 2
[108166] dfu_req_handling.c:1013 Storing 128 bytes at: 0x0005d280

[108176] nrf_ble_dfu.c:618 PRN reached 
[108182] nrf_dfu_flash.c:76 Flash write success: addr=0005d200, pending 1

[108196] nrf_ble_dfu.c:467 Set receipt notif 
[108197] nrf_ble_dfu.c:477 m_pkt_notif_target 5
[108198] nrf_ble_dfu.c:480 m_pkt_notif_target_cnt 5
[108199] nrf_ble_dfu.c:310 Sending Response: [0x2, 0x1]

[108213] nrf_dfu_flash.c:76 Flash write success: addr=0005d280, pending 0
[108239] nrf_dfu_flash.c:129 nrf_fstorage_write(addr=0005d300, len=0x80 bytes), queue usage: 1
[108242] dfu_req_handling.c:1013 Storing 128 bytes at: 0x0005d300
[108248] nrf_dfu_flash.c:129 nrf_fstorage_write(addr=0005d380, len=0x20 bytes), queue usage: 2
[108250] dfu_req_handling.c:1051 Storing 32 bytes at: 0x0005d380

[108251] nrf_ble_dfu.c:618 PRN reached[108254] nrf_ble_dfu.c:345 Sending CRC: [0x60, 0x03, 0x01, 0:x0003a3a0, CRC:0x2ee4a596]

[108273] nrf_ble_dfu.c:458 Received execute object
[108274] dfu_req_handling.c:1079 Before OP execute
[108275] dfu_req_handling.c:1093 Valid Data Execute : Update the offset and crc values for the last object written.
[108277] nrf_dfu_settings.c:194 Writing settings...
[108278] nrf_dfu_settings.c:195 Erasing old settings at: 0x0007f000
[108279] nrf_dfu_flash.c:150 nrf_fstorage_erase(addr=0x0007f000, len=1 pages), queue usage: 3
[108281] nrf_dfu_flash.c:129 nrf_fstorage_write(addr=0007f000, len=0x17c bytes), queue usage: 4
[108283] nrf_fstorage.c:162 writing...
[108284] dfu_req_handling.c:1109 Received the whole image. Doing postvalidate
[108850] dfu_req_handling.c:522 Hash failure

[Einar Thorsrud]

Hi,

I suspect this is caused by doing post validation before the last chunk of data has been written to flash. There is no part of the DFU protocol that gives you information about that, so I do not see any other solution than waiting a bit so that you are sure that all data is written to flash before doing post validation.