DFU compatibility for a mesh example

[DeveloperZ]

Hi,

I am interested to know how to make a mesh node (for example, a light server) compatible to perform a DFU.

I have gone through the DFU page and I noticed that the DFU is done over the Serial port. I understand that the dfu firmware is being replaced with blinky firmware. Correct me if I am wrong.

Now, what changes should I make to the commands inorder to make the light_server example node to be DFU compatible/capable over the air?

Would replacing the application hex file on step 8 to light_server instead of dfu solve it? or should I add any other commands to have light_server firmware first?

Thank you.

[Edvin]

Hello,

well. The short answer is that you need the mesh_evt_handler() to handle DFU packets, like it does in the DFU example. In addition, if you want the device to handle Serial DFU (the one who receives the packets from nrfutil), you need it to support serial DFU as well. 

This is done in nrf_mesh_serial_init(NULL); in mesh_init(void).

Inside this call, serial_handler_dfu_init() is called, which handles the DFU serial messages.

When you try to include the DFU packets (over Mesh) you will probably have to include everything you need around as well. Give it a go, and let me know if you get stuck.

BR,

Edvin

[DeveloperZ]

Hi,

Edvin said:

The short answer is that you need the mesh_evt_handler() to handle DFU packets, like it does in the DFU example.

I notice that nrf_mesh_evt_handler_t is used in only few examples like EnOcean, light switch provisioner, Lpn examples. 

So, in order to make other examples DFU capable, I need to add the nrf_mesh_evt_handler while mesh initilization.

Edvin said:

Give it a go, and let me know if you get stuck.

If I am trying to push the dfu packets over the air but not serial, I should push it from the nrf connect app right? or can we do it from the mesh app? Sorry but I have no idea about how to push it OTA!!

Thank you.

[Edvin]

DeveloperZ said:

Regarding the other data that is being stored, we are storing them using random entry handles like 7EFF and 5EFF(which we assumed are the memory locations on flash). 

 what do you mean by "entry handles"? Are you using FDS to store this data? Or perhaps fstorage?

[DeveloperZ]

Edvin said:

what do you mean by "entry handles"

It is the same way that you guys store the Enocean data in the enocean example. Create an entry handle and use app_flash_store, app_flash_data_store, app_flash_clear functions to write into and erase the flash.

[Edvin]

Ok. I see.

You need to tell the bootloader that you are using some of the flash area for application data, so that the bootloader or flash manager doesn't try to use this for something else.

Please take a look at what my collague, Hung writes in this post.

Basically, this define, FLASH_MANAGER_RECOVERY_PAGE_OFFSET_PAGES, will determine the flash that is reserved in between your bootloader and the flash_manager_recovery_page. Make sure that you store all your application data in this space.

Best regards,

Edvin

[DeveloperZ]

Edvin said:

Basically, this define, FLASH_MANAGER_RECOVERY_PAGE_OFFSET_PAGES, will determine the flash that is reserved in between your bootloader and the flash_manager_recovery_page.

I see that this particular define is initialized as 0. I changed it to 1.

The Enocean entry handle is specified as 00001. So, I had two variables that I am storing in flash. I assigned entry handles as 00002 and 00003. 

The DFU fails again. Should I also increase the APP_FLASH_PAGE_COUNT value which is set to 1 initially?

If I increase the count, how can the chip understand which flash page should it save my variables into?

I can't see anything in the app_flash_data_store, load and clear functions which specifies the flash page.

I am unable to figure out the changes needed to make the DFU work when I am storing data in the flash using app_flash_data_store, load and clear functions. If I can get the exact info on all the changes needed, that would be a great help.

Thank you.

[Edvin]

Just to verify that this is actually the issue. Can you try to perform the DFU without storing the custom data. Does the DFU pass through then?

It is not easy to tell "the exact info on all the changes needed" when I don't know your project. 

The assert you get, that is in the application after you performed the DFU, right? Does it happen if you just flash that image as well? Remember to erase the flash completely before you program your application if you have modified the flash setup, using e.g. "nrfjprog -e".

[DeveloperZ]

Edvin said:

Can you try to perform the DFU without storing the custom data.

When I don't store the data in the flash, DFU process is successful.

Edvin said:

The assert you get, that is in the application after you performed the DFU, right? Does it happen if you just flash that image as well?

When I flash the example with data stored in flash after erasing the board, it doesn't give back any assert. 

Now, I am pretty sure that the assert is occurring while initializing the flash (app_flash_init() function) while loading the new firmware.

Edvin said:

Please take a look at what my collague, Hung writes in this post.

I did go through the post and it says that the address for storing MBR and Softdevice starts at 0x00000 and is of size 0x26000. If you can see the enocean example, in enocean_switch_example.h file, the Entry handle for storing the enocean commissioning data is 0x00001.

Edvin said:

You need to tell the bootloader that you are using some of the flash area for application data, so that the bootloader or flash manager doesn't try to use this for something else.

I don't know how the boot loader understands that that particular location stores enocean data and can't be used for anything else. Where does the application specify it? Because I tried storing them in 0x00002 and 0x00003 locations and it fails.

I am trying to find other ways to store the custom data into the flash by either using ads or fstorage but it would be great if I get it working with this app_flash_init().

I can send you a simple example(zip file) which is similar to what I am trying and I would be glad if you can find some time to check what's going wrong and suggest the changes needed. :)

[Edvin]

If you store data in a new area in the flash, the flash manager needs to be aware. Changing this with a FW update, either a DFU or by manually programming the device will not erase the flash part that is used for application data storage, and hence the flash manager is still initialized with the old settings. So it is not trivial to change the size that the flash manager will use after it has been initialized.

I don't think this is actually DFU related at all. You need your initial FW to use the same flash manager settings as the one you intend to use in the end.

[DeveloperZ]

Hi,

I have been trying to analyze the messages sent back and forth between the PC and the target while establishing the initial UART communication.

Case1:

Sending DFU start packet, afterwards we wait for the flash on target to be initialized before continuing.
PC -> target: 0502aabbccdd
target -> PC: 0582aabbccdd
Got echo response
Sending DFU init packet
PC -> target: 1378fdff040fd3260a7859000000010003000000
target -> PC: 03847800
PC -> target: 1378fdff040fd3260a7859000000010003000000
target -> PC: 03847800
PC -> target: 1478fcff0000d3260a78ffffffff5d71000040000c
target -> PC: 03847887

I assume that the PC -> target message contains 59-company id, 1 - app-id, 3- application version that I sent. Is that right?

The response from the target to PC is 03847800. What does this indicate?

The third response back from the target to PC is 03847887. This is different from the first two responses. When the PC gets this response back, the DFU stops(fails). What does this value indicate?

Is there any link that I can refer to regarding these messages?

Case2:

Sending DFU start packet, afterwards we wait for the flash on target to be initialized before continuing.
PC -> target: 0502aabbccdd
target -> PC: 0582aabbccdd
Got echo response
Sending DFU init packet
PC -> target: 1378fdff040f2625fa6859000000010004000000
target -> PC: 16a6045900000001000400000059000000010001000000
target -> PC: 03847800
PC -> target: 1378fdff040f2625fa6859000000010004000000
target -> PC: 03847800
PC -> target: 1478fcff00002625fa68ffffffff5995000040000c
target -> PC: 0ea30104590000000100040000000d
target -> PC: 03847800
Sending firmware file
[------------------------------------] 1% 00:11:49PC -> target: 1978fcff01002625fa6800f0002071630200596302005b630200
target -> PC: 03847887

In this case, the target sends two message as a reply which wasn't done in the first case. Why does the target send variable amount of messages?

Also, the transfer began and then it was stopped when it received a different response back from the target. What might cause this behavior?

There have been situations where I successfully ran the DFU when the target responded back with one message and with two messages.

Why different responses for same/similar message packets sent from PC to target?

[Edvin]

Hello,

Please check out the Serial commands, Serial events and Serial status codes in the mesh sdk documentation.

So, basically, 03847887 means:

03: Internal Events Report

84: Cmd Response

78: DFU Data

87: ERROR_INVALID_DATA

So it means that the init packet is not accepted.

We have seen some weird behavior on certain compiler versions. So what IDE/compiler do you use? Segger Embedded Version? What version?

[DeveloperZ]

Edvin said:

We have seen some weird behavior on certain compiler versions. So what IDE/compiler do you use? Segger Embedded Version? What version?

Since you suggested previously, I have been using only SES v3.40. But I keep getting this response frequently.

Additionally, I have noticed that once the DFU transfer stops in the middle(for example, my PC went to sleep after transferring 87% which stopped sending packets); there is no way to resume the DFU from that point. I tried pushing the new update with increased version number. It failed sending the error code for ERROR_INVALID_DATA.

How does the flash for new FW image get freed? Does it gets overridden overtime new DFU is pushed or does the flash gets erased after the node loads the new FW?

The error that I mentioned, is there any chance that it could be because of the DFU packets stored during the previous transfer.