DFU compatibility for a mesh example

[DeveloperZ]

Hi,

I am interested to know how to make a mesh node (for example, a light server) compatible to perform a DFU.

I have gone through the DFU page and I noticed that the DFU is done over the Serial port. I understand that the dfu firmware is being replaced with blinky firmware. Correct me if I am wrong.

Now, what changes should I make to the commands inorder to make the light_server example node to be DFU compatible/capable over the air?

Would replacing the application hex file on step 8 to light_server instead of dfu solve it? or should I add any other commands to have light_server firmware first?

Thank you.

[Edvin]

Hello,

well. The short answer is that you need the mesh_evt_handler() to handle DFU packets, like it does in the DFU example. In addition, if you want the device to handle Serial DFU (the one who receives the packets from nrfutil), you need it to support serial DFU as well. 

This is done in nrf_mesh_serial_init(NULL); in mesh_init(void).

Inside this call, serial_handler_dfu_init() is called, which handles the DFU serial messages.

When you try to include the DFU packets (over Mesh) you will probably have to include everything you need around as well. Give it a go, and let me know if you get stuck.

BR,

Edvin

[DeveloperZ]

Hi,

Edvin said:

The short answer is that you need the mesh_evt_handler() to handle DFU packets, like it does in the DFU example.

I notice that nrf_mesh_evt_handler_t is used in only few examples like EnOcean, light switch provisioner, Lpn examples. 

So, in order to make other examples DFU capable, I need to add the nrf_mesh_evt_handler while mesh initilization.

Edvin said:

Give it a go, and let me know if you get stuck.

If I am trying to push the dfu packets over the air but not serial, I should push it from the nrf connect app right? or can we do it from the mesh app? Sorry but I have no idea about how to push it OTA!!

Thank you.

[DeveloperZ]

Edvin said:

Basically, this define, FLASH_MANAGER_RECOVERY_PAGE_OFFSET_PAGES, will determine the flash that is reserved in between your bootloader and the flash_manager_recovery_page.

I see that this particular define is initialized as 0. I changed it to 1.

The Enocean entry handle is specified as 00001. So, I had two variables that I am storing in flash. I assigned entry handles as 00002 and 00003. 

The DFU fails again. Should I also increase the APP_FLASH_PAGE_COUNT value which is set to 1 initially?

If I increase the count, how can the chip understand which flash page should it save my variables into?

I can't see anything in the app_flash_data_store, load and clear functions which specifies the flash page.

I am unable to figure out the changes needed to make the DFU work when I am storing data in the flash using app_flash_data_store, load and clear functions. If I can get the exact info on all the changes needed, that would be a great help.

Thank you.

[Edvin]

Just to verify that this is actually the issue. Can you try to perform the DFU without storing the custom data. Does the DFU pass through then?

It is not easy to tell "the exact info on all the changes needed" when I don't know your project. 

The assert you get, that is in the application after you performed the DFU, right? Does it happen if you just flash that image as well? Remember to erase the flash completely before you program your application if you have modified the flash setup, using e.g. "nrfjprog -e".

[DeveloperZ]

Edvin said:

Can you try to perform the DFU without storing the custom data.

When I don't store the data in the flash, DFU process is successful.

Edvin said:

The assert you get, that is in the application after you performed the DFU, right? Does it happen if you just flash that image as well?

When I flash the example with data stored in flash after erasing the board, it doesn't give back any assert. 

Now, I am pretty sure that the assert is occurring while initializing the flash (app_flash_init() function) while loading the new firmware.

Edvin said:

Please take a look at what my collague, Hung writes in this post.

I did go through the post and it says that the address for storing MBR and Softdevice starts at 0x00000 and is of size 0x26000. If you can see the enocean example, in enocean_switch_example.h file, the Entry handle for storing the enocean commissioning data is 0x00001.

Edvin said:

You need to tell the bootloader that you are using some of the flash area for application data, so that the bootloader or flash manager doesn't try to use this for something else.

I don't know how the boot loader understands that that particular location stores enocean data and can't be used for anything else. Where does the application specify it? Because I tried storing them in 0x00002 and 0x00003 locations and it fails.

I am trying to find other ways to store the custom data into the flash by either using ads or fstorage but it would be great if I get it working with this app_flash_init().

I can send you a simple example(zip file) which is similar to what I am trying and I would be glad if you can find some time to check what's going wrong and suggest the changes needed. :)

[Edvin]

If you store data in a new area in the flash, the flash manager needs to be aware. Changing this with a FW update, either a DFU or by manually programming the device will not erase the flash part that is used for application data storage, and hence the flash manager is still initialized with the old settings. So it is not trivial to change the size that the flash manager will use after it has been initialized.

I don't think this is actually DFU related at all. You need your initial FW to use the same flash manager settings as the one you intend to use in the end.

[DeveloperZ]

Hi,

I have been trying to analyze the messages sent back and forth between the PC and the target while establishing the initial UART communication.

Case1:

Sending DFU start packet, afterwards we wait for the flash on target to be initialized before continuing.
PC -> target: 0502aabbccdd
target -> PC: 0582aabbccdd
Got echo response
Sending DFU init packet
PC -> target: 1378fdff040fd3260a7859000000010003000000
target -> PC: 03847800
PC -> target: 1378fdff040fd3260a7859000000010003000000
target -> PC: 03847800
PC -> target: 1478fcff0000d3260a78ffffffff5d71000040000c
target -> PC: 03847887

I assume that the PC -> target message contains 59-company id, 1 - app-id, 3- application version that I sent. Is that right?

The response from the target to PC is 03847800. What does this indicate?

The third response back from the target to PC is 03847887. This is different from the first two responses. When the PC gets this response back, the DFU stops(fails). What does this value indicate?

Is there any link that I can refer to regarding these messages?

Case2:

Sending DFU start packet, afterwards we wait for the flash on target to be initialized before continuing.
PC -> target: 0502aabbccdd
target -> PC: 0582aabbccdd
Got echo response
Sending DFU init packet
PC -> target: 1378fdff040f2625fa6859000000010004000000
target -> PC: 16a6045900000001000400000059000000010001000000
target -> PC: 03847800
PC -> target: 1378fdff040f2625fa6859000000010004000000
target -> PC: 03847800
PC -> target: 1478fcff00002625fa68ffffffff5995000040000c
target -> PC: 0ea30104590000000100040000000d
target -> PC: 03847800
Sending firmware file
[------------------------------------] 1% 00:11:49PC -> target: 1978fcff01002625fa6800f0002071630200596302005b630200
target -> PC: 03847887

In this case, the target sends two message as a reply which wasn't done in the first case. Why does the target send variable amount of messages?

Also, the transfer began and then it was stopped when it received a different response back from the target. What might cause this behavior?

There have been situations where I successfully ran the DFU when the target responded back with one message and with two messages.

Why different responses for same/similar message packets sent from PC to target?

[Edvin]

Hello,

Please check out the Serial commands, Serial events and Serial status codes in the mesh sdk documentation.

So, basically, 03847887 means:

03: Internal Events Report

84: Cmd Response

78: DFU Data

87: ERROR_INVALID_DATA

So it means that the init packet is not accepted.

We have seen some weird behavior on certain compiler versions. So what IDE/compiler do you use? Segger Embedded Version? What version?

[DeveloperZ]

Edvin said:

We have seen some weird behavior on certain compiler versions. So what IDE/compiler do you use? Segger Embedded Version? What version?

Since you suggested previously, I have been using only SES v3.40. But I keep getting this response frequently.

Additionally, I have noticed that once the DFU transfer stops in the middle(for example, my PC went to sleep after transferring 87% which stopped sending packets); there is no way to resume the DFU from that point. I tried pushing the new update with increased version number. It failed sending the error code for ERROR_INVALID_DATA.

How does the flash for new FW image get freed? Does it gets overridden overtime new DFU is pushed or does the flash gets erased after the node loads the new FW?

The error that I mentioned, is there any chance that it could be because of the DFU packets stored during the previous transfer.

[Edvin]

DeveloperZ said:

Since you suggested previously, I have been using only SES v3.40. But I keep getting this response frequently.

 Sorry. I am out travelling, so I didn't read through the thread.

 

DeveloperZ said:

I tried pushing the new update with increased version number. It failed sending the error code for ERROR_INVALID_DATA.

 That is because the device is probably expecting the next packet in the sequence. It should be possible to resume by sending the next packet, but this is not something that nrfutil currently supports. You would have to do this by altering the nrfutil FW, or wait until the DFU times out, and start over.

 

DeveloperZ said:

The error that I mentioned, is there any chance that it could be because of the DFU packets stored during the previous transfer.

 Yes. If the device is still in DFU mode with a previous image, then you have to wait until it times out. Check out the reply that describes TIMER_DATA_TIMEOUT_US. There is also a define TIMER_START_TIMEOUT_US (both in nrf_mesh_dfu.c). If any of these times out, the DFU is stopped, and it will be ready to receive a new packet.

[DeveloperZ]

Hi,

How many DFU pushes can the board handle?

Is there any time frame needed between each push?

I am having hard time pushing the DFU multiple times.I have been pushing the same FW image of DFU example as well as customized example with DFU over and over again but the second DFU push fails most of the time. Sometimes, second push is successful but third fails. It fails sending the error code for ERROR_INVALID_DATA.

I did try it by sending commands individually and also using scripts(version number is increased everytime). If there was something wrong with either commands or the scripts, it would have failed in the first push but it fails on second or third push. Also,  I am not even changing the hex files for pushing. It is the same file that was used for the first push.

In the customized example, I do change the name and push it so that I see the indication that the device is successfully updated with new FW image.

DeveloperZ said:

How does the flash for new FW image get freed? Does it gets overridden overtime new DFU is pushed or does the flash gets erased after the node loads the new FW?

I dunno why but I feel that the flash which stores the new FW image isn't accepting the new image as I can see that the initial messages that are sent to check the ID's reply back with Success code which is 0x00.

The DFU start message which carries the start address, size of the application is the one that is returning back with the ERROR_INVALID_DATA reply

The only difference that I see with the DFU start message when the DFU is pushed successfully and when it fails is the transfer ID field in the message packet. I don't see anything else changing. So, I am still unsure on what is going wrong while trying to push the same FW again and again.

It would be great if you or any of your colleagues find time to push DFU example multiple times onto the same board and see if there is anything wrong with that?

[Edvin]

DeveloperZ said:

How many DFU pushes can the board handle?

 1 at the time.

 

DeveloperZ said:

Is there any time frame needed between each push?

 Yes.

You need to check out the defines that i mentioned in the previous reply. TIMER_DATA_TIMEOUT_US and TIMER_START_TIMEOUT_US. It will not accept a new image until any of these times out. Please let me know if you can't find them.

As mentioned. I am out of office, so I don't have any HW to test this on. Please open a new ticket so that one of the guys in the office can check. 

 

DeveloperZ said:

while trying to push the same FW again and again.

 One of these two is happening:

1: The application that is stored is already up to date. You need to increase the application version to make the device accept it.

2: The device is still expecting the next packet from a previous session because TIMER_DATA_TIMEOUT_US has not timed out and TIMER_START_TIMEOUT_US has not timed out.

Best regards,

Edvin