DFU compatibility for a mesh example

[DeveloperZ]

Hi,

I am interested to know how to make a mesh node (for example, a light server) compatible to perform a DFU.

I have gone through the DFU page and I noticed that the DFU is done over the Serial port. I understand that the dfu firmware is being replaced with blinky firmware. Correct me if I am wrong.

Now, what changes should I make to the commands inorder to make the light_server example node to be DFU compatible/capable over the air?

Would replacing the application hex file on step 8 to light_server instead of dfu solve it? or should I add any other commands to have light_server firmware first?

Thank you.

[Edvin]

Hello,

well. The short answer is that you need the mesh_evt_handler() to handle DFU packets, like it does in the DFU example. In addition, if you want the device to handle Serial DFU (the one who receives the packets from nrfutil), you need it to support serial DFU as well. 

This is done in nrf_mesh_serial_init(NULL); in mesh_init(void).

Inside this call, serial_handler_dfu_init() is called, which handles the DFU serial messages.

When you try to include the DFU packets (over Mesh) you will probably have to include everything you need around as well. Give it a go, and let me know if you get stuck.

BR,

Edvin

[DeveloperZ]

Hi,

Edvin said:

The short answer is that you need the mesh_evt_handler() to handle DFU packets, like it does in the DFU example.

I notice that nrf_mesh_evt_handler_t is used in only few examples like EnOcean, light switch provisioner, Lpn examples. 

So, in order to make other examples DFU capable, I need to add the nrf_mesh_evt_handler while mesh initilization.

Edvin said:

Give it a go, and let me know if you get stuck.

If I am trying to push the dfu packets over the air but not serial, I should push it from the nrf connect app right? or can we do it from the mesh app? Sorry but I have no idea about how to push it OTA!!

Thank you.

[Edvin]

If you store data in a new area in the flash, the flash manager needs to be aware. Changing this with a FW update, either a DFU or by manually programming the device will not erase the flash part that is used for application data storage, and hence the flash manager is still initialized with the old settings. So it is not trivial to change the size that the flash manager will use after it has been initialized.

I don't think this is actually DFU related at all. You need your initial FW to use the same flash manager settings as the one you intend to use in the end.

[DeveloperZ]

Hi,

I have been trying to analyze the messages sent back and forth between the PC and the target while establishing the initial UART communication.

Case1:

Sending DFU start packet, afterwards we wait for the flash on target to be initialized before continuing.
PC -> target: 0502aabbccdd
target -> PC: 0582aabbccdd
Got echo response
Sending DFU init packet
PC -> target: 1378fdff040fd3260a7859000000010003000000
target -> PC: 03847800
PC -> target: 1378fdff040fd3260a7859000000010003000000
target -> PC: 03847800
PC -> target: 1478fcff0000d3260a78ffffffff5d71000040000c
target -> PC: 03847887

I assume that the PC -> target message contains 59-company id, 1 - app-id, 3- application version that I sent. Is that right?

The response from the target to PC is 03847800. What does this indicate?

The third response back from the target to PC is 03847887. This is different from the first two responses. When the PC gets this response back, the DFU stops(fails). What does this value indicate?

Is there any link that I can refer to regarding these messages?

Case2:

Sending DFU start packet, afterwards we wait for the flash on target to be initialized before continuing.
PC -> target: 0502aabbccdd
target -> PC: 0582aabbccdd
Got echo response
Sending DFU init packet
PC -> target: 1378fdff040f2625fa6859000000010004000000
target -> PC: 16a6045900000001000400000059000000010001000000
target -> PC: 03847800
PC -> target: 1378fdff040f2625fa6859000000010004000000
target -> PC: 03847800
PC -> target: 1478fcff00002625fa68ffffffff5995000040000c
target -> PC: 0ea30104590000000100040000000d
target -> PC: 03847800
Sending firmware file
[------------------------------------] 1% 00:11:49PC -> target: 1978fcff01002625fa6800f0002071630200596302005b630200
target -> PC: 03847887

In this case, the target sends two message as a reply which wasn't done in the first case. Why does the target send variable amount of messages?

Also, the transfer began and then it was stopped when it received a different response back from the target. What might cause this behavior?

There have been situations where I successfully ran the DFU when the target responded back with one message and with two messages.

Why different responses for same/similar message packets sent from PC to target?

[Edvin]

Hello,

Please check out the Serial commands, Serial events and Serial status codes in the mesh sdk documentation.

So, basically, 03847887 means:

03: Internal Events Report

84: Cmd Response

78: DFU Data

87: ERROR_INVALID_DATA

So it means that the init packet is not accepted.

We have seen some weird behavior on certain compiler versions. So what IDE/compiler do you use? Segger Embedded Version? What version?

[DeveloperZ]

Edvin said:

We have seen some weird behavior on certain compiler versions. So what IDE/compiler do you use? Segger Embedded Version? What version?

Since you suggested previously, I have been using only SES v3.40. But I keep getting this response frequently.

Additionally, I have noticed that once the DFU transfer stops in the middle(for example, my PC went to sleep after transferring 87% which stopped sending packets); there is no way to resume the DFU from that point. I tried pushing the new update with increased version number. It failed sending the error code for ERROR_INVALID_DATA.

How does the flash for new FW image get freed? Does it gets overridden overtime new DFU is pushed or does the flash gets erased after the node loads the new FW?

The error that I mentioned, is there any chance that it could be because of the DFU packets stored during the previous transfer.

[Edvin]

DeveloperZ said:

Since you suggested previously, I have been using only SES v3.40. But I keep getting this response frequently.

 Sorry. I am out travelling, so I didn't read through the thread.

 

DeveloperZ said:

I tried pushing the new update with increased version number. It failed sending the error code for ERROR_INVALID_DATA.

 That is because the device is probably expecting the next packet in the sequence. It should be possible to resume by sending the next packet, but this is not something that nrfutil currently supports. You would have to do this by altering the nrfutil FW, or wait until the DFU times out, and start over.

 

DeveloperZ said:

The error that I mentioned, is there any chance that it could be because of the DFU packets stored during the previous transfer.

 Yes. If the device is still in DFU mode with a previous image, then you have to wait until it times out. Check out the reply that describes TIMER_DATA_TIMEOUT_US. There is also a define TIMER_START_TIMEOUT_US (both in nrf_mesh_dfu.c). If any of these times out, the DFU is stopped, and it will be ready to receive a new packet.

[DeveloperZ]

Hi,

How many DFU pushes can the board handle?

Is there any time frame needed between each push?

I am having hard time pushing the DFU multiple times.I have been pushing the same FW image of DFU example as well as customized example with DFU over and over again but the second DFU push fails most of the time. Sometimes, second push is successful but third fails. It fails sending the error code for ERROR_INVALID_DATA.

I did try it by sending commands individually and also using scripts(version number is increased everytime). If there was something wrong with either commands or the scripts, it would have failed in the first push but it fails on second or third push. Also,  I am not even changing the hex files for pushing. It is the same file that was used for the first push.

In the customized example, I do change the name and push it so that I see the indication that the device is successfully updated with new FW image.

DeveloperZ said:

How does the flash for new FW image get freed? Does it gets overridden overtime new DFU is pushed or does the flash gets erased after the node loads the new FW?

I dunno why but I feel that the flash which stores the new FW image isn't accepting the new image as I can see that the initial messages that are sent to check the ID's reply back with Success code which is 0x00.

The DFU start message which carries the start address, size of the application is the one that is returning back with the ERROR_INVALID_DATA reply

The only difference that I see with the DFU start message when the DFU is pushed successfully and when it fails is the transfer ID field in the message packet. I don't see anything else changing. So, I am still unsure on what is going wrong while trying to push the same FW again and again.

It would be great if you or any of your colleagues find time to push DFU example multiple times onto the same board and see if there is anything wrong with that?

[Edvin]

DeveloperZ said:

How many DFU pushes can the board handle?

 1 at the time.

 

DeveloperZ said:

Is there any time frame needed between each push?

 Yes.

You need to check out the defines that i mentioned in the previous reply. TIMER_DATA_TIMEOUT_US and TIMER_START_TIMEOUT_US. It will not accept a new image until any of these times out. Please let me know if you can't find them.

As mentioned. I am out of office, so I don't have any HW to test this on. Please open a new ticket so that one of the guys in the office can check. 

 

DeveloperZ said:

while trying to push the same FW again and again.

 One of these two is happening:

1: The application that is stored is already up to date. You need to increase the application version to make the device accept it.

2: The device is still expecting the next packet from a previous session because TIMER_DATA_TIMEOUT_US has not timed out and TIMER_START_TIMEOUT_US has not timed out.

Best regards,

Edvin

[DeveloperZ]

Edvin said:

You need to check out the defines that i mentioned in the previous reply. TIMER_DATA_TIMEOUT_US and TIMER_START_TIMEOUT_US.

I found these and they are both around 10seconds and 50 seconds respectively. I am waiting for few minutes between each push.

Edvin said:

The application that is stored is already up to date. You need to increase the application version to make the device accept it.

I am sure that the version number is increased as I can see the version number in the DFU messages exchanged before the push.

Edvin said:

The device is still expecting the next packet from a previous session because TIMER_DATA_TIMEOUT_US has not timed out and TIMER_START_TIMEOUT_US has not timed out.

I don't think that this might be the case because the customized application that I have advertises its name and I can see the name change on the node after the DFU is successfully pushed. Similarly I added a command for LED's to blink after loading the new FW on DFU example so that it gives me an indication of successful DFU transfer and updating the firmware. 

Edvin said:

Please open a new ticket so that one of the guys in the office can check. 

Okay. I will try this.

While going through all this, I just programmed the dev board with DFU example in Mesh SDK3.1.0 using Segger embedded studio. I noticed this which I feel is weird.

Are the logs supposed to look like this?

[Edvin]

It shouldn't print that in an unmodified example/SDK.

Can you try with a clean SDK, download and unzip from here.

Maybe you have defined "HOST" in some file in another project that affects the DFU example. If you look in nrf_meshdfu_init() in nrf_mesh_dfu.c, it will set the dfu_cmd_handler if "HOST" is not defined. So if it is defined, this cmd handler is never set, and you will get the error that you are currently seeing.

Try with an unmodified SDK. If that works, try to open your custom project in this SDK, and see if it behaves the same.

[DeveloperZ]

Okay.

Finally I was able to have the DFU working. It looks like the size of the DFU was an issue. I was pushing the image created using the hex file generated in debug mode. I tried it with the hex file built in release mode along with the patch given by your colleague in this thread.

 Now, I changed the company ID from nordic's to our own company ID. It fails to push.
If I try to push it with Nordic's company ID, the DFU is successful.

I changed the CompanyID in the FW(access.h), device page. Tried to push the update(with new Company ID) which failed replying back ERROR_INVALID_DATA.

Is there any hardcoded information in the nrfutil tool or somewhere else where we need to add new Company ID's?

Or what might be the reason that the update fails when I push with a company ID which isn't Nordic's?