Serial LTE Modem Application (NRF9160) FOTA does not apply new update

[ronald-blackline]

Hi,

I have the SLM application running in a custom board with uses the NRF9160. The SLM is driven an external microprocessor within the same board. Everything seems to work but FOTA will not apply the update.

Basically, we send the following commands after we confirmed successful network registration:

AT#XFOTA=8
AT#XFOTA=1,"">xxxxxx.amazonaws.com/app_update.bin"

This sequence of commands downloaded the FOTA image successfully. Then, we apply the AT#RESET command and this is what we are seeing from RTT

00> *** Booting Zephyr OS build v2.7.0-ncs1 ***
00> I: Starting bootloader
00> I: Primary image: magic=good, swap_type=0x4, copy_done=0x1, image_ok=0x1
00> I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3
00> I: Boot source: none
00> I: Swap type: test
00> D: erasing trailer; fa_id=3
00> D: initializing status; fa_id=3
00> D: writing swap_info; fa_id=3 off=0x6ffd8 (0x7ffd8), swap_type=0x2 image_num=0x0
00> D: writing swap_size; fa_id=3 off=0x6ffd0 (0x7ffd0)
00> D: writing magic; fa_id=3 off=0x6fff0 (0x7fff0)
00> D: erasing trailer; fa_id=7
00> D: writing copy_done; fa_id=3 off=0x6ffe0 (0x7ffe0)
00> I: Bootloader chainload address offset: 0x10000
00> I: Jumping to the first image slot
00> *** Booting Zephyr OS build v2.7.0-ncs1 ***
00> I: Starting bootloader
00> I: Primary image: magic=good, swap_type=0x2, copy_done=0x1, image_ok=0x3
00> I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
00> I: Boot source: none
00> I: Swap type: revert
00> I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
00> D: erasing trailer; fa_id=7
00> D: writing image_ok; fa_id=7 off=0x6ffe8 (0xeffe8)
00> D: writing swap_size; fa_id=7 off=0x6ffd0 (0xeffd0)
00> D: writing magic; fa_id=7 off=0x6fff0 (0xefff0)
00> D: erasing trailer; fa_id=3
00> D: initializing status; fa_id=3
00> D: writing swap_info; fa_id=3 off=0x6ffd8 (0x7ffd8), swap_type=0x4 image_num=0x0
00> D: writing image_ok; fa_id=3 off=0x6ffe8 (0x7ffe8)
00> D: writing swap_size; fa_id=3 off=0x6ffd0 (0x7ffd0)
00> D: writing magic; fa_id=3 off=0x6fff0 (0x7fff0)
00> D: erasing trailer; fa_id=7
00> D: writing copy_done; fa_id=3 off=0x6ffe0 (0x7ffe0)
00> I: Bootloader chainload address offset: 0x10000
00> I: Jumping to the first image slot

After this, the SLM gets run again and we read the following response from the application:

b'\x00Ready\r\n'
b'\r\n'
b'#XFOTA: 5,1,-9\r\n'

This basically indicates failure in the FOTA process. Can someone please help.

[Håkon Alseth]

Hi,

 

ronald-blackline said:

i tried programming the app_moved_test_update.hex but the result is the same, getting reverted back.

Are you certain that the original pm_static.yml is the one that you're currently using?

Meaning, are you certain that the static addresses are indeed matching those that your mcuboot expects?

 

I do not see this by using your initial pm_static.yml, but that might also be because I use that as the "starting point" to ensure that the partition layout is equal for the pending update image.

 

What I did for testing your process:

* used your pm_static.yml, and set FPROTECT off.

* Configured ncs v1.5.0 SLM and flashed that.

* checked out ncs v1.8.0, reconfigured SLM

* Changed the printout, just to see that the image booted (from "Ready" to "Update Ready")

* flashed the ncs v1.8.0 "app_moved_test_update.hex" using nrfjprog into the mcuboot secondary slot (starting at 0x75000).

* Reset the board, and saw that the image booted and printed my modified string (ie. Update Ready).

 

* (optional step) call boot_write_img_confirmed() on the ncs v1.8.0 image to make the new image permanent. If this is not done, the image will revert to ncs v1.5.0 on the next reset.

 

Kind regards,

Håkon

[ronald-blackline]

Håkon Alseth said:

Meaning, are you certain that the static addresses are indeed matching those that your mcuboot expects?

How do I verify this?

[Håkon Alseth]

Hi,

 

ronald-blackline said:

How do I verify this?

Do you have the original build (and artifacts, such as the partitions.yml?)?

 

Could you confirm if the "update images" gets executed at all, or is it specifically only the "boot_write_img_confirmed()" that is not making the image permanent? From the logs that you have provided, it seems that the app_*.hex files does not seem to run properly.

  

Kind regards,

Håkon

[ronald-blackline]

I was able to make it work but I have to remove one of the required configuration I have to use in my project. I have this config defined:

CONFIG_SLM_START_SLEEP=y

Whenever I removed this configuration in the project, the actual image swap is successful. If I put it back in, the swap reverts back (thus failing). I investigated and whenever the following APIs are called (in main.c) ...

nrf_power_resetreas_get(NRF_POWER_NS);
nrf_power_resetreas_clear(NRF_POWER_NS, 0x70017);

... the swap fails.

As a workaround I moved the following calls before the *nrf_power_resetreas_* APIs:

handle_nrf_modem_lib_init_ret();
handle_mcuboot_swap_ret();

This actually makes swapping successful although the FOTA completion message says otherwise:

#XFOTA: 5,1,-9

This is the last issue Im looking and I think the swapping issue is now behind us.

[Håkon Alseth]

Hi,

 

Thank you very much for the detailed update.

ronald-blackline said:

I was able to make it work but I have to remove one of the required configuration I have to use in my project. I have this config defined:

CONFIG_SLM_START_SLEEP=y

Whenever I removed this configuration in the project, the actual image swap is successful. If I put it back in, the swap reverts back (thus failing). I investigated and whenever the following APIs are called (in main.c) ...

nrf_power_resetreas_get(NRF_POWER_NS);
nrf_power_resetreas_clear(NRF_POWER_NS, 0x70017);

... the swap fails.

Yes, I can see that this scenario will fail on my end as well. I have submitted a bug report internally on the SLM project to make the developers aware of this issue! Thank you for updating and helping us improve the application!

 

ronald-blackline said:

As a workaround I moved the following calls before the *nrf_power_resetreas_* APIs:

handle_nrf_modem_lib_init_ret();
handle_mcuboot_swap_ret();

This actually makes swapping successful although the FOTA completion message says otherwise:

#XFOTA: 5,1,-9

This is the last issue Im looking and I think the swapping issue is now behind us.

I believe that the XFOTA return error is a issue that was recently fixed in ncs v1.9.0:

https://github.com/nrfconnect/sdk-nrf/pull/6918

 

Kind regards,

Håkon