Update from nRF5 SDK to nRF Connect SDK

[QuentinD]

Hello,

For a month I have been trying to find a solution to update my devices which are at my customers (these devices have a bootloader, softdevice and app coming from the nrf5 SDK) to the mcuboot and app coming from the NCS SDK.

To do this, I relied on this link: https://devzone.nordicsemi.com/f/nordic-q-a/79571/dfu-update-from-nrf5-sdk-to-nrf-connect-sdk

Here are my steps:

1. I modified the secure_bootloader_uart_mbr_pca10056_debug of the nrf5 SDK so that it accepts replacing the sofdevice and the app (nrf5 SDK) with my new app (NCS SDK):
   - comments "return NRF_ERROR_INTERNAL" in the sd_activate() function of the nrf_bootloader_fw_activation.c file
   - comments "result = false;" of the if(SD_MAGIC_NUMBER_GET(sd_start_addr) of the softdevice_info_ok(...) function of the nrf_dfu_validation.c file
2. I created a file containing this bootloader patched with the softdevice and my app (nrf5 SDK):
   nrfutil settings generate --family NRF52840 --application BT840-FW-0.1.0.12.hex --application-version-string "1.0.12" --bootloader-version 3 --bl-settings-version 2 --softdevice BT840- SD-0.7.2.0.hex --key-file priv.pem bootloader_setting.hex
   mergehex --merge bootloader_setting.hex BT840-BL-0.3.hex --output boot_with_settings.hex
   mergehex --merge boot_with_settings.hex BT840-SD-0.7.2.0.hex --output boot_with_settingsAndSofdevice.hex
   mergehex --merge boot_with_settingsAndSofdevice.hex BT840-FW-0.1.0.12.hex --output firmware-BT840-HW3-SD7_2_0-BOOT3-FW1_0_12.hex
3. I uploaded this file to my nrf52840
4. To generate the update packages for the bootloader (MCUboot) and my app from the NCS SDK, I did this:
   nrfutile pkg generate --hw-version 3 --app-boot-validation VALIDATE_GENERATED_CRC --bootloader-version 4 --sd-req 0x0,0xCA,0x0100,0x0101 --bootloader Application/TA_Smart/mcuboot/zephyr/zephyr.hex - -key-file Application/child_image/key/mcuboot_private.pem BT840_update_with_boot.zip
   nrfuutil pkg generate --hw-version 3 --app-boot-validation VALIDATE_GENERATED_CRC --sd-req 0x0,0xCA,0x0100,0x0101 --softdevice Application/TA_Smart/zephyr/app_update.bin --key-file Application/child_image/ key/mcuboot_private.pem BT840_update_with_app.zip
5. I send the DFU update of the softdevice
6. Once it is finished, I send the bootloader DFU update

Here are the logs of these 2 updates:

 Test_update_to_new_sdk.log

Here is the memory map of my devices working with the nrf5 SDK:

mbr: 0x0 -> 0x1000
softdevice: 0x1000 -> 0x27000
app: 0x27000 -> 0xD0000 --------> the application uses BLE and Thread
storage: 0xD0000 -> 0xD1000
unused page: 0xD1000 + 0xE0000
ot_flash_data: 0xE0000 -> 0xE4000
bootloader: 0xE4000 -> 0xF0000
unused_page2: 0xF0000 -> 0xFE000
mbr_params_page: 0xFE000 -> 0xFF000
bootloader_settings_page: 0xFF000 -> 0x100000

Here is the content of my pm_static.yml from my project using the NCS SDK: 

mbr:
  address: 0x0
  region: flash_primary
  size: 0x1000
app:
  address: 0x1200
  region: flash_primary
  size: 0xD0E00
mcuboot:
  address: 0xE4000
  placement:
    after:
    - mcuboot_secondary
  region: flash_primary
  size: 0x1A000
mcuboot_pad:
  address: 0x1000
  placement:
    align:
      start: 0x1000
    before:
    - mcuboot_primary_app
  region: flash_primary
  size: 0x200
mcuboot_primary:
  address: 0x1000
  orig_span: &id001
  - app
  - mcuboot_pad
  region: flash_primary
  sharers: 0x1
  size: 0xD1000
  span: *id001
mcuboot_primary_app:
  address: 0x1200
  orig_span: &id002
  - app
  region: flash_primary
  size: 0xD0E00
  span: *id002
mcuboot_secondary:
  address: 0xD2000
  placement:
    after:
    - mcuboot_primary
    align:
      start: 0x1000
  region: flash_primary
  share_size:
  - mcuboot_primary
  size: 0x10000
settings_storage:
  address: 0xE2000
  placement:
    before:
    - mcuboot
  region: flash_primary
  size: 0x2000
mbr_params_page:
  address: 0xFE000
  region: flash_primary
  size: 0x1000
bootloader_settings_page:
  address: 0xFF000
  region: flash_primary
  size: 0x1000  
sram_primary:
  address: 0x20000000
  region: sram_primary
  size: 0x40000

Additional information:

• After updating the softdevice (step 5), my file was written between address 0x1000 and address 0xC11DF (thus overwriting the old softdevice and app). On the other hand, in the logs, I see "<debug> nrf_dfu_validation: Hash verification. start address: 0x27000, size: 0x9A1D4": is it normal that it says "0x27000" and not "0x1000"?
• If after this update, I send the bootloader (step 6), it also writes the file at address 0x1000 and not at address 0xE4000: do you know why?
• If I update the bootloader before the softdevice, I see in the logs that it starts writing from address 0x99000 (in the logs and physically): why is it at this address?
• It's obvious but of course, my project using the NCS SDK works when I use it with the immutable bootloader + MCUboot + app

Once I updated my device to MCUboot and the app from the NCS SDK, I no longer see anything in the RTT logs and I can't see which instruction the nrf52840 is stuck on.
Do you have any idea where the problem could be coming from?

Thank you

[Sigurd Hellesvik]

Hi

Have you seen  Hang with nRF5 SDK 17.1.0 Bootloader and nRF Connect SDK 2.1.0 application ?

I don't have an answer for all of your questions, so I will ask some questions so we can get more info.

You say you crash after DFU, right. Can I suggest that you try to flash hex files using nrfjprog to the device and see if the application works then?

It's obvious but of course, my project using the NCS SDK works when I use it with the immutable bootloader + MCUboot + app

Do logging work for this?
Do you log from the application only, or also from MCUboot?

Regards,
Sigurd Hellesvik

[QuentinD]

Hi,

Yes I have already seen this link.

Ok I'll test that a bit later.

Yes, I activated the MCUboot and app logs: both work.

Another question: here I do a softdevice update (in this file, I have my NCS SDK app) which overwrites the memory area of the softdevice but also a part of the memory area of the app. So, when the bootloader restarts it will consider the softdevice is validated but the app is invalidated. So how can I make it jump to address 0x1000 (new app address) after the update?

[QuentinD]

After the file was written to nrf52840 at address 0x27000, I had a reboot during the phase of copying the fisrt part of the file to the app partition (0x1000).
In fact, if there was a restart during the copy, afther the restart it goes into the postvalidate_sd_bl() function and since softdevice_info_ok(...) returns false (SD_MAGIC_NUMBER_GET(sd_start_addr) != SD_MAGIC_NUMBER) because the address does not is no longer the right one.

To continue to be able to update the bootlaoder, what more should I do?

[Vidar Berg]

It appears that the problem arises in the nrf_bootloader_fw_activate() function, where SD_MAGIC_NUMBER_GET() is checked:

00> <debug> app: Enter nrf_bootloader_fw_activate
00> <debug> app: Valid SD + BL
00> <debug> app: Enter nrf_dfu_sd_bl_continue
00> <debug> app: Enter nrf_bootloader_dfu_sd_continue
00> <error> app: Source address does not contain a valid SoftDevice.
00> <error> app: SD+BL: SD copy failed

This is after the sd+bl image has been postvalidated. What doesn't make sense is why the bootloader fails to read the magic number in nrf_bootloader_dfu_sd_continue(). To debug this further, it would help if you could print the 'src_addr' before the SD_MAGIC_NUMBER_GET() check.

[QuentinD]

As I remember it, this is how it worked:
1. The update file was written to address 0x27000
2. The first part of the update file was copied to address 0x1000. During the copy, the nrf52840 was restarted.
The problem was that since my update file had a size of 0xA9D70 bytes, the copy process overwrote the value of the magic number which was at address 0x29004 since the reboot happened more or less when he had written to the address 0x7E81b.
So, on reboot, there was no longer a magic number at address 0x29004 since it had been overwritten.

You didn't answer me: to continue to be able to update the bootlaoder, what more should I do?

Thank you

[Vidar Berg]

QuentinD said:

The problem was that since my update file had a size of 0xA9D70 bytes, the copy process overwrote the value of the magic number which was at address 0x29004 since the reboot happened more or less when he had written to the address 0x7E81b.

The log shows that the copy routine did not start, so I don't see how the SD info struct could have been overwritten.

QuentinD said:

You didn't answer me: to continue to be able to update the bootlaoder, what more should I do?

The bootloader and SoftDevice should be activated simultaneously, but the activation is aborted due to the issue discussed above.

[QuentinD]

We do not see it in the logs because the last logs before the restart were not indicated in the log file (it did not display logs during the copy phase). If you look at the log file, you will see that the last log before the reboot was truncated.

I think you misunderstood my question. Now that I have successfully replaced the softdevice+app+bootloader with the app+MCUboot, what should I put in place to successfully update the bootloader later? (update MCUboot)

[Vidar Berg]

QuentinD said:

We do not see it in the logs because the last logs before the restart were not indicated in the log file (it did not display logs during the copy phase). If you look at the log file, you will see that the last log before the reboot was truncated.

It takes quite a while for the destination address to reach the SD info struct in the copy routine, so it is odd that the info structure would have been erased when there are no logs indicating that the copy routine even started. Have you done anything to confirm that this is indeed the case? E.g., by reading the flash content at 0x29004 with nrfjprog.

QuentinD said:

I think you misunderstood my question. Now that I have successfully replaced the softdevice+app+bootloader with the app+MCUboot, what should I put in place to successfully update the bootloader later? (update MCUboot)

As illustrated by the picture I refered to earlier, the Softdevice (zephyr app in this case) and bootloader (mcuboot) are updated after the copy routine is finished. 

[QuentinD]

Yes, it was by rereading the flash that I realized what I said above.
I just had to extend the timeout of my other microcontroller (the timeout triggers the reset pin of the nrf52840) for the copy to be done without problem.

I'm sorry but it seems you don't understand my question.
Now I can do like you in your project (I can only have these 3 element on my device: MBR+app+MCUboot). It works and I can use MCUboot to update
with my future applications.
But if one day I want to be able to update the MCUboot to a new version, what should I do now to be able to update the MCUboot in my device?

[Vidar Berg]

QuentinD said:

Yes, it was by rereading the flash that I realized what I said above.
I just had to extend the timeout of my other microcontroller (the timeout triggers the reset pin of the nrf52840) for the copy to be done without problem.

Thank you for confirming. This means there is a critical window in the copy routine where a sudden reset could cause the SD activation to fail. The fix would likely be to remove the check. The Softdevice was already validated in the postvalidation step.

QuentinD said:

I'm sorry but it seems you don't understand my question.
Now I can do like you in your project (I can only have these 3 element on my device: MBR+app+MCUboot). It works and I can use MCUboot to update
with my future applications.
But if one day I want to be able to update the MCUboot to a new version, what should I do now to be able to update the MCUboot in my device?

I understand your question now, but MCUBoot is not upgradeable. 

Vidar Berg said:

The original bootloader will be replaced by the MCUBoot bootloader if the update was completed, and this bootloader is not upgradable.

[QuentinD]

Yes exactly.

Yes I know that the MCUboot is not upgradeable: what I would like to know is in these conditions (update a device whitch is from nrf5 sdk), would it be possible to add the "immutable bootlaoder" (s0) in addition to the MCUboot in the update file to allow updating the bootloader later?

[Vidar Berg]

I don't think you will have enough flash memory to add an immutable bootloader and use MCUBoot as a second-stage bootloader. Since MCUBoot does not include its own BLE DFU transport, there normally should not be a reason to update.

That said, it might be possible to make a mechanism to update the bootloader from the app since you have the MBR with the bootloader copy function present on the device (unless MCUBoot overlaps with the MBR params page).