Update from nRF5 SDK to nRF Connect SDK

[QuentinD]

Hello,

For a month I have been trying to find a solution to update my devices which are at my customers (these devices have a bootloader, softdevice and app coming from the nrf5 SDK) to the mcuboot and app coming from the NCS SDK.

To do this, I relied on this link: https://devzone.nordicsemi.com/f/nordic-q-a/79571/dfu-update-from-nrf5-sdk-to-nrf-connect-sdk

Here are my steps:

1. I modified the secure_bootloader_uart_mbr_pca10056_debug of the nrf5 SDK so that it accepts replacing the sofdevice and the app (nrf5 SDK) with my new app (NCS SDK):
   - comments "return NRF_ERROR_INTERNAL" in the sd_activate() function of the nrf_bootloader_fw_activation.c file
   - comments "result = false;" of the if(SD_MAGIC_NUMBER_GET(sd_start_addr) of the softdevice_info_ok(...) function of the nrf_dfu_validation.c file
2. I created a file containing this bootloader patched with the softdevice and my app (nrf5 SDK):
   nrfutil settings generate --family NRF52840 --application BT840-FW-0.1.0.12.hex --application-version-string "1.0.12" --bootloader-version 3 --bl-settings-version 2 --softdevice BT840- SD-0.7.2.0.hex --key-file priv.pem bootloader_setting.hex
   mergehex --merge bootloader_setting.hex BT840-BL-0.3.hex --output boot_with_settings.hex
   mergehex --merge boot_with_settings.hex BT840-SD-0.7.2.0.hex --output boot_with_settingsAndSofdevice.hex
   mergehex --merge boot_with_settingsAndSofdevice.hex BT840-FW-0.1.0.12.hex --output firmware-BT840-HW3-SD7_2_0-BOOT3-FW1_0_12.hex
3. I uploaded this file to my nrf52840
4. To generate the update packages for the bootloader (MCUboot) and my app from the NCS SDK, I did this:
   nrfutile pkg generate --hw-version 3 --app-boot-validation VALIDATE_GENERATED_CRC --bootloader-version 4 --sd-req 0x0,0xCA,0x0100,0x0101 --bootloader Application/TA_Smart/mcuboot/zephyr/zephyr.hex - -key-file Application/child_image/key/mcuboot_private.pem BT840_update_with_boot.zip
   nrfuutil pkg generate --hw-version 3 --app-boot-validation VALIDATE_GENERATED_CRC --sd-req 0x0,0xCA,0x0100,0x0101 --softdevice Application/TA_Smart/zephyr/app_update.bin --key-file Application/child_image/ key/mcuboot_private.pem BT840_update_with_app.zip
5. I send the DFU update of the softdevice
6. Once it is finished, I send the bootloader DFU update

Here are the logs of these 2 updates:

 Test_update_to_new_sdk.log

Here is the memory map of my devices working with the nrf5 SDK:

mbr: 0x0 -> 0x1000
softdevice: 0x1000 -> 0x27000
app: 0x27000 -> 0xD0000 --------> the application uses BLE and Thread
storage: 0xD0000 -> 0xD1000
unused page: 0xD1000 + 0xE0000
ot_flash_data: 0xE0000 -> 0xE4000
bootloader: 0xE4000 -> 0xF0000
unused_page2: 0xF0000 -> 0xFE000
mbr_params_page: 0xFE000 -> 0xFF000
bootloader_settings_page: 0xFF000 -> 0x100000

Here is the content of my pm_static.yml from my project using the NCS SDK: 

mbr:
  address: 0x0
  region: flash_primary
  size: 0x1000
app:
  address: 0x1200
  region: flash_primary
  size: 0xD0E00
mcuboot:
  address: 0xE4000
  placement:
    after:
    - mcuboot_secondary
  region: flash_primary
  size: 0x1A000
mcuboot_pad:
  address: 0x1000
  placement:
    align:
      start: 0x1000
    before:
    - mcuboot_primary_app
  region: flash_primary
  size: 0x200
mcuboot_primary:
  address: 0x1000
  orig_span: &id001
  - app
  - mcuboot_pad
  region: flash_primary
  sharers: 0x1
  size: 0xD1000
  span: *id001
mcuboot_primary_app:
  address: 0x1200
  orig_span: &id002
  - app
  region: flash_primary
  size: 0xD0E00
  span: *id002
mcuboot_secondary:
  address: 0xD2000
  placement:
    after:
    - mcuboot_primary
    align:
      start: 0x1000
  region: flash_primary
  share_size:
  - mcuboot_primary
  size: 0x10000
settings_storage:
  address: 0xE2000
  placement:
    before:
    - mcuboot
  region: flash_primary
  size: 0x2000
mbr_params_page:
  address: 0xFE000
  region: flash_primary
  size: 0x1000
bootloader_settings_page:
  address: 0xFF000
  region: flash_primary
  size: 0x1000  
sram_primary:
  address: 0x20000000
  region: sram_primary
  size: 0x40000

Additional information:

• After updating the softdevice (step 5), my file was written between address 0x1000 and address 0xC11DF (thus overwriting the old softdevice and app). On the other hand, in the logs, I see "<debug> nrf_dfu_validation: Hash verification. start address: 0x27000, size: 0x9A1D4": is it normal that it says "0x27000" and not "0x1000"?
• If after this update, I send the bootloader (step 6), it also writes the file at address 0x1000 and not at address 0xE4000: do you know why?
• If I update the bootloader before the softdevice, I see in the logs that it starts writing from address 0x99000 (in the logs and physically): why is it at this address?
• It's obvious but of course, my project using the NCS SDK works when I use it with the immutable bootloader + MCUboot + app

Once I updated my device to MCUboot and the app from the NCS SDK, I no longer see anything in the RTT logs and I can't see which instruction the nrf52840 is stuck on.
Do you have any idea where the problem could be coming from?

Thank you

[Sigurd Hellesvik]

Hi

Have you seen  Hang with nRF5 SDK 17.1.0 Bootloader and nRF Connect SDK 2.1.0 application ?

I don't have an answer for all of your questions, so I will ask some questions so we can get more info.

You say you crash after DFU, right. Can I suggest that you try to flash hex files using nrfjprog to the device and see if the application works then?

It's obvious but of course, my project using the NCS SDK works when I use it with the immutable bootloader + MCUboot + app

Do logging work for this?
Do you log from the application only, or also from MCUboot?

Regards,
Sigurd Hellesvik

[QuentinD]

Hi,

Yes I have already seen this link.

Ok I'll test that a bit later.

Yes, I activated the MCUboot and app logs: both work.

Another question: here I do a softdevice update (in this file, I have my NCS SDK app) which overwrites the memory area of the softdevice but also a part of the memory area of the app. So, when the bootloader restarts it will consider the softdevice is validated but the app is invalidated. So how can I make it jump to address 0x1000 (new app address) after the update?

[QuentinD]

No, the problem is that after the update, the app will be invalidated and therefore, since the app is invalid, it will no longer jump and will therefore remain in the bootlaoder.

Yes I had already seen this solution but how can I overwrite the softdevice and the nrf5 bootloader with this solution?

In this link (devzone.nordicsemi.com/.../hang-with-nrf5-sdk-17-1-0-bootloader-and-nrf-connect-sdk-2-1- 0-application), if I understand the solution given by Vidar correctly, it's to do an "app update" with a file containing the new app and the MCUboot. But then, how are these files transferred to the right addresses so as to overwrite the old bootloader and softdevice?

[Sigurd Hellesvik]

QuentinD said:

if I understand the solution given by Vidar correctly, it's to do an "app update" with a file containing the new app and the MCUboot. But then, how are these files transferred to the right addresses so as to overwrite the old bootloader and softdevice?

The nRF5 bootloader has a feature to update both the bootloader and softdevice at the same time.
This feature is used to overwrite the bootloader with the MCUboot and the softdevice with the new app.

QuentinD said:

n this link (devzone.nordicsemi.com/.../hang-with-nrf5-sdk-17-1-0-bootloader-and-nrf-connect-sdk-2-1- 0-application),

Can I suggest that you test the sample that Vidar has uploaded there, to learn the procedure?

[Sigurd Hellesvik]

Bonus:
I found this blog on the subject from somewhere else which might be of some help:
https://www.embeddedrelated.com/showarticle/1573.php. It is not from us, so take it with a pinch of salt

[QuentinD]

Thank you.
This link is very interesting but I don't have SPI flash in my hardware so unfortunately I won't be able to use this solution as is. In my hardware, the update file is saved in a larger microcontroller (STM32) with its external flash. This larger microcontroller communicates with the nrf52840 via uart and it's him who updates the nrf52840.

I would like to test Vidar's solution on my test board (PCA10056) but unfortunately I can no longer launch an update with nrfutil (even though it worked a few years ago). When I use the command "nrfutil dfu serial -pkg BT840_update_with_boot_and_app.zip -p COM4 -b 115200 -prn 1", the update gets stuck at 0% even though I am sure it is in DFU mode.

The only thing I don't understand in his example is how by writing the new app+boot instead of the old app, that it goes after re-writing these files at address 0x1000 (after the MBR) and the 'other file at the end of memory (bootloader)?

[Sigurd Hellesvik]

QuentinD said:

This larger microcontroller communicates with the nrf52840 via uart and it's him who updates the nrf52840.

Oh, then you can definitively use the Dongle method if you want.

The downside of the donlge method is that you first upload the bootloader and then the new app, which works poorly if you use BLE.

However, if you have UART available, you can run Serial Recovery and upload the new application directly via MCUboot at the next reboot. Does that make sense?

In this case, MCUboot can write over whatever parts of flash it want, except for itself and the old bootloader and mbr.

QuentinD said:

The only thing I don't understand in his example is how by writing the new app+boot instead of the old app, that it goes after re-writing these files at address 0x1000 (after the MBR) and the 'other file at the end of memory (bootloader)?

I do not think I understand the question?

[QuentinD]

Yes I know that but how can I replace the softdevice and the old bootloader with this method ? If I write the new app and MCUboot to the app partition of my old device, how can I remove the softdevice and the nrf5 bootloader ? (I don't want to loose memory)

If you have a device that works with the nrf5 SDK and has MBR+softdevice+app+bootloader in its memory, how would you do to replace (steps) the softdevice+app+bootloader by the app+MCUboot from the NCS SDK?

[Vidar Berg]

The idea is to package the Zephyr app and MCUBoot images as a SoftDevice+bootloader update, allowing the existing SoftDevice and bootloader to be replaced, as illustrated by the picture in my post here:  RE: Hang with nRF5 SDK 17.1.0 Bootloader and nRF Connect SDK 2.1.0 application  

[QuentinD]

Thank you for your explanation.

Ah I thought when I saw the image that we had to send an update of the app with a file which contains the NCS App and MCUboot. So you should instead do a combined update of the softdevice+bootloader (in which the softdevice update file will contain the NCS app and the bootloader update file will contain the MCUboot)?

Can we send the softdevice and after the next reboot to send the bootloader or must both be written at the same time?

Does this solution require you to first go through a patched bootlaoder?

[Vidar Berg]

QuentinD said:

Ah I thought when I saw the image that we had to send an update of the app with a file which contains the NCS App and MCUboot. So you should instead do a combined update of the softdevice+bootloader (in which the softdevice update file will contain the NCS app and the bootloader update file will contain the MCUboot)?

Correct.

QuentinD said:

Can we send the softdevice and after the next reboot to send the bootloader or must both be written at the same time?

They must be written at the same time.

QuentinD said:

Does this solution require you to first go through a patched bootlaoder?

You can include the Softdevice info structure in the zephyr app to trick the bootloader into thinking it is a valid Softdevice. See the example attached to my other devzone post for how you can include this structure.

[QuentinD]

Hi Vidar,

Thank you very much for your help: I have made a lot of progress :)

But here I still have a problem: the file that I am trying to write is not completely written in memory (the end of the file is missing) although according to the logs everything was written correctly.
Just to check that there is no problem in the bootloader code for writing large files, I tested what it would do if I had only updated the application by filling almost all the "app" partition. For this, I wrote from address 0x27000 to address 0xC2C3B (size=0x9BC3B) and everything was written correctly.

If I now send my file containing the NCS App + MCUboot to overwrite the softdevice (file of 0xA9E80 bytes), it writes in flash from the address 0x1000 to an address around the address 0x7EB00 (I say "around" since if I change the delay between sending 2 frames to the bootloader, the last written address changes (but not much)) whereas it should go up to address 0xAAE80.

Here is the memory map of my nrf52840 before the update:

• mbr: 0x0 -> 0x1000
• softdevice: 0x1000 -> 0x26000
• app: 0x27000 -> 0xD0000 (0xA9000) -> in reality my app goes from 0x27000 to 0x98CEB
• storage: 0xD0000 -> 0xD1000
• unused page: 0xd1000 -> 0xe0000
• ot_flash_data: 0xe0000 -> 0xe4000
• bootloader: 0xE4000 -> 0xF0000
• unused_page2: 0xF0000 -> 0xFE000
• mbr_params_page: 0xFE000 -> 0xFF000
• bootloader_settings_page: 0xFF000 -> 0x100000

Here are the bootloader logs during the update:
Test_BT840_V0.1.0.12.log

I'm trying to send these files:

sd_bl.bin

sd_bl.dat

This file was built from these 2 files:
app_signed.hex 1411.zephyr.hex

Remarks:

• We see that physically, the update overwrites in flash the softdevice and part of the NRF5 application with part of the NCS SDK application. On the other hand, we see this in the logs:
  <debug> nrf_dfu_flash: nrf_fstorage_write(addr=0x00027000, src=0x200046A0, len=64 bytes), queue usage: 1
  ...
  <debug> nrf_dfu_flash: nrf_fstorage_write(addr=0x000D0E40, src=0x2000461C, len=64 bytes), queue usage: 1
  Why is it indicated "0x27000" as the starting address and not 0x1000 because physically that is where it is written? Isn't the problem I'm having related to this?
• I can write the entire "app" partition if I don't start writing by starting at the address of the softdevice so we already know that there is no problem writing in flash for large files and no problem writing to addresses where I have a problem.
• In the logs, all the writes are marked as "success" but in reality, rereading the flash, I see that the end is not saved, do you have any idea?