Update from nRF5 SDK to nRF Connect SDK

[QuentinD]

Hello,

For a month I have been trying to find a solution to update my devices which are at my customers (these devices have a bootloader, softdevice and app coming from the nrf5 SDK) to the mcuboot and app coming from the NCS SDK.

To do this, I relied on this link: https://devzone.nordicsemi.com/f/nordic-q-a/79571/dfu-update-from-nrf5-sdk-to-nrf-connect-sdk

Here are my steps:

1. I modified the secure_bootloader_uart_mbr_pca10056_debug of the nrf5 SDK so that it accepts replacing the sofdevice and the app (nrf5 SDK) with my new app (NCS SDK):
   - comments "return NRF_ERROR_INTERNAL" in the sd_activate() function of the nrf_bootloader_fw_activation.c file
   - comments "result = false;" of the if(SD_MAGIC_NUMBER_GET(sd_start_addr) of the softdevice_info_ok(...) function of the nrf_dfu_validation.c file
2. I created a file containing this bootloader patched with the softdevice and my app (nrf5 SDK):
   nrfutil settings generate --family NRF52840 --application BT840-FW-0.1.0.12.hex --application-version-string "1.0.12" --bootloader-version 3 --bl-settings-version 2 --softdevice BT840- SD-0.7.2.0.hex --key-file priv.pem bootloader_setting.hex
   mergehex --merge bootloader_setting.hex BT840-BL-0.3.hex --output boot_with_settings.hex
   mergehex --merge boot_with_settings.hex BT840-SD-0.7.2.0.hex --output boot_with_settingsAndSofdevice.hex
   mergehex --merge boot_with_settingsAndSofdevice.hex BT840-FW-0.1.0.12.hex --output firmware-BT840-HW3-SD7_2_0-BOOT3-FW1_0_12.hex
3. I uploaded this file to my nrf52840
4. To generate the update packages for the bootloader (MCUboot) and my app from the NCS SDK, I did this:
   nrfutile pkg generate --hw-version 3 --app-boot-validation VALIDATE_GENERATED_CRC --bootloader-version 4 --sd-req 0x0,0xCA,0x0100,0x0101 --bootloader Application/TA_Smart/mcuboot/zephyr/zephyr.hex - -key-file Application/child_image/key/mcuboot_private.pem BT840_update_with_boot.zip
   nrfuutil pkg generate --hw-version 3 --app-boot-validation VALIDATE_GENERATED_CRC --sd-req 0x0,0xCA,0x0100,0x0101 --softdevice Application/TA_Smart/zephyr/app_update.bin --key-file Application/child_image/ key/mcuboot_private.pem BT840_update_with_app.zip
5. I send the DFU update of the softdevice
6. Once it is finished, I send the bootloader DFU update

Here are the logs of these 2 updates:

 Test_update_to_new_sdk.log

Here is the memory map of my devices working with the nrf5 SDK:

mbr: 0x0 -> 0x1000
softdevice: 0x1000 -> 0x27000
app: 0x27000 -> 0xD0000 --------> the application uses BLE and Thread
storage: 0xD0000 -> 0xD1000
unused page: 0xD1000 + 0xE0000
ot_flash_data: 0xE0000 -> 0xE4000
bootloader: 0xE4000 -> 0xF0000
unused_page2: 0xF0000 -> 0xFE000
mbr_params_page: 0xFE000 -> 0xFF000
bootloader_settings_page: 0xFF000 -> 0x100000

Here is the content of my pm_static.yml from my project using the NCS SDK: 

mbr:
  address: 0x0
  region: flash_primary
  size: 0x1000
app:
  address: 0x1200
  region: flash_primary
  size: 0xD0E00
mcuboot:
  address: 0xE4000
  placement:
    after:
    - mcuboot_secondary
  region: flash_primary
  size: 0x1A000
mcuboot_pad:
  address: 0x1000
  placement:
    align:
      start: 0x1000
    before:
    - mcuboot_primary_app
  region: flash_primary
  size: 0x200
mcuboot_primary:
  address: 0x1000
  orig_span: &id001
  - app
  - mcuboot_pad
  region: flash_primary
  sharers: 0x1
  size: 0xD1000
  span: *id001
mcuboot_primary_app:
  address: 0x1200
  orig_span: &id002
  - app
  region: flash_primary
  size: 0xD0E00
  span: *id002
mcuboot_secondary:
  address: 0xD2000
  placement:
    after:
    - mcuboot_primary
    align:
      start: 0x1000
  region: flash_primary
  share_size:
  - mcuboot_primary
  size: 0x10000
settings_storage:
  address: 0xE2000
  placement:
    before:
    - mcuboot
  region: flash_primary
  size: 0x2000
mbr_params_page:
  address: 0xFE000
  region: flash_primary
  size: 0x1000
bootloader_settings_page:
  address: 0xFF000
  region: flash_primary
  size: 0x1000  
sram_primary:
  address: 0x20000000
  region: sram_primary
  size: 0x40000

Additional information:

• After updating the softdevice (step 5), my file was written between address 0x1000 and address 0xC11DF (thus overwriting the old softdevice and app). On the other hand, in the logs, I see "<debug> nrf_dfu_validation: Hash verification. start address: 0x27000, size: 0x9A1D4": is it normal that it says "0x27000" and not "0x1000"?
• If after this update, I send the bootloader (step 6), it also writes the file at address 0x1000 and not at address 0xE4000: do you know why?
• If I update the bootloader before the softdevice, I see in the logs that it starts writing from address 0x99000 (in the logs and physically): why is it at this address?
• It's obvious but of course, my project using the NCS SDK works when I use it with the immutable bootloader + MCUboot + app

Once I updated my device to MCUboot and the app from the NCS SDK, I no longer see anything in the RTT logs and I can't see which instruction the nrf52840 is stuck on.
Do you have any idea where the problem could be coming from?

Thank you

[Sigurd Hellesvik]

Hi

Have you seen  Hang with nRF5 SDK 17.1.0 Bootloader and nRF Connect SDK 2.1.0 application ?

I don't have an answer for all of your questions, so I will ask some questions so we can get more info.

You say you crash after DFU, right. Can I suggest that you try to flash hex files using nrfjprog to the device and see if the application works then?

It's obvious but of course, my project using the NCS SDK works when I use it with the immutable bootloader + MCUboot + app

Do logging work for this?
Do you log from the application only, or also from MCUboot?

Regards,
Sigurd Hellesvik

[QuentinD]

Hi,

Yes I have already seen this link.

Ok I'll test that a bit later.

Yes, I activated the MCUboot and app logs: both work.

Another question: here I do a softdevice update (in this file, I have my NCS SDK app) which overwrites the memory area of the softdevice but also a part of the memory area of the app. So, when the bootloader restarts it will consider the softdevice is validated but the app is invalidated. So how can I make it jump to address 0x1000 (new app address) after the update?

[Sigurd Hellesvik]

QuentinD said:

This larger microcontroller communicates with the nrf52840 via uart and it's him who updates the nrf52840.

Oh, then you can definitively use the Dongle method if you want.

The downside of the donlge method is that you first upload the bootloader and then the new app, which works poorly if you use BLE.

However, if you have UART available, you can run Serial Recovery and upload the new application directly via MCUboot at the next reboot. Does that make sense?

In this case, MCUboot can write over whatever parts of flash it want, except for itself and the old bootloader and mbr.

QuentinD said:

The only thing I don't understand in his example is how by writing the new app+boot instead of the old app, that it goes after re-writing these files at address 0x1000 (after the MBR) and the 'other file at the end of memory (bootloader)?

I do not think I understand the question?

[QuentinD]

Yes I know that but how can I replace the softdevice and the old bootloader with this method ? If I write the new app and MCUboot to the app partition of my old device, how can I remove the softdevice and the nrf5 bootloader ? (I don't want to loose memory)

If you have a device that works with the nrf5 SDK and has MBR+softdevice+app+bootloader in its memory, how would you do to replace (steps) the softdevice+app+bootloader by the app+MCUboot from the NCS SDK?

[Vidar Berg]

The idea is to package the Zephyr app and MCUBoot images as a SoftDevice+bootloader update, allowing the existing SoftDevice and bootloader to be replaced, as illustrated by the picture in my post here:  RE: Hang with nRF5 SDK 17.1.0 Bootloader and nRF Connect SDK 2.1.0 application  

[QuentinD]

Thank you for your explanation.

Ah I thought when I saw the image that we had to send an update of the app with a file which contains the NCS App and MCUboot. So you should instead do a combined update of the softdevice+bootloader (in which the softdevice update file will contain the NCS app and the bootloader update file will contain the MCUboot)?

Can we send the softdevice and after the next reboot to send the bootloader or must both be written at the same time?

Does this solution require you to first go through a patched bootlaoder?

[Vidar Berg]

QuentinD said:

Ah I thought when I saw the image that we had to send an update of the app with a file which contains the NCS App and MCUboot. So you should instead do a combined update of the softdevice+bootloader (in which the softdevice update file will contain the NCS app and the bootloader update file will contain the MCUboot)?

Correct.

QuentinD said:

Can we send the softdevice and after the next reboot to send the bootloader or must both be written at the same time?

They must be written at the same time.

QuentinD said:

Does this solution require you to first go through a patched bootlaoder?

You can include the Softdevice info structure in the zephyr app to trick the bootloader into thinking it is a valid Softdevice. See the example attached to my other devzone post for how you can include this structure.

[QuentinD]

Hi Vidar,

Thank you very much for your help: I have made a lot of progress :)

But here I still have a problem: the file that I am trying to write is not completely written in memory (the end of the file is missing) although according to the logs everything was written correctly.
Just to check that there is no problem in the bootloader code for writing large files, I tested what it would do if I had only updated the application by filling almost all the "app" partition. For this, I wrote from address 0x27000 to address 0xC2C3B (size=0x9BC3B) and everything was written correctly.

If I now send my file containing the NCS App + MCUboot to overwrite the softdevice (file of 0xA9E80 bytes), it writes in flash from the address 0x1000 to an address around the address 0x7EB00 (I say "around" since if I change the delay between sending 2 frames to the bootloader, the last written address changes (but not much)) whereas it should go up to address 0xAAE80.

Here is the memory map of my nrf52840 before the update:

• mbr: 0x0 -> 0x1000
• softdevice: 0x1000 -> 0x26000
• app: 0x27000 -> 0xD0000 (0xA9000) -> in reality my app goes from 0x27000 to 0x98CEB
• storage: 0xD0000 -> 0xD1000
• unused page: 0xd1000 -> 0xe0000
• ot_flash_data: 0xe0000 -> 0xe4000
• bootloader: 0xE4000 -> 0xF0000
• unused_page2: 0xF0000 -> 0xFE000
• mbr_params_page: 0xFE000 -> 0xFF000
• bootloader_settings_page: 0xFF000 -> 0x100000

Here are the bootloader logs during the update:
Test_BT840_V0.1.0.12.log

I'm trying to send these files:

sd_bl.bin

sd_bl.dat

This file was built from these 2 files:
app_signed.hex 1411.zephyr.hex

Remarks:

• We see that physically, the update overwrites in flash the softdevice and part of the NRF5 application with part of the NCS SDK application. On the other hand, we see this in the logs:
  <debug> nrf_dfu_flash: nrf_fstorage_write(addr=0x00027000, src=0x200046A0, len=64 bytes), queue usage: 1
  ...
  <debug> nrf_dfu_flash: nrf_fstorage_write(addr=0x000D0E40, src=0x2000461C, len=64 bytes), queue usage: 1
  Why is it indicated "0x27000" as the starting address and not 0x1000 because physically that is where it is written? Isn't the problem I'm having related to this?
• I can write the entire "app" partition if I don't start writing by starting at the address of the softdevice so we already know that there is no problem writing in flash for large files and no problem writing to addresses where I have a problem.
• In the logs, all the writes are marked as "success" but in reality, rereading the flash, I see that the end is not saved, do you have any idea?

[QuentinD]

If I flash the app_signed.hex and zephyr.hex file to my nrf52840, I have these logs:

00> *** Booting nRF Connect SDK v2.5.0 ***
00> [00:00:00.000,213] <inf> mcuboot: Starting bootloader
00> [00:00:00.000,701] <wrn> mcuboot: Cannot upgrade: not a compatible amount of sectors
00> [00:00:10.572,021] <inf> mcuboot: Bootloader chainload address offset: 0x1000
00> [00:00:10.572,021] <inf> mcuboot: Jumping to the first image slot
00> [00:00:10.572,052] <err> mcuboot: Protect mcuboot flash failed, cancel startup.

Do you know what can cause this kind of problem?

[Vidar Berg]

Hi,

For this error:

QuentinD said:

00> [00:00:10.572,052] <err> mcuboot: Protect mcuboot flash failed, cancel startup.

Try to build mcuboot with CONFIG_FPROTECT=n. I recall there were some issues with the flash protection mechanism after relocating the bootloader to the end of flash (in nRF Connect SDK the bootloader starts at the beginning of flash). 

QuentinD said:

00> [00:00:00.000,701] <wrn> mcuboot: Cannot upgrade: not a compatible amount of sectors

This suggests that the MCUBoot secondary slot is not allocated in flash. Please run the "memory report" action in VS code and post the flash partition layout here.

Best regards,

Vidar 

[QuentinD]

Hi,

Putting "CONFIG_FPROTECT=n" was enough: now if I flash the files directly, it works! THANKS !

On the other hand, when I send the file by DFU and then I read the flash again, I notice that it does not write the end of the file. It does erase the block in the flash but it does not write the data. During my last test, it correctly wrote the data up to address 0x7E81b in flash but after that it no longer wrote anything (it should go up to address 0x9CED3) but nevertheless in the bootloader logs all the writes are "success": do you have any idea what could be causing this problem?

Can you look at my bootloader log from my previous post to see if you see a problem?

[Vidar Berg]

Hi,

I'm glad to hear that it resolved the issue with MCUBoot not booting.

QuentinD said:

On the other hand, when I send the file by DFU and then I read the flash again, I notice that it does not write the end of the file. It does erase the block in the flash but it does not write the data. During my last test, it correctly wrote the data up to address 0x7E81b in flash but after that it no longer wrote anything (it should go up to address 0x9CED3) but nevertheless in the bootloader logs all the writes are "success": do you have any idea what could be causing this problem?

The merged Zephyr app + bootloader binary is first loaded to 0x27000. After this, the bootloader is supposed to activate the images by copying them to their respective slots. However, the activation step is skipped because the bootloader does not see the Zephyr app as a valid Softdevice:

00> <error> app: Source address does not contain a valid SoftDevice.