how to dfu from bare metal with s132 to zephyr

[ziv123]

Hello Nordic

We have a device, working on nrf52832, with s132 and nRF5 sdk11.0.0, at customers around the world.

It contains a bootloader mechanism that copies a received “.bin” image to memory and then after restart the bootloader checks the GPREGRET register and in dfu is in order it copies the image to the address of application start. 

We wish to move from bare metal (using the softdevice and the old sdk) to zephyr and use the smp service to perform dfu. So there are some question i would like to ask:

Is there a difference between the “.bin” image structure headers / magics etc.  when building with zephyr in comparison to bare metal ?

Is there some difference in memory layout ? 

Zephyr doesn't contain sofdevice but what if a softdevice already exists in memory ?

Are there different address locations and spaces for the application in zephyr and bare metal? 

Does a system running on zephyr also contain an MBR (master boot record) at address 0x00000000 ?  If not, then is it the mcuboot that is in address 0x00000000 ? and does the "west flash --erase" command puts it in address 0x00000000, cause i guess the whole application goes to slot-0, or slot-1 address, is the mcuboot part of the image generated with zephyr ?

How would you suggest upgrading such an old dfu mechanism which currently enables only to update the application (softdevice is fixed since production) to zephyr ?

 

hope to read from you soon

best regards

Ziv

[Edvin]

Hello,

There is no trivial way to upgrade from the nRF5 SDK to NCS (Zephyr) over DFU. The easiest way is to manually erase and flash the new FW using a programmer. However, I realize that this is not always an option.

You should really consider whether it is actually worth the considerate amount of work to get this working, and not to mention, test whether it is actually possible. What features from the new nRF Connect SDK do you want in your product that you can't get from the nRF5 SDK? The most common way is probably to update to NCS in a new version of the product, programmed with NCS in the factory. 

Background:

It looks like you have some fair knowledge on the bootloader from the nRF5 SDK. A BLE application from the nRF5 SDK consists of the application itself, the softdevice, the MBR and possibly a bootloader.The MBT is usually part of the softdevice. It is located at 0x00000000 -> 0x00001000 (0x0000FFFF), and this is not changed between the different softdevice versions.

In NCS however, this has changed. The Softdevice is replaced by the Softdevice Controller, which is fundamentally different. The bootloader layout is also different from the nRF5 SDKs bootloader. So for you to update via DFU from nRF5 to NCS, you need to create a DFU image that you tell the old bootloader contains a BL + SD update, so that the bootloader will remove the softdevice and the bootloader with the new bootloader setup.

As a disclaimer, this is not something that we support, but this is some pointers in case you want to test this yourself. It is also not certain that the flash will be able to hold all of these components at the same time, and in that case, it is not possible to do this. The DFU in NCS is "background DFU", which means that if you want to update to NCS, you need to be able to fit the following in flash at the same time:

MBR, SD, BL (<- all of these from the nRF5 SDK) + (all of these from NCS ->) Immutable bootloader, MCUBOOT bootloader, NCS application.

It has been tested successfully on the nRF52840, but I suspect that the nRF52832 will not fit this. 

If you are still interested in this, I suggest that you start by looking into the immutable bootloader + MCUBOOT and test this in itself before you start to try to DFU from nRF5 to NCS.

Best regards,

Edvin

[ziv123]

hi Edvin

how are you doing ?

https://devzone.nordicsemi.com/f/nordic-q-a/43516/field-upgrade-from-nrf5-sdk-to-zephyr

i saw this thread (from 3 years ago) that if i understood correctly only suggest to build the zephyr for an address not 0x0000000 (not sure how it is done either but its a direction) ... any thoughts on if it is indeed possible and how should it be done ?

Edvin said:

you need to create a DFU image that you tell the old bootloader contains a BL + SD update, so that the bootloader will remove the softdevice and the bootloader with the new bootloader setup

when you write "tell the bootloader" you mean with a bootloader settings file ? it is not so practical in our case since we use our own bootloader which only update application and not rewriting the softdevice .. 

Edvin said:

MBR, SD, BL (<- all of these from the nRF5 SDK) + (all of these from NCS ->) Immutable bootloader, MCUBOOT bootloader, NCS application.

 i am still not clear if the MCUBoot is build within the zephyr image or separately since to my understanding it is the MCUBoot that is in address 0x00000000 and not zephyr ?? 

also, if i understood your suggestion right, then because we need the existing softdevice for the ble, and we need the app to copy the new zephyr image to somewhere, that is why we will have both MBR+SD+BL+APP with all the zephyr image components as well and you are not sure if there will be enough space (off course it will also need to change the build start address for zephyr to another address) ... 

so if i understand correct, .. i think i will need to create a minimal and Lynn version of zephyr, any ideas on that ? 

maybe a lynn application just for coping the new zephyr image as well, any ideas on that ?

how can i change the zephyr build start address from 0x00000000 to something else ? and again is it the MCUBoot's address or zephyr (i am new to zephyr so have not figured out yet how it is built ) ?

can the above somehow be divided to steps in case there is a flash size limitation ?

if all the above works, how can i move back to address 0x00000000 do i copy the new image to 0x00000000 to overwrite all what is there yet, seems like i will also need place for 2 zephyrs on disk, is it possible ?

sorry for so many question 

hope to read from you soon 

best regards

Ziv 

[Edvin]

ziv123 said:

i saw this thread (from 3 years ago) that if i understood correctly only suggest to build the zephyr for an address not 0x0000000 (not sure how it is done either but its a direction) ... any thoughts on if it is indeed possible and how should it be done ?

I am not completely sure either, but for now, let's wait with this.

ziv123 said:

when you write "tell the bootloader" you mean with a bootloader settings file ? it is not so practical in our case since we use our own bootloader which only update application and not rewriting the softdevice .. 

When I say "tell the bootloader", I mean the old nRF5 bootloader. When you generate an application image you will use nrfutil to generate this image, right? nrfutil pkg generate --application-version ... --application <path to application> ...

Something like that, right? 

What do you mean by:

ziv123 said:

it is not so practical in our case since we use our own bootloader which only update application and not rewriting the softdevice .. 

Did you actively disable the possibility for softdevice and bootloader updates, or have you just not tested it? If so, can you tell me how you did that?

We can look into the details later, but for now you should check whether you are able to update the bootloader and softdevice via DFU, as this is an essensial step in being able to delete them. If you can't delete these two, then you will have very little space for your NCS app in the future.

please specify what you mean by "only update application and not rewriting softdevice" in your current bootloader.

Best regards,

Edvin

[ziv123]

hi Edvin

Edvin said:

build the zephyr for an address not 0x0000000 (not sure how it is done either but its a direction) ... any thoughts on if it is indeed possible and how should it be done ?

I am not completely sure either, but for now, let's wait with this.

What is not clear for me in that aspect, and also later, is how can an MBR be present with zephyr RTOS ?

The MBR beside having the address of where to go bootloader / application also has an interrupt vector table, it receives all the interrupt from the hardware in the system and direct it to the softdevice (ble) or relevant drivers, if i understand it correct, so if there is a softdevice and an NCS on the flash,

1. how does it know not to use the softdevice ?

2. can it be configured to ignor the softdevice ?

3. can it be configured to disable current bootloader or jump directly to app/ zephyr address and not go through the bootloader, until a new one is copied ?

4. can all this be done from a running application ? 

5. Also more general, how zephyr RTOS perform/work if the MBR has the vector table ? 

6. How can the MCUboot be overwriting the MBR ? in advanced stages ?

Edvin said:

please specify what you mean by "only update application and not rewriting softdevice" in your current bootloader.

Our current bootloader is self made, it is not the bootloader from the sdk dfu example with the micro ecc etc. it has no bootloader setting .hex file, it only copies a, previously to restart, received application from an external flash to local flash at an address after softdevice end address and it does use the "sd_init()"

i put the main() of the our bootloader here 

int main(void) {
    //this is to apply errata fixes that are missing in system_nrf52.c
    apply_additional_errata_fixes();

    APP_ERROR_CHECK(NRF_LOG_INIT());

    NRF_LOG_PRINTF("Augury bootloader %s, welcome!\n", BOOTLOADER_VERSION);
    nrf_gpio_cfg_output(AUGU_GPIO_GREEN_LED_PIN);
    nrf_gpio_cfg_output(AUGU_GPIO_RED_LED_PIN);
    nrf_gpio_cfg_output(AUGU_GPIO_BLUE_LED_PIN);
    
    augu_dfu_settings_t settings;
    augu_dfu_settings_get(&settings);
    bool app_reset = (NRF_POWER->GPREGRET == BOOTLOADER_DFU_START);

    augu_init_flash_spi();

    if (NRF_POWER->GPREGRET == BOOTLOADER_DFU_START || (settings.valid && settings.dirty)) {
        NRF_LOG_PRINTF("entering DFU!\n");
        int source_bank = 1;

        if (!settings.dirty) {
            settings.dirty = true;
            augu_dfu_settings_save(&settings);
        }
        
        nrf_gpio_pin_set(AUGU_GPIO_BLUE_LED_PIN);
        copy_from_data_flash_to_code_flash(source_bank, CODE_REGION_1_START, CODE_SIZE);
        nrf_gpio_pin_clear(AUGU_GPIO_BLUE_LED_PIN);

        settings.dirty = false;
        augu_dfu_settings_save(&settings);
        
        NRF_POWER->GPREGRET = 0;
        NVIC_SystemReset();
    }

    if (!settings.valid) {
        NRF_LOG_PRINTF("bootloader settings not initialized.\n");
        NRF_POWER->GPREGRET = 0;
    }
        
    if (NRF_POWER->GPREGRET == BOOT_FAILURE) {
        NRF_LOG_PRINTF("boot failure, switching code banks...\n");
        // int bank = 1 - settings.code_from_bank;

        settings.dirty = true;
        augu_dfu_settings_save(&settings);
        nrf_gpio_pin_set(AUGU_GPIO_RED_LED_PIN);
        copy_from_data_flash_to_code_flash(0, CODE_REGION_1_START, CODE_SIZE);
        nrf_gpio_pin_clear(AUGU_GPIO_RED_LED_PIN);
        
        // settings.code_from_bank = bank;
        settings.dirty = false;
        augu_dfu_settings_save(&settings);

        NRF_POWER->GPREGRET = 0;
        NVIC_SystemReset();
    }

    NRF_POWER->GPREGRET = BOOT_FAILURE;

    augu_uninit_flash_spi();

    sd_init(!app_reset);
    
    NRF_LOG_PRINTF("starting app...\n");

    uint32_t err_code = nrf_drv_timer_init(&BOOTLOADER_TIMER, NULL, bootloader_timeout_handler);
    APP_ERROR_CHECK(err_code);

    uint32_t time_ticks = nrf_drv_timer_ms_to_ticks(&BOOTLOADER_TIMER, 5000);
    
     nrf_drv_timer_extended_compare(
         &BOOTLOADER_TIMER, NRF_TIMER_CC_CHANNEL0, time_ticks, NRF_TIMER_SHORT_COMPARE0_CLEAR_MASK, true);

    nrf_drv_timer_enable(&BOOTLOADER_TIMER);

    bootloader_app_start(CODE_REGION_1_START);
    NVIC_SystemReset();
}

The softdevice and bootloader were flashed first in production. The devices, as mentioned, are now out there in the world

Edvin said:

f you can't delete these two, then you will have very little space for your NCS app in the future.

cause our current bootloader is very limited, What we think is can we dfu an application that say to the MBR “if the system resets go straight to the application again and don’t go to the bootloader", then the application will overwrite the current bootloader with a new bootloader (if it is possible) 

And if that is possible then it will be step 1, next we will have to understand what the next bootloader should be able to do.

At this stage we are still using the softdevice for the ble connection, cause we don’t yet have another tool for the ble and the midel-way application is still sdk based and not zephyr, and in general i don't think i can erase the softdevice until i put the NCS on and only then to my understanding i can remove the softdevice to have more space

hope you can help me with getting a better understanding of the MBR and how can it work with zephyr and also figure out how to do this sdk - zephyr transfer via ble :)

hope to read from you soon

best regards

Ziv

[Edvin]

Hello Ziv,

If you are not able to update the bootloader or the softdevice, you will succeed with this project. I am sorry. I know it is not what you want to hear, but there is not enough memory on the nRF52832 to do this then.

To answer your question, let us assume that you had the normal SDK bootloader. It would be possible to generate an init packet telling the bootloader that this update was a softdevice and bootloader update. In that case, it would receive this packet, validate it, then erase the old bootloader and softdevice, and then program the new "softdevice+bootloader" to the right placement. However, in this case the "softdevice and bootloader" would contain the softdevice controller(NCS), bootloader, application and the immutable bootloader. The MBR would remain, and the immutable bootloader would start after the MBR. The MBR would forward all interrupts to the immutable bootloader, as if it was located on 0x00000000. 

I don't know if you would be able to update your bootloader from your bootloader. You would probably need to update the bootloader itself first. You can see if you manage to do so using the method you describe above. Exactly how this is done in the MBR I am not sure. Look at the implementation of the bootloader from the SDK if you can figure out how it is done. As far as I can see, you don't protect the flash area of the bootloader from the bootloader itself, is that correct?

ziv123 said:

1. how does it know not to use the softdevice ?

2. can it be configured to ignor the softdevice ?

3. can it be configured to disable current bootloader or jump directly to app/ zephyr address and not go through the bootloader, until a new one is copied ?

4. can all this be done from a running application ? 

I don't know the details on this, but the MBR is write protected, so you can't change it from the application (ever). Only with a programmer.

ziv123 said:

5. Also more general, how zephyr RTOS perform/work if the MBR has the vector table ? 

The workaround would require some piece of code in place of the old bootloader redirecting the vector table. That is also a drawback doing this.

ziv123 said:

6. How can the MCUboot be overwriting the MBR ? in advanced stages ?

It can't. The MBR will remain.

Again. See if you can replace your bootloader with the SDK bootloader via DFU. I really think you should look into this before you proceed because it is necessary. And Again, I have not tested this myself. And I can in no way guarantee that this will work at all with the amount of flash on the nRF52832.

I wanted to not post this yet, but perhaps it will answer some of your questions. This is an answer from a colleague of mine who attempted something similar on the nRF52840 (which has more memory). It is also tested on an older version of NCS, so it may not work out of the box anymore.

-------------------

Are you looking to support OTA DFU to NCS on existing devices in the field? The MCUboot is placed at address 0x0 by default, same place as where the MBR used to reside. It's however possible to relocate it with the partition manager, as you mentioned. I actually did this as an experiment a little while ago to see if I could do DFU from nRF5 SDK 17 to NCS v.1.4.0. Attached below are the projects I made if you want to review them. 

Memory layout (defined by pm_static.yml)

$ west build -b nrf52840dk_nrf52840 // West command to build peripheral_uart sampe (MCUboot is automatically built as a child image, see Multi-image builds

$ west build -t rom_report // Ninja build target to display flash layout as shown below

FOTA from nRF5 SDK to NCS test

6607.dfu_nRF5_NCS.zip

Some quick notes about my test:

1. I'm updating MCUboot and the zephyr application image in one go by calling it a Softdevice+bootloader (SoftDevice and bootloader) update. I did this to be able to overwrite the softdevice with the zephyr app. However, it did require some minor modifications to the existing bootloader to make it accept zephyr application as a Softdevice image.I can provide a 'diff' to show the changes

2. The flash protection mechanism in MCUboot assumes that the mcuboot_primary slot is always placed after the mcuboot partition. I disabled flash protection to get around this limitation.

-------------------

Best regards,

Edvin