Why is MCUboot scratch algorithm disabled on Thingy:91?

[kFYatek]

The original firmware for Thingy:91 in version 2019-11-29_d3130d77 and earlier, included MCUboot configured to use the scratch partition for swapping the firmware images.

Since version 2020-04-29_bc7ade8b, the default MCUboot version no longer does this, instead using the generic "move" algorithm - even though a 120KiB scratch partition is still reserved in the static partition layout.

Is it by design, or is this a bug? When distributing e.g. open source projects intended to run on the Thingy:91, it creates a problem - the published application shouldn't require an external debug probe to work, and if FOTA functionality is desired, then one would need to make sure that the application doesn't fill up the entire partition, so that the "move" algorithm can work. And in either case, without modifying the partition layout (which I believe is discouraged for the Thingy), the 120 KiB of the flash memory just go to waste and are not used for either MCUboot, the app image, or storage space.

Of course I can build my own version of MCUboot that uses the scratch algorithm, but that won't be usable by people who don't have access to an external debug probe.

Is there something that I'm missing? Could you comment on this issue?

[Sigurd Hellesvik]

Hi,

Is it by design, or is this a bug? When distributing e.g. open source projects intended to run on the Thingy:91, it creates a problem - the published application shouldn't require an external debug probe to work, and if FOTA functionality is desired, then one would need to make sure that the application doesn't fill up the entire partition, so that the "move" algorithm can work. And in either case, without modifying the partition layout (which I believe is discouraged for the Thingy), the 120 KiB of the flash memory just go to waste and are not used for either MCUboot, the app image, or storage space.

The "move" algorith also needs a partition to do the move, but it is not included in the partitioning.
This is a bug, and I believe https://github.com/nrfconnect/sdk-mcuboot/pull/185 fixes it.

So your observation is correct, but it does not relate to the change of algorithm, but rather only to the "move" algorithm.

Did I understand your report correctly?

Regards,
Sigurd Hellesvik

[kFYatek]

Hi Sigurd,

Thanks for looking into this.

The PR you mentioned indeed seems to address this issue, at least partially - I don't fully understand it, but from the looks of it, it seems to force reserving an area within the primary partition to make sure that the firmware update procedure works, which is definitely a welcome change.

I wasn't able to test the effects myself, because after attempting to apply it, I got the following error when attempting to configure the project:

-- Found partition manager static configuration: /home/kfyatek/nrf-zephyr/ncs/nrf/boards/arm/thingy91_nrf9160/thingy91_pm_static.yml
Partition 'mcuboot' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_pad' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_primary' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_primary_app' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_secondary' is not included in the dynamic resolving since it is statically defined.
Partition 'settings_storage' is not included in the dynamic resolving since it is statically defined.
Partition 'nonsecure_storage' is not included in the dynamic resolving since it is statically defined.
Partition 'tfm_secure' is not included in the dynamic resolving since it is statically defined.
Partition 'tfm_nonsecure' is not included in the dynamic resolving since it is statically defined.
Partition 'tfm' is not included in the dynamic resolving since it is statically defined.
Dropping partition 'nrf_modem_lib_trace' since its size is 0.
Traceback (most recent call last):
  File "/home/kfyatek/nrf-zephyr/ncs/nrf/scripts/partition_manager.py", line 1977, in <module>
    main()
  File "/home/kfyatek/nrf-zephyr/ncs/nrf/scripts/partition_manager.py", line 1027, in main
    solution.update(solve_region(pm_config, region, region_config,
  File "/home/kfyatek/nrf-zephyr/ncs/nrf/scripts/partition_manager.py", line 970, in solve_region
    get_region_config(partitions, region_config, static_partitions, system_reqs=pm_config)
  File "/home/kfyatek/nrf-zephyr/ncs/nrf/scripts/partition_manager.py", line 758, in get_region_config
    solve_complex_region(pm_config, start, size, placement_strategy, region_name, device, static_conf, dp, system_reqs)
  File "/home/kfyatek/nrf-zephyr/ncs/nrf/scripts/partition_manager.py", line 866, in solve_complex_region
    set_addresses_and_align(pm_config, sub_partitions, solution, free_size, dp, start=start, system_reqs=system_reqs)
  File "/home/kfyatek/nrf-zephyr/ncs/nrf/scripts/partition_manager.py", line 417, in set_addresses_and_align
    reqs[dp]['size'] = dynamic_partitions_size(reqs, size, dp)
  File "/home/kfyatek/nrf-zephyr/ncs/nrf/scripts/partition_manager.py", line 388, in dynamic_partitions_size
    size = total_size - sum([req['size'] for name, req in reqs.items() if 'size' in req.keys() and name != dp])
TypeError: unsupported operand type(s) for +: 'int' and 'str'
CMake Error at /home/kfyatek/nrf-zephyr/ncs/nrf/cmake/partition_manager.cmake:304 (message):
  Partition Manager failed, aborting.  Command:
  /usr/bin/python3.10;/home/kfyatek/nrf-zephyr/ncs/nrf/scripts/partition_manager.py;--input-files;/home/kfyatek/nrf-zephyr/ncs/build/mcuboot/zephyr/include/generated/pm.yml;/home/kfyatek/nrf-zephyr/ncs/build/zephyr/include/generated/pm.yml;/home/kfyatek/nrf-zephyr/ncs/build/modules/nrf/subsys/partition_manager/pm.yml.settings;/home/kfyatek/nrf-zephyr/ncs/build/modules/nrf/subsys/partition_manager/pm.yml.libmodem;/home/kfyatek/nrf-zephyr/ncs/build/modules/nrf/subsys/partition_manager/pm.yml.tfm;/home/kfyatek/nrf-zephyr/ncs/build/modules/nrf/subsys/partition_manager/pm.yml.trustzone;--regions;sram_primary;otp;flash_primary;--output-partitions;/home/kfyatek/nrf-zephyr/ncs/build/partitions.yml;--output-regions;/home/kfyatek/nrf-zephyr/ncs/build/regions.yml;--static-config;/home/kfyatek/nrf-zephyr/ncs/nrf/boards/arm/thingy91_nrf9160/thingy91_pm_static.yml;--sram_primary-size;0x40000;--sram_primary-base-address;0x20000000;--sram_primary-placement-strategy;complex;--sram_primary-dynamic-partition;sram_primary;--otp-size;756;--otp-base-address;0xff8108;--otp-placement-strategy;start_to_end;--flash_primary-size;0x100000;--flash_primary-base-address;0x0;--flash_primary-placement-strategy;complex;--flash_primary-device;flash_controller;--flash_primary-default-driver-kconfig;CONFIG_SOC_FLASH_NRF
Call Stack (most recent call first):
  /home/kfyatek/nrf-zephyr/ncs/zephyr/cmake/modules/kernel.cmake:246 (include)
  /home/kfyatek/nrf-zephyr/ncs/zephyr/cmake/modules/zephyr_default.cmake:117 (include)
  /home/kfyatek/nrf-zephyr/ncs/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:66 (include)
  /home/kfyatek/nrf-zephyr/ncs/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:92 (include_boilerplate)
  CMakeLists.txt:7 (find_package)

..so I cannot be sure about what's going on - but the PR also doesn't seem to address the fact that the mcuboot_scratch partition is there defined in nrf/boards/arm/thingy91_nrf9160/thingy91_pm_static.yml, yet it is not used for anything - unless I'm missing something. It seems to me that it would make sense to either use the "scratch" algorithm, after all, or reallocate this space for other purposes.

[Sigurd Hellesvik]

kFYatek said:

..so I cannot be sure about what's going on - but the PR also doesn't seem to address the fact that the mcuboot_scratch partition is there defined in nrf/boards/arm/thingy91_nrf9160/thingy91_pm_static.yml, yet it is not used for anything - unless I'm missing something. It seems to me that it would make sense to either use the "scratch" algorithm, after all, or reallocate this space for other purposes.

Ah, I missed that part of the question.

It is not clear in our bootloader docs, so I have both given internal feedback to add this there, and created a PR to give users a warning.
But from our partition manager docs:

"However, if you have a deployed product that consists of multiple images, where only a subset of the included images can be upgraded through a firmware update mechanism, the upgradable images must be statically configured."

To translate this, it means that if you use only MCUboot, you can not change partitioning between updates, as this may break DFU for future updates.
This is because when the device is first flashed, MCUboot knows only the old partitioning and assumes that this is where all future partitions will be swapped from/to.
So if the application then tries to write updates to other places in memory, things may break.

If I were to guess, this is the reason for the static partitioning: So that the updates will be valid for both old and new versions of the thingy.

If you want to update the partitioning, you must program the Thingy:91 with an external debugger (or DK).

Regards,
Sigurd Hellesvik

[kFYatek]

Yes, this makes perfect sense.

However, what I still don't understand is why did you change the algorithm from "scratch" to "move" between the 2019 and 2020 revisions of the original firmware? Since the scratch partition has been already reserved, it would make sense to keep using it. Was there some problem with the "scratch" algorithm?

[Sigurd Hellesvik]

The move algorithm has less flash wear than the skratch algorithm.

See https://www.youtube.com/watch?v=KFgK3TRDNR4

Regards,
Sigurd Hellesvik

[kFYatek]

OK then. I can't say that I'm particularly happy that the 120 KiB originally reserved for a scratch partition are now basically unusable without repartitioning the device, but if that's intentional, then I guess it is what it is.

Just to confirm: commercially available Thingy:91 units are generally expected to be flashed with the latest firmware available at the time of manufacturing, is that right? So I cannot assume that MCUboot on any unit straight from the factory will use either the "move" or "scratch" algorithm?

[Sigurd Hellesvik]

kFYatek said:

commercially available Thingy:91 units are generally expected to be flashed with the latest firmware available at the time of manufacturing, is that right?

Generally, but there may be some lag here.

kFYatek said:

So I cannot assume that MCUboot on any unit straight from the factory will use either the "move" or "scratch" algorithm?

Correct.
I guess this is also a bonus when the partitioning is the same, as you can treat both the same.

kFYatek said:

I can't say that I'm particularly happy that the 120 KiB originally reserved for a scratch partition are now basically unusable without repartitioning the device, but if that's intentional, then I guess it is what it is.

I understand that.

If it is to some consolidation, you can use our nRF9160DK or nRF5340DK to program the Thingy91, which lets you repartition.

Regards,
Sigurd Hellesvik