Fixing the "known issue" Partitioning limitation with MCUboot swap move

[variant]

The subject issue is documented here as "NCSDK-20567: Partitioning limitation with MCUboot swap move":


This issue has been open for while. 

These two DevZone tickets refer to this issue and mention the impact is that you can use only around 95% of the flash space:
1. https://devzone.nordicsemi.com/f/nordic-q-a/112343/mcuboot-image-in-the-primary-slot-is-not-valid-unable-to-find-bootable-image

2. https://devzone.nordicsemi.com/f/nordic-q-a/107599/mcuboot-sign-image-possible-wrong-slot-size/475558

Here are my questions:

1. In the Workaround description, there is a formula to calculate the approximate size limitation. What CONFIG item is referred to for mcuboot_primary_size?  I am not able to figure out how to use that equation to result in a value of approximately 95% of the "Region Size" for the application image indicated in the build log.

2. Why is "additional" margin suggested?  Can that margin be quantified?

3. Are there still plans to fix this?  A lot of internal flash space seems to be wasted due to this issue.

4. When the fix is available, do you think the fix can be patched by us to an earlier NCS version, such as NCS 2.6?

[Vidar Berg]

Hi,

The size of the primary slot is defined by the PM_MCUBOOT_PRIMARY_SIZE symbol in pm_config.h which is generated by the partition manager during the build.

Fixing this issue is still in our backlog. I’ve requested that we prioritize addressing it so you can receive a build time error if the image will exceed the maximum size for the secondary slot. As you may know, the reason the FW image can't fill the whole partition is that we need to leave room for the image trailer at the end. 

1. In the Workaround description, there is a formula to calculate the approximate size limitation. What CONFIG item is referred to for mcuboot_primary_size?  I am not able to figure out how to use that equation to result in a value of approximately 95% of the "Region Size" for the application image indicated in the build log.

The percentage will vary depending on the size of the partition. Larger partitions can have utilization above 95%

2. Why is "additional" margin suggested?  Can that margin be quantified?

This was to account for any changes in the image trailer size. For example, the image trailer becomes larger if encryption is enabled. The best way to verify that there is enough room for the image trailer with your configuration is to test the DFU of the image locally before distributing the update. If the image is too large, the bootloader will fail to copy the update to the primary slot. It is also important that you are using the same static partition layout across FW versions (Configuring static partitions)

4. When the fix is available, do you think the fix can be patched by us to an earlier NCS version, such as NCS 2.6?

The planned fix is to calculate the actual space available for the firmware image at build time and provide the user with an error if the image is too large for DFU.

Best regards,

Vidar

[variant]

Thanks Vidar for the very detailed and clear reply.
I will have a follow-up question on your answer shortly.  I haven't had a chance to research something yet, but I wanted to get the thank you sent now.

[variant]

Hi Vidar,

Ok, so for the equation in NCSDK-20567 to calculate max app size:
mcuboot_primary_size - 80 - (mcuboot_primary_size/ 4096) * 12 -4096

you said the mcuboot_primary_size is PM_MCUBOOT_PRIMARY_SIZE.

1. For my case, 
PM_MCUBOOT_PRIMARY_SIZE is 0xee000, or 974848.

The max app size equation in NCSDK-20567 results in 967816

Our build log "Memory region" for FLASH lists "Region Size" as 974336.

If hypothetically my actual app size were 967816 as being the "Used Size" in the build log, the build log would report "%age Used" as 967816 / 974336 = 99.33%.   I know that 99.33% can't be correct because we hit the max size issue at 96% as reported by the build log.  What am I doing wrong in this example?

I don't think the NCSDK-20567's "Some additional margin is suggested" would account for the discrepancy.

2. According to image trailer, 


If the image trailer size depends on BOOT_MAX_IMG_SECTORS, then why doesn't the equation in NCSDK-20567 include that value in the equation?  More specifically, does BOOT_MAX_IMG_SECTORS have any impact on the max app size that can be FW updated?

Thanks in advance!

[Vidar Berg]

Hi,

Yes, this does not appear to be correct, unfortunately. I'm awaiting feedback from the developers on this. The value of BOOT_MAX_IMG_SECTORS * 4096 must be bigger than or equal to the size of the mcuboot slot, and since the BOOT_MAX_IMG_SECTORS value affects the image trailer size, it seems like it should have been included in the calculation as you said.

[variant]

Hi Vidar, is there any update on your last reply?  Can you at least confirm for sure that the value for BOOT_MAX_IMG_SECTORS does indeed affect the max size of the updateable image.

[Dominik-NS]

Hi,

The BOOT_MAX_IMG_SECTORS is actually mentioned in NCSDK-20567, as CONFIG_BOOT_MAX_IMG_SECTORS, and how it weights against (image_slot_size_in_bytes - erase_page_size_in_bytes) / erase_page_size_in_bytes. Unfortunately I did calculations from the point of hardware to software impact and so there was no clear indication how the BOOT_MAX_IMG_SECTORS  impacts calculations, as a software configuration item.

The BOOT_MAX_IMG_SECTORS  does indeed impact size of footer. It is mainly used, in MCUboot, to calculate log size and to allocate array of sectors (erase units on device) (as explained in NCSDK-20567)

Internal details are below, if interested.

The first usage is done here boot_status_internal_off (note that this is unfortunate name for a partial offset, not full offset from the end of slot):
https://github.com/nrfconnect/sdk-mcuboot/blob/b82206c15fff357c151c24bf97c99c4348d14a46/boot/bootutil/src/swap_move.c#L221 (for MOVE algorithm, SCRATCH has own function for that) - basically the function is called given write-block-size of a device (elem_sz) and calculates log for 3 states (BOOT_STATUS_MOVE_STATE_COUNT) per page/sector moved (BOOT_MAX_IMG_SECTORS).
Above can also be seen in swap_read_status_bytes https://github.com/nrfconnect/sdk-mcuboot/blob/b82206c15fff357c151c24bf97c99c4348d14a46/boot/bootutil/src/swap_move.c#L204 when calculating maximum expected log entries, without footer "header" (that is why it is "partial").

The other usage is in allocating table for sector information; the array is used to store information on offset and size of each sector that covers slot - this happens, because MCUboot has been designed with devices with non-uniform sector layout, like stm32, where each sector can have different size, https://github.com/nrfconnect/sdk-mcuboot/blob/b82206c15fff357c151c24bf97c99c4348d14a46/boot/bootutil/src/loader.c#L96.
The BOOT_MAX_IMG_SECTORS needs to be provided by user, because it is not always possible to get that info from DTS, for example spi devices do not have such info and so far non-uniform devices have no such definition.

Regards,
Dominik

[Dominik-NS]

One more update.

As it may be tempting to set the BOOT_MAX_IMG_SECTORS to (image_slot_size_in_bytes - erase_page_size_in_bytes) / erase_page_size_in_bytes, it would not be correct.
The BOOT_MAX_IMG_SECTORS needs to be set to highest of image_slot_size_in_bytes / erase_page_size_in_bytes of two values, assuming two slots may have different sizes, as it has to hold information on layout for both slots, as we will erase any page in the slot range.

So in reality, after the BOOT_MAX_IMG_SECTORS  has been established, the trailer itself, can be calculated as
(((BOOT_MAX_IMG_SECTORS * write_block_size) * 3 + 80.

But aside from the above value, the image will also loose one additional page for the "move" page.