Fixing the "known issue" Partitioning limitation with MCUboot swap move

[variant]

The subject issue is documented here as "NCSDK-20567: Partitioning limitation with MCUboot swap move":


This issue has been open for while. 

These two DevZone tickets refer to this issue and mention the impact is that you can use only around 95% of the flash space:
1. https://devzone.nordicsemi.com/f/nordic-q-a/112343/mcuboot-image-in-the-primary-slot-is-not-valid-unable-to-find-bootable-image

2. https://devzone.nordicsemi.com/f/nordic-q-a/107599/mcuboot-sign-image-possible-wrong-slot-size/475558

Here are my questions:

1. In the Workaround description, there is a formula to calculate the approximate size limitation. What CONFIG item is referred to for mcuboot_primary_size?  I am not able to figure out how to use that equation to result in a value of approximately 95% of the "Region Size" for the application image indicated in the build log.

2. Why is "additional" margin suggested?  Can that margin be quantified?

3. Are there still plans to fix this?  A lot of internal flash space seems to be wasted due to this issue.

4. When the fix is available, do you think the fix can be patched by us to an earlier NCS version, such as NCS 2.6?

[Vidar Berg]

Hi,

The size of the primary slot is defined by the PM_MCUBOOT_PRIMARY_SIZE symbol in pm_config.h which is generated by the partition manager during the build.

Fixing this issue is still in our backlog. I’ve requested that we prioritize addressing it so you can receive a build time error if the image will exceed the maximum size for the secondary slot. As you may know, the reason the FW image can't fill the whole partition is that we need to leave room for the image trailer at the end. 

1. In the Workaround description, there is a formula to calculate the approximate size limitation. What CONFIG item is referred to for mcuboot_primary_size?  I am not able to figure out how to use that equation to result in a value of approximately 95% of the "Region Size" for the application image indicated in the build log.

The percentage will vary depending on the size of the partition. Larger partitions can have utilization above 95%

2. Why is "additional" margin suggested?  Can that margin be quantified?

This was to account for any changes in the image trailer size. For example, the image trailer becomes larger if encryption is enabled. The best way to verify that there is enough room for the image trailer with your configuration is to test the DFU of the image locally before distributing the update. If the image is too large, the bootloader will fail to copy the update to the primary slot. It is also important that you are using the same static partition layout across FW versions (Configuring static partitions)

4. When the fix is available, do you think the fix can be patched by us to an earlier NCS version, such as NCS 2.6?

The planned fix is to calculate the actual space available for the firmware image at build time and provide the user with an error if the image is too large for DFU.

Best regards,

Vidar

[variant]

Thanks Vidar for the very detailed and clear reply.
I will have a follow-up question on your answer shortly.  I haven't had a chance to research something yet, but I wanted to get the thank you sent now.

[Dominik-NS]

One more update.

As it may be tempting to set the BOOT_MAX_IMG_SECTORS to (image_slot_size_in_bytes - erase_page_size_in_bytes) / erase_page_size_in_bytes, it would not be correct.
The BOOT_MAX_IMG_SECTORS needs to be set to highest of image_slot_size_in_bytes / erase_page_size_in_bytes of two values, assuming two slots may have different sizes, as it has to hold information on layout for both slots, as we will erase any page in the slot range.

So in reality, after the BOOT_MAX_IMG_SECTORS  has been established, the trailer itself, can be calculated as
(((BOOT_MAX_IMG_SECTORS * write_block_size) * 3 + 80.

But aside from the above value, the image will also loose one additional page for the "move" page.

[variant]

Is there going to be any reply from Nordic?

[Vidar Berg]

Sorry, I assumed the questions got answered by Dominik-NS  comments above. He is a Nordicer who is working on MCUBoot. Could you upload the configuration file (build/mcuboot/zephyr/.config) for your bootloader so I can have a look at the configuration?

[variant]

Thanks for the reply Vidar Berg . And I apologize Dominik-NS that I didn't realize you were replying as a Nordic rep. Your reply was very detailed, but I wasn't sure it was sanctioned by Nordic. 
Attached is the build/mcuboot/zephyr/.config file that Vidar suggested I post.

Thanks again.

.config_for_mcuboot.txt

[Dominik-NS]

I think we are mistaking binary with mcuboot image, these are different things:

the top one is binary, that is the thing build logs size form; the bottom one is image, that is uploaded to the device.
Once your image is signed there is additional information added to it, and build log does not show that info.
For experiment I have appended my binary with enough data to fill, after signing, entire slot - 2 * 4096 bytes, and that worked fine. But I have been using equal slot sizes, that is why I have to subtract on slot for move area and one for trailer rounded up to one slot.
In primary slot trailer may not be written in the same page as the swapped in image is.

In secondary slot trailer may be written to the same page as uploaded, because there is no "move" page between trailer and image.
The move page needs to be erased independently, so it can not be either part of image nor trailer.
But that image will be swapped to the primary slot, and at that point that slot has to be able to

It seems that you have CONFIG_BOOT_MAX_IMG_SECTORS=256, the nrf5340 has a 4096B page and 4B write block, 974848 primary slot.

The 974848 takes 238 slots, so 238 * 3 * 4 + 80 = 2936 ~= 4096
So primary slot can take max 974848  - 2 * 4096 = 966656

Please check your PM_MCUBOOT_SECONDARY_SIZE, because it seems that this is significantly shorter. The trailer equation will only work with the secondary slot, because here the trailer can touch the last page of binary, something it can not do in primary slot.

[smith_tristan]

Hi Dominik-NS  - I work with variant , trying to catch up on this thread...

Q1) What do you mean when you say: "Please check your PM_MCUBOOT_SECONDARY_SIZE, because it seems that this is significantly shorter."? Shorter than what?

In partitions.yml, I see mcuboot_secondary: size: 0xee000 (974848 bytes).

Just to make sure I'm understanding correctly:

1) Primary Slot Size: has to contain the image + trailer + 1 page for move page

2) BOOT_MAX_IMG_SECTORS must be larger than: ( Largest Image Slot Size ) / Erase Page Size

3) Erase Page Size = 4kBytes for the NRF5340

4) Trailer Size (bytes) = ( BOOT_MAX_IMG_SECTORS * Block Size ) * 3 + 80

So for the nRF5340:

5) We start with a primary image slot size = 974848 bytes (0xEE000)

6) BOOT_MAX_IMG_SECTORS >= 238 = 974848 / 4096

7) Given that the trailer is 4096 bytes, this gives us a useable space of 966656 bytes. 

Q2) However, because of the MCUBOOT pad, should we actually start with 974336 bytes as the primary image slot size? That is address space of 0x10200 to 0xFE000.

Q3) Also, why is there not a general recommended BOOT_MAX_IMG_SECTORS configuration for any nRF5340 project? Would 256 always be the recommended number of sectors?

Thank you!

Tristan

[Dominik-NS]

1) Primary Slot Size: has to contain the image + trailer + 1 page for move page
Yes, trailer is rounded to N pages, where page is 4096 bytes (min for hardware)

2) BOOT_MAX_IMG_SECTORS must be larger than: ( Largest Image Slot Size ) / Erase Page Size
Yes. Because BOOT_MAX_IMG_SECTORS determines size of swap log size. You are swapping BOOT_MAX_IMG_SECTORS  at most, which is slot size/page size

3) Erase Page Size = 4kBytes for the NRF5340
Yes,; when external mem is used, the bigger size is used, but it can not be less than physical limitation of any of memories (4k is min for nrf53/nrf52)

4) Trailer Size (bytes) = ( BOOT_MAX_IMG_SECTORS * Block Size ) * 3 + 80

Block size == Write block size (4 bytes); yes, but rounded up to pages. MCUboot has to copy/erase by pages.

5) We start with a primary image slot size = 974848 bytes (0xEE000)

OK

6) BOOT_MAX_IMG_SECTORS >= 238 = 974848 / 4096

ok

7) Given that the trailer is 4096 bytes, this gives us a useable space of 966656 bytes.

should

Q2) However, because of the MCUBOOT pad, should we actually start with 974336 bytes as the primary image slot size? That is address space of 0x10200 to 0xFE000.

No.The pad is just shifting application image by the size required for image header, but header is part of the image that is being moved around. This thing is for compiler/linker to figure out where code actually should start. After you build application the 0x200 bytes are added in front of by imgtool when image is signed. There is another chunk of data added at the end in form of TLV that store info like signature. The real size that is swapped between slots is what you see as app_update.bin (or zephyr.signed.bin), not the thing that build reports in the end.

Q3) Also, why is there not a general recommended BOOT_MAX_IMG_SECTORS configuration for any nRF5340 project? Would 256 always be the recommended number of sectors?

Generally 256 is ok if size of slot is < 1MiB. As you can see this has to round up to a page anyway, and 256 logs take up to 3072 so plenty more space to fit the rest of trailer. The space gets shorter when using encryption (because keys are also dropped there), in which case you may wan to cut the the BOOT_MAX_IMG_SECTORS to actual size, to not take another page.

The best way to test your setup is build your app, check size difference between signed and unsigned. Then add, to the end of binary, enough bytes to fill the slot, save the header (0x200), the difference checked above and - 2* 4096 in pages. Now when you sign what you have you should have signed binary of size of slot - 2* 4096 bytes; that thing should correctly upload to device and swap into primary slot.