Partition Manager not using flash efficiently

[Cla]

Dear Nordic Team

When I run the command `west build -t partition_manager_report` from inside our build folder I get :

  flash_primary (0x100000 - 1024kB): 
+--------------------------------------------------+
| 0x0: mcuboot (0xc000 - 48kB)                     |
| 0xc000: EMPTY_0 (0x4000 - 16kB)                  |
+---0x10000: mcuboot_primary (0x70000 - 448kB)-----+
+---0x10000: tfm_secure (0xc200 - 48kB)------------+
| 0x10000: mcuboot_pad (0x200 - 512B)              |
+---0x10200: mcuboot_primary_app (0x6fe00 - 447kB)-+
| 0x10200: tfm (0xc000 - 48kB)                     |
+---0x1c200: tfm_nonsecure (0x63e00 - 399kB)-------+
| 0x1c200: app (0x63e00 - 399kB)                   |
+--------------------------------------------------+
| 0x80000: mcuboot_secondary (0x70000 - 448kB)     |
| 0xf0000: EMPTY_1 (0x8000 - 32kB)                 |
+---0xf8000: nonsecure_storage (0x6000 - 24kB)-----+
| 0xf8000: nvs_storage (0x6000 - 24kB)             |
+---0xfe000: tfm_storage (0x2000 - 8kB)------------+
| 0xfe000: tfm_its (0x2000 - 8kB)                  |
+--------------------------------------------------+

This seems like an inefficient use of flash and flash is quite valuable to us. We have to make use of every kB we can.
It seems like a waste in the following respects:

1. Why is there an EMPTY_0 partition? Potential gain 16kB
2. Why is there an EMPTY_1 partition? Potential gain 32kB
3. Why does mcuboot take 48kB, it only needs 33996B in our build? Potential gain at least ~12kB

My main question is, how can the flash layout be optimized (by Hand)? If it cannot, how can partition manager be disable, so that the partitions from the device-tree are used instead?

My assumption is, that to optimize the flash layout one has to move to a static partition setup as described here: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/scripts/partition_manager/partition_manager.html#static-configuration
So I copied the `build/paritions.yml` file to the `pm_static.yml` and started hand-optimizing. This is quite difficult, but I managed to achieve the following temporary result:

  flash_primary (0x100000 - 1024kB): 
+-------------------------------------------------+
| 0x0: mcuboot (0x9000 - 36kB)                    |
+---0x9000: mcuboot_primary (0x77000 - 476kB)-----+
+---0x9000: tfm_secure (0x13200 - 76kB)-----------+
| 0x9000: mcuboot_pad (0x200 - 512B)              |
+---0x9200: mcuboot_primary_app (0x76e00 - 475kB)-+
| 0x9200: tfm (0x13200 - 76kB)                    |
+---0x1c200: tfm_nonsecure (0x63e00 - 399kB)------+
| 0x1c400: app (0x63c00 - 399kB)                  |
+-------------------------------------------------+
| 0x80000: EMPTY_1 (0x1000 - 4kB)                 |
| 0x81000: mcuboot_secondary (0x77000 - 476kB)    |
+---0xf8000: nonsecure_storage (0x6000 - 24kB)----+
| 0xf8000: nvs_storage (0x6000 - 24kB)            |
+---0xfe000: tfm_storage (0x2000 - 8kB)-----------+
| 0xfe000: tfm_its (0x2000 - 8kB)                 |
+-------------------------------------------------+

I managed to reduce the empty partitions and move the free space into the mcuboot_primary and secondary partitions respectively. To achieve this, I had to increase the tfm-partitions, instead the goal is to increase the app partitions. Trying to reduce the tfm partition size again to the original 48kB and moving the free space into the app partition I run into a linker issue.

That leaves me with the following secondary questions:
1. Is this approach to optimize the flash-layout correct?
2. Is there a reason, that the tfm partition is expected to be located at a certain address?
3. What is the mcubootpad partition for?
4. Why do some of the partitions use the keyword "align" in the pm_static.yml? The documentation reads "Ensure the alignment of the start or the end of the partition by specifying a dict with a start or end key respectively, where the value is the number of bytes to align to. If necessary, empty partitions are inserted in front of or behind the partition to ensure that the alignment is correct...." In our case it is "align: -start: 0x8000", does this mean, that the start of those partitions needs to be a multiple of 0x8000 for maximum space efficiency (or the partition manager will insert empty partitions)? What are the alignment requirements?
5. We are using mcuboot for firmware upgrades as described here: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/mcuboot/design.html#swap-without-using-scratch. As described there, the primary partition needs to be bigger by a certain amount than the secondary partition. Can this be baked into the flash layout, such that an offending (too big) image is not only discovered when trying to switch to it after reboot when upgrading, but earlier? Ideally the secondary slot has exactly the size, such that it fits into the primary slot with enough padding for the move algorithm as described in the link.

[Cla]

Update:
On 4. I found some helpful infos here: Interpretation of the partition manager report
Is the CONFIG_NRF_SPU_FLASH_REGION_SIZE = 0x8000 in the nRF9160 ?

On 3. It seems it is for the image header answer found here: Partition manager primary and secondary slot sizes

On 5. It seems others have also observed:
Partition manager primary and secondary slot sizes

 Big primary image prevents FOTA 

Regarding the empty partitions also see:
RE: west build generates large EMPTY partitions

After investigating a bit, I think I have to update my assumptions. Maybe someone could confirm?
- On the nRF9160 the flash sector size is 0x8000=32KiB. It has a total of 1MiB/32KiB=32 sectors.
- The sectors occupied by mcuboot can not hold anything else. Since the build > 32KiB it takes 2 sectors
- The storages (nvs and tfm) currently takes 1 sector, exactly 32kiB size.
- that leaves 32 - 3 = 29 sectors for the mcuboot primary and secondary partition. They need to be equal size (leaving out the caveat about the move algorithm for now). They also need to be aligned to the flash sectors (if so, why?). That means the sector count has to be even 29-1=28. Each partition needs half of the space 28/2=14. Which is what we see in the above report (14*32KiB = 448KiB).

[Sigurd Hellesvik]

Hi,

My main question is, how can the flash layout be optimized (by Hand)? If it cannot, how can partition manager be disable, so that the partitions from the device-tree are used instead?

Yes, static partitioning (pm_static.yml) is the way to optimize partitions "by hand".

If you use child images, Partition Manager is included by default, so I think it will be hard to disable it if you want to use MCUboot.

1. Is this approach to optimize the flash-layout correct?

You have two different ways to configure mcuboot flash usage:

1. CONFIG_PM_PARTITION_SIZE_MCUBOOT and CONFIG_PM_PARTITION_SIZE_MCUBOOT_PAD. Ref  RE: west build generates large EMPTY partitions 
2. pm_static.yml

MCUboot operates with static partitioning.
Meaning that you can not change partitioning in an DFU update to MCUboot.

Because of this, I think it is just as well to use pm_static.yml, as you might want static partitions either way.

2. Is there a reason, that the tfm partition is expected to be located at a certain address?

I do not know of any certain address requirements of tfm. Can you explain a bit further?

If we imagine an application without a bootloader for a moment: The tfm partition is placed at the beginning of flash to make sure it is the first thing that is executed.

Cla said:

On 3. It seems it is for the image header answer found here: Partition manager primary and secondary slot sizes

Yes

Cla said:

On 4. I found some helpful infos here: Interpretation of the partition manager report
Is the CONFIG_NRF_SPU_FLASH_REGION_SIZE = 0x8000 in the nRF9160 ?

The general block size of the flash is 4kB(0x1000), see INFO.CODEPAGESIZE in the FICR.
However, the SPU splits the flash for partitions, and have a size of 32kB(0x8000). See SPU Flash access control.

Use Kcofig(in VS Code) search to verify what this configuration is set to for your build specifically.
Or "west build -t menuconfig" in the Terminal.

Cla said:

On 5. It seems others have also observed:
Partition manager primary and secondary slot sizes

 Big primary image prevents FOTA 

Yes, and there seems to be a pull request for this as well:
https://github.com/nrfconnect/sdk-nrf/pull/6245

Cla said:

After investigating a bit, I think I have to update my assumptions. Maybe someone could confirm?
- On the nRF9160 the flash sector size is 0x8000=32KiB. It has a total of 1MiB/32KiB=32 sectors.
- The sectors occupied by mcuboot can not hold anything else. Since the build > 32KiB it takes 2 sectors
- The storages (nvs and tfm) currently takes 1 sector, exactly 32kiB size.
- that leaves 32 - 3 = 29 sectors for the mcuboot primary and secondary partition. They need to be equal size (leaving out the caveat about the move algorithm for now). They also need to be aligned to the flash sectors (if so, why?). That means the sector count has to be even 29-1=28. Each partition needs half of the space 28/2=14. Which is what we see in the above report (14*32KiB = 448KiB).

Yes, both I and one of my colleagues agree with your assumptions.

I hope I answered your questions.
Let me know if I missed some or if you have more.
Good job with searching and finding relevant DevZone cases yourself by the way.

Regards,
Sigurd Hellesvik

[Cla]

Thanks for the detailed explanation.

Sigurd Hellesvik said:

The general block size of the flash is 4kB(0x1000), see INFO.CODEPAGESIZE in the FICR.
However, the SPU splits the flash for partitions, and have a size of 32kB(0x8000). See SPU Flash access control.

These are unfortunately not accessible to me. I get a "503 Service unavailable".

Sigurd Hellesvik said:

I do not know of any certain address requirements of tfm. Can you explain a bit further?

When I hand edited the `pm_static` I ran into a linker issue with tfm. Even when I did not modify the size of the partition. This was unexpected. Now that I learned more, I guess that it was an alignment issue. Nevertheless: Do you know, what the alignment requirements are for the tfm_secure partition? I would have guessed, that it will need its own 32KiB sector also, but that does not seem to be, what the partition manager puts out.

Sigurd Hellesvik said:

Yes, and there seems to be a pull request for this as well:
https://github.com/nrfconnect/sdk-nrf/pull/6245

as far as I could see this was closed.
Alternatively, we considered reserving the required amount in the `pm_static`, but I was unsure about what that amount should be (hence my question here: MCUBoot Swap using move, how to calculate size?). Now I start to understand, that the block size is 4kB and to be used in that calculation whereas the sectors/partitions are 32KiB to be used for partitioning. Is this correct?

Sigurd Hellesvik said:

Good job with searching and finding relevant DevZone cases yourself by the way.

Thanks. Maybe I should have searched a bit more before asking though ;)

[Sigurd Hellesvik]

Cla said:

These are unfortunately not accessible to me. I get a "503 Service unavailable".

Infocenter is now up and running again.

Cla said:

Do you know, what the alignment requirements are for the tfm_secure partition?

I do not know.
I will try to create a sample to move TF-M partitions to test this tomorrow or Monday.

Cla said:

as far as I could see this was closed.

I reached out to the guy who closed it and it seems like it was closed due to it being 1 year since it was created.
If you comment the issue it will show that users are interested in this feature, which may speed up its progress.

Cla said:

Alternatively, we considered reserving the required amount in the `pm_static`, but I was unsure about what that amount should be (hence my question here: MCUBoot Swap using move, how to calculate size?). Now I start to understand, that the block size is 4kB and to be used in that calculation whereas the sectors/partitions are 32KiB to be used for partitioning. Is this correct?

I agree yes.

Regards,
Sigurd Hellesvik

[Cla]

Sigurd Hellesvik said:

I will try to create a sample to move TF-M partitions to test this tomorrow or Monday.

Thanks

And thanks for the quick and detailed answer.

[Cla]

Sigurd Hellesvik said:

I will try to create a sample to move TF-M partitions to test this tomorrow or Monday.

Could it be, that tfm is using sub-regions?

https://infocenter.nordicsemi.com/pdf/nRF9160_PS_v2.0.pdf#%5B%7B%22num%22%3A2810%2C%22gen%22%3A0%7D%2C%7B%22name%22%3A%22XYZ%22%7D%2C85.039%2C465.123%2Cnull%5D

"6.15.2.1 Non-secure callable (NSC) region definition in flash"

I interpret this section to mean, that in each region there can be a "non-secure" and a "secure"-callable sub-region. This might explain, why the tfm-partition does not need to be aligned to the region border?

[Sigurd Hellesvik]

To test the placement of the tfm partition, I did the following:

Use nrf/samples/crypto/aes_cbc, with CONFIG_BOOTLOADER_MCUBOOT=y.

Now the partition manager report is as follows:

+-------------------------------------------------+
| 0x0: mcuboot (0xc000 - 48kB)                    |
+---0xc000: mcuboot_primary (0x74000 - 464kB)-----+
+---0xc000: tfm_secure (0x40200 - 256kB)----------+
| 0xc000: mcuboot_pad (0x200 - 512B)              |
+---0xc200: mcuboot_primary_app (0x73e00 - 463kB)-+
| 0xc200: tfm (0x40000 - 256kB)                   |
+---0x4c200: tfm_nonsecure (0x33e00 - 207kB)------+
| 0x4c200: app (0x33e00 - 207kB)                  |
+-------------------------------------------------+
| 0x80000: mcuboot_secondary (0x74000 - 464kB)    |
| 0xf4000: EMPTY_0 (0x4000 - 16kB)                |
+---0xf8000: tfm_storage (0x8000 - 32kB)----------+
| 0xf8000: tfm_its (0x2000 - 8kB)                 |
| 0xfa000: tfm_otp_nv_counters (0x2000 - 8kB)     |
| 0xfc000: tfm_ps (0x4000 - 16kB)                 |
+-------------------------------------------------+

  otp (0x2fc - 764B): 
+------------------------------+
| 0xff8100: otp (0x2fc - 764B) |
+------------------------------+

  sram_primary (0x80000 - 512kB): 
+------------------------------------------------+
+---0x20000000: sram_secure (0x40000 - 256kB)----+
| 0x20000000: tfm_sram (0x40000 - 256kB)         |
+---0x20040000: sram_nonsecure (0x40000 - 256kB)-+
| 0x20040000: sram_primary (0x40000 - 256kB)     |
+------------------------------------------------+

To move the tfm partition, I increased the size of mcuboot.
Then the tfm partition would have to move to make space.

I used the following pm_static.yml:

mcuboot:
  address: 0x0
  end_address: 0x10000
  placement:
    before:
    - mcuboot_primary
  region: flash_primary
  size: 0x10000

And get the following partition manager report:

+--------------------------------------------------+
| 0x0: mcuboot (0x10000 - 64kB)                    |
+---0x10000: mcuboot_primary (0x74000 - 464kB)-----+
+---0x10000: tfm_secure (0x40200 - 256kB)----------+
| 0x10000: mcuboot_pad (0x200 - 512B)              |
+---0x10200: mcuboot_primary_app (0x73e00 - 463kB)-+
| 0x10200: tfm (0x40000 - 256kB)                   |
+---0x50200: tfm_nonsecure (0x33e00 - 207kB)-------+
| 0x50200: app (0x33e00 - 207kB)                   |
+--------------------------------------------------+
| 0x84000: mcuboot_secondary (0x74000 - 464kB)     |
+---0xf8000: tfm_storage (0x8000 - 32kB)-----------+
| 0xf8000: tfm_its (0x2000 - 8kB)                  |
| 0xfa000: tfm_otp_nv_counters (0x2000 - 8kB)      |
| 0xfc000: tfm_ps (0x4000 - 16kB)                  |
+--------------------------------------------------+

  otp (0x2fc - 764B): 
+------------------------------+
| 0xff8100: otp (0x2fc - 764B) |
+------------------------------+

  sram_primary (0x80000 - 512kB): 
+------------------------------------------------+
+---0x20000000: sram_secure (0x40000 - 256kB)----+
| 0x20000000: tfm_sram (0x40000 - 256kB)         |
+---0x20040000: sram_nonsecure (0x40000 - 256kB)-+
| 0x20040000: sram_primary (0x40000 - 256kB)     |
+------------------------------------------------+

Observe that in the first build, tfm starts at 0xc200.
At the second build, tfm starts at 0x10200

Regards,
Sigurd Hellesvik

[Cla]

Very good, thanks.
So, just for my understanding, the tfm partition does not have to be aligned to the flash-regions (and can therefore share a region with the `app`), because it makes use of non-secure-callable sub-regions (https://infocenter.nordicsemi.com/pdf/nRF9160_PS_v2.0.pdf#%5B%7B%22num%22%3A2810%2C%22gen%22%3A0%7D%2C%7B%22name%22%3A%22XYZ%22%7D%2C85.039%2C465.123%2Cnull%5D)?

[Sigurd Hellesvik]

Hi,

Cla said:

So, just for my understanding, the tfm partition does not have to be aligned to the flash-regions (and can therefore share a region with the `app`), because it makes use of non-secure-callable sub-regions

Since we have 2 different regions we talk about here, I am a bit unsure what you ask about.
But I asked our experts the following question:

"Does the TF-M partition need to be aligned to fit with The SPUs 32 KiB regions?"

And they answer:

"The answer is yes.
Or to be more precise. The application needs to start on an SPU aligned 32 / 16 KB region.
So that means if you have bootloaders, that affects it also. As the sum of bootloaders + TF-M sizes is what you need to look at."

and
"Correct it needs to be aligned within the locking granularity of https://infocenter.nordicsemi.com/topic/ps_nrf9160/spu.html?cp=2_0_0_5_14_1#register.FLASHREGION.PERM
So that you are ensured that the flash can be set to secure.
"

Is this what you were looking for?

My example might have misled you some.
When testing it, I had to increase the size of MCUboot by exactly 16KiB as you see.
If I remember correctly, it did not build for smaller increases.
I should have realized this when testing, but oh well.
Good job asking questions about this, it increases both our understanding on the matter.

Regards,
Sigurd Hellesvik

[Cla]

Sigurd Hellesvik said:

Is this what you were looking for?

Not quite. I am sorry, that I did not express myself well.

I was wondering, why the end of the `tfm` partition, bordering the `app` did not have to be aligned to `0x8000`. My guess would be, that FLASHNSC[n].REGION and FLASHNSC[n].SIZE are use to define the secure-callable and non-secure-callable sub-regions within a region?

This is more an understanding question out of interest. The original question can be considered answered.
Thanks for your help.

[Sigurd Hellesvik]

Cla said:

This is more an understanding question out of interest. The original question can be considered answered.

This also helps me get more understanding into this complicated topic, so I am glad to continue answering.

I might take a bit longer to answer than usual, since the important part of the case is solved, but I will figure this out eventually.

Regards,
Sigurd Hellesvik

[Sigurd Hellesvik]

First I was spending a bit of time to understand that you were talking about the last 0x200 part of the tfm partition.

And now that I understand it, I it looks odd to me, and I think it could even be a bug in the partitioning.

I think that TF-M does not really care that much about the partitions of zephyr, as it does not really build in the "zephyr way".

Either way, I have asked our developers about this.

Regards,
Sigurd Hellesvik