Partition Manager not using flash efficiently

[Cla]

Dear Nordic Team

When I run the command `west build -t partition_manager_report` from inside our build folder I get :

  flash_primary (0x100000 - 1024kB): 
+--------------------------------------------------+
| 0x0: mcuboot (0xc000 - 48kB)                     |
| 0xc000: EMPTY_0 (0x4000 - 16kB)                  |
+---0x10000: mcuboot_primary (0x70000 - 448kB)-----+
+---0x10000: tfm_secure (0xc200 - 48kB)------------+
| 0x10000: mcuboot_pad (0x200 - 512B)              |
+---0x10200: mcuboot_primary_app (0x6fe00 - 447kB)-+
| 0x10200: tfm (0xc000 - 48kB)                     |
+---0x1c200: tfm_nonsecure (0x63e00 - 399kB)-------+
| 0x1c200: app (0x63e00 - 399kB)                   |
+--------------------------------------------------+
| 0x80000: mcuboot_secondary (0x70000 - 448kB)     |
| 0xf0000: EMPTY_1 (0x8000 - 32kB)                 |
+---0xf8000: nonsecure_storage (0x6000 - 24kB)-----+
| 0xf8000: nvs_storage (0x6000 - 24kB)             |
+---0xfe000: tfm_storage (0x2000 - 8kB)------------+
| 0xfe000: tfm_its (0x2000 - 8kB)                  |
+--------------------------------------------------+

This seems like an inefficient use of flash and flash is quite valuable to us. We have to make use of every kB we can.
It seems like a waste in the following respects:

1. Why is there an EMPTY_0 partition? Potential gain 16kB
2. Why is there an EMPTY_1 partition? Potential gain 32kB
3. Why does mcuboot take 48kB, it only needs 33996B in our build? Potential gain at least ~12kB

My main question is, how can the flash layout be optimized (by Hand)? If it cannot, how can partition manager be disable, so that the partitions from the device-tree are used instead?

My assumption is, that to optimize the flash layout one has to move to a static partition setup as described here: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/scripts/partition_manager/partition_manager.html#static-configuration
So I copied the `build/paritions.yml` file to the `pm_static.yml` and started hand-optimizing. This is quite difficult, but I managed to achieve the following temporary result:

  flash_primary (0x100000 - 1024kB): 
+-------------------------------------------------+
| 0x0: mcuboot (0x9000 - 36kB)                    |
+---0x9000: mcuboot_primary (0x77000 - 476kB)-----+
+---0x9000: tfm_secure (0x13200 - 76kB)-----------+
| 0x9000: mcuboot_pad (0x200 - 512B)              |
+---0x9200: mcuboot_primary_app (0x76e00 - 475kB)-+
| 0x9200: tfm (0x13200 - 76kB)                    |
+---0x1c200: tfm_nonsecure (0x63e00 - 399kB)------+
| 0x1c400: app (0x63c00 - 399kB)                  |
+-------------------------------------------------+
| 0x80000: EMPTY_1 (0x1000 - 4kB)                 |
| 0x81000: mcuboot_secondary (0x77000 - 476kB)    |
+---0xf8000: nonsecure_storage (0x6000 - 24kB)----+
| 0xf8000: nvs_storage (0x6000 - 24kB)            |
+---0xfe000: tfm_storage (0x2000 - 8kB)-----------+
| 0xfe000: tfm_its (0x2000 - 8kB)                 |
+-------------------------------------------------+

I managed to reduce the empty partitions and move the free space into the mcuboot_primary and secondary partitions respectively. To achieve this, I had to increase the tfm-partitions, instead the goal is to increase the app partitions. Trying to reduce the tfm partition size again to the original 48kB and moving the free space into the app partition I run into a linker issue.

That leaves me with the following secondary questions:
1. Is this approach to optimize the flash-layout correct?
2. Is there a reason, that the tfm partition is expected to be located at a certain address?
3. What is the mcubootpad partition for?
4. Why do some of the partitions use the keyword "align" in the pm_static.yml? The documentation reads "Ensure the alignment of the start or the end of the partition by specifying a dict with a start or end key respectively, where the value is the number of bytes to align to. If necessary, empty partitions are inserted in front of or behind the partition to ensure that the alignment is correct...." In our case it is "align: -start: 0x8000", does this mean, that the start of those partitions needs to be a multiple of 0x8000 for maximum space efficiency (or the partition manager will insert empty partitions)? What are the alignment requirements?
5. We are using mcuboot for firmware upgrades as described here: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/mcuboot/design.html#swap-without-using-scratch. As described there, the primary partition needs to be bigger by a certain amount than the secondary partition. Can this be baked into the flash layout, such that an offending (too big) image is not only discovered when trying to switch to it after reboot when upgrading, but earlier? Ideally the secondary slot has exactly the size, such that it fits into the primary slot with enough padding for the move algorithm as described in the link.

[Cla]

Update:
On 4. I found some helpful infos here: Interpretation of the partition manager report
Is the CONFIG_NRF_SPU_FLASH_REGION_SIZE = 0x8000 in the nRF9160 ?

On 3. It seems it is for the image header answer found here: Partition manager primary and secondary slot sizes

On 5. It seems others have also observed:
Partition manager primary and secondary slot sizes

 Big primary image prevents FOTA 

Regarding the empty partitions also see:
RE: west build generates large EMPTY partitions

After investigating a bit, I think I have to update my assumptions. Maybe someone could confirm?
- On the nRF9160 the flash sector size is 0x8000=32KiB. It has a total of 1MiB/32KiB=32 sectors.
- The sectors occupied by mcuboot can not hold anything else. Since the build > 32KiB it takes 2 sectors
- The storages (nvs and tfm) currently takes 1 sector, exactly 32kiB size.
- that leaves 32 - 3 = 29 sectors for the mcuboot primary and secondary partition. They need to be equal size (leaving out the caveat about the move algorithm for now). They also need to be aligned to the flash sectors (if so, why?). That means the sector count has to be even 29-1=28. Each partition needs half of the space 28/2=14. Which is what we see in the above report (14*32KiB = 448KiB).

[Sigurd Hellesvik]

Hi,

My main question is, how can the flash layout be optimized (by Hand)? If it cannot, how can partition manager be disable, so that the partitions from the device-tree are used instead?

Yes, static partitioning (pm_static.yml) is the way to optimize partitions "by hand".

If you use child images, Partition Manager is included by default, so I think it will be hard to disable it if you want to use MCUboot.

1. Is this approach to optimize the flash-layout correct?

You have two different ways to configure mcuboot flash usage:

1. CONFIG_PM_PARTITION_SIZE_MCUBOOT and CONFIG_PM_PARTITION_SIZE_MCUBOOT_PAD. Ref  RE: west build generates large EMPTY partitions 
2. pm_static.yml

MCUboot operates with static partitioning.
Meaning that you can not change partitioning in an DFU update to MCUboot.

Because of this, I think it is just as well to use pm_static.yml, as you might want static partitions either way.

2. Is there a reason, that the tfm partition is expected to be located at a certain address?

I do not know of any certain address requirements of tfm. Can you explain a bit further?

If we imagine an application without a bootloader for a moment: The tfm partition is placed at the beginning of flash to make sure it is the first thing that is executed.

Cla said:

On 3. It seems it is for the image header answer found here: Partition manager primary and secondary slot sizes

Yes

Cla said:

On 4. I found some helpful infos here: Interpretation of the partition manager report
Is the CONFIG_NRF_SPU_FLASH_REGION_SIZE = 0x8000 in the nRF9160 ?

The general block size of the flash is 4kB(0x1000), see INFO.CODEPAGESIZE in the FICR.
However, the SPU splits the flash for partitions, and have a size of 32kB(0x8000). See SPU Flash access control.

Use Kcofig(in VS Code) search to verify what this configuration is set to for your build specifically.
Or "west build -t menuconfig" in the Terminal.

Cla said:

On 5. It seems others have also observed:
Partition manager primary and secondary slot sizes

 Big primary image prevents FOTA 

Yes, and there seems to be a pull request for this as well:
https://github.com/nrfconnect/sdk-nrf/pull/6245

Cla said:

After investigating a bit, I think I have to update my assumptions. Maybe someone could confirm?
- On the nRF9160 the flash sector size is 0x8000=32KiB. It has a total of 1MiB/32KiB=32 sectors.
- The sectors occupied by mcuboot can not hold anything else. Since the build > 32KiB it takes 2 sectors
- The storages (nvs and tfm) currently takes 1 sector, exactly 32kiB size.
- that leaves 32 - 3 = 29 sectors for the mcuboot primary and secondary partition. They need to be equal size (leaving out the caveat about the move algorithm for now). They also need to be aligned to the flash sectors (if so, why?). That means the sector count has to be even 29-1=28. Each partition needs half of the space 28/2=14. Which is what we see in the above report (14*32KiB = 448KiB).

Yes, both I and one of my colleagues agree with your assumptions.

I hope I answered your questions.
Let me know if I missed some or if you have more.
Good job with searching and finding relevant DevZone cases yourself by the way.

Regards,
Sigurd Hellesvik

[Cla]

Very good, thanks.
So, just for my understanding, the tfm partition does not have to be aligned to the flash-regions (and can therefore share a region with the `app`), because it makes use of non-secure-callable sub-regions (https://infocenter.nordicsemi.com/pdf/nRF9160_PS_v2.0.pdf#%5B%7B%22num%22%3A2810%2C%22gen%22%3A0%7D%2C%7B%22name%22%3A%22XYZ%22%7D%2C85.039%2C465.123%2Cnull%5D)?

[Sigurd Hellesvik]

Hi,

Cla said:

So, just for my understanding, the tfm partition does not have to be aligned to the flash-regions (and can therefore share a region with the `app`), because it makes use of non-secure-callable sub-regions

Since we have 2 different regions we talk about here, I am a bit unsure what you ask about.
But I asked our experts the following question:

"Does the TF-M partition need to be aligned to fit with The SPUs 32 KiB regions?"

And they answer:

"The answer is yes.
Or to be more precise. The application needs to start on an SPU aligned 32 / 16 KB region.
So that means if you have bootloaders, that affects it also. As the sum of bootloaders + TF-M sizes is what you need to look at."

and
"Correct it needs to be aligned within the locking granularity of https://infocenter.nordicsemi.com/topic/ps_nrf9160/spu.html?cp=2_0_0_5_14_1#register.FLASHREGION.PERM
So that you are ensured that the flash can be set to secure.
"

Is this what you were looking for?

My example might have misled you some.
When testing it, I had to increase the size of MCUboot by exactly 16KiB as you see.
If I remember correctly, it did not build for smaller increases.
I should have realized this when testing, but oh well.
Good job asking questions about this, it increases both our understanding on the matter.

Regards,
Sigurd Hellesvik

[Cla]

Sigurd Hellesvik said:

Is this what you were looking for?

Not quite. I am sorry, that I did not express myself well.

I was wondering, why the end of the `tfm` partition, bordering the `app` did not have to be aligned to `0x8000`. My guess would be, that FLASHNSC[n].REGION and FLASHNSC[n].SIZE are use to define the secure-callable and non-secure-callable sub-regions within a region?

This is more an understanding question out of interest. The original question can be considered answered.
Thanks for your help.

[Sigurd Hellesvik]

Cla said:

This is more an understanding question out of interest. The original question can be considered answered.

This also helps me get more understanding into this complicated topic, so I am glad to continue answering.

I might take a bit longer to answer than usual, since the important part of the case is solved, but I will figure this out eventually.

Regards,
Sigurd Hellesvik

[Sigurd Hellesvik]

First I was spending a bit of time to understand that you were talking about the last 0x200 part of the tfm partition.

And now that I understand it, I it looks odd to me, and I think it could even be a bug in the partitioning.

I think that TF-M does not really care that much about the partitions of zephyr, as it does not really build in the "zephyr way".

Either way, I have asked our developers about this.

Regards,
Sigurd Hellesvik

[Sigurd Hellesvik]

Our developers agree that the part of the tfm partition that exceeds the secure flash is there because we have forgot to take the padding into account.

This means it is a bug, and we have created an internal ticket for it.

Thanks for pointing this out!

Regards,
Sigurd Hellesvik

[Cla]

It is good to hear, that this discussion helped in finding a bug.

Unfortunately I do not really understand it to 100% yet.

Cla said:

I was wondering, why the end of the `tfm` partition, bordering the `app` did not have to be aligned to `0x8000`. My guess would be, that FLASHNSC[n].REGION and FLASHNSC[n].SIZE are use to define the secure-callable and non-secure-callable sub-regions within a region?

My focus is on the internal workings here, just for better understanding. Did I understand this correctly?

[Sigurd Hellesvik]

Hi,

Cla said:

I was wondering, why the end of the `tfm` partition, bordering the `app` did not have to be aligned to `0x8000`.

If you take the end -0x200, I think you will get it aligned to 0x8000, right?
If yes, then it is the bug I talked about.
Let me know if you see other numbers.

Cla said:

My guess would be, that FLASHNSC[n].REGION and FLASHNSC[n].SIZE are use to define the secure-callable and non-secure-callable sub-regions within a region?

The Non-Secure callable regions are part of the secure flash.
From Non-secure callable (NSC) region definition in flash:
A non-secure callable sub-region can only exist within an existing secure region

The way I understand this, the secure region still has the same boundaries limited to 32KiB.
And the NSC regions are just a sub-region of the secure region.

Regards,
Sigurd Hellesvik

[Cla]

In your example yes, but in my example it is way off:

tfm:
address: 0x8200
end_address: 0x14200

That lead me to think, that it need not be aligned.

Sigurd Hellesvik said:

The way I understand this, the secure region still has the same boundaries limited to 32KiB.
And the NSC regions are just a sub-region of the secure region.

I understand it like that too. My guess was, that it will configure the region that contains both (tfm and app) as secure with a non-secure subregion. Lets say like this:

32 KiB | secure     | tfm part 1 (start: 0x8000, end: 0xf000)
32 KiB | secure     | tfm part 2 (start: 0x0f000, end: 0x14200)
                    | app part 1 (non-secure subregion, start: 0x14200, end: 0x18000)
32 KiB | non-secure | app part 2 (start: 0x18000, end: 0x1f000)
...

[Sigurd Hellesvik]

Cla said:

tfm:
address: 0x8200
end_address: 0x14200

That looks 0x4000 = 16KiB aligned, which is odd.

Cla said:

I understand it like that too. My guess was, that it will configure the region that contains both (tfm and app) as secure with a non-secure subregion

That could be. It would be a bit confusing if actual secure and non-secure regions were mixed in such a way.

Can you share the project or the pm_static.yml to achieve your partitioning so I can try it myself?

Regards,
Sigurd Hellesvik