Increase size of custom SPM partition

[NelsonGoncalves]

Hello,

I am using NCS v1.5.0 and my application runs in non-secure mode, and uses a custom SPM. Initially all build and ran correctly, and the SPM fit in a 32 KB partition. Then I added some secure services and the SPM image size increased and I got this build error:

Memory region         Used Size  Region Size  %age Used
           FLASH:         64 KB        48 KB    133.33%
            SRAM:        8568 B        64 KB     13.07%
        IDT_LIST:          40 B         2 KB      1.95c:/nordic/v1.5.0/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: zephyr\zephyr_prebuilt.elf section `.gnu.sgstubs' will not fit in region `FLASH'
c:/nordic/v1.5.0/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: SPM and app are sharing an SPU region. Cannot partition flash correctly into secure and non-secure. Adjust partitions sizes so they are placed in separate regions.
c:/nordic/v1.5.0/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: region `FLASH' overflowed by 16384 bytes
collect2.exe: error: ld returned 1 exit status

Looking at the partition manager setup for the SPM pm.yml file, I noticed that the size of the SPM partition was set by this config item CONFIG_PM_PARTITION_SIZE_SPM.

So in the custom SPM project options (file prj.conf) I added CONFIG_PM_PARTITION_SIZE_SPM=0x010000, making the SPM partition size 64 KB because the size of the custom SPM became 48 KB and the SoC internal flash has 32 KB pages.

After this change everything builds correctly:

[27/352] Performing build step for 'spm_subimage'
[190/195] Linking C executable zephyr\zephyr_prebuilt.elf
Memory region Used Size Region Size %age Used
FLASH: 64 KB 64 KB 100.00%
SRAM: 8568 B 64 KB 13.07%
IDT_LIST: 40 B 2 KB 1.95%
[195/195] Linking C executable zephyr\zephyr.elf

but when it runs it is misteriously stuck in the function Zephyr OS initial thread and ends up in function "arch_cpu_idle()".

Then I changed config item value from hexadecimal to decimal, CONFIG_PM_PARTITION_SIZE_SPM=65536. But the build fails:

[24/352] Performing build step for 'spm_subimage'
[190/195] Linking C executable zephyr\zephyr_prebuilt.elf
Memory region         Used Size  Region Size  %age Used
           FLASH:         64 KB     415029 B     15.79%
            SRAM:        8568 B        64 KB     13.07%
        IDT_LIST:          40 B         2 KB      1.95%
[195/195] Linking C executable zephyr\zephyr.elf

(... building the application ...)

[346/352] Linking C executable zephyr\zephyr_prebuilt.elf
FAILED: zephyr/zephyr_prebuilt.elf
cmd.exe /C "cd . && C:\Nordic\v1.5.0\toolchain\opt\bin\arm-none-eabi-gcc.exe    zephyr/CMakeFiles/zephyr_prebuilt.dir/misc/empty_file.c.obj  -o zephyr\zephyr_prebuilt.elf  -Wl,-T  zephyr/linker.cmd  -Wl,-Map=C:/workspace/repos/DNG/build/zephyr/zephyr_prebuilt.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/aarch32/libarch__arm__core__aarch32.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/mpu/libarch__arm__core__aarch32__cortex_m__mpu.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/cmse/libarch__arm__core__aarch32__cortex_m__cmse.a  zephyr/lib/libc/newlib/liblib__libc__newlib.a  zephyr/lib/posix/liblib__posix.a  zephyr/soc/arm/common/cortex_m/libsoc__arm__common__cortex_m.a  zephyr/subsys/net/libsubsys__net.a  zephyr/subsys/net/ip/libsubsys__net__ip.a  zephyr/subsys/random/libsubsys__random.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/spi/libdrivers__spi.a  zephyr/drivers/flash/libdrivers__flash.a  zephyr/drivers/entropy/libdrivers__entropy.a  modules/nrf/lib/nrf_modem_lib/lib..__nrf__lib__nrf_modem_lib.a  modules/nrf/lib/fprotect/lib..__nrf__lib__fprotect.a  modules/nrf/subsys/fw_info/lib..__nrf__subsys__fw_info.a  C:/Nordic/v1.5.0/nrfxlib/nrf_modem/lib/cortex-m33/hard-float/libmodem.a  modules/nrfxlib/nrf_security/src/mbedtls/libmbedtls_base_vanilla.a  modules/hal_nordic/libmodules__hal_nordic.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -L"c:/nordic/v1.5.0/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/thumb/v8-m.main+fp/hard"  -LC:/workspace/repos/DNG/build/zephyr  -lgcc  -Wl,--print-memory-usage  ../mbus/build/libmbusStack.a  ../dlms_server/build/libDlmsServer.a  zephyr/arch/common/libisr_tables.a  modules/nrfxlib/nrf_security/src/mbedtls/shared/libmbedcrypto_shared.a  modules/nrfxlib/nrf_security/src/mbedtls/oberon/libmbedcrypto_oberon.a  -mcpu=cortex-m33  -mthumb  -mabi=aapcs  -mfpu=fpv5-sp-d16  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -no-pie  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  spm/libspmsecureentries.a  -lm  -Wl,-lc  -L"C:/Nordic/v1.5.0/toolchain/opt/arm-none-eabi"/lib/thumb/v8-m.main+fp/hard  -Wl,-lgcc  -lc  -specs=nano.specs && cd ."
c:/nordic/v1.5.0/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe:zephyr/linker.cmd:58 cannot move location counter backwards (from 00065744 to 00065738)
collect2.exe: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: 'C:\Nordic\v1.5.0\toolchain\opt\bin\cmake.EXE' --build 'c:\workspace\repos\DNG\build'

Notice that the indication of the SPM used space is now correct, from when I used the hexadecimal value. But the build still fails:

c:/nordic/v1.5.0/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe:zephyr/linker.cmd:58 cannot move location counter backwards (from 00065744 to 00065738)

What is the proper way to specify the size of the SPM partion, when using a custom build SPM ?

[Heidi]

Hi!

Using the hexadecimal value is correct.

Can you check that CONFIG_INIT_ARCH_HW_AT_BOOT is enabled (check <build_folder>/spm/zephyr/.config)?

Also, please paste the output of running ninja partition_manager_report in the build folder.

Best regards,

Heidi

[NelsonGoncalves]

Hello Heidi,

[EDIT 30/06/2021] CONFIG_INIT_ARCH_HW_AT_BOOT is enabled in <build_folder>/spm/zephyr/.config. I initially said it as not, but was I looking in (the wrong place)  <build>/zephyr/.config.

This is the ouptut of ninja partition_manager_report when the build succeeds (that is CONFIG_PM_PARTITION_SIZE_SPM=0x10000)

c:\workspace\repos\DNG\build>ninja partition_manager_report
[0/10] Performing build step for 'spm_subimage'
ninja: no work to do.
[2/2] cmd.exe /C "cd /D C:\workspace\repos\DNG\build\modules\nrf && C:\Nordic\v1.5.0\toolchain\opt\bin\python.exe C:/Nordic/v1.5.0/zephyr/../nrf/scripts/partition_manager_report.py --input C:/workspace/repos/DNG/build/partitions.yml"
  external_flash (0x800000 - 8192kB):
+----------------------------------------------+
| 0x180000: external_flash (0x800000 - 8192kB) |
+----------------------------------------------+

  flash_primary (0x100000 - 1024kB):
+--------------------------------+
| 0x0: spm (0x10000 - 64kB)      |
| 0x10000: app (0xf0000 - 960kB) |
+--------------------------------+

  otp (0x2f4 - 756B):
+------------------------------+
| 0xff8108: otp (0x2f4 - 756B) |
+------------------------------+

  sram_primary (0x40000 - 256kB):
+--------------------------------------------------+
+---0x20000000: sram_secure (0x10000 - 64kB)-------+
| 0x20000000: spm_sram (0x10000 - 64kB)            |
+---0x20010000: sram_nonsecure (0x30000 - 192kB)---+
+---0x20010000: nrf_modem_lib_sram (0x44e8 - 17kB)-+
| 0x20010000: nrf_modem_lib_ctrl (0x4e8 - 1kB)     |
| 0x200104e8: nrf_modem_lib_tx (0x2000 - 8kB)      |
| 0x200124e8: nrf_modem_lib_rx (0x2000 - 8kB)      |
+--------------------------------------------------+
| 0x200144e8: sram_primary (0x2bb18 - 174kB)       |
+--------------------------------------------------+

[NelsonGoncalves]

Hi,

I aded CONFIG_INIT_ARCH_HW_AT_BOOT=y as well to the application prj.conf, and now it is defined in the <build>/zephyr/.config file.

However the behaviour is the same, at startup it gets stuck at the Zephyr idle loop.

[NelsonGoncalves]

Hi,

I don't think this strange behavior (application stuck in the Zephyr idle task) has anything to do with the SPM partition.

I checked out my last working version, and increased the size of the SPM partition to 64 KB. The application built and launched correctly.

Other than the size of the SPM partition, the difference is that I am now using the Zephyr SPI flash driver. Could the issue be wrong a wrong configuration ? 

[Heidi]

Hi, my original comment was incorrect, I meant to check in <build_folder>/spm/zephyr/.config. I have edited it now to reflect this.

 

Can you find linker.cmd located both in <build>/zephyr/ and <build>/spm/zephyr and attach them? 

I want to see if there is any overlap in the memory regions, which would explain why it's getting stuck at startup. 

[NelsonGoncalves]

Hi,

Here are the files:

spm_zephyr_linker.zip

build_zephyr_linker.zip

I did not find anything wrong with the linker placements. I believe the issue is related to these configuration options:

CONFIG_SPI=y
CONFIG_SPI_NOR=y
CONFIG_SPI_NOR_FLASH_LAYOUT_PAGE_SIZE=4096
CONFIG_FLASH=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FLASH_JESD216_API=y

I can reproduce the issue when these options are enabled. And for reference, this is the device tree SPI external flash definition:

&spi3 {
	status = "okay";
	compatible = "nordic,nrf-spim";
	sck-pin = <13>;
	mosi-pin = <11>;
	miso-pin = <12>;
	cs-gpios = <&gpio0 25 GPIO_ACTIVE_LOW>;
	mx25r64: mx25r6435f@0 {
		compatible = "jedec,spi-nor";
		reg = <0>;
		spi-max-frequency = <8000000>;
		jedec-id = [c2 20 15];
		label = "MX25R6435F";
		size = <67108864>;
	};	
};