Matter bridge build process fails with "region `FLASH' overflowed"

[Ondro]

Hi guys,

I'm playing with : Matter Bridge from SDK v2.6.0.rc1 on nRF7002-DK. I tried to extend Matter data model with "Temperature color light" and add a new device but it caused that its flash is overflowed:

...
-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/zephyr_pre0.elf section `datas' will not fit in region `FLASH'
/home/ubuntu/Projects/nordic-4.1.2/toolchains/19127b768d/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: region `FLASH' overflowed by 1784 bytes

My question is very simple: How to fix it? What exacly should I change to solve it?
I know that DK has 1MB internal flash and 8MB external flash...I hope. I try to read documentation about Partition manager, Multi-image builds, external flash, XIP but I'm a bit lost. I also tried to find something here in issues. Would it be possible to somehow re-arrange internal flash? or is possible to use external flash instead of internal one? I've played with it a couple of days but without any success. There are too many options for me which must match together (files: *.conf, *.overlay, *.yml, ...). Optimizations like "-Os", "no debug symbols" didn't help me much.

[massimodutte]

You've already mentioned using optimization flags like -Os to optimize code size. Ensure that these flags are applied correctly and consider experimenting with other optimization flags if necessary. If your device has an external flash memory, you may need to configure the Partition Manager to utilize it effectively. This involves defining memory regions and their sizes in the partition manager configuration file. Make sure that your firmware image is configured to be stored in the external flash if you have one. If your device supports multi-image builds, you can split your firmware into multiple images, each stored in different memory regions. This can help manage memory usage more efficiently.

[Ondro]

Thank you for the fast response.

- You didn't mentioned re-arrange internal flash. Does it mean it is not possible?
- I mentioned optimizations only to show a big picture what I tried to use. I'm not going to follow this way.
- I checked again the presence of the external flash and found it here in file PCA10143_Schematic_And_PCB.pdf
where is MX25R6435F chip connected over SPI. The file ./boards/nrf7002dk_nrf5340_cpuapp.overlay contains:

/ {
  chosen {
    nordic,pm-ext-flash = &mx25r64;
  };
};

which matches with the chip. Then I would guess it is OK.


- I'm sure that my device supports multi-image builds because this project is multi-image one and it works well in the original state. It is also possible to see in the folder structure:

./child_image/
    - mcuboot
    - hci_ipc

- If we are talking about "partition manager configuration file", I think that it is about this file ./configuration/nrf7002dk_nrf5340_cpuapp/pm_static_dfu.yml

Here I made a few experiments but without success, e.g:
- I increased a bit of "app" size or
- I removed all blocks with "region: external_flash" and changed a region in blocks with "region: flash_primary" to "region: external_flash"

I assume this file is connected to files:

- ./prj.conf
- ./child_image/mcuboot/prj.conf

where I added (in both of them), e.g:

CONFIG_SPI=y
CONFIG_SPI_NOR=y
CONFIG_FLASH=y
CONFIG_PM_OVERRIDE_EXTERNAL_DRIVER_CHECK=y
CONFIG_PM_PARTITION_REGION_SETTINGS_STORAGE_EXTERNAL=y  

I'm not sure if it is everything or should I add something else.

I'm afraid that everything what I tried to edit to solve the issue was ignored due to one of this warnings appeared during the build:

=== child image b0n - CPUNET (inherited) end ===
...

          ---------------------------------------------------------------------
          --- WARNING: Using a bootloader without pm_static.yml.            ---
          --- There are cases where a deployed product can consist of       ---
          --- multiple images, and only a subset of these images can be     ---
          --- upgraded through a firmware update mechanism. In such cases,  ---
          --- the upgradable images must have partitions that are static    ---
          --- and are matching the partition map used by the bootloader     ---
          --- programmed onto the device.                                   ---
          ---------------------------------------------------------------------
...
=== child image hci_ipc - CPUNET end ===
...
=== child image mcuboot -  end ===
...
-- Found partition manager static configuration: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/configuration/nrf7002dk_nrf5340_cpuapp/pm_static_dfu.yml
Partition 'mcuboot' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_pad' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_primary' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_primary_app' is not included in the dynamic resolving since it is statically defined.
Partition 'settings_storage' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_primary_1' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_secondary' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_secondary_1' is not included in the dynamic resolving since it is statically defined.
Partition 'pcd_sram' is not included in the dynamic resolving since it is statically defined.
CMake Warning at /home/ubuntu/Projects/nordic-4.1.2/v2.6.0-rc1/nrf/cmake/partition_manager.cmake:593 (message):
  Could not find memory mapped address for XIP.  Generated update hex files
  will not have the correct base address.  Hence they can not be programmed
  directly to the external flash

The whole log is matter-bridge.build.zip. It is from the build process with edited prj.conf files only.

You also mentioned "memory regions". I have an overview what is it but I have no clue how they are connected to the binary code which should filled there. Is my issue "section `datas' will not fit in region `FLASH'" connected to the "app" block? If yes where I can see it?

I would like to ask you to be more specific in your answer because I'm completely new to this field.

[AHaug]

Hi,

Unfortunately the Matter over Wfi version of the samples are quite large due to the requirement of both the Wifi stack and Matter libraries, but we're working on creating ROM efficient solutions that can fit on the internal flash, but the bridge application is quite demanding in general

A couple of items that you can consider to do:

1. Wifi patch relocation to external flash: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/2.6.0/nrf/device_guides/working_with_nrf/nrf70/developing/fw_patches_ext_flash.html
   1. This might save you 50-120kB depending on the wifi firmware patch
   2. Disclaimer, if you choose the xip
2. Secondary mcuboot application slots on external flash (as is default on the DKs) is a must for DFU
3. CONFIG_PM_PARTITION_REGION_SETTINGS_STORAGE_EXTERNAL=y   to allocate settings partitions to external flash is also a good solution as you state you've done
4. Minimal mcuboot as seen here https://github.com/nrfconnect/sdk-nrf/blob/main/samples/matter/lock/child_image/mcuboot/prj_release.conf (used in the release version of for instance the Matter door lock)
5. General footprint optimization https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/test_and_optimize/optimizing/index.html 

massimodutte said:

you may need to configure the Partition Manager to utilize it effectively

When massimodutte refers to this I believe they're suggesting that you customize the partitions to fit what you need through static configuration of the partition manager. There aren't any good samples that illustrates how to do this, so the best way to learn more is to go through the documentation I mentioned, and this lesson in the intermediate course https://academy.nordicsemi.com/courses/nrf-connect-sdk-intermediate/lessons/lesson-8-bootloaders-and-dfu-fota/ with focus on the section regarding partition manager and static configuration.

Ondro said:

which matches with the chip. Then I would guess it is OK.

Thats correct, the nRF7002DK has the mx25 external flash device and is enabled through the board files. You can inspect the partitioning by using the partition manager tool in the extension or by navigating to the build folder and using the command ninja partition_manager_report to see how it is set up (if you remove what makes it overflow you should get a successful build).

Ondro said:

- I'm sure that my device supports multi-image builds because this project is multi-image one and it works well in the original state. It is also possible to see in the folder structure:

This is correct

Ondro said:

I'm not sure if it is everything or should I add something else.

If you have any other partitions that you don't need to execute or that does not need to be in a secure part of the ROM (i.e don't put keys on external flash), then you are limited with what you can do put on the external flash.

Regarding XIP, this is a possible option but I would hardly recommend that your first XIP project is a Matter project. Start with something simpler that you have more control over to get familiarized with the qspi driver and 

A couple of big key items with executing code from external flash is

1. It demands a lot more power than executing code from internal flash
2. You will need to have control over every single part that you execute from there w.r.t timers and interrupts. Nothing that is depending on timing critical code should be put on the external flash for code execution.

Let me know if this helps you get any closer with shrinking your modified Matter bridge application. And disclaimer, do note that this application is only experimental as of now

Kind regards,
Andreas

[Ondro]

Hi Andreas,

thank you for the response.

At the first look, Wifi patch relocation to external flash seemed to be promising but unfortunately it is for external flashes connected over QSPI and not SPI. QSPI is used for Wifi chip. According to the documentation "...fw_patches_ext_flash.html", nRF7002DK is not supported. I found something related to it at devzone: ncs-external-flash-ota-change-qspi-to-spi but I didn't go deeper.

Any attemps to use external flash finished without any impact. I used for verification:

west build -t partition_manager_report
west build -t rom_report

Fortunately,  switching release build + Os optimization  helps me enough but now I'm at 99% of flash usage so we will see.

I've read the intermediate course too and I would say that matter_bridge application is very complicated especially on nRF7002DK. According to this: https://github.com/nrfconnect/sdk-nrf/blob/v2.6.0-rc1/applications/matter_bridge/sysbuild/mcuboot/boards/nrf7002dk_nrf5340_cpuapp.conf

it seems that is it:
- Simultaneous multi-image DFU
- MCUboot’s serial recovery of the networking core image

There are also:
- nRF7002DK needs a special configuration in comparing to nRF5340DK due to QSPI vs SPI issue
- nRF5340DK has 2 cores(app, net) which increases the number of partition regions

On the base of this I have no clue what to do.

[Ondro]

I tried to put every settings what could make a sense to *.conf files. It seems that redundant settings are ignored and conflicting ones caused errors thus I didn't spent time with distinguishing what is relevant and what not. Then I updated pm_static_dfu.yml and it seems that I have a progress. Files are here: shared.zip

Sizes in regions where are not a part of external flash was increased due to:

[138/148] Generating zephyr/mcuboot_primary_app.hex
[139/148] Generating zephyr/mcuboot_primary.hex
[140/148] Generating ../../zephyr/app_update.bin
FAILED: zephyr/app_update.bin /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_update.bin
cd /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/modules/mcuboot && /home/ubuntu/Projects/nordic-4.1.2/toolchains/19127b768d/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-objcopy --input-target=ihex --output-target=binary --gap-fill=0xff /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/mcuboot_primary_app.hex /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_to_sign.bin && /home/ubuntu/Projects/nordic-4.1.2/toolchains/19127b768d/usr/local/bin/python3.9 /home/ubuntu/Projects/nordic-4.1.2/v2.6.0-rc1/bootloader/mcuboot/scripts/imgtool.py sign --key /home/ubuntu/Projects/nordic-4.1.2/v2.6.0-rc1/bootloader/mcuboot/root-rsa-2048.pem --header-size 0x200 --align 4 --version 2.5.99+0 --pad-header --slot-size 0xeb000 /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_to_sign.bin /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_update.bin
Usage: imgtool.py sign [OPTIONS] INFILE OUTFILE
Try 'imgtool.py sign -h' for help.

Error: Image size (0xeb798) + trailer (0x630) exceeds requested size 0xeb000
image.py: sign the payload
[141/148] Generating ../../zephyr/app_signed.hex

I was a bit confused why it generated primary_app when I wanted to use the mcuboot_secondary.

At the moment I facing to a problem with flashing the image to the device/chip:

west flash -d /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build --skip-rebuild --dev-id 1050712793 --erase

-- west flash: using runner nrfjprog
-- runners.nrfjprog: mass erase requested
-- runners.nrfjprog: reset after flashing requested
-- runners.nrfjprog: Flashing file: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/merged_domains.hex
-- runners.nrfjprog: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/merged_domains.hex targets both nRF53 coprocessors; splitting it into: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_NETWORK_merged_domains.hex and /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_APPLICATION_merged_domains.hex
[ #################### ]   1.167s | Erase file - Done erasing                                                          
[ #################### ]   1.949s | Program file - Done programming                                                    
[ #################### ]   1.565s | Verify file - Done verifying                                                       
[error] [ Client] - Encountered error -173: Command erase_file executed for 422 milliseconds with result -173          
[error] [ Worker] - Address range [0x0013B000 - 0x0013B591] does not map to a known memory.
ERROR: The file specified is not a valid hex file, has data outside valid areas
ERROR: or does not have data in valid areas.
NOTE: For additional output, try running again with logging enabled (--log).
NOTE: Any generated log error messages will be displayed.
FATAL ERROR: command exited with status 52: nrfjprog --program /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_APPLICATION_merged_domains.hex --chiperase --verify -f NRF53 --coprocessor CP_APPLICATION --snr 1050712793

In my opinion, I don't see any restriction why an region of the external flash should be "unknown". What could be a reason of this error?