Matter bridge build process fails with "region `FLASH' overflowed"

RE: Matter bridge build process fails with "region `FLASH' overflowed"

AHaug — Wed, 17 Apr 2024 11:43:59 GMT

Hi,

Great

Just 1 thing for information: I would recommend that you wait until NCS 2.6.1 before you attempt wifi patch relocation which requires qspi due to a bug in the qspi driver in v2.6.0.

This has been fixed upstream and will be released within a couple of weeks

Fix posted upstream: https://github.com/zephyrproject-rtos/zephyr/pull/71373
Cherry-pick into NCS: https://github.com/nrfconnect/sdk-nrf/pull/14812

Kind regards,
Andreas

RE: Matter bridge build process fails with "region `FLASH' overflowed"

Ondro — Tue, 16 Apr 2024 12:41:12 GMT

Hi,

I have no progress. I'm working on the build that fits on the internal flash:

[quote userid="127836" url="~/f/nordic-q-a/109935/matter-bridge-build-process-fails-with-region-flash-overflowed/477779"]Fortunately, switching release build + Os optimization helps me enough but now I'm at 99% of flash usage so we will see[/quote]

I'm not expecting that the image size would increase more. And if yes then I would try to enable the wifi fw patch relocation. In my point of view, it could be closed for now.

Thank you and BR,

Ondro

RE: Matter bridge build process fails with "region `FLASH' overflowed"

AHaug — Tue, 16 Apr 2024 08:00:55 GMT

Hi,

It's been a while since you've heard from me. Have you had any success with setting up your partitions since last time?

If not, could you share the entire project in a zip?

Kind regards,
Andreas

RE: Matter bridge build process fails with "region `FLASH' overflowed"

Ondro — Wed, 10 Apr 2024 10:59:50 GMT

[quote userid="107683" url="~/f/nordic-q-a/109935/matter-bridge-build-process-fails-with-region-flash-overflowed/477981"]Might be that the device does not know of the external flash. How did you set it up in your devicetree files?[/quote]

I didn't make any changes in devicetree. All what I changed was in shared.zip

RE: Matter bridge build process fails with "region `FLASH' overflowed"

AHaug — Wed, 10 Apr 2024 09:05:49 GMT

Hi,

[quote user="Ondro"]At the first look, Wifi patch relocation to external flash seemed to be promising but unfortunately it is for external flashes connected over QSPI and not SPI. QSPI is used for Wifi chip. According to the documentation "...fw_patches_ext_flash.html", nRF7002DK is not supported. I found something related to it at devzone: ncs-external-flash-ota-change-qspi-to-spi but I didn't go deeper.[/quote]

Ah yeah you're right. The relocation through the XIP features in that documentation requires qspi, so the alternative would be to do a similar relocation using flash API and your own proprietary implementation of this. https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/applications/matter_bridge/doc/matter_bridge_description.html#building-and-running explains how you can enable the wifi fw patch relocation as well

[quote user="Ondro"] nRF7002DK is not supported[/quote]

This is note quite right, but the demo is showcased using a nRF5340DK with a 7002DK as a shield. The nRF7002 supports the firmware patch relocation. But the 7002 is only a host-chip, and it requires a companion chip to work. On the 7002DK there is two nRF devices: The nRF7002 host chpi which is used for Wifi and Wifi-stuff only, connected with SPI to a nRF5340 that is acting as the companion chip (i.e where the application runs). In addition there is an external flash (mx25) that is by default connected to the 5340 with QSPI. Your explanation leads me to believe that you're not quite aware that this is how the 7002Dk is set up, but I could be wrong.

[quote user="Ondro"]I've read the intermediate course too and I would say that matter_bridge application is very complicated especially on nRF7002DK.[/quote]

It is a comprehensive application, yes.

[quote user="Ondro"]On the base of this I have no clue what to do.[/quote]

It might be better to start with simpler applications, i.e familiarize yourself with the other Matter over Thread and Matter over Wifi samples and see how you can optimize their footprint through either fw relocation to the external flash or through

[quote user="Ondro"]In my opinion, I don't see any restriction why an region of the external flash should be "unknown". What could be a reason of this error?[/quote]

[error] [ Worker] - Address range [0x0013B000 - 0x0013B591] does not map to a known memory.

Might be that the device does not know of the external flash. How did you set it up in your devicetree files?

Kind regards,
Andreas

RE: Matter bridge build process fails with "region `FLASH' overflowed"

Ondro — Wed, 10 Apr 2024 08:34:08 GMT

I tried to put every settings what could make a sense to *.conf files. It seems that redundant settings are ignored and conflicting ones caused errors thus I didn't spent time with distinguishing what is relevant and what not. Then I updated pm_static_dfu.yml and it seems that I have a progress. Files are here: devzone.nordicsemi.com/.../shared.zip

Sizes in regions where are not a part of external flash was increased due to:

[138/148] Generating zephyr/mcuboot_primary_app.hex
[139/148] Generating zephyr/mcuboot_primary.hex
[140/148] Generating ../../zephyr/app_update.bin
FAILED: zephyr/app_update.bin /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_update.bin
cd /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/modules/mcuboot && /home/ubuntu/Projects/nordic-4.1.2/toolchains/19127b768d/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-objcopy --input-target=ihex --output-target=binary --gap-fill=0xff /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/mcuboot_primary_app.hex /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_to_sign.bin && /home/ubuntu/Projects/nordic-4.1.2/toolchains/19127b768d/usr/local/bin/python3.9 /home/ubuntu/Projects/nordic-4.1.2/v2.6.0-rc1/bootloader/mcuboot/scripts/imgtool.py sign --key /home/ubuntu/Projects/nordic-4.1.2/v2.6.0-rc1/bootloader/mcuboot/root-rsa-2048.pem --header-size 0x200 --align 4 --version 2.5.99+0 --pad-header --slot-size 0xeb000 /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_to_sign.bin /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_update.bin
Usage: imgtool.py sign [OPTIONS] INFILE OUTFILE
Try 'imgtool.py sign -h' for help.

Error: Image size (0xeb798) + trailer (0x630) exceeds requested size 0xeb000
image.py: sign the payload
[141/148] Generating ../../zephyr/app_signed.hex

I was a bit confused why it generated primary_app when I wanted to use the mcuboot_secondary.

At the moment I facing to a problem with flashing the image to the device/chip:

west flash -d /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build --skip-rebuild --dev-id 1050712793 --erase

-- west flash: using runner nrfjprog
-- runners.nrfjprog: mass erase requested
-- runners.nrfjprog: reset after flashing requested
-- runners.nrfjprog: Flashing file: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/merged_domains.hex
-- runners.nrfjprog: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/merged_domains.hex targets both nRF53 coprocessors; splitting it into: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_NETWORK_merged_domains.hex and /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_APPLICATION_merged_domains.hex
[ #################### ]   1.167s | Erase file - Done erasing                                                          
[ #################### ]   1.949s | Program file - Done programming                                                    
[ #################### ]   1.565s | Verify file - Done verifying                                                       
[error] [ Client] - Encountered error -173: Command erase_file executed for 422 milliseconds with result -173          
[error] [ Worker] - Address range [0x0013B000 - 0x0013B591] does not map to a known memory.
ERROR: The file specified is not a valid hex file, has data outside valid areas
ERROR: or does not have data in valid areas.
NOTE: For additional output, try running again with logging enabled (--log).
NOTE: Any generated log error messages will be displayed.
FATAL ERROR: command exited with status 52: nrfjprog --program /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_APPLICATION_merged_domains.hex --chiperase --verify -f NRF53 --coprocessor CP_APPLICATION --snr 1050712793

In my opinion, I don't see any restriction why an region of the external flash should be "unknown". What could be a reason of this error?

RE: Matter bridge build process fails with "region `FLASH' overflowed"

Ondro — Tue, 09 Apr 2024 11:29:52 GMT

Hi Andreas,

thank you for the response.

At the first look, Wifi patch relocation to external flash seemed to be promising but unfortunately it is for external flashes connected over QSPI and not SPI. QSPI is used for Wifi chip. According to the documentation "...fw_patches_ext_flash.html", nRF7002DK is not supported. I found something related to it at devzone: ncs-external-flash-ota-change-qspi-to-spi but I didn't go deeper.

Any attemps to use external flash finished without any impact. I used for verification:

west build -t partition_manager_report
west build -t rom_report

Fortunately, switching release build + Os optimization helps me enough but now I'm at 99% of flash usage so we will see.

I've read the intermediate course too and I would say that matter_bridge application is very complicated especially on nRF7002DK. According to this: https://github.com/nrfconnect/sdk-nrf/blob/v2.6.0-rc1/applications/matter_bridge/sysbuild/mcuboot/boards/nrf7002dk_nrf5340_cpuapp.conf

it seems that is it:
- Simultaneous multi-image DFU
- MCUboot’s serial recovery of the networking core image

There are also:
- nRF7002DK needs a special configuration in comparing to nRF5340DK due to QSPI vs SPI issue
- nRF5340DK has 2 cores(app, net) which increases the number of partition regions

On the base of this I have no clue what to do.

RE: Matter bridge build process fails with "region `FLASH' overflowed"

AHaug — Fri, 05 Apr 2024 12:16:11 GMT

Hi,

Unfortunately the Matter over Wfi version of the samples are quite large due to the requirement of both the Wifi stack and Matter libraries, but we're working on creating ROM efficient solutions that can fit on the internal flash, but the bridge application is quite demanding in general

A couple of items that you can consider to do:

Wifi patch relocation to external flash: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/2.6.0/nrf/device_guides/working_with_nrf/nrf70/developing/fw_patches_ext_flash.html
1. This might save you 50-120kB depending on the wifi firmware patch
2. Disclaimer, if you choose the xip
Secondary mcuboot application slots on external flash (as is default on the DKs) is a must for DFU
CONFIG_PM_PARTITION_REGION_SETTINGS_STORAGE_EXTERNAL=y to allocate settings partitions to external flash is also a good solution as you state you've done
Minimal mcuboot as seen here https://github.com/nrfconnect/sdk-nrf/blob/main/samples/matter/lock/child_image/mcuboot/prj_release.conf (used in the release version of for instance the Matter door lock)
General footprint optimization https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/test_and_optimize/optimizing/index.html

[quote user="massimodutte"] you may need to configure the Partition Manager to utilize it effectively[/quote]

When massimodutte refers to this I believe they're suggesting that you customize the partitions to fit what you need through static configuration of the partition manager. There aren't any good samples that illustrates how to do this, so the best way to learn more is to go through the documentation I mentioned, and this lesson in the intermediate course https://academy.nordicsemi.com/courses/nrf-connect-sdk-intermediate/lessons/lesson-8-bootloaders-and-dfu-fota/ with focus on the section regarding partition manager and static configuration.

[quote user="Ondro"]
which matches with the chip. Then I would guess it is OK.[/quote]

Thats correct, the nRF7002DK has the mx25 external flash device and is enabled through the board files. You can inspect the partitioning by using the partition manager tool in the extension or by navigating to the build folder and using the command ninja partition_manager_report to see how it is set up (if you remove what makes it overflow you should get a successful build).

[quote user="Ondro"]- I'm sure that my device supports multi-image builds because this project is multi-image one and it works well in the original state. It is also possible to see in the folder structure:[/quote]

This is correct

[quote user="Ondro"]I'm not sure if it is everything or should I add something else.[/quote]

If you have any other partitions that you don't need to execute or that does not need to be in a secure part of the ROM (i.e don't put keys on external flash), then you are limited with what you can do put on the external flash.

Regarding XIP, this is a possible option but I would hardly recommend that your first XIP project is a Matter project. Start with something simpler that you have more control over to get familiarized with the qspi driver and

A couple of big key items with executing code from external flash is

It demands a lot more power than executing code from internal flash
You will need to have control over every single part that you execute from there w.r.t timers and interrupts. Nothing that is depending on timing critical code should be put on the external flash for code execution.

Let me know if this helps you get any closer with shrinking your modified Matter bridge application. And disclaimer, do note that this application is only experimental as of now

Kind regards,
Andreas

RE: Matter bridge build process fails with "region `FLASH' overflowed"

Ondro — Fri, 05 Apr 2024 09:08:19 GMT

Thank you for the fast response.

- You didn't mentioned re-arrange internal flash. Does it mean it is not possible?
- I mentioned optimizations only to show a big picture what I tried to use. I'm not going to follow this way.
- I checked again the presence of the external flash and found it here in file PCA10143_Schematic_And_PCB.pdf
where is MX25R6435F chip connected over SPI. The file ./boards/nrf7002dk_nrf5340_cpuapp.overlay contains:

/ {
  chosen {
    nordic,pm-ext-flash = &mx25r64;
  };
};

which matches with the chip. Then I would guess it is OK.

- I'm sure that my device supports multi-image builds because this project is multi-image one and it works well in the original state. It is also possible to see in the folder structure:

./child_image/
    - mcuboot
    - hci_ipc

- If we are talking about "partition manager configuration file", I think that it is about this file ./configuration/nrf7002dk_nrf5340_cpuapp/pm_static_dfu.yml

Here I made a few experiments but without success, e.g:
- I increased a bit of "app" size or
- I removed all blocks with "region: external_flash" and changed a region in blocks with "region: flash_primary" to "region: external_flash"

I assume this file is connected to files:

- ./prj.conf
- ./child_image/mcuboot/prj.conf

where I added (in both of them), e.g:

CONFIG_SPI=y
CONFIG_SPI_NOR=y
CONFIG_FLASH=y
CONFIG_PM_OVERRIDE_EXTERNAL_DRIVER_CHECK=y
CONFIG_PM_PARTITION_REGION_SETTINGS_STORAGE_EXTERNAL=y

I'm not sure if it is everything or should I add something else.

I'm afraid that everything what I tried to edit to solve the issue was ignored due to one of this warnings appeared during the build:

=== child image b0n - CPUNET (inherited) end ===
...

          ---------------------------------------------------------------------
          --- WARNING: Using a bootloader without pm_static.yml.            ---
          --- There are cases where a deployed product can consist of       ---
          --- multiple images, and only a subset of these images can be     ---
          --- upgraded through a firmware update mechanism. In such cases,  ---
          --- the upgradable images must have partitions that are static    ---
          --- and are matching the partition map used by the bootloader     ---
          --- programmed onto the device.                                   ---
          ---------------------------------------------------------------------
...
=== child image hci_ipc - CPUNET end ===
...
=== child image mcuboot -  end ===
...
-- Found partition manager static configuration: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/configuration/nrf7002dk_nrf5340_cpuapp/pm_static_dfu.yml
Partition 'mcuboot' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_pad' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_primary' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_primary_app' is not included in the dynamic resolving since it is statically defined.
Partition 'settings_storage' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_primary_1' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_secondary' is not included in the dynamic resolving since it is statically defined.
Partition 'mcuboot_secondary_1' is not included in the dynamic resolving since it is statically defined.
Partition 'pcd_sram' is not included in the dynamic resolving since it is statically defined.
CMake Warning at /home/ubuntu/Projects/nordic-4.1.2/v2.6.0-rc1/nrf/cmake/partition_manager.cmake:593 (message):
  Could not find memory mapped address for XIP.  Generated update hex files
  will not have the correct base address.  Hence they can not be programmed
  directly to the external flash

The whole log is devzone.nordicsemi.com/.../matter_2D00_bridge.build.zip. It is from the build process with edited prj.conf files only.

You also mentioned "memory regions". I have an overview what is it but I have no clue how they are connected to the binary code which should filled there. Is my issue "section `datas' will not fit in region `FLASH'" connected to the "app" block? If yes where I can see it?

I would like to ask you to be more specific in your answer because I'm completely new to this field.

RE: Matter bridge build process fails with "region `FLASH' overflowed"

massimodutte — Thu, 04 Apr 2024 16:12:51 GMT

You've already mentioned using optimization flags like -Os to optimize code size. Ensure that these flags are applied correctly and consider experimenting with other optimization flags if necessary. If your device has an external flash memory, you may need to configure the Partition Manager to utilize it effectively. This involves defining memory regions and their sizes in the partition manager configuration file. Make sure that your firmware image is configured to be stored in the external flash if you have one. If your device supports multi-image builds, you can split your firmware into multiple images, each stored in different memory regions. This can help manage memory usage more efficiently.