Matter bridge build process fails with "region `FLASH' overflowed"

Hi guys,

I'm playing with : Matter Bridge from SDK v2.6.0.rc1 on nRF7002-DK. I tried to extend Matter data model with "Temperature color light" and add a new device but it caused that its flash is overflowed:


...
-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/zephyr_pre0.elf section `datas' will not fit in region `FLASH'
/home/ubuntu/Projects/nordic-4.1.2/toolchains/19127b768d/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: region `FLASH' overflowed by 1784 bytes



My question is very simple: How to fix it? What exacly should I change to solve it?
I know that DK has 1MB internal flash and 8MB external flash...I hope. I try to read documentation about Partition manager, Multi-image builds, external flash, XIP but I'm a bit lost. I also tried to find something here in issues. Would it be possible to somehow re-arrange internal flash? or is possible to use external flash instead of internal one? I've played with it a couple of days but without any success. There are too many options for me which must match together (files: *.conf, *.overlay, *.yml, ...). Optimizations like "-Os", "no debug symbols" didn't help me much.

Parents
  • You've already mentioned using optimization flags like -Os to optimize code size. Ensure that these flags are applied correctly and consider experimenting with other optimization flags if necessary. If your device has an external flash memory, you may need to configure the Partition Manager to utilize it effectively. This involves defining memory regions and their sizes in the partition manager configuration file. Make sure that your firmware image is configured to be stored in the external flash if you have one. If your device supports multi-image builds, you can split your firmware into multiple images, each stored in different memory regions. This can help manage memory usage more efficiently.

  • Thank you for the fast response.

    - You didn't mentioned re-arrange internal flash. Does it mean it is not possible?
    - I mentioned optimizations only to show a big picture what I tried to use. I'm not going to follow this way.
    - I checked again the presence of the external flash and found it here in file PCA10143_Schematic_And_PCB.pdf
    where is MX25R6435F chip connected over SPI. The file ./boards/nrf7002dk_nrf5340_cpuapp.overlay contains:

    / {
      chosen {
        nordic,pm-ext-flash = &mx25r64;
      };
    };


    which matches with the chip. Then I would guess it is OK.


    - I'm sure that my device supports multi-image builds because this project is multi-image one and it works well in the original state. It is also possible to see in the folder structure:

    ./child_image/
        - mcuboot
        - hci_ipc


    - If we are talking about "partition manager configuration file", I think that it is about this file ./configuration/nrf7002dk_nrf5340_cpuapp/pm_static_dfu.yml

    Here I made a few experiments but without success, e.g:
    - I increased a bit of "app" size or
    - I removed all blocks with "region: external_flash" and changed a region in blocks with "region: flash_primary" to "region: external_flash"

    I assume this file is connected to files:

    - ./prj.conf
    - ./child_image/mcuboot/prj.conf

    where I added (in both of them), e.g:

    CONFIG_SPI=y
    CONFIG_SPI_NOR=y
    CONFIG_FLASH=y
    CONFIG_PM_OVERRIDE_EXTERNAL_DRIVER_CHECK=y
    CONFIG_PM_PARTITION_REGION_SETTINGS_STORAGE_EXTERNAL=y  


    I'm not sure if it is everything or should I add something else.

    I'm afraid that everything what I tried to edit to solve the issue was ignored due to one of this warnings appeared during the build:


    === child image b0n - CPUNET (inherited) end ===
    ...
    
              ---------------------------------------------------------------------
              --- WARNING: Using a bootloader without pm_static.yml.            ---
              --- There are cases where a deployed product can consist of       ---
              --- multiple images, and only a subset of these images can be     ---
              --- upgraded through a firmware update mechanism. In such cases,  ---
              --- the upgradable images must have partitions that are static    ---
              --- and are matching the partition map used by the bootloader     ---
              --- programmed onto the device.                                   ---
              ---------------------------------------------------------------------
    ...
    === child image hci_ipc - CPUNET end ===
    ...
    === child image mcuboot -  end ===
    ...
    -- Found partition manager static configuration: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/configuration/nrf7002dk_nrf5340_cpuapp/pm_static_dfu.yml
    Partition 'mcuboot' is not included in the dynamic resolving since it is statically defined.
    Partition 'mcuboot_pad' is not included in the dynamic resolving since it is statically defined.
    Partition 'mcuboot_primary' is not included in the dynamic resolving since it is statically defined.
    Partition 'mcuboot_primary_app' is not included in the dynamic resolving since it is statically defined.
    Partition 'settings_storage' is not included in the dynamic resolving since it is statically defined.
    Partition 'mcuboot_primary_1' is not included in the dynamic resolving since it is statically defined.
    Partition 'mcuboot_secondary' is not included in the dynamic resolving since it is statically defined.
    Partition 'mcuboot_secondary_1' is not included in the dynamic resolving since it is statically defined.
    Partition 'pcd_sram' is not included in the dynamic resolving since it is statically defined.
    CMake Warning at /home/ubuntu/Projects/nordic-4.1.2/v2.6.0-rc1/nrf/cmake/partition_manager.cmake:593 (message):
      Could not find memory mapped address for XIP.  Generated update hex files
      will not have the correct base address.  Hence they can not be programmed
      directly to the external flash


    The whole log is matter-bridge.build.zip. It is from the build process with edited prj.conf files only.

    You also mentioned "memory regions". I have an overview what is it but I have no clue how they are connected to the binary code which should filled there. Is my issue "section `datas' will not fit in region `FLASH'" connected to the "app" block? If yes where I can see it?

    I would like to ask you to be more specific in your answer because I'm completely new to this field.

  • Hi,

    Unfortunately the Matter over Wfi version of the samples are quite large due to the requirement of both the Wifi stack and Matter libraries, but we're working on creating ROM efficient solutions that can fit on the internal flash, but the bridge application is quite demanding in general

    A couple of items that you can consider to do:

    1. Wifi patch relocation to external flash: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/2.6.0/nrf/device_guides/working_with_nrf/nrf70/developing/fw_patches_ext_flash.html
      1. This might save you 50-120kB depending on the wifi firmware patch
      2. Disclaimer, if you choose the xip
    2. Secondary mcuboot application slots on external flash (as is default on the DKs) is a must for DFU
    3. CONFIG_PM_PARTITION_REGION_SETTINGS_STORAGE_EXTERNAL=y   to allocate settings partitions to external flash is also a good solution as you state you've done
    4. Minimal mcuboot as seen here https://github.com/nrfconnect/sdk-nrf/blob/main/samples/matter/lock/child_image/mcuboot/prj_release.conf (used in the release version of for instance the Matter door lock)
    5. General footprint optimization https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/test_and_optimize/optimizing/index.html 

    massimodutte said:
    you may need to configure the Partition Manager to utilize it effectively

    When massimodutte refers to this I believe they're suggesting that you customize the partitions to fit what you need through static configuration of the partition manager. There aren't any good samples that illustrates how to do this, so the best way to learn more is to go through the documentation I mentioned, and this lesson in the intermediate course https://academy.nordicsemi.com/courses/nrf-connect-sdk-intermediate/lessons/lesson-8-bootloaders-and-dfu-fota/ with focus on the section regarding partition manager and static configuration.

    Ondro said:

    which matches with the chip. Then I would guess it is OK.

    Thats correct, the nRF7002DK has the mx25 external flash device and is enabled through the board files. You can inspect the partitioning by using the partition manager tool in the extension or by navigating to the build folder and using the command ninja partition_manager_report to see how it is set up (if you remove what makes it overflow you should get a successful build).

    Ondro said:
    - I'm sure that my device supports multi-image builds because this project is multi-image one and it works well in the original state. It is also possible to see in the folder structure:

    This is correct

    Ondro said:
    I'm not sure if it is everything or should I add something else.

    If you have any other partitions that you don't need to execute or that does not need to be in a secure part of the ROM (i.e don't put keys on external flash), then you are limited with what you can do put on the external flash.

    Regarding XIP, this is a possible option but I would hardly recommend that your first XIP project is a Matter project. Start with something simpler that you have more control over to get familiarized with the qspi driver and 

    A couple of big key items with executing code from external flash is

    1. It demands a lot more power than executing code from internal flash
    2. You will need to have control over every single part that you execute from there w.r.t timers and interrupts. Nothing that is depending on timing critical code should be put on the external flash for code execution.

    Let me know if this helps you get any closer with shrinking your modified Matter bridge application. And disclaimer, do note that this application is only experimental as of now

    Kind regards,
    Andreas

  • Hi Andreas,

    thank you for the response.

    At the first look, Wifi patch relocation to external flash seemed to be promising but unfortunately it is for external flashes connected over QSPI and not SPI. QSPI is used for Wifi chip. According to the documentation "...fw_patches_ext_flash.html", nRF7002DK is not supported. I found something related to it at devzone: ncs-external-flash-ota-change-qspi-to-spi but I didn't go deeper.

    Any attemps to use external flash finished without any impact. I used for verification:

    west build -t partition_manager_report
    west build -t rom_report

    Fortunately,  switching release build + Os optimization  helps me enough but now I'm at 99% of flash usage so we will see.

    I've read the intermediate course too and I would say that matter_bridge application is very complicated especially on nRF7002DK. According to this: https://github.com/nrfconnect/sdk-nrf/blob/v2.6.0-rc1/applications/matter_bridge/sysbuild/mcuboot/boards/nrf7002dk_nrf5340_cpuapp.conf

    it seems that is it:
    - Simultaneous multi-image DFU
    - MCUboot’s serial recovery of the networking core image

    There are also:
    - nRF7002DK needs a special configuration in comparing to nRF5340DK due to QSPI vs SPI issue
    - nRF5340DK has 2 cores(app, net) which increases the number of partition regions

    On the base of this I have no clue what to do.

  • I tried to put every settings what could make a sense to *.conf files. It seems that redundant settings are ignored and conflicting ones caused errors thus I didn't spent time with distinguishing what is relevant and what not. Then I updated pm_static_dfu.yml and it seems that I have a progress. Files are here: shared.zip

    Sizes in regions where are not a part of external flash was increased due to:

    [138/148] Generating zephyr/mcuboot_primary_app.hex
    [139/148] Generating zephyr/mcuboot_primary.hex
    [140/148] Generating ../../zephyr/app_update.bin
    FAILED: zephyr/app_update.bin /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_update.bin
    cd /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/modules/mcuboot && /home/ubuntu/Projects/nordic-4.1.2/toolchains/19127b768d/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-objcopy --input-target=ihex --output-target=binary --gap-fill=0xff /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/mcuboot_primary_app.hex /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_to_sign.bin && /home/ubuntu/Projects/nordic-4.1.2/toolchains/19127b768d/usr/local/bin/python3.9 /home/ubuntu/Projects/nordic-4.1.2/v2.6.0-rc1/bootloader/mcuboot/scripts/imgtool.py sign --key /home/ubuntu/Projects/nordic-4.1.2/v2.6.0-rc1/bootloader/mcuboot/root-rsa-2048.pem --header-size 0x200 --align 4 --version 2.5.99+0 --pad-header --slot-size 0xeb000 /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_to_sign.bin /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_update.bin
    Usage: imgtool.py sign [OPTIONS] INFILE OUTFILE
    Try 'imgtool.py sign -h' for help.
    
    Error: Image size (0xeb798) + trailer (0x630) exceeds requested size 0xeb000
    image.py: sign the payload
    [141/148] Generating ../../zephyr/app_signed.hex

    I was a bit confused why it generated primary_app when I wanted to use the mcuboot_secondary.

    At the moment I facing to a problem with flashing the image to the device/chip:

    west flash -d /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build --skip-rebuild --dev-id 1050712793 --erase
    
    -- west flash: using runner nrfjprog
    -- runners.nrfjprog: mass erase requested
    -- runners.nrfjprog: reset after flashing requested
    -- runners.nrfjprog: Flashing file: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/merged_domains.hex
    -- runners.nrfjprog: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/merged_domains.hex targets both nRF53 coprocessors; splitting it into: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_NETWORK_merged_domains.hex and /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_APPLICATION_merged_domains.hex
    [ #################### ]   1.167s | Erase file - Done erasing                                                          
    [ #################### ]   1.949s | Program file - Done programming                                                    
    [ #################### ]   1.565s | Verify file - Done verifying                                                       
    [error] [ Client] - Encountered error -173: Command erase_file executed for 422 milliseconds with result -173          
    [error] [ Worker] - Address range [0x0013B000 - 0x0013B591] does not map to a known memory.
    ERROR: The file specified is not a valid hex file, has data outside valid areas
    ERROR: or does not have data in valid areas.
    NOTE: For additional output, try running again with logging enabled (--log).
    NOTE: Any generated log error messages will be displayed.
    FATAL ERROR: command exited with status 52: nrfjprog --program /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_APPLICATION_merged_domains.hex --chiperase --verify -f NRF53 --coprocessor CP_APPLICATION --snr 1050712793

    In my opinion, I don't see any restriction why an region of the external flash should be "unknown". What could be a reason of this error?

Reply
  • I tried to put every settings what could make a sense to *.conf files. It seems that redundant settings are ignored and conflicting ones caused errors thus I didn't spent time with distinguishing what is relevant and what not. Then I updated pm_static_dfu.yml and it seems that I have a progress. Files are here: shared.zip

    Sizes in regions where are not a part of external flash was increased due to:

    [138/148] Generating zephyr/mcuboot_primary_app.hex
    [139/148] Generating zephyr/mcuboot_primary.hex
    [140/148] Generating ../../zephyr/app_update.bin
    FAILED: zephyr/app_update.bin /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_update.bin
    cd /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/modules/mcuboot && /home/ubuntu/Projects/nordic-4.1.2/toolchains/19127b768d/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-objcopy --input-target=ihex --output-target=binary --gap-fill=0xff /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/mcuboot_primary_app.hex /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_to_sign.bin && /home/ubuntu/Projects/nordic-4.1.2/toolchains/19127b768d/usr/local/bin/python3.9 /home/ubuntu/Projects/nordic-4.1.2/v2.6.0-rc1/bootloader/mcuboot/scripts/imgtool.py sign --key /home/ubuntu/Projects/nordic-4.1.2/v2.6.0-rc1/bootloader/mcuboot/root-rsa-2048.pem --header-size 0x200 --align 4 --version 2.5.99+0 --pad-header --slot-size 0xeb000 /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_to_sign.bin /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/app_update.bin
    Usage: imgtool.py sign [OPTIONS] INFILE OUTFILE
    Try 'imgtool.py sign -h' for help.
    
    Error: Image size (0xeb798) + trailer (0x630) exceeds requested size 0xeb000
    image.py: sign the payload
    [141/148] Generating ../../zephyr/app_signed.hex

    I was a bit confused why it generated primary_app when I wanted to use the mcuboot_secondary.

    At the moment I facing to a problem with flashing the image to the device/chip:

    west flash -d /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build --skip-rebuild --dev-id 1050712793 --erase
    
    -- west flash: using runner nrfjprog
    -- runners.nrfjprog: mass erase requested
    -- runners.nrfjprog: reset after flashing requested
    -- runners.nrfjprog: Flashing file: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/merged_domains.hex
    -- runners.nrfjprog: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/merged_domains.hex targets both nRF53 coprocessors; splitting it into: /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_NETWORK_merged_domains.hex and /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_APPLICATION_merged_domains.hex
    [ #################### ]   1.167s | Erase file - Done erasing                                                          
    [ #################### ]   1.949s | Program file - Done programming                                                    
    [ #################### ]   1.565s | Verify file - Done verifying                                                       
    [error] [ Client] - Encountered error -173: Command erase_file executed for 422 milliseconds with result -173          
    [error] [ Worker] - Address range [0x0013B000 - 0x0013B591] does not map to a known memory.
    ERROR: The file specified is not a valid hex file, has data outside valid areas
    ERROR: or does not have data in valid areas.
    NOTE: For additional output, try running again with logging enabled (--log).
    NOTE: Any generated log error messages will be displayed.
    FATAL ERROR: command exited with status 52: nrfjprog --program /home/ubuntu/Projects/matter3/examples/nordic/bridge-for-ble-devices/build/zephyr/GENERATED_CP_APPLICATION_merged_domains.hex --chiperase --verify -f NRF53 --coprocessor CP_APPLICATION --snr 1050712793

    In my opinion, I don't see any restriction why an region of the external flash should be "unknown". What could be a reason of this error?

Children
  • Hi,

    Ondro said:
    At the first look, Wifi patch relocation to external flash seemed to be promising but unfortunately it is for external flashes connected over QSPI and not SPI. QSPI is used for Wifi chip. According to the documentation "...fw_patches_ext_flash.html", nRF7002DK is not supported. I found something related to it at devzone: ncs-external-flash-ota-change-qspi-to-spi but I didn't go deeper.

    Ah yeah you're right. The relocation through the XIP features in that documentation requires qspi, so the alternative would be to do a similar relocation using flash API and your own proprietary implementation of this. https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/applications/matter_bridge/doc/matter_bridge_description.html#building-and-running explains how you can enable the wifi fw patch relocation as well

    Ondro said:
    nRF7002DK is not supported

    This is note quite right, but the demo is showcased using a nRF5340DK with a 7002DK as a shield. The nRF7002 supports the firmware patch relocation. But the 7002 is only a host-chip, and it requires a companion chip to work. On the 7002DK there is two nRF devices: The nRF7002 host chpi which is used for Wifi and Wifi-stuff only, connected with SPI to a nRF5340 that is acting as the companion chip (i.e where the application runs). In addition there is an external flash (mx25) that is by default connected to the 5340 with QSPI. Your explanation leads me to believe that you're not quite aware that this is how the 7002Dk is set up, but I could be wrong.

    Ondro said:
    I've read the intermediate course too and I would say that matter_bridge application is very complicated especially on nRF7002DK.

    It is a comprehensive application, yes.

    Ondro said:
    On the base of this I have no clue what to do.

    It might be better to start with simpler applications, i.e familiarize yourself with the other Matter over Thread and Matter over Wifi samples and see how you can optimize their footprint through either fw relocation to the external flash or through 

    Ondro said:
    In my opinion, I don't see any restriction why an region of the external flash should be "unknown". What could be a reason of this error?

    [error] [ Worker] - Address range [0x0013B000 - 0x0013B591] does not map to a known memory.

    Might be that the device does not know of the external flash. How did you set it up in your devicetree files?

    Kind regards,
    Andreas

  • Might be that the device does not know of the external flash. How did you set it up in your devicetree files?

    I didn't make any changes in devicetree. All what I changed was in shared.zip

  • Hi,

    It's been a while since you've heard from me. Have you had any success with setting up your partitions since last time?

    If not, could you share the entire project in a zip?

    Kind regards,
    Andreas

  • Hi,

    I have no progress. I'm working on the build that fits on the internal flash:

    Fortunately,  switching release build + Os optimization  helps me enough but now I'm at 99% of flash usage so we will see

    I'm not expecting that the image size would increase more. And if yes then I would try to enable the wifi fw patch relocation. In my point of view, it could be closed for now.

    Thank you and BR,

    Ondro

  • Hi,

    Great

    Just 1 thing for information: I would recommend that you wait until NCS 2.6.1 before you attempt wifi patch relocation which requires qspi due to a bug in the qspi driver in v2.6.0.

    This has been fixed upstream and will be released within a couple of weeks

    Fix posted upstream: https://github.com/zephyrproject-rtos/zephyr/pull/71373
    Cherry-pick into NCS: https://github.com/nrfconnect/sdk-nrf/pull/14812

    Kind regards,
    Andreas

Related