MCUBoot, ExFat USB Filesystem, and Flash Region Overflows.

[FinnBiggs]

Hello again folks,

Recently, I closed a ticket regarding MCUBoot, DFU, and the USB Mass Filesystem. Unfortunately, I'm still having problems. That ticket, for the curious, is here:  MCUBoot DFU Interference with ExFat Filesystem In that ticket, the problem ended up being to do with the method of adding an affiliation tag to my external storage partition.

When that was resolved, I closed the ticket. However, I'm still stuck in trying to get DFU Enabled alongside a USB Filesystem. Not on the DevKit, where it's working, but in the transfer from devKit to my production board.

As always, here are the details of my environment.

Nordic SDK v2.5.0

Hardware: (working) nRF52840dk

Hardware: (faulting) Xiao BLE Sense

Now, I'll describe the problem.

I have developed an application based on a few different nordic samples. In particular, I am using the usb/mass sample for my filesystem (configured as an ExFat, not littlefs system). I am also using pieces of the LBS sample, and nRF Desktop to enable OTA DFU that preserves my filesystem.

This application compiles, runs, and performs DFU OTA on my nRF52840dk. My current application (if anyone wants it as a sample) is built on top of this one, which was the resolution of my last ticket. https://github.com/FinnWBiggs/mass_pm

From there, I've added child_image subdirectories, and changed my partition manager tool from CONFIG_PM_SINGLE_IMAGE to CONFIG_BOOTLOADER_MCUBOOT, which does add a layer of complexity.

Additionally, I've enabled the options used in nRF Desktop for DFU by following the options implied by those options and including them in my prj.conf

All together, this makes for a complicated app. Regardless, it runs well on the dev kit, and updates the firmware while preserving our filesystem.

The complication arises when building for Xiao BLE sense. Despite sharing a common prj.conf (actually a prj_mcuboot_smp.conf), and having identical structures for the creation of the child image, the Xiao build overflows by the same exact same amount of space no matter what changes I make.

Rather than attaching a git repository, I will attach a zip file of this project so that it can't disappear in the future. (As an aside, what is the preferred method of attaching a lot of information to these help forum posts? Git / Zip / In-text code blocks?)

In this attached zip is a Notes.md, which details a large portion of the changes I've made and the effects those changes have had.

The ones I will call out here to keep folks from having to dig in:

• Adding a chosen nordic,pm-ext-flash to the xiao build (using the &p2516h), the occupied flash size of a compiled test application with no DFU.
• The pm_static.yml that is shared by both xiao_ble_sense and nrf52840dk_nrf52840 provides:
  • a primary and secondary mcuboot partition of 488kB (size = 0x7a000)
  • which yields (allotting for the mcuboot pad) an app and mcuboot_primary_app partition size of 0x79e00
• The changes made to the pm_static.yml that differ from the default partitioning are as follows:
  • Before: 1MB 52840 flash divided as:
    • 
    • (And please ignore arbitrary_partition, I was using it for other tests and it got left in while I was taking notes)
  • After: 1MB 52840 flash divided as:
    
    • 
    • Notable changes:
      • settings storage moved from primary->external flash
      • mcuboot primary and secondary expanded by half of the size of settings_storage
      • app increased by half size of settings_storage
      • address of mcuboot_secondary shifted to match new endpoint of app and mcuboot_primary
• The size of my compiled application is significantly smaller than my app partition (which is fine, but important to note regarding the overflow on the other board)
  • The compiled application size for nrf52840 is 378216B (378.2 kB), which is placed in a partition of size 487.5 kB.
• I can't validate the compiled size of the application for the xiao_ble_sense, since the application will not compile and always fails in the same way.

  • c:/ncs/toolchains/c57af46cb7/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd.exe: zephyr\zephyr_pre0.elf section `rodata' will not fit in region `FLASH'
    c:/ncs/toolchains/c57af46cb7/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd.exe: region `FLASH' overflowed by 2408 bytes

• Changing the compiler optimizations does not change the amount by which the flash overflows.
  • All of Debug, Speed, and Size optimizations overflow by 2408B (2.4 kB), which is significantly less than the ~100kB I had spare in my build for nrf52840dk

I would be very grateful for advice on how to proceed. Thank you very much in advance for your time.

All the best,

   -  Finn

ATTACHED FILE: mass_pm_dfu.zip (zip of relevant project directory, including my builds).

mass_pm_dfu_TOSHARE.zip

[Sigurd Hellesvik]

Welcome back!

Great job on the formatting on the ticket! As you say this project is starting to become complicated, so good formatting on the ticket will enable us to help you a loot faster than if not.

I tried to build your project for the xiao baord and the nRF52840DK, but get a different error. So I think I am doing something wrong.
Could you include build instructions, so I know I am getting to the right issue?
I work from the CLI, so a single west command would be preffered, if not too much to ask.

Regards,
Sigurd Hellesvik

[FinnBiggs]

Hi again Sigurd,

I'm happy to provide a build command. The error you ran into was likely my fault -- I checked to see if I could build from the zip I sent, and it looks like having the build directories included there has some interaction with CMakeCache.txt that prevents a build.

That cache problem can be fixed either by removing all build* directories, or by using the new zip at the end of this file (which is the old zip, but with the builds removed).

For brevity, I'm going to say "dk" in place of "nrf52840dk_nrf52840", and "xiao" instead of "xiao_ble_sense".

• My build configuration
  • Configuration file (both xiao and dk)
    • ./configuration/common/prj_mcuboot_smp.conf
  • Board
    • xiao
      • xiao_ble_sense
    • dk
      • nrf52840dk_nrf52840
  • DeviceTree Overlay
    • xiao
      • ./configuration/xiao_ble_sense/xiao_ble_sense.overlay
    • dk
      • ./configuration/nrf52840dk_nrf52840/nrf52840dk_nrf52840.overlay
  • Optimization
    • xiao and dk
      • Default is fine
• West commands
  • xiao
    • west build --build-dir ./build_xiao . --pristine --board xiao_ble_sense --no-sysbuild -- -DNCS_TOOLCHAIN_VERSION=NONE -DCONF_FILE=./configuration/common/prj_mcuboot_smp.conf -DEXTRA_DTC_OVERLAY_FILE=configuration/xiao_ble_sense/xiao_ble_sense.overlay
  • dk
    • west build --build-dir ./build_dk . --pristine --board nrf52840dk_nrf52840 --no-sysbuild -- -DNCS_TOOLCHAIN_VERSION=NONE -DCONF_FILE=./configuration/common/prj_mcuboot_smp.conf -DEXTRA_DTC_OVERLAY_FILE=configuration/nrf52840dk_nrf52840/nrf52840dk_nrf52840.overlay

Thank you for all your time.

Best,

    - Finn

ATTACHED: mass_pm_dfu.zip (same project as before, but with build directories absent and some unused files deleted)

7587.mass_pm_dfu.zip

As an aside, I had to delete and remake this reply. Putting the zip file in the middle of the text block caused a number of bizarre formatting problems that rendered my reply unreadable.

[Sigurd Hellesvik]

So let start with something a bit less complex, but which I think is wrong:

Your MCUboot configuration does not seem to be included.

You are missing ";" after "okay" in both of these, but you do not get an error?
That would mean that the file is not included.

To make sure that all your files are included, go through them one by one and add an obvious error + build them. This way, you should see that it crashes once per file, and thus you know that what you expect to be included really has been.

[FinnBiggs]

Argh!

Sorry to leave you hanging on such a trivial issue -- I had some travel and personal things to take care of.

Here's a follow up. Excluding those (mcuboot conf) files but adding the content I cared about (making sure xiao's external flash was enabled via devicetree) gives the exact same behavior.

Below is the filetree of the project (both builds)

mass_pm_dfu/
┣ configuration/
┃ ┣ common/
┃ ┃ ┗ prj_mcuboot_smp.conf
┃ ┣ nrf52840dk_nrf52840/
┃ ┃ ┣ child_image/
┃ ┃ ┃ ┗ mcuboot/
┃ ┃ ┃   ┣ mcuboot_private.pem
┃ ┃ ┃   ┗ prj_mcuboot_smp.conf
┃ ┃ ┗ nrf52840dk_nrf52840.overlay
┃ ┗ xiao_ble_sense/
┃   ┣ child_image/
┃ ┃ ┃ ┗ mcuboot/
┃ ┃ ┃   ┣ mcuboot_private.pem
┃ ┃ ┃   ┗ prj_mcuboot_smp.conf
┃   ┗ xiao_ble_sense.overlay
┣ notes/
┃ ┣ filetree.md
┃ ┣ Notes.md
┃ ┗ screenshot_pm.png
┣ src/
┃ ┗ main.c
┣ CMakeLists.txt
┣ Kconfig
┣ pm_static.yml
┗ sample.yaml

---

• Both board configurations use /common/prj_mcuboot_smp.conf as their primary configurations
• Both boards use pm_static.yml as their partition manager
• The  /<board>/ subfolder-structures are identical
  • the /child_image/mcuboot/mrj_mcuboot_smp.conf are both blank
  • the private keys are identical4
• The error is exactly the same (ro_data overflows by 2408 bytes)

Are the mcuboot conf files strictly necessary as long as I have all the values that I want?

Anyways, here's a new zip with those changes, and I'll reattach the west build commands to reproduce as well.

NOTE: I've added the west commands (though same command as before) to this zip under /cmds/xiao.sh and cmds/dk.sh

If you have this folder open in your terminal, you can build either one with

./cmds/xiao.sh

or

./cmds/dk.sh

(Please check that the contents of the bash script match what you expect - I don't want to be your security risk. If you like, just grab the text from those and run it)

Thank you very much

     - Finn

mass_pm_dfu_2024-08-02.zip

[Elfving]

Hello Finn,

Just letting you know that Sigurd is currently out of office, but will be back sometime next week. If your problem is urgent and doesn't allow you to wait until then, let me know.

Regards,

Elfving

[Sigurd Hellesvik]

Hi Finn,

I am back from vacation now, and am catching up today.
I will get back to you later this week. Hopefully tomorrow.

[Sigurd Hellesvik]

Hi Finn,

the Xiao build overflows by the same exact same amount of space no matter what changes I make.

This is probably because MCUboot overflow.

One trick to reduce space in MCUboot is to do this in MCUboot config:

CONFIG_CBPRINTF_NANO=y

That seems to fix the overflow for me, but I get some other errors. I hope that those other erros will not be present on your side. So test this one first.

Alternativley you could change partitioning to increase MCUboot space

[FinnBiggs]

Hi Sigurd,

Welcome back from vacation, I hope it was pleasant!

I've been pushing at this with no luck. As part of trying to get things working, I restructured the project to match the structure from the nrf connect sdk intermediate course lesson on FOTA: (https://github.com/NordicDeveloperAcademy/ncs-inter/tree/main/lesson8/inter_less8_exer3_solution)

My new file tree:

mass_pm_dfu/
┣ .vscode/
┃ ┗ settings.json
┣ boards/
┃ ┣ nrf52840dk_nrf52840.conf
┃ ┣ nrf52840dk_nrf52840.overlay
┃ ┣ xiao_ble_sense.conf
┃ ┗ xiao_ble_sense.overlay
┣ child_image/
┃ ┣ mcuboot.conf
┃ ┣ mcuboot_private.pem
┃ ┣ nrf52840dk_nrf52840.overlay
┃ ┗ xiao_ble_sense.overlay
┣ scripts/
┃ ┣ dk_build.sh
┃ ┗ xiao_build.sh
┣ src/
┃ ┗ main.c
┣ CMakeLists.txt
┣ Kconfig
┣ pm_static.yml
┗ prj.conf

The new build command is really annoying.

• The build system will not automatically find the child_image conf file
  • I include the following build command
  • -Dmcuboot_EXTRA_CONF_FILE="$BASE_DIR/child_image/mcuboot.conf"
• Much to my chagrin, the mcuboot_EXTRA_CONF_FILE seems to search at
  • C:/ncs/v2.5.0/bootloader/mcuboot/boot/zephyr/
• As a fix, I made bash scripts that passes the mcuboot_EXTRA_CONF_FILE with a $(pwd)
  
  • -Dmcuboot_EXTRA_CONF_FILE="$(pwd)/child_image/mcuboot.conf"
• My recommendation for building this project is to run:
  • cd <PROJECT BASE DIRECTORY>
  • ./scripts/dk_build.sh
  • ./scripts/xiao_build.sh
• Or, direct CLI Build
  • cd into the project directory and run one of the following depending on build target

  • # build nrf52840dk_nrf52840
    west build \
        --build-dir build_dk \
        . --pristine \
        --board nrf52840dk_nrf52840 \
        --no-sysbuild \
    -- \
        -DNCS_TOOLCHAIN_VERSION=NONE \
        -DEXTRA_CONF_FILE=boards/nrf52840dk_nrf52840.conf \
        -DDTC_OVERLAY_FILE=boards/nrf52840dk_nrf52840.overlay \
        -Dmcuboot_EXTRA_CONF_FILE="$(pwd)/child_image/mcuboot.conf"

  • # build xiao_ble_sense
    west build \
        --build-dir build_xiao \
        . --pristine \
        --board xiao_ble_sense \
        --no-sysbuild \
    -- \
        -DNCS_TOOLCHAIN_VERSION=NONE \
        -DEXTRA_CONF_FILE=boards/xiao_ble_sense.conf \
        -DDTC_OVERLAY_FILE=boards/xiao_ble_sense.overlay \
        -Dmcuboot_EXTRA_CONF_FILE="$(pwd)/child_image/mcuboot.conf"

I notice no difference in overflow amount when I include/exclude CONFIG_CBPRINTF_NANO=y. Intellisense seems to think that the option is already enabled.

(That seems to be an option already handled in /c/ncs/v2.5.0/bootloader/mcuboot/boot/zephyr/prj.conf)

When I set the CONFIG_CBPRINTF_NANO=n, I get the same amount of overflow as when it's 'y'.

I do however notice a difference in the overflow amount as a result of the restructuring. Now, instead of a 2408B overflow, it's 1560B. 

----------------------

Seems like the Nano option isn't the right approach, which brings me to your second suggestion -- change the partitioning to increase mcuboot space.

Is it permitted to separate the MCUBoot partitions? Otherwise, it's not possible to make more space.

The flash_primary partition runs from 0x0000000 to 0x000fffff

Would it be feasible to try moving mcuboot_secondary into external_flash?

Thank you very much, I appreciate your advice.

Best,

    - Finn

[Sigurd Hellesvik]

Hi Finn,

You forgot to upload the new project I think.

I will upload a fixed project at end of this message still. But first let me answer your questions:

FinnBiggs said:

As part of trying to get things working, I restructured the project to match the structure from the nrf connect sdk intermediate course lesson on FOTA

I like this. Maybe just because I am more familiar with this structure. But I like it.

FinnBiggs said:

The build system will not automatically find the child_image conf file

It should. See my project.

FinnBiggs said:

When I set the CONFIG_CBPRINTF_NANO=n, I get the same amount of overflow as when it's 'y'.

Right yes I see that I was mistaken in my previous reply.

FinnBiggs said:

Seems like the Nano option isn't the right approach, which brings me to your second suggestion -- change the partitioning to increase mcuboot space.

Is it permitted to separate the MCUBoot partitions? Otherwise, it's not possible to make more space.

I think you are mixing up partitions here. Let me explain them:

• mcuboot: The bootloader MCUboot lives here. It is 0xC000 by default
• mcuboot_primary: The application lives here. It will be half of available space left after all other partitioning is done.
• mcuboot_secondary: Your update lives here. It will be the same size as mcuboot_primary.

The flash overflow is in mcuboot. This means that moving or resizing mcuboot_primary/mcuboot_secondary will not matter too much. As long as there is space for the app inside them ofc.

FinnBiggs said:

Would it be feasible to try moving mcuboot_secondary into external_flash?

It is very possible and maybe even recommended. But it won't fix this issue specifically.

Fixed project

Here is what I did:
I took this one:

FinnBiggs said:

mass_pm_dfu_2024-08-02.zip

and reformatted it to use my preferred directory structure.

Then I added a child image folder to add a pair of configurations to MCUboot.

Then I removed most things from pm_static.yml. Why?
In my mind, pm_static has two use-cases:
1. To define custom partitions
2. To freeze partitioning for release. (This can be done by copying build/partitions.yml when done developing) ((Freezing partitioning for release is a hard  requirement by DFU))

For none of these, you need to define it all for developing. Instead I recommend only defining partitions as you need them, and then use Kconfig options to change sizes. This way, the build system will automatically partition your project for you, and you are not as limited in what you can do. See that I added CONFIG_PM_PARTITION_SIZE_MCUBOOT to the project to upscale MCUboot. (This upscaling should fix the flash overflow)
And I use similar configurations for Settings.

And here is the project.

6165.mass_pm_dfu_urd_2024-08-14.zip

Let me know how this works on your  side.

FinnBiggs said:

Welcome back from vacation, I hope it was pleasant!

Thanks, it was very nice indeed!