MCUBoot, ExFat USB Filesystem, and Flash Region Overflows.

[FinnBiggs]

Hello again folks,

Recently, I closed a ticket regarding MCUBoot, DFU, and the USB Mass Filesystem. Unfortunately, I'm still having problems. That ticket, for the curious, is here:  MCUBoot DFU Interference with ExFat Filesystem In that ticket, the problem ended up being to do with the method of adding an affiliation tag to my external storage partition.

When that was resolved, I closed the ticket. However, I'm still stuck in trying to get DFU Enabled alongside a USB Filesystem. Not on the DevKit, where it's working, but in the transfer from devKit to my production board.

As always, here are the details of my environment.

Nordic SDK v2.5.0

Hardware: (working) nRF52840dk

Hardware: (faulting) Xiao BLE Sense

Now, I'll describe the problem.

I have developed an application based on a few different nordic samples. In particular, I am using the usb/mass sample for my filesystem (configured as an ExFat, not littlefs system). I am also using pieces of the LBS sample, and nRF Desktop to enable OTA DFU that preserves my filesystem.

This application compiles, runs, and performs DFU OTA on my nRF52840dk. My current application (if anyone wants it as a sample) is built on top of this one, which was the resolution of my last ticket. https://github.com/FinnWBiggs/mass_pm

From there, I've added child_image subdirectories, and changed my partition manager tool from CONFIG_PM_SINGLE_IMAGE to CONFIG_BOOTLOADER_MCUBOOT, which does add a layer of complexity.

Additionally, I've enabled the options used in nRF Desktop for DFU by following the options implied by those options and including them in my prj.conf

All together, this makes for a complicated app. Regardless, it runs well on the dev kit, and updates the firmware while preserving our filesystem.

The complication arises when building for Xiao BLE sense. Despite sharing a common prj.conf (actually a prj_mcuboot_smp.conf), and having identical structures for the creation of the child image, the Xiao build overflows by the same exact same amount of space no matter what changes I make.

Rather than attaching a git repository, I will attach a zip file of this project so that it can't disappear in the future. (As an aside, what is the preferred method of attaching a lot of information to these help forum posts? Git / Zip / In-text code blocks?)

In this attached zip is a Notes.md, which details a large portion of the changes I've made and the effects those changes have had.

The ones I will call out here to keep folks from having to dig in:

• Adding a chosen nordic,pm-ext-flash to the xiao build (using the &p2516h), the occupied flash size of a compiled test application with no DFU.
• The pm_static.yml that is shared by both xiao_ble_sense and nrf52840dk_nrf52840 provides:
  • a primary and secondary mcuboot partition of 488kB (size = 0x7a000)
  • which yields (allotting for the mcuboot pad) an app and mcuboot_primary_app partition size of 0x79e00
• The changes made to the pm_static.yml that differ from the default partitioning are as follows:
  • Before: 1MB 52840 flash divided as:
    • 
    • (And please ignore arbitrary_partition, I was using it for other tests and it got left in while I was taking notes)
  • After: 1MB 52840 flash divided as:
    
    • 
    • Notable changes:
      • settings storage moved from primary->external flash
      • mcuboot primary and secondary expanded by half of the size of settings_storage
      • app increased by half size of settings_storage
      • address of mcuboot_secondary shifted to match new endpoint of app and mcuboot_primary
• The size of my compiled application is significantly smaller than my app partition (which is fine, but important to note regarding the overflow on the other board)
  • The compiled application size for nrf52840 is 378216B (378.2 kB), which is placed in a partition of size 487.5 kB.
• I can't validate the compiled size of the application for the xiao_ble_sense, since the application will not compile and always fails in the same way.

  • c:/ncs/toolchains/c57af46cb7/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd.exe: zephyr\zephyr_pre0.elf section `rodata' will not fit in region `FLASH'
    c:/ncs/toolchains/c57af46cb7/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd.exe: region `FLASH' overflowed by 2408 bytes

• Changing the compiler optimizations does not change the amount by which the flash overflows.
  • All of Debug, Speed, and Size optimizations overflow by 2408B (2.4 kB), which is significantly less than the ~100kB I had spare in my build for nrf52840dk

I would be very grateful for advice on how to proceed. Thank you very much in advance for your time.

All the best,

   -  Finn

ATTACHED FILE: mass_pm_dfu.zip (zip of relevant project directory, including my builds).

mass_pm_dfu_TOSHARE.zip

[Sigurd Hellesvik]

Thanks for the info.
I will get back to you tomorrow or Monday on this

[Sigurd Hellesvik]

I must confess that I did not read all your explanations. So if I missed any questions about theory, please let me know and I will explain.

However, I think I fixed the issue at hand.
First, I will say that I just tested on the DK so far. I do got an Xiao laying around somewhere, but i think and hope that when we have solved the DK issue the Xiao will be good as well.

BOOT_MAX_IMG_SECTORS=____ - too small?

This error message will also appear if the flash driver does not fail.
This is a bug that is fixed in newer versions of MCUboot.

But still, we need to know why the driver does not work. It is likely configured wrong.

To find this, I built a working sample for the NRF52840DK, and  tested this to make sure.

With two samples, I can compare configurations.
Since it fails in MCUboot I will compare MCUboot configurations.
So for each sample I compare the following files:

• build/mcuboot/zephyr/zephyr.dts
• build/mcuboot/zephyr/.config

With this, I found that in .config, your sample is missing CONFIG_NORDIC_QSPI_NOR, so I added that.

And for DTS, I found that "nordic,pm-ext-flash = &mx25r64;" is missing altogether.
So I added "aosihfjkasbfjkbsakj" to the overlay file. And this did not give a build  error, meaning that the file itself is not parsed.

We can fix this in two ways:

• Either move the file to child_image/mcuboot/nrf52840dk_nrf52840.overlay.
  • I think this should work at least
• Or to add the path of the mcuboot overlay to the build script.
  • This is the method I used
  • This must use the full path, cause unintuitively it will use the relative path from https://github.com/nrfconnect/sdk-mcuboot/blob/main/boot/zephyr/ and not from your app.

mass_pm_dfu_urd_2024-08-19.zip

This fixes the "mcuboot cannot find  external flash" issue. I think that is the one issue you had here right?

[FinnBiggs]

Hi Sigurd,

I have good and bad news.

The good news: your changes work perfectly on the Dev Kit.

The bad news: the Xiao version fails with no build errors, serial/console output, or RTT output. For Xiao, this application builds correctly, but does not boot, and requires a JLink probe to recover back into its bootloader mode.

I have no idea where to go from here - there's not a ton of information I can send to you because I can't get any output from this application on my target hardware.

--------------------

Sigurd Hellesvik said:

So if I missed any questions about theory, please let me know and I will explain.

There's one aspect of your process where I got lost:

How did you know that the overflow was coming from the mcuboot partition, and not one of the others?

On that same note, why would the required size for that partition change?

--------------------

I'm extremely grateful for all the help you've provided on these last few responses. Can I send a cake or a box of pastries to your office as a thank you?

Cheers,

    - Finn

[Sigurd Hellesvik]

FinnBiggs said:

How did you know that the overflow was coming from the mcuboot partition, and not one of the others?

First and foremost experience.

Secondly, if you change the app, the overflow stays the same, so it must be something else, and for the nRF52840, that can only be MCUboot.

But if you read closely in the build log, you can see that the paths close before the flash overflow error, you will see the path to MCUboot ncs/bootloaders/mcuboot/boot/zephyr.

FinnBiggs said:

On that same note, why would the required size for that partition change?

Default size is 0xC000, which is true for the default MCUboot configuration, plus eventual board-specific MCUboot configurations.

If you add more functionality to MCUboot, you may get a bigger MCUboot app.
This is typically adding exeternal flash drivers, or Serial Recovery functionality.

The flash overflow issue has two different solutions:

• Increasing MCUboot partition size.
• Reducing MCUboot app size by disabling stuff in Kconfig
  • See for example prj_minimal.conf
  • At some point you will have optimied away everything you do not need, at which case the other option is the only left.

The latter is technically more clean if possible, but harder.

Another point to make here is:
MCUboot will inherit configurations from board files.
This means that you could get into a state where MCUboot works for nRF52840 but fails for the Xiao.

And this is a good seg-way to the issue.

I propose the following steps, all for the Xiao:

1. Verify that MCUboot is the error
   1. Build without and with MCUboot. Does it work without?
2. Compare the following files between Xiao and nRF52840DK:
   1. build/mcuboot/zephyr/zephyr.dts
   2. build/mcuboot/zephyr/.config
   3. Feel free to send me those files as well, and I can have a look.
3. When comparing above, we can expect some differences based on the boards. But keep a lookout for differing configs that are not board specific.
   1. A common thing is that some boards have logging configs in board config, such as enabling RTT, USB or UART or other stuff. And those are not nessecarily useful for MCUboot.
   2. I can also mention that MCUboot is known to be a bit unstable with RTT.
4. Try to fix things you found and see what happens

If those steps does not work, I have the following questions for another potential path:

FinnBiggs said:

a JLink probe to recover back into its bootloader mode.

What is the "bootloader mode" here?

FinnBiggs said:

I'm extremely grateful for all the help you've provided on these last few responses. Can I send a cake or a box of pastries to your office as a thank you?

Thanks for the offer, but I decline.
Still, it warms my heart to be able to help and to have thankful customers! It is what makes this job fun!