MCUBoot DFU Interference with ExFat Filesystem

[FinnBiggs]

Hi Folks!

I've got a rather bulky project that's having a rather bulky project. In short, my project tries to enable both MCUBoot DFU and an ExFat Filesystem. I've narrowed the problem down to some kind of conflict happening between KConfig option for enabling each of these features.

In particular, CONFIG_DISK_DRIVER_FLASH, an implication by CONFIG_APP_MSC_STORAGE_FLASH_FATFS, causes a cascade of errors.

I've created a public Github repo that includes my entire project, configuration settings and files, and a readme that details how to set up my project and enable both of these features at once.

That's located here: https://github.com/FinnWBiggs/mini_desktop

I would be quite grateful for any help detangling DFU and Filesystem options.

Best,

    - Finn

[Sigurd Hellesvik]

Hi,

I've got a rather bulky project that's having a rather bulky project. In short, my project tries to enable both MCUBoot DFU and an ExFat Filesystem. I've narrowed the problem down to some kind of conflict happening between KConfig option for enabling each of these features.

You probably have thought about this, but:

Can I suggest that you try a simpler sample with only MCUboot and ExFat and see if that works?
I find that having a lot of other things in a project as well tent to complicate debugging

Regards,
Sigurd Hellesvik

[FinnBiggs]

Hi Sigurd,

I'll set up a minimized version and update here.

Update:

Here is a minimized sample causing a collision between ExFat and MCUBoot.

https://github.com/FinnWBiggs/MCUBoot_ExFat

I do get a different error than I did in mini_desktop, but that's okay. I believe that if we can find where the two tools conflict, it will tell me enough about the system that I can follow the same logic to get my project working.

When MCUBoot is disabled, my application finds my devicetree's "some_storage_partition" (whose name I changed to avoid potential naming and defines, such as in flash_map_pm).The sample runs and sets up a USB storage system as intended.

When I enable MCUBoot, I get errors regarding the definition of PM_some_storage_partition_ID, which implies that MCUBoot is somehow affecting my devicetree's storage partitioning IDs. 

How do you recommend making MCUBoot and Mass storage compatible based on this?

Thank you very much for your time,

    - Finn

[Sigurd Hellesvik]

FinnBiggs said:

I added pm.yml

No wait.
pm.yml should not be parsed by user applications. This file is specifically only used for the build system, and not to be used by applications. Applications use pm_static.yml.

FinnBiggs said:

(The root application does not need to define a pm.yml file, because its partition size and placement is implied by the size and placement of the child image partitions. If a root application defines a pm.yml file, it is silently ignored.)

Ah yes I see you already learned what I said above.

FinnBiggs said:

What's the correct way to implement a partition manager while PM_SINGLE_IMAGE is enabled?

Just configure the already enabled partition manager with pm_static.yml.

FinnBiggs said:

What do you recommend for a DFU-compatible Filesystem?

Let us fix the file system alone first and then add DFU?
Or are you considering using another file-system than the one you try to handle now altogether?
I do not think I understand the question fully.

FinnBiggs said:

What would be the right resource to go learn about these tools so I don't eat up time on DevZone?

All I got are these:
https://academy.nordicsemi.com/courses/nrf-connect-sdk-intermediate/lessons/lesson-8-bootloaders-and-dfu-fota/
https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/scripts/partition_manager/partition_manager.html
https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/config_and_build/multi_image.html

However, this mostly just requires experience, and this stuff is complicated.
I think that you can still continue to ask here on DevZone.
That being said, summer vacations will start here from next week, so we will be a bit low on staff in the following weeks.

FinnBiggs said:

Apologies for not being as helpful as I'd like to be, I'm new to this and feeling lost.

No worries at all, this is not easy, and we know it.
We should be making it easier, and I hope that we will in the future, but for now we use tech support as a way to make it easier for you guys to use. (At least that is how I see it).
I think that you are doing a great job, both in trying to learn this and in being friendly!

Alright, now let me try to explain how I would do this:

Create the following pm_static.yml:

fatfs_storage:
  address: 0xf0000
  end_address: 0x100000
  region: flash_primary
  size: 0x10000
  affiliation: disk
  extra_params: {
      disk_name: "SD",
      disk_cache_size: 4096,
      disk_sector_size: 512,
      disk_read_only: 0
  }

You may want to change the region to your external storage, and the address range to either the start or end of that region.

I also used an arbitrary size, so maybe tweak that a bit.

When you build with this, you should be able to verify that it was added to partitioning by looking at build/partitioning.yml, which contains the final partitioning for the project.

Now that the partition does have the affiliation "disk", you should hopefully no longer get the error PM_FOREACH_SOMETHING_SOMETHING_disk that I was getting when building the project.

Let me know how this goes.

[FinnBiggs]

Hi Sigurd,

Thanks for the kind words. I've made a little bit of progress here, but mostly I've been on other issues. After devoting my whole workday to this issue I've decided to come back for more help.

Sigurd Hellesvik said:

Or are you considering using another file-system than the one you try to handle now altogether?
I do not think I understand the question fully.

At this point I'm open to any solution that meets my two needs -- DFU OTA that works alongside a USB File System. If there's a way to those features that uses some different Zephyr tool set, I'd be happy to swap.

In this thread, we've boiled down the problem to some conflict between partition manager and flash.

In the ReadMe here: https://github.com/FinnWBiggs/MCUBoot_ExFat, I lay out build instructions, and the config options which lead to the error.

----------------------------------------------------------

In the process of trying to debug and come up with a minimal example that narrows the scope of this problem, I went back to the mass sample.

I believe it may be a valid minimal example add CONFIG_PM_SINGLE_IMAGE to zephyr\samples\subsys\usb\mass sample.

How can I enable the partition manager in this sample?

Thanks again for the help.

Best regards,

    - Finn

[Sigurd Hellesvik]

I have been on vacation the last two weeks, but I am back now.
Seems like this ticket has not been picket up in the meantime in the summer rush.

FinnBiggs said:

In the process of trying to debug and come up with a minimal example that narrows the scope of this problem, I went back to the mass sample.

I believe it may be a valid minimal example add CONFIG_PM_SINGLE_IMAGE to zephyr\samples\subsys\usb\mass sample.

How can I enable the partition manager in this sample?

I think this is a good plan. I will try to make this example work with the partition manager on my side.

By the way, another customer recently also asked about this in  USB MASS sample and partition manager build issues

[Sigurd Hellesvik]

I think this should solve the USB mass sample (with a workaround. There seems to be a bug in partititon manager)
Bug in partition_manager_output.py leading to incorrect PM_FOREACH_AFFILIATED_TO_disk 

Can you give it a go and let me know what you think?

[FinnBiggs]

Hi Sigurd,

Thank you so much, great find on that solution. You're right that that solves the USB sample.

In a bizarre twist, I sunk my entire workday into trying to get my GitHub repo to build and compile and ended up in a strange state where I was getting very opaque assembler errors.

[55/180] Building C object zephyr/CMakeFiles/zephyr.dir/misc/generated/configs.c.obj
FAILED: zephyr/CMakeFiles/zephyr.dir/misc/generated/configs.c.obj 
C:\ncs\toolchains\c57af46cb7\opt\zephyr-sdk\arm-zephyr-eabi\bin\arm-zephyr-eabi-gcc.exe -DKERNEL -DNRF52840_XXAA -DPICOLIBC_INTEGER_PRINTF_SCANF -DUSE_PARTITION_MANAGER=1 -D_FORTIFY_SOURCE=1 -D_POSIX_C_SOURCE=200809 -D__LINUX_ERRNO_EXTENSIONS__ -D__PROGRAM_START -D__ZEPHYR__=1 -IC:/ncs/v2.5.0/zephyr/kernel/include -IC:/ncs/v2.5.0/zephyr/arch/arm/include -IC:/ncs/v2.5.0/zephyr/include -Izephyr/include/generated -IC:/ncs/v2.5.0/zephyr/soc/arm/nordic_nrf/nrf52 -IC:/ncs/v2.5.0/zephyr/soc/arm/nordic_nrf/common/. -IC:/ncs/v2.5.0/zephyr/subsys/usb/device -IC:/ncs/v2.5.0/nrf/include -IC:/ncs/v2.5.0/nrf/tests/include -IC:/ncs/v2.5.0/modules/hal/cmsis/CMSIS/Core/Include -IC:/ncs/v2.5.0/zephyr/modules/cmsis/. -IC:/ncs/v2.5.0/modules/hal/nordic/nrfx -IC:/ncs/v2.5.0/modules/hal/nordic/nrfx/drivers/include -IC:/ncs/v2.5.0/modules/hal/nordic/nrfx/mdk -IC:/ncs/v2.5.0/zephyr/modules/hal_nordic/nrfx/. -IC:/ncs/v2.5.0/modules/debug/segger/SEGGER -IC:/ncs/v2.5.0/modules/debug/segger/Config -isystem C:/ncs/v2.5.0/nrfxlib/crypto/nrf_cc310_platform/include -fno-strict-aliasing -Os -imacros C:/Users/finnb/Documents/auli/MCUBoot_ExFat/build/zephyr/include/generated/autoconf.h -fno-printf-return-value -fno-common -g -gdwarf-4 -fdiagnostics-color=always -mcpu=cortex-m4 -mthumb -mabi=aapcs -mfp16-format=ieee --sysroot=C:/ncs/toolchains/c57af46cb7/opt/zephyr-sdk/arm-zephyr-eabi/arm-zephyr-eabi -imacros C:/ncs/v2.5.0/zephyr/include/zephyr/toolchain/zephyr_stdint.h -Wall -Wformat -Wformat-security -Wno-format-zero-length -Wno-pointer-sign -Wpointer-arith -Wexpansion-to-defined -Wno-unused-but-set-variable -Werror=implicit-int -fno-pic -fno-pie -fno-asynchronous-unwind-tables -ftls-model=local-exec -fno-reorder-functions --param=min-pagesize=0 -fno-defer-pop -fmacro-prefix-map=C:/Users/finnb/Documents/auli/MCUBoot_ExFat=CMAKE_SOURCE_DIR -fmacro-prefix-map=C:/ncs/v2.5.0/zephyr=ZEPHYR_BASE -fmacro-prefix-map=C:/ncs/v2.5.0=WEST_TOPDIR -ffunction-sections -fdata-sections --specs=picolibc.specs -std=c99 -MD -MT zephyr/CMakeFiles/zephyr.dir/misc/generated/configs.c.obj -MF zephyr\CMakeFiles\zephyr.dir\misc\generated\configs.c.obj.d -o zephyr/CMakeFiles/zephyr.dir/misc/generated/configs.c.obj -c zephyr/misc/generated/configs.c
C:\Users\finnb\AppData\Local\Temp\ccu25E1Q.s: Assembler messages:
C:\Users\finnb\AppData\Local\Temp\ccu25E1Q.s:43: Error: missing expression
[76/180] Building C object zephyr/CMakeFiles/zephyr.dir/lib/os/cbprintf_complete.c.obj
ninja: build stopped: subcommand failed

Luckily, I've been saved by a coworker, who tried this out by starting from a clean mass sample and made it simply work using your suggestions.

Here is a link to a solution. (https://github.com/FinnWBiggs/mass_pm). To any and all future readers: please feel free to do anything you like with it. . 

I've added a warning to the malfunctioning ReadMe of that repo to let future viewers know, and will leave both repos up so that future viewers of this post still have a reference.

If you look at what's in my included error or in the repo (https://github.com/FinnWBiggs/MCUBoot_ExFat) and see an obvious problem, I'd be happy to know why I'm getting assembler errors.

If it looks complicated to ID a problem, then let's just close this ticket -- I'm able to continue on my project.

Regardless, I'm very glad to have a working filesystem, and I think we can close this ticket. Thank you Sigurd, you've been amazingly helpful. I hope you had a lovely and relaxing vacation.

Best regards,

    - Finn