USB MASS sample and partition manager build issues

RE: USB MASS sample and partition manager build issues

BrianW — Wed, 17 Jul 2024 13:43:44 GMT

[quote userid="106736" url="~/f/nordic-q-a/113000/usb-mass-sample-and-partition-manager-build-issues/494384"]s it okay for you if I mark your last reply as Verified and close the ticket?[/quote]

Sure.

A final couple of notes:

- if you change the size of the partition _after_ FAT FS has formatted it, it won't be taken into account and you need to re-format it (I did this by explicitly asking the flash driver to erase the partition at boot, and having

CONFIG_FS_FATFS_MOUNT_MKFS=y

(so that FAT FS rebuilt it after.that.

It might also be possible to ask the windows PC to format it - YMMV..

- probably useful to enable 'long file names'

CONFIG_FS_FATFS_LFN=y

as the old 8.3 format is very nasty...

I couldn't find the KConfig flag however to enable 'relative directory navigation' (FF_FS_RPATH >= 1, as per this page: http://elm-chan.org/fsw/ff/doc/filename.html#:~:text=In%20the%20generic%20FAT%20filesystems,characters%20%5Cx80%20to%20%5CxFF.) which would have been nice...

RE: USB MASS sample and partition manager build issues

Sigurd Hellesvik — Wed, 17 Jul 2024 13:19:04 GMT

Good job figuring this out!
And thanks a lot for summarizing, it will help future people for sure!

Is it okay for you if I mark your last reply as Verified and close the ticket?

RE: USB MASS sample and partition manager build issues

BrianW — Tue, 16 Jul 2024 08:08:48 GMT

[quote userid="134465" url="~/f/nordic-q-a/113000/usb-mass-sample-and-partition-manager-build-issues/493980"]

2/ USB mass storage driver finds the flash partition using the KConfig line in prj.conf:

CONFIG_MASS_STORAGE_DISK_NAME="EXT_FAT_FS"

as defined in the pm _static.yml extra_params section:

disk_name: "EXT_FAT_FS",

But I can't now work out why I get the EIO error when mounting the FS?

[/quote]

Solved - turns out that although the USB mass driver can find the flash partition, ELM FAT FS requires that the partition 'disk_name' must also be in its restricted list of volume names!

It would have been nice had this been documented somewhere, eg in comments in the 'mass' sample code?

Summary of what I've learned:

for nrf5340, PartitionManager is (currently) required if using 'multi-image' eg a networking stack. When using PM:

- you can add add specific partitions using a 'pm_static.yml' file in the root dir of the application

- DTS partition definitions are not required (ignored?)

For external flash type FAT FS ELM:

- in pm_static.yml, ensure affiliation attribute is a list with 'disk' eg

fatfs_storage:

region: external_flash

affiliation:

- disk

extra_params: {

disk_name: "NAND",

disk_cache_size: 4096,

disk_sector_size: 512,

disk_read_only: 0

}

placement:

after: [start]

align: {start: 4096}

# 2Mb size, external flash is 8Mb total

address: 0x0

size: 0x200000

(note: I called the partition "fatfs_storage", so in the code to find its partition id (which is in the generated file buiid/zephyr/include/generated/pm_config.h) the defines are:

#define STORAGE_PARTITION fatfs_storage

#define STORAGE_PARTITION_ID FIXED_PARTITION_ID(STORAGE_PARTITION)

- in the DTS overlay, you need to map the flash driver with the chosen for the nordic,pm-ext-flash set to the correct driver node eg:

/ {

chosen {

nordic,pm-ext-flash = &mx25r64;

};

- have fs_mount use mount point "/NAND:", KConfig has CONFIG_MASS_STORAGE_DISK_NAME="NAND" and the pm_static.yml has disk_name="NAND" or it won't mount... (or one of the other volume name strings from list in fs.h)

RE: USB MASS sample and partition manager build issues

BrianW — Mon, 15 Jul 2024 17:44:12 GMT

Some proress;

1/ ELM FAT can only mount o certain volume names -> can' t use "/flash"

Changed to "/NAND:", now I get a EIO error

00:00:00.367,767] <inf> base: Flash USB mass storage opened : area 1 at 0x0 on mx25r6435f@0 for 2097152 bytes

[00:00:00.378,295] <err> fs: fs mount error (-5)
[00:00:00.383,575] <err> base: Failed to mount filesystem on /NAND:

2/ USB mass storage driver finds the flash partition using the KConfig line in prj.conf:

CONFIG_MASS_STORAGE_DISK_NAME="EXT_FAT_FS"

as defined in the pm _static.yml extra_params section:

disk_name: "EXT_FAT_FS",

But I can't now work out why I get the EIO error when mounting the FS?

RE: USB MASS sample and partition manager build issues

BrianW — Mon, 15 Jul 2024 15:43:10 GMT

[quote userid="106736" url="~/f/nordic-q-a/113000/usb-mass-sample-and-partition-manager-build-issues/493930"]This means that it works, right?[/quote]

Ah, it compiles! Not quite the same thing....

When run, I get this logs/error:

Area 1 at 0x0 on mx25r6435f@0 for 2097152 bytes
[00:00:00.359,222] <err> fs: fs mount error (-19)
[00:00:00.364,624] <err> base: Failed to mount filesystem

which comes from this code (pretty much directly from the mass sample code)

#define STORAGE_PARTITION fatfs_storage

#define STORAGE_PARTITION_ID FIXED_PARTITION_ID(STORAGE_PARTITION)

static struct fs_mount_t fs_mnt;

static int _setup_flash(struct fs_mount_t *mnt)

{

int rc = 0;

#if CONFIG_DISK_DRIVER_FLASH

unsigned int id;

const struct flash_area *pfa;

mnt->storage_dev = (void *)STORAGE_PARTITION_ID;

id = STORAGE_PARTITION_ID;

rc = flash_area_open(id, &pfa);

printk("Area %u at 0x%x on %s for %u bytes\n",

id, (unsigned int)pfa->fa_off, pfa->fa_dev->name,

(unsigned int)pfa->fa_size);

if (rc < 0 && IS_ENABLED(CONFIG_APP_WIPE_STORAGE)) {

printk("Erasing flash area ... ");

rc = flash_area_erase(pfa, 0, pfa->fa_size);

printk("%d\n", rc);

}

if (rc < 0) {

flash_area_close(pfa);

}

#endif

return rc;

}

static int _mount_app_fs(struct fs_mount_t *mnt)

{

int rc;

#if CONFIG_FAT_FILESYSTEM_ELM

static FATFS fat_fs;

mnt->type = FS_FATFS;

mnt->fs_data = &fat_fs;

mnt->mnt_point = "/flash:";

#else

log_warn("no file system type defined in firmware build, no FS mount available.");

return -1;

#endif

rc = fs_mount(mnt);

return rc;

}

static void _setup_disk(void)

{

struct fs_mount_t *mp = &fs_mnt;

struct fs_dir_t dir;

struct fs_statvfs sbuf;

int rc;

fs_dir_t_init(&dir);

if (IS_ENABLED(CONFIG_DISK_DRIVER_FLASH)) {

rc = _setup_flash(mp);

if (rc < 0) {

LOG_ERR("Failed to setup flash area");

return;

}

if (!IS_ENABLED(CONFIG_FAT_FILESYSTEM_ELM)) {

LOG_INF("No file system selected");

return;

}

rc = _mount_app_fs(mp);

if (rc < 0) {

LOG_ERR("Failed to mount filesystem");

return;

}

.... etc

So it finds the partition, and the characteristics seem right. but the mount fails.

The run continues, including:

[00:00:00.370,727] <inf> base: USB mass storage mode enabled.

and from the PC I see the USB mass storage as a new drive, and can create/read files on it...

So why does the FS mount fail?

I also don't quite see how the USB mass driver "knows" which partition to expose to the USB? How is this link made? is it looking for a specific partition name or a label (the DTS config from 'mass' had an overlay config for an msc_disk0 node liek this:

msc_disk0 {

compatible = "zephyr,flash-disk";

partition = <&storage_partition>;

disk-name = "NAND";

cache-size = <4096>;

};

&mx25r64 {

partitions {

compatible = "fixed-partitions";

#address-cells = <1>;

#size-cells = <1>;

storage_partition: partition@0 {

label = "storage";

reg = <0x00000000 0x00020000>;

};

But i have that commented out right now as the flashdisk.c code with PM enabled uses the PM config to find the disk partition...maybe the USB exposed file system is not actually on the fatfs_storage partition but actually using a different partition??

Any pointers? thanks!

RE: USB MASS sample and partition manager build issues

Sigurd Hellesvik — Mon, 15 Jul 2024 13:41:29 GMT

[quote user="BrianW"]and everything compiles as it should including the generated partition id for fatfs_storage.[/quote]

This means that it works, right?
If so, great news!

RE: USB MASS sample and partition manager build issues

BrianW — Mon, 15 Jul 2024 12:56:07 GMT

I updated my yml file (called pm_ext_fs.yml) with the fix from that report (make affiliation a list)

fatfs_storage:

region: external_flash

affiliation:

- disk

extra_params: {

disk_name: "NAND",

disk_cache_size: 4096,

disk_sector_size: 512,

disk_read_only: 0

}

placement:

after: [start]

align: {start: 4096}

# 2Mb size, external flash is 8Mb total

size: 0x200000

To get PM to load it, I added to CMakeList.txt in root of application:

if(CONFIG_PARTITION_MANAGER_ENABLED)

ncs_add_partition_manager_config(pm_ext_fatfs.yml)

message( INFO " Added pm_ext_fatfs.yml to partition manager...")

endif()

This passes and I get the log message, but the file contents seem to be ignored by PM : no effect on the generated pm_config.h / Pm.config or partitions.yml files -> still no success.

Next, I tried renaming it to pm_static.yml in root dir. (found this piece of magic in that bug report also)

This time the compilation includes the log "-- Found partition manager static configuration: C:/work/dev/if-device-nrf53/cc1-med/pm_static.yml".But it then bombs because it requires an 'address' field....(I tried with the placement before: [end], same result)

Adding

address: 0x0

and everything compiles as it should including the generated partition id for fatfs_storage.

RE: USB MASS sample and partition manager build issues

Sigurd Hellesvik — Mon, 15 Jul 2024 10:53:44 GMT

Reported and explained in:

Bug in partition_manager_output.py leading to incorrect PM_FOREACH_AFFILIATED_TO_disk

RE: USB MASS sample and partition manager build issues

BrianW — Mon, 15 Jul 2024 09:46:13 GMT

[quote userid="107683" url="~/f/nordic-q-a/113000/usb-mass-sample-and-partition-manager-build-issues/493833"]Instead, lets take a step back and check something that should be verified support for: Could you try to test with APP_MSCC_STORAGE_FLASH_LITTLEFS instead? This [/quote]

OK. the 'mass' sample builds ok with

west build --build-dir mass/build mass --pristine -DCONFIG_APP_MSC_STORAGE_FLASH_LITTLEFS=y

also with

west build --build-dir mass/build mass --pristine -DCONFIG_APP_MSC_STORAGE_FLASH_FATFS=y

BUT not with:

west build --build-dir mass/build mass --pristine -DCONFIG_APP_MSC_STORAGE_FLASH_LITTLEFS=y -DCONFIG_PM_SINGLE_IMAGE=y

ie it does NOT work if the partition manager is enabled (which is a must when building for nRF5340 and using bluetooth....)

So its not a question of using FAT or LittleFS file systems. (btw, FATFS is required as otherwise the PC attached to the USB will not be able to access the FS...)

For me, the problem is because flashdisk.c in the core zephyr code needs to find a flash partition with the 'disk' affiliation, and this means the PM requires a pm.yml file to define this, yes?

SO: how do I get PM to pick up my pm.yml file? Putting it in the root dir doesn't work. Putting it in 'child_image' sub folder doesn't work.

Can you try building the mass sample with PM enabled for a nrf5340DK, using the external flash for the USB mass storage and tell me what is required to get it to build?

thanks

RE: USB MASS sample and partition manager build issues

AHaug — Mon, 15 Jul 2024 09:11:47 GMT

Hi,

I understand that this is complex and there is quite a few configuration options to navigate through..

A couple of thoughts here:

APP_MSC_STORAGE_FLASH_FATFS != APP_MSC_STORAGE_FLASH_LITTLEFS, while our pm.yml.file_system concerns littlefs
Partition manager does not seem to have out of the box support for that specific FAT filesystem
As you've stated you're already attempting, you'll have to use static partitioning

Instead, lets take a step back and check something that should be verified support for: Could you try to test with APP_MSCC_STORAGE_FLASH_LITTLEFS instead? This

Kind regards,
Andreas

RE: USB MASS sample and partition manager build issues

BrianW — Mon, 15 Jul 2024 08:36:49 GMT

[quote userid="134465" url="~/f/nordic-q-a/113000/usb-mass-sample-and-partition-manager-build-issues/493659"]The PM doc includes a snippet for defining a fat-fs partition - where do I put the pm.yml file? The doc says it is ignored if its in the root application dir (which seems to be true!).[/quote]

More reading tells me that in a multi-image application on a nrf5340, the 'app' is always the 'root' application. The image for the cpu-net (Bluetooth HCI RPMsg from the sample/...) is a 'child-image', witha "child_image" subfolder and a hci_rpmsg.conf file.

But since the app, which is the bit that needs the pm.yml, is the root, and the root's pm.yml is ignored, how to build a nrf5340 bluetooth application with any partition configuration????

This all seems excessively complex.... and the documentation is a maze of twisty litle configuration options....

RE: USB MASS sample and partition manager build issues

BrianW — Fri, 12 Jul 2024 14:38:25 GMT

[quote userid="134465" url="~/f/nordic-q-a/113000/usb-mass-sample-and-partition-manager-build-issues"]

I tried to build the 'mass' sample standalone copied directly from the NCS; but it also fails to compile with this error:

CMake Error at CMakeLists.txt:10 (message):
No disk access settings detected.

Any ideas what I'm doing wrong?

[/quote]

Ok, so this error is due to not giving the -DCONFIG_APP_MSC_STORAGE_FLASH_FATFS=y option on the build line (which only works if you build from the parent directory as

west build --build-dir mass/build mass --pristine=auto -DCONFIG_APP_MSC_STORAGE_FLASH_FATFS=y

as otherwise the -D... is taken as the application name).

And this builds ok as the PM is NOT enabled - if I force the PM with

CONFIG_PM_SINGLE_IMAGE=y

then it fails to compile also....

RE: USB MASS sample and partition manager build issues

BrianW — Fri, 12 Jul 2024 14:06:44 GMT

[quote userid="9456" url="~/f/nordic-q-a/113000/usb-mass-sample-and-partition-manager-build-issues/493601"]Yes, you guessed right. You need to use the Partition Manager because you are building a multi-image application. This is the case up to nRF Connect SDK (NCS) v2.6.1.[/quote]

Ok. The partition manger doc (which I have read but haven't managed to map onto my project...) says:

Note

When you build a multi-image application using the Partition Manager, the devicetree source flash partitions are ignored.

So why do the nrf5340 DTS board board files all include partition definitions in nrf5340_cpuapp_partition_conf.dtsi, and then in nrf5340_cpuapp_common.dtsi for &flash0? are these just ignored? the build/partitions.yml file doesn't seem to include them...

For the FAT-FS in external flash then:

The PM doc includes a snippet for defining a fat-fs partition - where do I put the pm.yml file? The doc says it is ignored if its in the root application dir (which seems to be true!).

here's what I put in it:

fatfs_storage:

region: external_flash

affiliation: disk

extra_params: {

disk_name: "NAND",

disk_cache_size: 4096,

disk_sector_size: 512,

disk_read_only: 0

}

placement:

after: [start]

align: {start: 4096}

# 2Mb size, external flash is 8Mb total

size: 0x200000

But no fastfs_storage partition is in build/partitions.yml, and the compilation of flashdisk.c fails in the same way.

since I put

/ {

chosen {

nordic,pm-ext-flash = &mx25r64;

};

into my dts overlay, an "external-flash" region now appears in build/regions.yml, as well as a partition called 'external-flash' in build/partitions.yml, which uses all the region...

I feel its not far away but just stuck with the PM...

RE: USB MASS sample and partition manager build issues

Hieu — Fri, 12 Jul 2024 09:28:52 GMT

Hi BrianW,

[quote user=""]

Looking at the flashdisk.c code it seems to be in the USE_PARTITION_MANAGER case, so I guess its not finding some PM config to define the flash disk parameters?

Q: do I need to use the Partition Manager sub system? The DTS files appear to define the partitions required just fine...The mass example does not have a pm.yml either?

If I have to use PM (my reading implies 'yes', because the nrf5340 requires a multi-image hex due to the 2 processors?),

[/quote]

Yes, you guessed right. You need to use the Partition Manager because you are building a multi-image application. This is the case up to nRF Connect SDK (NCS) v2.6.1.
(From NCs v2.7.0, the build system changes quite a bit, and I honestly haven't caught up on that, so I refrain from commenting about v2.7.0 here.)

If I understand correctly, you are using FAT FS, right? This issue happens mostly with FAT FS.

[quote user=""]where, how to configure its partitions if not in the DTS?[/quote]

The Partition Manager dynamically resolve all partition requirement during compile time. You can add a requirement for the FAT FS partition.

The format of the file and how to use it are documented here: https://docs.nordicsemi.com/bundle/ncs-2.6.1/page/nrf/scripts/partition_manager/partition_manager.html

I tested in the past with NCS v2.5.0, and this worked:

app:
  address: 0x00000000
  end_address: 0x000f0000
  region: flash_primary
  size: 0x000f0000
storage:
  address: 0x000f0000
  end_address: 0x00100000
  region: flash_primary
  size: 0x00010000
  
  affiliation: 
    - disk
  extra_params: {
    disk_name: "SD",
    disk_cache_size: 4096,
    disk_sector_size: 512,
    disk_read_only: 0
  }

Alternatively, you could also provide a static configuration of the entire flash memory partitioning. The details are also included in the document page I linked above.

A quick way to get a starting point with static configurations is to copy the result of a dynamic Partition Manager run. It is located at: $BUILD_DIR/partitions.yml.

Hieu