Mass storage initialization error with nsib and mcuboot

RE: Mass storage initialization error with nsib and mcuboot

Hieu — Wed, 08 Nov 2023 23:18:53 GMT

With things like this, I think it is reasonable for you to move forward with a cherry-picking version of NCS v2.3.0. To sync up, are there any issues left with that setup?

Next, just for the purpose of providing information, and not suggesting a different direction, I want to explain about the support of FAT FS with Partition Manager in NCS v2.5.0.

Sigvartmh's own PR 38 contains experimental support to automatically create a FAT FS partition. In NCS v2.5.0, without that patch, you will still be able to setup a FAT FS partition manually using a static Partition Manager configuration YML.

While the sample code in PR 11802 doesn't work. I can get FAT FS working with Partition Manager just fine using a slightly different pm_static.yml file like this:
(note the affiliation attribute of the storage partition)

app:
  address: 0x00000000
  end_address: 0x000f0000
  region: flash_primary
  size: 0x000f0000
storage:
  address: 0x000f0000
  end_address: 0x00100000
  region: flash_primary
  size: 0x00010000
  
  affiliation: 
    - disk
  extra_params: {
    disk_name: "SD",
    disk_cache_size: 4096,
    disk_sector_size: 512,
    disk_read_only: 0
  }

I have checked with the author, and the issue is registered and in investigation. However, right now, the feature works as above.

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Fri, 03 Nov 2023 09:17:34 GMT

Okay.

[quote userid="9456" url="~/f/nordic-q-a/101687/mass-storage-initialization-error-with-nsib-and-mcuboot/453891"]The pull request you linked seems internal to Sigvartmh's repository. The support for flash disk API should have been completed in NCS v2.5.0.[/quote]

Without this patch, I am getting code build error.

[quote userid="9456" url="~/f/nordic-q-a/101687/mass-storage-initialization-error-with-nsib-and-mcuboot/453891"]I see that the change to swap_misc.c was provided by Sigvartmh below. I believe Sigvartmh is investigating whether the change has to be made permanently or not, so we cannot confirm whether it will be merged in NCS yet.[/quote]

Please let me know, if you get any solution for the same.

RE: Mass storage initialization error with nsib and mcuboot

Hieu — Fri, 03 Nov 2023 08:16:26 GMT

Hi Mithi,

If the project is in its final development stage, then I cannot recommend changing SDK. Your project will need to weigh the cost and benefits.

The pull request you linked seems internal to Sigvartmh's repository. The support for flash disk API should have been completed in NCS v2.5.0.

Where did you get that change in swap_misc.c from, and why do you think you need to apply it to NCS v2.5.0?

I see that the change to swap_misc.c was provided by Sigvartmh below. I believe Sigvartmh is investigating whether the change has to be made permanently or not, so we cannot confirm whether it will be merged in NCS yet.

Also, as Priyanka has said earlier, on DevZone, we also cannot comment on which NCS release will contain which change.

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Fri, 03 Nov 2023 05:06:23 GMT

Hi Sigvartmh/Hieu,

The project is in its final development stage.

I downloaded the NCS v2.5.0, but it requires at least one cherry-pick:

https://github.com/sigvartmh/fw-nrfconnect-nrf-1/pull/38/.

I also need to change swap_misc.c:

Can I expect these two changes in the upcoming NCS release?

RE: Mass storage initialization error with nsib and mcuboot

Hieu — Thu, 02 Nov 2023 18:25:18 GMT

Hi Mithi,

I must remind you that if the question is not directly related to the main topic of this case (FAT FS with Partition Manager in NCS v2.3.0), you should have opened a new question. I will give a quick answer, but please open a new post for follow-up, if it is unrelated to the main topic.

When the bootloader is updated:

During the first reboot, the "new MCUboot" is still in the secondary slot, where it was downloaded. Then the "old MCUboot" swap the "new MCUboot" to the correct slot.
During the second reboot, the "new MCUboot" is in the correct slot, and NSIB boots it.

When the application is updated, MCUboot can swap the application into the correct slot, and boot it right away.

By the way, what stage is your project at? NCS v2.5.0 has been released, where the two-stage bootloader support for nRF5340 is completed. You can use it and don't have to worry about any cherry picking.

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Thu, 02 Nov 2023 17:08:36 GMT

Thank you for the clarification.

For the mcuboot update we need an extra reboot the device, how can we know that we need to reboot the device again to change the slot for mcuboot?

RE: Mass storage initialization error with nsib and mcuboot

sigvartmh — Thu, 02 Nov 2023 15:58:56 GMT

You cannot just write the zip file to Flash, unfortunately, as MCUBoot does not support this. Also, MCUBoot has no system to receive both updates in the secondary slot and select the correct one.

I think you have misinterpreted the documentation:

>This file can be used by FOTA servers (for example, nRF Cloud) to serve both s0 and s1 to the device. The device can then select the firmware file for the slot that is currently not in use.

This tries to explain that the device needs to request either S0 or S1 from the server, depending on which slot is active. So you cannot just provide it with both (in Flash), and it will select the correct one. Your FOTA solution could provide both files and then the device could select the correct one based on the active slot.

As an example, the nRF Cloud server uses the manifest.json file to provide the device with the correct file according to what it requests.

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Thu, 02 Nov 2023 15:23:47 GMT

Thanks, Sigvartmh, Now I am able to update the mcuboot.

We need separated binary files for updating slot S0 and S1, but I prefer not to monitor these for FOTA updates.

I found a solution for the same:

But When I am writing "dfu_mcuboot.zip" file to flash, I don't see any updates.

Please help me, do I need to write any parser for this or It will write same as signed_by_mcuboot_and_b0_s0_image_update.bin and signed_by_mcuboot_and_b0_s1_image_update.bin file.

RE: Mass storage initialization error with nsib and mcuboot

sigvartmh — Wed, 01 Nov 2023 15:24:25 GMT

Seems it's trying to do a trailer erase out of bounds within the storage space of the new bootloader.

I assume you tried to investigate some more with your configuration? If not here is one with a larger FS you can still change the PM_static file just make sure you align the sizes.

devzone.nordicsemi.com/.../nsib_5F00_mcuboot_5F00_with_5F00_log_5F00_larger_5F00_fs.zip

I was able to get it working with scratch at least if you apply this patch to mcuboot

diff --git a/boot/bootutil/src/swap_misc.c b/boot/bootutil/src/swap_misc.c
index 733a3974..39e89582 100644
--- a/boot/bootutil/src/swap_misc.c
+++ b/boot/bootutil/src/swap_misc.c
@@ -66,15 +66,17 @@ swap_erase_trailer_sectors(const struct boot_loader_state *state,
     sector = boot_img_num_sectors(state, slot) - 1;
     trailer_sz = boot_trailer_sz(BOOT_WRITE_SZ(state));
     total_sz = 0;
-    do {
-        sz = boot_img_sector_size(state, slot, sector);
-        off = boot_img_sector_off(state, slot, sector);
-        rc = boot_erase_region(fap, off, sz);
-        assert(rc == 0);
-
-        sector--;
-        total_sz += sz;
-    } while (total_sz < trailer_sz);
+    if(!(fa_id_primary == PM_S1_ID || fa_id_primary == PM_S0_ID)) {
+	    do {
+		    sz = boot_img_sector_size(state, slot, sector);
+		    off = boot_img_sector_off(state, slot, sector);
+		    rc = boot_erase_region(fap, off, sz);
+		    assert(rc == 0);
+
+		    sector--;
+		    total_sz += sz;
+	    } while (total_sz < trailer_sz);
+    }

     return rc;
 }

If you want the PM with scratch are I have it here:

EMPTY_0:
  address: 0x19200
  end_address: 0x1a000
  placement:
    before:
    - s1_pad
  region: flash_primary
  size: 0xe00
EMPTY_1:
  address: 0x2a200
  end_address: 0x2b000
  placement:
    before:
    - mcuboot_pad
  region: flash_primary
  size: 0xe00
app:
  address: 0x2b200
  end_address: 0xfa000
  region: flash_primary
  size: 0xcee00
app_image:
  address: 0x2b200
  end_address: 0xfa000
  orig_span: &id001
  - app
  region: flash_primary
  size: 0xcee00
  span: *id001
b0:
  address: 0x0
  end_address: 0x8000
  placement:
    after:
    - start
  region: flash_primary
  size: 0x8000
b0_container:
  address: 0x0
  end_address: 0x9000
  orig_span: &id002
  - b0
  - provision
  region: flash_primary
  size: 0x9000
  span: *id002
fatfs_storage:
  address: 0xb1000
  affiliation:
  - disk
  device: DT_CHOSEN(nordic_pm_ext_flash)
  extra_params:
    disk_cache_size: 0x1000
    disk_name: NAND
    disk_read_only: 0x0
    disk_sector_size: 0x200
  placement:
    align:
      start: 0x1000
    before:
    - tfm_storage
    - end
  region: external_flash
  size: 0x74f000
mcuboot:
  address: 0x9200
  end_address: 0x19200
  placement:
    before:
    - mcuboot_primary
  region: flash_primary
  sharers: 0x1
  size: 0x10000
mcuboot_pad:
  address: 0x2b000
  end_address: 0x2b200
  placement:
    align:
      start: 0x1000
    before:
    - mcuboot_primary_app
  region: flash_primary
  sharers: 0x2
  size: 0x200
mcuboot_primary:
  address: 0x2b000
  end_address: 0xfa000
  orig_span: &id003
  - app
  - mcuboot_pad
  region: flash_primary
  size: 0xb1000
  span: *id003
mcuboot_primary_app:
  address: 0x2b200
  end_address: 0xfa000
  orig_span: &id004
  - app
  region: flash_primary
  size: 0xcee00
  span: *id004
mcuboot_secondary:
  address: 0x0
  device: DT_CHOSEN(nordic_pm_ext_flash)
  placement:
    align:
      start: 0x4
  region: external_flash
  share_size:
  - mcuboot_primary
  size: 0xb1000
nvs_storage:
  address: 0xfa000
  end_address: 0x100000
  placement:
    before:
    - end
  region: flash_primary
  size: 0x6000
provision:
  address: 0x8000
  end_address: 0x9000
  placement:
    after:
    - b0
    align:
      start: 0x1000
  region: flash_primary
  size: 0x1000
s0:
  address: 0x9000
  end_address: 0x19200
  orig_span: &id005
  - mcuboot
  - s0_pad
  region: flash_primary
  size: 0x10200
  span: *id005
s0_image:
  address: 0x9200
  end_address: 0x19200
  orig_span: &id006
  - mcuboot
  region: flash_primary
  size: 0x10000
  span: *id006
s0_pad:
  address: 0x9000
  end_address: 0x9200
  placement:
    after:
    - b0_container
    align:
      start: 0x1000
  region: flash_primary
  share_size:
  - mcuboot_pad
  size: 0x200
s1:
  address: 0x1a000
  end_address: 0x2a200
  orig_span: &id007
  - s1_pad
  - s1_image
  region: flash_primary
  size: 0x10200
  span: *id007
s1_image:
  address: 0x1a200
  end_address: 0x2a200
  placement:
    after:
    - s1_pad
    - s0
  region: flash_primary
  share_size:
  - mcuboot
  size: 0x10000
s1_pad:
  address: 0x1a000
  end_address: 0x1a200
  placement:
    after:
    - s0
    align:
      start: 0x1000
  region: flash_primary
  share_size:
  - mcuboot_pad
  size: 0x200
mcuboot_scratch:
  address: 0xdc000
  end_address: 0xfa000
  placement:
    after:
    - app
    align:
      start: 0x1000
  region: flash_primary
  size: 0x1e000
sram_primary:
  address: 0x20000000
  end_address: 0x20040000
  region: sram_primary
  size: 0x40000

I think the patch should work with MCUBoot using move however I haven't tested this.

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Fri, 27 Oct 2023 05:11:21 GMT

Hi Sigvartmh,

Did you get a chance to investigate the issue?

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Mon, 16 Oct 2023 06:42:20 GMT

Hi,

I tested the nsib_mcuboot_mass_1_modified.zip project by disabling the assert which you mentioned but could not get success.

When I tried nsib_mcuboot_with_log.zip project, the host displayed a mass storage disk size of approx. 45 KB, preventing me to pasting the binary file into the disk because of size.

I am expecting that, after allocating required size to the mcuboot secondary partition, all flash memory will be available for the file system i.e. approx. 7MB.

RE: Mass storage initialization error with nsib and mcuboot

sigvartmh — Tue, 10 Oct 2023 14:04:42 GMT

I was able to do it but I see that I disabled asserts which is why you probably are experiencing the same error as I encountered trying to look into your issues:

I: Starting swap using move algorithm.
assertion "rc == 0" failed: file "WEST_TOPDIR/bootloader/mcuboot/boot/bootutil/src/swap_move.c", line 367, function: boot_move_sector_up
and
assertion "rc == 0" failed: file "WEST_TOPDIR/bootloader/mcuboot/boot/bootutil/src/swap_misc.c", line 73, function: swap_erase_trailer_sectors
abort()

devzone.nordicsemi.com/.../nsib_5F00_mcuboot_5F00_with_5F00_log.zip

removing the asserts will allow it to swap at least once(haven't tested more) but it's not ideal.

attached you can find a project wtih log enabled but figuring out why this asserts would require some more investigations.

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Sun, 08 Oct 2023 10:30:16 GMT

My objective is to update mcuboot and application by binary file stored in external flash. I want to use external flash in two partition 1- for file system and 2- for mcuboot secondary partition.

I tried your modified project code but not able to update bootloader but able to update application.

To debug the failure on updating MCUBOOT, I enabled the log of mcuboot, but encountered a flash overflow error.

Are you able to update MCUBOOT?, if so please let me know which SDK you are using(ncs installed by toolchain manger or installed the main sdk by west) and which cherry-pick you have selected?

RE: Mass storage initialization error with nsib and mcuboot

sigvartmh — Fri, 06 Oct 2023 12:59:16 GMT

devzone.nordicsemi.com/.../nsib_5F00_mcuboot_5F00_mass_5F00_1_5F00_modified.zip

Here is a modified version of your project that can do MCUBoot and Application updates. It features a much smaller MCUBoot without logging.

The problem is that I don't know what you intended with your setup, so I kept it as close to what you initially setup as possible, but I had to modify MCUBoot to fit the alignment requirements. This is not necessarily needed, as you could have also made it bigger, but it depends on your project and what you intend to do.

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Thu, 05 Oct 2023 04:51:41 GMT

Could you please fix this issue in pm_static.yml file for both application and bootloader upgrade, and please share the updated file.

RE: Mass storage initialization error with nsib and mcuboot

sigvartmh — Wed, 04 Oct 2023 15:58:48 GMT

*** Booting nRF Connect SDK v2.4.99-dev2-68-g386b811dfb17 ***
Attempting to boot slot 0.
Attempting to boot from address 0x9200.
Verifying signature against key 0.
Hash: 0x8f...3b
Firmware signature verified.
Firmware version 11
Booting (0x9200).
*** Booting nRF Connect SDK v2.4.99-dev2-68-g386b811dfb17 ***
[00:00:00.250,854] <inf> mcuboot: Starting bootloader
[00:00:00.252,868] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.253,326] <inf> mcuboot: Secondary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.253,356] <inf> mcuboot: Boot source: none
[00:00:00.255,340] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.255,798] <inf> mcuboot: Secondary image: magic=good, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.255,828] <inf> mcuboot: Boot source: none
[00:00:00.256,561] <inf> mcuboot: Image index: 1, Swap type: test
[00:00:00.409,942] <inf> mcuboot: Starting swap using move algorithm.
[00:00:00.409,973] <wrn> mcuboot: Not enough free space to run swap upgrade
[00:00:00.409,973] <wrn> mcuboot: required 69632 bytes but only 65536 are available
*** Booting nRF Connect SDK v2.4.99-dev2-68-g386b811dfb17 ***
[00:00:00.000,518] <inf> flashdisk: Initialize device NAND
[00:00:00.000,518] <inf> flashdisk: offset 0, sector size 512, page size 4096, volume size 7516160
[00:00:00.004,608] <wrn> main: Image build time: Oct  4 2023 13:39:34
[00:00:00.004,669] <inf> main: Set up button at gpio@50000000 pin 11

So I tired to run your setup on a newer version of MCUBoot which gives this error message which probably means that the swap before happened with a too small space meaning it copies some wrong data into the flash resulting in an invalid image.

see:

[00:00:00.409,973] <wrn> mcuboot: Not enough free space to run swap upgrade
[00:00:00.409,973] <wrn> mcuboot: required 69632 bytes but only 65536 are available

So using MCUBoot with Swap using Move requires empty 1 free flash page(4k) at the end of the image for swapping. So what needs to be fixed is your pm_static.yml file. Details on the algorithm here: https://www.youtube.com/watch?v=YgILPGzCdxc

more info with regards to the flash block here: Partition Manager not using flash efficientlyy

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Tue, 19 Sep 2023 13:05:29 GMT

Hi Sigvartmh,

I've uploaded the sample code, please look into this, if i am doing any wrong configuration:

devzone.nordicsemi.com/.../nsib_5F00_mcuboot_5F00_mass_5F00_1.zip

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Mon, 18 Sep 2023 04:49:08 GMT

Hi Sigvartmh,

I’ve installed the ncsV2.4.2 by the Toolchain manger and cherry-pick the patch in nrf directory by the following way:

a. For https://github.com/nrfconnect/sdk-nrf/pull/11802/ :

git cherry-pick ea4d4c4fafd07b20ba9c7a1a6bb284cbeeb87c63
git cherry-pick d1f11e92a9de0eb6d5d6ec40cf7715b56b79f45f

b. For https://github.com/sigvartmh/fw-nrfconnect-nrf-1/pull/38/ :

git remote add upstream https://github.com/sigvartmh/fw-nrfconnect-nrf-1
git fetch upstream
git cherry-pick 42c15a1adf1f1f4a0a620721b9b08ed8f756b9e5

After building and running the code(include NSIB, MCUBOOT and MSD), got the same error as before merging patch:

Now I removed the ncsV2.4.2 and installed the latest nRF connect SDK manually by entering the flowing commands:

west init -m https://github.com/nrfconnect/sdk-nrf --mr main

2- west update

In the latest SDK, the patch from https://github.com/nrfconnect/sdk-nrf/pull/11802/ has already been merged, So I only required to add patch from https://github.com/sigvartmh/fw-nrfconnect-nrf-1/pull/38/ .

After building and running the code, the above encountered flash disk related error has gone.

I’m now able to update the application but encountered an error of "Unable to find bootable image" when trying to update the bootloader.

When I attempted to verify this issue using the SMP server sample, encountered the same error also while trying to update the bootloader by mcumgr:

The issue I am facing is, after writing the bootloader image, named signed_by_mcuboot_and_b0_s1_image_update.bin, and rebooting the device, device swap the image and directly jumps to S1, gives error "Unable to find bootable image" and device stop here.

Ideally, it should initially jump to the application after swap, reboot the device again to load the new version of MCUBOOT and jump to application.

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Thu, 14 Sep 2023 06:16:01 GMT

Hi Priyanka,

As this is merged: https://github.com/nrfconnect/sdk-nrf/pull/11802/ it will be resolved in the NCS 2.5 release. However if you want to start testing it out you can cherry-pick this patch.

As Sigvartmh suggested me for the cherry-pick the patch, So i think he/she will reply in this ticket, if not so then i'll create new ticket for the same.

RE: Mass storage initialization error with nsib and mcuboot

Priyanka — Thu, 14 Sep 2023 06:06:39 GMT

Hi Mithilesh,

I would suggest you to create a new ticket for the same, as it is a bit different from the topic of this case, and also it will help future seekers fins answers better.

Regards,

Priyanka

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Wed, 13 Sep 2023 17:56:07 GMT

Hi,

I've installed ncs v2.4.1 by toolchain manager, I am trying to cherry-pick the patch in nrf directory, but getting the following error:

C:\ncs_v2.4.1\v2.4.1\nrf>git cherry-pick 2d5f2cb4718bc9e84c449fecd6f99058f57b3638
fatal: bad object 2d5f2cb4718bc9e84c449fecd6f99058f57b3638

Could you please guide me on how to cherry-pick this patch and specify the directory?

RE: Mass storage initialization error with nsib and mcuboot

Priyanka — Fri, 25 Aug 2023 12:27:13 GMT

We cannot comment about the roadmaps. But you could contact your Regional Sales Manager (RSM) in order to know about this.

-Priyanka

RE: Mass storage initialization error with nsib and mcuboot

Mithilesh Yadav — Fri, 25 Aug 2023 05:35:01 GMT

Thank You! I will check it once I get time. Does it work with LittleFs?

When is nordic planning to release the NCS v2.5?

RE: Mass storage initialization error with nsib and mcuboot

sigvartmh — Thu, 24 Aug 2023 08:45:50 GMT

As this is merged: https://github.com/nrfconnect/sdk-nrf/pull/11802/ it will be resolved in the NCS 2.5 release. However if you want to start testing it out you can cherry-pick this patch.

I made a prototype implementation for how you could potentially integrate parts of this into the build system see:
https://github.com/sigvartmh/fw-nrfconnect-nrf-1/pull/38/

and here is the diff to the MSC sample in zephyr:

diff --git a/samples/subsys/usb/mass/prj.conf b/samples/subsys/usb/mass/prj.conf
index 0954afcd0e..a8bc60848d 100644
--- a/samples/subsys/usb/mass/prj.conf
+++ b/samples/subsys/usb/mass/prj.conf
@@ -12,3 +12,4 @@ CONFIG_USB_MASS_STORAGE_LOG_LEVEL_ERR=y
 CONFIG_USB_DEVICE_INITIALIZE_AT_BOOT=n

 CONFIG_MAIN_STACK_SIZE=1536
+CONFIG_PM_PARTITION_FATFS_DISK_NAME="NAND"
diff --git a/samples/subsys/usb/mass/src/main.c b/samples/subsys/usb/mass/src/main.c
index 0d8d6ba982..b40681ef76 100644
--- a/samples/subsys/usb/mass/src/main.c
+++ b/samples/subsys/usb/mass/src/main.c
@@ -28,8 +28,8 @@ LOG_MODULE_REGISTER(main);
 FS_LITTLEFS_DECLARE_DEFAULT_CONFIG(storage);
 #endif

-#define STORAGE_PARTITION              storage_partition
-#define STORAGE_PARTITION_ID           FIXED_PARTITION_ID(STORAGE_PARTITION)
+#define STORAGE_PARTITION              fatfs_storage
+#define STORAGE_PARTITION_ID           FIXED_PARTITION_ID(FATFS_STORAGE)

 static struct fs_mount_t fs_mnt;

RE: Mass storage initialization error with nsib and mcuboot

Priyanka — Wed, 19 Jul 2023 06:12:26 GMT

Hi Mithilesh,

Hieu is on vacation and I am looking into the case for the time being. I have inquired your request internally and will get back to you as soon as I get a response. Please expect a slight delay as we are currently short staffed because of the summer holidays here in Norway.

Thank you very much for your patience and understanding.

-Priyanka