nRF52840 Possible Flash Damage

RE: nRF52840 Possible Flash Damage

Sigurd Hellesvik — Tue, 27 Dec 2022 09:08:44 GMT

Hi,

[quote user="DavidKaplan"]1) Does loading a new program when debugging count as an erasure?
I have used sensor#1 quite a bit in debugging.[/quote]

When I run "debug" in VS Code, it calles this command to flash the chip:

west flash -d c:\ncs\v2.2.0\zephyr\samples\hello_world\build --skip-rebuild -r nrfjprog --dev-id 1050084572

So it flashes the chip, but does not erase the flash.

[quote user="DavidKaplan"]2) I erase the slot on the sensor before starting its file transfer and do not use CONFIG_IMG_ERASE_PROGRESSIVELY=y.
Would it be better to use progressive erasures and not erase it at the start?[/quote]

From CONFIG_IMG_ERASE_PROGRESSIVELY:
"If enabled, flash is erased as necessary when receiving new firmware, instead of erasing the whole image slot at once. This is necessary on some hardware that has long erase times, to prevent long wait times at the beginning of the DFU process."

It seems that this configuration is for changing how fast the DFU goes.
I suggest that you try to time your DFU with and without this configuration, to find which setting works best for you.

[quote user="DavidKaplan"]3) The default nRF52840 block size is 512 bytes. Would it help to increase it if possible?[/quote]

I assume that you mean: "Will it help to decrease flash wear?"

The way I understand it, the scratch size is mostly important if you use swap-scratch, as it will decrease the amount of times you write to the same flash.

However, you should use swap-move, as it reduces wear on the flash.
In this case, here is from Swap without using scratch:
"The algorithm does two erase cycles on the primary slot and one on the secondary slot during each swap. Assuming that receiving a new image by the DFU application requires 1 erase cycle on the secondary slot, this should result in leveling the flash wear between the slots."

From this it looks like the swap-move will only do a set number of erases, no matter the block size.

For more information on swap alternatives, see MCUboot swap-scratch algorithm and The Swap-Move algorithm in MCUboot .

Regards,
Sigurd Hellesvik

RE: nRF52840 Possible Flash Damage

DavidKaplan — Sun, 25 Dec 2022 07:13:16 GMT

To recap things:

I previously did not define CONFIG_BOOT_SWAP_USING_MOVE=y since I was unaware of the consequences.

In my current sensor code, I added a child_image folder and copied the mcuboot's prog.conf file adding the CONFIG_BOOT_SWAP_USING_MOVE=y #define in a mcuboot.conf file.
I then programmed two sensors with the new code and then transferred the new image (with only the date changed) by my non-SMP BLE file transfer procedure.
One sensor was programmed OK and the other not.
I then connected the debugger and issued nrfjprog --readcode on both the good and bad sensors and uploaded to you the hex files along with the bootloader report.
1.hex for the first sensor (failed) and nrfjprog --readcode 2.hex for the good sensor.
The sensors will soon be in field tests and if I am doing something wrong, I need to know now.

1) Does loading a new program when debugging count as an erasure?
I have used sensor#1 quite a bit in debugging.

2) I erase the slot on the sensor before starting its file transfer and do not use CONFIG_IMG_ERASE_PROGRESSIVELY=y.
Would it be better to use progressive erasures and not erase it at the start?

Before I start the DFU, I call img_mgmt_impl_erase_slot() to erase slot1 and flash_img_init().
Then, I use img_mgmt_impl_write_image_data() to write incoming 512-byte blocks of the image.
When I have finished writing the whole image to slot 1, I call img_mgmt_read_info() and img_mgmt_impl_write_pending()
with the write pending permanent flag false. It is all in the code in my post.

3) The default nRF52840 block size is 512 bytes. Would it help to increase it if possible?
I tried once 1024 but it did not work or maybe I did not change the proper define.

Thanks David

RE: nRF52840 Possible Flash Damage

Sigurd Hellesvik — Thu, 22 Dec 2022 14:32:44 GMT

Hi,

As Andreas said, I will continue to help with this case.

To make sure I have not missed anything, can you confirm if this is your current questions?

You do want to know which swap type you use
You are concerned that you have a limited amount of DFUs available before you use up the flash.

Let me know if there was anything I have missed in the case.

Swap Type:

To figure out which swap type your project has, you can find that in VS Code after you built your project:

Which swap type does your project use?

Regards,
Sigurd Hellesvik

RE: nRF52840 Possible Flash Damage

AHaug — Thu, 22 Dec 2022 10:11:15 GMT

Hi,

Unfortunately II couldn't identify the cause to the issue before the Holidays. I'll be out of office until the 29th, but in the meanwhile a colleague of mine will handle this case. Hopefully they will have the time to look into the issue and come with suggestions, but due to the staffing throughout the holiday period it's expected that the response time will be longer until the first week of January where we're fully staffed again.

I hope this does not cause too many issues for you,

Kind regards,
Andreas

RE: nRF52840 Possible Flash Damage

DavidKaplan — Tue, 20 Dec 2022 12:01:53 GMT

Here is my.config file.

devzone.nordicsemi.com/.../2330.config.txt

My project proj.conf

devzone.nordicsemi.com/.../0285.prj.txtI have a overlay-secure_bootloader.conf file along the proj.conf

devzone.nordicsemi.com/.../overlay_2D00_secure_5F00_bootloader.txtAnd in the child_image a mcuboot.conf

devzone.nordicsemi.com/.../mcuboot.txt My boards directory is based on the generic nRF52840

devzone.nordicsemi.com/.../boards.zip

I did not see some of the defines you highlighted in my build .config file.

I just want to make sure I am not using the swap with the 256 upgrade limitation.

Thanks David

RE: nRF52840 Possible Flash Damage

AHaug — Tue, 20 Dec 2022 10:22:25 GMT

Hi,

I will give an answer to some of your questions now today so you have something to work with, and I will come back to verify the items below that needs some more input tomorrow.

[quote user="DavidKaplan"]After reading the MCUboot documentation you inked, I understand that the swap-using-scratch is the default and that it may leave us with only 256 upgrades.[/quote]

Yes, that's correct for the case where you're performing DFU using scratch. However the default is swap using move if the SoC family is nRF, I am not 100% sure that is the case in your application if you've created it from scratch, so we'll have to look closer into this. Could you:

Share your configurations (proj.conf and overlay files)?
Could you navigate to <your app>/build/mcuboot/zephyr/.config and see what is enabled or not? As a sample here, you can see that by default the sample I've chosen uses move and not scratch.

For reference, the sample where this .config is fetched from is this sample:https://github.com/hellesvik-nordic/samples_for_nrf_connect_sdk/tree/main/bootloader_samples/nrf5340/mcuboot_smp_ble_simultaneous

Swapping using move should not cause the same issues as with scratch, so I might've been too fast in suggesting that the issue lies there. However, your configuration files should reveal if scratch or move is being used.

[quote user="DavidKaplan"]2) I erase the slot on the sensor before starting its file transfer and do not use CONFIG_IMG_ERASE_PROGRESSIVELY=y. Would it be better to use progressive erasures and not erase it at the start?[/quote]

I will have to verify this,

[quote user="DavidKaplan"]

3) The default nRF52840 block size is 512 bytes. Would it help to increase it if possible?

[/quote]

As far as I know what is used is the slot size of 4k (8 blocks or 1 page) that is being section used for cycling the image. I believe it should be possible to increase this in multiples of a page size, but I need to look closer into if there exists any configs or if the 52840 has any limitations w.r.t. maximum slot sector size. As the documentation states under mcuboot: "Usual minimum size of 4k.."

Kind regards,
Andreas

RE: nRF52840 Possible Flash Damage

DavidKaplan — Mon, 19 Dec 2022 13:37:55 GMT

After reading the MCUboot documentation you inked, I understand that the swap-using-scratch is the default and that it may leave us with only 256 upgrades.
1) Does loaded a new program when debugging count?

2) I erase the slot on the sensor before starting its file transfer and do not use CONFIG_IMG_ERASE_PROGRESSIVELY=y. Would it be better to use progressive erasures and not erase it at the start?

3) The default nRF52840 block size is 512 bytes. Would it help to increase it if possible?

I added a child_image folder and copied the mcuboot's prog.conf file adding the CONFIG_BOOT_SWAP_USING_MOVE=y #define in a mcuboot.conf file.

I understand that this should drastically increase the number of possible upgrades.

The sensor has only a primary and a secondary slot, so I see no reason not to do this.

  flash_primary (0x100000 - 1024kB):
+-------------------------------------------------+
| 0x0: mcuboot (0xc000 - 48kB)                    |
+---0xc000: mcuboot_primary (0x74000 - 464kB)-----+
| 0xc000: mcuboot_pad (0x200 - 512B)              |
+---0xc200: mcuboot_primary_app (0x73e00 - 463kB)-+
| 0xc200: app (0x73e00 - 463kB)                   |
+-------------------------------------------------+
| 0x80000: mcuboot_secondary (0x74000 - 464kB)    |
| 0xf4000: littlefs_storage (0x6000 - 24kB)       |
| 0xfa000: nvs_storage (0x6000 - 24kB)            |
+-------------------------------------------------+

  sram_primary (0x40000 - 256kB):
+--------------------------------------------+
| 0x20000000: sram_primary (0x40000 - 256kB) |
+--------------------------------------------+

I FOTA DFU two of my nRF52840 sensors. One failed at the last block written or when calling img_mgmt_read_info() and img_mgmt_impl_write_pending(). The other one most always works.

I connected each sensor to my dk and issued the nrfjprog --readcode 1.hex for the first sensor (failed) and nrfjprog --readcode 2.hex for the good sensor. I compared the files and see that they are different.

devzone.nordicsemi.com/.../2548.hex

devzone.nordicsemi.com/.../2.hex

Thanks and happy holidays.

RE: nRF52840 Possible Flash Damage

AHaug — Fri, 16 Dec 2022 13:59:59 GMT

Hi,

[quote user="DavidKaplan"]What address are slot1?[/quote]

The first thing that runs starts at address 0x0. Depending on if you're using MCUboot and/or an Immutable bootloader (b0), the location will change.

If you only have the application, slot1 should be starting at 0x0. If you have MCUboot in addition to the application, then MCUboot is what starts at 0x0, and the application (primary slot) starts after. If you're also using b0, then b0 starts at 0x0, then followed by MCUboot, then followed by the application

You could get an indication of where the various partitions are located by navigating to the projects build folder and use ninja partition_manager_report to see where the slots are located.

[quote user="DavidKaplan"]It does not seem likely that even with a software DFU bug I have exceeded writing 10,000 times.[/quote]

I had a closer look in the MCUboot documentation and I found this: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/mcuboot/design.html#swap-using-scratch. If you're using the scratch algorithm when flashing, there might be some scratch specific wear when your doing upgrades which dramatically reduces the number of cycles you can update the device. That number could be more plausible than 10 000. I suggest you have a look at that documentation to see if that could be a possible scenario or not

[quote user="DavidKaplan"]I will get back to you.[/quote]

Sounds like a plan. Just a heads up, I will be working on and off for the next couple of weeks due to the christmas holidays in Norway, so the response time might be a bit longer than usual

Kind regards,
Andreas

RE: nRF52840 Possible Flash Damage

DavidKaplan — Thu, 15 Dec 2022 10:00:01 GMT

It does not seem likely that even with a software DFU bug I have exceeded writing 10,000 times.

The 3 sensor I have bricked do not respond to Seggar connect tries so any software above it also gives an error, so it may be another problem and not the flash with those units.

The sensor that gives a hard fault after I have written the entire image to Slot1 caused me to think that maybe some flash region could not be programmed. I wanted to debug the reason but was unable to switch the BLE off so it would not give timing faults when debugging. How can I prevent the BLE timing errors when debugging this issue?

I can try to read the nrfjprog to read out the flash-area. The flash areas are defaults for the nR52840.

What address are slot1?

I did not write a custom SMP server but use raw BLE transfers from our Gateway Central.

But I do have the SMP stuff enabled for phone App DUF updates.

I tried Simon's test SMP DFU code but could not get it to work and decided to try custom raw transfers without the SMP complicating things.

Before I start the DFU, I call img_mgmt_impl_erase_slot() to erase slot1 and flash_img_init().

Then, I use img_mgmt_impl_write_image_data() to write incoming 512-byte blocks of the image.

When I have finished writing the whole image to slot 1, I call img_mgmt_read_info() and img_mgmt_impl_write_pending() with the write pending permanent flag false. It is all in the code in my post.

It seems to work.

I guess the img_mgmt_impl_erase_slot() call counts as one of the 10,000 flash writes.

I will get back to you.

Thanks David

RE: nRF52840 Possible Flash Damage

AHaug — Thu, 15 Dec 2022 08:57:18 GMT

Hi,

[quote user=""]How many times can the flash be written?[/quote]

From the PS for the 52840, it is specified that the device has 10,000 write/erase cycles for the nRF52840 flash. During normal circumstances you should not need to worry that you will reach this limit, as my colleague has answered in this devzone case. The linked case has a different use-case but I think the information there can be relevant for your project.

However since you state that you can update one of the sensors but not the other it could however indicate that it is a issue with the flash, but I can't say for sure. Do you think you've written that many cycles to the flash during the devices lifespan? Do you have a less used device you can test with, or do you only have one device available?

[quote user=""]My DFU process:[/quote]

Am I correct in assuming that you've written your own custom SMP server?

Could you use nrfjprog to read out the flash-area with the DFU images from all 3 devices and compare them to verify that the images are different?

I would also like to suggest these two resources containing a lot of great samples with DFU and bootloaders in nRF Connect SDK just in case you haven't seen them already. They might not contain what you need to update the sensors, but I find them great to have available when developing anything that should use the related topics:

https://github.com/hellesvik-nordic/samples_for_nrf_connect_sdk
- A great repo with guides, explanations and samples for doing DFU with various means and various configurations
Add DFU support to your application
- A similar guide with a slightly different approach