Updating bootloader in SDK11 erases old bootloader flash pages but never copies the new bootloader in its place

[Richard]

Using the SDK11 bootloader, if I recompile the bootloader, package it with nrfutil 0.5.3, and perform a DFU update the original bootloader is erased but the new bootloader never gets copied to the final location by the MBR. I've confirmed that the DFU transfer was successful, valid, etc, and it gets all the way to dfu_bl_image_swap() which calls sd_mbr_command() to copy the actual bootloader from the temporary location to the real location - but this is where it fails.

Reading back the flash after the DFU operation, I can see the new bootloader at 0x49000 (the address passed to sd_mbr_command) but the original bootloader location at 0x77000 is erased. UICR->NRFFW[0] contains the correct bootloader address of 0x77000, and UICR->NRFFW[1] contains the correct MBR settings page address of 0x7E000.

This is being done with the stock SDK11 bootloader and S132 v2.0.1 (tested with v2.0.0 too) with the exception that I've changed the ORIGIN for FLASH in the linker script to 0x77000 (and length correspondingly to 0x7000) and BOOTLOADER_REGION_START to match as 0x77000.

What would cause sd_mbr_command to not copy the valid new bootloader to its destination?

[Turbo J]

Is the watchdog active?

[Richard]

No. There isn't even an application programmed - this is a bare bones test of just the SD + BL programmed after the chip is erased.

[Turbo J]

Next to check: MBR parameter page (MBRPARAMADDR 0x7E000).

I did not find a public documentation of what is in there, but it should at least not be fully erased.

The MBR is designed to continue a BL replacement even after a power failure (POR) - which means there should be some information about it in the MBR param page.

[Richard]

I've reverse engineered most parts of the MBR associated with this process now, and the MBR settings page is fine. It's erased, updated, and verified (hash/CRC and magic number) correctly. The MBR loads the saved SD command from the MBR settings page, and continues through, passing all validations of the command, source, and destination memory addresses.

I've traced this down to the actual bootloader erase/copy step doing nothing, and I'm lost as to why. The MBR is looping over all pages of the bootloader (0x77000 to +0x7000); enabling FLASH erasing, erasing the page, and disabling erasing (i.e. nrf_nvmc_page_erase()). However, looking at the memory while it does this nothing changes (yet when I read back the memory later, it has been erased)! Once that is done it does the equivalent of looping over the new bootloader memory (0x49000) and writing it to the real bootloader address (0x77000) with nrf_nvmc_write_word(). Again, nothing is changed in flash (and indeed, reading it back it's blank)! A similar process (some of the same functions actually) is used for updating the MBR settings page and it works fine, and I can see the erase and writing live in the debugger as it happens.

I could believe the erase being a caching issue with the debugger (Ozone), even though the exact same erase code is used by the MBR Settings Page and works fine there, but the write to the newly erased memory isn't doing anything at all.

Is there anything that could be causing flash erases and writes to be silently ignored? It's not BPROT as that's set correctly by the MBR (I've verified that by disabling BPROT->DISABLEINDEBUG so if it was wrong it would cause a hardfault). This is just so odd.

[Richard]

OK after much debugging, I've found the problem!

Firstly, yes Ozone was caching the flash memory contents and lying to me. I found out that I could read back the flash with J-Flash while paused in the debugger, which let me have a ground-truth for the state of the flash. From there I narrowed in on the problem - I was switching from a dual-bank bootloader to a single-bank bootloader, and there was a few references remaining to BANK_1, which no longer meant anything, and the single-bank bootloader always used BANK_0. BANK_0 was always erased, so never matched the new bootloader when the bootloader compared the memory to check if the update was successful. Because of the mis-match, it would fire off another SD command for copying the bootloader from BANK_0, and thus overwriting the new bootloader with erased memory.

That was... not a fun debugging journey. On the bright side, I now know precisely how nearly the entire MBR and bootloader updating process works.