Unknown hangs with secure bootloader without debug

[Alex Tishchenko]

Hello,

This is a strange problem and I do hope I am missing something. There are two bootloaders in the examples/dfu section for a particular transport - with debug and without debug. Making my own custom build of the bootloader for the project (private key, buttonless, no uart, etc), I stumbled upon an unexplainable for me problem: The same bootloader, with the same code works properly when the debug information is included and hangs silently when not.

Things that are different between the two:

sdk_config.h:

-----------------

A few nuts and bolts concerning RTT logging configuration (same as in the example code provided) and LOG_LEVEL settings.

.emProject:

---------------

The one without debug omits a pair of the debug related macros - DEBUG_NRF and NRF_DFU_DEBUG_VERSION

Flash placement differs: The one with debug takes flash from x71000, the one without debug from x78000 (as in examples), RAM placement is the same for both.

Flash sections order is a tiny bit different, but that doesn't look like critical problem, there is no 4x4 overlaps in the final image and Programmer verifies it as correct.

Everything else, as far I managed to check, is the same. What am I missing?

Does anybody else meet with this situation?

[Hung Bui]

Hi Alexander, 

Could you give some more information about when the bootloader hang ? Was it hang right when booting ? or when you were doing DFU ? 

Please try to turn on some log. 

You can also move the non-debug version of the bootloader to address 0x71000 and then change the optimization level to 0 . This way you can step into the code and check where it stuck. 
Have you tried to test the stock bootloader with no modification ? 

[Alex Tishchenko]

Hello Hung,

Ok, this is what I did to check right now:

1) I built the no-debug bootloader from the SDK with the following changes:

DFU key, set to my own,

NRF_BL_DFU_ENTER_METHOD_BUTTON=0

NRF_BL_DFU_ENTER_METHOD_BUTTONLESS=1

NRF_DFU_BLE_REQUIRES_BONDS=1

NRF_SDH_BLE_SERVICE_CHANGED=1

2) I build buttonless app, with single modification: NRF_DFU_BLE_BUTTONLESS_SUPPORTS_BONDS=1

3) usual mantra:

$ nrfutil settings generate --family NRF52 --application app.hex --application-version 1 --bootloader-version 1 --bl-settings-version 2 settings.hex

Bootloader DFU Settings:
* File:                     settings.hex
* Family:                   nRF52
* Start Address:            0x0007F000
* CRC:                      0x5EF46751
* Settings Version:         0x00000002 (2)
* App Version:              0x00000001 (1)
* Bootloader Version:       0x00000001 (1)
* Bank Layout:              0x00000000
* Current Bank:             0x00000000
* Application Size:         0x0000D0B0 (53424 bytes)
* Application CRC:          0x1603126C
* Bank0 Bank Code:          0x00000001
* Softdevice Size:          0x00000000 (0 bytes)
* Boot Validation CRC:      0x3DCFD0A2
* SD Boot Validation Type:  0x00000000 (0)
* App Boot Validation Type: 0x00000001 (1)

$  mergehex.exe -m sd.hex app.hex boot.hex settings.hex -o init.hex

Parsing input files.
Merging file "sd.hex" into output.
Merging file "app.hex" into output.
Merging file "boot.hex" into output.
Merging file "settings.hex" into output.
Storing merged file.

4) Flashed it into 52-DK with emStudio (Target -> Connect, Erase All, Download iHex..) with a hang as a result, with LEDs 1 and 2 wildly flashing, nothing in RTT, nothing on UART and nothing in the air.

Here comes the interesting part :^)

If I try to open the resulting .hex file with Programmer, it gives me an overlap error:

BUT, (I just thought of trying it along the way) -  If I add every  file to the Progammer file list one by one, it takes them in silently AND flashing that combo to the board does miracle! The bootloader is up, the app is up, dfu service is up, bonded and buttonless - DFU upgrades work and upgrading bootloader (same type for the same type!) works too.

I will further investigate the initial hang, that I had in the beginning  - OTA upgrade of the running bootloader-with-debug  with a replacement(!) bootloader without debug - for the effects of that flash difference. I am not happy moving no-debug bootloader to 71000, as it looses all the benefit of being no-debug.

But the problem with the mergehex is of another sort, it takes in quietly the following bit from the app.hex:

:103040003905030045050300530503000000000097
:023050002E0050
:1030540000000000002000400000000000200040AC

If you notice there is a gap of two bytes between the second and the third data records. And this is what mergehex does with it:

:103040003905030045050300530503000000000097

:103050002E00030045050300530503000000000097
:1030540000000000002000400000000000200040AC

I marked the piece of a record in bold, so you can visually compare it to the first. The bold part is a leftover from the previous wite combined with the data from the second record in the first set (E0 00), and in the third record mergehex instructs the flasher to trace back 12 bytes and overwrite them with new data. No wonder "something doesn't work"

Programmer, when it loads the files separately deals with the hole of two bytes correctly, but given the combined file refuses to trace back. emStudio doesn't do such checks, whatever results from it - depends on flash type and write instruction used...

OMG, I was nearly pulling hairs out my head, 'cause there is absolutely no black magic in three pieces of software spread like a butter in the flash, and the answer was just that simple, bug in a tool...

[Alex Tishchenko]

As I checked with the .map files, this 2 byte gap is usually generated by linker on a text/rodata or text/observer section boundary...

Don't know if it matters, but I am using SES 5.44 at the moment.

[Alex Tishchenko]

I verified, replacing debug bootloader with no debug one doesn't work. Probably, I've seen somewhere something like the size of the bootloader doesn't change during the lifecycle of the application, so I guess its position in flash too.

[Hung Bui]

Hi Alexander,

Could you check which version of mergehex you have ? 
We had an issue with v10.12.1 , you can have a look here: https://devzone.nordicsemi.com/f/nordic-q-a/75729/possible-mergehex-issue-v10-12-1

Note that when you switch between the bootloader hex files, especially those with different start addresses in flash, you should do an nrfjprog --eraseall. The reason is that the start address of the bootloader is written in the UICR (and the MBR area) so it's required if you want to change the start address of the bootloader. 

Could you confirm that by using the nRF Connect programmer (or nrfjprog ) and flash the hex files separately instead of mergehex it works for you ? both the debug version and non debug version ?