NRF52833 UART Secure DFU - Not booting after app start

[dsweet]

I'm working on developing a UART Secure DFU platform on the NRF52833 with SDK 17.1 and soft-device s140 v7.2.0.

As there are no direct examples for this application, I followed the steps outlined in this blog post as a starting point. I then updated my application to leave enough space for the larger SD and implemented the steps to get the uECC crypto working in place of the hardware crypto on the 52840. 

I have things working to the stage that I can successfully dfu both the soft-device and my application, but when the bootloader attempts to start my application (via the SD) at address 0x1000, the system never boots.

After both DFU's are complete, I see the following debug messages:

00> <debug> app: App is valid
00> <warning> nrf_dfu_settings: No additional data erased
00> <info> nrf_dfu_settings: Backing up settings page to address 0x7E000.
00> <debug> nrf_dfu_settings: Destination settings are identical to source, write not needed. Skipping.
00> <debug> app: Running nrf_bootloader_app_start with address: 0x00001000
00> <debug> app: Disabling interrupts. NVIC->ICER[0]: 0x0

And nothing afterwards. Seems like it thinks my app is valid but somewhere in the handoff to the SD to App the system gets lost.

Some notes:

- I created copies of my dfu/application project with logging enabled in the bootloader to get the above prints, but overall behavior seems the same in both projects.

- Both the bootloader and application projects work fine if I program them in on their own from SES. 

- I do an erase all and program only the bootloader and the MBR into the device (no SD nor app hard-programmed) and then do 2 separate DFU operations for the SD then the application. This mimics my end-customers usage flow and (I think) avoids the need to do any bootloader settings merge operations for initial programming.

- Logging seems to indicate the uECC package verification is working as intended.

- Logging seems to indicate the bootloader is working (eg dfu is all good, no errors, tries to hand off to app but then fails)

- My application starts (FLASH_START) at 0x27000, which matches the required size indicated by the s140 v7.2.0 SD release notes.

- My bootloader project addresses (in my non-debug project) are:

- My application project addresses (in my non-debug project) are:

Any ideas on what I am missing here? I don't even know how to debug this because once it jumps to 0x1000 I can't debug or log anymore so it is a bit of a black box. I'm not sure how to try to determine what is happening after this jump. Unsure if its making to the beginning of my app and crashing due to an application problem (eg reinitializing some hardware already done by the bootloader), or if it is getting lost in the soft-device before it makes it to my application.

I've attached a full debug log of the sd DFU + app DFU sequence. I think it may be dropping some debug prints but gives an idea of what is going on. 

rtt_bootload_sequence.log

[Hung Bui]

Hi Dan, 

From what you described I couldn't find anything wrong. Note that you can debug the application after it's flashed by DFU the same as when it's flashed by an IDE. You can simply run it in debug and the debugger will wait until the PC counter jumps to the address of the reset vector in the application. 

I would suggest the following process to detect the issue: 

1.Start with debug version of the bootloader.

2. Try to update a very simple application with no softdevice, blinky for example. The application should not use any interrupt handler, just a normal NOP delay to blink LED. Please make sure you use the version that's compiled for MBR, which mean it starts at 0x1000 and memory starts at 0x20000008

3. Try to test with an application that actually use interrupt handler, e.g RTC example. Same requirement on the memory as above

4. Test with an example that uses softdevice, ble_app_hrs for example. 

Let us know if it fails at any step. 

Another way to debug is to do a hex dump and then compare it with what you have when flashing the application + SD with the programmer to see if there is any different. 

[dsweet]

Thanks for the response. I will give a simpler application a try (starting with no SD) to see if I see the same issues. 

I did some more debugging with my existing debug application and don't believe I'm making it to my application. It appears I am hitting a hard fault in the soft device. 

Specifically, from the `jump_to_addr()` call in the bootloader, I can see the following sequence (in Disassembly):

1. Starts with ~4 instructions at 0xA60, which is odd because this is in the MBR and I thought `jump_to_addr` was targeting address 0x1000 (start of SD).

2. Jumps from 0xA68 to a region at 0x25F46, which is towards the end of the soft device region.

3.  Runs about a dozen instructions before jumping to 0x273A2, which is the HardFault handler in my application space. As far as I can tell this is the only/first time I make it to my application space (I tried running the debugger in my application as you suggest and it breaks here).

Let me know if the above gives you any clues as to what might be happening and I will try a blinky application in the meantime.

[dsweet]

Hung Bui Another set of interesting updates:

My comment above about ending up at 0xA60 seemed wrong/suspicious and I recalled running across this issue related to a bug in the jump_to_address optimization. This issue was supposed to be fixed in v17.0+ SDK, so I originally moved on, but decided to revisit messing with optimizations:

My current debug project has an optimization level of 'None' as my baseline. I then tried the following tests:

1. Change full bootloader project to 'Level 2 for Size'. -> Now I get a new error: 'Failed running nrf_dfu_mbr_irq_forward_address_set()' right before starting the application. App does not boot.

2. Return full bootloader  project to 'Level 0', but change only nrf_bootloader_start_final.c to 'Level 2 for Size' -> Bootloading now works, app boots and runs as expected.

3. Change full bootloader project to 'Level 3' -> Bootloading works, app boots and runs as expected.

Note these changes are only applied to my bootloader project, my SD and APP packages remained the same for all tests.

I find these results a bit surprising and confusing. Particularly it confuses me that test 1 does not work the same as test 2. Seems like it is possible my issue is not the same as the original jump_to_address bug, but perhaps similar. Another idea is that there is some sort of word alignment issue somewhere and changing various optimizations just moves things around in a way to just happen to work. 

Thoughts? I plan to try applying the above modifications back to my original non-debug version of my project to see if I end up with similar results.

[dsweet]

Hung Bui It seems I have been able to simplify what works and does not work to a very simple code change, but I'm not sure I understand why.

By adding `volatile` to the sd_mbr_command_t command variable in `nrf_dfu_mbr_irq_forward_address_set()`, my bootloader appears to work with any optimization setting.

Applying this change back to my original (non-debug) projects is also working. Seems to bootload and run my full application (with BLE / SD dependencies) just fine.

I'm unsure what sd_mbr_command is doing under the surface but wondering if the address variable was somehow being optimized incorrectly causing an invalid jump/forwarding address.

Does this make any sense to you and any chance this is related to any known bug/issue? Seems surprising I'm the only one to run into this during new bootloader development on the v17.1 SDK.

[Hung Bui]

Hi Dan, 
Please let me know the toolchain setup you have. 
We do see an issue with newer toolchain (IDE/compiler) when compiling with our nRF5 SDK. Note that the SDK is now considered legacy and we stopped any new development for it since last year. 

An update to the compiler may break the compatibility with the SDK. 

Could you reproduce the same issue on a nRF52832 or nRF52840 so that there is no modification needed to the bootloader project ? 

[dsweet]

Hi Hung Bui 

My IDE / gcc info is below, let me know if you need any other details:

Incidentally, this current effort started as a migration from a prior project on a nRF52840 that was originally on SDK 15.3 and s140 6.1.1 SD. I first updated this 52840 project to the same SDK17.1 / SD 7.2.0, which worked great with minimal/expected changes. AKA I wasn't able to reproduce these same issues on the nRF52840 with the same SDK/SD. 

It wasn't until I worked on migrating to the nRF52833 where I started running into problems, it is unfortunate there was no working dfu example to start from for this platform. 

I will note that I think I updated from SES version 4.22 (IDE used when I did the original 52840 project) to SES 6.32b right around the time when I started trying to port to the 52833 because debugging kept crashing on 4.22 on the 52833. It seemed like updating to 6.32b fixed this issue.

[Hung Bui]

Hi Dan, 
It's recommended to use the exact SES version that we mentioned in the release note of SDK v17.1 , which is v5.42a. 

 
Please see here.