Release build with MCUboot occasionally bricks custom board

RE: Release build with MCUboot occasionally bricks custom board

Edvin — Mon, 15 Dec 2025 13:28:18 GMT

Can you please try to remove the configs CONFIG_STACK_CANARIES=y and CONFIG_ENTROPY_GENERATOR (if that makes the issue happen again), and then try to add CONFIG_DEBUG_THREAD_INFO=y in that same file, to see if you can determine what thread that is causing this issue. The idea then is to try to increas this thread's stack size. It is typically done through a Kconfig option, but I don't know which one to search for without knowing the name of the thread that caused the issue.

Best regards,

Edvin

RE: Release build with MCUboot occasionally bricks custom board

jyv — Thu, 11 Dec 2025 17:57:18 GMT

Hi Edvin, sorry for the delay. I had to work on other stuff and am just getting back to this now. I have been making sure that the CONFIG_MCUBOOT_IMGTOOL_SIGN_VERSION value is higher than the current value when I perform a DFU. I've also been erasing the secondary image slot on the external flash, but that doesn't fix the "bricking" issue.

I've been doing some more debugging by trying to look at the bootloader logs and I was able to capture these log messages once on a bricked board:

[00:00:00.599,395] <err> os: ***** BUS FAULT *****
[00:00:00.599,700] <err> os:   Imprecise data bus error
[00:00:00.600,006] <err> os: r0/a1:  0x8bbbf596  r1/a2:  0x1abd4294  r2/a3:  0x00000128
[00:00:00.605,468] <err> os: r3/a4:  0x00000001 r12/ip:  0x00000040 r14/lr:  0x0000a5db
[00:00:00.605,895] <err> os:  xpsr:  0x21000000
[00:00:00.606,170] <err> os: Faulting instruction address (r15/pc): 0x0000a1b6
[00:00:00.606,567] <err> os: >>> ZEPHYR FATAL ERROR 26: Unknown error on CPU 0
[00:00:00.606,964] <err> os: Current thread: 0x20000660 (unknown)
[00:00:00.607,299] <err> os: Halting system

I then ran addr2line:

% arm-zephyr-eabi-addr2line -e build-release/mcuboot/zephyr/zephyr.elf 0x0000a1b6
/opt/nordic/ncs/v2.9.2/modules/crypto/mbedtls/library/bignum.c:936

So it looks like there was a fault in this function in mbedtls:

/*
 * Compare signed values
 */
int mbedtls_mpi_cmp_mpi(const mbedtls_mpi *X, const mbedtls_mpi *Y)
{
    size_t i, j;

    for (i = X->n; i > 0; i--) {
        if (X->p[i - 1] != 0) {
            break;
        }
    }

    for (j = Y->n; j > 0; j--) {
        if (Y->p[j - 1] != 0) {
            break;
        }
    }

    if (i == 0 && j == 0) {
        return 0;
    }

    if (i > j) {
        return X->s;
    }
    if (j > i) {
        return -Y->s;
    }

    if (X->s > 0 && Y->s < 0) {
        return 1;
    }
    if (Y->s > 0 && X->s < 0) {
        return -1;
    }

    for (; i > 0; i--) {
        if (X->p[i - 1] > Y->p[i - 1]) {
            return X->s;
        }
        if (X->p[i - 1] < Y->p[i - 1]) {
            return -X->s;
        }
    }

    return 0;
}

I have not been able to determine any more information from this though and am still working on it.

Also, another thing I have found is that enabling the compiler stack canary in my child_image/mcuboot/prj.conf:

CONFIG_ENTROPY_GENERATOR=y
CONFIG_STACK_CANARIES=y

Seems to consistently make the application properly boot on a bricked board. If I flash a release build again on to the same board, this time with the stack canaries disabled, the board will brick again.

I am still working on this to try and figure out more info, but in the meantime do you have any ideas on:

Why I'm hitting that bus fault in the bootloader?
Why does enabling compiler stack canaries let my application boot normally?

RE: Release build with MCUboot occasionally bricks custom board

Edvin — Fri, 31 Oct 2025 13:56:04 GMT

One thing to check is that the CONFIG_MCUBOOT_IMGTOOL_SIGN_VERSION="0.0.1" is higher than the previous one. If it is lower, it will be rejected. However, if you erase the chip (nrfutil device erase) then there should be no traces of another application image with a higher application sign version.

Let me know what you find!

BR,

Edvin

RE: Release build with MCUboot occasionally bricks custom board

jyv — Thu, 30 Oct 2025 16:59:35 GMT

Yes, if I flash a debug build to a bricked board it will run as normal, which leads me to believe that the hardware itself is fine as well as the application.

Reproduction is one of the issues I am facing in debugging this problem as I have not found any clear reproduction steps. Most of the time when I flash a release build it runs fine, and occasionally it bricks.

I have not made any changes to the bootloader. Besides the Kconfigs (child_image/mcuboot/prj.conf) and Devicetree properties (child_image/mcuboot.overlay) that I included earlier, I also have a child_image/mcuboot.conf:

CONFIG_BOOT_SIGNATURE_KEY_FILE="/Users/jyv/Projects/my_project/keys/secret.pem"

I also have a release.conf (which extends my application's prj.conf) that has some bootloader related Kconfigs set:

CONFIG_ASSERT=n

CONFIG_WATCHDOG=y
CONFIG_BRICK_GUARD=y
CONFIG_BRICK_GUARD_TIMEOUT=60000

CONFIG_FLASH_LOG_LEVEL_WRN=y

CONFIG_ARCH_LOG_LEVEL_OFF=y
CONFIG_KERNEL_LOG_LEVEL_OFF=y
CONFIG_MPU_LOG_LEVEL_OFF=y

CONFIG_SIZE_OPTIMIZATIONS=y
CONFIG_LTO=y
CONFIG_ISR_TABLES_LOCAL_DECLARATION=y

CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_MCUBOOT_BUILD_STRATEGY_FROM_SOURCE=y
CONFIG_MCUBOOT_IMGTOOL_SIGN_VERSION="0.0.1"
CONFIG_PM_EXTERNAL_FLASH_MCUBOOT_SECONDARY=y
CONFIG_PM_OVERRIDE_EXTERNAL_DRIVER_CHECK=y

CONFIG_ZCBOR=y
CONFIG_MCUMGR=y
CONFIG_MCUMGR_TRANSPORT_BT=y
CONFIG_MCUMGR_TRANSPORT_BT_PERM_RW=y

CONFIG_STREAM_FLASH=y
CONFIG_IMG_MANAGER=y
CONFIG_MCUMGR_GRP_IMG=y
CONFIG_MCUMGR_GRP_OS=y

CONFIG_MCUMGR_MGMT_NOTIFICATION_HOOKS=y
CONFIG_MCUMGR_GRP_IMG_STATUS_HOOKS=y

CONFIG_BT_CTLR_ASSERT_OVERHEAD_START=n

CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=3072

When I uploaded an image to the device using the nRF Connect for Mobile App, I would use the app_update.bin file generated by my build configuration and I would select the test and confirm option. This process would usually take about a minute to complete (image upload, new image is moved to primary slot, board resets and confirms new image version via logs).

I have not tried performing a DFU via the nRF Device Manager app, I will try that and get back to you.

Thank you.

RE: Release build with MCUboot occasionally bricks custom board

Edvin — Thu, 30 Oct 2025 13:09:04 GMT

But, if you flash a "debug build" (without a bootloader) to one of the bricked boards, they run as normal, right? So the HW itself is working, right? And apparently also the application?

I guess there is no way for me to reproduce what you are seeing using a DK?

Did you do any changes to the bootloader?

How did you upload it? You mentioned the nRF Connect for Mobile App, but when you did, did you select "test", "run", "validate" or something like that? Can you please try uploading it using the nRF Device Manager for mobile?

Best regards,

Edvin

RE: Release build with MCUboot occasionally bricks custom board

jyv — Wed, 29 Oct 2025 20:32:40 GMT

Hi Edvin,

Thanks for getting back to me.

[quote userid="26071" url="~/f/nordic-q-a/125277/release-build-with-mcuboot-occasionally-bricks-custom-board/552769"]1: If you flash the release build, and it "bricks", and you flash the release build once more, without flashing anything else in between. Is it still bricked? Even if you erase the chip in between? And does it brick immediately, or after a little while?[/quote]

Yes, if I reflash the release build the device is still bricked, even if I erase the chip in between. I usually erase the chip using one of these commands:

nrfutil device erase
nrfutil device recover

The device bricks immediately (the application never starts so I believe the bricked devices are getting stuck in the bootloader).

[quote userid="26071" url="~/f/nordic-q-a/125277/release-build-with-mcuboot-occasionally-bricks-custom-board/552769"]2: If you flash the bricked release build to another identical board, and it doesn't brick, does it behave completely like normal?[/quote]

Yes, the second board that doesn't brick behaves completely like normal.

[quote userid="26071" url="~/f/nordic-q-a/125277/release-build-with-mcuboot-occasionally-bricks-custom-board/552769"]3: Before the brick. Was there ever done a DFU (since the last time the device was reprogrammed using a debugger)? [/quote]

I have been performing DFU's on our devices with the release build using the Android version of the nRF Connect mobile app. However, I cannot say for sure that all of the bricked boards (currently 3) have had DFU's performed on them.

[quote userid="26071" url="~/f/nordic-q-a/125277/release-build-with-mcuboot-occasionally-bricks-custom-board/552769"]4: What are the symptoms of a bricked device? Do you have any log output?[/quote]

There are a few symptoms I see:

We have set up our application to use the Zephyr logging API via RTT, but I don't see any logs
Our application plays an LED sequence on startup, but I do not see this occurring
Our application has a vibration motor triggered via a touch sensor, this is unresponsive
An unbricked device can be discovered via the nRF Connect mobile app, but a bricked device will not show up in the app

Thank you!

jyv

RE: Release build with MCUboot occasionally bricks custom board

Edvin — Wed, 29 Oct 2025 13:42:20 GMT

Hello,

[quote user=""]

This is not a true brick however, because if I flash a debug build onto the bricked board the app will run fine, but then if I reflash the release build the board will brick again. Flashing an identical release build onto an identical custom board will usually not brick the second board.

[/quote]

In this case:

1: If you flash the release build, and it "bricks", and you flash the release build once more, without flashing anything else in between. Is it still bricked? Even if you erase the chip in between? And does it brick immediately, or after a little while?

2: If you flash the bricked release build to another identical board, and it doesn't brick, does it behave completely like normal?

3: Before the brick. Was there ever done a DFU (since the last time the device was reprogrammed using a debugger)?

4: What are the symptoms of a bricked device? Do you have any log output?

Best regards,

Edvin