FOTA with Nordic Connect SDK (ncs) not working

RE: FOTA with Nordic Connect SDK (ncs) not working

Sunil vignesh — Wed, 17 Mar 2021 18:19:42 GMT

Hi Mike,

I am having issue on nrf52832 NCS FOTA . https://devzone.nordicsemi.com/f/nordic-q-a/72803/ncs-ota-dfu-failed-in-nrf52832/300422#300422

I have the problem

during OTA from android mobile process gets rejected. no changes happened

If possible can you share your working example code "LBS code" . so that it will be helpful.

RE: FOTA with Nordic Connect SDK (ncs) not working

Einar Thorsrud — Tue, 02 Mar 2021 09:59:36 GMT

Hi Mike,

I am glad to hear it works. Regarding signing the smp_svr sample documentation does not take into account the work done by the partition manager script that is part of NCS which handles signing for you (using default key when nothing is specified), so it is not entirely accurate in the NCS context.

Einar

RE: FOTA with Nordic Connect SDK (ncs) not working

Mike Hibbett — Mon, 01 Mar 2021 21:32:15 GMT

That worked, with only a small modification!

My modification was changing the board's Kconfig file, setting DCDC to "n", as my custom board does not have the external DCDC converter circuit, as I dont need it.

My issue was that the smp_svr build instructions, on the link below, include a step (signing the image) that simply fails. So I never proceeded. It did not state that the failing step was not required.

Great, thanks for that, I can now work from your build instructions to integrate FOTA into my existing application.

Thanks,

Mike

developer.nordicsemi.com/.../README.html

RE: FOTA with Nordic Connect SDK (ncs) not working

Einar Thorsrud — Mon, 01 Mar 2021 13:15:15 GMT

Hi Mike,

Thank you. This is puzzling, to be honest. Would it be possible to provide your example that reproduce this on a DK, with instructions on how to reproduce? That way I can debug on my side as well. You can create a private ticket and refer to this one if you do not want to share the code in public.

[quote user="Mike Hibbett"]Can you provide me with an existing example of creating a FOTA build, based on the nrf52832? I presume this has been tested. I could then use that as the basis for working out my own settings.[/quote]

Regarding the question about an existing FOTA example that is present in the nRF Connect SDK. I understand you are using NCS 1.3.0, and I just verified that the example works there. You can use it like this:

Navigate to zephyr\samples\subsys\mgmt\mcumgr\smp_svr\ (using NCS 1.3.0 in this case, but method is same for newer versions)
Build with west build -b nrf52dk_nrf52832 . -- -DOVERLAY_CONFIG=overlay-bt.conf
Program build/zephyr/merged.hex from the first build. This way you have MCUboot and the application which has smp_svr and bluetooth transport on it.
(Open a UART PuTTY terminal to the nRF so that you can observe the log)
Build the example again to get a different app binary (build time stamp is part of the app due to logging it)
Copy build/zephyr/app_update.bin to your phone (call it something like first_app_update.bin to be able to distinguish it)
Program the .bin from via DFU (I use nRF Connect for iOS but nRF Connect for Android should do the same)
Observe from logs that it works
Go back to step 5 and to this again, observe that second DFU upgrade also works.

Einar

RE: FOTA with Nordic Connect SDK (ncs) not working

Mike Hibbett — Sat, 27 Feb 2021 18:48:10 GMT

RE: FOTA with Nordic Connect SDK (ncs) not working

Mike Hibbett — Fri, 26 Feb 2021 11:59:23 GMT

I will recreate that set-up and provide the information.

Can you provide me with an existing example of creating a FOTA build, based on the nrf52832? I presume this has been tested. I could then use that as the basis for working out my own settings.

There must be a description somewhere of "here are the config setting / dts memory settings to build FOTA images". I have probably just set some setting incorrectly.

Regards,

Mike

RE: FOTA with Nordic Connect SDK (ncs) not working

Einar Thorsrud — Fri, 26 Feb 2021 11:43:45 GMT

Hi Mike,

That is very interesting, though I am not able to say why. To know more from this, can you select frame 3, then check "esf" variable to get R0-R3 + SP and LR content?

Einar

RE: FOTA with Nordic Connect SDK (ncs) not working

Mike Hibbett — Thu, 25 Feb 2021 15:14:08 GMT

Some additional info, if it helps. I enabled more debug output in mcuboot. When I FOTA an image over, the device reboots, displaying this output, and no more.

[00:00:00.274,536] <inf> mcuboot: Starting bootloader
[00:00:00.280,548] <inf> mcuboot: Swap type: perm
[00:00:00.550,292] <inf> mcuboot: Image upgrade secondary slot -> primary slot
[00:00:00.557,983] <inf> mcuboot: Erasing the primary slot
[00:00:00.658,386] <inf> mcuboot: Copying the secondary slot to the primary slot: 0x2cec4 bytes
[00:00:03.891,265] <dbg> mcuboot.boot_write_magic: writing magic; fa_id=2 off=0x34ff0 (0x40ff0)
[00:00:03.900,726] <dbg> mcuboot.boot_copy_image: erasing secondary header
[00:00:03.910,095] <dbg> mcuboot.boot_copy_image: erasing secondary trailer
[00:00:04.183,990] <inf> mcuboot: Bootloader chainload address offset: 0xc000
[00:00:04.191,589] <inf> mcuboot: Jumping to the first image slot

Attaching the debugger, I am in a hard fault condition. Here's the call stack:

RE: FOTA with Nordic Connect SDK (ncs) not working

Einar Thorsrud — Wed, 24 Feb 2021 16:12:08 GMT

Hi Mike,

I am not able to explain this at this point. I need to look more into it before I can say something meaningful, but I hope to get back to you tomorrow.

Einar

RE: FOTA with Nordic Connect SDK (ncs) not working

Mike Hibbett — Tue, 23 Feb 2021 21:52:12 GMT

So a small amount of progress. Let me refresh where I am.

SDK v1.4.2. peripheral LBS application. Modified to operate with mcuboot & fota, by the following changes

main() function has these calls added:

printk("Advertising successfully started\n");
// For FOTA
os_mgmt_register_group();
img_mgmt_register_group();
smp_bt_register();

prj.conf is like this:

CONFIG_NCS_SAMPLES_DEFAULTS=n
CONFIG_BT=y
CONFIG_BT_PERIPHERAL=y
CONFIG_BT_DEVICE_NAME="Nordic_Blinky"
CONFIG_BT_LBS=y
CONFIG_BT_LBS_POLL_BUTTON=y
CONFIG_DK_LIBRARY=y
CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_MCUMGR=y
CONFIG_MCUMGR_CMD_OS_MGMT=y
CONFIG_MCUMGR_CMD_IMG_MGMT=y
CONFIG_MCUMGR_SMP_BT=y
CONFIG_IMG_ERASE_PROGRESSIVELY=y
CONFIG_BT_L2CAP_TX_MTU=252
CONFIG_BT_L2CAP_RX_MTU=252
CONFIG_BT_RX_BUF_LEN=260
CONFIG_BT_CTLR_TX_BUFFER_SIZE=251
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_MCUMGR_SMP_BT_AUTHEN=n
CONFIG_DEBUG_OPTIMIZATIONS=y
CONFIG_MCUMGR_SMP_SHELL=y
CONFIG_FILE_SYSTEM=y
CONFIG_FILE_SYSTEM_LITTLEFS=n
CONFIG_MCUMGR_CMD_FS_MGMT=y
CONFIG_STATS_NAMES=n
CONFIG_LOG=n
CONFIG_FLASH=y
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048

board dts file has this flash setup:

partitions {
compatible = "fixed-partitions";
#address-cells = <1>;
#size-cells = <1>;

boot_partition: partition@0 {
label = "mcuboot";
reg = <0x00000000 0xc000>;
};
slot0_partition: partition@c000 {
label = "image-0";
reg = <0x0000C000 0x34000>;
};
slot1_partition: partition@3e000 {
label = "image-1";
reg = <0x00040000 0x34000>;
};
/* scratch_partition: partition@70000 {
label = "image-scratch";
reg = <0x00074000 0xa000>;
};
*/ storage_partition: partition@7a000 {
label = "storage";
reg = <0x0007a000 0x00006000>;
};

Note the change I made to the slots, to allow the application to fit.

Built the code, it's this size:

Programming 37.2 KB of addresses 00000000 — 000094eb
Programming 207.4 KB of addresses 0000c000 — 0003fd9b

so will fit in the slots.

Erased the chip and downloaded the image. serial debug on mcu shows this:

*** Booting Zephyr OS build v2.4.0-ncs2 ***
[00:00:00.400,878] <inf> mcuboot: Starting bootloader
[00:00:00.406,921] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.416,870] <inf> mcuboot: Boot source: none
[00:00:00.422,302] <inf> mcuboot: Swap type: none
[00:00:00.724,212] <inf> mcuboot: Bootloader chainload address offset: 0xc000
[00:00:00.731,811] <inf> mcuboot: Jumping to the first image slot
*** Booting Zephyr OS build v2.4.0-ncs2 ***
Starting Bluetooth Peripheral LBS example
Application build time: Feb 23 2021 21:16:22
Bluetooth initialized
Advertising successfully started

Rebuilt the code to generate a different binary (I have time & date coded in a printf)

Started a download via nrf connect for mobile using CONFIRM ONLY. Download proceeds, at a slow rate, 1.5kB/s

At the end, nrf connect for mobile says "validating", and the mcu reboots, showing this:

*** Booting Zephyr OS build v2.4.0-ncs2 ***
[00:00:00.274,505] <inf> mcuboot: Starting bootloader
[00:00:00.280,517] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.290,466] <inf> mcuboot: Boot source: none
[00:00:00.295,898] <inf> mcuboot: Swap type: perm

... a few seconds pass, then this is printed ...

[00:00:12.354,919] <inf> mcuboot: Bootloader chainload address offset: 0xc000
[00:00:12.362,518] <inf> mcuboot: Jumping to the first image slot

The android app disconnects with no messages.

nrf logger dump of these final stages of dfu are shown below.

So the image upload bricks the device; rebooting the device leaves the boot sequence as this:

*** Booting Zephyr OS build v2.4.0-ncs2 ***
[00:00:00.274,475] <inf> mcuboot: Starting bootloader
[00:00:00.280,487] <inf> mcuboot: Primary image: magic=good, swap_type=0x3, copy_done=0x1, image_ok=0x1
[00:00:00.290,344] <inf> mcuboot: Boot source: none
[00:00:00.295,776] <inf> mcuboot: Swap type: none
[00:00:00.566,192] <inf> mcuboot: Bootloader chainload address offset: 0xc000
[00:00:00.573,822] <inf> mcuboot: Jumping to the first image slot

Completely stuck at this point and out of ideas.

I hope these details reveal what is going wrong.

Regards,

Mike.

RE: FOTA with Nordic Connect SDK (ncs) not working

Mike Hibbett — Tue, 23 Feb 2021 17:59:57 GMT

Ah, so adding that debug config caused this to appear, interesting bit in bold

*** Booting Zephyr OS build v2.4.0-ncs2 ***
[00:00:00.274,414] <inf> mcuboot: Starting bootloader
[00:00:00.280,395] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.290,313] <inf> mcuboot: Boot source: none
[00:00:00.295,776] <inf> mcuboot: Swap type: test
[00:00:00.565,460] <wrn> mcuboot: Not enough free space to run swap upgrade
[00:00:00.883,209] <inf> mcuboot: Bootloader chainload address offset: 0xc000
[00:00:00.890,808] <inf> mcuboot: Jumping to the first image slot
*** Booting Zephyr OS build v2.4.0-ncs2 ***
Starting Bluetooth Peripheral LBS example

I am building this on NRF52832 - thats only 512KB flash compared to NRF52840 1024KB

Is there an alternative FOTA process without swap method, that can cope with the smaller size? Ive done FOTA on Nordic chips with less memory than 512KB

Cheers,

Mike

RE: FOTA with Nordic Connect SDK (ncs) not working

Mike Hibbett — Tue, 23 Feb 2021 14:52:03 GMT

Hi Einar, the device is inoperative after

[00:00:11.939,270] <inf> mcuboot: Jumping to the first image slot

and no further information is displayed.

I will rebuild & reflash with the optimisations enabled, and let you know.

Thanks!

Mike

RE: FOTA with Nordic Connect SDK (ncs) not working

Einar Thorsrud — Tue, 23 Feb 2021 14:29:47 GMT

Hi Mike,

I do not see that you lack any configurations. You can compare with the smp_svr sample, but this looks good.

What happens after getting "mcuboot: Jumping to the first image slot" in the bootloader log? What state is the nRF in? Do you get more usefull debug information if building with CONFIG_DEBUG_OPTIMIZATIONS=y?

Einar

RE: FOTA with Nordic Connect SDK (ncs) not working

Mike Hibbett — Tue, 23 Feb 2021 11:33:07 GMT

By adding the following lines, which I found someone using through a random google search

CONFIG_BT_RX_BUF_LEN=258
CONFIG_BT_L2CAP_RX_MTU=247
CONFIG_BT_L2CAP_TX_MTU=247
CONFIG_BT_CTLR_TX_BUFFER_SIZE=251
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_MCUMGR_SMP_BT_AUTHEN=n

FOTA now proceeds as expected, although slowly, 1.3KB/s. At the end, the application booted.

A second rebuild of the app failed to download, and the device now no longer gets past the bootloader.

*** Booting Zephyr OS build v2.4.0-ncs2 ***
[00:00:00.274,566] <inf> mcuboot: Starting bootloader
[00:00:00.280,578] <inf> mcuboot: Primary image: magic=good, swap_type=0x4, copy_done=0x1, image_ok=0x1
[00:00:00.290,435] <inf> mcuboot: Boot source: none
[00:00:00.295,867] <inf> mcuboot: Swap type: test
[00:00:11.931,640] <inf> mcuboot: Bootloader chainload address offset: 0xc000
[00:00:11.939,270] <inf> mcuboot: Jumping to the first image slot

So please point us to the correct technique for enabling FOTA.

Thanks,

Mike

RE: FOTA with Nordic Connect SDK (ncs) not working

Mike Hibbett — Tue, 23 Feb 2021 10:21:54 GMT

I have resolved the build issues (header files moved in the SDK); the behaviour is the same. FYI, the changes I made are as follows:

Added to prj.conf:

# Added for FOTA
CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_MCUMGR=y
CONFIG_MCUMGR_CMD_OS_MGMT=y
CONFIG_MCUMGR_CMD_IMG_MGMT=y
CONFIG_MCUMGR_SMP_BT=y
CONFIG_IMG_ERASE_PROGRESSIVELY=y

Added to main.c

// For FOTA
#include <tinycbor/cbor.h>
#include "cborattr/cborattr.h"
#include <mgmt/mgmt.h>
#include <mgmt/mcumgr/smp_bt.h>
#include <mgmt/mcumgr/buf.h>
#include <mgmt/mcumgr/smp.h>
#include "os_mgmt/os_mgmt.h"
#include "os_mgmt/os_mgmt_impl.h"
#include "os_mgmt/os_mgmt_config.h"
#include "img_mgmt/image.h"
#include "img_mgmt/img_mgmt.h"
#include "img_mgmt/img_mgmt_impl.h"
#include "img_mgmt/img_mgmt_config.h"

and in main()

printk("Application build time: %s %s \n", __DATE__, __TIME__);

...

printk("Advertising successfully started\n");

// For FOTA
os_mgmt_register_group();
img_mgmt_register_group();
smp_bt_register();

I reloaded the project, built and flashed. The image booted ok

*** Booting Zephyr OS build v2.4.0-ncs2 ***
[00:00:00.403,625] <inf> mcuboot: Starting bootloader
[00:00:00.409,667] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, cop y_done=0x3, image_ok=0x3
[00:00:00.419,677] <inf> mcuboot: Boot source: none
[00:00:00.425,140] <inf> mcuboot: Swap type: none
[00:00:00.716,369] <inf> mcuboot: Bootloader chainload address offset: 0xc000
[00:00:00.724,029] <inf> mcuboot: Jumping to the first image slot
*** Booting Zephyr OS build v2.4.0-ncs2 ***
Starting Bluetooth Peripheral LBS example
Application build time: Feb 23 2021 09:53:14
I: 2 Sectors of 4096 bytes
I: alloc wra: 0, ff0
I: data wra: 0, 0
I: SoftDevice Controller build revision:
I: cf 5c 0f 11 88 9c d7 02 |.\......
I: 15 27 c7 c3 ca 60 19 85 |.'...`..
I: b7 c4 50 e3 |..P.
I: No ID address. App must call settings_load()
Bluetooth initialized
Advertising successfully started

I rebuilt the code, to generate a different binary image (TIME being different) and attempted to DFU this using nrf connect mobile. "Validating" was displayed for a few seconds, then the Android app reverted without attempting the actual DFU.

The target debug output showed

Connected
Disconnected (reason 19)
Connected
W: Unhandled ATT code 0x1d
Passkey for 4d:a0:4e:95:16:f2 (random): 128349
Security changed: 4d:a0:4e:95:16:f2 (random) level 4
Pairing completed: a8:87:b3:74:2d:56 (public), bonded: 1

I assume I am missing something. Please advise, this product development is now blocked.