DFU using USB file system (not serial emulation)?

Question

My project has a nrf5340+external flash, and an external USB-C connector. The device exposes a USB file system (FAT) which is hosted on the external flash partition, and appears as an external disk when connected to a host. This all works well and is used to update config, image files etc for the device to use. 
 I would like to be able to do application / network firmware updates by simply copying the new image (hex or bin) to this file system and rebooting. Has anyone done this? I'm guessing it involves using MCUBoot bootloader and copying the image into the secondary image slot somehow? I also want to put the secondard image slot on my external flash as won't have space on the internal flash - even better would be if MCUBoot could take the image directly from the file in the FS.... 
 by the way, to be clear, I do NOT want to use the 'DFU over USB serial emulation' method, as this requires the user to install the specific tool to the host machine, which may not be available for their host or may not be allowed by IT policy (or by their level of expertise...). 
 thanks for any pointers!

BrianW · Answer

BrianW said: but it doesn't mention having to set a buffer for mcuboot before the call to dfu_target_init() And nowhere in dfu_target.c does it do this?! What is the point of having a 'standard' api dfu_target to mask the different 'targets', if app has to explicitly call a target specific function to get it to work? And not described in the doc either? So... I changed my code to just use the dfu_target_mscuboot_XXX API directly. This all seems to work just fine, except that after dfu_target completes (with no errors), mcuboot still refuses to update from the secondary slot : the relevant logs: [00:00:00.544,769] base: dfu: opened app_update.bin, image type identified as mcuboot [00:00:00.555,236] base: dfu: starting update of image for app_update.bin Copying to secondard slot.................................................................................................................................................................................................................................. DONE [00:00:27.828,277] dfu_target_mcuboot: MCUBoot image-0 upgrade scheduled. Reset device to apply [00:00:27.838,562] base: dfu: no image file for net_core_app_update.bin [00:00:27.846,130] base: dfu : found new images, rebooting in 5s to update....see you on the other side... Rebooting... *** Booting nRF Connect SDK 3758bcbfa5cd *** I: Starting bootloader W: Cannot upgrade: not a compatible amount of sectors I: Bootloader chainload address offset: 0xe000 After a search on this forum, I found this: https://devzone.nordicsemi.com/f/nordic-q-a/94313/mcuboot-runtime-error-cannot-upgrade-not-a-compatible-amount-of-sectors-with-external-flash-on-nrf9160 Turns out that the size of the secondard slot MUST be exactly equal to the primary (not just bigger), and that a) this is not enforced at build time and b) the share_size directive does not work in your pm_static.yml. [as an aside, I note that post was over 2 years ago, and the information about these 2 points was supposed to being fed back to developers.... but still no fix or even any documentation....] Making my mcuboot_secondary partition be the same size as the mcuboot_primary_app one fixed the mcuboot log and now I get: ............................................................................................................................................................................................................................ DONE [00:00:27.582,580] dfu_target_mcuboot: MCUBoot image-0 upgrade scheduled. Reset device to apply [00:00:27.592,864] base: dfu: no image file for net_core_app_update.bin [00:00:27.600,402] base: dfu :System write boot ok flag to NVM [00:00:27.607,360] base: dfu : found new images, rebooting in 5s to update.... [00:00:30.319,671] base: System running ok, write boot ok flag to NVM Rebooting...see you on the other side... *** Booting nRF Connect SDK 3758bcbfa5cd *** I: Starting bootloader I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3 I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3 I: Boot source: none I: Image index: 0, Swap type: test I: Starting swap using move algorithm. I: Bootloader chainload address offset: 0xe000 And miracle : a working DFU from USB filesystem! For those also looking to do this: the 'dfu from local file' fn (call it with the name of the file on the FS, and reboot if it returns true). You will need to implement the usb_fs_XX functions, but these are just wrappers round the FAT FS fs_XX api. // DFU related code static void _dfu_cb(enum dfu_target_evt_id evt_id) { log_info("dfu: callback says event %d", evt_id); } // read blocks of 4K from fle and pass to DFU updater // Returns false if couldn't access or dfu the given file #define DFU_BUF_SZ (4096) static bool _do_dfu(char* filename, int img_num) { int32_t fsize = usb_fs_get_file_size(filename); if (fsize<0) { log_info("dfu: no image file for %s",filename); return false; } file_handle_t fh = usb_fs_open_file(filename, true); // open in read only if (fh==NULL) { log_warn("dfu : failed to open file %s", filename); return false; } uint8_t* buf = Util_amalloc(DFU_BUF_SZ); if (buf==NULL) { usb_fs_close_file(fh); log_warn("dfu : failed to alloc buffer to read %s", filename); return false; } uint32_t buf_sz = 0; // Read first block to let dfu analyse file type buf_sz = usb_fs_read_file(fh, buf, DFU_BUF_SZ); if (buf_sz!=DFU_BUF_SZ) { // image file must be > 4K so if fail to read then give up usb_fs_close_file(fh); Util_afree(buf); log_warn("dfu : failed to read 4K from file %s, %d", filename, buf_sz); return false; } if (!dfu_target_mcuboot_identify(buf)) { usb_fs_close_file(fh); Util_afree(buf); log_warn("dfu : file %s is not an mcuboot image, cannot DFU", filename); return false; } log_info("dfu: opened %s, image type identified as mcuboot", filename); dfu_target_mcuboot_reset(); // always start from beginning as not downloading // Give it a scratch buffer for flash writing uint8_t* mcuboot_buf = Util_amalloc(DFU_BUF_SZ); dfu_target_mcuboot_set_buf(mcuboot_buf, DFU_BUF_SZ); int ret = dfu_target_mcuboot_init(fsize, img_num, _dfu_cb); if (ret<0) { usb_fs_close_file(fh); Util_afree(buf); Util_afree(mcuboot_buf); log_warn("dfu : init for %s image identified as type mcuboot failed code %d", filename, ret); return false; } log_info("dfu: starting update of image for %s", filename); printk("Copying to secondard slot."); while(buf_sz==DFU_BUF_SZ) { // write the block int wret = dfu_target_mcuboot_write(buf, buf_sz); if (wret==0) { // read the next buf_sz = usb_fs_read_file(fh, buf, DFU_BUF_SZ); printk("."); } else { buf_sz = wret; printk("[WRITE FAIL %d] ", wret); } } if (buf_sz>0) { // Write the final block buf_sz = dfu_target_mcuboot_write(buf, buf_sz); printk(". DONE "); } usb_fs_close_file(fh); Util_afree(buf); Util_afree(mcuboot_buf); if (buf_sz<0) { log_warn("dfu : failed during install of file %s, %d", filename, buf_sz); dfu_target_mcuboot_done(false); // TBD : should we delete the 'bad' bin file? or try again next boot? return false; } // Success! Delete the file (we don't want it hanging about and triggering dfu again next boot time!) usb_fs_delete_file(filename); int dret = dfu_target_mcuboot_done(true); if (dret==0) { dret = dfu_target_mcuboot_schedule_update(img_num); if (dret==0) { return true; } else { log_warn("dfu : schedule_update() returns error %d, dfu filed for %s", dret, filename); } } else { log_warn("dfu : done() returns error %d, dfu filed for %s", dret, filename); } return false; } And my pm_static.yml also, as this is also a pain in the neck to get right... mcuboot: address: 0x0 end_address: 0xe000 region: flash_primary size: 0xe000 mcuboot_pad: address: 0xe000 end_address: 0xe200 region: flash_primary size: 0x200 app: address: 0xe200 end_address: 0x100000 region: flash_primary size: 0xf1e00 mcuboot_primary_app: address: 0xe200 end_address: 0x100000 region: flash_primary size: 0xf1e00 span: [app] mcuboot_primary: address: 0xe000 end_address: 0x100000 region: flash_primary size: 0xf2000 span: [mcuboot_pad, mcuboot_primary_app] # define partition for fatfs disk, on the external flash storage (probably the device defined by nordic,pm-ext-flash in the DTS) fatfs_storage: region: external_flash affiliation: - disk extra_params: { disk_name: "NAND", disk_cache_size: 4096, disk_sector_size: 512, disk_read_only: 0 } # 6Mb size, external flash is 8Mb total address: 0x0 size: 0x600000 nvs_storage: region: external_flash affiliation: - nvs # 64kB size, its just for small stuff address: 0x600000 size: 0x10000 mcuboot_secondary_1: region: external_flash affiliation: - mcuboot # 256kb size, its for netcore flash image address: 0x610000 size: 0x40000 mcuboot_secondary: region: external_flash affiliation: - mcuboot address: 0x650000 # MUST be same size as the primary application slot size, and share_size does not appear to work size: 0xf1e00 #share_size: [mcuboot_primary_app] # internal cuisine otp: address: 0xff8100 end_address: 0xff83fc region: otp size: 0x2fc pcd_sram: address: 0x20000000 end_address: 0x20002000 #placement: # after: # - start region: sram_primary size: 0x2000 rpmsg_nrf53_sram: address: 0x20070000 end_address: 0x20080000 #placement: # before: # - end region: sram_primary size: 0x10000 sram_primary: address: 0x20002000 end_address: 0x20070000 region: sram_primary size: 0x6e000 # TODO need fake flash ram flash partition to do cpunet DFU... #mcuboot_primary_1: # address: 0x0 # device: nordic_ram_flash_controller # end_address: 0x40000 # region: ram_flash # size: 0x40000 # #ram_flash: # address: 0x40000 # end_address: 0x40000 # region: ram_flash # size: 0x0 (you're welcome) Note app_update.bin is the file generated by the build, and has the mcuboot magic number at the start... I haven't yet tried the CPUNET update (as you can see the ram flash simulator is still commented out).