Need help with a custom DFU transport

RE: Need help with a custom DFU transport

Inia — Fri, 28 Oct 2022 15:30:33 GMT

Hi brooksp,

Your post is very interesting. I am in the same configuration. I want to create a custom bootloader that act as DFU controller and DFU target.

The dfu firmware (.zip) is downloaded in the context of the application and save in another flash.

Can you confirm that you achieve to make it work when you modify nrf_bootloader_init() to launch the process of dfu ? Can you confirm that you are using a firmware create by dfu (.zip file with softdevice, bootsetings, etc) ?

Thanks for your time and help.

RE: Need help with a custom DFU transport

nevi — Mon, 21 Mar 2022 20:22:45 GMT

Hi brooksp,

I am just trying to create a similar custom DFU. Could you share your results?

Thanks in advance.

RE: Need help with a custom DFU transport

brooksp — Sun, 11 Oct 2020 20:24:10 GMT

Yeah, I can probably help. Send me a message; I don't think the DevZone is the best place for collaboration.

RE: Need help with a custom DFU transport

Rycco — Wed, 07 Oct 2020 14:45:38 GMT

Hi, @brooksp

I was wondering if you would mind sharing your code/structure for this issue, We are trying to achieve exactly the same thing, having BLE DFU with another transport running from SPI memory.

If it is something you are willing to discuss/share i would really appreciate the help, as not to invent the wheel again 🙂

Thanks

RE: Need help with a custom DFU transport

Edvin — Mon, 06 Jul 2020 09:55:19 GMT

Not that I am aware of. I believe you would need some changes for the bootloader in order to change it from a "passive" to an "active" bootloader.

Glad to hear that it is working now 🙂

BR,

Edvin

RE: Need help with a custom DFU transport

brooksp — Thu, 02 Jul 2020 16:54:55 GMT

OK, I think I fixed it! I was only calling Object Execute once for the whole firmware image, instead of once per object:

I.e. Old way:

while (more data to send)
{
    Object Create
    Object Write
}
Object Execute

New way:

while (more data to send)
{
    Object Create
    Object Write
    Object Execute
}

With this change, I was able to get my DFU logic to work! The one additional thing I have to do currently is to call my DFU function from inside nrf_bootloader_init(), just before loop_forever() is called. I was really hoping to not modify the SDK at all, but I'm not sure if that's possible. If there is a workaround for this, please do let me know :)

RE: Need help with a custom DFU transport

brooksp — Thu, 02 Jul 2020 13:21:17 GMT

[quote userid="26071" url="~/f/nordic-q-a/63158/need-help-with-a-custom-dfu-transport/258001"] I would suggest leaving all interrupts, and wait for the callback.[/quote]

I was able to get Flash erase/write to work, yes, but I'm not sure how much I like my solution. Is there a way via the SDK to wait for the callback?

And for the firmware image size part:

My total firmware image size is not an even multiple of 4096. I was assume that's true for the majority of firmware images too.

I am using the same firmware image as I did in my previous bootloader (https://devzone.nordicsemi.com/f/nordic-q-a/62777/need-help-with-custom-dfu-bootloader-transferring-back-to-main-app) and was able to get that working, which leads me to believe the firmware image is good.

I've also used this firmware image successfully as a package and used it via the BLE DFU with the nRF Connect app.

In this case, my firmware image's size is 131,660 bytes. That's 32 full pages, and one incomplete page.

32 full pages = 131,072

last page = 131660 - 131,072 = 588

So when I'm writing the final page, the actual firmware image is not 4096 bytes, and that's when I'm running into issues. All previous 32 pages write successfully.

Attempt 1:

Object Create with size = 588

Object Write with size = 588

result: Object Create errors and says that "Object size must be page aligned"

Attempt 2:

Object Create with size = 4096

Object Write with size = 588

result: Object Write errors and says "Invalid data. expected: 4096, got: 588"

I'm not sure how to proceed here. The firmware image only has 588 bytes left; I'm guessing there's some way to handle this final smaller write. Do you know?

RE: Need help with a custom DFU transport

Edvin — Thu, 02 Jul 2020 08:36:41 GMT

[quote user="brooksp"]Right now this is failing when trying to erase/write to internal Flash. Here's a log:[/quote]

Did you solve this?

Erase failed with 0xD (12) is NRF_ERROR_TIMEOUT. This can e.g. be caused by not allowing the nRF to erase the flash, if you are waiting in an interrupt with a higher priority than the flash operations. The flash operations have a very low priority, so I would suggest leaving all interrupts, and wait for the callback.

[quote user="brooksp"]To fix that, I call ALIGN_TO_PAGE() on the Object Create size, but I only Object Write the actual size. After the firmware transfer is complete, I call Object Execute and I get the following error:[/quote]

So it looks like it is expecting 4096 bytes from the dfu settings, but the actual data_object_size is only 588 long, measured with:

s_dfu_settings.progress.firmware_image_offset -
s_dfu_settings.progress.firmware_image_offset_last;

So it looks like you have not stored the entire image properly, possibly because of some issues with the flash writing.

Have you tried to read out the flash of the nRF at this point in time, and checked what it looks like compared to the application hex file that you try to update to? Is it actually shorter, or is the image offset wrong, perhaps?

RE: Need help with a custom DFU transport

brooksp — Wed, 01 Jul 2020 22:36:51 GMT

OK, I'm getting close; I'm able to write from SPI Flash to Internal Flash!

Now I'm wondering about the Object Create and Object Write calls. When writing my firmware, it takes up multiple Flash pages, so I transfer the firmware one page at a time. For the last chunk, it is less than the size of a page, however Object Create says that "Object size must be page aligned", and errors.

To fix that, I call ALIGN_TO_PAGE() on the Object Create size, but I only Object Write the actual size. After the firmware transfer is complete, I call Object Execute and I get the following error:

<debug> nrf_dfu_req_handler: Handle NRF_DFU_OP_OBJECT_EXECUTE (data)
<error> nrf_dfu_req_handler: Invalid data. expected: 4096, got: 588
<debug> nrf_dfu_req_handler: Request handling complete. Result: 0x8

Now I'm not sure how to proceed. Can you help?

RE: Need help with a custom DFU transport

brooksp — Wed, 01 Jul 2020 14:38:54 GMT

Correct!

Yes, I'm using block SPI to read from my external SPI Flash.

In Version 1 of my Bootloader (the original post), I called blocking SPI commands from inside wa_dfu_transport_init(). This ended up crashing/failing because I queued up too many requests and filled up the App Scheduler's queue.

In Version 2 I added a call to app_sched_execute() immediately after I called nrf_dfu_req_handler_on_req(). However, this didn't work because nrf_dfu_req_handler::m_observer isn't initialized yet, so the Bootloader asserts/crashes when processing the Object Create/Write requests. To fix this, I need nrf_dfu_init() to be called before my update_device_firmware() is called.

In Version 3, I tried a work-around. In wa_dfu_transport_init(), I queued a new request to an unused request OP, *without* app_sched_execute(). I then set the request.callback.response to be update_device_firmware(). Unfortunately this doesn't work either. It does successfully delay calling update_device_firmware() until after nrf_dfu_init(), but the way the App Scheduler Queue works, the queue doesn't pop until the callback returns. And since my top-level function, update_device_firmware(), queues up more events, whenever app_sched_execute() runs, the first event is still update_device_firmware() (since it hasn't returned yet), which then causes update_device_firmware() to get run multiple times, and no other requests run.

In Version 4, I am modifying nrf_bootloader_init() to call my top-level function, update_device_firmware(), after nrf_dfu_init() is called. Right now this is failing when trying to erase/write to internal Flash. Here's a log:

<debug> nrf_dfu_req_handler: Handle NRF_DFU_OP_OBJECT_CREATE (data)
<debug> nrf_dfu_flash: nrf_fstorage_erase(addr=0x0x00026000, len=1 pages), queue usage: 0
<debug> nrf_dfu_flash: Flash erase failed (0xD): addr=0x00026000, len=0x1 bytes, pending 0
<debug> nrf_dfu_req_handler: Creating object with size: 4096. Offset: 0x00000000, CRC: 0x00000000
<debug> nrf_dfu_req_handler: Request handling complete. Result: 0x1

I'm investigating this currently. I'm wondering if it's a SoftDevice configuration issue? I'm not sure though. Any suggestions/recommendations are very much appreciated. Thank you!

RE: Need help with a custom DFU transport

Edvin — Wed, 01 Jul 2020 08:43:16 GMT

Hello,

Regarding the fault handler in the boot loader, in case you run into other faults at a later point in time. Check out app_error_weak.c's fault handler. ID 0x4002 refers to NRF_FAULT_ID_SDK_ASSERT, and ID 0x4001 refers to NRF_FAULT_ID_SDK_ERROR. Check out the implementation of these two in app_error_weak.c to see how to find out where the fault handler is pointing to.

[quote user="brooksp"]However, nrf_dfu_req_handler_init() hasn't been called yet[/quote]

It looks like your wa_dfu_transport_init() calls update_device_firmware()

There are two ways of using the SPI. Blocking and non-blocking. Basically, if you have an event handler, then the SPI transactions are non-blocking, and the event handler is called when a message is sent/received. If you don't have an event handler, then the SPI is blocking, so transmitting an SPI message will not return until the message has been sent.

Which of the two types do you use?

If you use the blocking SPI, do you start the process from your transport init function? Does wa_dfu_transport_init() return?

RE: Need help with a custom DFU transport

brooksp — Tue, 30 Jun 2020 18:37:20 GMT

More updates:

I've been making more progress, and now it looks like I've figured out how to do (4) from my original post. How I'm running into another issue:

So when nrf_dfu_transports_init() gets called by nrf_dfu_init(), my transport fires and begins my DFU. However, nrf_dfu_req_handler_init() hasn't been called yet. That means the nrf_dfu_req_handler::m_observer hasn't been initialized yet... So when my transport tries to do Object Create, it crashes.

Now I'm not sure where to put my DFU call now. Should I move it out of my transport init and instead call it from main(), or somewhere else? I'm gonna try moving it to main() now and will reply back when I know more.

RE: Need help with a custom DFU transport

brooksp — Tue, 30 Jun 2020 16:00:55 GMT

Update: The Fault above was due to a mismatch in my PRN Request function. I set the request type to the wrong value, which caused the callback to trigger an assert when it say the Request Type was different than expected.

RE: Need help with a custom DFU transport

brooksp — Tue, 30 Jun 2020 13:43:24 GMT

Hi Edvin, yes, let me elaborate; sorry for not being clear!

Yes, the External SPI Flash is a passive module. You are correct, it will not send anything automatically, so the Bootloader must manually read it. In effect, I'm trying to have the Bootloader be both the DFU Controller and DFU Target at once.
For BLE, no, I am not sending commands to tell the nRF to read from Flash. Really I'm trying to have two distinct and separate DFU transports. For the sake of this ticket, it's easiest to just image there is no BLE DFU.
Data gets into the SPI Flash during normal application execution (our board uses Cellular), but I'm not using the background DFU to do so.
And yes, in the Bootloader the DFU only needs to read from External SPI Flash, not write. I have confirmed that I can do this successfully.
At the of my first post, I was not seeing any callbacks. Some more info/updates below:

Today I traced down that I need to call app_sched_execute() in order for my DFU requests to get executed. I've added a call to app_sched_execute() after each of my calls to nrf_dfu_req_handler_on_req(). However now I'm seeing a Fault that I'm trying to track down. Here's a log:

<info> app: Inside main
<debug> app: In nrf_bootloader_init
<debug> nrf_dfu_settings: Calling nrf_dfu_settings_init()...
<debug> nrf_dfu_flash: Initializing nrf_fstorage_nvmc backend.
<debug> app: Enter nrf_bootloader_fw_activate
<info> app: No firmware to activate.
<debug> app: Enter nrf_dfu_app_is_valid
<debug> app: Return false in valid app check
<debug> app: DFU mode because app is not valid.
<info> nrf_bootloader_wdt: WDT is not enabled
<debug> app: in weak nrf_dfu_init_user
<info> app_timer: RTC: initialized.
<info> app: Entering DFU mode.
<debug> app: Initializing transports (found: 2)
<debug> nrf_dfu_ble: Initializing BLE DFU transport
<debug> nrf_dfu_ble: Setting up vector table: 0x000E0000
<debug> nrf_dfu_ble: Enabling SoftDevice.
<debug> nrf_dfu_ble: Configuring BLE stack.
<debug> nrf_dfu_ble: Enabling the BLE stack.
<debug> nrf_dfu_ble: No advertising name found
<debug> nrf_dfu_ble: Using default advertising name
<debug> nrf_dfu_ble: Advertising...
<debug> nrf_dfu_ble: BLE DFU transport initialized.
<debug> wa_dfu_transport: Initializing WA DFU transport...
<debug> wa_dfu_transport: { update_device_firmware
<debug> wa_dfu_req_handler: { wa_dfu_req_set_prn
<debug> nrf_dfu_req_handler: Handle NRF_DFU_OP_MTU_GET
<debug> nrf_dfu_req_handler: Request handling complete. Result: 0x1
<debug> wa_dfu_req_handler: { set_prn_callback
<error> app: Received a fault! id: 0x00004002, pc: 0x00000000, info: 0x2003FEA8

RE: Need help with a custom DFU transport

Edvin — Tue, 30 Jun 2020 13:32:15 GMT

Hello,

Just need to check. The external SPI flash is a passive module? That is, it will not send anything automatically, so you manually need to read the SPI flash from the bootloader?

I am not sure I understand your implementation correctly. Are you sending commands over BLE to tell the nRF to read from flash?

I assume that the data found in the SPI flash is transferred over BLE at some point, probably as a background DFU while you run your application?

If so, you should only need to read this back when you enter DFU mode, right?

What callbacks is it that you don't see?

BR,

Edvin