nrf5 SDK for Thread : Combining OTA DFU with MQTT-SN Examples

[Brian (Geek)]

Hello Developers,

I am currently developing an application on top of mqtt-sn publisher example. Now i want to add OTA DFU in my mqtt-sn based application. I have started up with simple dfu example present in Thread folder and it works fine. Than i created few firmware packages of my mqtt-sn based applications and updated it on the device flashed with dfu client example and it also works fine. Now i am looking to flash the end device with mqttsn client example integrated with dfu client example and than update it over the air with another mqtt-sn client example. I am using Segger Embedded Studio for development with, nRF52840 DK and nRF SDK for Thread and Zigbee SDK v4.1 .


I have followed this post on devzone along with dfu example docs from infocenter and started to integrate dfu client example and my mqtt-sn based example in Segger Embedded Studio:
-
https://devzone.nordicsemi.com/f/nordic-q-a/73401/ota-thread-dfu-merge-sdk-example-with-custom-application/307331#307331


I have attached my flash_placement.xml file based on the dfu client example, please refer it and tell me if i have to make any changes.

Fullscreen 1425.flash_placement.xml Download

<!DOCTYPE Linker_Placement_File>
<Root name="Flash Section Placement">
  <MemorySegment name="FLASH" start="$(FLASH_PH_START)" size="$(FLASH_PH_SIZE)">
    <ProgramSection load="no" name=".reserved_flash" start="$(FLASH_PH_START)" size="$(FLASH_START)-$(FLASH_PH_START)" />
    <ProgramSection alignment="0x100" load="Yes" name=".vectors" start="$(FLASH_START)" />
    <ProgramSection alignment="4" load="Yes" name=".init" />
    <ProgramSection alignment="4" load="Yes" name=".init_rodata" />
    <ProgramSection alignment="4" load="Yes" name=".text" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".crypto_data" inputsections="*(SORT(.crypto_data*))" address_symbol="__start_crypto_data" end_symbol="__stop_crypto_data" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".nrf_queue" inputsections="*(.nrf_queue*)" address_symbol="__start_nrf_queue" end_symbol="__stop_nrf_queue" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".pwr_mgmt_data" inputsections="*(SORT(.pwr_mgmt_data*))" address_symbol="__start_pwr_mgmt_data" end_symbol="__stop_pwr_mgmt_data" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".log_const_data" inputsections="*(SORT(.log_const_data*))" address_symbol="__start_log_const_data" end_symbol="__stop_log_const_data" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".log_backends" inputsections="*(SORT(.log_backends*))" address_symbol="__start_log_backends" end_symbol="__stop_log_backends" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".nrf_balloc" inputsections="*(.nrf_balloc*)" address_symbol="__start_nrf_balloc" end_symbol="__stop_nrf_balloc" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".dfu_trans" inputsections="*(SORT(.dfu_trans*))" address_symbol="__start_dfu_trans" end_symbol="__stop_dfu_trans" />
    <ProgramSection alignment="4" keep="Yes" load="No" name=".nrf_sections" address_symbol="__start_nrf_sections" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".fs_data" inputsections="*(.fs_data*)" runin=".fs_data_run" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".log_dynamic_data" inputsections="*(SORT(.log_dynamic_data*))" runin=".log_dynamic_data_run" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".log_filter_data" inputsections="*(SORT(.log_filter_data*))" runin=".log_filter_data_run" />
    <ProgramSection alignment="4" load="Yes" name=".dtors" />
    <ProgramSection alignment="4" load="Yes" name=".ctors" />
    <ProgramSection alignment="4" load="Yes" name=".rodata" />
    <ProgramSection alignment="4" load="Yes" name=".ARM.exidx" address_symbol="__exidx_start" end_symbol="__exidx_end" />
    <ProgramSection alignment="4" load="Yes" runin=".fast_run" name=".fast" />
    <ProgramSection alignment="4" load="Yes" runin=".data_run" name=".data" />
    <ProgramSection alignment="4" load="Yes" runin=".tdata_run" name=".tdata" />
  </MemorySegment>
  <MemorySegment name="RAM" start="$(RAM_PH_START)" size="$(RAM_PH_SIZE)">
    <ProgramSection load="no" name=".reserved_ram" start="$(RAM_PH_START)" size="$(RAM_START)-$(RAM_PH_START)" />
    <ProgramSection alignment="0x100" load="No" name=".vectors_ram" start="$(RAM_START)" address_symbol="__app_ram_start__" />
    <ProgramSection alignment="4" keep="Yes" load="No" name=".nrf_sections_run" address_symbol="__start_nrf_sections_run" />
    <ProgramSection alignment="4" keep="Yes" load="No" name=".fs_data_run" address_symbol="__start_fs_data" end_symbol="__stop_fs_data" />
    <ProgramSection alignment="4" keep="Yes" load="No" name=".log_dynamic_data_run" address_symbol="__start_log_dynamic_data" end_symbol="__stop_log_dynamic_data" />
    <ProgramSection alignment="4" keep="Yes" load="No" name=".log_filter_data_run" address_symbol="__start_log_filter_data" end_symbol="__stop_log_filter_data" />
    <ProgramSection alignment="4" keep="Yes" load="No" name=".nrf_sections_run_end" address_symbol="__end_nrf_sections_run" />
    <ProgramSection alignment="4" load="No" name=".fast_run" />
    <ProgramSection alignment="4" load="No" name=".data_run" />
    <ProgramSection alignment="4" load="No" name=".tdata_run" />
    <ProgramSection alignment="4" load="No" name=".bss" />
    <ProgramSection alignment="4" load="No" name=".tbss" />
    <ProgramSection alignment="4" load="No" name=".non_init" />
    <ProgramSection alignment="4" size="__HEAPSIZE__" load="No" name=".heap" />
    <ProgramSection alignment="8" size="__STACKSIZE__" load="No" place_from_segment_end="Yes" name=".stack" address_symbol="__StackLimit" end_symbol="__StackTop" />
    <ProgramSection alignment="8" size="__STACKSIZE_PROCESS__" load="No" name=".stack_process" />
  </MemorySegment>
  <MemorySegment name="mbr_params_page" start="0x000FE000" size="0x1000">
    <ProgramSection alignment="4" keep="Yes" load="No" name=".mbr_params_page" address_symbol="__start_mbr_params_page" end_symbol="__stop_mbr_params_page" start="0x000FE000" size="0x1000" />
  </MemorySegment>
  <MemorySegment name="bootloader_settings_page" start="0x000FF000" size="0x1000">
    <ProgramSection alignment="4" keep="Yes" load="No" name=".bootloader_settings_page" address_symbol="__start_bootloader_settings_page" end_symbol="__stop_bootloader_settings_page" start="0x000FF000" size="0x1000" />
  </MemorySegment>
  <MemorySegment name="ot_flash_data" start="0xf4000" size="0x4000">
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".ot_flash_data" address_symbol="__start_ot_flash_data" end_symbol="__stop_ot_flash_data" start="0xf4000" size="0x4000" />
  </MemorySegment>
  <MemorySegment name="uicr_bootloader_start_address" start="0x10001014" size="0x4">
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".uicr_bootloader_start_address" address_symbol="__start_uicr_bootloader_start_address" end_symbol="__stop_uicr_bootloader_start_address" start="0x10001014" size="0x4" />
  </MemorySegment>
  <MemorySegment name="uicr_mbr_params_page" start="0x10001018" size="0x4">
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".uicr_mbr_params_page" address_symbol="__start_uicr_mbr_params_page" end_symbol="__stop_uicr_mbr_params_page" start="0x10001018" size="0x4" />
  </MemorySegment>
</Root>

I have also attached the memory view of my application :

MQTT-SN DFU Client

So my mqtt-sn client example integrated with dfu client example and interfaced with sensor after build takes 437.0K code area and 209.9K  Data+RO area as per segger studio. Now in Memory Usage out of the 1MB flash 511.8KB of flash is used up and 135KB RAM is used up.

I have attached the image for reference.:

- Now i flash MBR, bootloader, and the merged hex file generated using application hex and bootloader setting hex on my end device. I can see my device gets connected to the network.

Here is my boot-loader settings file generated for my MQTT-SN based application:

Here is the memory read layout from the nRF Connect Programmer :




MQTT-SN DFU Client (To be updated via OTA)
- I have changed the sensor and built the project it takes approximately the same memory space as above. I have created a package file and started DFU process:

Three things happened:
1) One-time DFU process got stuck at 98percent and the DFU client running also got stopped

2) One-time DFU process was completed but the update didn't happen. The client got disconnected from the broker and got connected but with the previous firmware.

3) And one time my MQTT-sn firmware example got updated successfully. 

Can be a Problem and How to Solve this issue :
1. I think this can be the issue of flash memory as my hex file combined with other segments in flash make it around 500KB.


Secondly when I flash the end device with DFU Client example and then update it with MQTT-SN firmware 1. The firmware 1 starts running. Then on the same device running firmware 1, I try to update a new firmware MQTTSN firmware 2 and this firmware also gets updated successfully. So from this, I can also think there can be an issue with the creation of MQTT-SN-based DFU client.

SECONDLY, I WOULD ALSO LIKE TO KNOW HOW CAN I USE EXTERNAL FLASH FOR DFU Example because my example build may get increased in size. 

[Edvin]

Three things happened:
1) One-time DFU process got stuck at 98percent and the DFU client running also got stopped

2) One-time DFU process was completed but the update didn't happen. The client got disconnected from the broker and got connected but with the previous firmware.

3) And one time my MQTT-sn firmware example got updated successfully. 

Obviously, this didn't happen all at the same time. Do you mean that when you tried, either of these could happen, when you were running the same application, and tried to upgrade using the same DFU image?

Can be a Problem and How to Solve this issue :
1. I think this can be the issue of flash memory as my hex file combined with other segments in flash make it around 500KB.

When you are using the Openthread bootloader, you need to fit all .hex files at once, so this may be a memory issue. It needs to fit the bootloader (Red line on the nRF Connect Programmer screenshot), bootloader settings pages (black lines) , the openthread persistent storage (the green line below the bootloader), the mbr (orange) and the old application, and the new. Please note that the thickness of the lines in nRF Connect Programmer isn't proportional to their actual flash sizes. 

With a flash size around 500KB you are on the very limit. A workaround for this is to perform two DFUs. The first one is just to upgrade to an application that takes much less space, so that you have room for the new one. You can try this if you suspect that the size is the limit. 

If that doesn't seem to be the case, when the update fails, does it look like the nRF52840 enters bootloader mode? Does the log from the DFU client, or the DFU server say anything? If you are using the DK, do the LEDs indicate Bootloader mode? Have you tried enabling logging on the bootloader project? If you do this, you probably have to test with a smaller application, since this requires you to increase the size of the bootloader, and change the bootloader's start address.

Best regards,

Edvin

[Brian (Geek)]

Hello, I have implemented DFU with the MQTT-SN example and it works very well when the devices are near to each other but it takes more time when I keep devices at some distance.  When DFU clients are kept far from the server the DFU gets stuck more often and I have to restart it many times to complete the full DFU process. Whenever I restart the DFU, it starts from where it got interrupted previously. 

Edvin said:

With a flash size around 500KB you are on the very limi

Do you mean flash size of 500KB for the downloaded firmware during the thread DFU process? Because as per my knowledge the running application is stored in bank0 while bank1 stores the downloaded firmware during the device firmware update process before getting swapped out by the bootloader.  

Edvin said:

A workaround for this is to perform two DFUs. The first one is just to upgrade to an application that takes much less space, so that you have room for the new one.

So can you please elaborate on your suggestion regarding multiple DFUs? And can't we try to store newer applications on external memory like QSPI flash on DK. 

Also now I am facing new problems as mentioned below. I am trying device firmware update on 4 devices: 

Problem 1: Sometimes when I run the Unicast DFU the server gets stuck at  " Thread DFU server is running... Press <Ctrl + D> to stop " . I have waited for 5-10 minutes too but no response. 

Problem 2: How to increase the data rate of DFU. It takes me around 30 minutes 482KB zip file.  I can see there is a -r parameter for rate but didn't find any valid value and doc of how to use it.  I have tried passing a few values but didn't affect anything. 

Problem 3: Sometimes when my DFU update is currently going on it gives me this type of error (Cannot convert the argument to integer). I have faced this issue very frequently, what can be the reason behind it?  And how to solve the same. 

 

Problem 4: Sometimes even after successful completion of DFU on the server-side the clients are not getting updated with the new firmware. So what could be the reason for the error in post validation?  

And what's the difference between the unicast (without passing IP address) and multicast DFU command,  as both take ff03::1 as address. Also the unicast in this case is more informative as I am able to see the IP address on which the data is getting updated, while in multicast I am not even able to check whether even any device is getting updated or not.  

[Edvin]

Brian (Geek) said:

Do you mean flash size of 500KB for the downloaded firmware during the thread DFU process? Because as per my knowledge the running application is stored in bank0 while bank1 stores the downloaded firmware during the device firmware update process before getting swapped out by the bootloader.  

It is correct that there is a bank0 and bank1, but these are dynamic, so if the application is large, bank1 starts at a higher address. If however, the current application is very small, then bank1 is larger than bank0. 

Perhaps it is possible to store the application on an external flash device, but for that question, please open a new ticket, as I am not familiar with that process.

Brian (Geek) said:

Problem 1: Sometimes when I run the Unicast DFU the server gets stuck at  " Thread DFU server is running... Press <Ctrl + D> to stop " . I have waited for 5-10 minutes too but no response. 

Does it look like the DFU server is able to join the network when it is stuck in this state?

Brian (Geek) said:

Problem 2: How to increase the data rate of DFU. It takes me around 30 minutes 482KB zip file.  I can see there is a -r parameter for rate but didn't find any valid value and doc of how to use it.  I have tried passing a few values but didn't affect anything. 

Check out "nrfutil dfu thread --help". It states:

-r, --rate FLOAT                Multicast upload rate in blocks per second.

I know that 30 mins seems slow, but please remember that DFU in mesh networks (Zigbee, OpenThread, Bluetooth Mesh) is slow. The reason for this is that the throughput of such networks is in general slow, and you wouldn't want to flood your network with these packets, because it may block other packets from being treated in the network. Remember that in an end user usecase, the user will not notice that the DFU is taking place, because the old application is running in the background while the DFU is taking place. 

Brian (Geek) said:

Problem 3: Sometimes when my DFU update is currently going on it gives me this type of error (Cannot convert the argument to integer). I have faced this issue very frequently, what can be the reason behind it?  And how to solve the same. 

I see that you are using python 2.7. Try upgrading to python 3.8. I have not seen this error in particular before, but at least the later versions of nrfutil is using python 3.8 as a preferred python version. Try the combo:

Brian (Geek) said:

Problem 4: Sometimes even after successful completion of DFU on the server-side the clients are not getting updated with the new firmware. So what could be the reason for the error in post validation? 

Are you sure the packet is correctly set up using the correct bootloader and public/private key combination? I know it is a bit difficult, but as I mentioned in my first reply, can you try to enable logging in the bootloader? This requires you to change the bootloader's start address. Let me know what IDE you are using if you need help with this. It will also require you to use a smaller app, so that you can fit the larger bootloader and the new application image. Perhaps you can start with just the dfu_client, and then try to update to your current application. (The reason being, as I mentioned, that the bank sizes are dynamic, depending on your current application size.

Best regards,

Edvin