Setting up App Data preservation with DFU

[erosembedded]

I have the BMD350 from Rigado. I am using Segger embedded studio with SDK v 15.2.0

I have the buttonless DFU working with my custom application that has the buttonless DFU and the nordic uart service (NUS).

I would like to use the app data section of flash that does not get written during a DFU. I would like to store application configuration data there.

I am having a hard time finding clear examples of doing this. I have read some topics mentioning the "pstorage_platform",  the "device manager",
and have seen some hard coded addresses. Also have read many topics about the "dfu_types" to edit the DFU_APP_DATA_RESERVED value.
This has not lead me to a path to get started.

I'm looking in the "flash_placement.xml" file and do not see a section dedicated to "app data" or "user data" or "pstorage", so I'm thinking it's not as simple as defining a C array or structure
with the "SECTION" attribute in the linker?

If you could point me to an example of using this application data section of flash or list a quick step by step getting started that would be great.

Below is the segger flash_placement.xml for my project.

<!DOCTYPE Linker_Placement_File>
<Root name="Flash Section Placement">
  <MemorySegment name="FLASH" start="$(FLASH_PH_START)" size="$(FLASH_PH_SIZE)">
    <ProgramSection load="no" name=".reserved_flash" start="$(FLASH_PH_START)" size="$(FLASH_START)-$(FLASH_PH_START)" />
    <ProgramSection alignment="0x100" load="Yes" name=".vectors" start="$(FLASH_START)" />
    <ProgramSection alignment="4" load="Yes" name=".init" />
    <ProgramSection alignment="4" load="Yes" name=".init_rodata" />
    <ProgramSection alignment="4" load="Yes" name=".text" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".sdh_ble_observers" inputsections="*(SORT(.sdh_ble_observers*))" address_symbol="__start_sdh_ble_observers" end_symbol="__stop_sdh_ble_observers" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".sdh_soc_observers" inputsections="*(SORT(.sdh_soc_observers*))" address_symbol="__start_sdh_soc_observers" end_symbol="__stop_sdh_soc_observers" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".pwr_mgmt_data" inputsections="*(SORT(.pwr_mgmt_data*))" address_symbol="__start_pwr_mgmt_data" end_symbol="__stop_pwr_mgmt_data" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".log_const_data" inputsections="*(SORT(.log_const_data*))" address_symbol="__start_log_const_data" end_symbol="__stop_log_const_data" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".sdh_stack_observers" inputsections="*(SORT(.sdh_stack_observers*))" address_symbol="__start_sdh_stack_observers" end_symbol="__stop_sdh_stack_observers" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".sdh_req_observers" inputsections="*(SORT(.sdh_req_observers*))" address_symbol="__start_sdh_req_observers" end_symbol="__stop_sdh_req_observers" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".sdh_state_observers" inputsections="*(SORT(.sdh_state_observers*))" address_symbol="__start_sdh_state_observers" end_symbol="__stop_sdh_state_observers" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".log_backends" inputsections="*(SORT(.log_backends*))" address_symbol="__start_log_backends" end_symbol="__stop_log_backends" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".nrf_balloc" inputsections="*(.nrf_balloc*)" address_symbol="__start_nrf_balloc" end_symbol="__stop_nrf_balloc" />
    <ProgramSection alignment="4" keep="Yes" load="No" name=".nrf_sections" address_symbol="__start_nrf_sections" />
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".log_dynamic_data"  inputsections="*(SORT(.log_dynamic_data*))" runin=".log_dynamic_data_run"/>
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".log_filter_data"  inputsections="*(SORT(.log_filter_data*))" runin=".log_filter_data_run"/>
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".fs_data"  inputsections="*(.fs_data*)" runin=".fs_data_run"/>
    <ProgramSection alignment="4" load="Yes" name=".dtors" />
    <ProgramSection alignment="4" load="Yes" name=".ctors" />
    <ProgramSection alignment="4" load="Yes" name=".rodata" />
    <ProgramSection alignment="4" load="Yes" name=".ARM.exidx" address_symbol="__exidx_start" end_symbol="__exidx_end" />
    <ProgramSection alignment="4" load="Yes" runin=".fast_run" name=".fast" />
    <ProgramSection alignment="4" load="Yes" runin=".data_run" name=".data" />
    <ProgramSection alignment="4" load="Yes" runin=".tdata_run" name=".tdata" />
  </MemorySegment>
  <MemorySegment name="RAM" start="$(RAM_PH_START)" size="$(RAM_PH_SIZE)">
    <ProgramSection load="no" name=".reserved_ram" start="$(RAM_PH_START)" size="$(RAM_START)-$(RAM_PH_START)" />
    <ProgramSection alignment="0x100" load="No" name=".vectors_ram" start="$(RAM_START)" address_symbol="__app_ram_start__"/>
    <ProgramSection alignment="4" keep="Yes" load="No" name=".nrf_sections_run" address_symbol="__start_nrf_sections_run" />
    <ProgramSection alignment="4" keep="Yes" load="No" name=".log_dynamic_data_run" address_symbol="__start_log_dynamic_data" end_symbol="__stop_log_dynamic_data" />
    <ProgramSection alignment="4" keep="Yes" load="No" name=".log_filter_data_run" address_symbol="__start_log_filter_data" end_symbol="__stop_log_filter_data" />
    <ProgramSection alignment="4" keep="Yes" load="No" name=".fs_data_run" address_symbol="__start_fs_data" end_symbol="__stop_fs_data" />
    <ProgramSection alignment="4" keep="Yes" load="No" name=".nrf_sections_run_end" address_symbol="__end_nrf_sections_run" />
    <ProgramSection alignment="4" load="No" name=".fast_run" />
    <ProgramSection alignment="4" load="No" name=".data_run" />
    <ProgramSection alignment="4" load="No" name=".tdata_run" />
    <ProgramSection alignment="4" load="No" name=".bss" />
    <ProgramSection alignment="4" load="No" name=".tbss" />
    <ProgramSection alignment="4" load="No" name=".non_init" />
    <ProgramSection alignment="4" size="__HEAPSIZE__" load="No" name=".heap" />
    <ProgramSection alignment="8" size="__STACKSIZE__" load="No" place_from_segment_end="Yes" name=".stack"  address_symbol="__StackLimit" end_symbol="__StackTop"/>
    <ProgramSection alignment="8" size="__STACKSIZE_PROCESS__" load="No" name=".stack_process" />
  </MemorySegment>
  <MemorySegment name="uicr_bootloader_start_address" start="0x10001014" size="0x4">
    <ProgramSection alignment="4" keep="Yes" load="Yes" name=".uicr_bootloader_start_address" address_symbol="__start_uicr_bootloader_start_address" end_symbol="__stop_uicr_bootloader_start_address" start = "0x10001014" size="0x4" />
  </MemorySegment>
</Root>

[Vidar Berg]

Hi, 

Device manager and pstorage have been replaced with the peer manager and FDS/fstorage. Instructions on how you can implement persistent storage depends on whether you intend to use the peer manager (support BLE pairing) or not and if you want to use fstorage or directly or use FDS on top (file system). Please let me know if you are using the peer manager and if you prefer fds or fstorage. 

It's not necessary to define the app data section in your linker script but you need to make sure that the application does not grow into this section. 

[erosembedded]

The  buttonless BLE DFU PCA100040 example uses the peer manager so it is also in my code as well.

It has this function:

/**@brief Function for the Peer Manager initialization.
 */
static void peer_manager_init()
{
    ble_gap_sec_params_t sec_param;
    ret_code_t           err_code;

    err_code = pm_init();
    APP_ERROR_CHECK(err_code);

    memset(&sec_param, 0, sizeof(ble_gap_sec_params_t));

    // Security parameters to be used for all security procedures.
    sec_param.bond           = SEC_PARAM_BOND;
    sec_param.mitm           = SEC_PARAM_MITM;
    sec_param.lesc           = SEC_PARAM_LESC;
    sec_param.keypress       = SEC_PARAM_KEYPRESS;
    sec_param.io_caps        = SEC_PARAM_IO_CAPABILITIES;
    sec_param.oob            = SEC_PARAM_OOB;
    sec_param.min_key_size   = SEC_PARAM_MIN_KEY_SIZE;
    sec_param.max_key_size   = SEC_PARAM_MAX_KEY_SIZE;
    sec_param.kdist_own.enc  = 1;
    sec_param.kdist_own.id   = 1;
    sec_param.kdist_peer.enc = 1;
    sec_param.kdist_peer.id  = 1;

    err_code = pm_sec_params_set(&sec_param);
    APP_ERROR_CHECK(err_code);

    err_code = pm_register(pm_evt_handler);
    APP_ERROR_CHECK(err_code);
}

I do not need to use a file system. I just need to define a simple C structure with maybe 1kBytes of data max. I need to be able to update this app data in the application code as well. If the configuration changes or the data structure changes I need to write new data.

It is also important that I be able to define the structure I put in the app data as "packed" (packed attribute) and sub structures within it as packed as well. It also needs to be volatile as well

So I'm guess using fstorage directly with peer manager as you mentioned would be best?

[Vidar Berg]

You may know this but there are a few things to keep in mind when dealing with flash storage. 1. flash must be erased before you can write to it. I.e.,, to update existing data in flash you need to ensure that the flash page you are writing to is erased. 2. Flash can only be erased in units of flash pages (4kB on nRF52 series). 3. Page erase is a relatively time consuming task (<85 ms)  and the softdevice may fail do schedule it if there's too much BLE activity (short connection interval, etc), see Flash API for more information on how the scheduling works.

Generally I think FDS is the best option, especially if you need to update data regularly. Instead of erasing a flash page every time existing data is updated FDS will invalidate the old record by writing one bit then write a new one. When there is no more flash pages you can run garbage collection to remove invalidated data.

erosembedded said:

I do not need to use a file system. I just need to define a simple C structure with maybe 1kBytes of data max. I need to be able to update this app data in the application code as well. If the configuration changes or the data structure changes I need to write new data.

FDS/Fstorage does not care about data types but the data source must be from one contiguous block in memory and the length must be a multiple of 4 bytes (word). It might make sense to split up the structure in smaller data records if you parts of the structure is being updated more frequently. 

erosembedded said:

It is also important that I be able to define the structure I put in the app data as "packed" (packed attribute) and sub structures within it as packed as well. It also needs to be volatile as well

FDS/Fstorage does not care whether the structure is packed or not. I'm not sure what you mean by 'volatile' in this context but the source data must be kept in static memory until the flash operation is completed.

The following examples demonstrate usage of  fstorage and FDS in the SDK:

Flash Data Storage Example

Flash Storage Example

Note: if you end up using FDS, it is not necessary to initialize it as it being done by the PM. Just register a new user.

void flash_callback(fds_evt_t const * p_evt)
{
    switch (p_evt->id)
    {
        case FDS_EVT_INIT:
            if(p_evt->result == FDS_SUCCESS)
            {
                NRF_LOG_INFO("FDS initialized");
            }
            break;
            
        //TODO: add fds event handling
}

void flash_init(void)
{
    uint32_t err_code;

    err_code = fds_register(flash_callback);
    APP_ERROR_CHECK(err_code);
}

[erosembedded]

FDS sounds like the way to go.

I had a question, does the FDS automatically use the persistent "app data" region in flash that the bootloader does not write to? I don't see any set up for where the file system is located. Is it set up so that whatever I write to the file system will not be overwritten by the bootloader?

For Volatile, I meant define it so the compiler will not optimize out any usage I have of the data. But using the FDS may get rid of the need for that. 

I don't plan on writing to my configuration file very often. It will only be written to once in a while with long periods (days, weeks) in between.

[Vidar Berg]

Yes, FDS will automatically use the app data region which gets placed below the bootloader as shown here. The region end add address (where bootloader starts) is found by reading out the BL start address stored in NRF_UICR->NRFFW[0], see fds.c ->flash_end_addr() .

You may chose how many flash pages you want allocated to FDS by changing FDS_VIRTUAL_PAGES define in sdk_config.h. But note that APP_DATA_RESERVED define in the bootloader code must correspond with FDS_VIRTUAL_PAGES value used by the app to ensure that user data is not erased by a DFU. 

erosembedded said:

For Volatile, I meant define it so the compiler will not optimize out any usage I have of the data. But using the FDS may get rid of the need for that. 

FDS will simply copy the data from a given source address so you should assess your code to determine the structure should be declared volatile.