nrf52832 mesh bricks after OTA update

[Mr Sunshine]

Hello,

for clarification I use the nrf5_SDK_for_Mesh_v2.1.1.

After an OTA update (with use of the nRF toolbox app) the nrf52832 seem to be in a state of constant reboots. After some digging I see that the error happens during the mesh initialization in the "flash_manager_add()" function located at "flash_manager.c". In my case the when updating through OTA the "flash_area_is_valid()" will turn false. In addition the following is written in the function "If the area build fails, it's because we can't fit all the metadata in the action queue. Consider increasing its size."

What can I do to fix this issue? How can I increase this action queue size?  Am I looking in the right direction or am I just fixing a symptom instead of the cause?

[Edvin]

The size of the nRF52832 is 512kb flash, so I don't think your application is 939kB. The .hex file may be, but this is not translatable to the application size. Could you send the old application .hex file and the new one, and I can check. That is, the old one I can see from your screenshot. If you don't want to send the .hex file, drag and drop the new one to the File Memory layout to the right, and do the same measurement (start and end flash addresses).

Did you come any further as to why the flash_area_is_valid() returns false? If not, is there any way for me to reproduce this? Preferably, a .bat script to perform the DFU is welcome, so I don't need to set it up from scratch. It will be faster for both of us

Best regards,

Edvin

[Mr Sunshine]

Edvin said:

Could you send the old application .hex file and the new one, and I can check.

Currently I'm stuck so I'll give you the hex files:

 flash_new.hex flash_old.hex

Edvin said:

Did you come any further as to why the flash_area_is_valid() returns false?

Sorry wan't able to. I didn't had acces to the J-link, and have to do other work activities. If I look at the flash_area_is_valid() then I see that this goes deeper into scructs. 

Edvin said:

If not, is there any way for me to reproduce this? Preferably, a .bat script to perform the DFU is welcome, so I don't need to set it up from scratch. It will be faster for both of us

Don't you mean .dat file? the generated DFU zip file only includes .dat files and I don't see .bat files. I can give you the zip file so that you can reproduce this through the nRF tools app.

new_DFU.zip

EDIT:

the new DFU will not continuously reboot because I altered the code that It will ignore the first errors at the start of the program. But if you debug it you can see errors occurring during boot. I altered this for debug purposes.

In addition I went deeper into the flash_area_is_valid() code and can confirm that the error occurs at p_area[0].

[Edvin]

Mr Sunshine said:

Don't you mean .dat file? the generated DFU zip file only includes .dat files and I don't see .bat files. I can give you the zip file so that you can reproduce this through the nRF tools app.

 No, I meant a script that will perform the DFU when I run it. This way, if I compile the application, and run the script, it would take a DK and start transferring the image. How do you normally perform the DFU?

And for the new_flash.hex, I was thinking only of the hex file that is generated when you compile the application that you use to generate the DFU image later.

BR,

Edvin

[Mr Sunshine]

Edvin said:

I can give you the zip file so that you can reproduce this through the nRF tools app.

I use the nRF toolbox app on android. 

Edvin said:

And for the new_flash.hex, I was thinking only of the hex file that is generated when you compile the application that you use to generate the DFU image later.

I use the new_DFU.zip file given in my previous message for DFU updates. I can give you the application file without the bootloader and softdevice if that is what you mean.

 6180.application.hex

in addition (I made a edit in my last message) I went deeper into the flash_area_is_valid() code and I see that the error occurs at p_area[0] of the struct "&p_manager->config.p_area[i].metadata".

[Edvin]

Which of the checks in:

static inline bool metadata_is_valid(const flash_manager_metadata_t * p_metadata)
{
    return (p_metadata->metadata_len != 0xFF &&
            p_metadata->metadata_len >= 8 &&
            p_metadata->page_index != 0xFF &&
            p_metadata->pages_in_area != 0xFF);
}

is it that returns false?

And a difficult question: What flash area is it looking at? To answer this, you need to check where is  flash_manager_add() called from in the case where it fails? add_flash_manager(), init_flash_storage() or build_flash_area()? And what is manager_config in that case?

In case:

static void add_flash_manager(void)
{

    static fm_mem_listener_t flash_add_mem_available_struct = {
        .callback = flash_manager_mem_available,
        .p_args = add_flash_manager
    };
    flash_manager_config_t manager_config;
    manager_config.write_complete_cb = flash_write_complete;
    manager_config.invalidate_complete_cb = flash_invalidate_complete;
    manager_config.remove_complete_cb = flash_remove_complete;
    manager_config.min_available_space = WORD_SIZE;
#ifdef ACCESS_FLASH_AREA_LOCATION
    manager_config.p_area = (const flash_manager_page_t *) ACCESS_FLASH_AREA_LOCATION;
#else
    manager_config.p_area = (const flash_manager_page_t *) (((const uint8_t *) dsm_flash_area_get()) - (ACCESS_FLASH_PAGE_COUNT * PAGE_SIZE));
#endif
    manager_config.page_count = ACCESS_FLASH_PAGE_COUNT;
    m_flash_not_ready = true;
    uint32_t status = flash_manager_add(&m_flash_manager, &manager_config);
    if (NRF_SUCCESS != status)
    {
        flash_manager_mem_listener_register(&flash_add_mem_available_struct);
    }
    else
    {
        m_flash_not_ready = false;
    }
}

What is dsm_flash_get()? What address is it pointing to?

In case:

static void build_flash_area(void)
{
    bool success = false;
    flash_manager_config_t manager_config;
    manager_config.write_complete_cb      = flash_write_complete;
    manager_config.invalidate_complete_cb = flash_invalidate_complete;
    manager_config.remove_complete_cb     = flash_remove_complete;
    manager_config.min_available_space    = 0;
    manager_config.p_area = dsm_flash_area_get();
    manager_config.page_count = DSM_FLASH_PAGE_COUNT;

    /* Lock the bearer event handler to ensure that we don't enter and leave the BUILDING state
     * between adding and checking. */
    bearer_event_critical_section_begin();
    if (flash_manager_add(&m_flash_manager, &manager_config) == NRF_SUCCESS)
    {
        /* If we have to build the flash manager, it means that it's new, and there's no metainfo. */
        if (m_flash_manager.internal.state == FM_STATE_BUILDING)
        {
            success = flash_store_metainfo();
        }
        else
        {
            success = true;
        }
    }
    bearer_event_critical_section_end();

    if (success)
    {
        m_flash_is_available = true;
    }
    else
    {
        /* Register the listener and wait for some memory to be freed up before we retry. */
        static fm_mem_listener_t mem_listener = {.callback = flash_mem_listener_callback,
                                                 .p_args = build_flash_area};
        flash_manager_mem_listener_register(&mem_listener);
    }
}

What is: manager_config.p_area = dsm_flash_area_get(); ?

In case:

static void init_flash_storage(void)
{
    flash_manager_config_t config;
    memset(&config, 0, sizeof(config));
    config.min_available_space = 0;
    config.p_area = net_state_flash_area_get();
    config.page_count = NET_FLASH_PAGE_COUNT;
    config.write_complete_cb = flash_write_complete;

    if (flash_manager_add(&m_flash_manager, &config) != NRF_SUCCESS)
    {
        static fm_mem_listener_t mem_listener = {.callback = flash_mem_available,
                                                 .p_args = init_flash_storage};
        flash_manager_mem_listener_register(&mem_listener);
    }
}

What is:

config.p_area = net_state_flash_area_get(); ?

In case you are curious, the net_state_flash_area_get() fetches the bootloader address:

net_state_flash_area_get() -> flash_manager_recovery_page_get() -> flash_manager_defrag_recovery_page_get() -> mp_recovery_area,

which is set in flash_manager_defrag_init():

mp_recovery_area = p_flash_end - FLASH_MANAGER_RECOVERY_PAGE_OFFSET_PAGES - 1; /* pointer arithmetic */

where p_flash_end is:

p_flash_end = (flash_manager_recovery_area_t *) (BOOTLOADERADDR() - PAGE_SIZE);

Which was why I asked for that earlier.

So, what area is it trying to initialize the flash manager on, and which of the checks is it that fails, and what is currently in the flash where you are trying to use?

As for the application .hex file, it has start address 0x26000 and end address 0x4545F, so it is 0x1F45F large (rounded up to 0x20000)

Your screenshot from nRF Connect programmer:

The previous application stops at 0x4BD58 (effectively 0x4C000), and the Flash manager starts at 0x6B000. This means that the free space between the two is 0x6B000-0x4C000 = 0x1F000. But since you are using a BLE bootloader, that should be fine (not background). However, you may want to check that your bootloader saves the correct amount of flash for FDS (flash manager). 

What is your DFU_APP_DATA_RESERVED defined as in sdk_config.h in your bootloader? Depending on what SDK version you use, it may have different, but similar name.

I your case this size should reflect the distance from the bootloader area 0x70000 to the start of your flash manager (0x6B000), so try setting it to 20480 (or alternatively, 5*CODE_PAGE_SIZE).

BR,

Edvin