Struggling to write in the flash memory correctly.

[Giannis Anastasopoulos]

Dear all,

I am trying to adopt the flash_fds example to fit my needs. What I initially need to get sorted out is how can I erase the memory pages after I have fully written on them. I have changed the handler of the example with the following code:

static void fds_evt_handler(fds_evt_t const * p_evt)
{
    NRF_LOG_GREEN("Event: %s received (%s)",
                  fds_evt_str[p_evt->id],
                  fds_err_str[p_evt->result]);

    switch (p_evt->id)
    {
        case FDS_EVT_INIT:
            if (p_evt->result == FDS_SUCCESS)
            {
                m_fds_initialized = true;
            }
            break;

        case FDS_EVT_WRITE:
        {
            if (p_evt->result == FDS_SUCCESS)
            {
                NRF_LOG_GREEN("Record ID:\t0x%04x",  p_evt->write.record_id);
                NRF_LOG_INFO("File ID:\t0x%04x",    p_evt->write.file_id);
                NRF_LOG_INFO("Record key:\t0x%04x", p_evt->write.record_key);
            }
            else 
            {
              delete_all_begin();
              fds_gc();
            }
        } break;

        case FDS_EVT_DEL_RECORD:
        {
            if (p_evt->result == FDS_SUCCESS)
            {
                NRF_LOG_INFO("Record ID:\t0x%04x",  p_evt->del.record_id);
                NRF_LOG_INFO("File ID:\t0x%04x",    p_evt->del.file_id);
                NRF_LOG_INFO("Record key:\t0x%04x", p_evt->del.record_key);
            }
            m_delete_all.pending = false;
        } break;

        default:
            break;
    }
}

The change that I tried to make is that in case the p_evt->result doesn't return SUCCESS, then I will print a simple message in the terminal. Ideally I would like to print my message when p_evt->result = FDS_ERR_NO_SPACE_IN_FLASH or p_evt->result = FDS_ERR_NO_SPACE_IN_QUEUES.

In the main loop I am updating the configuration entry to the point that I am exhausting the memory that I have allocated for my project. This part appears to work, although there is a problem with the APP_ERROR_CHECK(rc), that I am also unable to figure out.

My whole main looks like this:

int main(void)
{
    ret_code_t rc;

#ifdef SOFTDEVICE_PRESENT
    ble_stack_init();
#else
    clock_init();
#endif

    timer_init();
    log_init();
    cli_init();

    NRF_LOG_INFO("FDS example started.")

    /* Register first to receive an event when initialization is complete. */
    (void) fds_register(fds_evt_handler);

    NRF_LOG_INFO("Initializing fds...");

    rc = fds_init();
    APP_ERROR_CHECK(rc);

    /* Wait for fds to initialize. */
    wait_for_fds_ready();

    NRF_LOG_INFO("Available commands:");
    NRF_LOG_INFO("- print all\t\tprint records");
    NRF_LOG_INFO("- print config\tprint configuration");
    NRF_LOG_INFO("- update\t\tupdate configuration");
    NRF_LOG_INFO("- stat\t\tshow statistics");
    NRF_LOG_INFO("- write\t\twrite a new record");
    NRF_LOG_INFO("- delete\t\tdelete a record");
    NRF_LOG_INFO("- delete_all\tdelete all records");
    NRF_LOG_INFO("- gc\t\trun garbage collection");

    NRF_LOG_INFO("Reading flash usage statistics...");

    fds_stat_t stat = {0};

    rc = fds_stat(&stat);
    APP_ERROR_CHECK(rc);

    NRF_LOG_INFO("Found %d valid records.", stat.valid_records);
    NRF_LOG_INFO("Found %d dirty records (ready to be garbage collected).", stat.dirty_records);

    fds_record_desc_t desc = {0};
    fds_find_token_t  tok  = {0};

    rc = fds_record_find(CONFIG_FILE, CONFIG_REC_KEY, &desc, &tok);

    if (rc == FDS_SUCCESS)
    {
        /* A config file is in flash. Let's update it. */
        fds_flash_record_t config = {0};

        /* Open the record and read its contents. */
        rc = fds_record_open(&desc, &config);
        APP_ERROR_CHECK(rc);

        /* Copy the configuration from flash into m_dummy_cfg. */
        memcpy(&m_dummy_cfg, config.p_data, sizeof(configuration_t));

        NRF_LOG_INFO("Config file found, updating boot count to %d.", m_dummy_cfg.boot_count);

        /* Update boot count. */
        m_dummy_cfg.boot_count++;

        /* Close the record when done reading. */
        rc = fds_record_close(&desc);
        APP_ERROR_CHECK(rc);

        /* Write the updated record to flash. */
        rc = fds_record_update(&desc, &m_dummy_record);
        APP_ERROR_CHECK(rc);
    }
    else
    {
        /* System config not found; write a new one. */
        NRF_LOG_INFO("Writing config file...");

        rc = fds_record_write(&desc, &m_dummy_record);
        //APP_ERROR_CHECK(rc);
    }

    cli_start();

    /* Enter main loop. */
    for (;;)
    {
        if (!NRF_LOG_PROCESS())
        {
            power_manage();
        }
        rc = fds_record_write(&desc, &m_dummy_record);
        //APP_ERROR_CHECK(rc);
        cli_process();
        delete_all_process();
    }
}

I am using SDK 15.03, Segger Studio and Putty.

Any ideas/advice would be much appreciated.

[Vidar Berg]

Hi,

If you have used up the allocated memory, the call to FDS write will not be queued (i.e., no new task to trigger fds callback), but return FDS_ERR_NO_SPACE_IN_FLASH. Try to make the code changes shown below and see if it works for you.

in main():

        rc = fds_record_write(&desc, &m_dummy_record);
        NRF_LOG_GREEN("FDS write returned \"%s\"",
                       fds_err_str[rc]);

        if (rc == FDS_ERR_NO_SPACE_IN_FLASH)
        {
            delete_all_begin();
        }
        else
        {
            APP_ERROR_CHECK(rc);
        }

Then start garbage collection once all records have been deleted:

/**@brief   Process a delete all command.
 *
 * Delete records, one by one, until no records are left.
 */
void delete_all_process(void)
{
    if (   m_delete_all.delete_next
        & !m_delete_all.pending)
    {
        NRF_LOG_INFO("Deleting next record.");

        m_delete_all.delete_next = record_delete_next();
        if (!m_delete_all.delete_next)
        {
            NRF_LOG_CYAN("No records left to delete. Start GC");
            fds_gc();
        }
    }
}

[Giannis Anastasopoulos]

Dear Vidar,

Thank you for tour response and help.

My aim is to write a code that there is no command line input. The aim is to write/update a value in the flash memory and once the memory I have allocated is full to clear it and start over.

Using the flash_fds example, I have extracted the following functions from the cli.c (unaltered, appart from replacing NRF_LOG_INFO with printf):

static void record_delete(uint32_t fid, uint32_t key)
{
    fds_find_token_t tok   = {0};
    fds_record_desc_t desc = {0};

    if (fds_record_find(fid, key, &desc, &tok) == FDS_SUCCESS)
    {
        ret_code_t rc = fds_record_delete(&desc);
        if (rc != FDS_SUCCESS)
        {
            return;
        }
    }
}

bool record_delete_next(void)
{
    fds_find_token_t  tok   = {0};
    fds_record_desc_t desc  = {0};

    if (fds_record_iterate(&desc, &tok) == FDS_SUCCESS)
    {
        ret_code_t rc = fds_record_delete(&desc);
        if (rc != FDS_SUCCESS)
        {
            return false;
        }

        return true;
    }
    else
    {
        /* No records left to delete. */
        return false;
    }
}

I follow your suggestion and I do this:

void update_total_moves_flash(void)
{
    if(prev_total_moves != total_moves)
    {
        ret_code_t rc;
        fds_record_desc_t desc = {0};
        fds_find_token_t  tok  = {0};
        rc = fds_record_find(CONFIG_FILE, CONFIG_REC_KEY, &desc, &tok);
        if(rc == FDS_SUCCESS)
        {
            m_dummy_cfg.m_total_moves = total_moves;
            rc = fds_record_update(&desc, &m_dummy_record);
            if(rc == FDS_ERR_NO_SPACE_IN_FLASH)
            {
              printf("!!!!!!!!!!!!!!!!!!!!!!!Error Occured. Deleting everyhting...!!!!!!!!!!!!!!!!!!!!\n");
              
              uint8_t is_nested;
              sd_nvic_critical_region_enter ( &is_nested ) ;

              delete_all_begin();
              delete_all_process();
              fds_gc();

              sd_nvic_critical_region_exit ( is_nested );

              fds_stat_t stat = {0};
              rc = fds_stat(&stat);
              APP_ERROR_CHECK(rc);
              printf("\n!!!!!!!!!!!!!!!!!!!!AFTER DELETION!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n");
              printf("Found %d valid records.\n", stat.valid_records);
              printf("Found %d dirty records (ready to be garbage collected).\n", stat.dirty_records);
              printf("!!!!!!!!!!!!!!!!!!!!AFTER DELETION!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n\n");

              rc = fds_record_update(&desc, &m_dummy_record);
              APP_ERROR_CHECK(rc);
            }
        }
        APP_ERROR_CHECK(rc);
        prev_total_moves = total_moves;

        fds_stat_t stat = {0};
        rc = fds_stat(&stat);
        APP_ERROR_CHECK(rc);

        printf("Found %d valid records.\n", stat.valid_records);
        printf("Found %d dirty records (ready to be garbage collected).\n", stat.dirty_records);
    }
}

In my main the relevant code I am using is the following:

/* Register first to receive an event when initialization is complete. */
        (void) fds_register(fds_evt_handler);
        rc = fds_init();
        APP_ERROR_CHECK(rc);
        /* Wait for fds to initialize. */
        wait_for_fds_ready();

        fds_stat_t stat = {0};


        rc = fds_stat(&stat);
        APP_ERROR_CHECK(rc);

        printf("Found %d valid records.\n", stat.valid_records);
        printf("Found %d dirty records (ready to be garbage collected).\n", stat.dirty_records);

        fds_record_desc_t desc = {0};
        fds_find_token_t  tok  = {0};
        
        rc = fds_record_find(CONFIG_FILE, CONFIG_REC_KEY, &desc, &tok);

        if (rc == FDS_SUCCESS)
        {
            /* A config file is in flash. Let's update it. */
            fds_flash_record_t config = {0};

            /* Open the record and read its contents. */
            rc = fds_record_open(&desc, &config);
            APP_ERROR_CHECK(rc);

            /* Copy the variable value from flash into m_dummy_cfg. */
            memcpy(&m_dummy_cfg, config.p_data, sizeof(flash_entry));

            printf("Previous total moves retrieved %d.\n", m_dummy_cfg.m_total_moves);
            total_moves = m_dummy_cfg.m_total_moves;

            /* Close the record when done reading. */
            rc = fds_record_close(&desc);
            APP_ERROR_CHECK(rc);
        }
        else
        {
            m_dummy_cfg.m_total_moves = 0;
            rc = fds_record_write(&desc, &m_dummy_record);
            if (rc == FDS_ERR_NO_SPACE_IN_FLASH)
            {
                delete_all_begin();
                delete_all_process();
                fds_gc();
            }
            else
            {
                APP_ERROR_CHECK(rc);
            }
        }
        for (;;)
        {
               update_total_moves_flash();
        }

And this is the handler:

static void fds_evt_handler(fds_evt_t const * p_evt)
{
    printf("Event: %s received (%s)\n",
                  fds_evt_str[p_evt->id],
                  fds_err_str[p_evt->result]);

    switch (p_evt->id)
    {
        case FDS_EVT_INIT:
            if (p_evt->result == FDS_SUCCESS)
            {
                m_fds_initialized = true;
            }
            break;

        case FDS_EVT_WRITE:
        {
            if (p_evt->result == FDS_SUCCESS)
            {
                printf("Record ID:\t0x%04x\n",  p_evt->write.record_id);
                printf("File ID:\t0x%04x\n",    p_evt->write.file_id);
                printf("Record key:\t0x%04x\n", p_evt->write.record_key);
            }
        } break;

        case FDS_EVT_DEL_RECORD:
        {
            if (p_evt->result == FDS_SUCCESS)
            {
                printf("Record ID:\t0x%04x\n",  p_evt->del.record_id);
                printf("File ID:\t0x%04x\n",    p_evt->del.file_id);
                printf("Record key:\t0x%04x\n", p_evt->del.record_key);
            }
            m_delete_all.pending = false;
        } break;

        default:
            break;
    }
}

What I see is that once the allocated momeory is full, the code enters the if statement, but it doesn't do the deletion and the garbage collection, so then the code fails at the APP_ERROR_CHECK.

Perhaps there is an obvious mistake, but I don't see it.

[Vidar Berg]

Your code quite different from what I suggested. Could you try the code I posted first to verify that it works on your side?  I.e., verify that you can free the flash pages and continue writing afterwards. I worked for me when I tried it.