Write and read several various records key in the same file of flash using FDS

[PRETOTEmma]

Hi everybody,

I am currently trying to write in flash several records key in a same file using fds_write, fds_update, fds_records_find.

My page is FILE_ID = 0x1000 and my 3 record keys are: ITEM_INTRO = 0x01, ITEM_DATA = 0x02 and ITEM_TOKEN = 0x03.

These 3 record keys are associated to 3 tab respectively: learning_tab_intro, learning_tab_data and learning_tab token.

Whatever the records key called, it seems that the data is written or read at the same place in flash, that way when I shut down and next turn on my system, when I read the ITEM_INTRO records key I can find the data of record key  ITEM_DATA and ITEM_TOKEN in.

Here is the learning_tab_intro that I wrote :

Here is the learning_tab_data that I wrote :

Here i the learning_tab_token that I wrote :

Here is the learning_tab_intro that I read after shutting the system down:

We see clearly that the ITEM_TOKEN record has been written in the ITEM_INTRO record

Inspired by the solution of this link https://devzone.nordicsemi.com/f/nordic-q-a/32208/what-should-i-start-in-order-to-use-the-maximum-memory-space-in-fstorage I created these two functions :

Here is my write function :

ret_code_t fds_function_write(uint8_t item , uint8_t size,uint16_t tab[])
{		
	static uint32_t write_tab[LEARNING_TAB_SIZE/2] ={0};//max size

	memcpy(write_tab, tab, size*2);//in number of byte, 1 uint16 = 2 bytes
	
	fds_record_t        record;
	fds_record_desc_t   record_desc;

	// Set up record.
	record.file_id              = FILE_ID;
	record.key              		= item;
	record.data.p_data       		= &write_tab;
	record.data.length_words   	= (size+1)/2;//from uint16 to uint32, 1 uint16 = 0.5 uint32
 	
	fds_find_token_t    ftok ={0};//Important, make sure you zero init the ftok token
	if(fds_record_find(FILE_ID, item, &record_desc, &ftok) == FDS_SUCCESS)
	{
		ret_code_t ret = fds_record_update(&record_desc, &record);
		if (ret != FDS_SUCCESS)
		{
			return ret;
		}

		NRF_LOG_INFO("Updating Record ID = %d \r\n",record_desc.record_id);
		
		ret = fds_gc();
		if (ret != FDS_SUCCESS)
		{
			return ret;
		}
		NRF_LOG_INFO("FDS_GC = ok \r\n");
	}
	else
	{
		ret_code_t ret = fds_record_write(&record_desc, &record);
		if (ret != FDS_SUCCESS)
		{
			return ret;
		}
		NRF_LOG_INFO("Writing Record ID = %d \r\n",record_desc.record_id);		
	}		
	return NRF_SUCCESS;
}

And here my read function :

ret_code_t fds_function_read(uint8_t item , uint8_t size, uint16_t tab[])
{	
	static uint16_t read_tab[LEARNING_TAB_SIZE] = {0};//max size
		
	fds_flash_record_t  flash_record;
	fds_record_desc_t   record_desc;
	fds_find_token_t    ftok ={0};//Important, make sure you zero init the ftok token
	uint16_t *data;
	uint32_t err_code;

	err_code = fds_record_find(FILE_ID, item, &record_desc, &ftok);
	if ( err_code == FDS_SUCCESS)
	{
		err_code = fds_record_open(&record_desc, &flash_record);
		if ( err_code != FDS_SUCCESS)
		{
			return err_code;		
		}
		
		//NRF_LOG_INFO("Found Record ID = %d\r\n",record_desc.record_id);
		data = (uint16_t *) flash_record.p_data;

		for (uint8_t i=0;i<((flash_record.p_header->length_words)*2);i++)//1 word = 1 uint32 = 2 uint16
		{
			read_tab[i] = data[i];
		}
	    memcpy(tab, read_tab, (size));

		// Close the record when done.
		err_code = fds_record_close(&record_desc);
		if (err_code != FDS_SUCCESS)
		{
			return err_code;	
		}
	}
	else
	{
		return err_code;
	}
	return NRF_SUCCESS;	
}

Do you see any mistake or have you got an idea of how could I resolve my issue ?

I thanks you all in advance !

[tesc]

Hi,

I suspect that you make three calls to fds_function_write() right after each other, without waiting for the FDS_EVT_WRITE or FDS_EVT_UPDATE event in-between.

FDS write and update operations are asynchronous, i.e. the operation is queued and then the actual writing to flash happens at a later point. For both, you must keep the buffer containing the data for as long as it takes for the operations to finish. If you use the same buffer for multiple operations, as you do, you must make sure to only queue one operation using that buffer at a time.

Consider using one buffer for each record key. That way you can queue all three writes in one go. You probably also want to add a mechanism preventing you from starting a new write to the same record ID while an old one is still pending.

I also see that you run fds_gc() every time you do fds_record_update(). This is not recommended. You should only run garbage collection when needed, as the process of garbage collection involves several flash writes and erases which may wear down the flash if used too extensively.

Regards,
Terje

[PRETOTEmma]

Hi tesc,

Firstly, I thank you for your quick answer and for your help.

I tried this solution, creating a wait_for_fds_write function that I called after every write or update.

/**@brief   Wait for fds_write to be done. */
static void wait_for_fds_write(void)
{
    while (!write_flag)
    {
        power_manage();
    }
}

And my m_learning_fds_evt_handler was supposed to activate the write_flag but it seems to never happen so my system reboot after some seconds.

static void m_learning_fds_evt_handler(fds_evt_t const * p_fds_evt)
{
	switch (p_fds_evt->id)
    {
        case FDS_EVT_INIT:
            if (p_fds_evt->result != FDS_SUCCESS)
            {
                // Initialization failed.
            }
			if (p_fds_evt->result == FDS_SUCCESS)
            {								
				_fds_initialized = true;
            }
                
            break;
		case FDS_EVT_UPDATE:
		case FDS_EVT_WRITE:
			if (p_fds_evt->result == FDS_SUCCESS)
				{
					write_flag = true;
				}
			break;
        default:
            break;
    }
}

I can understand that writing to flash is not immediate, but I am pretty sure that it doesn’t normally take more than a second. 

Regards.

[tesc]

Hi,

When operating on a variable from multiple contexts, you need to declare the variable volatile. This will ensure that the variable actually gets read from memory and written to memory every time, and not just kept in a register. Without volatile you also run the risk of the variable getting optimized away.

I do not see where you set the write_flag to false, but I assume that you do that in your write function.

Also make sure that your code will work when the p_fds_evt->result is not FDS_SUCCESS. I suspect that with your current implementation you may end up in a state where write_flag is false even though there is no write or update pending.

Regards,
Terje

[PRETOTEmma]

Hi Terje,

My write_flag is actually a volatile variable :

static bool volatile write_flag = false;

Indeed I initialize it at false and set it to false after every call of wait_for_fds_write function.

And the return value from fds_record_write and fds_record_update functions is tested, and returned directly as a failure if it’s different from FDS_SUCCESS. If it’s equal to FDS_SUCCESS then the wait_for_fds_write function is called.

I tried to create a new volatile variable to avoid the possibility of staying in the loop because of a non FDS_SUCCESS.

static bool volatile write_failure = false;

Now my m_learning_fds_evt_handler looks like that : 

case FDS_EVT_UPDATE:
case FDS_EVT_WRITE:
	if (p_fds_evt->result == FDS_SUCCESS)
	{
		write_flag = true;
		write_failure = false;
	}
	else
	{
		write_flag = true; 
		write_failure = true;
	}
    break;

My wait_for_fds_write function :

static ret_code_t wait_for_fds_write(void)
{
    while (!write_flag)
    {
       power_manage();
    };
	write_flag = false;
	if (write_failure == false)
	{
		return FDS_SUCCESS;
	}	
	else
	{
		write_failure = false;
		return LEARNING_FAILURE_MEMORY;
	}			
}

And I call it after every write and update function this way : 

ret_code_t ret = wait_for_fds_write();
if (ret != FDS_SUCCESS)
{
    NRF_LOG_INFO("Write_flag NOK\r\n");	
	return ret;
}
NRF_LOG_INFO("Write_flag OK\r\n");	
return NRF_SUCCESS;

But it always seems that the FDS_EVT_WRITE event never occurred.

Regards.

[PRETOTEmma]

Hi Terje,

I tried another way to resolve my issue as you suggested in your first answer.

I created one static uint32_t write_tab for every various table that I wanted to write successively in flash.

Same with static uint16_t read_tab.

And in works well.

I might have tried it earlier, I thanks you a lot for your help.

Regards.

[tesc]

Hi,

I am happy to see that you found a solution! I have not had time to look at your other attempt yet, but I guess it is not needed now that you have solved the issue.

Regards,
Terje