How store large amount data 73Kbytes into flash using FDS?

[vishal]

We are using nRF52840 with sdk 15.0 version & segger embedded studio. We are able to store some small amount of configuration data into flash using FDS. But now we want to store large data array near about 73Kbytes into flash. I am writing example code for storing this data and created one dummy array of size 73728 bytes. But I getting following ERROR:

FDS_ERR_RECORD_TOO_LARGE

I am confused for configuration of FDS for storing 73Kbytes 

Here is bellow edited in sdk_config.h

FDS_VIRTUAL_PAGES 10

FDS_VIRTUAL_PAGE_SIZE 2048

FDS_OP_QUEUE_SIZE 4

Will you please help me how i use this above configuration for storing 73KB data. & how i fixed FDS_ERR_RECORD_TOO_LARGE ERROR?

Is it possible to store this data in one fds_record_write (...) call request?

Thanks in advanced..!!

[Ashish]

The number that you give for the FDS_VIRTUAL_PAGE_SIZE is actually 4-byte words. So you can save at a maximum of (2048-5)*4 = 8172 bytes. So you will need to split up your data if you want to save more than that.

[vishal]

Ok, Is it possible to increase FDS_VIRTUAL_PAGE_SIZE as per required data storage above of 2048. If i increase this i got ERROR 11 i.e. NRF_ERROR_INVALID_DATA when fds_flash_init();

We will split our data with smaller chunks with multiple of above maximum limit. But still confused for is it needed to change record key every request of that.

here is bellow my sample code snippet for storing dummy array of 73728bytes.

 

void fds_data_write() {
  uint32_t length = 74000;
  uint8_t data_write[length];
//  memset(data_write,2,sizeof(data_write));
  for (int i = 0; i<=length; i++) {
   data_write[i] = i;
  }
//  NRF_LOG_INFO("fds_data_write");
  ret_code_t ret = fds_write(FILE_ID_READING_1, REC_KEY_READING_1,(char *)data_write,length);
  if (ret == FDS_SUCCESS) {
    NRF_LOG_DEBUG("FDS data stored");
  } else {
    NRF_LOG_DEBUG("FDS failed error code:%d", ret);
  }
}

Will you please provide me program snippet for how i call multiple request for storing 73KB data with FILE-ID and REC_KEY. Because after stored this data we want to also retrieve this data as required time.

Lets say from above whole array first 8172 bytes stored of 1 request with specific FILE_ID & RECKEY, Then i should need to call next chunk data for that is it necessary to change RECKEY.

Because if KECKEY is not change for second call request it will overwritten data on old. Is any other way to store this large data with one FILEID & KECKEY?

Thanks.....

[vishal]

Hi,

Now i am able to store 73Kbytes data into flash with sending multiple request up to 19 pages. I have changed rec key every request let say first request reckey is 1 then it 2, 3, 4.. etc. 

But now problem getting when i want to retrieve this stored data for all this pages.  First of kec key 1 & 2 got data but when i send read request with 3 reckey that time my application is crash. Nothing say any error but when i debugging it get __aeabi_memcpy4 when this bellow function reached:

fds_find_token_t ftok;
memset(&ftok, 0x00, sizeof(fds_find_token_t)); 

Will you please help me how i read this stored data from flash with proper way?

Thanks

[Ashish]

Hey,

I tried to do the same thing myself and I was able to read and write 10 records.

This is my "app_fds.c" file. This implements the write and read functions. In write function, the same array is getting written 10 times. The file_id is same in all cases but the rec_id changes.

#include "fds.h"
#include "nrf_log.h"
#include "nrf_log_ctrl.h"
#include "nrf_log_default_backends.h"

/**************************Private Function Definitions**************************/

uint32_t file_id = 0x01000;
uint32_t rec_id = 0x1111;

/* Data to write in flash. */
struct 
{
  char string[100];                         //Change this size as required.
}fds_data;

/* A record containing dummy configuration data. */
fds_record_t record =
{
  .file_id           = 0x00,
  .key               = 0x00,
  .data.p_data       = &fds_data,
  /* The length of a record is always expressed in 4-byte units (words). */
  .data.length_words = 25,                  //Also change this if array size is changed.
};

/* Flag to check fds initialization. */
static bool volatile m_fds_initialized = false;;
static bool volatile m_fds_write = false;
static bool volatile m_gc_complete = false;


/**
* @brief             Event handler for FDS events
* @param[in]         p_evt              Structure with FDS data
* @retval            NONE
*/
static void fds_evt_handler(fds_evt_t const * p_evt)
{
  /*NRF_LOG_INFO("Event: %s received (%s)",
               fds_evt_str[p_evt->id],
               fds_err_str[p_evt->result]);
  */
  switch (p_evt->id)
  {
  case FDS_EVT_INIT:
    if (p_evt->result == FDS_SUCCESS)
    {
      m_fds_initialized = true;
    }
    break;
    
  case FDS_EVT_WRITE:
    {
      if (p_evt->result == FDS_SUCCESS)
      {
        m_fds_write = true;
      }
    } break;
    
  case FDS_EVT_DEL_RECORD:
    {
      if (p_evt->result == FDS_SUCCESS)
      {
        NRF_LOG_INFO("Record ID:\t0x%04x",  p_evt->del.record_id);
        NRF_LOG_INFO("File ID:\t0x%04x",    p_evt->del.file_id);
        NRF_LOG_INFO("Record key:\t0x%04x", p_evt->del.record_key);
      }
      ///m_delete_all.pending = false;
    } break;
    
  case FDS_EVT_GC:
    {
      NRF_LOG_INFO("GC Done");
      m_gc_complete = true;
    }break;
    
  default:
    break;
  }
}
/**
* @brief             Function to go to low power when waiting for FDS
* @param[in]         NONE
* @retval            NONE
*/
static void power_manage(void)
{
#ifdef SOFTDEVICE_PRESENT
  (void) sd_app_evt_wait();
#else
  __WFE();
#endif
}

/**
* @brief             The FDS takes some time for init. Waiting till it gets initialised properly.
* @param[in]         NONE
* @retval            NONE
*/
static void wait_for_fds_ready(void)
{
  while (!m_fds_initialized)
  {
    power_manage();
  }
}

static void wait_for_write(void)
{
  while (!m_fds_write)
  {
    power_manage();
  }
  m_fds_write = false;
}


void fds_write()
{
  ret_code_t rc;
  fds_record_desc_t desc = {0};
  record.file_id = file_id;

  for(int i=0;i<10;i++)
  {  
    memset(&desc, 0x00, sizeof(fds_record_desc_t)); 
    record.key = rec_id;
    rc = fds_record_write(&desc, &record);
    wait_for_write();
    if(rc == FDS_SUCCESS) {
      NRF_LOG_INFO("Data written with id %d",rec_id);
    }
    else {
      NRF_LOG_INFO("Write Failed, id %d",rc);
    }
    rec_id++;
  }  
  rec_id = 0x1111;
}

void fds_read()
{
  NRF_LOG_INFO("Starting Read");
  ret_code_t rc;
  
  fds_record_desc_t desc = {0};
  fds_find_token_t  tok  = {0};
  
  for(int i=0;i<10;i++)
  {
    memset(&desc, 0x00, sizeof(fds_record_desc_t)); 
    memset(&tok, 0x00, sizeof(fds_find_token_t)); 
    
    rc = fds_record_find(file_id, rec_id, &desc, &tok);
  
    if (rc == FDS_SUCCESS)
    {
      fds_flash_record_t config = {0};
      
      /* Open the record and read its contents. */
      rc = fds_record_open(&desc, &config);
      APP_ERROR_CHECK(rc);

      NRF_LOG_INFO("File Found with id %d",rec_id);
      
      /* Close the record when done reading. */
      rc = fds_record_close(&desc);
      APP_ERROR_CHECK(rc);
    }
    rec_id++;
  }
}

/**
* @brief             Initialise FDS
* @param[in]         NONE
* @retval            NONE
*/
void init_fds()
{
  ret_code_t rc;
  
  /* Register first to receive an event when initialization is complete. */
  (void) fds_register(fds_evt_handler);
  
  NRF_LOG_INFO("Initializing fds...");
  
  rc = fds_init();
  NRF_LOG_INFO("fds init %d",rc);
  APP_ERROR_CHECK(rc);
  
  /* Wait for fds to initialize. */
  wait_for_fds_ready();
}

This is simple "main.c" file

#include "app_fds.h"
#include "nrf_pwr_mgmt.h"
#include "nrf_delay.h"
#include "nrf_log.h"
#include "nrf_log_ctrl.h"
#include "nrf_log_default_backends.h"

/**************************Private Function Definitions**************************/

/**
* @brief                       Function to call the power management
* @param[in]                   NONE
* @retval                      NONE
*/
void idle_state_handle()
{
  nrf_pwr_mgmt_run();
}

/**
* @brief             Initialise the peripherals
* @param[in]         NONE
* @retval            NONE
*/
void init_peripherals()
{  
  NRF_LOG_INIT(NULL);
  NRF_LOG_DEFAULT_BACKENDS_INIT();                                                      //Use RTT as default backend
  NRF_LOG_INFO("Start\r\n");
  
  init_fds();
}

/**
* @brief             Main Function
* @param[in]         NONE
* @retval            NONE
*/
int main()
{
  init_peripherals();
  fds_write();
  fds_read();
  
  while(1)
  {
    NRF_LOG_PROCESS();
    idle_state_handle();
  }
}

In "sdk_config.h"

FDS_VIRTUAL_PAGES 11

FDS_VIRTUAL_PAGE_SIZE 2048

I don't know if this is the best way to do this, but it has worked for me.

[vishal]

Thanks Ashisk,

Now we are able to store 74 KB data in flash also read this data, but got one issue for wait_for_write(); function. My program is not work when mobile application is connected wit device that time want to write data into flash. Because it continuous on this loop not triggering FDS_EVT_WRITE event, since this flash is not true. This happening only when BLE is connected to phone. If phone is not connected then is working and also trigger FDS event. 

We still stuck here why this happening with BLE connection established and still it connected that time only FDS write event is not trigger.

Thanks..

[Ashish]

Hey Vishal,

A couple of things:

Are you using the nrf_fstorage_sd.c library or nrf_fstorage_nvmc.c? You should be using the former one with softdevice enabled. 

In sdk_config.h, FDS_BACKEND should be 2 to work with softdevice.

Can you check what is the return value that you are getting in case when FDS_EVT_WRITE doesn't give FDS_SUCCESS?

Are you calling the fds write function in another callback or another handler?

[vishal]

Thanks for fast reply,

Actually i am using ble_app_uart example and modified this example as per our application with FDS library.

Ashish said:

Are you using the nrf_fstorage_sd.c library or nrf_fstorage_nvmc.c? You should be using the former one with softdevice enabled. 

Yes i am using nrf_fstorage_sd.c lib 

I checked in my sdk_config.h FDS_BACKEND  is 2.

The return value is 0 when but not trigger FDS_EVT_WRITE.

I am calling fds write function in nus_data_handler(...); something command received from mobile app.

But when i comment wait_for_write();function that time it working..

what happening with ble connected for fds event???

Thanks....