Problems writing/reading struct to NVS

[Mike Austin (LPI)]

I've been slowly adding NVS functionality to my code.  Has been working fine, up until today when I attempted to write a struct I have for storing historical data to flash.

This is the struct:

struct log_data {
	// data structure to store individual strike events
	uint8_t	strike_level;  // SML = 1, MED = 2 or LRG = 4
	uint16_t	total_strike_count;  // Total number of strikes recorded to date
	struct bt_cts_current_time time_detected;  //Time stamp for each strike
};

I then have an array of this struct.  At this point, MAX_STRIKE_RECORDS = 10, but ultimately I want to see this at 250.

struct log_data lsr_strike_log[MAX_STRIKE_RECORDS];

In my code to read the data in from Flash, I have the following:

	uint32_t rc = 0;
	rc = nvs_read(&fs, HISTORY_ID, &lsr_strike_log, sizeof(lsr_strike_log));
	printk("sizeof = %u\n", sizeof(lsr_strike_log));
	printk("strike history rc = %u\n", rc);

The return value from nvs_read() is supposed to be the amount of data read in, if everything works as expected.  In my case, this should be equal to sizeof(lsr_strike_log), which is 160 bytes.  If the return value is larger than this, it apparently means there is more data to read in.  This is what I get from my two printk() statements:

sizeof = 160
strike history rc = 4294967294

So, its like the nvs_read() is going outside my defined flash area, or something - I'm not sure, hence why I'm posting here to see if someone can help me understand what's going on better.

My flash is initialised as per the following, and I don't get any of the error messages output via UART, so it appears to be initialising OK.

flash_dev = FLASH_AREA_DEVICE(STORAGE_NODE_LABEL);
	if (!device_is_ready(flash_dev)) {
		printk("Flash device %s is not ready\n", flash_dev->name);
		return -EINVAL;
	}
	fs.offset = FLASH_AREA_OFFSET(storage);
	rc = flash_get_page_info_by_offs(flash_dev, fs.offset, &info);
	if (rc) 
	{
		printk("Unable to get page info\n");
		return rc;
	}
	fs.sector_size = info.size;
	fs.sector_count = 3U;

	rc = nvs_init(&fs, flash_dev->name);
	if (rc) 
	{
		printk("Flash Init failed\n");
		return rc;
	}

If I enable CONFIG_LOG=y in my proj.config file, I see the following output:

I suspect I'm doing something obvious wrong. Problem is, its not obvious to me.

Thanks and regards,

Mike

[Mike Austin (LPI)]

Thanks Haakonsh.

If I simplify what I'm trying to write to NVS, it seems to work OK.  I changed my data structure to just a simple array:

uint16_t lsr_strike_log[9];

And if I write/read this to NVS, I can see that the data is being stored.  I'm still getting weird values for the return value from nvs_write() and nvs_read() though.

This is the log info I see when writing the simple array:

If I try and write my struct to NVS in the form I ultimately want to, I crash my DK and it keeps rebooting.

In my code I initialse NVS before I initialise anything to do with BLE, so not sure if the BLE initialisation then changes what I've done when setting up NVS?

Thanks and regards,

Mike

[Simon]

Hello Mike,

What is the current state on this issue? 

Best regards,

Simon

[Mike Austin (LPI)]

Hi Simon,

I’m just about to get back into this part of it after struggling with issues around GPIO/System OFF/MCUBoot for nearly a month.

I’ll update this thread in a week or so when I’ve had a chance to do some more testing

Cheers,

Mike

[Simon]

Mike Austin (LPI) said:

I’ll update this thread in a week or so when I’ve had a chance to do some more testing

Sounds good

[Mike Austin (LPI)]

Hi Simon,

I seem to have this working now.  Only problem is, I'm not able to get everything I want stored in NVS as I am running out of room.  And I'm not sure how to make more space, or indeed if I can.

My struct takes up 12 bytes per array element, and I am trying to have capacity for up to 250 array elements.  At the moment, I hit a wall around 167.  I've got a few other bytes of info I store in NVS that are separate to this array - they take up about 15 bytes in total.

This is how the flash is allocated in my .dts file (this is the default for the nRF52-DK)

&flash0 {

	partitions {
		compatible = "fixed-partitions";
		#address-cells = <1>;
		#size-cells = <1>;

		boot_partition: partition@0 {
			label = "mcuboot";
			reg = <0x00000000 0xc000>;
		};
		slot0_partition: partition@c000 {
			label = "image-0";
			reg = <0x0000C000 0x32000>;
		};
		slot1_partition: partition@3e000 {
			label = "image-1";
			reg = <0x0003E000 0x32000>;
		};
		scratch_partition: partition@70000 {
			label = "image-scratch";
			reg = <0x00070000 0xa000>;
		};
		storage_partition: partition@7a000 {
			label = "storage";
			reg = <0x0007a000 0x00006000>;
		};
	};
};

And this is how I have my flash device set up in my code (based on the example code)

int16_t flash_initialise(void)
{
    int16_t rc;

	flash_dev = FLASH_AREA_DEVICE(STORAGE_NODE_LABEL);
	if (!device_is_ready(flash_dev)) {
		#ifdef DEBUG_NVS
			printk("Flash device %s is not ready\n", flash_dev->name);
		#endif
		return -EINVAL;
	}
	printk("Flash device %s is ready\n",flash_dev->name);

	fs.offset = FLASH_AREA_OFFSET(storage);
	rc = flash_get_page_info_by_offs(flash_dev, fs.offset, &info);
	if (rc !=0) 
	{
		#ifdef DEBUG_NVS		
			printk("Unable to get page info\n");
		#endif
		return rc;
	}
	#ifdef DEBUG_NVS
		printk("Page info OK\n");
	#endif
	fs.sector_size = info.size;
	fs.sector_count = 2U;

	rc = nvs_init(&fs, flash_dev->name);
	if (rc !=0) 
	{
		#ifdef DEBUG_NVS
			printk("Flash Init failed\n");
		#endif
		return rc;
	}
	#ifdef DEBUG_NVS
		printk("Flash initialised OK\n");
	#endif

	// Check if data is stored in flash
	rc = check_for_nvs_key();
	return rc;
	
}

As I understand it, the available flash storage size is 0x6000, which is about 24kB.  And I have allocated two sectors, each of 4096 bytes.

So, I'm not sure why I'm running into storage problems.

Sorry if this sounds like a pretty basic question, but I can't seem to find any info that would help me understand why I'm running out of storage space and how to increase it

Cheers,

Mike

[Simon]

Sorry for the delay, I was gone on Friday.

Could you check if the partition manager is enabled?

You can do that by searching for CONFIG_PARTITION_MANAGER_ENABLED in <sample>build/zephyr/.config and see if it's set to 'y'.

Partition Manager is enabled when you're including child images (like mcuboot) and the dts partitions will be ignored.

Best regards,

Simon

[Mike Austin (LPI)]

Hi Simon,

Yes, I have partition manager enabled via:

#

# Partition Manager

#

CONFIG_PARTITION_MANAGER_ENABLED=y

CONFIG_FLASH_MAP_CUSTOM=y

CONFIG_SRAM_SIZE=64

CONFIG_SRAM_BASE_ADDRESS=0x20000000

Cheers,

Mike

[Simon]

Ah, okay.

Check the note in the documentation about the Partition Manager:

"When you build a multi-image application using the Partition Manager, the Device Tree Source flash partitions are ignored."

So you should create the storage partition through a pm_static.yml file

Vidar provided one example how to go about this in this reply:  RE: Problem to read back flash with NVS when concurrent use with Bluetooth 

Then you should be able to access the size and address of the user storage from your application (generated defines will be located in <sample>\build\zephyr\include\generated\pm_config.h I believe), which you can use to set up NVS to store custom data.

By the way, Bluetooth bonding uses NVS as well (settings_storage), so that may interfere with your custom NVS data, in the ticket above that was resolved by creating a separate partition (user_storage).

However, I think it should be possible to use the same partition for both cases, see  Concurrent usage of NV and settings on the same partition 

Best regards,

Simon