Thingy91 flash memory storage size

[Huba]

Hey everyone,

I know its quite a lot of text, but i tried given as much usefull information as possible.

Setup
Thingy91, SDK 2.5.0, using the NVS sample as test code developer.nordicsemi.com/.../nvs.html

Goal
Save pressure data (uint32_t) every 5s for a long period of time (>6 months).

Using the 1MB flash-memory seems like a good option, correct me if i am wrong.

Each datapoint is stored in flash as metadata (8 byte) and the data (4 byte), so every datapoint consumes 12 bytes. The storage size is 24.576 bytes (24 KiB), with a sector size of 4096 bytes, this comes down to 24.576/4096 = 6 sectors. In total 24.576/12 = 2048 datapoints could be stored. Which in my case comes down to 2048*5s = 170 hour of storage.

I have a few questions that I hope can be clarified.

Questions
1) The metadata occupies twice as much space as the actual data. Is it possible to decrease the metadata usage by saving datapoints in an array and then storing this array to flash memory? Is the following calculation correct?

For example: i save 100 datapoints in an array, this would consume 8 byte (metadata) + 100*4 = 400 bytes of data = 408 bytes in total for 100 datapoints. This way i could store: 24.576/408 = 6000 datapoints, almost three times as much.


2) Increasing the storage size would be the easiest solution. The flash is divided into 7 partitions as shown below:
- image-0 192 KiB
- image-0-nonsecure 256 KiB
- image-1 192 KiB
- image-1-nonsecure 256 KiB
- image-scratch 40 KiB
- mcuboot 64 KiB
- storage 24 KiB

Are all the following 4 partitions: image-0, image-0-nonsecure, image-1, image-1-nonsecure, used for flashing an image to the board?

What partition could i savely reduce to increase the storage partition? Or is it a better idea to use image-1 as storage partition, as done in this post: devzone.nordicsemi.com/.../repurposing-flash-partition-for-non-volatile-storage-on-thingy91


3) The NVS sample works fine with a sector count of 2, however when i increase it to 4 or 6 (fs.sector_count = 6U;), which should be possible according to my calculation i get the error: invalid address. Any idea what i am doing wrong?

When using image-1 as storage partition i can use 48 sectors with no errors. 196.608 (192 KiB) / 4096 = 48 sectors. Strange thing is the code even works when given 100 sectors, which should not be possible?

// #define NVS_PARTITION storage_partition
#define NVS_PARTITION image_1
#define NVS_PARTITION_DEVICE FIXED_PARTITION_DEVICE(NVS_PARTITION)
#define NVS_PARTITION_OFFSET FIXED_PARTITION_OFFSET(NVS_PARTITION)

fs.sector_count = 100U;

Thanks in advance!

[Naeem Maroof]

Hello,

4B vs 12B would definitely be 3x more datapoints

Which in my case comes down to 2048*5s = 170 hour of storage.

this however is wrong and would be 170 minutes instead of hours

I don't see a conclusion for the ticket that you have mentioned.

I will respond back once I have more information.

/BR, Naeem

[Huba]

Ah my bad, miscalculation on my part.

I did another test, in which pressure data is saved in the image_1 partition for 45min with a 1s interval. In total i would expect roughly 45*60 =  2700 datapoints to be stored.

The output from the Putty log indeed shows the 2700 datapoints, however reading from the flash memory only 1800 datapoints were retrieved. From the graph it looks like over time datapoints are either not saved or not read out correctly from the flash memory.

Code for saving data to flash:

while (1) {
		sensor_sample_fetch(dev);
		sensor_channel_get(dev, SENSOR_CHAN_PRESS, &press);
		nvs_write(&fs, PRESSURE_ID, &press, sizeof(press));
		k_msleep(SLEEP_TIME_MS);
	}

Code for reading the flash memory data:

cnt_his = 0;
while (1) {
	rc = nvs_read_hist(&fs, PRESSURE_ID, &press, sizeof(press), cnt_his);
	if (rc < 0) {
		break;
	}
	printk("%d.%06d\n", press.val1, press.val2);
	cnt_his++;
}

Real strange, no idea what is going wrong

[Naeem Maroof]

Good to know you have made the progress.

Can you also check for the error code after writing (nvs_write) to make sure that every write is successful?

[Huba]

No errors are given while writing to the flash memory.

However i investigated a little more and discovered what is actually going wrong.

Comparing the Putty column with the Flash read column: when the value is equal to the previous value it is not being written, without giving an error or warning.

Re-reading the NVS documentation: NVS checks the id-data pair before writing data to flash. If the id-data pair is unchanged no write to flash is performed.

Classic example of RTFM (Read the fking manual xD)

Solution in case someone else also want to write all data, even duplicates.

In the nvs_write function, comment out the following line of code:

/* do not try to compare if lengths are not equal */
/* compare the data and if equal return 0 */
rc = nvs_flash_block_cmp(fs, rd_addr, data, len);
if (rc <= 0) {
	// return rc; comment out to write duplicates
}