Sending DATA larger than RAM through UART

[CNONE]

Hi all,

My goal is to store large data which is approximately 300KB and send it using UART. To do it, I think I should the library of NVS on zephyr. I achieved to use it. But, creating and reading large data is my issue. My issues are the following:

1) How can I create 300KB data on flash using nvs or something?

2) How can I send these data through UART?

I found this example and I can use nvs like that:

/* ADDRESS_ID is used to store an address, lets see if we can
 * read it from flash, since we don't know the size read the
 * maximum possible
 */
rc = nvs_read(&fs, ADDRESS_ID, &buf, sizeof(buf));
if (rc > 0) { /* item was found, show it */
	printk("Id: %d, Address: %s\n", ADDRESS_ID, buf);
} else   {/* item was not found, add it */
	strcpy(buf, "192.168.1.1");
	printk("No address found, adding %s at id %d\n", buf,
	       ADDRESS_ID);
	nvs_write(&fs, ADDRESS_ID, &buf, strlen(buf)+1);
}

Thanks for your interest.

[Didrik Rokhaug]

Hi.

How is your data structured/generated?

The NVS library/filesystem uses key-value pairs, which is useful if you want to store several pieces of data. However, if you want to store sensor data that is generated periodically (or other similar scenarios), the Flash Circular Buffer (FCB) library might be better suited. A third option is to use a filesystem that lets you store different files. You can also just use the flash as a region of flash, and write your own "filesystem".

Also, 300kB data is quite a lot for a device with 1MB of flash. You might want to use an external flash chip, instead of using the internal flash. However, it should not be difficult to use an external flash instead of the internal as long as you have a driver for the external flash chip that implements Zephyr's flash API.

Either way, you will generally assign a partition in the device tree, then provide that partition to your "storage library". See here for the default partitions on the nRF9160: https://github.com/NordicPlayground/fw-nrfconnect-zephyr/blob/master/boards/arm/nrf9160_pca10090/nrf9160_pca10090_common.dts#L146

As to how you can send the data over UART, you first have to read out part of the data. How to best do this depends on how the data is stored. If it is stored as multiple small records, it would make sense to read one (or more) records at the time, then send those. Once they are sent, you read more. If the data is stored as one consecutive stream of bytes, you can read out a sensible number, send those over UART, then keep read out the next "batch".

As for how to actually print some data over UART, you can either set UART as Zephyr's console's backend (which is done by default) and use printk, or you can use Zephyr's UART driver: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/1.2.0/zephyr/reference/peripherals/uart.html

Best regards,

Didrik

[CNONE]

Firstly, thanks for your comprehensive answer. I will download the data which will be stored in flash. This data will probably be the format of ZIP. So, actually I should store this type in flash. Is it possible with NVS? If yes, do you have any suggestion for it? 

In addition, I want to use internal flash instead of external. I think it is not problem because the size of data will never be larger than 350KB. 

[Didrik Rokhaug]

CNONE said:

Is it possible with NVS?

 Yes, it is. However, I do not think that would be the best way to do it. As your application would only see it as a big block of bytes, I think it is best to treat it like that.

I would probably just manage the flash partition manually. I.e., you create a flash partition big enough to hold the file, then, when you download a chunk of the file you store it at the beginning of the flash partition. When you receive the next chunk, you store it right after the first chunk, and so on.

As you would have no need for reading out chunk number 3, it is not necessary to store the extra metadata needed to read out chunk 3.

Check out the flash_map and flash_area libraries: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/1.2.0/zephyr/reference/storage/flash_map/flash_map.html

If you want something a bit nicer (though with more overhead), you can take a look at the Flash Circular Buffer: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/1.2.0/zephyr/reference/storage/fcb/fcb.html