Possible to pre-load data to NVS when the nrf9160 is programmed?

[matharman]

Hello,

We are using the nrf9160 in a custom hardware solution, with SDK v1.2.0 and modem firmware v1.1.1. There are some data items that we would like to load into NVS at the time the nrf9160 is programmed, for example if it were being flashed at the factory. Some are not security-critical, such as target server IP(s) and port(s), our in-house device IDs, and so forth. However we do have security-critical items to store -- 2 PSKs for TLS connections.

I am aware that TLS credentials can be written to/read from the modem flash using the AT command %CMNG, but only written when the modem is offline. There are certain scenarios when the application will need to refresh the PSK while maintaining an LTE connection, so I am not sure that the modem credential storage is the right solution for us. We are also rolling the wolfSSL TLS stack rather than the native TLS so we can use existing code from another cross-platform solution we've written. This means we cannot simply specify a sec_tag in the modem to use for TLS when creating a socket.

My question has 2 parts, due to the different considerations for secure and non-secure data.

1. Is there an example/guide as to how one might pre-write a set of data into NVS at the time the application processor is programmed? Is it even possible to read/write to NVS outside of runtime?

2. Are there any alternatives to modem flash for secure storage in the same vein as No. 1, that can be pre-written at the time of apps proc programming?

I appreciate any help/advice you can offer. Thank you!

[matharman]

In case future readers stumble across this thread, I wanted to share the solution I've arrived at with Heidi's advice above.

The native_posix_64 target is indeed able to output NVS-formatted data. Getting the partitions' starting addresses to align was more trouble than it was worth, however. I will share prj.conf, source code, and python script needed for my method at the end of this post.

The method I've come up with is as follows:
1. Run a native_posix_64 project to output the NVS key-value pairs you want to provision in your device.  This will be written to build/flash.bin. The sector size and number of sectors reserved must match up to your application's as far as I can tell, or the metadata will be in the wrong flash address.

2. Extract the bytes of the storage partition, making sure to include the NVS metadata. The starting address for my native_posix_64 storage partition was 0xfc000 (1032192 in decimal).The NVS metadata was stored beginning at 0xfcff0.

You can find the starting address in build/zephyr/zephyr.dts, or see for yourself by running 

hd build/flash.bin

 and observing where your data is. The hd command is how I determined where the NVS metadata began.

   I extracted the data and metadata together with the following command:
   

tail -c +1032193 build/flash.bin | head -c 16384 > nvs.bin

   Note that I run tail from the starting address+1. This was to skip over a 0xFF byte preceding the data I wrote with NVS. You may wish to hexdump your build/flash.bin to examine if you also need to start at the next index.

   I take 16384 bytes of that partition arbitrarily, just enough to really be sure I included the NVS metadata. There is probably a more "correct" size to use, such as the (number of reserved sectors) * (sector size) but I haven't experimented with that yet.

3. Translate the nvs.bin slice into a HEX file that is mergeable with your NRF9160 merged.hex file.
   To do this, I created a python script using IntelHex which should already be installed as a prerequisite for west.

4. Merge your NVS hex file with the NRF9160 application hex file, like so:

mergehex --merge build/zephyr/merged.hex nvs.hex -o merged.hex

   If your merge is successful, the merged hexfile is flashable, and you should see the sectors of your storage partition be erased in the output of nrfjprog

EDIT (4/30/202): Merging the hex files is completely unecessary actually -- As long as the generated nvs.hex has the correct starting address encoded, you can flash it separately with

nrfjprog --program nvs.hex --sectorerase

as this will only erase the relevant blocks of the storage partition

I haven't come up with a great automation for this process yet, so after generating nvs.hex I'm simply copying it over to my application folder.

prj.conf:

CONFIG_FLASH=y
CONFIG_FLASH_SIMULATOR=y
CONFIG_NVS=y

main.c:

#include <zephyr.h>
#include <device.h>
#include <drivers/flash.h>
#include <fs/nvs.h>

#include <string.h>
#include <errno.h>

#define TEST_DATA_ID 0U
#define TEST_DATA "ABCDEFGHIJKLMNOPQRSTUVWXYZ123456789"

void main(void) {
    struct device *flash_dev = device_get_binding(DT_FLASH_DEV_NAME);
    if(!flash_dev) {
        printk("No flash device available!\n");
        return;
    }

    struct flash_pages_info flash_inf;
    flash_get_page_info_by_offs(flash_dev, DT_FLASH_AREA_STORAGE_OFFSET, &flash_inf);

    struct nvs_fs my_fs;
    my_fs.offset = flash_inf.start_offset;
    my_fs.sector_size = flash_inf.size;
    my_fs.sector_count = 5U;

    printk("Sector size: %u\n", my_fs.sector_size);

    if(nvs_init(&my_fs, DT_FLASH_DEV_NAME) != 0) {
        printk("Failed to init NVS!\n");
        return;
    }

    const char *data = TEST_DATA;

    if(nvs_write(&my_fs, 0, data, strlen(data)) < 0) {
        printk("Failed to write test data, err: %d\n", errno);
        return;
    }

    printk("Available space: %d\n", (int)nvs_calc_free_space(&my_fs));
}

python script:

#!/usr/bin/env python
from intelhex import IntelHex

nvs = IntelHex()

# Specifying offset generates the hexfile encodings starting at that address.

# For my NRF9160 application, DT_FLASH_AREA_STORAGE_OFFSET_0=0xfa000

# nvs.bin is the output of the tail | head oneliner that parsed flash.bin
nvs.loadbin('nvs.bin', offset=0xfa000)
nvs.tofile("nvs.hex", format='hex')