Problem to read back flash with NVS when concurrent use with Bluetooth

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Vidar Berg — Wed, 29 Sep 2021 07:31:27 GMT

The settings module will use the 'settings_storage' instead of the 'storage' partition now when the Partition Manager is enabled, so you can remove the storage partition and keep settings_storage + user_storage.

CONFIG_SETTINGS_NVS_SECTOR_COUNT will work in the same way as before.

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Romain — Mon, 27 Sep 2021 14:37:58 GMT

Vidar,

The error still remains.

This is not clear for me, the dts and overlay files will be ignored about &flash0 part only ?

The content seems not identical to my overlay file... Is the code below the "perfect" equivalence of my overlay ? (it doesn't work neither)

storage:
  address: 0xf8000
  size: 0x6000
  end_address: 0xfe000
  placement:
    before: 
    - user_storage
  region: flash_primary
user_storage:
  address: 0xfe000
  size: 0x2000
  end_address: 0x100000
  placement:
    before:
    - end
  region: flash_primary

Edit: I can build now

According to my pm_static.yml there is 3 "storage" partitions

- settings_storage size 0x2000 (generated only with MCU Boot)

- storage size 0x6000 (default 0x8000)

- user_storage size 0x2000 (new custom partition from storage)

Knowing this, CONFIG_SETTINGS_NVS_SECTOR_COUNT=4 should still influence on storage ?

...
settings_storage:
  address: 0xf6000
  end_address: 0xf8000
  placement:
    before:
    - end
  region: flash_primary
  size: 0x2000
sram_primary:
  address: 0x20000000
  end_address: 0x20040000
  region: sram_primary
  size: 0x40000
storage:
  address: 0xf8000
  end_address: 0xfe000
  placement:
    before:
    - user_storage
  region: flash_primary
  size: 0x6000
user_storage:
  address: 0xfe000
  end_address: 0x100000
  placement:
    before:
    - end
  region: flash_primary
  size: 0x2000

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Vidar Berg — Mon, 27 Sep 2021 11:03:47 GMT

Hi,

I should have pointed this out earlier, but the partition manager will take over and manage the memory layout once you start doing Multi-image builds. This means that the DTS configuration will be ignored.

Please try to add the yaml code below into a file named "pm_static.yml" and place it in your project directory. This should configure the partition manager to include your partition.

user_storage:
  address: 0xfc000
  size: 0x2000
  end_address:0xfe000
  placement:
    before: 
    - settings_storage
  region: flash_primary
settings_storage:
  address: 0xfe000
  size: 0x2000
  end_address: 0x100000
  placement:
    before:
    - end
  region: flash_primary

Best regards,

Vidar

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Romain — Mon, 27 Sep 2021 07:58:13 GMT

Hi Vidar,
It used to work fine but now I face problem to compile the same code with MCUBoot enabled...

This MACRO generates an error

fs.offset = FLASH_AREA_OFFSET(user_storage);

If I use one partition named storage this work but as soon as I split it in storage + user-storage it fails.

The overlay of my custom board nRF52840 is as below

/* User storage */
&flash0 {
    partitions {
        storage_partition: partition@f8000 {
            label = "storage";
            reg = < 0xf8000 0x6000 >;
        };
            
        user_partition: partition@fe000 {
            label = "user-storage";
            reg = < 0xfe000 0x2000 >;
        };
    };
};

proj.conf extract

NVS concern:

# Enable bonding
CONFIG_SETTINGS=y
CONFIG_BT_BONDABLE=y
CONFIG_BT_SETTINGS=y
CONFIG_FLASH=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FLASH_MAP=y
CONFIG_NVS=y
CONFIG_SETTINGS_NVS=y
CONFIG_SETTINGS_NVS_SECTOR_COUNT=4

New lines MCUBoot concern:

# Bootloader
CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_MCUBOOT_UTIL_LOG_LEVEL_DBG=y
CONFIG_MCUMGR_LOG_LEVEL_DBG=y

# DFU and OTA update
CONFIG_DFU_TARGET=y
CONFIG_DFU_TARGET_MCUBOOT=y
CONFIG_FLASH=y
CONFIG_IMG_ERASE_PROGRESSIVELY=y
CONFIG_IMG_MANAGER=y
CONFIG_MCUBOOT_IMG_MANAGER=y
CONFIG_RING_BUFFER=y

# Firmware signing
CONFIG_SIGN_IMAGES=y

Any idea on what could be wrong ?

Best regards.

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Vidar Berg — Fri, 27 Aug 2021 09:07:48 GMT

Thanks for the update.

[quote userid="24186" url="~/f/nordic-q-a/77781/problem-to-read-back-flash-with-nvs-when-concurrent-use-with-bluetooth/326922#326922"]So your point is that every user NVS data should go in a dedicated partition and the OS NVS data will take place in the original storage partition.[/quote]

I think creating a new storage partition might be the easiest way to ensure separation between the NVS area used by Settings from the NVS area you are using in your application. But it's not the only solution, you could also use the CONFIG_SETTINGS_NVS_SECTOR_COUNT configuration and prevent Settings from allocating the whole partition to itself.

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Romain — Fri, 27 Aug 2021 08:40:57 GMT

[quote userid="4240" url="~/f/nordic-q-a/77781/problem-to-read-back-flash-with-nvs-when-concurrent-use-with-bluetooth/326914#326914"]Are these 2 sectors currently placed in the same partition as Bluetooth settings?[/quote]

Yes they were...

But I tried your proposal and it seems working fine.
The Bluetooth settings now takes 4 sectors instead of 8.

So your point is that every user NVS data should go in a dedicated partition and the OS NVS data will take place in the original storage partition.

Thank you

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Vidar Berg — Fri, 27 Aug 2021 08:12:39 GMT

Hi,

[quote userid="24186" url="~/f/nordic-q-a/77781/problem-to-read-back-flash-with-nvs-when-concurrent-use-with-bluetooth/326906#326906"]you propose to separate NVS to 2x16kb partitions. Could that be really OK ?[/quote]

This was just an example. Bluetooth settings will allocate 4 sectors with this partitioning. My point is that your NVS instance should not cover the same flash area as the other NVS instance used by Settings.

[quote userid="24186" url="~/f/nordic-q-a/77781/problem-to-read-back-flash-with-nvs-when-concurrent-use-with-bluetooth/326906#326906"]custom NVS parameters defines 2 (3 in Simon example) sectors[/quote]

Are these 2 sectors currently placed in the same partition as Bluetooth settings?

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Romain — Fri, 27 Aug 2021 07:50:45 GMT

Hi Vidar,

[quote userid="4240" url="~/f/nordic-q-a/77781/problem-to-read-back-flash-with-nvs-when-concurrent-use-with-bluetooth/325502#325502"] have you defined a new flash partition for your NVS instance[/quote]

No I didn't.

But I am not clear with sectors overlapping on the storage partition which is 32kb. Bluetooth settings defines 8 sectors of 4096b (the whole storage partition as defined in dts), my custom NVS parameters defines 2 (3 in Simon example) sectors of 4096b and you propose to separate NVS to 2x16kb partitions. Could that be really OK ?

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Vidar Berg — Wed, 18 Aug 2021 12:33:08 GMT

Hi,

It sounds like NVS should have plenty of time to complete the write operations then. So the question why CONFIG_BT_GATT_CACHING=n seems to improve the behaviour still remains. Would you be able to share your project so I can try to debug it here (assuming it can run on a nordic DK without too much effort). Otherwise I'll just try to set up a new project.

Edit: have you defined a new flash partition for your NVS instance so there is not any conflict with the NVS instance used by Settings?

e.g.

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Romain — Tue, 17 Aug 2021 09:21:36 GMT

Hi Vidar,

I am reading and writing a characteristic manually through an app so there is a few seconds between a successive write and read back. I tried again with more than 5s but nothing changed.

[quote userid="4240" url="~/f/nordic-q-a/77781/problem-to-read-back-flash-with-nvs-when-concurrent-use-with-bluetooth/325160#325160"]I didn't manage to replicate the issue with the sample code Simon had uploaded here[/quote][quote userid="24186" url="~/f/nordic-q-a/77781/problem-to-read-back-flash-with-nvs-when-concurrent-use-with-bluetooth/324850#324850"] you need to connect to a central to reproduce the issue[/quote]

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Vidar Berg — Mon, 16 Aug 2021 14:43:25 GMT

Hello,

I didn't manage to replicate the issue with the sample code Simon had uploaded here, but I'm guessing the problem in your case might be that the app is attempting to read back the data before it has actually been stored to flash (Note: write API is not synchronous).

Have you tried to add a delay (k_sleep(), etc) after the write to give NVS more time to complete the flash writes before reading the data back again?

[quote user="rop_"]How is GATT caching interacting with NVS ?[/quote]

GATT caching causes more data to be stored with NVS so that may explain why it takes longer to store your user data.

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Romain — Fri, 13 Aug 2021 12:40:32 GMT

When I implicitly set CONFIG_BT_GATT_CACHING=n (default value is y) it works fine until step 6.

I guess you need to connect to a central to reproduce the issue as there is no issue until step 3.

How is GATT caching interacting with NVS ?

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Romain — Thu, 12 Aug 2021 14:11:15 GMT

Hi Simon,

Yes I got (almost) the same log except that

I had to add CONFIG_LOG_PRINTK=y in proj.conf
The order of printk messages is not the same (due to priority level ?)

*** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***
Before nvs_init() from app
[00:00:10.525,390] [0m<dbg> fs_nvs.nvs_recover_last_ate: Recovering last ate from sector 0[0m
[00:00:10.530,975] [0m<inf> fs_nvs: 3 Sectors of 4096 bytes[0m
[00:00:10.530,975] [0m<inf> fs_nvs: alloc wra: 0, ff0[0m
[00:00:10.530,975] [0m<inf> fs_nvs: data wra: 0, 0[0m
[00:00:10.531,280] [0m<dbg> fs_nvs.nvs_recover_last_ate: Recovering last ate from sector 0[0m
[00:00:10.536,804] [0m<inf> fs_nvs: 8 Sectors of 4096 bytes[0m
[00:00:10.536,804] [0m<inf> fs_nvs: alloc wra: 0, ff0[0m
[00:00:10.536,834] [0m<inf> fs_nvs: data wra: 0, 0[0m
[00:00:10.536,926] [0m<inf> sdc_hci_driver: SoftDevice Controller build revision: 
                                         58 5d 8b 31 54 67 00 e9  b8 4a a7 df a9 9c e4 1c |X].1Tg.. .J......
                                         b3 0b ce 74                                      |...t             [0m
[00:00:10.538,604] [0m<inf> bt_hci_core: No ID address. App must call settings_load()[0m
Enable bluetooth sucess!
Item at id 1 was found: data1
Item at id 2 was found: data2
Item at id 3 was found: data3

I will try to expose some code to reproduce our issue...

Basically we are doing the following

init nvs
read then write the default value if empty nvs
enable Bluetooth (which inits the nvs again)
connect to a central and enable some characteristics
write a new value to nvs
read the last value from nvs

RE: Problem to read back flash with NVS when concurrent use with Bluetooth

Simon — Sun, 25 Jul 2021 14:48:49 GMT

I tried to reproduce this issue but was not able to, can you test the attached sample with NCS v1.6.0 and check if you get the same output:

*** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***
Before nvs_init() from app
Enable bluetooth sucess!
Item at id 1 was found: data1
Item at id 2 was found: data2
Item at id 3 was found: data3
> fs_nvs.nvs_recover_last_ate: Recovering last ate from sector 0
[00:00:00.013,549] <inf> fs_nvs: 3 Sectors of 4096 bytes
[00:00:00.013,549] <inf> fs_nvs: alloc wra: 0, ff0
[00:00:00.013,549] <inf> fs_nvs: data wra: 0, 0
[00:00:00.017,303] <dbg> fs_nvs.nvs_recover_last_ate: Recovering last ate from sector 0
[00:00:00.022,796] <inf> fs_nvs: 8 Sectors of 4096 bytes
[00:00:00.022,827] <inf> fs_nvs: alloc wra: 0, ff0
[00:00:00.022,827] <inf> fs_nvs: data wra: 0, 0
[00:00:00.022,918] <inf> sdc_hci_driver: SoftDevice Controller build revision:
                                         58 5d 8b 31 54 67 00 e9  b8 4a a7 df a9 9c e4 1c |X].1Tg.. .J......
                                         b3 0b ce 74                                      |...t
[00:00:00.024,688] <inf> bt_hci_core: No ID address. App must call settings_load()

devzone.nordicsemi.com/.../nvs_5F00_and_5F00_ble.zip

If I have misunderstood your issue, please attach a sample that demonstrates it.

Best regards,

Simon