[Zigbee] OTA compatibility & custom memory configuration

RE: [Zigbee] OTA compatibility & custom memory configuration

Maria Gilje — Fri, 12 Apr 2024 15:54:59 GMT

Good summary.

[quote user="pwpot"]I assume the "Flash memory configuration" applies only to the flash layout. Is my assumption correct? [/quote]

Yes, you are correct.

Best regards,

Maria

RE: [Zigbee] OTA compatibility & custom memory configuration

Pawel(embeddedsolutions.pl) — Wed, 10 Apr 2024 08:45:52 GMT

Hi Maria,

[quote userid="116814" url="~/f/nordic-q-a/108673/zigbee-ota-compatibility-custom-memory-configuration/477156"]Note that the Flash memory configuration is still important and will cause incompatibility if they differ between current and new image.[/quote]

Ok, to sum things up.

ZBOSS 3.11.2.1 RAM -> These configs can be changed between FOTA/DFU updates.

ZBOSS 3.11.2.1 FLASH -> That data cannot be changed between FOTA/DFU updates but I do not see any configurable configs. I assume the "Flash memory configuration" applies only to the flash layout. Is my assumption correct?

Best regards,

Pawel

RE: [Zigbee] OTA compatibility & custom memory configuration

Maria Gilje — Thu, 04 Apr 2024 13:50:58 GMT

Hi Pawel,

[quote user="pwpot"]So since today, that note is false-positive and each config (see screenshot below) can be changed between FOTA updates without a risk, am I right?[/quote]

Note that the Flash memory configuration is still important and will cause incompatibility if they differ between current and new image.

But yes, the memory configuration for RAM can be changed between FOTA updates.

[quote user="pwpot"]Regarding the production config partition it turned out that the best option is to do the following: [/quote][quote user="pwpot"]

The build system is happy because the zboss_product_config partition exists but does not occupy any memory.

Commenting out the flash_area_open in zb_osif_nvram.c: zb_osif_nvram_init() for me seems not mandatory. In case of a failure it will generate only false-positive log.

[/quote]

Thank you for sharing your solution.

Best regards,

Maria

RE: [Zigbee] OTA compatibility & custom memory configuration

Pawel(embeddedsolutions.pl) — Wed, 03 Apr 2024 10:33:35 GMT

Hi Maria,

Thank you for your effort.

[quote userid="116814" url="~/f/nordic-q-a/108673/zigbee-ota-compatibility-custom-memory-configuration/476764"]This note is not relevant for RAM memory configuration and the note will be removed in a future documentation update.[/quote]

So since today, that note is false-positive and each config (see screenshot below) can be changed between FOTA updates without a risk, am I right?

Regarding the production config partition it turned out that the best option is to do the following:

# prj.conf
CONFIG_PM_PARTITION_SIZE_ZBOSS_PRODUCT_CONFIG


# main.c 

int main(void){

    # some other important zigbee stuff
    zb_production_config_disable(ZB_TRUE);
    zigbee_enable();

}

The build system is happy because the zboss_product_config partition exists but does not occupy any memory.

Commenting out the flash_area_open in zb_osif_nvram.c: zb_osif_nvram_init() for me seems not mandatory. In case of a failure it will generate only false-positive log.

Regards,

Pawel

RE: [Zigbee] OTA compatibility & custom memory configuration

Maria Gilje — Wed, 03 Apr 2024 07:42:22 GMT

Hi Pawel,

Regarding your original question:

[quote user=""]I am looking for clarification regarding that small note [Developing with ZBOSS]:
The memory configuration must not be changed between firmware versions if the firmware is going to be upgraded using the OTA DFU.[/quote]

This note is not relevant for RAM memory configuration and the note will be removed in a future documentation update.

The note is important for Flash memory configuration. It is a common requirement for DFU in general, not just for Zigbee OTA DFU.

[quote user="pwpot"]Additionaly: Is it possible to build app without zboss_product_config partition ? I am not going to use it anyway and at this point 1kB of memory is simply being lost.[/quote]

I got one more insight about this:

zb_production_config_disable() can be used before zboss_start(), in order for ZBOSS not to call zb_osif_prod_cfg_ functions. However the proposed modifications are still needed if you want to remove the partition (in order to avoid build issues).

Best regards,

Maria

RE: [Zigbee] OTA compatibility & custom memory configuration

Maria Gilje — Thu, 21 Mar 2024 14:45:48 GMT

Hi Pawel,

I got a detailed answer from the developers today. Please see their full response below. Note: The workaround described is not tested and if you decide to use it it will be at your own risk.

Hi, regarding the zboss_product_config partition:

It can’t be removed since the libraries are built with ZB_PRODUCTION_CONFIG enabled and zb_osif_prod_cfg_check_presence will try to read from this region. Unexpected behavior can happen if random data is present there.

It’s minimum size is 128 B.

NVS needs to start at an erase-size alignment position. Thus, zboss_product_config can’t be less than 0x1000.

One could think about fitting zboss_product_config at the end of zboss_nvram, but this protection exists for that: https://github.com/nrfconnect/sdk-nrf/blob/v2.4.1/subsys/zigbee/osif/zb_nrf_nvram.c#L18-L19

The only workaround I can think about, at the own risk of the customer, is:

Build with CONFIG_PM_PARTITION_SIZE_ZBOSS_PRODUCT_CONFIG=0x0 or remove the partition from PM.

Modify zb_osif_prod_cfg_check_presence to just return FALSE, without any check.

Modify zb_osif_prod_cfg_read_header to just return RET_ERROR, without any check.

Modify zb_osif_prod_cfg_read to just return RET_ERROR, without any check.

Modify zb_osif_nvram_init to skip the ZB_PRODUCTION_CONFIG part.

The third and fourth bullet point answers this question you had:

[quote user="pwpot"]Will NVS work with such unaligned memory space? [/quote]

Regarding this question:

[quote user="pwpot"]By reducing the size of zboss_product_config you mean setting it to a value lower than the single flash page size of nrf chip? [/quote]

Yes, this was what I meant. See the details in the quote from our developer and please note that if you use the workaround it is at your own risk.

Best regards,

Maria

RE: [Zigbee] OTA compatibility & custom memory configuration

Pawel(embeddedsolutions.pl) — Mon, 18 Mar 2024 13:55:00 GMT

Hi Marie,

By reducing the size of zboss_product_config you mean setting it to a value lower than the single flash page size of nrf chip?

As I mentioned my goal is to use that space for other purposes like custom partition with NVS on top of it.
Setting CONFIG_PM_PARTITION_SIZE_ZBOSS_PRODUCT_CONFIG to 0x80 will give me 0xF80 (0x1000-0x80) available space. Will NVS work with such unaligned memory space?

Best regards,

Pawel

RE: [Zigbee] OTA compatibility & custom memory configuration

Maria Gilje — Wed, 13 Mar 2024 12:01:09 GMT

Hi Pawel,

I have some small updates for you.

[quote user="Maria Gilje"]Can zboss_product_config be completely removed?[/quote]

Removing zboss_product_config causes build errors, so this is not advised.

[quote user="Maria Gilje"]What is the minimum allowed size for zboss_product_config?[/quote]

I'm still waiting for a detailed answer from the developers, but I want to share that my colleague has found that it is possible to configure the size of zboss_product_config with the Kconfig symbol CONFIG_PM_PARTITION_SIZE_ZBOSS_PRODUCT_CONFIG. Reducing it from the default size caused some errors during startup, but the functionality of the device seemed to be as expected. When assigning a new size with the symbol, the value should be in a hex format, i.e. 0x1000.

Note that reducing the size can cause unexpected behaviour.

[quote user="pwpot"]Regarding "How it works under the hood" I meant the memory configuration layout. I would like to know what is causing the OTA incompatibility. [/quote]

I am still awaiting an answer from the developers regarding this.

Best regards,

Maria

RE: [Zigbee] OTA compatibility & custom memory configuration

Pawel(embeddedsolutions.pl) — Wed, 06 Mar 2024 15:08:56 GMT

Indeed, having production data in zboss_product_config is not mandatory. ZBOSS determines if there is something in it by MAGIC value and CRC at init. The problem is that the NCS build forces to add that specific flash memory space but probably it is like that because ZBOSS forces that on NCS.

Thank you for your effort.

Looking forward to hearing from you,

Pawel

RE: [Zigbee] OTA compatibility & custom memory configuration

Maria Gilje — Wed, 06 Mar 2024 12:52:30 GMT

I do not recommend using zboss_product_config for storing other data.

I have sent these questions to the developers:

What is the minimum allowed size for zboss_product_config?
Can zboss_product_config be completely removed?

I included the last question because the list of data which can be saved to zboss_product_config does not seem like a requirement. This was my assumption when I first read it.

This data is:

802.15.4 channel mask
Device IEEE address
Definition of the maximum TX power per channel per page
Install code of the device

These data points need to be saved to your device if you are using them -- and the zboss_product_config is a suitable place to store them.

I want to also let you know that your question about the memory configuration layout will be looked into when possible. Note that it can still take some time.

Best regards,

Maria

RE: [Zigbee] OTA compatibility & custom memory configuration

Pawel(embeddedsolutions.pl) — Fri, 01 Mar 2024 15:06:36 GMT

Thank you,

Regarding zboss_product_config as far as I can see ZBOSS NVRAM does not have an API for writing to that partition.
It seems that it should not be a problem of using NVS on top of it with additional offset of 128Bytes.
Based on: https://github.com/NordicSemiconductor/pc-nrfutil/blob/master/nordicsemi/zigbee/prod_config.py#L54

The idea is to do not waste the memory space, so by adding the 128B offset in NVS the ZBOSS will not even notice that something is placed inside its zboss_product_config partition and because there is no API for writing to that memory space my data should be safe.

What do you think about it ?

Regards,

Pawel

RE: [Zigbee] OTA compatibility & custom memory configuration

Maria Gilje — Fri, 01 Mar 2024 14:35:48 GMT

Hi Pawel,

[quote user="pwpot"]Regarding "How it works under the hood" I meant the memory configuration layout. I would like to know what is causing the OTA incompatibility. [/quote]

I have forwarded this question to the developers and will update you when I have any updates.

[quote user="pwpot"]Additionaly: Is it possible to build app without zboss_product_config partition ? I am not going to use it anyway and at this point 1kB of memory is simply being lost.[/quote]

The system part of the production configuration is mandatory, and the application-specific part is optional. See more in the Production configuration part of the Programming principles documentation.

[quote user="pwpot"]I am satisfied with answers for other questions:) [/quote]

Great!

Best regards,

Maria

RE: [Zigbee] OTA compatibility & custom memory configuration

Pawel(embeddedsolutions.pl) — Wed, 28 Feb 2024 20:47:12 GMT

Hi Maria,

Thank you for fast response.

Regarding "How it works under the hood" I meant the memory configuration layout. I would like to know what is causing the OTA incompatibility.

Additionaly: Is it possible to build app without zboss_product_config partition ? I am not going to use it anyway and at this point 1kB of memory is simply being lost.

I am satisfied with answers for other questions:)

Regards,

Pawel

RE: [Zigbee] OTA compatibility & custom memory configuration

Maria Gilje — Tue, 27 Feb 2024 14:53:40 GMT

Hi Pawel,

[quote user=""] [Developing with ZBOSS]: [/quote]

This link is to the documentation of an old ZBOSS version. The most recent -- and the one applicable for NCS v2.4.1 is found here (ZBOSS 3.11.2.1 for production). Please update bookmarks if you have an older version of the documentation saved. This can save you some confusion in the future.

[quote user=""]Is it valid for all configs or only for part of them ? [/quote]

It's valid for all the configs.

[quote user=""]How this works under the hood ? [/quote]

What are your referring to here? The memory configuration layout, the OTA DFU process? Please elaborate.

[quote user=""]Is OTA incompatibility still valid if I am using static partition layout ? [/quote]

Yes, the requirement for using the same memory configuration for future firmware versions applies when you are using a static partition layout. Note that static partition is recommended when you are planning to do DFU in the future.

[quote user=""]Do I really need to put my custom memory configuration header after zboss_api.h everywhere I am using the zboss_api.h ?[/quote]

No, not everywhere. It is needed in your main application file (often named main.c) and the memory configuration header needs to be included after including zboss_api.h. See the Where to include header files section of the documentation.

[quote user=""]Can it be done in other way in Cmake for eg. ?[/quote]

No, the method is as described in the documentation: including it in the main application file after zboss_api.h is included.

Best regards,

Maria