[Zigbee] OTA compatibility & custom memory configuration

[Pawel(embeddedsolutions.pl)]

Setup: 
NCS v2.4.1

nrf52840DK (ZR)

Hi, 

I am looking for clarification regarding that small note [Developing with ZBOSS]: 
The memory configuration must not be changed between firmware versions if the firmware is going to be upgraded using the OTA DFU.

1. Is it valid for all configs or only for part of them ? 
2. How this works under the hood ? 
3. Is OTA incompatibility still valid if I am using static partition layout ? 
4. Do I really need to put my custom memory configuration header after zboss_api.h everywhere I am using the zboss_api.h ? Can it be done in other way in Cmake for eg. ?

I look forward to hearing from you,

Pawel

[Maria Gilje]

Hi Pawel,

[Developing with ZBOSS]: 

This link is to the documentation of an old ZBOSS version. The most recent -- and the one applicable for NCS v2.4.1 is found here (ZBOSS 3.11.2.1 for production). Please update bookmarks if you have an older version of the documentation saved. This can save you some confusion in the future.

Is it valid for all configs or only for part of them ? 

It's valid for all the configs.

How this works under the hood ? 

What are your referring to here? The memory configuration layout, the OTA DFU process? Please elaborate.

Is OTA incompatibility still valid if I am using static partition layout ? 

Yes, the requirement for using the same memory configuration for future firmware versions applies when you are using a static partition layout. Note that static partition is recommended when you are planning to do DFU in the future.

Do I really need to put my custom memory configuration header after zboss_api.h everywhere I am using the zboss_api.h ?

No, not everywhere. It is needed in your main application file (often named main.c) and the memory configuration header needs to be included after including zboss_api.h. See the Where to include header files section of the documentation.

Can it be done in other way in Cmake for eg. ?

No, the method is as described in the documentation: including it in the main application file after zboss_api.h is included.

Best regards,

Maria

[Pawel(embeddedsolutions.pl)]

Hi Maria, 

Thank you for fast response. 

Regarding "How it works under the hood" I meant the memory configuration layout. I would like to know what is causing the OTA incompatibility. 

Additionaly: Is it possible to build app without zboss_product_config partition ? I am not going to use it anyway and at this point 1kB of memory is simply being lost.

I am satisfied with answers for other questions:) 

Regards,

Pawel

[Maria Gilje]

Hi Pawel,

Pawel(embeddedsolutions.pl) said:

Regarding "How it works under the hood" I meant the memory configuration layout. I would like to know what is causing the OTA incompatibility. 

I have forwarded this question to the developers and will update you when I have any updates.

Pawel(embeddedsolutions.pl) said:

Additionaly: Is it possible to build app without zboss_product_config partition ? I am not going to use it anyway and at this point 1kB of memory is simply being lost.

The system part of the production configuration is mandatory, and the application-specific part is optional. See more in the Production configuration part of the Programming principles documentation.

Pawel(embeddedsolutions.pl) said:

I am satisfied with answers for other questions:) 

Great!

Best regards,

Maria

[Pawel(embeddedsolutions.pl)]

Thank you, 

Regarding zboss_product_config as far as I can see ZBOSS NVRAM does not have an API for writing to that partition. 
It seems that it should not be a problem of using NVS on top of it with additional offset of 128Bytes. 
Based on: https://github.com/NordicSemiconductor/pc-nrfutil/blob/master/nordicsemi/zigbee/prod_config.py#L54

The idea is to do not waste the memory space, so by adding the 128B offset in NVS the ZBOSS will not even notice that something is placed inside its zboss_product_config partition and because there is no API for writing to that memory space my data should be safe. 

What do you think about it ? 

Regards,

Pawel

[Maria Gilje]

I do not recommend using zboss_product_config for storing other data.

I have sent these questions to the developers:

• What is the minimum allowed size for zboss_product_config?
• Can zboss_product_config be completely removed?

I included the last question because the list of data which can be saved to zboss_product_config does not seem like a requirement. This was my assumption when I first read it.

This data is:

• 802.15.4 channel mask
• Device IEEE address
• Definition of the maximum TX power per channel per page
• Install code of the device

These data points need to be saved to your device if you are using them -- and the zboss_product_config is a suitable place to store them.

I want to also let you know that your question about the memory configuration layout will be looked into when possible. Note that it can still take some time.

Best regards,

Maria

[Pawel(embeddedsolutions.pl)]

Indeed, having production data in zboss_product_config is not mandatory. ZBOSS determines if there is something in it by MAGIC value and CRC at init. The problem is that the NCS build forces to add that specific flash memory space but probably it is like that because ZBOSS forces that on NCS.

Thank you for your effort. 

Looking forward to hearing from you,

Pawel

[Maria Gilje]

Hi Pawel,

I have some small updates for you.

Maria Gilje said:

Can zboss_product_config be completely removed?

Removing zboss_product_config causes build errors, so this is not advised.

Maria Gilje said:

What is the minimum allowed size for zboss_product_config?

I'm still waiting for a detailed answer from the developers, but I want to share that my colleague has found that it is possible to configure the size of zboss_product_config with the Kconfig symbol CONFIG_PM_PARTITION_SIZE_ZBOSS_PRODUCT_CONFIG. Reducing it from the default size caused some errors during startup, but the functionality of the device seemed to be as expected. When assigning a new size with the symbol, the value should be in a hex format, i.e. 0x1000.

Note that reducing the size can cause unexpected behaviour.

Pawel(embeddedsolutions.pl) said:

Regarding "How it works under the hood" I meant the memory configuration layout. I would like to know what is causing the OTA incompatibility. 

I am still awaiting an answer from the developers regarding this.

Best regards,

Maria

[Pawel(embeddedsolutions.pl)]

Hi Marie,

By reducing the size of zboss_product_config you mean setting it to a value lower than the single flash page size of nrf chip? 

As I mentioned my goal is to use that space for other purposes like custom partition with NVS on top of it.
Setting CONFIG_PM_PARTITION_SIZE_ZBOSS_PRODUCT_CONFIG to 0x80 will give me 0xF80 (0x1000-0x80) available space. Will NVS work with such unaligned memory space? 

Best regards,

Pawel

[Maria Gilje]

Hi Pawel,

I got a detailed answer from the developers today. Please see their full response below. Note: The workaround described is not tested and if you decide to use it it will be at your own risk.

Hi, regarding the zboss_product_config partition:

• It can’t be removed since the libraries are built with ZB_PRODUCTION_CONFIG enabled and zb_osif_prod_cfg_check_presence will try to read from this region. Unexpected behavior can happen if random data is present there.

• It’s minimum size is 128 B.

• NVS needs to start at an erase-size alignment position. Thus, zboss_product_config can’t be less than 0x1000.

• One could think about fitting zboss_product_config at the end of zboss_nvram, but this protection exists for that: https://github.com/nrfconnect/sdk-nrf/blob/v2.4.1/subsys/zigbee/osif/zb_nrf_nvram.c#L18-L19

• The only workaround I can think about, at the own risk of the customer, is:

  1. Build with CONFIG_PM_PARTITION_SIZE_ZBOSS_PRODUCT_CONFIG=0x0 or remove the partition from PM.

  2. Modify zb_osif_prod_cfg_check_presence to just return FALSE, without any check.

  3. Modify zb_osif_prod_cfg_read_header to just return RET_ERROR, without any check.

  4. Modify zb_osif_prod_cfg_read to just return RET_ERROR, without any check.

  5. Modify zb_osif_nvram_init to skip the ZB_PRODUCTION_CONFIG part.

The third and fourth bullet point answers this question you had:

Pawel(embeddedsolutions.pl) said:

Will NVS work with such unaligned memory space? 

Regarding this question:

Pawel(embeddedsolutions.pl) said:

By reducing the size of zboss_product_config you mean setting it to a value lower than the single flash page size of nrf chip? 

Yes, this was what I meant. See the details in the quote from our developer and please note that if you use the workaround it is at your own risk.

Best regards,

Maria

[Maria Gilje]

Hi Pawel,

Regarding your original question:

I am looking for clarification regarding that small note [Developing with ZBOSS]: 
The memory configuration must not be changed between firmware versions if the firmware is going to be upgraded using the OTA DFU.

This note is not relevant for RAM memory configuration and the note will be removed in a future documentation update.

The note is important for Flash memory configuration. It is a common requirement for DFU in general, not just for Zigbee OTA DFU.

Pawel(embeddedsolutions.pl) said:

Additionaly: Is it possible to build app without zboss_product_config partition ? I am not going to use it anyway and at this point 1kB of memory is simply being lost.

I got one more insight about this:

zb_production_config_disable() can be used before zboss_start(), in order for ZBOSS not to call zb_osif_prod_cfg_ functions. However the proposed modifications are still needed if you want to remove the partition (in order to avoid build issues).

Best regards,

Maria