Troubles to use FOTA + NVS Lib

[Felipe Dantas]

Hi everyone,

I working on a projetc that i have to use the NVS lib to store several user data on external flash, but i need put FOTA capability in this firmware and when i´m included the needed configs on prj.conf  and created the thread that process a connection with nRFClud, the drive of spi flash memory seems that don´t worked propertly. This issue occurs on my proprietrary board but don´t occurs on 9160DK despiste that i´ve use same parts used in DK, but with a diferent pinout in flash IC


here is my overlay:

arduino_spi: &spi3 {
	compatible = "nordic,nrf-spim";
	#address-cells = <0x1>;
	#size-cells = <0x0>;
	reg = <0xb000 0x1000>;
	interrupts = <0xb 0x1>;
	max-frequency = <0x7a1200>;
	easydma-maxcnt-bits = <0xd>;
	status = "okay";
	cs-gpios = <&arduino_header 0x10 0x1>, <&gpio0 0x1C 0x1>;
	pinctrl-0 = <&spi3_default>;
	pinctrl-1 = <&spi3_sleep>;
	pinctrl-names = "default", "sleep";
	mx25r64: mx25r6435f@1 {
		compatible = "jedec,spi-nor";
		status = "okay";
		reg = <0x1>;
		spi-max-frequency = <0x7a1200>;
		jedec-id = [ C2 28 17  ];
		sfdp-bfp = [ E5 20 F1 FF FF FF FF 03 44 EB 08 6B 08 3B 04 BB EE FF FF FF FF FF 00 FF FF FF 00 FF 0C 20 0F 52 10 D8 00 FF 23 72 F5 00 82 ED 04 CC 44 83 48 44 30 B0 30 B0 F7 C4 D5 5C 00 BE 29 FF F0 D0 FF FF  ];
		size = <0x4000000>;
		has-dpd;
		t-enter-dpd = <0x2710>;
		t-exit-dpd = <0x88b8>;
		wp-gpios = <&gpio0 26 0x1>;
		reset-gpios = <&gpio0 29 0x1>;
		mxicy,mx25r-power-mode = "high-performance";
	};
};

&pinctrl {

	uart0_default: uart0_default {
		group1 {
			psels = <NRF_PSEL(UART_TX, 0, 11)>; //6 para ftdi // 11 para 52X
		};
		group2 {
			psels = <NRF_PSEL(UART_RX, 0, 10)>; //5 para ftdi // 10 para 52X
			bias-pull-up;
		};
	};
	uart0_sleep: uart0_sleep {
		group1 {
			psels = <NRF_PSEL(UART_TX, 0, 11)>,
					<NRF_PSEL(UART_RX, 0, 10)>;
			low-power-enable;
		};
	};

	spi3_default: spi3_default {
		group1 {
			psels = <NRF_PSEL(SPIM_SCK, 0, 30)>,
			        <NRF_PSEL(SPIM_MISO, 0, 27)>,
			        <NRF_PSEL(SPIM_MOSI, 0, 31)>;
			nordic,drive-mode = <NRF_DRIVE_H0H1>;
		};
	};

	spi3_sleep: spi3_sleep {
		group1 {
			psels = <NRF_PSEL(SPIM_SCK, 0, 30)>,
			        <NRF_PSEL(SPIM_MISO, 0, 27)>,
			        <NRF_PSEL(SPIM_MOSI, 0, 31)>;
			low-power-enable;
		};
	};
};

&i2c2 {
	status = "disabled";
};

&pwm0 {
	status = "disabled";
};

&adc {
	status = "disabled";
};

/* Enable the external flash device (required) */
&mx25r64 {
	status = "okay";
};

/* Configure partition manager to use mx25r64 as the external flash device */
/ {
	chosen {
		nordic,pm-ext-flash = &mx25r64;
	};
};

/* Enable high performance mode to increase write/erase performance */
&mx25r64 {
	mxicy,mx25r-power-mode = "high-performance";
};

and here is my prj.conf:

#============================================================================
# BOOT OPTIONS
#============================================================================

CONFIG_BOOT_BANNER=y

# MCUBOOT
CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_IMG_MANAGER=y
CONFIG_MCUBOOT_IMG_MANAGER=y
CONFIG_STREAM_FLASH_ERASE=y


#============================================================================
# LOGGING
#============================================================================

CONFIG_LOG=y  
CONFIG_LOG_BACKEND_UART=y 
CONFIG_LOG_BACKEND_RTT=n  
CONFIG_USE_SEGGER_RTT=n    

CONFIG_LOG_MODE_DEFERRED=y
CONFIG_LOG_BUFFER_SIZE=4096

CONFIG_PRINTK=y 

#============================================================================
# PRINTF / FPU
#============================================================================

# Enable print float on console
CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y
CONFIG_FPU=y

#============================================================================
# UART / AT COMMANDS
#============================================================================

CONFIG_SERIAL=y
CONFIG_UART_ASYNC_API=y

CONFIG_AT_HOST_LIBRARY=n

CONFIG_UART_INTERRUPT_DRIVEN=y

# AT monitor
CONFIG_AT_MONITOR_HEAP_SIZE=2048
CONFIG_AT_HOST_STACK_SIZE=2048

#============================================================================
# NEWLIB
#============================================================================

CONFIG_NEWLIB_LIBC=y

#============================================================================
# NETWORKING
#============================================================================

CONFIG_NETWORKING=y
CONFIG_NET_NATIVE=y      ##Sob analise - pos FOTA, original = n.
CONFIG_NET_SOCKETS_OFFLOAD=y
CONFIG_NET_SOCKETS=y
CONFIG_NET_SOCKETS_POSIX_NAMES=y

#============================================================================
# MEMORY / STACKS 
#============================================================================

CONFIG_MAIN_STACK_SIZE=16384
CONFIG_HEAP_MEM_POOL_SIZE=24576
#CONFIG_HEAP_MEM_POOL_SIZE=8192
CONFIG_ISR_STACK_SIZE=8192
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=8192



#============================================================================
# MODEM LIBRARY / MODEM CONFIG
#============================================================================

CONFIG_NRF_MODEM_LIB=y
CONFIG_NRF_MODEM_LIB_TRACE=n

CONFIG_EVENTS=y
CONFIG_RESET_ON_FATAL_ERROR=y

#============================================================================
# LTE LINK CONTROL / NETWORK
#============================================================================

CONFIG_LTE_LINK_CONTROL=y
CONFIG_LTE_NETWORK_MODE_LTE_M_NBIOT=y

#============================================================================
# PSM CONFIG 
#============================================================================

CONFIG_LTE_PSM_REQ=y
CONFIG_LTE_PSM_REQ_FORMAT_SECONDS=y
CONFIG_LTE_PSM_REQ_RAT_SECONDS=0
CONFIG_LTE_PSM_REQ_RPTAU_SECONDS=14400

#============================================================================
# MQTT
#============================================================================

CONFIG_MQTT_LIB=y
CONFIG_MQTT_CLEAN_SESSION=y

CONFIG_MQTT_KEEPALIVE=120

CONFIG_MQTT_MESSAGE_BUFFER_SIZE=4096
CONFIG_MQTT_PAYLOAD_BUFFER_SIZE=4096
CONFIG_MQTT_LOG_LEVEL_DBG=y

#============================================================================
# FLASH / NVS
#============================================================================

CONFIG_FLASH=y
CONFIG_NVS=y
CONFIG_FLASH_MAP=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_STREAM_FLASH=y
CONFIG_MPU_ALLOW_FLASH_WRITE=y
CONFIG_PM_EXTERNAL_FLASH_MCUBOOT_SECONDARY=n

#============================================================================
# DATE / TIME
#============================================================================

CONFIG_DATE_TIME=y
CONFIG_DATE_TIME_AUTO_UPDATE=y
CONFIG_DATE_TIME_NTP=y
CONFIG_DATE_TIME_MODEM=n
CONFIG_DATE_TIME_UPDATE_INTERVAL_SECONDS=86400

#============================================================================
# SPI
#============================================================================

CONFIG_SPI=y
CONFIG_SPI_NOR=y
CONFIG_SPI_NOR_SFDP_DEVICETREE=y
CONFIG_PM_OVERRIDE_EXTERNAL_DRIVER_CHECK=y
CONFIG_SPI_NOR_FLASH_LAYOUT_PAGE_SIZE=4096
CONFIG_SPI_COMPLETION_TIMEOUT_TOLERANCE=10000

#============================================================================
# TLS
#============================================================================

CONFIG_MQTT_LIB_TLS=y
CONFIG_MODEM_KEY_MGMT=y

#============================================================================
# FTP
#============================================================================

CONFIG_FTP_CLIENT=y

#============================================================================
# NRF CLOUD E FOTA
#============================================================================

CONFIG_NRF_CLOUD_MQTT=y   # Habilita stack nRF Cloud MQTT
CONFIG_NRF_CLOUD_FOTA=y   # Habilita FOTA

CONFIG_NRF_CLOUD_SEND_DEVICE_STATUS=y
CONFIG_NRF_CLOUD_SEND_DEVICE_STATUS_NETWORK=y
CONFIG_NRF_CLOUD_SEND_DEVICE_STATUS_SIM=y
CONFIG_NRF_CLOUD_SEND_SERVICE_INFO_FOTA=y
CONFIG_NRF_CLOUD_SEND_SERVICE_INFO_UI=y

# nRF Cloud alert/log
CONFIG_NRF_CLOUD_ALERT=y
CONFIG_NRF_CLOUD_LOG_DIRECT=y
CONFIG_NRF_CLOUD_LOG_OUTPUT_LEVEL=3

#============================================================================
# DOWNLOAD CLIENT
#============================================================================

CONFIG_DOWNLOAD_CLIENT=y
CONFIG_DOWNLOAD_CLIENT_HTTP_FRAG_SIZE_1024=y
CONFIG_DOWNLOAD_CLIENT_STACK_SIZE=4096
CONFIG_DOWNLOAD_CLIENT_BUF_SIZE=2300
CONFIG_DOWNLOAD_CLIENT_MAX_HOSTNAME_SIZE=128

#============================================================================
# LOCATION SERVICES
#============================================================================

CONFIG_LOCATION=y
CONFIG_LOCATION_METHOD_GNSS=y
CONFIG_LOCATION_METHOD_CELLULAR=y
CONFIG_NRF_CLOUD_AGNSS=y
CONFIG_NRF_CLOUD_LOCATION=y
CONFIG_MODEM_INFO=y
CONFIG_MODEM_INFO_ADD_NETWORK=y
CONFIG_NRF_CLOUD_PGPS=y
CONFIG_NRF_CLOUD_PGPS_REPLACEMENT_THRESHOLD=4
CONFIG_NRF_CLOUD_PGPS_REQUEST_UPON_INIT=y

CONFIG_LTE_LINK_CONTROL=y 

#============================================================================
# SETTINGS
#============================================================================

CONFIG_SETTINGS=y
CONFIG_SETTINGS_FCB=y
CONFIG_FCB=y

#============================================================================
# REDE - CONFIGS EXTRAS
#============================================================================

CONFIG_NET_MGMT_EVENT_STACK_SIZE=2048
CONFIG_NET_CONNECTION_MANAGER_MONITOR_STACK_SIZE=1024

CONFIG_NET_IPV4=y
CONFIG_NET_IPV6=y
CONFIG_NET_IPV6_NBR_CACHE=n
CONFIG_NET_IPV6_MLD=n
CONFIG_NET_CONNECTION_MANAGER=y
CONFIG_NRF_MODEM_LIB_ON_FAULT_APPLICATION_SPECIFIC=y
CONFIG_NRF_MODEM_LIB_NET_IF=y
CONFIG_NRF_MODEM_LIB_NET_IF_AUTO_DOWN=y
CONFIG_NRF_MODEM_LIB_NET_IF_DOWN_DEFAULT_LTE_DISCONNECT=y

Finally, he is that i receive on log:

[00:00:00.252,716] <ESC>[1;31m<err> spi_nor: Device id 00 00 00 does not match config c2 28 17<ESC>[0m<CR><LF>
*** Booting nRF Connect SDK 3758bcbfa5cd ***<CR><LF>
[00:00:00.258,331] <ESC>[0m<inf> fs_nvs: 6 Sectors of 4096 bytes<ESC>[0m<CR><LF>
[00:00:00.258,331] <ESC>[0m<inf> fs_nvs: alloc wra: 0, e90<ESC>[0m<CR><LF>
[00:00:00.258,361] <ESC>[0m<inf> fs_nvs: data wra: 0, 518<ESC>[0m<CR><LF>
[00:00:00.258,392] <ESC>[1;31m<err> PluviDB_IoT: Flash ex device mx25r6435f@1 is not ready<LF>
<ESC>[0m<CR><LF>
[00:00:00.258,392] <ESC>[0m<inf> PluviDB_IoT: Initializing modem library.<ESC>[0m<CR><LF>
Inicializando a nRF Cloud teste A<CR><LF>
Inicializando First Boot nRF Cloud<CR><LF>
[00:00:00.544,403] <ESC>[0m<inf> PluviDB_IoT: match<LF>
<ESC>[0m<CR><LF>
[00:00:00.544,403] <ESC>[0m<inf> PluviDB_IoT: Connecting to LTE network.<ESC>[0m<CR><LF>
[00:00:00.835,540] <ESC>[0m<dbg> PluviDB_IoT: get_bat_voltage: VBAT: 3.696 V<ESC>[0m<CR><LF>
[00:00:00.844,512] <ESC>[1;31m<err> PluviDB_IoT: AT XMonitor full raw: %XMONITOR: 0<CR><LF>
OK<CR><LF>
.<CR><LF>

Note the first line. This only occurs on my proprietary board. 

Some help are welcome.

[Simonr]

Hi

You say this only occurs on a custom board. The error message here points to the SPI_NOR library not being able to communicate with the external flash memory. Can you confirm whether you have an external NOR flash on your custom board or not? The application expects that you have an mx25r6435f on the board that's connected to the device.

Best regards,

Simon

[Felipe Dantas]

Hi Simon,

Of course! As said, i have an external flash on board, and like you can see on my code, this is an mx25r6435.

Best regards,

Felipe

[Hieu]

Hi Felipe,

What model of external flash device do you have on your custom board? Like Simon said, the SPI_NOR library isn't able to read the JEDEC ID from the external flash correctly.

It could be a hardware issue. For that, let's go through the usual checks:

• Is the physical connection between the two devices intact?
• Are the chosen pins on the nRF9160 right for the purpose? I.e., do they have any usage restriction?
• Have this been tested with multiple boards, or just one?
• Does the board design ensure that the external flash is fully powered and ready before the nRF9160 starts communicating with it?

From the log, it seems that the error is thrown from the bootloader. Does the application have any features with the external flash? If so, do they work normally?

Best regards,

Hieu

[Felipe Dantas]

Hi Hieu,

So, about ic model, i think that i understood somethink wrong. On my initial post, i´ve said that i´ve used the same parts used in 9160DK, that have an mx25r6435f IC flash memory and this is are declared on code, but you and Simon asked me the same think. The information that you and Simon are needing is other think? Sorry, i really didnt´t understood.

We´re in pre production stage, so i´ve some like 10 bords in lab for tests now. About the eletrical connectios, are good. As i said, the firmware work normally without the code pieces needed by FOTA scheme. I can write and read normally in the mx25r6435, booth my board and in dk.

But the most stranger thing here is that the firmware with fota work perfectly on dk, but not in mine, despiste that i´ve declared the correct pinout on overlay file like you can see in my code (and if is wrong, this firmware wouldn´t in any condition and any board).

So, trying be a little more clearer:

- IC flash memory model(i think): mx25r6435f

Work conditions:

- DK and firmware without fota: works normally. can read and write on external flash memory mx25r6435f normally.

- DK and firmware with fota: works normally. can read and write on external flash memory mx25r6435f normally and make fota update throug nrfCloud.

- Custom boar without fota: works normally. can read and write on external flash memory mx25r6435f normally.

- Custom board with fota: don´t work propertly. external flash memory mx25r6435f are not recognized by spim drive like yoou can see in firt line of application log. 

This seems to me more like a competition for pins going on behind the scenes by something in Zephyr or something called by the libraries included by FOTA.

So... is it for now.

Best regards,

Felipe

[Hieu]

Hi Felipe,

Felipe Dantas said:

- Custom boar without fota: works normally. can read and write on external flash memory mx25r6435f normally.

- Custom board with fota: don´t work propertly. external flash memory mx25r6435f are not recognized by spim drive like yoou can see in firt line of application log. 

How is the read and write on external flash done in your test no-FOTA case? In particular, I want to know if you are using Zephyr driver which initialize before main() even starts.

For the with-FOTA case, does the application work? Like I mentioned, the log suggests that the error is in the bootloader stage, but the application boot up still works. I want to know if the application can access the external flash normally.

This will help tell if the external flash isn't started early enough.

By the way, have you had a hardware design review case with us yet?

Best regards,

Hieu

[Felipe Dantas]

Hi Hieu,

To use the mx25r6435f, i followed the steps on page in the link below (Exept the initial step "Programming board controller" cause i´m in custom boar):

https://docs.nordicsemi.com/bundle/ncs-2.9.0/page/nrf/app_dev/device_guides/nrf91/nrf9160_external_flash.html

So, i include nvs.h lib and use him following his documentation in the link below:

https://docs.nordicsemi.com/bundle/ncs-latest/page/zephyr/services/storage/nvs/nvs.html

About the hardware design review case, we never had yet.

Best regards,

Felipe Dantas

[Hieu]

Hi Felipe,

What about this case?

Hieu said:

For the with-FOTA case, does the application work? Like I mentioned, the log suggests that the error is in the bootloader stage, but the application boot up still works. I want to know if the application can access the external flash normally.

Can the application access the external flash and use it normally?

Could you please also open a private case for hardware review? Please mention this case in that new case.

Best regards,

Hieu

[Felipe Dantas]

Hi Hieu,

I think that this is you want to know:

Felipe Dantas said:

Work conditions:

- DK and firmware without fota: works normally. can read and write on external flash memory mx25r6435f normally.

- DK and firmware with fota: works normally. can read and write on external flash memory mx25r6435f normally and make fota update throug nrfCloud.

- Custom boar without fota: works normally. can read and write on external flash memory mx25r6435f normally.

- Custom board with fota: don´t work propertly. external flash memory mx25r6435f are not recognized by spim drive like yoou can see in firt line of application log. 

Above i described all 4 scenaries that we have now.

So, being more clarify about your question: In any case, if memory aren´t recognized, any nvs funtion like nvs_init(), nvs_read(), nvs_write() will return error, but the rest of application still running.

I´ll open a private case as soon as possible. Thanks for your help.

Best regards,

Felipe