Regarding flash overflow when dfu is enabled

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Fri, 13 Sep 2024 13:38:13 GMT

Are you using the unmodified download client sample with nRF7002DK (since you mentioned MS12SF1(nrf7002+nrf5340) in the original post)?

Since we setup nginx web server serving both http and https a big file (1 MB size) and then tested the download_client library using the samples/net/download/ sample application and everything worked fine for both http and https connections, there was no issue with the download_client library. Therefore, I would suggest you test with other web servers such nginx web server.

RE: Regarding flash overflow when dfu is enabled

Rakshita — Fri, 13 Sep 2024 05:22:20 GMT

U tested with a very small file
In my case the file size is around740kb can u once test with that

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Thu, 12 Sep 2024 19:30:37 GMT

Your heap is ok because the TLS connection was established in your logs.

I have run the unmodified Download client sample on nRF7002DK and it doesn't have any issues.

Please test with the unmodified Download client sample on nRF7002DK and refer to it.

RE: Regarding flash overflow when dfu is enabled

Rakshita — Wed, 11 Sep 2024 12:56:30 GMT

Hey Amanda

So when I enabled the extra configurations which u mentioned above the issue is that I am using CONFIG_MBEDTLS_HEAP_SIZE=81920 and u used 220000 which is quite large and is overflowing the ram
Wanted to know that does increasing this value has some impact on the downloading procedure?
The previous results which I provided was of nrf7002DK only

Also wanted to know that why does after a certain size which I mentioned in my above configurations also, the response which I receive is of connection: close and it is always happening after sending 100 GET requests

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Tue, 10 Sep 2024 19:12:26 GMT

Hi,

We setup nginx web server serving both http and https a big file (1 MB size). Then tested the download_client library using the samples/net/download/ sample application and everything worked fine for both http and https connections, there was no issue with the download_client library.

The following setup was used in the test:

west build -p -b native_sim samples/net/download/ -d ../build/fota-download -- \
-DCONFIG_NATIVE_UART_AUTOATTACH_DEFAULT_CMD="\"gnome-terminal -- screen %s\"" \
-DCONFIG_NET_SHELL=y \
-DCONFIG_SAMPLE_FILE_CUSTOM=y \
-DCONFIG_SAMPLE_FILE_URL="\"https://192.0.2.2/big-file.txt\\"" \
-DCONFIG_DOWNLOAD_CLIENT_LOG_LEVEL_DBG=y \
-DCONFIG_DOWNLOAD_CLIENT_LOG_HEADERS=y \
-DCONFIG_NET_SOCKETS_LOG_LEVEL_DBG=y \
-DCONFIG_LOG=y -DCONFIG_NET_LOG=y \
-DCONFIG_NET_CONTEXT_RCVTIMEO=y \
-DCONFIG_NET_CONTEXT_SNDTIMEO=y \
-DCONFIG_MBEDTLS_DEBUG=y CONFIG_MBEDTLS_LOG_LEVEL_DBG=y \
-DCONFIG_NET_SOCKETS_POLL_MAX=10 \
-DCONFIG_MBEDTLS=y \
-DCONFIG_MBEDTLS_ENABLE_HEAP=y \
-DCONFIG_MBEDTLS_HEAP_SIZE=220000 \
-DCONFIG_MBEDTLS_TLS_VERSION_1_2=y \
-DCONFIG_MBEDTLS_BUILTIN=y \
-DCONFIG_MBEDTLS_CIPHER_CHACHA20_ENABLED=y \
-DCONFIG_MBEDTLS_KEY_EXCHANGE_ALL_ENABLED=y \
-DCONFIG_MBEDTLS_CIPHER_ALL_ENABLED=y \
-DCONFIG_MBEDTLS_ECP_ALL_ENABLED=y \
-DCONFIG_MBEDTLS_HASH_ALL_ENABLED=y \
-DCONFIG_MBEDTLS_GENPRIME_ENABLED=y \
-DCONFIG_MBEDTLS_HMAC_DRBG_ENABLED=y \
-DCONFIG_MBEDTLS_ECDH_C=y \
-DCONFIG_MBEDTLS_ECDSA_C=y \
-DCONFIG_MBEDTLS_ECJPAKE_C=y \
-DCONFIG_MBEDTLS_ECP_C=y \
-DCONFIG_MBEDTLS_MAC_POLY1305_ENABLED=y \
-DCONFIG_MBEDTLS_MAC_ALL_ENABLED=y

If the above cannot help, could you test with nRF7002DK and provide the project for nRF7002DK?

RE: Regarding flash overflow when dfu is enabled

Rakshita — Wed, 04 Sep 2024 19:27:15 GMT

But how is that possible that after every 204800 bytes only the downloading stops, if the buffer size is 2048 ? If I suppose decrease the buffer size to 512 also the downloading stops at 51200 bytes, something related with 100
Do u know what that could be? Also I am attaching my prj.conf file for better understanding

# Wi-Fi
CONFIG_WIFI=y
CONFIG_WIFI_NRF700X=y
CONFIG_WPA_SUPP=y

# Wi-Fi credentials
CONFIG_WIFI_CREDENTIALS=y
CONFIG_WIFI_CREDENTIALS_STATIC=y
CONFIG_WIFI_CREDENTIALS_STATIC_SSID="UltraHuman-4th Floor"
CONFIG_WIFI_CREDENTIALS_STATIC_PASSWORD="Ultra@Human$!"

# Networking Management API
CONFIG_NET_MGMT=y
CONFIG_NET_MGMT_EVENT=y
CONFIG_NET_MGMT_EVENT_INFO=y
CONFIG_NET_MGMT_EVENT_STACK_SIZE=4096
CONFIG_NET_CONNECTION_MANAGER=y
CONFIG_WIFI_MGMT_EXT=y

# Networking
CONFIG_NETWORKING=y
CONFIG_NET_NATIVE=y
CONFIG_NET_SOCKETS=y
CONFIG_NET_SOCKETS_POSIX_NAMES=y
CONFIG_POSIX_MAX_FDS=12

CONFIG_NET_L2_ETHERNET=y
CONFIG_NET_IPV4=y
CONFIG_NET_IPV6=n
CONFIG_NET_TCP=y
CONFIG_NET_DHCPV4=y
CONFIG_DNS_RESOLVER=y

# Memory
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=4096
CONFIG_MAIN_STACK_SIZE=4096
CONFIG_HEAP_MEM_POOL_SIZE=153600

CONFIG_NET_TCP_WORKQ_STACK_SIZE=2048
CONFIG_NET_TX_STACK_SIZE=4096
CONFIG_NET_RX_STACK_SIZE=4096
CONFIG_NET_BUF_RX_COUNT=20
CONFIG_NET_BUF_TX_COUNT=20

CONFIG_NET_BUF_DATA_SIZE=256
CONFIG_NET_TC_TX_COUNT=0

# # HTTP
# STEP 1 -  Enable the HTTP client library
CONFIG_HTTP_CLIENT=y

CONFIG_LOG=y
CONFIG_USE_SEGGER_RTT=y
CONFIG_LOG_BACKEND_RTT=y
CONFIG_LOG_BACKEND_UART=n
CONFIG_LOG_PRINTK=y

CONFIG_USE_SEGGER_RTT=y
CONFIG_RTT_CONSOLE=y
CONFIG_CBPRINTF_FP_SUPPORT=y

CONFIG_SETTINGS=y                                   #Enables the settings subsystem
CONFIG_FLASH=y                                      #Enables the flash drivers
CONFIG_FLASH_PAGE_LAYOUT=y                          #API for retrieving the layout of pages
CONFIG_FLASH_MAP=y                                  #Enables the flash map abstraction module
CONFIG_NVS=y                                       #Enables support for non-volatile storage
CONFIG_SETTINGS_NVS=y

CONFIG_WPA_SUPP_CRYPTO_PSA=y

# # TLS
# # STEP 1.1 - Enable the TLS library and security relevant configurations
CONFIG_NET_SOCKETS_SOCKOPT_TLS=y
CONFIG_TLS_CREDENTIALS=y
CONFIG_MBEDTLS_RSA_C=y
CONFIG_MBEDTLS_DHM_C=y 

# # # STEP 1.2 - Enable support for server name indication (SNI)
CONFIG_MBEDTLS_SSL_SERVER_NAME_INDICATION=y


CONFIG_MBEDTLS=y
CONFIG_MBEDTLS_TLS_LIBRARY=y
CONFIG_TLS_CREDENTIALS_BACKEND_PROTECTED_STORAGE=y


CONFIG_POSIX_CLOCK=y

CONFIG_NET_CONNECTION_MANAGER=y
CONFIG_NET_CONNECTION_MANAGER_MONITOR_STACK_SIZE=1024

# CONFIG_BUILD_WITH_TFM=y

CONFIG_NORDIC_SECURITY_BACKEND=y
CONFIG_CC3XX_BACKEND=y
CONFIG_MBEDTLS_TLS_LIBRARY=y
CONFIG_MBEDTLS_ECDSA_DETERMINISTIC=n
CONFIG_MBEDTLS_SSL_SRV_C=n
CONFIG_MBEDTLS_X509_CSR_WRITE_C=y
CONFIG_MBEDTLS_X509_CREATE_C=y
CONFIG_MBEDTLS_PK_WRITE_C=y
CONFIG_MBEDTLS_SSL_SERVER_NAME_INDICATION=y
CONFIG_MBEDTLS_PKCS1_V15=y

# For hardware crypto accelerator
CONFIG_PSA_CRYPTO_DRIVER_OBERON=n
CONFIG_PSA_CRYPTO_DRIVER_CC3XX=y

CONFIG_MBEDTLS_PSA_CRYPTO_C=y
CONFIG_MBEDTLS_ENABLE_HEAP=y
CONFIG_MBEDTLS_HEAP_SIZE=81920
CONFIG_MBEDTLS_ENABLE_HEAP=y
CONFIG_MBEDTLS_MAC_SHA256_ENABLED=y
# CONFIG_MBEDTLS_SSL_MAX_CONTENT_LEN=8192
CONFIG_MBEDTLS_TLS_LIBRARY=y

#FOTA
# CONFIG_FOTA_DOWNLOAD=y
CONFIG_DOWNLOAD_CLIENT=y
CONFIG_DOWNLOAD_CLIENT_STACK_SIZE=4096

# DFU Target
# CONFIG_DFU_TARGET=y

# # Application Upgrade support
# CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_DOWNLOAD_CLIENT_LOG_HEADERS=y
CONFIG_DOWNLOAD_CLIENT_LOG_LEVEL_DBG=y
# CONFIG_FOTA_DOWNLOAD_LOG_LEVEL_DBG=y


CONFIG_LOG_BUFFER_SIZE=8192

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Wed, 04 Sep 2024 18:23:32 GMT

The CONFIG_DOWNLOAD_CLIENT_BUF_SIZE and CONFIG_DOWNLOAD_CLIENT_HTTP_FRAG_SIZE parameters in the Kconfig file are not directly related to the total size of the file you want to download. They define the size of the buffer used for each fragment of the download and the maximum size of each HTTP fragment respectively.

For " <err> download_client: Unable to connect, errno 2" in your log, see my colleague's reply in this post and this pot.

In the log, <err> fota: Error -117 means the Host is down.

RE: Regarding flash overflow when dfu is enabled

Rakshita — Wed, 04 Sep 2024 15:58:17 GMT

Yes, so the maximum size of CONFIG_DOWNLOAD_CLIENT_BUF_SIZE and CONFIG_DOWNLOAD_CLIENT_HTTP_FRAG_SIZE is 4096 as per defined in the Kconfig file but the size of binary file which I need to download is of size 740kb For this do I need to add one more size value in the Kconfig like 8192? The issue is that it doesn't reconnects again why is that

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Wed, 04 Sep 2024 15:44:39 GMT

If you are using HTTPS, you need to check the configuration described in the Download client for HTTP and HTTPS (TLS 1.2).

Set the CONFIG_DOWNLOAD_CLIENT_BUF_SIZE and CONFIG_DOWNLOAD_CLIENT_HTTP_FRAG_SIZE Kconfig options, so that the buffer is large enough to accommodate the entire HTTP header of the request and the response.

Also this post.

RE: Regarding flash overflow when dfu is enabled

Rakshita — Wed, 04 Sep 2024 08:51:24 GMT

Thanks
Yes I was able to download file but the issue is that if my file size is large enough like 500kb or something, the downloading process stops at 204800 bytes if CONFIG_DOWNLOAD_CLIENT_BUF_SIZE=2048

I followed https://devzone.nordicsemi.com/f/nordic-q-a/113704/fota-download-breaking-up this thread and also made the required changes in the download_client library but still facing the same issue.

[00:01:43.639,343] <wrn> download_client: Peer closed connection, will re-connect
[00:01:43.639,373] <dbg> download_client: download_thread: Receiving up to 2048 bytes at 0x2000095c...
[00:01:44.354,797] <dbg> download_client: handle_received: Read 2048 bytes from socket
[00:01:44.354,827] <inf> download_client: Downloaded 204800/555880 bytes (36%)

[  36% ] |==================                                | (204800/555880 bytes)[00:01:44.355,529] <inf> download_client: Reconnecting...
[00:01:44.370,086] <dbg> download_client: client_connect: Port not specified, using default: 443
[00:01:44.370,208] <dbg> download_client: host_lookup: Failed to resolve hostname s3.amazonaws.com on IPv6
[00:01:44.393,005] <dbg> download_client: client_socket_connect: family: 1, type: 1, proto: 258
11122[00:01:44.393,707] <inf> download_client: Setting up TLS credentials, sec tag count 1
[00:01:44.393,798] <inf> download_client: Connecting to 52.217.125.16
[00:01:44.393,829] <dbg> download_client: client_socket_connect: fd 9, addrlen 8, fam IPv4, port 443
[00:01:44.394,287] <err> download_client: Unable to connect, errno 2
[00:01:44.394,622] <err> fota: Error -2 during download

[00:01:44.399,963] <inf> fota: Network disconnected
[00:01:44.418,762] <dbg> download_client: set_state: state = 0
[00:01:44.418,792] <inf> fota: Socket closed

[00:01:44.418,792] <err> fota: Error -117 during download

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Tue, 03 Sep 2024 14:33:30 GMT

Hi,

There are AWS IoT and Azure IoT Hub samples in NCS demonstrate FOTA over Wi-Fi. In addition, there is an exercise in the nRF Connect SDK Intermediate course that guides you through the process of using AWS to perform FOTA over Wi-Fi for an nRF7002DK. The exercise code is based on the AWS IoT Sample.

-Amanda H.

RE: Regarding flash overflow when dfu is enabled

Rakshita — Tue, 03 Sep 2024 10:42:36 GMT

Do we have any sample code for fota over wifi through http/https?

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Thu, 29 Aug 2024 15:33:17 GMT

Hi,

[quote user="Rakshita"]can we do something like that if I enable the BOOTLOADER_MCUBOOT configuration it doesn't creates 2 slots and creates a single slot with more size and when we want to add a new firmware it can get replaced with the older one?[/quote]

The CONFIG_SINGLE_APPLICATION_SLOT is only supported with DFU via USB/UART, not for BLE DFU.

[quote user="Rakshita"]So actually for the net core I wanted to move all the other configurations except the network related to net core including the main application too
Wanted to test it once, how can I make a separate source file for net core and run it along with the app core?[/quote]

The net core is for radio communication and for real-time processing tasks involving low-level radio protocol layers. You cannot move all the other configurations except the network related to net core.

For your use case, I would suggest using an external flash as the secondary slot for BLE DFU.

-Amanda H.

RE: Regarding flash overflow when dfu is enabled

Rakshita — Thu, 29 Aug 2024 14:41:09 GMT

So actually for the net core I wanted to move all the other configurations except the network related to net core including the main application too
Wanted to test it once, how can I make a separate source file for net core and run it along with the app core?

RE: Regarding flash overflow when dfu is enabled

Rakshita — Thu, 29 Aug 2024 08:03:43 GMT

Hey
I was curious that can we do something like that if I enable the BOOTLOADER_MCUBOOT configuration it doesn't creates 2 slots and creates a single slot with more size and when we want to add a new firmware it can get replaced with the older one?

Actually I saw one more configuration CONFIG_SINGLE_APPLICATION_SLOT which does this but I think it doesn't supports dfu through ble so won't be helping but if I could make a change in the existing procedure which will still support dfu through ble that could be very helpful

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Wed, 28 Aug 2024 19:11:01 GMT

Hi,

It's possible to run the full Bluetooth LE stack on the network core (currently supported for development only), but not for the network. See this doc.

-Amanda H.

RE: Regarding flash overflow when dfu is enabled

Rakshita — Wed, 28 Aug 2024 04:57:57 GMT

I think my question is a little incorrect. I want to add network related activities in the net core and main application functionalities in the app core. Is that possible?

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Tue, 27 Aug 2024 21:27:59 GMT

Unfortunately, no.

RE: Regarding flash overflow when dfu is enabled

Rakshita — Tue, 27 Aug 2024 19:36:53 GMT

Is it possible that I can add a child image for net core and can add the wifi and ble configurations in the net core so that it can consume the network storage for these configurations and the flash would be reduced in the app core?

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Tue, 27 Aug 2024 19:17:32 GMT

Yes, I think so. Currently, there is no example of using an SD card as the secondary slot directly. Maybe you can consider DFU over serial (UART or USB) to use single-slot updating. By default, MCUboot splits the non-volatile memory into two slots. If you use Serial Recovery, we can use only one slot instead to get more available non-volatile memory.

RE: Regarding flash overflow when dfu is enabled

Rakshita — Tue, 27 Aug 2024 17:04:16 GMT

But I think I will still face the flash overflow issues as by enabling the below configurations

CONFIG_BOOTLOADER_MCUBOOT=y

CONFIG_NCS_SAMPLE_MCUMGR_BT_OTA_DFU=y

I face the issue, so even if I manage to operate the sd card as an external flash I still need to enable the above configurations and it will still generate overflow issue right?

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Tue, 27 Aug 2024 16:56:50 GMT

Check out this post Bootloader from SDCard .

RE: Regarding flash overflow when dfu is enabled

Rakshita — Tue, 27 Aug 2024 05:29:29 GMT

I do have sd card in my device so can I use that as an external flash?

RE: Regarding flash overflow when dfu is enabled

Amanda Hsieh — Mon, 26 Aug 2024 18:37:45 GMT

Refer to the Memory footprint optimization doc. to reduce the size.

RE: Regarding flash overflow when dfu is enabled

Rakshita — Mon, 26 Aug 2024 18:22:29 GMT

There is no external flash present in the given module