Wi-Fi Fundamentals Lesson 5, Exercise 3 will not compile with debug options - FLASH overflows

[Saxman58]

When I try to build the solution with "Optimize for debugging (-Og)" the linker fails saying `rodata' will not fit in region `FLASH'.  FLASH overflowed by 46056 bytes.  This happens with both my exercise code and your solution code.

When I look at the output of the default build, which works, I see that the Zephyr.elf has used a huge amount of the FLASH and RAM, 621,984 of 704KB of the FLASH.  This exercise is a very simple HTTP interface, and that's all.  Why does it use up so much space?  It doesn't seem like there is very much left for a serious application, or even merging in some other functionality like Wi-Fi provisioning through BLE!  And it would be much more difficult to develop an application without being able to use the debugger.  (Based on this exercise, I am seriously concerned about whether our planned app with HTTP server and BLE Wi-Fi provisioning will fit.)  What is going on here?

The full output from a pristine build with "Optimize for debugging (-Og)" is included below:

 *  Executing task: nRF Connect: Build [pristine]: wififund_less5_exer3_solution/build (active) 

Building wififund_less5_exer3_solution
west build --build-dir /Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build /Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution --pristine --board nrf7002dk_nrf5340_cpuapp_ns --no-sysbuild -- -DNCS_TOOLCHAIN_VERSION=NONE -DBOARD_ROOT=/Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution -DCONFIG_DEBUG_OPTIMIZATIONS=y -DCONFIG_DEBUG_THREAD_INFO=y -DCACHED_CONF_FILE=/Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/prj.conf;/Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/boards/nrf7002dk_nrf5340_cpuapp_ns.conf

-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: /Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution
-- CMake version: 3.21.0
-- Found Python3: /opt/nordic/ncs/toolchains/580e4ef81c/opt/[email protected]/bin/python3.9 (found suitable version "3.9.6", minimum required is "3.8") found components: Interpreter 
-- Cache files will be written to: /Users/Glen/Library/Caches/zephyr
-- Zephyr version: 3.5.99 (/opt/nordic/ncs/v2.6.1/zephyr)
-- Found west (found suitable version "1.2.0", minimum required is "0.14.0")
-- Board: nrf7002dk_nrf5340_cpuapp_ns
-- Found host-tools: zephyr 0.16.5 (/opt/nordic/ncs/toolchains/580e4ef81c/opt/zephyr-sdk)
-- Found toolchain: zephyr 0.16.5 (/opt/nordic/ncs/toolchains/580e4ef81c/opt/zephyr-sdk)
-- Found Dtc: /opt/nordic/ncs/toolchains/580e4ef81c/bin/dtc (found suitable version "1.6.1", minimum required is "1.4.6") 
-- Found BOARD.dts: /opt/nordic/ncs/v2.6.1/nrf/boards/arm/nrf7002dk_nrf5340/nrf7002dk_nrf5340_cpuapp_ns.dts
-- Generated zephyr.dts: /Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build/zephyr/zephyr.dts
-- Generated devicetree_generated.h: /Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build/zephyr/include/generated/devicetree_generated.h
-- Including generated dts.cmake file: /Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build/zephyr/dts.cmake

warning: NRF_SECURITY (defined at /opt/nordic/ncs/v2.6.1/nrf/subsys/nrf_security/Kconfig:33) was
assigned the value 'n' but got the value 'y'. See
http://docs.zephyrproject.org/latest/kconfig.html#CONFIG_NRF_SECURITY and/or look up NRF_SECURITY in
the menuconfig/guiconfig interface. The Application Development Primer, Setting Configuration
Values, and Kconfig - Tips and Best Practices sections of the manual might be helpful too.


warning: Experimental symbol WIFI_CREDENTIALS is enabled.


warning: Experimental symbol TFM_EXPERIMENTAL is enabled.


warning: Experimental symbol NET_CONNECTION_MANAGER is enabled.

Parsing /opt/nordic/ncs/v2.6.1/zephyr/Kconfig
Loaded configuration '/opt/nordic/ncs/v2.6.1/nrf/boards/arm/nrf7002dk_nrf5340/nrf7002dk_nrf5340_cpuapp_ns_defconfig'
Merged configuration '/Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/prj.conf'
Merged configuration '/Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/boards/nrf7002dk_nrf5340_cpuapp_ns.conf'
Merged configuration '/Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build/zephyr/misc/generated/extra_kconfig_options.conf'
Configuration saved to '/Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build/zephyr/.config'
Kconfig header saved to '/Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build/zephyr/include/generated/autoconf.h'
-- Found GnuLd: /opt/nordic/ncs/toolchains/580e4ef81c/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd (found version "2.38") 
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /opt/nordic/ncs/toolchains/580e4ef81c/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc
-- Using ccache: /opt/nordic/ncs/toolchains/580e4ef81c/bin/ccache
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build
-- west build: building application
[1/492] Preparing syscall dependency handling

[3/492] Generating include/generated/version.h
-- Zephyr version: 3.5.99 (/opt/nordic/ncs/v2.6.1/zephyr), build: v3.5.99-ncs1-1-1-gef358de7de84
[9/492] Generating ../../tfm/CMakeCache.txt
CMake Warning at cmake/version.cmake:22 (message):
  Actual TF-M version is not available from Git repository.  Settled to
  v2.0.0
Call Stack (most recent call first):
  CMakeLists.txt:22 (include)


-- Found Git: /opt/nordic/ncs/toolchains/580e4ef81c/bin/git (found version "2.37.3") 
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /opt/nordic/ncs/toolchains/580e4ef81c/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc
-- Found Python3: /opt/nordic/ncs/toolchains/580e4ef81c/opt/[email protected]/bin/python3.9 (found version "3.9.6") found components: Interpreter 
CMake Deprecation Warning at /opt/nordic/ncs/v2.6.1/zephyr/cmake/modules/FindDeprecated.cmake:121 (message):
  'PYTHON_PREFER' variable is deprecated.  Please use Python3_EXECUTABLE
  instead.
Call Stack (most recent call first):
  /opt/nordic/ncs/v2.6.1/zephyr/cmake/modules/python.cmake:16 (find_package)
  /opt/nordic/ncs/v2.6.1/zephyr/cmake/modules/user_cache.cmake:30 (include)
  /opt/nordic/ncs/v2.6.1/zephyr/cmake/modules/extensions.cmake:5 (include)
  /opt/nordic/ncs/v2.6.1/nrf/subsys/nrf_security/tfm/CMakeLists.txt:38 (include)


-- Found Python3: /opt/nordic/ncs/toolchains/580e4ef81c/opt/[email protected]/bin/python3.9 (found suitable version "3.9.6", minimum required is "3.8") found components: Interpreter 
-- Cache files will be written to: /Users/Glen/Library/Caches/zephyr
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CRYPTO_AEAD_MODULE_ENABLED
    CRYPTO_ASYM_SIGN_MODULE_ENABLED
    CRYPTO_CIPHER_MODULE_ENABLED
    CRYPTO_HASH_MODULE_ENABLED
    CRYPTO_KEY_DERIVATION_MODULE_ENABLED
    CRYPTO_KEY_MODULE_ENABLED
    CRYPTO_MAC_MODULE_ENABLED
    CRYPTO_RNG_MODULE_ENABLED
    MBEDTLS_PSA_CRYPTO_USER_CONFIG_FILE
    PYTHON_PREFER


-- Build files have been written to: /Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build/tfm
[221/225] Linking C executable bin/tfm_s.axf
Memory region         Used Size  Region Size  %age Used
           FLASH:      163296 B       320 KB     49.83%
             RAM:       57692 B        96 KB     58.69%
[17/492] Performing install step for 'tfm'
-- Install configuration: "Debug"
----- Installing platform NS -----
[485/492] Linking C executable zephyr/zephyr_pre0.elf
FAILED: zephyr/zephyr_pre0.elf zephyr/zephyr_pre0.map /Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build/zephyr/zephyr_pre0.map 
: && ccache /opt/nordic/ncs/toolchains/580e4ef81c/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc  -gdwarf-4 zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj -o zephyr/zephyr_pre0.elf  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  tfm/api_ns/interface/lib/s_veneers.o  -fuse-ld=bfd  -T  zephyr/linker_zephyr_pre0.cmd  -Wl,-Map=/Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build/zephyr/zephyr_pre0.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/libarch__arm__core.a  zephyr/arch/arch/arm/core/cortex_m/libarch__arm__core__cortex_m.a  zephyr/arch/arch/arm/core/cortex_m/cmse/libarch__arm__core__cortex_m__cmse.a  zephyr/arch/arch/arm/core/mpu/libarch__arm__core__mpu.a  zephyr/lib/libc/newlib/liblib__libc__newlib.a  zephyr/lib/libc/common/liblib__libc__common.a  zephyr/lib/posix/liblib__posix.a  zephyr/soc/soc/arm/nordic_nrf/libsoc__arm__nordic_nrf.a  zephyr/subsys/random/libsubsys__random.a  zephyr/subsys/net/libsubsys__net.a  zephyr/subsys/net/l2/ethernet/libsubsys__net__l2__ethernet.a  zephyr/subsys/net/l2/wifi/libsubsys__net__l2__wifi.a  zephyr/subsys/net/ip/libsubsys__net__ip.a  zephyr/subsys/net/lib/config/libsubsys__net__lib__config.a  zephyr/subsys/net/lib/dns/libsubsys__net__lib__dns.a  zephyr/subsys/net/lib/http/libsubsys__net__lib__http.a  zephyr/subsys/net/conn_mgr/libsubsys__net__conn_mgr.a  zephyr/drivers/clock_control/libdrivers__clock_control.a  zephyr/drivers/console/libdrivers__console.a  zephyr/drivers/entropy/libdrivers__entropy.a  zephyr/drivers/flash/libdrivers__flash.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/mbox/libdrivers__mbox.a  zephyr/drivers/pinctrl/libdrivers__pinctrl.a  zephyr/drivers/serial/libdrivers__serial.a  zephyr/drivers/timer/libdrivers__timer.a  modules/nrf/lib/dk_buttons_and_leds/lib..__nrf__lib__dk_buttons_and_leds.a  modules/nrf/lib/fatal_error/lib..__nrf__lib__fatal_error.a  modules/nrf/subsys/nrf_security/src/zephyr/libmbedtls_zephyr.a  modules/nrf/subsys/net/lib/wifi_credentials/libwifi_credentials.a  modules/nrf/subsys/net/lib/wifi_mgmt_ext/lib..__nrf__subsys__net__lib__wifi_mgmt_ext.a  modules/nrf/modules/trusted-firmware-m/lib..__nrf__modules__trusted-firmware-m.a  modules/nrf/modules/trusted-firmware-m/libtfm_api_nrf.a  modules/nrf/modules/hostap/lib..__nrf__modules__hostap.a  modules/nrf/drivers/wifi/nrf700x/lib..__nrf__drivers__wifi__nrf700x.a  modules/trusted-firmware-m/libtfm_api.a  modules/hal_nordic/nrfx/libmodules__hal_nordic__nrfx.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  -L"/opt/nordic/ncs/toolchains/580e4ef81c/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/thumb/v8-m.main/nofp"  -L/Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build/zephyr  -lgcc  zephyr/arch/common/libisr_tables.a  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -Wl,-no-pie  -lm  -Wl,-lc  -L"/opt/nordic/ncs/toolchains/580e4ef81c/opt/zephyr-sdk/arm-zephyr-eabi/arm-zephyr-eabi"/lib/thumb/v8-m.main/nofp  -Wl,-lgcc  -lc  modules/nrf/subsys/nrf_security/src/libmbedtls.a  modules/nrf/subsys/nrf_security/src/libmbedx509.a  modules/nrf/subsys/nrf_security/src/libmbedcrypto.a  /opt/nordic/ncs/v2.6.1/nrfxlib/crypto/nrf_oberon/lib/cortex-m33/soft-float/liboberon_mbedtls_3.0.14.a  modules/nrf/subsys/nrf_security/src/libmbedcrypto_base.a  -mcpu=cortex-m33  -mthumb  -mabi=aapcs  -mfp16-format=ieee  /opt/nordic/ncs/v2.6.1/nrfxlib/crypto/nrf_oberon/lib/cortex-m33/soft-float/liboberon_3.0.14.a  -lc && cd /Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build/zephyr && /opt/nordic/ncs/toolchains/580e4ef81c/Cellar/cmake/3.21.0/bin/cmake -E true
/opt/nordic/ncs/toolchains/580e4ef81c/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/zephyr_pre0.elf section `rodata' will not fit in region `FLASH'
/opt/nordic/ncs/toolchains/580e4ef81c/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: region `FLASH' overflowed by 46056 bytes
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: /opt/nordic/ncs/toolchains/580e4ef81c/bin/cmake --build /Users/Glen/Projects/nordic/wifi-fund/lesson5/wififund_less5_exer3_solution/build

[runsiv]

Hi 

The samples on Devacademy might not necessary be optimized for flash and RAM usage. If you are developing your own application I would recommend having a look at the Wi-Fi stack documentation and select the Kconfigs you need depending the application. As you mentioned BLE I would also suggest to have a look at our Wi-Fi: Bluetooth LE coexistence sample. 

The only thing I would note is that the documentation in the link recommends CONFIG_HEAP_MEM_POOL_SIZE=230000. I would rather try to set it lower at the start. 

Regards

Runar

[Saxman58]

Thanks Runar,

I have run the Wi-Fi Bluetooth coexistence example, but I hadn't see the Wi-Fi stack documentation.  That is very useful.

To be honest, I have quite a bit of Wi-Fi functionality in the beginnings of our own app, and the Flash usage is still fairly low at 55%.  (RAM is higher for buffers for performance).  I'd have to spend some time figuring out what is taking up so much memory on the Dev Academy app, but it looks like the command line interface takes a lot more resources than you'd expect.  That was one of the first things I stripped out when creating the base of our own app.

I developed several products with the nRF5 SDK and always had high visibility into where the memory was being used.  With Zephyr and the network stack memory usage can jump quite quickly with different features, and it seems harder to figure out which modules used it.  Segger Embedded Studio, for example, would list how many bytes were used for code and data, by module.  Is there any equivalent to that with the nRF Connect SDK?  I found the zephyr.stat file, but that only list the sizes for very high level sections like "text".  Can you point me towards some generated file, or maybe a west command that would provide the level of detail that the SES provides like below?

Glen

[runsiv]

I will expand this post when I get back in to work tomorrow. In general my experience is that the zephyr shell can be nice however you have to be careful to include what you need. The CLI is great while testing and learning but I would not include it in a production application.

However I would just mention that you can use the memory report feature to view ROM, RAM and Partition usage in VScode. 

If you prefer a more CLI then you can use the commands

west build -t ram_report
west build -t rom_report

https://docs.zephyrproject.org/latest/develop/optimizations/tools.html

Update: Memory view is the way to go in NCS. I checked if there was anything else that I had forgotten to mention but memory view is the way if you don't want to look at the map file itself.  

Its also possible to store the nrf700x patch on external flash if it is needed. We can also try to help you with optimization of the application if needed. 

Regards

Runar

[Saxman58]

Thanks Runar,

I had seen the Memory Report when I first tried nRF Connect a few months ago, but had completely forgotten about it.  It has been hidden under the Connected Devices panel on my VS Code setup, but no longer.  That is very useful!

Thank you also for the west commands to generate the reports.  That will also be very useful, for example to diff two different builds and see exactly what changed, or to do a text search.  The reports are very detailed, right down to the function level!

Storing the nRF7002 patch on external flash is a good idea.  I found the notes on that: https://docs.nordicsemi.com/bundle/ncs-latest/page/nrf/app_dev/device_guides/nrf70/nrf70_fw_patch_update.html, and this DevZone post: Flash nRF7002 ROM patch via external SPI flash
Our external flash will be with a SPI bus.  We need to use the QSPI for the nRF7002 for the Wi-Fi throughput we need.  That patch note won't work on the nRF7002-DK board because it has the same arrangement we have (QSPI to 7002, SPI to external Flash), so it can't use XIP to access the patch.  We'd have to do some extra work to program the flash and load the patch into the RAM heap during boot.  But that would work.

But I'm going to try to avoid that by watching the memory usage carefully to keep the size under control.

It is not obvious where the nRF700x patch shows up in the memory map, but I'm guessing that might be the 117,336 byte chunk marked as (hidden).

[runsiv]

That seems like a reasonable approch to me. I would not be surprised if we see more optimization on the nrf700x drivers in the future, however this is not something I would take into consideration while developing. I will check where the nrf700x patch is in the memory view and give a update

Regards

Runar