nrf_wifi fails : memory allocation failure at boot and crash at config

[BrianW]

Using an nrf5340 with nrf7002 wifi, connected via SPI4. Using a shared (chip) antenna, and the COEX signals (pretty much exactly as per the dev board schema).

DTS snippets:

    // Wifi nrf7002 : SPI4 using SCK P0.08, MOSI P0.09, MISO P0.10, CS P0.11

    // BUCKEN P0.12, IRQ P0.23, COEX_GRANT P0.24, COEX_REQ P0.28, STATUS0 P0.30, STATUS1 P0.29, NRF7002_IOVDD_EN ioexp0/GPB0

    nrf_radio_coex: nrf7001-coex {

        status = "okay";

        compatible = "nordic,nrf700x-coex";

        req-gpios = <&gpio0 28 GPIO_ACTIVE_HIGH>;

        status0-gpios = <&gpio0 30 GPIO_ACTIVE_HIGH>;

        grant-gpios = <&gpio0 24 (GPIO_PULL_DOWN | GPIO_ACTIVE_LOW)>;

        swctrl1-gpios = <&gpio0 29 GPIO_ACTIVE_HIGH>;

        // btrf-switch-gpios = <&gpio1 10 GPIO_ACTIVE_HIGH>; No, controlled by nrf7002 directly???

    };

    nordic_wlan0: nordic_wlan0 {

        compatible = "nordic,wlan0";

        status = "okay";

    };

&spi4 {                     // Wifi nrf7002

    status = "okay";

    compatible = "nordic,nrf-spim";

    pinctrl-0 = <&spi4_default>;

    pinctrl-1 = <&spi4_sleep>;

    cs-gpios = <&gpio0 11 GPIO_ACTIVE_LOW>; /* P0.11 */

    pinctrl-names = "default", "sleep";

    nrf700x: nrf7002@0 {

        status = "okay";

        compatible = "nordic,nrf700x-spi";

        reg = <0>;

        spi-max-frequency = <33000000>;

//      sck-frequency = <24000000>;

        /* Wi-Fi Pins used */

        iovdd-ctrl-gpios = <&ioexp0 8 GPIO_ACTIVE_HIGH>;            // GPB00

        bucken-gpios = <&gpio0 12 GPIO_ACTIVE_HIGH>;

        host-irq-gpios = <&gpio0 23 GPIO_ACTIVE_HIGH>;

    };

};

I have configured prj.conf to have the wifi STA 'iot' operation, as per the docs:

# WIFI config
CONFIG_WIFI=y
CONFIG_WIFI_NRF700X=y

# WPA supplicant
CONFIG_WPA_SUPP=y
CONFIG_NRF700X_AP_MODE=n
CONFIG_NRF700X_P2P_MODE=n

# Networking layers
CONFIG_NETWORKING=y
CONFIG_NET_SOCKETS=y
CONFIG_NET_LOG=y
CONFIG_NET_IPV4=y
CONFIG_NET_UDP=y
CONFIG_NET_TCP=y
CONFIG_NET_DHCPV4=y

CONFIG_NET_IF_UNICAST_IPV4_ADDR_COUNT=2
CONFIG_NET_MAX_CONTEXTS=1
CONFIG_NET_CONTEXT_SYNC_RECV=y

CONFIG_NET_L2_ETHERNET=y

CONFIG_NET_CONFIG_SETTINGS=y
CONFIG_NET_CONFIG_INIT_TIMEOUT=0

CONFIG_NET_SOCKETS_POLL_MAX=1

# Memory usage
CONFIG_NET_TX_STACK_SIZE=4096
CONFIG_NET_RX_STACK_SIZE=4096

# Optimized networking settings for memory usage (STA MODE IOT DEVICES profile)
# https://docs.nordicsemi.com/bundle/ncs-latest/page/nrf/protocols/wifi/stack_configuration.html
CONFIG_NRF700X_MAX_TX_TOKENS=5
CONFIG_NRF700X_MAX_TX_AGGREGATION=1
CONFIG_NRF700X_RX_NUM_BUFS=6

CONFIG_NET_TC_TX_COUNT=1
CONFIG_NET_PKT_RX_COUNT=6
CONFIG_NET_PKT_TX_COUNT=6

CONFIG_NET_BUF_RX_COUNT=6
CONFIG_NET_BUF_TX_COUNT=12
# can take BUF_DATAT_SIZE down to 500
CONFIG_NET_BUF_DATA_SIZE=800

However, at boot time I see these logs:

[00:00:00.015,686] <inf> wifi_nrf_bus: SPIM spi@a000: freq = 33 MHz
[00:00:00.022,369] <inf> wifi_nrf_bus: SPIM spi@a000: latency = 1
[00:00:01.074,981] <err> wifi_nrf: hal_fw_patch_load: Failed to allocate memory for patch UMAC-bin: chunk 1/6, size: 8192
[00:00:01.086,639] <err> wifi_nrf: nrf_wifi_fmac_fw_load: UMAC patch load failed

[00:00:01.094,818] <err> wifi_nrf: nrf_wifi_fw_load: nrf_wifi_fmac_fw_load failed
[00:00:01.103,027] <err> wifi_nrf: nrf_wifi_fmac_dev_add_zep: nrf_wifi_fw_load failed
[00:00:01.113,891] <err> wifi_nrf: nrf_wifi_if_start_zep: nrf_wifi_fmac_dev_add_zep failed

Note I also have configured malloc to use the system heap, and for it to occupy all remaining RAM:

CONFIG_COMMON_LIBC_MALLOC=y

CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=-1

The heap size at boot time is 233576 bytes so no reason for a malloc to fail?

Then when I attempt to initialise the wifi I get this crash

[00:00:13.322,784] <err> os: ***** USAGE FAULT *****
[00:00:13.328,399] <err> os: Unaligned memory access
[00:00:13.334,197] <err> os: r0/a1: 0xa07eeaff r1/a2: 0x00000068 r2/a3: 0x0000005c
[00:00:13.342,834] <err> os: r3/a4: 0x20047130 r12/ip: 0x00000000 r14/lr: 0x00063dc9
[00:00:13.351,440] <err> os: xpsr: 0x61000000
[00:00:13.356,628] <err> os: Faulting instruction address (r15/pc): 0x0009bafe
[00:00:13.364,501] <err> os: >>> ZEPHYR FATAL ERROR 31: Unknown error on CPU 0
[00:00:13.372,375] <err> os: Current thread: 0x20007340 (sysworkq)
[00:00:13.379,180] <err> os: Halting system

in  nrf_wifi_osal_mem_zalloc() according to zephyr.map.

This is called somewherr from nrf_wifi_coex_config_non_pta():

    /* Configure Coexistence Hardware */

    log_info("netwifi: Configuring non-PTA registers. seperate antennas:%s  is_sr_protocol_ble: %s",separate_antennas?"YES":"NO", is_sr_protocol_ble?"YES":"NO");

    ret = nrf_wifi_coex_config_non_pta(separate_antennas, is_sr_protocol_ble);

    if (ret != 0) {

        log_warn("netwifi: Configuring non-PTA registers of CoexHardware FAIL");

        return false;

    }

Note the wifi sample projects (wifi_radio_test, wifi_shell)all run just FINE (with no error logs at boot) so the hardware is apparently good, but I haven't managed to get from them to my full app without this problem occuring...

Any ideas?

[Charlie]

Hi Brian，

Sorry for the late reply. I have been travling last week.

I did a try with the following configurations added to original NCS v2.7.0 https://docs.nordicsemi.com/bundle/ncs-2.7.0/page/nrf/samples/wifi/sta/README.html sample on nRF7002DK.

CONFIG_COMMON_LIBC_MALLOC=y

CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=-1

CONFIG_RESET_ON_FATAL_ERROR=n

Here is the log:

*** Booting nRF Connect SDK v2.7.0-6a541e852d54 ***
*** Using Zephyr OS v3.6.99-100befc70c74 ***
[00:00:00.609,130] <inf> net_config: Initializing network
[00:00:00.609,161] <inf> net_config: Waiting interface 1 (0x20000ed8) to be up...
[00:00:00.609,313] <inf> net_config: IPv4 address: 192.168.1.99
[00:00:00.609,344] <inf> net_config: Running dhcpv4 client...
[00:00:00.610,198] <inf> sta: Starting nrf7002dk with CPU frequency: 64 MHz
[00:00:01.610,382] <inf> sta: QSPI Encryption disabled
[00:00:01.610,443] <inf> sta: Static IP address (overridable): 192.168.1.99/255.255.255.0 -> 192.168.1.1
[00:00:01.610,443] <inf> sta: Waiting for Wi-Fi to be ready
[00:00:03.150,756] <inf> wifi_mgmt_ext: Connection requested
[00:00:03.150,787] <inf> sta: Connection requested
[00:00:03.150,848] <inf> sta: ==================
[00:00:03.150,878] <inf> sta: State: SCANNING
[00:00:03.451,019] <inf> sta: ==================
[00:00:03.451,049] <inf> sta: State: SCANNING
[00:00:03.751,190] <inf> sta: ==================
[00:00:03.751,220] <inf> sta: State: SCANNING
[00:00:04.051,361] <inf> sta: ==================
[00:00:04.051,361] <inf> sta: State: SCANNING
[00:00:04.351,531] <inf> sta: ==================
[00:00:04.351,562] <inf> sta: State: SCANNING
[00:00:04.651,702] <inf> sta: ==================
[00:00:04.651,733] <inf> sta: State: SCANNING
[00:00:04.951,873] <inf> sta: ==================
[00:00:04.951,904] <inf> sta: State: SCANNING
[00:00:05.252,044] <inf> sta: ==================
[00:00:05.252,075] <inf> sta: State: SCANNING
[00:00:05.552,215] <inf> sta: ==================
[00:00:05.552,215] <inf> sta: State: SCANNING
[00:00:05.852,386] <inf> sta: ==================
[00:00:05.852,416] <inf> sta: State: SCANNING
[00:00:06.152,557] <inf> sta: ==================
[00:00:06.152,587] <inf> sta: State: SCANNING
[00:00:06.452,728] <inf> sta: ==================
[00:00:06.452,758] <inf> sta: State: SCANNING
[00:00:06.752,899] <inf> sta: ==================
[00:00:06.752,899] <inf> sta: State: SCANNING
[00:00:07.053,070] <inf> sta: ==================
[00:00:07.053,100] <inf> sta: State: AUTHENTICATING
[00:00:07.236,053] <inf> sta: Connected
[00:00:07.251,525] <inf> net_dhcpv4: Received: 192.168.50.23
[00:00:07.251,678] <inf> net_config: IPv4 address: 192.168.50.23
[00:00:07.251,678] <inf> net_config: Lease time: 86400 seconds
[00:00:07.251,739] <inf> net_config: Subnet: 255.255.255.0
[00:00:07.251,770] <inf> net_config: Router: 192.168.50.1
[00:00:07.251,892] <inf> sta: DHCP IP address: 192.168.50.23
[00:00:07.363,464] <inf> sta: ==================
[00:00:07.363,494] <inf> sta: State: COMPLETED
[00:00:07.363,525] <inf> sta: Interface Mode: STATION
[00:00:07.363,555] <inf> sta: Link Mode: WIFI 6 (802.11ax/HE)
[00:00:07.363,555] <inf> sta: SSID: WiFi6
[00:00:07.363,586] <inf> sta: BSSID: C8:7F:54:91:94:50
[00:00:07.363,616] <inf> sta: Band: 2.4GHz
[00:00:07.363,616] <inf> sta: Channel: 10
[00:00:07.363,647] <inf> sta: Security: WPA2-PSK
[00:00:07.363,677] <inf> sta: MFP: Optional
[00:00:07.363,677] <inf> sta: RSSI: -26

I see you are using different NCS version, could you have a try with NCS v2.7.0?

Best regards,

Charlie

[BrianW]

Charlie said:

I see you are using different NCS version, could you have a try with NCS v2.7.0?

Well, I'm quite wary about doing an update to 2.7.0, as I understand there are some quite major changes to the build system (use of sysbuild), also maybe changes to the hardware DTS models?

I don't want to get even further behind in the project...

Can you try the change with NCS 2.6.2? 

[Charlie]

Hi Brain,

Sorry for late reply. We can reproduce this issue on NCS 2.6.2 but it is not easy to debug the root cause of it, this is due to the mix usage of different libcs with wifi libraries in zephyr and NCS own.

Currently we are working intenstively on NCS 2.8.0 release which is epxected coming in the end of October. The NCS 2.8.0 has a unified memory management method which will help to all heap increase or overflow issues related to Wi-Fi.

sdk-zephyr/drivers/wifi/nrfwifi/Kconfig.nrfwifi at main · nrfconnect/sdk-zephyr · GitHub

I strongly encourage you to have a try with this new solution for your current product. Migration guide for nRF Connect SDK v2.7.0 may help you about build system and hardware DTS update, feel free to generate new ticket If you need any help on this topic.

Best regards,

Charlie

[BrianW]

thats very disappointing as an answer. The stability of NCS is apparently still not 100% I see.... I guess this will the response for other wifi library related issues as well....

[Charlie]

Hi Brian,

I understand this is not an answer that you will satisfy, but I have to tell you to the right direction to avoide further struggling, and no company can provide 100% stability guarantee for software and hardware.

We are keeping improving our solution with both bug fixes and new features added based on developer feedback. NCS bascily have new realse every three month to include these changes.

One more thing I want to reminder is that you should keep "sck-frequency = <24000000>", and this is the max SPI speed nRF7002 support now.

Best regards,

Charlie

[BrianW]

Charlie said:

One more thing I want to reminder is that you should keep "sck-frequency = <24000000>", and this is the max SPI speed nRF7002 support now.

If I define sck-frequency, I get an error indicating this is not an attribute listed for nordic,nrf700x-spi.

devicetree error: 'sck-frequency' appears in /soc/peripheral@50000000/spi@a000/nrf7002@0 in C:/work/dev/if-device-nrf53/cc1-med/build/zephyr/zephyr.dts.pre, but is not declared in 'properties:' in C:/work/dev/nordic_connect/nrf/dts/bindings\wifi\nordic,nrf700x-spi.yaml

spi-max-frequency is ok...

[Charlie]

Sorry, 'sck-frequency' is only a property defined for nRF700x qspi in nrf\dts\bindings\wifi\nordic,nrf700x-qspi.yaml. "spi-max-frequency" is the property for nRF700x spi interface inherit from general spi-device.yaml.

Both of them define the max communication clock for nRF7002 and should be below 24000000.