Zigbee ZBOSS NCP HOST compile for ARMv7 (or other 32-bit arch)

[dpape]

Hello,

I have compiled the Zigbee ZBOSS NCP HOST on an ARMv7 (32-bit) architecture according to the docs (https://developer.nordicsemi.com/nRF_Connect_SDK/doc/zboss/3.11.2.0/zboss_ncp_host.html). I have
compiled both the ZBOSS libraries (according to 'Rebuilding the ZBOSS libraries for host') and the simple_gw
application.

However when I run the application is segfaults almost immediately and the following is written in the log:

0 76fbcbf0 2022-7-28 14:0:15 0/0/000.000 /usr/src/debug/ncp-host/2.2.0-r0/src/common/zb_init_ncp.c:141  [INFO1] FFD build
1 76fbcbf0 2022-7-28 14:0:15 0/0/000.013 /usr/src/debug/ncp-host/2.2.0-r0/src/common/zb_init_ncp.c:145  [INFO1] sizes: g_zb 10420 sched 1356 bpool 6660 addr 1612
2 76fbcbf0 2022-7-28 14:0:15 0/0/000.014 /usr/src/debug/ncp-host/2.2.0-r0/src/common/zb_init_ncp.c:147  [INFO1] zcl 572
3 76fbcbf0 2022-7-28 14:0:15 0/0/000.014 /usr/src/debug/ncp-host/2.2.0-r0/src/common/zb_init_ncp.c:156  [INFO1] nvram 144
4 76fbcbf0 2022-7-28 14:0:15 0/0/000.014 /usr/src/debug/ncp-host/2.2.0-r0/src/common/zb_init_ncp.c:162  [INFO1] err_ind 4
5 76fbcbf0 2022-7-28 14:0:15 0/0/000.014 /usr/src/debug/ncp-host/2.2.0-r0/src/common/zb_init_ncp.c:169  [INFO1] scheduler q size 64
6 76fbcbf0 2022-7-28 14:0:15 0/0/000.014 /usr/src/debug/ncp-host/2.2.0-r0/src/common/zb_init_ncp.c:175  [INFO1] ZB_IOBUF_POOL_SIZE 40
  zbncp_ll_init_complete: (RX-IDLE : TX-IDLE)
           zbncp_ll_poll: @ time 0 [ack size 1159438920]
zbncp_ll_rx_start_receiving: (RX-IDLE)
   zbncp_ll_rx_set_state: (RX-IDLE --> RX-RECEIVING-HDR)

I suspect this as something to do with struct alignment as many structs in the source are packed and in the
ZBOSS library code some pointer arithmetic is done which is probably 64-bit specific. Is there an easy fix? Somehow
there must be as not many gateway's have 64-bit processors.

[Marte Myrvold]

Hi,

I will check this internally with the development team and get back to you.

Best regards,

Marte

[dpape]

Hello Marte, can you give us an update on this issue? It is quite urgent for us.

[dpape]

Hi Marte,

Any update, this is getting critical for us

Kind regards,
Daan

[Marte Myrvold]

Hi Daan,

No updates yet. The development team is aware of the issue, but they have not been able to look into it yet. As I said, an increase in response time is expected due to summer vacation. I will notify you when I have an update.

Thank you for your patience.

Best regards,

Marte

[Marte Myrvold]

Hi Daan,

Thank you for your patience, and sorry for the delay.

Can you try this workaround, then rebuild and retest the application:

1. In ncp_host_v2.2.0/src/ncp/Makefile find line "CFLAGS += -DZBNCP_DEBUG=1" and change it to "CFLAGS += -DZBNCP_DEBUG=0"
2. Rebuild zboss libraries, then rebuild the application

This workaround helped when our developers encountered a similar issue. They are working on a proper fix as well.

Best regards,

Marte

[dpape]

Hello Marte,

We have tested this using compiling on-board and with a cross compile toolchain but the issue persists. You say that the developers are working on a proper solution, can you give a timeline or a test version to help us out?

Kind regards,
Daan

[Marte Myrvold]

Hi Daan,

I cannot share roadmap or timeline information here on DevZone, so for that information you must contact your regional sales manager.

The developers have some questions:

• What board are you using?
• Did you follow the Rebuilding the ZBOSS libraries for host section in NCP Host User guide, available here: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/zboss/3.11.2.0/zboss_ncp_host.html#rebuilding_libs?
• Can you get callstack of the segfault so the developers can check if it is the same issue as they found? You can build debug version of zboss libraries - in ncp_host_v2.2.0/Options change CFLAGS_BUILD_CFG = $(RELEASE_CFLAGS) to CFLAGS_BUILD_CFG = $(DEBUG_CFLAGS), rebuild stack, application and then run their application with gdb to get callstack.

Best regards,

Marte

[dpape]

Hallo Marte,

In the meantime we have found a workaround. We have disabled any debug logs from the stack. At first we still got a segfault. But then we have reset the nrf52840 so that the adapter had a clean state and the application now runs.

• What board are you using: we are using an i.MX6UL Cortex A7 processor board with yocto linux.
• We have the issue in two ways:
  • Using the on-board gcc compiler and following the user guide exactly.
  • I have set-up a cross compile toolchain and changed the ZBOSS build system to a more portable single CMake file with the same results.
• As soon as we can reproduce the error again we will send the callstack. 

Kind regards,
Daan

[Marte Myrvold]

Hi Daan,

Good to hear that you have found a workaround that works for now. I have informed the developers of your workaround and the additional information you gave here.

Best regards,

Marte

[bfrottier]

Hello Marte,

I am facing a similar (the same?) issue as Daan.

The NCP host application segfault when I try to run it on my target device that embed a nRF52833.

I tried you suggested workaround by changing the CFLAGS but same as Daan, the problem persists.

Here is the call stack of the crash:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0xf7aa82ee in strlen () from /lib/libc.so.6
(gdb) bt
#0  0xf7aa82ee in strlen () from /lib/libc.so.6
#1  0xf7a94bc2 in __vfprintf_internal () from /lib/libc.so.6
#2  0x009c18fa in zb_osif_trace_vprintf (f=0x1d28190, format=0x9e1310 ' ' <repeats 19 times>, ">>> timeout %zu%s [ack size %zu]", arglist=...) at zb_osif_file.c:408
#3  0x009c1546 in zb_file_trace_vprintf (format=0x9e1310 ' ' <repeats 19 times>, ">>> timeout %zu%s [ack size %zu]", arglist=...) at zb_file_trace.c:310
#4  0x009dd624 in zbncp_dbg_trace (fmt=0x9e1310 ' ' <repeats 19 times>, ">>> timeout %zu%s [ack size %zu]") at low_level/zbncp_debug.c:145
#5  0x009dbc8a in zbncp_ll_poll (ll=0xa67150 <node+16>, q=0xff8498f8) at low_level/zbncp_ll_impl.c:1109
#6  0x009d8978 in ncp_ll_quant (unused=255 '\377') at high_level/host/zb_ncp_ll_host.c:246
#7  0x009c1f22 in zb_sched_loop_iteration () at zb_scheduler.c:276
#8  0x009c3b00 in zboss_main_loop () at zb_zboss_api_common.c:84
#9  0x009b1098 in main (argc=1, argv=0xff849b14) at simple_gw.c:209

This was run on an azurewave AW-PU551 but cross-compiled with an armv7 toolchain. I tried to compile with an arm64 toolchain but got the following error:

In file included from low_level/zbncp_mem.c:47:
low_level/zbncp_debug.h:73:3: error: static assertion failed: "Static assertion failed"
   73 |   _Static_assert((cond), "Static assertion failed");
      |   ^~~~~~~~~~~~~~
low_level/zbncp_mem.c:49:1: note: in expansion of macro ‘ZBNCP_DBG_STATIC_ASSERT’
   49 | ZBNCP_DBG_STATIC_ASSERT(sizeof(void*) <= sizeof(zbncp_uintptr_t))
      | ^~~~~~~~~~~~~~~~~~~~~~~
../../src/Options:138: recipe for target 'low_level/zbncp_mem.o' failed

If you have a proper workaround it would be appreciated in the meantime I will try what Daan suggested and disable all debug logs.

Regards,

Benjamin

[Marte Myrvold]

Hi Benjamin,

Have you been able to test the workaround suggested by Daan?

Best regards,

Marte

[bfrottier]

Hello Marte,

Marte Myrvold said:

Have you been able to test the workaround suggested by Daan?

Sorry for the delay. Yes I was and it seems to work as expected.

Any news regarding a potential fix of the sample from the dev team? Did the stacktrace I shared help?

Regards,

Benjamin