OpenThread crashes on 5340

RE: OpenThread crashes on 5340

Marte Myrvold — Thu, 01 Dec 2022 15:11:29 GMT

Hi,

In the CLI sample this is set in the configuration for the child image. If you look at the sample you will see there is a directory called child_image with the file 802154_rpmsg.conf where this is set. You can add the child image directory and 802154_rpmsg.conf with CONFIG_NRF_802154_ENCRYPTION=y in your project as well.

Best regards,

Marte

RE: OpenThread crashes on 5340

bdodge09 — Thu, 01 Dec 2022 14:17:45 GMT

Thanks, It turned out there was one critical difference in the .config

< # CONFIG_NRF_802154_ENCRYPTION is not set
---
> CONFIG_NRF_802154_ENCRYPTION=y

I haven't figured out why it is not set in my build. In the build output for CPUNET I see

NRF_802154_ENCRYPTION_ENABLED=1

but that is not the same thing?

I can force it by adding

-D802154_rpmsg_CONFIG_NRF_802154_ENCRYPTION=y

To my build command line and that resolves that diff in the .config, and my build now works on the 5340dk

I will try and figure out what is in my prj.conf that is turning that off, but at first look there is nothing in there I don't need for my project.

RE: OpenThread crashes on 5340

Marte Myrvold — Tue, 29 Nov 2022 08:36:05 GMT

Hi,

Is it possible for you to upload your project here so I can test on my side? If you do not want to share it publicly I can make the ticket private. In that case let me know.

If you look in your build directory you should have a directory named "802154_rpmsg". This is the build directory for the network core application. One thing you can do is to compare the 802154_rpmsg/zephyr/.config file in both projects, to check whether the same configurations are set. Another thing is if you do a pristine build, look for when the network core application is built, and see if the same files are used in the build. You should be able to find it by looking for the following line in the build output:

=== child image 802154_rpmsg - CPUNET begin ===

Best regards,

Marte

RE: OpenThread crashes on 5340

bdodge09 — Mon, 28 Nov 2022 14:06:44 GMT

When I flash merged_domains.hex to the 5340 is when I see the failure. See log below for the *failing* case. When I flash merged_domains.hex from the CLI sample, and then flash just the AP processor with my application things work OK. There must be some difference in the network app? How do I find that?

-- west flash: using runner nrfjprog
Using board 1050007064
-- runners.nrfjprog: Recovering and erasing flash memory for both the network and application cores.
Recovering device. This operation might take 30s.
Writing image to disable ap protect.
Erasing user code and UICR flash areas.
Recovering device. This operation might take 30s.
Writing image to disable ap protect.
Erasing user code and UICR flash areas.
-- runners.nrfjprog: Flashing file: /Users/bdodge/Mead/threadcpu/build_5340/zephyr/merged_domains.hex
-- runners.nrfjprog: /Users/bdodge/Mead/threadcpu/build_5340/zephyr/merged_domains.hex targets both nRF53 coprocessors; splitting it into: /Users/bdodge/Mead/threadcpu/build_5340/zephyr/GENERATED_CP_NETWORK_merged_domains.hex and /Users/bdodge/Mead/threadcpu/build_5340/zephyr/GENERATED_CP_APPLICATION_merged_domains.hex
Parsing image file.
Verifying programming.
Verified OK.
Parsing image file.
Verifying programming.
Verified OK.
Applying pin reset.
-- runners.nrfjprog: Board with serial number 1050007064 flashed successfully.

RE: OpenThread crashes on 5340

Marte Myrvold — Tue, 22 Nov 2022 13:05:09 GMT

Hi,

Sorry for the delay, I have been out of office.

So if I understand you correctly, the only time it works is if you start with an erased board (or erase it during the first flashing), program CLI sample with west flash command, and then program your application without erasing flash first? The file merged.hex only has the firmware for one core. If you program with west flash command it will program both cores using merged_domain.hex. Can you check whether you have a merged_domain.hex file in your build folder, or if you only have merged.hex? The issue might be that the network core is not programmed correctly.

Best regards,

Marte

RE: OpenThread crashes on 5340

bdodge09 — Thu, 10 Nov 2022 12:55:54 GMT

I have been trying to update to NCS 2.1.2 and cant build I get

o
/usr/local/GNU-Tools-ARM-Embedded/gcc-arm-none-eabi-10.3-2021.10/bin/../lib/gcc/arm-none-eabi/10.3.1/../../../../arm-none-eabi/bin/ld.bfd: modules/openthread/platform/libopenthread_platform.a(logging.c.obj): in function `z_log_msg_runtime_vcreate':
./build_5340/zephyr/include/generated/syscalls/log_msg.h:75: undefined reference to `z_impl_z_log_msg_runtime_vcreate'
collect2: error: ld returned 1 exit status

I have to use full (non minimal) logging to get the build to work. It seems openthread platform ignores CONFIG_LOG_MODE_MINIMAL=y but this used to work

At NCS v2.1.2 I am still seeing the exact same crash on the 5340. There are only minor changes I made to the zephyr SDK related to some device drivers not involved in Nordic radio or board. Everything else is directly from the sdk-nrf repo and your sdk-zephyr fork at v2.1.2

***

I have gotten the cli sample to work on the 5340. It seems something in flash is different for the sample and my build. If I flash the sample using west flash --recover it works, and then flash my application (nrfjprog merged.hex), my application works. If I use west flash --recover in my application it goes back to crashing. If I then flash the sample (nrfjprog merged.hex) it still crashes.

My application is using mcuboot now, but it was crashing before we started using the bootloader.

RE: OpenThread crashes on 5340

Marte Myrvold — Fri, 28 Oct 2022 07:44:13 GMT

Hi,

I tested with one nRF5340 DK and one nRF52840 DK, as it should not matter what the sender is when the issue is that the nRF5340 crashes when receiving thread packets. I also tested with v2.0.1, and I could not get it to crash there either.

Please answer my previous questions.

bdodge09 — Tue, 20 Sep 2022 17:31:04 GMT

Started with an OpenThread sample but have made a lot of changes. Doesn't seem to matter what the config is (I have taken everything out beside openthread with the same results). Sometimes it works for a bit, but mostly it gets in to this crash loop eventually.

Sending spinel frame
    data: 0x80 0x03 0x8d 0x78 0x43 0x98
    data_address: 0x20029af0
Received spinel frame
    data: 0x80 0x06 0x00 0x00
    data_address: 0x20070814
Received response for status_ok_await
    status: OK
Calling nrf_802154_ack_data_clear
    p_addr: 0x00 0x98
    p_addr_address: 0x20029bf4
    extended: false
Sending spinel frame
    data: 0x80 0x03 0xa1 0x78 0x02 0x00 0x00 0x98 0x01
    data_address: 0x20029a78
Received spinel frame
    data: 0x80 0x06 0xa1 0x78 0x00
    data_address: 0x20070a14
Received response for net_generic_bool_response_await
    net response: false
Calling nrf_802154_ack_data_clear
    p_addr: 0xd4 0xb6 0x3e 0x7b 0xe4 0xa0 0xd5 0xca
    p_addr_address: 0x20029bf8
    extended: true
Sending spinel frame
    data: 0x80 0x03 0xa1 0x78 0x08 0x00 0xd4 0xb6 0x3e 0x7b 0xe4 0xa0 0xd5 0xca 0x01
    data_address: 0x20029a78
Received spinel frame
    data: 0x80 0x06 0xa1 0x78 0x00
    data_address: 0x20070c14
Received response for net_generic_bool_response_await
    net response: false
Calling nrf_802154_receive_at_cancel
Sending spinel frame
    data: 0x80 0x03 0xa8 0x78 0x00 0x00 0x00 0x00
    data_address: 0x20029a90
Received spinel frame
    data: 0x80 0x06 0xa8 0x78 0x00
    data_address: 0x20070e14
Received response for net_generic_bool_response_await
    net response: false
Calling nrf_802154_receive_at_cancel
Sending spinel frame
    data: 0x80 0x03 0xa8 0x78 0x01 0x00 0x00 0x00
    data_address: 0x20029a90
Received spinel frame
    data: 0x80 0x06 0xa8 0x78 0x00
    data_address: 0x20071014
Received response for net_generic_bool_response_await
    net response: false
Calling nrf_802154_csl_writer_period_set
Sending spinel frame
    data: 0x80 0x03 0xac 0x78 0x00 0x00
    data_address: 0x20029a90
E: No response within timeout 500
ASSERTION FAIL @ WEST_TOPDIR/zephyr/drivers/ieee802154/ieee802154_nrf5.c:1149
    802.15.4 serialization error
E: r0/a1: 0x00000004 r1/a2: 0x0000047d r2/a3: 0x00000001
E: r3/a4: 0x00011e61 r12/ip: 0xa0000000 r14/lr: 0x0001960f
E: xpsr: 0x41000000
E: Faulting instruction address (r15/pc): 0x00055d58
E: >>> ZEPHYR FATAL ERROR 4: Kernel panic on CPU 0
E: Current thread: 0x

RE: OpenThread crashes on 5340

Marte Myrvold — Tue, 20 Sep 2022 08:55:35 GMT

Hi,

Is the OpenThread sample one of the samples in our SDK or based on one of them, and if so, which one?

Does it happen when receiving specific packets, or just when receiving packets in general?

Please try enabling logs from the serialization module (CONFIG_NRF_802154_SER_LOG=y), since that is where the assert happens.

Best regards,

Marte