Problem on nRF9160 making a modified lte_ble_gateway sample

[Christopher Smith]

Our application on a nRF9160 to act as intermediary between an AWS cloud and another nRF devicee over bluetooth.  So I attempted to merge the bluetooth IOT sample with the bluetooth portion lte_ble_gateway sample

When I tried running this I got this crash in the early portion of the bluetooth intialization, before any IOT code is running. 

[00:00:00.251,251] <inf> health_hub: The AWS IoT sample started, version: v1.0.0
[00:00:00.251,281] <inf> hp_ble: Initializing Bluetooth..
ASSERTION FAIL [err == 0] @ WEST_TOPDIR/zephyr/subsys/bluetooth/host/hci_core.c:338
command opcode 0x0c03 timeout with err -11
[00:00:10.251,617] <err> os: r0/a1: 0x00000003 r1/a2: 0x00000000 r2/a3: 0x00000002
[00:00:10.251,647] <err> os: r3/a4: 0x20011498 r12/ip: 0x00000010 r14/lr: 0x00020a35
[00:00:10.251,647] <err> os: xpsr: 0x41000000
[00:00:10.251,678] <err> os: Faulting instruction address (r15/pc): 0x00020a40
[00:00:10.251,708] <err> os: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0
[00:00:10.251,739] <err> os: Current thread: 0x20014010 (unknown)
*** Booting nRF Connect SDK v2.5.1 ***Resetting system

    net_buf_put(&bt_dev.cmd_tx_queue, net_buf_ref(buf));

    err = k_sem_take(&sync_sem, HCI_CMD_TIMEOUT);

    BT_ASSERT_MSG(err == 0, "command opcode 0x%04x timeout with err %d", opcode, err);

Somewhere in the k_sem_take()  we get that error 11 but it isn't clear what it is looking for at that point and stepping in doesn't work well.

Any ideas of what sort of problem can give this error?

Here is the prj.conf in case this is a missing configuration problem or something

#
# Copyright (c) 2020 Nordic Semiconductor ASA
#
# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
#
#CONFIG_DEBUG=y

# General
CONFIG_LOG=y
CONFIG_LOG_BUFFER_SIZE=2048
CONFIG_HW_ID_LIBRARY=y
CONFIG_ASSERT=y
CONFIG_JSON_LIBRARY=y

# Heap and stacks
#CONFIG_HEAP_MEM_POOL_SIZE=8192
#CONFIG_MAIN_STACK_SIZE=4096
#CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048

#CONFIG_BOOTLOADER_MCUBOOT=y

# Network
CONFIG_NETWORKING=y
CONFIG_NET_NATIVE=y
CONFIG_NET_IPV4=y
CONFIG_NET_CONNECTION_MANAGER=y

# AWS IoT library
CONFIG_AWS_IOT=y
CONFIG_AWS_IOT_CLIENT_ID_STATIC="my-thing"
CONFIG_AWS_IOT_BROKER_HOST_NAME="xxx.amazonaws.com"
CONFIG_AWS_IOT_SEC_TAG=201
CONFIG_AWS_IOT_APP_SUBSCRIPTION_LIST_COUNT=2
CONFIG_AWS_IOT_TOPIC_UPDATE_DELTA_SUBSCRIBE=y
CONFIG_AWS_IOT_TOPIC_GET_ACCEPTED_SUBSCRIBE=y
CONFIG_AWS_IOT_TOPIC_GET_REJECTED_SUBSCRIBE=y
CONFIG_AWS_IOT_LAST_WILL=y

# MQTT - Maximum MQTT keepalive timeout specified by AWS IoT Core
CONFIG_MQTT_KEEPALIVE=1200
CONFIG_MQTT_CLEAN_SESSION=y

# Console for user association
CONFIG_CONSOLE_SUBSYS=y
CONFIG_CONSOLE_GETCHAR=y

# Enable Bluetooth stack and libraries
CONFIG_BT=y
CONFIG_BT_H4=y
CONFIG_BT_WAIT_NOP=y
CONFIG_BT_CENTRAL=y
CONFIG_BT_GATT_CLIENT=y
CONFIG_BT_GATT_DM=y
CONFIG_BT_SCAN=y
CONFIG_BT_SCAN_FILTER_ENABLE=y
CONFIG_BT_SCAN_UUID_CNT=1

CONFIG_UART_INTERRUPT_DRIVEN=y

# Heap and stacks
CONFIG_HEAP_MEM_POOL_SIZE=16384
CONFIG_MAIN_STACK_SIZE=8192
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=4096

# Update Data Length and MTU (BT_L2CAP_RX_MTU = CONFIG_BT_BUF_ACL_RX_SIZE - BT_L2CAP_HDR_SIZE)
CONFIG_BT_USER_DATA_LEN_UPDATE=y
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_BT_BUF_ACL_RX_SIZE=251
CONFIG_BT_BUF_ACL_TX_SIZE=251
CONFIG_BT_L2CAP_TX_MTU=247

#================================================
# Little FS
#
# Optionally force the file system to be recreated
# CONFIG_APP_WIPE_STORAGE=y

# Let __ASSERT do its job
CONFIG_DEBUG=n

#CONFIG_FLASH=n
#CONFIG_FLASH_MAP=n
#CONFIG_FLASH_PAGE_LAYOUT=n

CONFIG_FILE_SYSTEM=y
CONFIG_FILE_SYSTEM_LITTLEFS=y

# CONFIG_PM_PARTITION_REGION_LITTLEFS_EXTERNAL=y

CONFIG_NVS=y
CONFIG_SETTINGS=y

[Christopher Smith]

in .config in the lte_ble_gateway program there is the section

#
# Serial Drivers
#
CONFIG_UART_NRFX=y
CONFIG_UART_NRFX_UARTE=y
CONFIG_UART_ASYNC_TX_CACHE_SIZE=8
CONFIG_UART_0_ENHANCED_POLL_OUT=y
CONFIG_UART_0_INTERRUPT_DRIVEN=y
# CONFIG_UART_0_NRF_PARITY_BIT is not set
CONFIG_UART_0_NRF_TX_BUFFER_SIZE=32
# CONFIG_UART_0_NRF_HW_ASYNC is not set
# CONFIG_UART_0_NRF_ASYNC_LOW_POWER is not set
# CONFIG_UART_2_INTERRUPT_DRIVEN is not set
CONFIG_UART_2_ASYNC=y
CONFIG_UART_2_ENHANCED_POLL_OUT=y
# CONFIG_UART_2_NRF_PARITY_BIT is not set
CONFIG_UART_2_NRF_TX_BUFFER_SIZE=32
CONFIG_UART_2_NRF_HW_ASYNC=y
# CONFIG_UART_2_NRF_ASYNC_LOW_POWER is not set
CONFIG_UART_2_NRF_HW_ASYNC_TIMER=2
CONFIG_UARTE_NRF_HW_ASYNC=y
CONFIG_UART_ENHANCED_POLL_OUT=y
# CONFIG_SYSCON is not set

-----------------------------------------------------------------------------

in the merged in aws_iot derived sample

#
# Serial Drivers
#
CONFIG_UART_NRFX=y
CONFIG_UART_NRFX_UARTE=y
CONFIG_UART_ASYNC_TX_CACHE_SIZE=8
CONFIG_UART_0_ENHANCED_POLL_OUT=y
CONFIG_UART_0_INTERRUPT_DRIVEN=y
# CONFIG_UART_0_NRF_PARITY_BIT is not set
CONFIG_UART_0_NRF_TX_BUFFER_SIZE=32
# CONFIG_UART_0_NRF_HW_ASYNC is not set
# CONFIG_UART_0_NRF_ASYNC_LOW_POWER is not set
# CONFIG_UART_1_INTERRUPT_DRIVEN is not set
CONFIG_UART_1_ASYNC=y
CONFIG_UART_1_ENHANCED_POLL_OUT=y
# CONFIG_UART_1_NRF_PARITY_BIT is not set
CONFIG_UART_1_NRF_TX_BUFFER_SIZE=32
# CONFIG_UART_1_NRF_HW_ASYNC is not set
# CONFIG_UART_1_NRF_ASYNC_LOW_POWER is not set
CONFIG_UART_2_INTERRUPT_DRIVEN=y
CONFIG_UART_2_ENHANCED_POLL_OUT=y
# CONFIG_UART_2_NRF_PARITY_BIT is not set
CONFIG_UART_2_NRF_TX_BUFFER_SIZE=32
# CONFIG_UART_2_NRF_ASYNC_LOW_POWER is not set
CONFIG_UART_ENHANCED_POLL_OUT=y
# CONFIG_SYSCON is not set

Why does the UART1 not get used in one sample?  Could this be an issue somehow?  what prof setting triggers this?

for completeness the basic aws_iot sample has:

#
# Serial Drivers
#
CONFIG_UART_NRFX=y
CONFIG_UART_NRFX_UARTE=y
CONFIG_UART_ASYNC_TX_CACHE_SIZE=8
CONFIG_UART_0_ENHANCED_POLL_OUT=y
CONFIG_UART_0_ASYNC=y
# CONFIG_UART_0_NRF_PARITY_BIT is not set
CONFIG_UART_0_NRF_TX_BUFFER_SIZE=32
# CONFIG_UART_0_NRF_HW_ASYNC is not set
# CONFIG_UART_0_NRF_ASYNC_LOW_POWER is not set
CONFIG_UART_1_ASYNC=y
CONFIG_UART_1_ENHANCED_POLL_OUT=y
# CONFIG_UART_1_NRF_PARITY_BIT is not set
# CONFIG_UART_1_NRF_HW_ASYNC is not set
# CONFIG_UART_1_NRF_ASYNC_LOW_POWER is not set
CONFIG_UART_ENHANCED_POLL_OUT=y
# CONFIG_SYSCON is not set

no information about UART2.

[Christopher Smith]

device tree differences

in the lte_ble_gateway program

chosen {
zephyr,entropy = &psa_rng;
zephyr,flash-controller = &flash_controller;
zephyr,console = &uart0;
zephyr,shell-uart = &uart0;
zephyr,uart-mcumgr = &uart0;
zephyr,flash = &flash0;
zephyr,sram = &sram0_ns;
zephyr,code-partition = &slot0_ns_partition;
zephyr,bt-uart = &lpuart;
};

uart1: arduino_serial: uart@9000 {
compatible = "nordic,nrf-uarte";
reg = < 0x9000 0x1000 >;
interrupts = < 0x9 0x1 >;
status = "disabled";
current-speed = < 0x1c200 >;
pinctrl-0 = < &uart1_default >;
pinctrl-1 = < &uart1_sleep >;
pinctrl-names = "default", "sleep";
};
uart2: uart@a000 {
compatible = "nordic,nrf-uarte";
reg = < 0xa000 0x1000 >;
interrupts = < 0xa 0x1 >;
status = "okay";
pinctrl-0 = < &uart2_default_alt >;
pinctrl-1 = < &uart2_sleep_alt >;
pinctrl-names = "default", "sleep";
current-speed = < 0xf4240 >;
lpuart: nrf-sw-lpuart {
compatible = "nordic,nrf-sw-lpuart";
status = "okay";
req-pin = < 0x15 >;
rdy-pin = < 0x13 >;
};
};

nrf52840_reset: gpio-reset {
compatible = "nordic,nrf9160dk-nrf52840-reset";
status = "okay";
gpios = < &interface_to_nrf52840 0x9 0x1 >;
};

-----------------------------------------------------------------

in the merged iot program

chosen {
zephyr,entropy = &psa_rng;
zephyr,flash-controller = &flash_controller;
zephyr,console = &uart0;
zephyr,shell-uart = &uart0;
zephyr,uart-mcumgr = &uart0;
zephyr,flash = &flash0;
zephyr,sram = &sram0_ns;
zephyr,code-partition = &slot0_ns_partition;
zephyr,bt-uart = &uart2;
nordic,modem-trace-uart = &uart1;
};

uart1: arduino_serial: uart@9000 {
compatible = "nordic,nrf-uarte";
reg = < 0x9000 0x1000 >;
interrupts = < 0x9 0x1 >;
status = "okay";
current-speed = < 0xf4240 >;
pinctrl-0 = < &uart1_default >;
pinctrl-1 = < &uart1_sleep >;
pinctrl-names = "default", "sleep";
hw-flow-control;
};
uart2: uart@a000 {
compatible = "nordic,nrf-uarte";
reg = < 0xa000 0x1000 >;
interrupts = < 0xa 0x1 >;
status = "okay";
pinctrl-0 = < &uart2_default_alt >;
pinctrl-1 = < &uart2_sleep_alt >;
pinctrl-names = "default", "sleep";
current-speed = < 0xf4240 >;
lpuart: nrf-sw-lpuart {
compatible = "nordic,nrf-sw-lpuart";
status = "okay";
req-pin = < 0x15 >;
rdy-pin = < 0x13 >;
};
};

nrf52840_reset: gpio-reset {
compatible = "nordic,nrf9160dk-nrf52840-reset";
status = "disabled";
gpios = < &interface_to_nrf52840 0x9 0x1 >;
};

[Kenneth]

Can you try to copy the content in the .overlay file for lte_ble_gateway to merged iot program? Ref:

Kenneth

[Christopher Smith]

I could not see how to configure the enable/disable state for the nrf52840_reset.  Building the other set, though, can't find the name __device_dts_ord_97.  It seems to be a uart name but I am not sure which one.

b/CMakeFiles/..__nrf__lib__nrf_modem_lib.dir/trace_backends/uart/uart.c.obj -MF modules\nrf\lib\nrf_modem_lib\CMakeFiles\..__nrf__lib__nrf_modem_lib.dir\trace_backends\uart\uart.c.obj.d -o modules/nrf/lib/nrf_modem_lib/CMakeFiles/..__nrf__lib__nrf_modem_lib.dir/trace_backends/uart/uart.c.obj -c C:/ncs/v2.5.1/nrf/lib/nrf_modem_lib/trace_backends/uart/uart.c
In file included from C:/ncs/v2.5.1/zephyr/include/zephyr/toolchain/gcc.h:92,
from C:/ncs/v2.5.1/zephyr/include/zephyr/toolchain.h:50,
from C:/ncs/v2.5.1/zephyr/include/zephyr/sys/time_units.h:10,
from C:/ncs/v2.5.1/zephyr/include/zephyr/sys/util.h:615,
from C:/ncs/v2.5.1/zephyr/include/zephyr/devicetree.h:25,
from C:/ncs/v2.5.1/zephyr/include/zephyr/device.h:12,
from C:/ncs/v2.5.1/zephyr/include/zephyr/drivers/uart.h:26,
from C:/ncs/v2.5.1/nrf/lib/nrf_modem_lib/trace_backends/uart/uart.c:7:
C:/ncs/v2.5.1/zephyr/include/zephyr/device.h:85:41: error: '__device_dts_ord_97' undeclared here (not in a function); did you mean '__device_dts_ord_17'?
85 | #define DEVICE_NAME_GET(dev_id) _CONCAT(__device_, dev_id)
| ^~~~~~~~~
C:/ncs/v2.5.1/zephyr/include/zephyr/toolchain/common.h:132:26: note: in definition of macro '_DO_CONCAT'
132 | #define _DO_CONCAT(x, y) x ## y
| ^
C:/ncs/v2.5.1/zephyr/include/zephyr/device.h:85:33: note: in expansion of macro '_CONCAT'
85 | #define DEVICE_NAME_GET(dev_id) _CONCAT(__device_, dev_id)
| ^~~~~~~
C:/ncs/v2.5.1/zephyr/include/zephyr/device.h:211:37: note: in expansion of macro 'DEVICE_NAME_GET'
211 | #define DEVICE_DT_NAME_GET(node_id) DEVICE_NAME_GET(Z_DEVICE_DT_DEV_ID(node_id))
| ^~~~~~~~~~~~~~~
C:/ncs/v2.5.1/zephyr/include/zephyr/device.h:228:34: note: in expansion of macro 'DEVICE_DT_NAME_GET'
228 | #define DEVICE_DT_GET(node_id) (&DEVICE_DT_NAME_GET(node_id))
| ^~~~~~~~~~~~~~~~~~
C:/ncs/v2.5.1/nrf/lib/nrf_modem_lib/trace_backends/uart/uart.c:18:46: note: in expansion of macro 'DEVICE_DT_GET'
18 | static const struct device *const uart_dev = DEVICE_DT_GET(UART_DEVICE_NODE);
| ^~~~~~~~~~~~~
[9/52] Building C object modules/nrf/subsys/net/lib/download_client/CMakeFiles/..__nrf__subsys__net__lib__download_client.dir/src/sanity.c.obj

[Christopher Smith]

In trying to clean up these issues I keep getting undefined device tree like __device_dts_ord_97.  Is there a good way to tie these to the original symbol?