btattach command hanging when using HCI UART sample nrf52840

[samanthacho]

When flashing the chip with a build from the samples/bluetooth/hci_uart via west build -b nrf52840dk/nrf52840 samples/bluetooth/hci_uart --pristine and west flash off the current main of sdk-zephyr, a btattach command from a linux host connected to the Zephyr device over UART hangs indefinitely:

btattach -B /dev/ttyMSM` -S 1000000 -P h4

Attaching Primary controller to /dev/ttyMSM2
Switched line discipline from 0 to 15
= New Index: 00:00:00:00:00:00 (Primary,UART,hci1) #0 [hci1] 4311.006931
= Open Index: 00:00:00:00:00:00 [hci1] 4311.007519
< HCI Command: Reset (0x03|0x0003) plen 0 #1 [hci1] 4311.007671
Device index 1 attached

The bluez version is 5.54, zephyr-sdk is the latest.

Looking at the backtrace on the device via gdb shows the CPU as idle, and no attempts to access the UART are seen.

When attempting to disable the config value CONFIG_BT_CTLR as suggested here, the build fails with the below error. What must be done to enable HCI UART support on the Zephyr device?

-- Zephyr version: 4.0.0-rc2 (/Users/samantha.cho/clones/zephyrproject/zephyr), build: v4.0.0-rc2-53-gb9fc4cc4151f
[139/144] Linking C executable zephyr/zephyr_pre0.elf
FAILED: zephyr/zephyr_pre0.elf zephyr/zephyr_pre0.map /Users/samantha.cho/clones/zephyrproject/build/zephyr/zephyr_pre0.map
: && ccache /Users/samantha.cho/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc  -gdwarf-4 -gdwarf-4 zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj -o zephyr/zephyr_pre0.elf  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -T  zephyr/linker_zephyr_pre0.cmd  -Wl,-Map=/Users/samantha.cho/clones/zephyrproject/build/zephyr/zephyr_pre0.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/libarch__arm__core.a  zephyr/arch/arch/arm/core/cortex_m/libarch__arm__core__cortex_m.a  zephyr/arch/arch/arm/core/mpu/libarch__arm__core__mpu.a  zephyr/lib/libc/picolibc/liblib__libc__picolibc.a  zephyr/lib/libc/common/liblib__libc__common.a  zephyr/lib/net_buf/liblib__net_buf.a  zephyr/soc/soc/nrf52840/libsoc__nordic.a  zephyr/subsys/bluetooth/common/libsubsys__bluetooth__common.a  zephyr/subsys/bluetooth/host/libsubsys__bluetooth__host.a  zephyr/drivers/clock_control/libdrivers__clock_control.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/pinctrl/libdrivers__pinctrl.a  zephyr/drivers/serial/libdrivers__serial.a  zephyr/drivers/timer/libdrivers__timer.a  modules/hal_nordic/nrfx/libmodules__hal_nordic__nrfx.a  modules/segger/libmodules__segger.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  -L/Users/samantha.cho/clones/zephyrproject/build/zephyr  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m4  -mthumb  -mabi=aapcs  -mfp16-format=ieee  -mtp=soft  -fuse-ld=bfd  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -Wl,-no-pie  -specs=picolibc.specs  -DPICOLIBC_LONG_LONG_PRINTF_SCANF -L"/Users/samantha.cho/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/thumb/v7e-m/nofp" -lc -lgcc && cd /Users/samantha.cho/clones/zephyrproject/build/zephyr && /opt/homebrew/Cellar/cmake/3.30.5/bin/cmake -E true
/Users/samantha.cho/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/subsys/bluetooth/host/libsubsys__bluetooth__host.a(hci_raw.c.obj):/Users/samantha.cho/clones/zephyrproject/zephyr/subsys/bluetooth/host/hci_raw.c:60: undefined reference to `__device_dts_ord_133'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

[samanthacho]

Yes, I verified the pins are connected for the HW flow control.

```

[00:02:03.923,492] <wrn> hci_uart: rx is ready
[00:02:03.923,522] <wrn> hci_uart: read 1 req 1
[00:02:03.923,522] <wrn> hci_uart: read 3 req 3
[00:02:03.923,553] <wrn> hci_uart: read 0 req 0
[00:02:03.923,583] <wrn> hci_uart: putting RX packet in queue.
[00:02:03.923,614] <wrn> hci_uart: spurious interrupt
[00:02:03.923,889] <wrn> hci_uart: buf 0x2000e620 type 1 len 7

```

I am seeing when using the RTT debugger that when the `hciconfig hci0 up` command is initiated from the host, although the DK receives packages over the UART, it never attempts to transmit via `tx_isr`. I also do not see the device moving the tx to enabled.

[samanthacho]

Upon further debugging efforts, what I am seeing is that it seems like the BT controller is not responding the way we are expected.
- On the first issue of the `btattach` command from the host, the DK responds with `0e 04 01 03 0c 00`, which I believe to be the successful completion of a reset of the BT controller.

- the `tx_isr` function is triggered via interrupt, and the transmit buffer currently contains 7 bytes, `04 0e 04 01 03 0c 00`, which is the reset response.

- the buffer is left with 6 bytes (the value of the rest of the reset command) and these are never transmitted

- subsequent requests are propagated to the BT controller, but the interrupt to attempt to send them is never triggered, thus the tx_isr is never called and no data is attempted to be transmitted over uart.

Should the whole message be sent in tx_isr via the `uart_fifo_fill` function? If not, when are the rest of the bytes supposed to be transmitted? Also, when data is added via `h4_send`, why is the interrupt to send the data via tx_isr not being triggered?

Also is this statement in hci_uart/src/main.c correct?
```

if (!(uart_irq_rx_ready(hci_uart_dev) ||

uart_irq_tx_ready(hci_uart_dev))) {

LOG_DBG("spurious interrupt");

}

```

I am wondering if it should be `if (!(uart_irq_rx_ready(hci_uart_dev) || uart_irq_tx_ready(hci_uart_dev))`?