nRF Connect SDK v2.8.0 - Experimental option for multiple domains causes flashing issues on Thingy:53, resulting in bt_hci_driver failure

[zpm1066]

Hello,

I have a Thingy:53 sensor hub application (extended the Nordic Thingy53 Sensor Hub blog to include IAQ sensor)  built under NCS v2.7.0 that runs fine.

When rebuilding the same application under NCS v2.8.0 without any code or build changes, the build is successful with no Bluetooth driver issues. However, when flashed to Thingy:53 I get the following errors.

00> rtt:~$ [00:00:00.007,934] <inf> BH1749: BH1749 initialized
00> rtt:~$ *** Booting nRF Connect SDK v2.8.0-a2386bfc8401 ***
00> rtt:~$ *** Using Zephyr OS v3.7.99-0bc3393fb112 ***
00> rtt:~$ [00:00:00.035,430] <inf> main: main
00> rtt:~$ [00:00:00.035,491] <inf> main: Zephyr RTOS: 3.7.99
00> 
00> rtt:~$ [00:00:00.035,491] <inf> main: thingy53
00> rtt:~$ Starting Sensor Hub application
00> rtt:~$ [00:00:01.036,773] <err> bt_hci_driver: Endpoint binding failed with -11
00> rtt:~$ [00:00:01.036,804] <err> bt_hci_core: HCI driver open failed (-11)
00> rtt:~$ Bluetooth init failed (err -11)
00> rtt:~$ rtt:~$ 

In nRF Connect for VSCode, I noticed the following for NCS v.2.8.0 flashing stage.

"WARNING: Specifying runner options for multiple domains is experimental.
If problems are experienced, please specify a single domain using '--domain <domain>'" 

The above warning doesn't exist in previous NCS version.

Flashing build_vsc_ncs280/thingy53_ns to <jlink sn masked>
west flash -d /Volumes/QuarrkD/Current_Dev_IoT/2024_dev/thingy53_projects/thingy53_sensor_hub_v3/build_vsc_ncs280/thingy53_ns --skip-rebuild --dev-id 801017276

WARNING: Specifying runner options for multiple domains is experimental.
If problems are experienced, please specify a single domain using '--domain <domain>'
-- west flash: using runner nrfjprog
-- runners.nrfjprog: reset after flashing requested
-- runners.nrfjprog: Flashing file: /Volumes/QuarrkD/Current_Dev_IoT/2024_dev/thingy53_projects/thingy53_sensor_hub_v3/build_vsc_ncs280/thingy53_ns/merged_CPUNET.hex
[ #################### ] 2.565s | Erase file - Done erasing 
[ #################### ] 0.308s | Program file - Done programming 
[ #################### ] 0.302s | Verify file - Done verifying 
Applying pin reset.
-- runners.nrfjprog: Board with serial number <jlink sn masked> flashed successfully.
-- west flash: using runner nrfjprog
-- runners.nrfjprog: reset after flashing requested
-- runners.nrfjprog: Flashing file: /Volumes/QuarrkD/Current_Dev_IoT/2024_dev/thingy53_projects/thingy53_sensor_hub_v3/build_vsc_ncs280/thingy53_ns/merged.hex
[ #################### ] 11.471s | Erase file - Done erasing 
[ #################### ] 2.648s | Program file - Done programming 
[ #################### ] 2.734s | Verify file - Done verifying 
Applying pin reset.
-- runners.nrfjprog: Board with serial number <jlink sn masked> flashed successfully.
* Terminal will be reused by tasks, press any key to close it.

The above flashing with the experimental multiple domains appears to cause Bluetooth driver issues, possibly as there is a build step for 'empty_net_core".

To verify that the application domain thingy53_sensor_hub_v3 built correctly for the NCS v2.8.0 build, after flashing the application with NCS v2.7.0 build, I flashed only the NCS v2.8.0 thingy53_sensor_hub_v3 domain from a command line session.

[16/18] cd /Volumes/QuarrkD/Current_Dev_IoT/2024_dev/thingy...toolchains/15b490767d/Cellar/cmake/3.21.0/bin/cmake -E true

-- west flash: using runner nrfjprog

-- runners.nrfjprog: reset after flashing requested

Using board <masked>

-- runners.nrfjprog: Flashing file: /Volumes/QuarrkD/Current_Dev_IoT/2024_dev/thingy53_projects/thingy53_sensor_hub_v3/build_vsc_ncs280/thingy53_ns/merged.hex

[ #################### ] 11.087s | Erase file - Done erasing

[ #################### ]  2.667s | Program file - Done programming 

[ #################### ]  2.748s | Verify file - Done verifying

Applying pin reset.

-- runners.nrfjprog: Board with serial number <masked> flashed successfully.

After above, the Thingy:53 application ran fine. There was no bt_hci_driver endpoint binding failure and Bluetooth initialized correctly, as shown below.

00> rtt:~$ [00:00:00.007,873] <inf> BH1749: BH1749 initialized
00> rtt:~$ *** Booting nRF Connect SDK v2.8.0-a2386bfc8401 ***
00> rtt:~$ *** Using Zephyr OS v3.7.99-0bc3393fb112 ***
00> rtt:~$ [00:00:00.035,369] <inf> main: main
00> rtt:~$ [00:00:00.035,430] <inf> main: Zephyr RTOS: 3.7.99
00> 
00> rtt:~$ [00:00:00.035,430] <inf> main: thingy53
00> rtt:~$ Starting Sensor Hub application
00> rtt:~$ [00:00:00.051,422] <inf> bt_hci_core: HW Platform: Nordic Semiconductor (0x0002)
00> rtt:~$ [00:00:00.051,452] <inf> bt_hci_core: HW Variant: nRF53x (0x0003)
00> rtt:~$ [00:00:00.051,483] <inf> bt_hci_core: Firmware: Standard Bluetooth controller (0x00) Version 214.51162 Build 1926957230
00> rtt:~$ [00:00:00.053,710] <inf> bt_hci_core: Identity: C3:A6:C8:C4:25:42 (random)
00> rtt:~$ [00:00:00.053,741] <inf> bt_hci_core: HCI: version 5.4 (0x0d) revision 0x21fb, manufacturer 0x0059
00> rtt:~$ [00:00:00.053,771] <inf> bt_hci_core: LMP: version 5.4 (0x0d) subver 0x21fb
00> rtt:~$ Bluetooth initialized
00> rtt:~$ Advertising successfully started
00> rtt:~$ rtt:~$ 

In the nRF Connect SDK for VSCode IDE, How can I disable the experimental runner options for multiple domain introduced in NCS v2.8.0 so that it is the same as in NCS v2.7.0?

If the above is not possible, What is the recommended fix in this instance for NCS v2.8.0 when building with nRF Connect SDK?  Thank you.

Regards,
Ravi

[Simonr]

Thank you for your thorough feedback Ravi. I will need to ask the developers about this behavior, and what the correct way to add runner options with sysbuild will be, as I can't find this in our documentation as is.

One thing you can try is to add the "domain" for the NET core separately int he command to make sure the NET core image is in fact build with the Bluetooth driver. Sysbuild is still new, and we're working on improving it continuously.

Best regards,

Simon

[zpm1066]

Hi Simon,

Thank you. You’re most welcome. This issue is most likely affecting a good portion of the customers trying to use sysbuild. It’s completely understandable that there are a few “hiccups“ on the way for a robust sysbuild implementation. It certainly is a much better approach than the old “child-parent image” approach.

btw - I did add a note in the ticket you linked so that folks there are aware of this issue’s progress.

How do I specify adding the domain for NET core as part of the build process from within nRF Connect for VSCode or the command line so that BT driver gets included in the build? Thank you.

Regards,
Ravi

[zpm1066]

Hi Simon,

I expect the following sysbuild configuration addition is the solution.

SB_CONFIG_NETCORE_HCI_IPC

The link below states that sysbuild Kconfig options are also available for nRF53-based devices.
https://docs.nordicsemi.com/bundle/ncs-latest/page/nrf/app_dev/config_and_build/sysbuild/sysbuild_images.html

it looks like currently sysbuild is defaulting to using the following sysbuild configuration and generating an empty Net Core Image.

SB_CONFIG_NETCORE_EMPTY

i’ll try out the sysbuild configuration addition later today and update you. 

Regards’
Ravi

[zpm1066]

Hi Simon,

As anticipated from my comments above, I can confirm that the following sysbuild configuration in sysbuild.conf resolved the issue for Thingy53  "Thingy:53 sensor hub" application, for both NCS v2.8.0 and NCS 2.7.0 sysbuild.

SB_CONFIG_NETCORE_HCI_IPC

I don't know why sysbuild defaults to SB_CONFIG_NETCORE_EMPTY for Thingy:53, as these nRF5340 devices obviously support Bluetooth.


Regards,
Ravi