Using SDK 2.5.1 at time of writing.
This is less a "request for help" and more of a "here's how I managed to do this and I'm documenting it here in case others want to do the same thing".
The existing available configuration settings and APIs aren't really set up to allow you to easily use an external oscillator as an LFCLK source when using an nRF5340 processor that gets kicked off at runtime, mostly because the zephyr kernel needs some kind of LFCLK source for use in its kernel at startup and that it also assumes it will not need to be switched at runtime. If you need an external oscillator for your design (such as using an external calendar chip with a backup battery whose time you want to stay synchronized to the processor while running), here's how to do it anyway. Note that you'll have to make adjustments as needed if your project setup is different from mine. You'll of course need to make sure that the low frequency external oscillator is connected to the XL1 pin as usual, and that XL2 is left floating.
You'll have to tell the SDK that you want the Bluetooth image loaded into the network core with the Bluetooth hci interface enabled. Easiest way to do that is make sure you have CONFIG_BT=y set in your project configuration file, along with all the other configuration settings you'd normally set up. I recommend starting with a sample application and modifying it if you're doing this for the first time.
In your build configuration settings, you'll need to lie to the network core and tell it that the "XTAL" interface will be available from startup (we'll fix the obvious consequence of this lie later). In your build configuration settings add -Dhci_rpmsg_CONFIG_CLOCK_CONTROL_NRF_K32SRC_XTAL="y" and -Dhci_rpmsg_CONFIG_CLOCK_CONTROL_NRF_K32SRC_20PPM="y" to the Extra CMake arguments section (or the appropriate PPM level as needed). If alternatively if you've gone the path of linking the network core blob as a child image of your project, you can put these settings in the child image configuration file instead.
Now we'll need to set up the clock configuration for the main application image. In your project configuration settings, set CONFIG_SOC_ENABLE_LFXO=y, CONFIG_SOC_LFXO_CAP_EXTERNAL=n, CONFIG_CLOCK_CONTROL_NRF_K32SRC_SYNTH=y, and CONFIG_CLOCK_CONTROL_NRF_K32SRC_20PPM=y. Essentially we're telling the system to set up the GPIO for external clock use with no capacitors, but at startup use the synthesized clock from the high frequency clock source for our low frequency clock source. We can't tell it to use the RC clock, because then the kernel drivers would schedule a periodic calibration for a clock source that isn't running, which is the same reason we told the network core to use something other than RC clock. Your chip will randomly crash later if you try to use RC clock instead. We also couldn't tell main application to use the external clock source on startup because it's not running yet and the kernel will hang before reaching the main function. Therefore synthesized clock is the only workable option for startup configuration, but we don't care about its inaccuracy or non-compatibility with Bluetooth because we'll be switching away from it almost immediately.
Edited: If you have a bootloader, you must set it to use the SYNTH clock at startup like the application. If you require network core calls from bootloader (such as over the air update), you'll also need to perform a similar runtime correction to what we're about to do for our application code below.
We're ready to modify our init routines to handle our unusual clock setup. The first thing we'll want to do at startup is to disable the network core using the network force off bit in the reset register (see datasheet). It is unable to start up right now anyway due to expecting an external clock that isn't running yet. Next we'll want to get our external oscillator started, this is just whatever the normal process is for your hardware design whether it's enabled via a GPIO pin or via a set of serial transactions. Once you get it running, you'll then want to tell the chip that you're using an external oscillator instead of an external crystal via the oscillator bypass register in the oscillators peripheral (again see datasheet), just write the bit to set it to rail to rail external source. When that's done, you'll finally perform the switchover to use the external oscillator. Set up the transition by setting the LFCLKSRC register in the clock controller registers to "xtal", which will really be external oscillator due to the bypass register adjustment we just made. Use a barrier to make sure the processor does not perform the next write out of order before the previous register settings. Finally, set the start trigger bit in the LFCLK start register to tell the peripheral to perform the transition over to the new clock source. Per the datasheet you may lose up to 1 LFCLK clock cycle during this transition, but that shouldn't be a problem for an initialization routine like this.
Due to the way the RTC clock source configuration is set up inside the nRF53, the last operation has switched the input of ALL RTC peripherals over to the external source, including the ones used by the kernel and the network core. So the last thing we have to do is remember that we had shut off the network core earlier and now we need to restart it. Clear the network force off bit now to get it reset and running again. Now you have a fully up and running system that will use the external oscillator as its low frequency clock source and be compatible with the Bluetooth core running, so you can perform the rest of your normal init routine and run as usual from here.
Edited: Final important note is that if you choose to reset the chip using NVIC_SystemReset, you'll want to shut off the network core, then switch back to SYNTH clock and wait for ready before calling the reset function.
