nRF52840 application won't completely run without power applied to VBUS

We've developed an application based on nRF52840 (via Raytac module MDBT50Q), utilizing the Adafruit Feather Express Bootloader as our basis to simplify the development.  Our application uses an external display driver with touch screen controller to interact with the user.

Our application runs and loads just fine while we have 5V power applied to VBUS (either from a USB connection, or directly connecting 5V).  Upon removing power to VBUS and powering on the rest of the board, we can blink LEDs but our display application won't load.  Our first circuit configuration was this, which matched one of the modes (or so we thought)

We thought maybe the lack of DCDC mode was the reason, so we updated the design to be:

And still no complete load of the application.  Since we can load a basic blinking led as well basic BLE and serial communications applications, I'm thinking it's something in their boot loader / our display driver code but I wanted to confirm the correct hardware configuration and make sure there weren't any configurations that we missed.  Ultimately the USB is used only to program the micro, but there are a handful of serial debug call we're making that won't need to be in the final product.  The micro circuit only needs 3.3V, but I do have 5V on the board for other purposes that I could power VBUS with.

We did happen to get a handful of boards that did not have the VDDH connection to VDD, but that did not seem to effect anything related to this issue. I have seen the other posts on that issue: