For a device like the nRF9160 Feather, you need a second USB to UART device for listening to the modem traces. When I go to select the Terminal Serial Port under Advanced Options, it only gives me one option (the external FTDI adapter). As far as I understand, they can't be mapped to the same interface.
Running on Linux. Will try on Mac to double check it's not a platform specific issue.