nrf52840 - Scan start delay

I'm using an nrf52840 devkit but I'm assuming this behavior happens on all nrf52 devices. I'm trying to control the scanner using app_timer because I need a higher scan interval than the maximum supported interval (BLE_GAP_SCAN_INTERVAL_MAX). When calling nrf_ble_scan_start I'm observing that the softdevice does not start immediately scanning but waits m_scan.interval before it starts the actual scan. In theory, this would be simple a case of adding that delay into my timer code which seems to be working fine, but it takes a while before the advertising starts back up. What is happening in my code:

I set scan_window to BLE_GAP_SCAN_WINDOW_MAX. I set scan_interval to 1000ms. 

1. I start a timer that fires every 10 seconds.

2. The timer fires, I call sd_ble_gap_scan_stop, I start a second stop-timer which I set to 1000ms+ my requested window (184ms in this case)

3. After 1000ms, the scan starts. (see screenshot)

4. After 184ms the stop-timer fires, after which the scan ends by calling sd_ble_gap_scan_stop, but advertisements seem to cease for about another second;

5. Only after about 1000ms the advertisements start back up. 

Is there a way for me to get the advertisements to start sooner? Decreasing the scan_interval partly resolves the issue (the advertising delay is only 300ms when I set scan_interval to 300ms), but causes the radio to briefly switch off and back on when scanning for a longer time, causing a % packet-loss received from other advertisers. Ideally, if there was a way to immediately start scanning when calling nrf_ble_scan_start, this would resolve my issue (this seems to happen with the first scan, but subsequent scans have the delay).