SPIS Interrupt Latency. Delay due to Zephyr Tasks

Question

Hardware: nRF5340DK 
 Software: SDK version v2.5.1 
 
 Description: 
 We are using the nrfx_spis driver and are debugging interrupt latency as we have tight timings to reload the SPIS DMA buffers. (<12.5us) 
 We are using CONFIG_ZERO_LATENCY_IRQS in an effort to keep the interrupt latency as short as possible. 
 
 Running our SPIS driver code with no other code running on the device, we see a ~4.5us interrupt latency from when the CS line goes HIGH and when our debug toggle IO. 
 Here is an example of this: I have a IO toggle in my own spis_handler as well as the irq_handler in nrfx_spis.c. Here the timing makes sense, <2us to enter the nrfx_spis handler, and another ~2us to get to our user handler.

Here is the relavant code: 
 #define BUFFER_SIZE (768)
#define BUFFER_COUNT 2UL // Double Buffer
static uint16_t dma_buf[BUFFER_COUNT][BUFFER_SIZE];
static uint16_t dummy_buf[BUFFER_SIZE] = {0};
 
 /////***** SPIS Setup *****/////
 nrfx_spis_config_t spis_config = {
 .miso_pin = MISO_PIN, //
 .mosi_pin = MOSI_PIN, //
 .sck_pin = SCK_PIN, //
 .csn_pin = CSN_PIN, //
 .mode = NRF_SPIS_MODE_0, //
 .bit_order = NRF_SPIS_BIT_ORDER_MSB_FIRST, //
 .csn_pullup = NRF_GPIO_PIN_NOPULL, //
 .miso_drive = NRF_GPIO_PIN_S0S1, //
 .def = 0xFF, //
 .orc = 0xFE, //
 .irq_priority = 0, //
 .skip_gpio_cfg = false, //
 .skip_psel_cfg = false, //
 };

err = nrfx_spis_init(&spis_inst, &spis_config, spis_handler, &spis_inst);
 if (err != NRFX_SUCCESS) {
 LOG_ERR("nrfx_spis_init");
 }

IRQ_DIRECT_CONNECT(NRFX_IRQ_NUMBER_GET(NRF_SPIS_INST_GET(SPIS_INST_IDX)), 0,
 NRFX_SPIS_INST_HANDLER_GET(SPIS_INST_IDX),
 IRQ_ZERO_LATENCY);
 
 /////***** Start first transaction *****/////
 status =
 nrfx_spis_buffers_set(&spis_inst, (uint8_t*)dummy_buf, sizeof(dummy_buf),
 (uint8_t*)&dma_buf[0], sizeof(dma_buf[0]));
 if (status != NRFX_SUCCESS) {
 LOG_ERR("nrfx_spis_buffers_set");
 }
 
static void spis_handler(nrfx_spis_evt_t const* p_event, void* p_context) {
 nrfx_err_t status;

if (p_event->evt_type == NRFX_SPIS_XFER_DONE) {
 gpio_pin_set_dt(&debug_pin, 1);
 static uint32_t buffer_inx = 0;

if (buffer_inx == 0) {
 buffer_inx = 1;
 buf0_ready = true;

} else {
 buffer_inx = 0;
 buf1_ready = true;
 }

// Queue next
 status = nrfx_spis_buffers_set(
 &spis_inst, (uint8_t*)dummy_buf, sizeof(dummy_buf),
 (uint8_t*)&dma_buf[buffer_inx], sizeof(dma_buf[buffer_inx]));
 if (status != NRFX_SUCCESS) {
 LOG_ERR("nrfx_spis_buffers_set");
 }
 gpio_pin_set_dt(&debug_pin, 0);
 }
}

Issue: 
 However, we are encountering a ~12.5us increase in latency when a background task is running. 
 Here is the additional relevant code: 
 #define TASK_STACKSIZE 1024
#define TASK_PRIORITY 4
#define TASK_NAME "fpga"

void FPGA_Init(void)
{
 k_tid_t fpga_task_tid =
 k_thread_create(&fpga_task_thread, fpga_task_stack,
 K_THREAD_STACK_SIZEOF(fpga_task_stack), fpga_task, NULL,
 NULL, NULL, TASK_PRIORITY, 0, K_NO_WAIT);
 k_thread_name_set(fpga_task_tid, TASK_NAME);
}

void fpga_task(void* arg1, void* arg2, void* arg3) {
 while (1) {
 gpio_pin_set_dt(&fpga_reset_pin, 1);
 k_msleep(5);
 gpio_pin_set_dt(&fpga_reset_pin, 0);
 k_msleep(5);
 }
} 
 
 Here is the logic capture:

The time-in-interrupt looks the same. There is simply a large shift in when the interrupt is serviced. 
 My hunch is that there is some kernel-call happening within the SPIS library irq_handler that is causing this delay sometimes, and this is related to a task running? 
 
 Any help is appreciated.