This is because of window widening (from BLE core spec v4.2, Vol 6, Part B, chapter 4.5.7). If it is a longer time since last connection interval, the RX has to be on for a longer time in order to be sure that the packet is received because of the clock drifting.
You can see my answer here, devzone.nordicsemi.com/.../, under 3. Window widening to see how to calculate this and how this relates to power consumption.
