Multiple notifications per connection interval

I saw this issue brought up 3 years ago, but I think the conditions are changed now since that was on NCS 1.

I am trying to do bt_hids_inp_rep_send about 10 times per connection interval, 16 byte report, but it seems that the fifo I have for sending these through is only able to get through 2 per 50ms:

So it's like the second one is blocking. Why is that? Isn't it supposed to queue?

Here's another log with prints right before and after bt_hids_inp_rep_send that continue to show the 50ms block on every second call:

Also note, I have 

CONFIG_BT_BUF_ACL_TX_COUNT=32
which sets CONFIG_BT_CONN_TX_MAX. Even if I didn't set it, the default is 3, so I wouldn't see a block on every second call. And I don't think it can have to do with ACKs, since hids sets input report characteristics as without response.
Furthermore, I am not making these calls on a workqueue. They are made in a thread with priority 0.
Edit: I now see that the first 40 or so sends go through fine without blocking (my tx buffer is 32), but then it blocks. So for whatever reason, if say the connection interval would be 50ms, since that's how long it blocks, it is not able to get through 5 sends of 16 bytes each (which is effectively what I'm doing since my report is 16 bytes and data is added to the fifo at 100hz)?
Edit 2: I created two samples to reproduce the issue between two nrf5340 DKs: https://github.com/Abrahamh08/hids_many_notifs
Put the central on one and peripheral on the other. Every time you flash peripheral again, I think central nvm has to be erased because of whatever bonding it does. Reset/turn on the peripheral when the central is flashed and running. The central will prompt in RTT to press button 1. When you press it and the auth completes, the prints in peripheral will start having error code 0 at the end instead of 13. Let it run a bit then disconnect, and capture the logs. You will see every 3rd "report sent" will be 25ms after the "sending", indicating 25ms of blocking.
Edit 3: This is a sniffer trace of the peripheral from the sample linked: hidsmanynotifs.pcapng
Related