ESB link in PRX mode requires re-initialization after minutes with nRF54L15

[galenchurch]

We are experiencing some ESB link failures on the receiver end of our system: nrf54L15.  After minutes to hours of use the receiver will stop getting data (we know the transmitter is still sending).  If we re-initialize esb subsystem everything is back to normal.  Unfortunately we have no good means to detect the link is down or know why. Its still in ESB_MODE_PRX we just are no longer getting any interrupts.  We have some timeouts on the configured `event_handler` but those are too long to be very useful.  Any advice you have to debug would be helpful but I also have two questions:

1. Is there some detailed documentation on how the ESB system/driver works on the nRF54L15? the driver code is difficult to understand on its own: https://github.com/nrfconnect/sdk-nrf/blob/main/subsys/esb/esb.c and the documentation page seems generic and high level (does not explain much of what the driver does): https://docs.nordicsemi.com/bundle/ncs-2.7.99-cs2/page/nrf/protocols/esb/index.html
2. Can you provide some ESB status register we might be able to monitor/read from to get more information beyond the three interrupt events and the mode that the ESB is in?

Thanks,

Galen

[Amanda Hsieh]

Hi, 

I am checking with the team and will update the case when I collect enough information. 

Regards,
Amanda H.

[Amanda Hsieh]

Hi, 

There is no other documentation than: https://docs.nordicsemi.com/bundle/ncs-latest/page/nrf/protocols/esb/index.html and https://docs.nordicsemi.com/bundle/nrf-apis-latest/page/group_esb.html, unfortunately.

A way to see more of what's happening would be to use logging. The ESB subsystem only logs a few error messages, but you can add debug logs to see where the execution gets stuck. 

Regarding the register that can be checked - ESB is not a driver nor a HW subsystem - it is a protocol implementation, so there is no dedicated register for ESB. However, you can check some radio registers to understand the issue better: e.g. State register: https://docs.nordicsemi.com/bundle/ps_nrf54L15/page/radio.html#ariaid-title102 or event registers.

Please provide the guidance or shared snippets of application code to help us reproduce the issue. 

-Amanda H. 

[galenchurch]

HiAmanda Hsieh , thanks for this initial info.  I will add in some additional logging around the radio and let you know when i have some better information/results from that.

[galenchurch]

Hi Amanda Hsieh 

I have added more logging and I can see when the PRX side hangs after what i assume is sending an ack payload via `esb.c::on_radio_disabled_rx_ack()`.  I know that function returns and normally we would get another event on `esb_radio_direct_irq_handler` to handle normal reception, but none occurs.  There is no event triggering that ISR to be called again and therefor `esb.c::radio_irq_handler()` is never called.  What could cause the system to be hung in this state?

Just to reiterate, if we reset the PRX everything returns to normal, i.e. no change to the PTX device is required to return to normal and we know the PTX is still transmitting.

Thank,

Galen

[Amanda Hsieh]

What NCS version are you using?