Radio lock-up

RE: Radio lock-up

Thomas_E — Fri, 06 Aug 2021 19:23:22 GMT

Thank you Håkon, for your advice. Yes it is no doubt a timing problem in the 1st issue, that we introduced in our modifications of esb.

Tommy

RE: Radio lock-up

Håkon Alseth — Mon, 02 Aug 2021 09:12:21 GMT

Hi Tommy,

[quote user="Thomas_E"]For the 1st issue we have ACK's disabled in esb and only use the no-ack branches. Slave processes each request via a callback as soon as the packet has arrived. The periodic packet with 2-byte payload instead of 1-byte payload from Master to Slave (with no response from Slave) has continued to prove successful in avoiding radio lock-up on the Slave side.[/quote]

By going from 1 b to 2 b payload, you're adding a bit of time to the whole transaction, which indicates that the timing is a bit too tight. I'm glad to hear that you found a workaround, although; it seems that the issue is still present.

[quote user="Thomas_E"]We've resolved the 2nd issue, it appears. We stopped pushing the Master's radio from active receiving "RX" state to "RXDISABLE" when things go awry, and instead just allow Master's radio to finish receiving any errant packets. The radio no longer locks up. Not yet anyway. I also checked the Master's receive buffer size and it is sufficiently large enough to accommodate any size incoming packet.[/quote]

Good to hear! Let me know if any other issues/questions comes up.

Cheers,

Håkon

RE: Radio lock-up

Thomas_E — Tue, 13 Jul 2021 22:33:40 GMT

Hi Håkon,

Thank you for your suggestions.

For the 1st issue we have ACK's disabled in esb and only use the no-ack branches. Slave processes each request via a callback as soon as the packet has arrived. The periodic packet with 2-byte payload instead of 1-byte payload from Master to Slave (with no response from Slave) has continued to prove successful in avoiding radio lock-up on the Slave side.

We've resolved the 2nd issue, it appears. We stopped pushing the Master's radio from active receiving "RX" state to "RXDISABLE" when things go awry, and instead just allow Master's radio to finish receiving any errant packets. The radio no longer locks up. Not yet anyway. I also checked the Master's receive buffer size and it is sufficiently large enough to accommodate any size incoming packet.

Tommy

RE: Radio lock-up

Håkon Alseth — Fri, 09 Jul 2021 11:07:08 GMT

[quote user="Thomas_E"]How long should we wait between powering off and powering on the radio, if we resort to that measure ?[/quote]

One 16M cycle (62.5 ns). You can generate such a wait condition by reading any events register, (void)NRF_TIMER3->EVENTS_COMPARE[0];.

[quote user="Thomas_E"]

1) Master transmits a packet with 1-byte payload to Slave on a periodic basis. Slave does not respond, only receives. Eventually the Slave's radio locks up.

If instead the Master sends packet with 2-byte payload then the Slave's radio does not ever lock up.

So this problem seems to be solved, though we don't know why 2-byte payload solves it. Probably something more subtle in our handling of the radio that we have not uncovered.

[/quote]

This sound like a timing issue, do you allow the slave to process the request?

Do you dynamically upload a ACK-payload when the slave receives a payload? If so, there's a very small window of opportunity where the ACK will actually go out on-air.

I would strongly recommend that you always send a payload to let the slave prepare for ACK payload, then a new dummy-payload to fetch it.

[quote user="Thomas_E"]

2) Master and Slave are continuously sending packets back and forth, with somewhat tight timing constraints. Master always initiates with a small request packet, then Slave sends a larger data packet. Our connection interval is 1.5 milliseconds.

If there is too much interference for too long, the Master's radio will lock up. The only thing unusual that we do is that sometimes the Master is receiving a lengthy data packet which unexpectedly spills over into the subsequent connection interval, and so we push the Master's radio from active receiving state "RX" to "RXDISABLE" to cut off the errant run-away packet. This appears to be legal from the radio state machine diagram.

[/quote]

Does your buffers allow for such spills?

Kind regards,

Håkon

RE: Radio lock-up

Thomas_E — Thu, 08 Jul 2021 22:42:20 GMT

Hi Håkon,

Thank you for asking.

Not entirely resolved.

We actually have 2 contexts where this issue presents itself:

1) Master transmits a packet with 1-byte payload to Slave on a periodic basis. Slave does not respond, only receives. Eventually the Slave's radio locks up.

If instead the Master sends packet with 2-byte payload then the Slave's radio does not ever lock up.

So this problem seems to be solved, though we don't know why 2-byte payload solves it. Probably something more subtle in our handling of the radio that we have not uncovered.

So we are otherwise still investigating the cause of this lock-up. Have not found anything out of the ordinary yet with regard to timers / ppi / pertinent radio registers, but we could certainly be missing something.

How long should we wait between powering off and powering on the radio, if we resort to that measure ?

Tommy

RE: Radio lock-up

Håkon Alseth — Wed, 07 Jul 2021 07:05:51 GMT

Hi Tommy,

I'm glad to hear this.

Did you find the root cause of your issue?

Kind regards,

Håkon

RE: Radio lock-up

Thomas_E — Tue, 06 Jul 2021 15:27:48 GMT

We are using SDK 17.0.2 .

Thank you your answer was very helpful.

Tommy

RE: Radio lock-up

Håkon Alseth — Mon, 05 Jul 2021 11:53:16 GMT

Hi,

As long as you follow the state diagram of the radio, you should not get into such a mode:

https://infocenter.nordicsemi.com/topic/ps_nrf52833/radio.html?cp=4_1_0_5_17_4#concept_rnf_nml_4r

[quote user=""]

ie, it is in NRF_RADIO_STATE_DISABLED state and there is

no obvious way to get it out of that state except for NVIC_SystemReset() call.

[/quote]

You could try catching this scenario, then enter debug state to see if all the radio registers are in order, as well as the PPI channels / timers are enabled.

If power cycling the peripheral itself (NRF_RADIO->POWER =0; wait; NRF_RADIO->POWER=1; init_radio()) does not work, it is a indication that the problem is outside of the scope of the radio itself (ie. timers or ppi channels)

Which version of the SDK are you using?

Kind regards,

Håkon