I2C gets stuck with -EBUSY if i2c_* function calls timeout on zephyr (NCS) for nRF52840

[Leowt]

To reproduce this problemm Ive added a device to the I2C that does not respond to a read command.

The `i2c_read` call first returns `-EOI` (-5) and for every other i2c funcion calls it will return `-EBUSY` (-16).

Apparently it locks the i2c subsystem, a `sys_reboot` makes the i2c BUS available again.

I wonder if this is a known issue, and if there is a way of recovering the i2c subsystem? `i2c_recover_bus` is useless for this effect.



a similar issues were posted here:

- https://devzone.nordicsemi.com/f/nordic-q-a/70481/what-means-errno--16-for-nrf52832/289628 
- https://devzone.nordicsemi.com/f/nordic-q-a/66928/zephyr-i2c-locked-into-ebusy

[Kenneth]

Hi,

Do you have a logic analyzer trace? Typically if you get a busy you should retry, so I am wondering does it fail when retrying? Is there any difference depending on the delay between calling i2c_read()?

Do you have the exact error codes from nrfx_twim_xfer() as asked for here:
https://devzone.nordicsemi.com/f/nordic-q-a/70481/what-means-errno--16-for-nrf52832/289628#289628

Do you see the same if no device is connected to the i2c bus? 

Kenneth

[Leowt]

This issue gets the bus stuck after the first failure.
I can confirm that if I remove the device from the bus physically (no devices connected to the bus), it still returns me -EBUSY for any i2c_* call.
Im not sure the logic analyser is of any help since I know that what triggers it is the target device not responding (timeout) for this case in particular and it persists after I disconnect the device from the BUS (no other device is connected), and the only way I could see to get the bus back to a usable state is to call a sys_reboot, because it gets stuck at -EBUSY indefinitely.

I verified myself that:
- The first -EOI error comes from here https://github.com/nrfconnect/sdk-zephyr/blob/d5fe6f2e7f5ba28c9637297368ab1d3ecce7183c/drivers/i2c/i2c_nrfx_twim.c#L145
- All the following -EBUSY errors come from here: github.com/.../i2c_nrfx_twim.c

[Kenneth]

Thanks for the clarification, I thought you meant by "every other" that it happened once every two transfers.

A logic analyzer trace could still be useful just to check where the NAK (I assume) exactly occurs during the I2C transfer here. The only explanation I can find is that event_handler() in i2c_nrfx_twim.c never fires, due to twim_irq_handler() did not fire in nrfx_twim.c, but I don't have an explanation to that. Are you able to find out a bit more by debugging or adding a logic analyzer trace:
https://github.com/nrfconnect/sdk-hal_nordic/blob/8f013ea950f41bf69b18bf688bfb0dd80a3fdc44/nrfx/drivers/src/nrfx_twim.c#L650 

[Leowt]

Im sorry, I posted the wrong links, Im using twi not twim. (Ive tried TWIM but same issue happens)

For reference, the following are the correct links:
- The first -EOI error comes from here: https://github.com/nrfconnect/sdk-zephyr/blob/d5fe6f2e7f5ba28c9637297368ab1d3ecce7183c/drivers/i2c/i2c_nrfx_twi.c#L127 
- All the following -EBUSY errors come from here: https://github.com/nrfconnect/sdk-zephyr/blob/d5fe6f2e7f5ba28c9637297368ab1d3ecce7183c/drivers/i2c/i2c_nrfx_twi.c#L106 

Ill try to get a logic analyser up and running soon

[Leowt]

The following is what happens after I:

// write 0x1 to 0x5
i2c_write(0x1, 0x5) ---> returns 0

// read one byte from 0x5
i2c_read(1, 0x5) ---> returns -5 from the line previously mentioned


We can see here that the line doesnt change anymore after the start of the i2c_read



Then all the calls to i2c_* get a -16 response, no change to the i2c line in the analyser as well...

[Kenneth]

If I understand it correctly you have tried both twim and twi. Are you able to try slightly older versions of ncs for test (e.g. 1.4.0, and possible also even 1.3)?

Kenneth

[Leowt]

I can confirm that the behaviour is the same for v1.4.0.

For v1.3.0, it hangs forever on read. I guess this is due to some change/addition of a timeout code?

The following is a short code to test (considering you have a device on the other end that doesn't write back):

 

uint8_t r_buff;
uint8_t w_buff = 0x02;

const struct device *i2c1_dev;
i2c1_dev = device_get_binding("I2C_1");

while (true)
{
	r_buff = 0;
	// Write something to device at 0x05
	ret = i2c_write(i2c1_dev, &w_buff, 1, 0x05);
	if(ret != 0) {
		printk("error write: %d\n", ret);
	}

	// Read byte from device at 0x05, this is where it is problematic
	// if the device doesnt write back
	ret = i2c_read(i2c1_dev, &r_buff, 1, 0x05);
	if(ret != 0) {
		printk("error read: %d\n", ret);
	}
	printk("r_buff: %u\n", r_buff);
	k_msleep(2000);
}

So far I've noticed that the bus gets stuck forever if:

• i2c_read times out
• i2c SDA and SCL are shorted for a short amount of time

My main problem is not being able to recover the bus, If I was able to do that in runtime without resetting I could have that workaround. Is there any way I can do this in the meanwhile?

[Kenneth]

I assume you can try to do some low level calls, like:

NRF_TWI1->ENABLE = TWI_ENABLE_ENABLE_Disabled << TWI_ENABLE_ENABLE_Pos;
NRF_TWI1->ENABLE = TWI_ENABLE_ENABLE_Enabled << TWI_ENABLE_ENABLE_Pos;

Assuming you are using TWI1.

You can even add a power cycle of the TWI1 peripheral by adding the following between the two lines above:

NRF_TWI1->ENABLE = TWI_ENABLE_ENABLE_Disabled << TWI_ENABLE_ENABLE_Pos;
*(volatile uint32_t *)0x40004FFC = 0;
*(volatile uint32_t *)0x40004FFC;
*(volatile uint32_t *)0x40004FFC = 1;
NRF_TWI1->ENABLE = TWI_ENABLE_ENABLE_Enabled << TWI_ENABLE_ENABLE_Pos;

I can't see how this should not reset TWI1 instance and release the pins.

If you are using TWI0 you should use address 0x40003FFC instead.

But I am not sure how the zephyr driver will like that.

Kenneth

[Leowt]

Just tried that after I receiving a -EBUSY error, no difference from the i2c api perspective afterwards...

It will apparently returns -EBUSY because the driver gets stuck with NRFX_ERROR_BUSY from the 

nrfx_twi_xfer call from the line I previously mentioned.


How will Nordic proceed with this issue? Will this be filled in as a bug?

[Kenneth]

I can add it as a bug internally, I am wondering does the suggestion release the pins? If the pins return to normal pull-up state, then the next step would be to make the driver also return to "default" state.

Kenneth

[Leowt]

I can confirm that the following PR fixes this issue: https://github.com/NordicSemiconductor/nrfx/pull/85/files

Mainstream status: github.com/.../75

Unfortunately it is still not available as of v1.5.1

Can you please report that internally so that this fix becomes available asap?

[Kenneth]

The plan is that the requested change will be included in upcoming nrfx release. Therefore it shouldn't be merged into hal_nordic

Kenneth