Corrupt I2C scanning when adding load to the system

[Daniel Maartensson]

How to reproduce the issue:

We have used the standard hello_world sample application in the zephyr repository. CONFIG_I2C, CONFIG_SHELL and CONFIG_I2C_SHELL configurations are added to the sample application. We have also changed the i2c2 clock-frequency to use I2C_BITRATE_STANDARD in an overlay since the device that we intend to connect only supports this bitrate. Using the “i2c scan i2c@a000” from the shell we can see that the GPIO expander is correctly detected at address 0x21. However, if we add load to the application (k_sleep(K_MSEC(1)); in a while 1 loop) doing a scan will often result in more devices showing up at random addresses.

HW setup: nRF9151-DK (0.9.0, 2024.30). The board configuration utility has been used to enable the IO Expander (I2C) on the board.

SW setup:  nRF Connect SDK v2.7.0, Modem FW: mfw_nrf91x1_2.0.2

Toolchain: nRF Connect SDK Toolchain v2.7.0

• Working (non-loaded system, only the GPIO expander is detected):

uart:~$ > i2c scan i2c@a000

i2c scan i2c@a000

     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f

00:             -- -- -- -- -- -- -- -- -- -- -- --

10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

20: -- 21 -- -- -- -- -- -- -- -- -- -- -- -- -- --

30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

70: -- -- -- -- -- -- -- --                        

1 devices found on i2c@a000

• Non-working (loaded system, other devices than the GPIO expander detected):

uart:~$ > i2c scan i2c@a000

i2c scan i2c@a000

     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f

00:             -- -- -- -- -- -- 0a -- -- -- -- --

10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

20: -- 21 -- -- -- -- -- -- -- -- -- -- -- -- -- --

30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

50: -- -- -- -- -- -- -- -- -- 59 -- -- -- -- -- --

60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

70: -- -- -- -- -- -- 76 --                        

4 devices found on i2c@a000

Looking at the I2C clock and data lines (corresponding to the incorrect scan above) we see that there is a dip on the lines before address 0x0a is scanned and this gets detected as a device present as shown in the snippet below.

The next snippet shows the same scan where the GPIO expander is detected (correctly) at address 0x21 later in the sequence.

Changes to the hello_world sample application in the zephyr repository:

diff --git a/samples/hello_world/i2c.conf b/samples/hello_world/i2c.conf
new file mode 100644
index 00000000000..5c9f7c9b0e3
--- /dev/null
+++ b/samples/hello_world/i2c.conf
@@ -0,0 +1,4 @@
+CONFIG_I2C=y
+
+CONFIG_SHELL=y
+CONFIG_I2C_SHELL=y
diff --git a/samples/hello_world/i2c.overlay b/samples/hello_world/i2c.overlay
new file mode 100644
index 00000000000..5ee0212715c
--- /dev/null
+++ b/samples/hello_world/i2c.overlay
@@ -0,0 +1,3 @@
+&i2c2 {
+             clock-frequency = <I2C_BITRATE_STANDARD>;
+};
diff --git a/samples/hello_world/src/main.c b/samples/hello_world/src/main.c
index c550ab461cb..33d2176bbaa 100644
--- a/samples/hello_world/src/main.c
+++ b/samples/hello_world/src/main.c
@@ -4,11 +4,19 @@
  * SPDX-License-Identifier: Apache-2.0
  */

+#include <zephyr/kernel.h>
#include <stdio.h>

 int main(void)
{
               printf("Hello World! %s\n", CONFIG_BOARD_TARGET);

+#if 0
+             while (1)
+             {
+                             k_sleep(K_MSEC(1));
+             }
+#endif
+
               return 0;
}

Please let us know if this an issue with the I2C or if we are using the system incorrectly.

Best regards,

Daniel.

[selddbk]

It seems like the issue you're facing is related to I2C bus instability when the system is under load, which could be affecting the reliability of the I2C scan. The extra devices showing up on the scan could be a result of I2C noise, clock stretching, or timing issues caused by the added load in the system.

Make sure that appropriate pull-up resistors are placed on the I2C lines (SCL and SDA). A weak or absent pull-up can cause noise on the bus, which may lead to false detections, especially under load. Ensure you're using the correct values for the pull-ups (typically 4.7kΩ to 10kΩ). Block Blast

[Daniel Maartensson]

Hi,

Thank you for your reply. I agree with your analysis.

The above measurements are taken on a stock nRF9151-DK (0.9.0, 2024.30). Nothing has been added to the board. According to the HW user guide there should be 4.7 kΩ pull-up resistors on the SCL and SDA lines.

From the nRF9151 DK Hardware (v.0.9.0):

    Best regards,

    Daniel.

[Edvin]

Hello,

Just wanted to chip in. I see the same, but also without the sleep in your main loop, so I don't think it is related to that. I do however, suspect that it is related to the board controller that is present on the DK (using I2C). I also see that it triggers from an extra pulse, like the one you circled in your logic trace. I need to do more investigation, but I will get back to you.

Best regards,

Edvin

[Edvin]

Hello,

I was a bit mistaken in my previous answer. It only happened when I added the sleeping loop in main(), as you said. 

However, I have not been able to reproduce this outside the shell application. Have you?

I tried using this sample, basically the hello world sample with added I2C functionality. 

hello_world_i2c.zip

Note that it uses I2C1, and different pins. You can change this, but I wanted to rule out that it was something else using I2C on the DK .

If you are able to reproduce it outside the shell sample, please let me know! But I have tried for a good while, without success.

Best regards,

Edvin

[Daniel Maartensson]

Hi Edvin,

Thank you for helping us to investigate the issue. We started to look into this after experiencing random loss of data to another I2C device that we connected to the nRF9151 DK. The issue seemed to happen more often as we added functionality so we assumed it was dependent on load or timing.

It is interesting that you couldn't reproduce it outside of the shell. We believe that we see this issue outside of the shell but we do have the CONFIG_SHELL option set in our application. I will try to see if we can narrow it down further on our side.

I think that the fact that a simple sleep in the hello_world application causes an invalid start sequence on the I2C bus when using the shell is an indication that there is a timing issue in the I2C framework or that there is an issue on the nRF9151 DK setup.

    Best regards,

    Daniel.