nrf52832 crash since uart/log disabled

[opasutto]

Hi, we are facing a very strange problem on one of our product since the deployment of it's last software release:

This product integrate an nrf52832 and unfortunately an old version of the softdevice S132 (v2.0 from sdk 11.0).

The problem seems to occur since we have removed some compilation options regarding logging/UART.

In the previous version the:

1. UART0 was enabled in nrf_drv_config.h and is not in the last version
2. In the makefile, the files  uart/app_uart_fifo.c and  uart/nrf_drv_uart.c where included in the C_SOURCE_FILES to be compiled and not in the last version
3. In the makefile, the C symbol NRF_LOG_USES_RAW_UART=1 was defined and not in the last version

In production software, the log module is not really used, only the NRF_LOG_INIT function followed by a startup message.

Now what are the symptoms: on some boards (a small percentage) it seems that suddenly, the nrf chip becomes unresponsive (the product is disconnected from nRF connect app and SPI communication with the master MCU is not possible). But the behaviour is not always the same. On some board, it only needs to connect to BLE via nRF connect to freeze the chip, but on some other boards the freeze has more a random behaviour, it can happens from time to time... And for the majority there isn't any problem...

I tried to attach and debug via a j-link probe and vscode cortex-debug plugin after a freeze and it seems that there is an HardFault_handler, but I am not able to detect what caused this fault.

From the call stack I can see the address 0x1ae42 which is located in the softdevice, but I don't have more details:

So, I reckon that it's a difficult question I don't really know if it's an hardware or software problem, but any help would be greatly appreciated to investigate this problem!

Thank you in advance

[Einar Thorsrud]

Hi,

How many devices do you see this issue on? Have you been able to see a pattern for when it is triggered on the devices where you see it? If so, perhaps it is possible to find a way to reproduce the issue more easily so that it can be debugged.

Which exact S132 2.0.x version are you using?

[opasutto]

Hi, we have currently identified about 30 devices with this issue out of several thousands. We are using the S132 2.0.0 (the one furnished with the SDK v11) and unfortunately, we aren't able to update the softdevice part in our product...

On some problematic devices, it's easy to reproduce the issue. For example, one one device, the only operation needed is to connect to it with the nRF connect mobile app, and instantanely, the connection is closed by the device:

Advertising is stopped, SPI is not responding and the only solution is to restart the device.

One another device, after the power on I can observe that the advertising is started but only for 1 or 2 seconds, it is stopped without reasons.

So far for all the problematic devices, the solution to correct the problem is to add at the start of the program a call to the function log_raw_uart_init:

uint32_t log_raw_uart_init()
{
    // Disable UART
    NRF_UART0->ENABLE = UART_ENABLE_ENABLE_Disabled;

    // Configure RX/TX pins
    nrf_gpio_cfg_output( TX_PIN_NUMBER );
    nrf_gpio_cfg_input(RX_PIN_NUMBER, NRF_GPIO_PIN_NOPULL);

    // Set a default baud rate of UART0_CONFIG_BAUDRATE
    NRF_UART0->PSELTXD  = TX_PIN_NUMBER;
    NRF_UART0->BAUDRATE = UART0_CONFIG_BAUDRATE;

    NRF_UART0->PSELRTS  = 0xFFFFFFFF;
    NRF_UART0->PSELCTS  = 0xFFFFFFFF;

    // Disable parity and interrupt
    NRF_UART0->CONFIG   = (UART_CONFIG_PARITY_Excluded  << UART_CONFIG_PARITY_Pos );
    NRF_UART0->CONFIG  |= (UART_CONFIG_HWFC_Disabled    << UART_CONFIG_HWFC_Pos   );

    // Re-enable the UART
    NRF_UART0->ENABLE           = UART_ENABLE_ENABLE_Enabled;
    NRF_UART0->INTENSET         = 0;
    NRF_UART0->TASKS_STARTTX    = 1;
    NRF_UART0->TASKS_STARTRX    = 1;

    return NRF_SUCCESS;
}

And later, if I comment the line NRF_UART0->TASKS_STARTTX    = 1; in this function,the problem reappears...

So it seems that there is a workaround to our problem, but we would like to understand what is going on, because maybe there are other problems not yet detected.

Thank you again

[Einar Thorsrud]

Hi,

Are the failing and non-failing devices running the exact same firmware? Meaning the issue only occurs in devices where you had disable logging in the firmware, but also only in a small portion of those devices?

It is difficult to say what goes wrong in your firmware, but logging changes timing of the application, and the Raw UART even more so, as it is blocking. So if you have parts of your application that is sensitive to changes in timing, those issues could surface now. I am unfortunately not able to be more specific, but that is my best hypothesis.

[opasutto]

Hi Einar, thank you for your answer.

Yes all the devices are running the exact firmware, the majority don't have any problem, but for those that have the problem, it is in the majority of the cases an easy to reproduce problem.

At first I was also thinking that it could be a timing related problem. But now I think that it may be an hardware problem since I have removed all the logging in my test firmware and found that for the problematic devices, I only had to initialize UART0 to have the problem, and to comment it to solve it. For the non problematic devices, this modification doesn't have any consequence.

I think it is more related to the UART controller having an influence on some other controllers as the initialisation of the RX and TX pins doesn't change anything, it is really the following lines that makes all the difference:

NRF_UART0->TASKS_STARTTX = 1;
NRF_UART0->TASKS_STARTRX = 1;

Anyway, I think that I will stop investigating for the moment, I just hope that there is not some more hidden problems... Thank you again