NRF52833DK Persistent Hard Faults

RE: NRF52833DK Persistent Hard Faults

Sigurd — Mon, 14 Mar 2022 09:08:38 GMT

[quote user="rm_89"]This did the job, 8192 is enough and anything below that crashes. [/quote]

Great!

[quote user="rm_89"]One last question, if main stack size was the culprit, why did the thread analyzer say that main stack was only at 28% usage when the crashes were happening?[/quote]

I assume that there was a sudden peak in the stack usage, this caused the crash, and that the thread analyzer did not have time to register or report this increase before the crash.

RE: NRF52833DK Persistent Hard Faults

rm_89 — Sat, 12 Mar 2022 20:08:54 GMT

[quote userid="15146" url="~/f/nordic-q-a/85493/nrf52833dk-persistent-hard-faults/357573#357573"]CONFIG_MAIN_STACK_SIZE=16384[/quote]

This did the job, 8192 is enough and anything below that crashes. So weird because I tried increasing to 2048 and 4096 initially and both caused the program to crash earlier than with 1024 so I never tried increasing it more.

One last question, if main stack size was the culprit, why did the thread analyzer say that main stack was only at 28% usage when the crashes were happening?

RE: NRF52833DK Persistent Hard Faults

Sigurd — Fri, 11 Mar 2022 09:19:35 GMT

1) the function MLX90641_I2CRead() is called from main thread context? correct?

2) Any difference if you set CONFIG_LOG2_MODE_IMMEDIATE=n ?

3) Have you tried setting stack and heap to some very high numbers? e.g.

# Stacks and heaps
CONFIG_MAIN_STACK_SIZE=16384
CONFIG_HEAP_MEM_POOL_SIZE=16384

RE: NRF52833DK Persistent Hard Faults

rm_89 — Fri, 11 Mar 2022 09:07:37 GMT

Similar crash output, the addresses traceback to random zephyr source files related to timing:

I also just installed NCS v1.9.1 and created a fresh VS Code workspace to recreate the project, pretty much the same errors still.

RE: NRF52833DK Persistent Hard Faults

Sigurd — Fri, 11 Mar 2022 09:01:35 GMT

Hi,

Could you try setting these, and see what information you get then.

CONFIG_DEBUG_OPTIMIZATIONS=y
CONFIG_DEBUG=y
CONFIG_HW_STACK_PROTECTION=y

RE: NRF52833DK Persistent Hard Faults

rm_89 — Fri, 11 Mar 2022 08:12:24 GMT

I randomly added another LOG_INF output outside of the function that crashes, and the crash changed to "Usage Fault" and it seems the function gets slightly further along although the crash happens at a lower memory address as can be seen below.

Instruction addresses traceback to random NRF/Zephyr files.

And now, even weirder, when I comment out everything below line 24 in main.c, the program does not run at all and crashes with another different message seen below. How is this possible if the crash happens in the first function? I don't see how removing code after this point should change anything, since it never even runs. It must be some obscure ridiculous memory issue causing these crashes?

Once again, the faulting instruction addresses in r15 and r14 trace back to random zephyr code (log_output.c and cbprintf_complete.c)

RE: NRF52833DK Persistent Hard Faults

rm_89 — Mon, 07 Mar 2022 19:17:27 GMT

I've now tried dynamically allocating memory, and I get a bus fault.

prj.cnf:

CONFIG_I2C=y

CONFIG_TINYCBOR=y
CONFIG_CBOR_FLOATING_POINT=y

CONFIG_LOG=y
CONFIG_LOG2_MODE_IMMEDIATE=y

CONFIG_HEAP_MEM_POOL_SIZE=4096

main.c:

#include <zephyr.h>
#include <sys/printk.h>

#include <kernel.h>

#include <logging/log.h>
LOG_MODULE_REGISTER(TTPMS);

#include "MLX90641_API.h"
#include "MLX90641_I2C_Driver.h"

#define TA_SHIFT 8 //Default shift for MLX90641 in open air

static float mlx90641To[768];
paramsMLX90641 mlx90641;

void main(void)
{
	LOG_INF("Running sensor_testing on %s", CONFIG_BOARD);

	MLX90641_I2CInit();

	int status;
    //uint16_t eeMLX90641[832];
	uint16_t *eeMLX90641;
	eeMLX90641 = k_malloc(832*sizeof(uint16_t));
	if (eeMLX90641 != NULL) {
		memset(eeMLX90641, 0, 832*sizeof(uint16_t));
		LOG_INF("Memory allocation succeeded");
	} else {
		LOG_INF("Memory allocation failed");
	}
    status = MLX90641_DumpEE(MLX90641_ADDR, eeMLX90641);
    if (status != 0) {
        printk("Failed to load system parameters\n");
    }
	
    status = MLX90641_ExtractParameters(eeMLX90641, &mlx90641);
    if (status != 0) {
        printk("Parameter extraction failed\n");
    }
	k_free(eeMLX90641);

	while(1) {
		for (uint8_t x = 0 ; x < 2 ; x++) { //Read both subpages
			uint16_t mlx90641Frame[834];
			int status = MLX90641_GetFrameData(MLX90641_ADDR, mlx90641Frame);
			if (status < 0) {
				printk("GetFrame Error: %d\n", status);
			}

			float vdd = MLX90641_GetVdd(mlx90641Frame, &mlx90641);
			float Ta = MLX90641_GetTa(mlx90641Frame, &mlx90641);

			float tr = Ta - TA_SHIFT; //Reflected temperature based on the sensor ambient temperature
			float emissivity = 0.95;

			MLX90641_CalculateTo(mlx90641Frame, &mlx90641, emissivity, tr, mlx90641To);
    	}

		for (int x = 0 ; x < 10 ; x++) {
			printk("Pixel %d: %fC\n", x, mlx90641To[x]);
		}

		k_sleep(K_SECONDS(1));
	}
}

Log output:

addr2line on r14/lr: 0x00008047 goes to SEGGER_RTT.c line 948

Interesting to note that the crash happens before the program even gets to LOG_INF on the first line of MLX90641_I2CRead

UPDATE: If I include string.h in main.c, the crash turns into an MPU_FAULT but happens at the same address traced back to SEGGER_RTT.c

RE: NRF52833DK Persistent Hard Faults

rm_89 — Mon, 07 Mar 2022 06:01:02 GMT

As stated in the original post, increasing CONFIG_MAIN_STACK_SIZE causes an immediate bus fault crash and the program does not even start. I just tried reducing the CONFIG_MAIN_STACK_SIZE to 512 (default is 1024) and now the program got further along (cnt: 575) but it crashed at the exact same memory address, 0x20001b60. I am now almost certain this is an issue with Zephyr/NRF52833 memory regions rather than faulty code. Sidenote: I'm now using LOG_INF with LOG2_MODE_IMMEDIATE instead of printk.