Unaligned memory access Fault

[daThinkingOne]

Hi, I am using LittleFS with some external flash.  This works great on it's own, but when part of our growing program it has started crashing with the "Unaligned memory access" fault.  It was fine before our project grew in size.  This may be a memory issue somewhere.  I am not sure how to locate the problem.

Here is the fault message:

[00:00:00.603,668] <err> os: ***** USAGE FAULT *****
[00:00:00.603,668] <err> os:   Unaligned memory access
[00:00:00.603,698] <err> os: r0/a1:  0x00000000  r1/a2:  0x00000001  r2/a3:  0x00000001
[00:00:00.603,698] <err> os: r3/a4:  0x200009ce r12/ip:  0x00000000 r14/lr:  0x00016cad
[00:00:00.603,729] <err> os:  xpsr:  0x61000000
[00:00:00.603,729] <err> os: s[ 0]:  0x200008bd  s[ 1]:  0x00000001  s[ 2]:  0x2000597c  s[ 3]:  0x2000a538
[00:00:00.603,790] <err> os: s[ 4]:  0x00000004  s[ 5]:  0x0002f048  s[ 6]:  0x00000000  s[ 7]:  0x0001667f
[00:00:00.603,790] <err> os: s[ 8]:  0x20001bf8  s[ 9]:  0x200008bd  s[10]:  0x2000a5c0  s[11]:  0x2000a5c0
[00:00:00.603,820] <err> os: s[12]:  0x000257e1  s[13]:  0x00000000  s[14]:  0x00000000  s[15]:  0x0001223d
[00:00:00.603,820] <err> os: fpscr:  0x20005858
[00:00:00.603,851] <err> os: Faulting instruction address (r15/pc): 0x0002e07a
[00:00:00.603,881] <err> os: >>> ZEPHYR FATAL ERROR 31: Unknown error on CPU 0
[00:00:00.603,912] <err> os: Current thread: 0x20005688 (unknown)
[00:00:01.046,905] <err> fatal_error: Resetting system
*** Booting nRF Connect SDK v3.5.99-ncs1-1 ***

addr2line = C:/ncs/v2.6.1/zephyr/kernel/mutex.c:57

line 57 = mutex->lock_count = 0U;

variable mutex is at 0x200009ce <flash_storage+240> while debugging.

This happens while mounting the littlefs on the external flash chip

Some more info:

CONFIG_HEAP_MEM_POOL_SIZE=2048

CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048

CONFIG_MAIN_STACK_SIZE=4096

CONFIG_LOG_BUFFER_SIZE=4000

I'm not sure where else to look next.  Any suggestions?

[Håkon Alseth]

Hi,

 

addr2line = C:/ncs/v2.6.1/zephyr/kernel/mutex.c:57

line 57 = mutex->lock_count = 0U;

variable mutex is at 0x200009ce <flash_storage+240> while debugging.

Is this the resolved r15/PC value? What about the r14/LR content? This can tell you where you branched from.

 

In addition, you can check the build/zephyr/zephyr.map file and search for the current thread address (omit the 0x, ie. if 0x20005688 is listed as failing thread, then search for 20005688)

 

Kind regards,

Håkon

[daThinkingOne]

[00:00:00.603,668] <err> os: ***** USAGE FAULT *****
[00:00:00.603,668] <err> os: Unaligned memory access
[00:00:00.603,698] <err> os: r0/a1: 0x00000000 r1/a2: 0x00000001 r2/a3: 0x00000001
[00:00:00.603,729] <err> os: r3/a4: 0x200009ce r12/ip: 0x00000000 r14/lr: 0x00016cad
[00:00:00.603,729] <err> os: xpsr: 0x61000000
[00:00:00.603,759] <err> os: s[ 0]: 0x200008bd s[ 1]: 0x00000001 s[ 2]: 0x2000420c s[ 3]: 0x20008db8
[00:00:00.603,790] <err> os: s[ 4]: 0x00000004 s[ 5]: 0x0002f048 s[ 6]: 0x00000000 s[ 7]: 0x0001667f
[00:00:00.603,790] <err> os: s[ 8]: 0x20001bf8 s[ 9]: 0x200008bd s[10]: 0x20008e40 s[11]: 0x20008e40
[00:00:00.603,820] <err> os: s[12]: 0x000257e1 s[13]: 0x00000000 s[14]: 0x00000000 s[15]: 0x0001223d
[00:00:00.603,820] <err> os: fpscr: 0x200040e8
[00:00:00.603,851] <err> os: Faulting instruction address (r15/pc): 0x0002e07a
[00:00:00.603,881] <err> os: >>> ZEPHYR FATAL ERROR 31: Unknown error on CPU 0
[00:00:00.603,912] <err> os: Current thread: 0x20003f18 (unknown)
[00:00:01.047,760] <err> fatal_error: Resetting system

thread: 0x20003f18 = .bss.z_main_thread
0x0000000020003f18 0xc8 zephyr/kernel/libkernel.a(init.c.obj)
0x0000000020003f18 z_main_thread

(r15/pc): 0x0002e07a = C:/ncs/v2.6.1/zephyr/kernel/mutex.c:57
mutex->lock_count = 0U; //int z_impl_k_mutex_init(struct k_mutex *mutex)

(r14/lr): 0x00016cad = C:/ncs/v2.6.1/zephyr/subsys/fs/littlefs_fs.c:890
fs_lock(fs); //static int littlefs_mount(struct fs_mount_t *mountp)

So it's in the main thread.  Looks like setting the lock_count to 0U is not aligned to 32-bits?  Is there a way to align it?  Maybe a compiler switch?

It looks like optimizing the build for speed or debug solves the problem. Optimizing for size gives the "unaligned memory access" fault. Is there another way around this?

[Håkon Alseth]

Hi,

 

Given that it fails only on one specific compiler optimization hints that this has to do with memory usage - could you try adjusting CONFIG_MAIN_STACK_SIZE higher just to ensure that this is not overflowing?

 

Can you also share the partitions.yml + zephyr.dts + .config file generated in your build folder?

 

Kind regards,

Håkon

[daThinkingOne]

Increasing the CONFIG_MAIN_STACK_SIZE from 4096 to 8192 didn't work.

https://www.dropbox.com/scl/fi/karrr6c63l7907xhy9hlp/zephyr.dts?rlkey=9nd03fmmxbuq9hheizxtihun3&st=cqupu032&dl=0

https://www.dropbox.com/scl/fi/tjwo86kdzmkubkbwkfknk/.config?rlkey=nzfocrrd12e9r425m69joi2s7&st=p6kab7s5&dl=0

I can't seem to find a partitions.yml

[Håkon Alseth]

Hi,

 

Thank you for sharing the files.

Based on your DT file, it seems that you are using w25q32jv external flash. The t-exit-dpd parameter seems to be a bit short?

As per the other declarations, it seems that 30 us (30000ns) is needed?

https://github.com/nrfconnect/sdk-zephyr/blob/v3.6.99-ncs2/boards/circuitdojo/feather/circuitdojo_feather_nrf9160_common.dtsi#L170-L171

 

Also, could you try to reduce the SPI speed to for instance 1 MHz for testing purposes?

 

Kind regards,

Håkon

[daThinkingOne]

Changing the t-exit-dpd to 30000 didn't work.  Slowing the SPI speed didn't work either.  The fault message keeps pointing to the same place in that mutex.c file.  Maybe I'll try changing the mutex structure to be packed.  Somehow that var in the structure ends up being mis-aligned in the memory when compiling for size.  If I can't solve this soon, I'll just compile for speed from now on.

[Håkon Alseth]

Hi,

 

Do you see this on all your boards, or on one specifically?

If this happens on more than one board:

Could you try to set a breakpoint in main, then when that has hit, a bp in "C:/ncs/v2.6.1/zephyr/kernel/mutex.c:57", and then check the input memory to see if the variable is unaligned?

 

Kind regards,

Håkon