having issues with saving coredump to flash or at all

Hi Nordic

I am working with nrf52840 and nrf52832 using ncs v2.8.0

I am trying to save coredump to flash according to instructions on this link - https://docs.nordicsemi.com/bundle/ncs-2.8.0/page/zephyr/services/debugging/coredump.html

I added this to my pm_static_my_board.yml

coredump_partition:
  address: 0xCF000
  size: 0x8000
  region: flash_primary

And this to my_board.overlay

&flash0 {
    /*
     * For more information, see:
     * http: //docs.zephyrproject.org/latest/guides/dts/index.html#flash-partitions
     */
    partitions {
        compatible = "fixed-partitions";
        #address-cells = <1>;
        #size-cells = <1>;

      ...
        coredump_partition: partition@000080000 { //THIS IS NOT LEGIT ADDRESS(END OF FLASH) BUT IT IS NOT TAKEN TO ACOUNT BECAUS PM_STATIC IS
            label = "coredump-partition";
            reg = <0x000080000 DT_SIZE_K(4)>;
        };
    };

A side note is that this is strange that I need to set it in the overlay which is basically ignored because pm_static partitions is the one that actually matters (unless i got something wrong ? )

And this configs to my prj.conf

# Coredump 
CONFIG_DEBUG_COREDUMP=y
CONFIG_DEBUG_COREDUMP_BACKEND_FLASH_PARTITION=y
CONFIG_DEBUG_COREDUMP_MEMORY_DUMP_THREADS=y

In my my_board/my_app/zephyr/.config i see this coredump related configs

CONFIG_ARCH_SUPPORTS_COREDUMP=y
CONFIG_ARCH_SUPPORTS_COREDUMP_THREADS=y

# CONFIG_COREDUMP_DEVICE is not set

CONFIG_DEBUG_THREAD_INFO=y
CONFIG_DEBUG_COREDUMP=y
# CONFIG_DEBUG_COREDUMP_BACKEND_LOGGING is not set
CONFIG_DEBUG_COREDUMP_BACKEND_FLASH_PARTITION=y
# CONFIG_DEBUG_COREDUMP_BACKEND_OTHER is not set
# CONFIG_DEBUG_COREDUMP_MEMORY_DUMP_MIN is not set
CONFIG_DEBUG_COREDUMP_MEMORY_DUMP_THREADS=y
# CONFIG_DEBUG_COREDUMP_MEMORY_DUMP_LINKER_RAM is not set
CONFIG_DEBUG_COREDUMP_FLASH_CHUNK_SIZE=64
CONFIG_DEBUG_COREDUMP_THREADS_METADATA=y

I am generating a coredump using this implementation 

void trigger_coredump(void)
{
    __ASSERT(0, "Forcing coredump");
}

When i try to read the flash area after generating the coredump with nrfjprog --memrd 0xCF000 --w 32 --n 0x8000
i get all 0xFF 

what i am missing ?

I also tried to check myself by replacing CONFIG_DEBUG_COREDUMP_BACKEND_FLASH_PARTITION=y

With CONFIG_DEBUG_COREDUMP_BACKEND_LOGGING=y

Hopping to see the coredump on my open rtt but nothing .. when coredump is triggered prints just stop

  1. What am I missing? Why can't I find a coredump on the flash partition or in the rtt log ?
  2. Can it be that the device does not have the time to write the coredump before the actual crash ? If so, how can I manage that ?
  3. Is there some auto deletion of the flash partition with the coredump so new coredumps can be saved or is it something i have to manage myself after i read the coredump from flash ? 

Hope to read you soon

Best regards

Ziv

  • I just stumbled across this PR https://github.com/nrfconnect/sdk-nrf/pull/21418 which I assume should address the limitation of not being able to enable ASSERTs when storing to internal flash.

    though i think my issue is different and my problem is before even trying to write to flash ..

    is there a quick way to know if this merged change is relevant to the ncs version i am working with (2.8.0) ? (planning to update but it will take some time) 

  • what i see is the same weather i try to assert or panic or set a pointer to forbidden address.

    i do not get into z_fatal_error in any of those cases, or to assert_print .. it seems like device just crashes before everything, regardless to writing or not writing the CD

    Please see my earlier response where I tried to explain this and linked to the Zephyr issue. What is relevant here is whether you have CONFIG_ASSERT enabled in your build not. 

    i actually thought about implementing my own hard-fault handler and in it call for z_fatal_error in order to have the CD written to INTERNAL flash (i am trying to use it as suppose to be supported by zephyr debug coredump) but i am not sure what is the API to use to overwrite the existing nrf hard-fault handler (would be great if you can point me for where to look or what the API is.

    ... i am guessing memfault does that as well, though i am not sure if it means they need to have different implementation for each hardware - nrf, stm, etc. cause i am guessing they don't all have the same API for fault handling) 

    You can simply redefine the weakly defined k_sys_fatal_error_handler() function in your application. 

    is there a quick way to know if this merged change is relevant to the ncs version i am working with (2.8.0) ? (planning to update but it will take some time) 

    The new flash backend added by https://github.com/nrfconnect/sdk-nrf/pull/21418 is required if you are going to have asserts enabled globally. 

  •   I reproduced the fault so I can provide a full log:

    [00:00:31.627,532] <err> os: ***** USAGE FAULT *****
    [00:00:31.633,148] <err> os:   Unaligned memory access
    [00:00:31.639,007] <err> os: r0/a1:  0x00000000  r1/a2:  0x00000000  r2/a3:  0x00000001
    [00:00:31.647,674] <err> os: r3/a4:  0xfffff000 r12/ip:  0x00041f9b r14/lr:  0x00024719
    [00:00:31.656,372] <err> os:  xpsr:  0x29100000
    [00:00:31.661,621] <err> os: s[ 0]:  0x20004a40  s[ 1]:  0x00042f93  s[ 2]:  0x00042f77  s[ 3]:  0x20022120
    [00:00:31.672,027] <err> os: s[ 4]:  0x0004f6dc  s[ 5]:  0x00000156  s[ 6]:  0x0004f654  s[ 7]:  0x00021673
    [00:00:31.682,464] <err> os: s[ 8]:  0x0004f654  s[ 9]:  0x00000156  s[10]:  0x20022120  s[11]:  0x200069f7
    [00:00:31.692,901] <err> os: s[12]:  0x20022120  s[13]:  0x200069fb  s[14]:  0x0004e19c  s[15]:  0x00041fa9
    [00:00:31.703,308] <err> os: fpscr:  0x20022120
    [00:00:31.708,526] <err> os: Faulting instruction address (r15/pc): 0x0002a2c4
    [00:00:31.716,461] <err> os: >>> ZEPHYR FATAL ERROR 31: Unknown error on CPU 0
    [00:00:31.724,365] <err> os: Current thread: 0x20001c20 (mp_main)
    [00:00:31.731,140] <err> coredump: #CD:BEGIN#
    [00:00:31.736,206] <err> coredump: #CD:5a450200030005001f000000
    [00:00:31.742,797] <err> coredump: #CD:4102004400
    [00:00:31.748,199] <err> coredump: #CD:00000000000000000100000000f0ffff9b1f040019470200c4a2020000001029
    [00:00:31.758,300] <err> coredump: #CD:6020022000000000000000000000000000000000000000000000000000000000
    [00:00:31.768,341] <err> coredump: #CD:00000000
    [00:00:31.773,590] <err> coredump: #CD:4d0100201c0020301d0020
    [00:00:31.780,029] <err> coredump: #CD:641e0020641e0020000000000180000000000000000000000000000000000000
    [00:00:31.790,100] <err> coredump: #CD:2f0a040000000000df72070000000000e5970a00000000000100000000000000
    [00:00:31.800,201] <err> coredump: #CD:10000000684900200080000000000000f0610500740000007903000010000000
    [00:00:31.810,272] <err> coredump: #CD:e820022000000000881c0020881c002000000000adbe01000000000000000000
    [00:00:31.820,373] <err> coredump: #CD:00000000000000006d705f6d61696e0000000000000000000000000000000000
    [00:00:31.830,444] <err> coredump: #CD:0000000000000000e849002010000000400202200020000000000000740d0020
    [00:00:31.840,545] <err> coredump: #CD:00000000f5ffffff000000000000000000000000000000000000000000000000
    [00:00:31.850,616] <err> coredump: #CD:0000000000000000000000000000000000000000000000000000000000000000
    [00:00:31.860,687] <err> coredump: #CD:000000000000000000ed000000000000
    [00:00:31.868,011] <err> coredump: #CD:4d01004002022040220220
    [00:00:31.874,450] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.884,521] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.894,622] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.904,693] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.914,794] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.924,865] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.934,967] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.945,037] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.955,139] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.965,209] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.975,311] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.985,382] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:31.995,483] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.005,554] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.015,655] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.025,726] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.035,827] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.045,928] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.055,999] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.066,101] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.076,171] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.086,273] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.096,343] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.106,445] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.116,546] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.126,617] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.136,718] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.146,789] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.156,890] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.166,961] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.177,062] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.187,164] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.197,235] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.207,336] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.217,407] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.227,508] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.237,579] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.247,680] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.257,751] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.267,852] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.277,923] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.288,024] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.298,095] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.308,197] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.318,298] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.328,369] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.338,470] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.348,571] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.358,642] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.368,743] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.378,845] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.388,916] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.399,017] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.409,118] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.419,189] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.429,290] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.439,361] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.449,462] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.459,564] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.469,635] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.479,736] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.489,837] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.499,908] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.510,009] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.520,111] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.530,181] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.540,283] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.550,384] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.560,455] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.570,556] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.580,627] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.590,728] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.600,830] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.610,900] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.620,971] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.631,072] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.641,174] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.651,245] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.661,346] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.671,417] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.681,518] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.691,589] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.701,690] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.711,791] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.721,862] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.731,933] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.742,034] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.752,105] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.762,207] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.772,277] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.782,379] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.792,480] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.802,551] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.812,652] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.822,723] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.832,794] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.842,895] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.852,966] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.863,067] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.873,138] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.883,209] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.893,310] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.903,381] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.913,482] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.923,553] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.933,654] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.943,725] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.953,826] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.963,928] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.973,999] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.984,100] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:32.994,171] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.004,272] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.014,343] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.024,444] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.034,515] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.044,616] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.054,687] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.064,788] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.074,859] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.084,960] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.095,031] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.105,133] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.115,203] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.125,305] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.135,375] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.145,477] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.155,578] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.165,649] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.175,750] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.185,821] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.195,922] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.205,993] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.216,094] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.226,165] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.236,267] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.246,368] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.256,439] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.266,540] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.276,611] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.286,712] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.296,783] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.306,884] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.316,955] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.327,056] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.337,127] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.347,229] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.357,299] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.367,401] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.377,502] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.387,573] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.397,674] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.407,745] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.417,846] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.427,917] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.437,988] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.448,089] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.458,190] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.468,261] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.478,332] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.488,433] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.498,504] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.508,605] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.518,676] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.528,778] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.538,848] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.548,950] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.559,020] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.569,091] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.579,193] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.589,263] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.599,365] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.609,436] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.619,506] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.629,608] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.639,678] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.649,780] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.659,851] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.669,921] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.680,023] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.690,124] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.700,195] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.710,296] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.720,367] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.730,468] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.740,539] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.750,640] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.760,711] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.770,812] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.780,883] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.790,985] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.801,055] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.811,157] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.821,258] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.831,329] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.841,430] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.851,501] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.861,602] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.871,673] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.881,774] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.891,845] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.901,947] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.912,017] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.922,119] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.932,189] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.942,291] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.952,362] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.962,463] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.972,534] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.982,635] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:33.992,736] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.002,807] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.012,908] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.022,979] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.033,081] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.043,151] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.053,253] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa01000000aaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.063,323] <err> coredump: #CD:aaaaaaaa11000000b81d022054f604008a000000b30100000700000008000000
    [00:00:34.073,425] <err> coredump: #CD:9e640500b52f0400772f0400b81d0220dcf6040011000000b81d022073160200
    [00:00:34.083,496] <err> coredump: #CD:10f70400b301000000000000e0f80400b301000095e901000000000000000000
    [00:00:34.093,597] <err> coredump: #CD:000200000700000000000000e8490020b301000055290300aaaaaaaa02000000
    [00:00:34.103,698] <err> coredump: #CD:30690020aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa0002000000000000
    [00:00:34.113,769] <err> coredump: #CD:081e022001aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.123,870] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.133,972] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.144,042] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.154,144] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.164,215] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.174,316] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.184,387] <err> coredump: #CD:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.194,488] <err> coredump: #CD:381f022001000000381f0220d8aa0400481f022001000000481f0220d8aa0400
    [00:00:34.204,589] <err> coredump: #CD:14060500c5460000601f022001000000601f0220d8aa040090200220c5460000
    [00:00:34.214,660] <err> coredump: #CD:a946000018200220801f022001000000801f0220d8aa0400901f022001000000
    [00:00:34.224,761] <err> coredump: #CD:901f0220d8aa040014060500c5460000a81f022001000000a81f0220d8aa0400
    [00:00:34.234,832] <err> coredump: #CD:d8200220c5460000a9460000602002200100000004000000bc00000003000000
    [00:00:34.244,934] <err> coredump: #CD:02000000d9a9010001000000180000007e2105002fd20300bc00000003000000
    [00:00:34.255,004] <err> coredump: #CD:02000000d9a901000100000028000000010000009a0d00003000000014210220
    [00:00:34.265,106] <err> coredump: #CD:4c6b0020040000000100000003000000010000009a0d00000000000000000000
    [00:00:34.275,207] <err> coredump: #CD:0100000000f0ffff9b1f040019470200c4a2020000001029404a0020932f0400
    [00:00:34.285,278] <err> coredump: #CD:772f040020210220dcf604005601000054f604007316020054f6040056010000
    [00:00:34.295,379] <err> coredump: #CD:20210220f769002020210220fb6900209ce10400a91f04002021022019470200
    [00:00:34.305,450] <err> coredump: #CD:28210220e7c30200d9c2020000000000d06a0020200000001c210220e4200220
    [00:00:34.315,551] <err> coredump: #CD:00000000d1640400a1640400e4200220d06a00203b570400602102200dcc0200
    [00:00:34.325,622] <err> coredump: #CD:213a020008210220000000000821022028210220200000002421022000000000
    [00:00:34.335,723] <err> coredump: #CD:0300000060200220000000000000000000000000000000000000000000000000
    [00:00:34.345,794] <err> coredump: #CD:582102200821022000000000406a002028210220200000000000000000000000
    [00:00:34.355,895] <err> coredump: #CD:0300000025240400406a0020f96900201821022003000000404a002010e60400
    [00:00:34.365,966] <err> coredump: #CD:9ce1040000000000706a00200000000000000000b06a00201600000000000000
    [00:00:34.376,068] <err> coredump: #CD:bc000000000000000000000000000000f0df0120e1a10200d06d002000000000
    [00:00:34.386,138] <err> coredump: #CD:0000000000000000aba102001822022016000000dc0500008169002000000000
    [00:00:34.396,240] <err> coredump: #CD:0000000000000000f0df01205821022000000000000000000000000000000000
    [00:00:34.406,341] <err> coredump: #CD:000000000000000000000000e849002000000000dc05000081690020401e0020
    [00:00:34.416,412] <err> coredump: #CD:404a0020c7c30100aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.426,513] <err> coredump: #CD:aaaaaaaaf75f0020d4650020f0680020cb080000aaaaaaaaaaaaaaaaaaaaaaaa
    [00:00:34.436,584] <err> coredump: #CD:00000000aaaaaaaaaaaaaaaa40040500aaaaaaaaaaaaaaaa2000000011000000
    [00:00:34.446,685] <err> coredump: #CD:706900200000000000000000402a0220101b002065550000000000003b510000
    [00:00:34.456,726] <err> coredump: #CD:END#
    [00:00:34.461,608] <err> os: Halting system

    The part with "USAGE FAULT" is the crash logs; and the part where every line starts with "CD" is the coredump.

      So your main suggestion is that Coredump storage to internal/ external flash is too much of a hassle and I should look into your RAM suggestion or redefine k_sys_fatal_error_handler() and make my own implementation to store the crash logs to internal flash and from there, use the "Faulting instruction address" to figure out which line of code caused the fault?

  • So your main suggestion is that Coredump storage to internal/ external flash is too much of a hassle and I should look into your RAM suggestion or redefine

    Yes, as the amount of data that can be included in the coredump is limited by the amount of memory you have available and I don’t see what additional information a minimal core dump provides compared to the crash log. 

  • Well, given a hypothetical scenario where we have a function "myFunc(int a)" and it's used in 7 different files in our project and the crash log just says that "myFunc(int a)" is the Faulting instruction, with the crash log we can't really figure out what caused the error, but given that the coredump provides a stack trace, we can.

Related