An issue with memory allocation in NRF52832 when working with JSON files

Hello,

I am using the NRF52832 processor on a custom board to collect data from a sensor and send it to a mobile app using BLE and NUS service in JSON format. However, after a few minutes, the transmission stops due to memory allocation failure.

Here is a piece of my code:

        root_G_P_R=cJSON_CreateObject();   

        cJSON_AddItemToObject(root_G_P_R, "info", info_G_P_R=cJSON_CreateObject());

        cJSON *plan_date = cJSON_CreateArray();

        for (int i= 0 ; i<APP_PlanCounter+1;i++)

        {

        cJSON_AddItemToArray(plan_date, cJSON_CreateNumber(APP_PlanDate[i]));

        }

        cJSON *TR = cJSON_CreateArray();

        for (int i= 0 ; i<APP_PlanCounter+1;i++)

        {

        cJSON_AddItemToArray(TR, cJSON_CreateNumber(APP_TRS[i]));

        }

        cJSON *TRCR = cJSON_CreateArray();

        for (int i= 0 ; i<APP_PlanCounter+1;i++)

        {

        cJSON_AddItemToArray(TRCR, cJSON_CreateNumber(App_TRCS[i]));

        }

        cJSON_AddItemToObject(root_G_P_R,    "key",            cJSON_CreateString("G_P_R"));

        cJSON_AddItemToObject(info_G_P_R,    "tr",     TR);

        cJSON_AddItemToObject(info_G_P_R,    "trac",     TRCR);

        cJSON_AddNumberToObject(info_G_P_R,  "plan",            plan);

        cJSON_AddNumberToObject(info_G_P_R,    "PT",           App_PT[APP_PlanCounter]);

        cJSON_AddItemToObject(info_G_P_R,    "date",           plan_date);

     if (print_preallocated(root_G_P_R) != 0) {

        cJSON_Delete(root_G_P_R);

        exit(EXIT_FAILURE);

    }

    cJSON_Delete(root_G_P_R);

and in the case of print_preallocated:

  static int print_preallocated(cJSON *root)

{

 

    char *out = NULL;

    char *buf = NULL;

    char *buf_fail = NULL;

    uint16_t len = 0;

    size_t len_fail = 0;

 

    out = cJSON_Print(root);

 

 

    len = strlen(out) + 5;

    buf = (char*)malloc(len);

 

   

    if (buf == NULL)

    {

        printf("Failed to allocate memory.\n");

        exit(1);

    }

    len_fail = strlen(out);

    buf_fail = (char*)malloc(len_fail);

    if (buf_fail == NULL)

    {

        printf("Failed to allocate memory.\n");

        exit(1);

    }

     ble_nus_data_send(&m_nus, (uint8_t *)out , &len, m_conn_handle);

    free(out);

    free(buf_fail);

    free(buf);

    return 0;

}

 

I have read in a thread that applying cJSON_Init() to the main would be helpful, but when I added this function, the JSON parse data process failed:

cJSON_Parse(received_data) --> cJSON_ParseWithOpts --> cJSON_New_Item --> cJSON_malloc

I suspect that there is something wrong with memory allocation. Could you please help me with this problem? Is there any configuration or consideration that I should have taken but missed out on?

 Thank you for your assistance.

Parents
  • Hi,

     

    I have read in a thread that applying cJSON_Init() to the main would be helpful, but when I added this function, the JSON parse data process failed:

    cJSON_Parse(received_data) --> cJSON_ParseWithOpts --> cJSON_New_Item --> cJSON_malloc

    I suspect that there is something wrong with memory allocation. Could you please help me with this problem? Is there any configuration or consideration that I should have taken but missed out on?

    What is the heap size set to on your end?

    Please note that you're using libc dynamic alloc functions, ie. "malloc()" instead of zephyr's kernel heap "k_malloc()"- this will use the heap by your selected libc (normally newlib or minimal libc, symbols NEWLIB_LIBC_MIN_REQUIRED_HEAP_SIZE / MINIMAL_LIBC_MALLOC_ARENA_SIZE)

     

    Kind regards,

    Håkon

  • Hi,
    Thank you for your prompt response.

    What is the heap size set to on your end?

    It's 8kb (0x20005910 - 0x2000790f).


    It seems that multiple function callings and transmitting the mentioned data cause the heap section to fill up after several uses. I have traced some allocated memory addresses in this scenario:


    allocated address : 200074dc
    allocated address : 2000741c
    allocated address : 200073dc
    allocated address : 200072dc
    allocated address : 2000728c
    allocated address : 200071cc
    allocated address : 2000717c
    allocated address : 200070fc
    allocated address : 2000709c
    allocated address : 2000706c
    allocated address : 20006fec
    allocated address : 20006fdc
    allocated address : 20006f2c
    allocated address : 20006e9c
    allocated address : 20006e2c
    allocated address : 20006dac
    allocated address : 20006d3c
    allocated address : 20006d2c
    allocated address : 20006c4c
    allocated address : 20006bdc
    allocated address : 20006b6c
    allocated address : 20006b3c
    allocated address : 20006afc
    allocated address : 20006acc
    allocated address : 20006a8c
    allocated address : 20006a7c
    allocated address : 200069cc
    allocated address : 200068fc
    allocated address : 200068cc
    allocated address : 2000681c
    allocated address : 2000677c
    allocated address : 2000673c
    allocated address : 2000669c
    allocated address : 2000664c
    allocated address : 2000657c
    allocated address : 2000655c
    allocated address : 200064dc
    allocated address : 200063ec
    allocated address : 200063cc
    allocated address : 2000635c
    allocated address : 200062ec
    allocated address : 2000626c
    allocated address : 2000619c
    allocated address : 2000611c
    allocated address : 2000609c
    allocated address : 2000608c
    allocated address : 2000601c
    allocated address : 20005fac
    allocated address : 20005f3c
    allocated address : 20005edc
    allocated address : 20005e9c
    allocated address : 20005e4c
    allocated address : 20005ddc
    allocated address : 20005d7c
    allocated address : 20005d1c
    allocated address : 20005cec
    allocated address : 20005c5c
    allocated address : 20005bec
    allocated address : 20005b5c
    allocated address : 20005b1c
    allocated address : 20005b8c
    allocated address : 20005b2c
    allocated address : 20005a4c

    As you may have noticed, the beginning and end of these addresses are quite close to the start and end of the heap section. Is there anything I can do to prevent this? My goal is for users to be able to access this data as frequently as they desire through the mobile app, but it currently only works for a limited time.

    I appreciate the time you have taken.

    ZaQa

  • Hi ZaQa,

     

    ZaQa said:
    It seems that multiple function callings and transmitting the mentioned data cause the heap section to fill up after several uses. I have traced some allocated memory addresses in this scenario:

    You can lock the function with a semaphore for instance.

    The allocation print indicate that you're not free'ing the data? It only goes in one direction.

    You have to debug to see where the memory is leaking.

     

         ble_nus_data_send(&m_nus, (uint8_t *)out , &len, m_conn_handle);

    Sorry, I assumed you were using NCS, but this function call points to nRF5 SDK.

     

    You must check your return here, if not; you risk losing data.

    In general, I would recommend that you check your returned pointers, also from the cJSON_* calls as well.

     

    Kind regards,

    Håkon

Reply
  • Hi ZaQa,

     

    ZaQa said:
    It seems that multiple function callings and transmitting the mentioned data cause the heap section to fill up after several uses. I have traced some allocated memory addresses in this scenario:

    You can lock the function with a semaphore for instance.

    The allocation print indicate that you're not free'ing the data? It only goes in one direction.

    You have to debug to see where the memory is leaking.

     

         ble_nus_data_send(&m_nus, (uint8_t *)out , &len, m_conn_handle);

    Sorry, I assumed you were using NCS, but this function call points to nRF5 SDK.

     

    You must check your return here, if not; you risk losing data.

    In general, I would recommend that you check your returned pointers, also from the cJSON_* calls as well.

     

    Kind regards,

    Håkon

Children
  • You must check your return here, if not; you risk losing data.

    That's correct. I investigated and now I'm verifying the value returned by the ble_nus_data_send function, which seems to be working properly.

    Upon thoroughly examining my code, I have come to the realization that the free() function does not release allocated space. Consequently, over time, the heap section becomes full and ultimately causes the code to fail.

    Could you please guide me on how to effectively resolve this issue? I would greatly appreciate your kind assistance.

  • Hi,

     

    Dynamic memory handling is not always easy to handle, and should ideally only be used when there's no other option.

    cJSON is one scenario where it makes sense to use dynamic allocation.

     

    ZaQa said:
    Upon thoroughly examining my code, I have come to the realization that the free() function does not release allocated space. Consequently, over time, the heap section becomes full and ultimately causes the code to fail.

    Which compiler are you using? is alloc used in other functions of your firmware?

     

    Kind regards,

    Håkon

  • Hi Håkon,

    Which compiler are you using? is alloc used in other functions of your firmware?

    I'm currently using Segger Embedded Studio as IDE and GCC as the compiler. As for the second question, alloc is only utilized within the cJSON code and not elsewhere. 

    I appreciate your response.

    ZaQa

  • Hi,

     

    Looking at it again I found one place where the memory is free'd:

    ZaQa said:
    allocated address : 20005b5c
    allocated address : 20005b1c
    allocated address : 20005b8c

    This is the only place where you actually free the allocated memory, thus the HEAP goes up again. You need to adjust your heap size and/or handle your timing accordingly.

     

    Try to print + when alloc'ing, and - when free'ing memory to see how often one occurs vs. the other.

     

    Kind regards,

    Håkon

  • Hi,

    This is the only place where you actually free the allocated memory, thus the HEAP goes up again. You need to adjust your heap size and/or handle your timing accordingly.

    I tried it again, but this time I increased the heap size to 16KB. However, I obtained the same result:

    allocated address : 2000955c

    allocated address : 2000949c

    allocated address : 2000945c

    allocated address : 2000934c

    allocated address : 2000930c

    allocated address : 200092dc

    allocated address : 2000920c

    allocated address : 200091dc

    allocated address : 2000915c

    allocated address : 2000910c

    allocated address : 2000908c

    allocated address : 2000904c

    allocated address : 2000901c

    allocated address : 20008fcc

    allocated address : 20008f1c

    allocated address : 20008eec

    allocated address : 20008edc

    allocated address : 20008eac

    allocated address : 20008e7c

    allocated address : 20008e6c

    allocated address : 20008e3c

    allocated address : 20008dbc

    allocated address : 20008d7c

    allocated address : 20008d0c

    allocated address : 20008c8c

    allocated address : 20008c2c

    allocated address : 20008bfc

    allocated address : 20008bec

    allocated address : 20008bbc

    allocated address : 20008b3c

    allocated address : 20008aec

    allocated address : 20008adc

    allocated address : 20008acc

    allocated address : 20008abc

    allocated address : 20008aac

    allocated address : 20008a9c

    allocated address : 20008a8c

    allocated address : 20008a7c

    allocated address : 20008a6c

    allocated address : 20008a5c

    allocated address : 20008a4c

    allocated address : 200089cc

    allocated address : 2000898c

    allocated address : 2000895c

    allocated address : 2000892c

    allocated address : 2000891c

    allocated address : 200088ec

    allocated address : 200088bc

    allocated address : 2000889c

    allocated address : 2000888c

    allocated address : 2000887c

    allocated address : 2000886c

    allocated address : 2000885c

    allocated address : 2000884c

    allocated address : 2000883c

    allocated address : 2000882c

    allocated address : 2000881c

    allocated address : 2000880c

    allocated address : 200087fc

    allocated address : 200087ec

    allocated address : 2000879c

    allocated address : 200086ec

    allocated address : 2000861c

    allocated address : 2000852c

    allocated address : 200084bc

    allocated address : 200083cc

    allocated address : 2000837c

    allocated address : 200082fc

    allocated address : 200082bc

    allocated address : 2000823c

    allocated address : 2000818c

    allocated address : 2000810c

    allocated address : 200080cc

    allocated address : 20007ffc

    allocated address : 20007fac

    allocated address : 20007f6c

    allocated address : 20007f3c

    allocated address : 20007f0c

    allocated address : 20007edc

    allocated address : 20007e9c

    allocated address : 20007e4c

    allocated address : 20007dac

    allocated address : 20007d4c

    allocated address : 20007cfc

    allocated address : 20007ccc

    allocated address : 20007c4c

    allocated address : 20007c1c

    allocated address : 20007b9c

    allocated address : 20007a7c

    allocated address : 20007a4c

    allocated address : 200079bc

    allocated address : 2000797c

    allocated address : 2000791c

    allocated address : 200078bc

    allocated address : 200077ec

    allocated address : 200077bc

    allocated address : 200077ac

    allocated address : 2000779c

    allocated address : 2000777c

    allocated address : 200076dc

    allocated address : 2000767c

    allocated address : 2000764c

    allocated address : 2000761c

    allocated address : 2000757c

    allocated address : 2000754c

    allocated address : 200074ec

    allocated address : 2000747c

    allocated address : 2000742c

    allocated address : 200073ac

    allocated address : 2000736c

    allocated address : 2000733c

    allocated address : 2000730c

    allocated address : 200072dc

    allocated address : 200072ac

    allocated address : 2000727c

    allocated address : 2000724c

    allocated address : 2000721c

    allocated address : 2000714c

    allocated address : 2000705c

    allocated address : 20006fbc

    allocated address : 20006f0c

    allocated address : 20006e4c

    allocated address : 20006e1c

    allocated address : 20006d3c

    allocated address : 20006d2c

    allocated address : 20006d1c

    allocated address : 20006d0c

    allocated address : 20006cbc

    allocated address : 20006bfc

    allocated address : 20006b3c

    allocated address : 20006afc

    allocated address : 20006aec

    allocated address : 20006adc

    allocated address : 20006acc

    allocated address : 20006abc

    allocated address : 20006aac

    allocated address : 20006a9c

    allocated address : 20006a8c

    allocated address : 20006a7c

    allocated address : 20006a6c

    allocated address : 20006a5c

    allocated address : 20006a4c

    allocated address : 200069ac

    allocated address : 2000691c

    allocated address : 200068ec

    allocated address : 200068cc

    allocated address : 2000683c

    allocated address : 200067dc

    allocated address : 200067bc

    allocated address : 2000678c

    allocated address : 2000673c

    allocated address : 2000670c

    allocated address : 2000668c

    allocated address : 200065fc

    allocated address : 200065cc

    allocated address : 2000656c

    allocated address : 200064cc

    allocated address : 2000646c

    allocated address : 2000643c

    allocated address : 2000640c

    allocated address : 2000639c

    allocated address : 2000634c

    allocated address : 200062fc

    allocated address : 200062cc

    allocated address : 2000629c

    allocated address : 2000623c

    allocated address : 200061ec

    allocated address : 2000617c

    allocated address : 2000615c

    allocated address : 200060ec

    allocated address : 2000608c

    allocated address : 2000603c

    allocated address : 2000601c

    allocated address : 20005ffc

    allocated address : 20005fdc

    allocated address : 20005fbc

    allocated address : 20005f9c

    allocated address : 20005f6c

    allocated address : 20005f4c

    allocated address : 20005f1c

    allocated address : 20005efc

    allocated address : 20005edc

    allocated address : 20005eac

    allocated address : 20005e8c

    allocated address : 20005e6c

    allocated address : 20005e5c

    allocated address : 20005e3c

    allocated address : 20005ddc

    allocated address : 20005d8c

    allocated address : 20005d2c

    allocated address : 20005d0c

    allocated address : 20005cdc

    allocated address : 20005c3c

    allocated address : 20005c0c

    allocated address : 20005bac

    allocated address : 20005b4c

    allocated address : 20005b1c

    allocated address : 20005aec

    allocated address : 2000596c

    allocated address : 20005a2c

    allocated address : 200059ec

    allocated address : 0

    Finally, for the first and last time allocated address goes up from 2000596c to 20005a2c and then decreases to 0.
    Is it possible to clear the entire heap and start anew, perhaps when the allocated address reaches 0?

    Try to print + when alloc'ing, and - when free'ing memory to see how often one occurs vs. the other.


    I'm unsure how to ensure the allocated memory has been freed since the free() function doesn't return any value to confirm its success. Do you have any suggestions on how to address this issue?

    Thank you for your time.

Related