Two simultaneous heaps combining: kernel heap and libc heap

[Egor Dolgalev]

Hello Support team,

I have an issue with two simultaneous heaps in my project. I use nrf Connect SDK 3.2.2 with the next configuration options:

CONFIG_REQUIRES_FULL_LIBCPP=y

CONFIG_HEAP_MEM_POOL_SIZE=64536

By default I have Picolibc library with COMMON_LIBC_MALLOC_ARENA_SIZE=-1 As I understand it means that libc uses it's own heap with size of none allocated space in RAM

Having two simultaneous heaps is not a good idea with many drawbacks. So:

1. I would like to have only one heap with ability to get current status of free and occupied size. Kernel heap provides such API.

2. It is good to have an ability to configure the heap memory size as it is done for libc with config COMMON_LIBC_MALLOC_ARENA_SIZE=-1 to occupy none allocated memory.

 

Is there any way to combine these two heaps to one? I know that with current configuration when I allocates memory with 'new' operator it uses libc heap. When I create threads dynamically it uses kernel heap. I don't have an ability to estimate libc heap usage (or I don't know about it).

As I understand I can override 'new' and 'delete' operator for C++. It might be possible to override malloc, calloc and realloc functions for C. But how I can do it correctly and what is the list of functions I should override?

So, what is the good solution for it? What do you recommend? 

[Justin012]

Hi Egor,

you’re right — having two heaps is not ideal, and unfortunately there’s no built-in way to truly merge them into one.

Simple approach that works:
 Redirect everything to the Zephyr (kernel) heap. heardle game

That means:

• Override malloc, free, calloc, realloc
• Override C++ new / delete
• Internally call k_malloc / k_free

This way you effectively use one heap only, and you can monitor it with Zephyr APIs.

Also, I’d avoid:

COMMON_LIBC_MALLOC_ARENA_SIZE = -1

Better set a fixed size or fully redirect allocations, otherwise memory usage is hard to control.

In short:
You can’t truly combine both heaps, but you can force everything to use the kernel heap, which is the cleanest solution.

Hope that helps

[Egor Dolgalev]

Justin,

Thank you for this hints. I managed it with the next changes:

1. Override only C allocation functions malloc, calloc, realloc and free to kernel equivalents. Did several experiments and I see that C++ allocation is done via these C functions

2. Exclude Libc implementation of allocation functions and set heap arena size to 0 using next config:

CONFIG_COMMON_LIBC_MALLOC=n

CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=0

CONFIG_COMMON_LIBC_CALLOC=n

CONFIG_COMMON_LIBC_REALLOCARRAY=n

It looks like these approach works in my case. May be you can add something to it.

---

But I faced with one issue for handling Settings subsytem over the MCU manager. I want to handle settings write and read using heap to not to allocate too much stack for McuMgr thread because we do settings read/write only during production stage and the size of the transferred data is significant. So, I tried to set MCUMGR_GRP_SETTINGS_BUFFER_TYPE_HEAP=y but got 'n' due to the dependency of the heap size which is 0

Here is the dependency from Kconfig:

config MCUMGR_GRP_SETTINGS_BUFFER_TYPE_HEAP
bool "Heap (dynamic size)"
depends on COMMON_LIBC_MALLOC_ARENA_SIZE > 0
help
Use dynamic heap memory allocation through malloc, if there is
insufficient heap memory for the allocation then the request will be
rejected.

What could be the solution for this issue?

[Priyanka]

Hi Egor,

Could you try to allocate:

CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=1 

instead of leaving it 0 ? (It's just to satisfy the condition, but the actual allocation should go through your overriden method.)

Please try this and let me know how it goes.

-Priyanka

[Egor Dolgalev]

Hi Priyanka,

I tried it. To set any value to CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE I have to enable CONFIG_COMMON_LIBC_MALLOC. Enabling CONFIG_COMMON_LIBC_MALLOC leads to malloc function compilation and I have conflict with my malloc implementation

[Priyanka]

Hi Egor,

Apologies for the delayed response. I think it would be better then to keep using the stack buffer and increase the MCUmgr thread stack size if needed.

CONFIG_MCUMGR_GRP_SETTINGS_BUFFER_TYPE_STACK=y
CONFIG_MCUMGR_TRANSPORT_WORKQUEUE_STACK_SIZE=<larger value if needed>

If you really need MCUMgr to use heap memory instead, then I think that the Zephyr/MCUmgr would need a small code change so it uses k_malloc() directly, instead of depending on Zephyr’s libc malloc heap. But that means carrying a local patch. So, it might be better to use the stack buffer.

-Priyanka

[Egor Dolgalev]

Hi Priyanka,

Thank you for this answer. I think that the dependency in Kconfig for MCUMGR_GRP_SETTINGS_BUFFER_TYPE_HEAP is not quite correct. It relies on COMMON_LIBC_MALLOC_ARENA_SIZE > 0 condition but in implementation it uses malloc call to allocate the memory and as we see the malloc function does not come from the LibC all the time. 

Anyway, any fix for heap allocation in my case needs to carry a local patch for now.