Default system heap listener

[wkpolara]

How can a heap listener be used on the system heap?  That is, the system heap that is used for k_malloc() and k_free(), with size set in kconfig as follows.  The kconfig to enable the listener is also added

CONFIG_HEAP_MEM_POOL_SIZE=/* size in bytes */
CONFIG_SYS_HEAP_LISTENER=y

In particular, I'm trying to find out what _heap_id should be used with the macros

HEAP_LISTENER_ALLOC_DEFINE(name, _heap_id, _alloc_cb)
HEAP_LISTENER_FREE_DEFINE(name, _heap_id, _free_cb)

I tried using "HEAP_ID_LIBC" for the _heap_id in the above listener define macros.  It builds, but no calls to the _alloc_cb or _free_cb functions occur, despite many uses of k_malloc() and k_free().  I would assume HEAP_ID_LIBC is for C library calls with malloc() and free(), and not the Zephyr memory pool heap functions k_malloc() and k_free().

What should be used for the heap ID with the heap memory pool?  Is there an example of using the heap listener API?

Thank you,

Walt

[Kenneth]

Hello,

Not tried it myself, but I suggest to check out the documentaiton and the test example for the heap:
https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/zephyr/kernel/memory_management/heap.html 
\zephyr\tests\lib\heap\src\

Best regards,
Kenneth

[wkpolara]

Fullscreen 76255.main.c Download

/*
 * Copyright (c) 2016 Intel Corporation
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include <zephyr.h>
#include <drivers/gpio.h>
#include <zephyr/sys/heap_listener.h>

/* 1000 msec = 1 sec */
#define SLEEP_TIME_MS   5000

/* The devicetree node identifier for the "led0" alias. */
#define LED0_NODE DT_ALIAS(led0)


void main_heap_listener_alloc_cbh(uintptr_t heap_id, void *mem, size_t bytes);
void main_heap_listener_free_cbh(uintptr_t heap_id, void *mem, size_t bytes);

//**SYSHP static struct sys_heap listener_heap;
//**SYSHP static void *heapmem[1024];

/* using defined system heap */
//**SYSHP HEAP_LISTENER_ALLOC_DEFINE(AllocListener, HEAP_ID_FROM_POINTER(&listener_heap), main_heap_listener_alloc_cbh);
//**SYSHP HEAP_LISTENER_FREE_DEFINE(FreeListener, HEAP_ID_FROM_POINTER(&listener_heap), main_heap_listener_free_cbh);

/* using default system heap */
HEAP_LISTENER_ALLOC_DEFINE(AllocListener, HEAP_ID_LIBC, main_heap_listener_alloc_cbh);
HEAP_LISTENER_FREE_DEFINE(FreeListener, HEAP_ID_LIBC, main_heap_listener_free_cbh);

void main_heap_listener_alloc_cbh(uintptr_t heap_id, void *mem, size_t bytes)
{
    printk("=> Allocate %i <=\n", bytes);
} // main_heap_listener_alloc_cbh()

void main_heap_listener_free_cbh(uintptr_t heap_id, void *mem, size_t bytes)
{
    printk("=> Free %i <=\n", bytes);
} // main_heap_listener_free_cbh()

/*
 * A build error on this line means your board is unsupported.
 * See the sample documentation for information on how to fix this.
 */
static const struct gpio_dt_spec led = GPIO_DT_SPEC_GET(LED0_NODE, gpios);

void main(void)
{
	int ret;

    /* init system heap */
//**SYSHP     sys_heap_init(&listener_heap, heapmem, 1024);
	
    /* Register heap listeners */
	heap_listener_register(&AllocListener);
	heap_listener_register(&FreeListener);    

    printk("Registered heap listeners..\n");

	if (!device_is_ready(led.port)) {
		return;
	}

	ret = gpio_pin_configure_dt(&led, GPIO_OUTPUT_ACTIVE);
	if (ret < 0) {
		return;
	}

//**SYSHP     void *mem = sys_heap_alloc(&listener_heap, 32U);
    void *mem = k_malloc(32);  /* default heap */

    char *str = (char*)mem;
    str[0] = 'h';

//**SYSHP     sys_heap_free(&listener_heap, mem);
    k_free(mem);        /* default heap */

	while (1) {
		ret = gpio_pin_toggle_dt(&led);
		if (ret < 0) {
			return;
		}
		k_msleep(SLEEP_TIME_MS);
	}
}

Kenneth,

Thank you for the reply.  That heap test source was very helpful for understanding using the system heap and heap listener.

I took the blinky example:
\ncs\v2.0.2\zephyr\samples\basic\blinky

and added a defined system heap and listeners to it.  I updated the prj.conf as follows:

CONFIG_GPIO=y

# System heap
CONFIG_HEAP_MEM_POOL_SIZE=1024
CONFIG_SYS_HEAP_LISTENER=y

And the main.c per the attached file.  I got the listeners working with the defined system heap, "listener_heap", and would see the following on the console:
*** Booting Zephyr OS build v3.0.99-ncs1-1 ***
Registered heap listeners..
=> Allocate 36 <=
=> Free 36 <=

I then commented out the defined heap lines using //**SYSHP, and changed the listener define macros  HEAP_LISTENER_ALLOC_DEFINE & HEAP_LISTENER_FREE_DEFINE to use the default heap ID, 

HEAP_ID_LIBC

I then changed the allocation and free lines to use the default heap:

//**SYSHP     void *mem = sys_heap_alloc(&listener_heap, 32U);
    void *mem = k_malloc(32);  /* default heap */

    char *str = (char*)mem;
    str[0] = 'h';

//**SYSHP     sys_heap_free(&listener_heap, mem);
    k_free(mem);        /* default heap *

 However, I can't get the listeners to work with the default heap.  Is there some other part of the setup for the default heap this is missing?

Thanks,

Walt

[Kenneth]

Good question, don't know, we don't have much experience with this kernel api in specific. I suggest to direct this question to the zephyr support, for instance the zephyr discord channel:
https://docs.zephyrproject.org/latest/introduction/index.html#resources 

Best regards,
Kenneth

[Jason]

Hi，

I am facing the same issue. Have you found a solution?

Thanks!

[wkpolara]

Hi Jason,

Unfortunately I have not.  I posted to the Zephyr users mailing list but got no replies.  Perhaps you might reply adding that you are also facing the same issue?  

https://lists.zephyrproject.org/g/users/message/3122?p=%2C%2C%2C20%2C0%2C0%2C0%3A%3ACreated%2C%2Cheap+listener%2C20%2C1%2C0%2C95045476

Regrettably, poor support is often a hallmark of open source software.

Best,

Walt

[Jason]

Hi, 

wkpolara said:

Regrettably, poor support is often a hallmark of open source software.

yes, but fortunately, in most cases, you may find out by reading the source code.

I have figured out how to get the default system heap. It is a global variable.

so you can call it simply as follow:

extern struct k_heap _system_heap;

void main_heap_listener_alloc_cbh(uintptr_t heap_id, void *mem, size_t bytes);

void main_heap_listener_free_cbh(uintptr_t heap_id, void *mem, size_t bytes);

HEAP_LISTENER_ALLOC_DEFINE(AllocListener, HEAP_ID_FROM_POINTER(&_system_heap.heap), main_heap_listener_alloc_cbh);

HEAP_LISTENER_FREE_DEFINE(FreeListener, HEAP_ID_FROM_POINTER(&_system_heap.heap), main_heap_listener_free_cbh);

[wkpolara]

Jason,

Awesome!  I was able to get it working using the default heap global variable.  Thank you for posting your solution; that's a big help. 

Well done,

Walt

(PS- for posterity, this was working with nRF Connect v2.2.0)

[egood]

This was so hard to find, thanks for posting this. I have been looking for a way to effectivity monitor the system heap use for a while and had finally found sys_heap_runtime_stats_get() and sys_heap_print_info(), but no way to get a ref to the system heap and your replay has done that. Thanks so much for posting this! 

I will add for other out there that may have been looking for the same you can use CONFIG_SYS_HEAP_RUNTIME_STATS to enable the heap stats APIs mentioned above. These can then be called in the following manner:

extern struct k_heap _system_heap;
static struct sys_memory_stats stats;

sys_heap_print_info(&_system_heap,true);
//or
sys_heap_runtime_stats_get(&_system_heap, &stats);

I hope this helps anyone that comes across the same problem.