I need to monitor how much heap was ever left available, as a minimum, after I have run my CI test suite. This is the true minimum heap ever remaining, so taking account of calls to malloc() as well as to Zephyr's memory management APIs, i.e. the true underlying heap low water mark.
I could do this on NRF52 as I had the implementation of _sbrk() that FreeRTOS called into (the Dave Nadler one).
What is the equivalent in the nRFConnect/Zephyr world on NRF53?
Rob
I'm not sure about this. I'll ask internally about it. You could also try to ask in any of the resources listed here.
Best regards,
Simon
FYI, just been pointed at this: https://github.com/zephyrproject-rtos/zephyr/issues/23076. Seems that heap monitoring is not yet implemented in Zephyr. But even if it was it doesn't answer the whole question because I guess Zephyr would likely sit on top of malloc() rather than below. How do I monitor the true heap usage on NRF53 using nRFConnect/.Zephyr?
I got some answers internally. It does not seem like there is something built-in for "heap painting". However, for stack there are, check out the config CONFIG_INIT_STACKS.
In order to achieve this, you would need to initialize the area on init, and clear/reset every allocated/free'd blob. Do you know how to go about this? If not I can look into it and try to make it work.
Best regards,
Simon
We have stack sorted (we call k_thread_stack_space_get()), it's heap that we can't measure.
I had thought that it would be possible to do a maximum heap extent measurement by simply monitoring newlib's calls to sbrk(), since newlib will not ask for more space until it needs it. But it might be that newlib calls for more space irrespective of whether there might be a free'd block it could use, deliberating putting off the freeing process as an expensive operation.
So yes, if you were able to devise a mechanism to monitor heap I would be very interested.
I'll put off some time Wednesday to look into this.
Best regards,
Simon
A colleague put some work into this, check it out here: https://github.com/zephyrproject-rtos/zephyr/pull/29735.
The approach I mentioned earlier, by filling the heap and looking at it, may be misleading. The minimum largest contiguous region is probably the best "low water mark", since it will react to fragmentation of the heap.
Best regards,
Simon
Agree that largest contiguous region is probably a better measure, also saw your colleague "bjda" post that code on the Github thread, thanks.
The remaining question, then, is how Zephyr uses the heap. We are, of course, going to be calling malloc()/free() as any C code would, does Zephyr wrap those functions somehow and do entirely it's own thing with the heap memory, or does it work in parallel with them and sit atop malloc()/free() itself? If the former then your colleague's solution is good, if the latter it won't help because it's only half the story.
Or do you suggest I take this conversation up with your colleague on the Github issue thread?
Agree that largest contiguous region is probably a better measure, also saw your colleague "bjda" post that code on the Github thread, thanks.
The remaining question, then, is how Zephyr uses the heap. We are, of course, going to be calling malloc()/free() as any C code would, does Zephyr wrap those functions somehow and do entirely it's own thing with the heap memory, or does it work in parallel with them and sit atop malloc()/free() itself? If the former then your colleague's solution is good, if the latter it won't help because it's only half the story.
Or do you suggest I take this conversation up with your colleague on the Github issue thread?
Could you try to ask about this in the GitHub issue? If you don't get any answers, please ping me here.
Will do.
FYI, since newlib is being used by Zephyr under the hood (at least on NRF53) it is almost possible to do this by calling the newlib mallinfo() function. The reason this is only "almost" is that mallinfo() is not the whole story: newlib calls sbrk() as required to give it heap memory from the ultimate heap, so the real number is mallinfo's fordblks plus what's left in sbrk(): without this the reported heap can end up going UP from it's original value, a "negative leak", which would cause any strict heap checking to fail.
Unfortunately, the variable that tracks the memory left inside sbrk(), heap_sz in lib/libc/newlib/libc-hooks.c is static so one can't get at it to do the free heap calculation properly.
I've noted this on https://github.com/zephyrproject-rtos/zephyr/issues/23076 and suggested that the simplest change would be to remove the static from that variable, then people can solve this problem themselves.
Thanks for sharing your findings. Appreciated.
