Finding the heap low water mark when using nRFConnect/Zephyr

[RobMeades]

I need to monitor how much heap was ever left available, as a minimum, after I have run my CI test suite.  This is the true minimum heap ever remaining, so taking account of calls to malloc() as well as to Zephyr's memory management APIs, i.e. the true underlying heap low water mark.

I could do this on NRF52 as I had the implementation of _sbrk() that FreeRTOS called into (the Dave Nadler one).

What is the equivalent in the nRFConnect/Zephyr world on NRF53?

Rob

[Simon]

I'm not sure about this. I'll ask internally about it. You could also try to ask in any of the resources listed here.

Best regards,

Simon

[RobMeades]

FYI, just been pointed at this: https://github.com/zephyrproject-rtos/zephyr/issues/23076. Seems that heap monitoring is not yet implemented in Zephyr.  But even if it was it doesn't answer the whole question because I guess Zephyr would likely sit on top of malloc() rather than below.  How do I monitor the true heap usage on NRF53 using nRFConnect/.Zephyr?

[Simon]

I got some answers internally. It does not seem like there is something built-in for "heap painting". However, for stack there are, check out the config CONFIG_INIT_STACKS.

In order to achieve this, you would need to initialize the area on init, and clear/reset every allocated/free'd blob. Do you know how to go about this? If not I can look into it and try to make it work.

Best regards,

Simon

[RobMeades]

We have stack sorted (we call k_thread_stack_space_get()), it's heap that we can't measure.

I had thought that it would be possible to do a maximum heap extent measurement by simply monitoring newlib's calls to sbrk(), since newlib will not ask for more space until it needs it.  But it might be that newlib calls for more space irrespective of whether there might be a free'd block it could use, deliberating putting off the freeing process as an expensive operation.

So yes, if you were able to devise a mechanism to monitor heap I would be very interested.

[Simon]

I'll put off some time Wednesday to look into this.

Best regards,

Simon

[Simon]

A colleague put some work into this, check it out here: https://github.com/zephyrproject-rtos/zephyr/pull/29735.

The approach I mentioned earlier, by filling the heap and looking at it, may be misleading. The minimum largest contiguous region is probably the best "low water mark", since it will react to fragmentation of the heap.

Best regards,

Simon

[RobMeades]

Agree that largest contiguous region is probably a better measure, also saw your colleague "bjda" post that code on the Github thread, thanks.

The remaining question, then, is how Zephyr uses the heap.  We are, of course, going to be calling malloc()/free() as any C code would, does Zephyr wrap those functions somehow and do entirely it's own thing with the heap memory, or does it work in parallel with them and sit atop malloc()/free() itself?  If the former then your colleague's solution is good, if the latter it won't help because it's only half the story.

Or do you suggest I take this conversation up with your colleague on the Github issue thread?

[Simon]

Could you try to ask about this in the GitHub issue? If you don't get any answers, please ping me here.

[RobMeades]

Will do.