Finding the heap low water mark when using nRFConnect/Zephyr

RE: Finding the heap low water mark when using nRFConnect/Zephyr

Simon — Fri, 20 Nov 2020 09:22:53 GMT

Thanks for sharing your findings. Appreciated.

RE: Finding the heap low water mark when using nRFConnect/Zephyr

RobMeades — Thu, 19 Nov 2020 13:54:39 GMT

FYI, since newlib is being used by Zephyr under the hood (at least on NRF53) it is almost possible to do this by calling the newlib mallinfo() function. The reason this is only "almost" is that mallinfo() is not the whole story: newlib calls sbrk() as required to give it heap memory from the ultimate heap, so the real number is mallinfo's fordblks plus what's left in sbrk(): without this the reported heap can end up going UP from it's original value, a "negative leak", which would cause any strict heap checking to fail.

Unfortunately, the variable that tracks the memory left inside sbrk(), heap_sz in lib/libc/newlib/libc-hooks.c is static so one can't get at it to do the free heap calculation properly.

I've noted this on https://github.com/zephyrproject-rtos/zephyr/issues/23076 and suggested that the simplest change would be to remove the static from that variable, then people can solve this problem themselves.

RE: Finding the heap low water mark when using nRFConnect/Zephyr

RobMeades — Thu, 05 Nov 2020 14:41:30 GMT

Will do.

RE: Finding the heap low water mark when using nRFConnect/Zephyr

Simon — Thu, 05 Nov 2020 14:35:37 GMT

Could you try to ask about this in the GitHub issue? If you don't get any answers, please ping me here.

RE: Finding the heap low water mark when using nRFConnect/Zephyr

RobMeades — Thu, 05 Nov 2020 10:54:26 GMT

Agree that largest contiguous region is probably a better measure, also saw your colleague "bjda" post that code on the Github thread, thanks.

The remaining question, then, is how Zephyr uses the heap. We are, of course, going to be calling malloc()/free() as any C code would, does Zephyr wrap those functions somehow and do entirely it's own thing with the heap memory, or does it work in parallel with them and sit atop malloc()/free() itself? If the former then your colleague's solution is good, if the latter it won't help because it's only half the story.

Or do you suggest I take this conversation up with your colleague on the Github issue thread?

RE: Finding the heap low water mark when using nRFConnect/Zephyr

Simon — Tue, 03 Nov 2020 08:31:34 GMT

A colleague put some work into this, check it out here: https://github.com/zephyrproject-rtos/zephyr/pull/29735.

The approach I mentioned earlier, by filling the heap and looking at it, may be misleading. The minimum largest contiguous region is probably the best "low water mark", since it will react to fragmentation of the heap.

Best regards,

Simon

RE: Finding the heap low water mark when using nRFConnect/Zephyr

Simon — Mon, 02 Nov 2020 12:24:17 GMT

I'll put off some time Wednesday to look into this.

Best regards,

Simon

RE: Finding the heap low water mark when using nRFConnect/Zephyr

RobMeades — Mon, 02 Nov 2020 11:20:02 GMT

We have stack sorted (we call k_thread_stack_space_get()), it's heap that we can't measure.

I had thought that it would be possible to do a maximum heap extent measurement by simply monitoring newlib's calls to sbrk(), since newlib will not ask for more space until it needs it. But it might be that newlib calls for more space irrespective of whether there might be a free'd block it could use, deliberating putting off the freeing process as an expensive operation.

So yes, if you were able to devise a mechanism to monitor heap I would be very interested.

RE: Finding the heap low water mark when using nRFConnect/Zephyr

Simon — Mon, 02 Nov 2020 10:37:44 GMT

I got some answers internally. It does not seem like there is something built-in for "heap painting". However, for stack there are, check out the config CONFIG_INIT_STACKS.

In order to achieve this, you would need to initialize the area on init, and clear/reset every allocated/free'd blob. Do you know how to go about this? If not I can look into it and try to make it work.

Best regards,

Simon

RE: Finding the heap low water mark when using nRFConnect/Zephyr

RobMeades — Wed, 28 Oct 2020 09:41:19 GMT

FYI, just been pointed at this: https://github.com/zephyrproject-rtos/zephyr/issues/23076. Seems that heap monitoring is not yet implemented in Zephyr. But even if it was it doesn't answer the whole question because I guess Zephyr would likely sit on top of malloc() rather than below. How do I monitor the true heap usage on NRF53 using nRFConnect/.Zephyr?

RE: Finding the heap low water mark when using nRFConnect/Zephyr

RobMeades — Tue, 27 Oct 2020 09:21:52 GMT

No, no response from Zephyr to my questions either on their Slack channel or via their mailing list. Very odd: like being asked to buy a car without a fuel gauge. Can't be normal can it?

RE: Finding the heap low water mark when using nRFConnect/Zephyr

RobMeades — Sat, 24 Oct 2020 16:11:36 GMT

OK, have asked on the Zephyr Slack but I expect they'll bounce me back here for a "platform implementation" issue.

RE: Finding the heap low water mark when using nRFConnect/Zephyr

Simon — Fri, 23 Oct 2020 08:12:37 GMT

I'm not sure about this. I'll ask internally about it. You could also try to ask in any of the resources listed here.

Best regards,

Simon