NRF52 Modules Stop Working after weeks

[Garelli]

We have developed a custom board using Raytac MDBT50Q with NRF52840. It has an external EEPROM and a RTC. We made an app that we use to program 4 daily alarms through BLE in our board, the nrf52 goes to Idle State when there is not an active connection and waits until the RTC wakes it up. When the nrf52 wakes up it activates a Motor for a while and then goes back to idle state. 

This process has been working perfectly for a month, with the board waking up exactly at the times it should. But suddenly yesterday the board did not activate the motor for any of the 4 daily alarms. I synched again the alarms and the current time (there was no problem with the communication), and again nothing happened (I heard a click of one of the 3 relays when the motor should have started, but nothing else). Until I reset the system, everything started working again. 

I ran some tests to check if the RTC was the problem, but it seems to be working fine.

I´m wondering if the GPIOTE Module could stop working after weeks? or any other module that could be affecting the process (TWI, timers, etc), what could be wrong to make this happen? Is it possible that the nrf52 had some RAM problems after some time? 

If anyone has seen this behavior, having some modules stop working after weeks, I would appreciate a comment. 

Thanks.

- Using SDK 15.2, Softdevice S140 6.1.0 and secure bootloader ble S140

[Jared]

Hi,

Could you try flashing the device with the GPIOTE example from the SDK? You should first change the macro GPIO_OUTPUT_PIN_NUMBER to one of the pins that you're experiencing the issue with. The state of the pin should toggle every 200 ms. If you're not observing this, then it would indicate that this is probably a hardware issue.

You can find the code at <InstallFolder>\examples\peripheral\gpiote

Best regards

Jared 

[Garelli]

Hi Jared, thanks for your answer. I have tried that, I have tested it for hours and I don´t get errors.

The program I made works perfectly fine for weeks, and suddenly it behaves wrong. I think I could have a memory leak somewhere, probably in a library. Do you know any method to detect memory leaks?

Best regards, 

Luis

[Jared]

Memory leak?

Are you using malloc? We don't recommend using dynamic memory allocation as problems such as fragmention often occur. 

[Garelli]

That could be the problem, yes I´m using malloc to create a linked list, and every time I clear a node I free the memory. Do you know if making a soft system reset with NVIC_SystemReset() would clear all the RAM and avoid the fragmentation?

[wpaul]

NVIC_SystemReset() will reboot the CPU completely. Anything not saved in flash will be discarded and your program will start over again, re-initialize all the hardware and reset the heap. So yes, this will "clear all RAM."

But periodic resets are not really the correct answer. You should identify the root cause and fix it.

Also, note that a memory leak and memory fragmentation are not the same thing. A memory leak would be due to a bug in your code (you think you're releasing a buffer after a malloc(), but you're not, or maybe you're using the wrong pointer when you free()). Memory fragmentation is more of a design flaw.

Fragmentation occurs when you allocate and release buffers of varying sizes. This can occur if the buffer size is somehow influenced by user input, which you can't easily predict, and you allocate and release buffers in unpredictable order. Say you allocate a big buffer, then a bunch of little buffers, then you release the big buffer, then you want to allocate a little buffer again. Releasing the big buffer created a chunk of free memory in the heap, but it's bordered by chunks of memory which are still allocated. To make the most efficient use of RAM, the allocator might decide to use memory from that free chunk to satisfy your current small buffer request, but that leaves you with a slightly smaller chunk of free memory left. If you don't try to allocate a chunk that exactly matches that size, that free chunk will remain unused. As this continues it can become harder and harder to satisfy large memory requests.

The recommended practice (especially for safety-critical systems) is to allocate lists of fixed-sized buffers. The size of the buffers is dictated by your application. Choose a value like 128 or 256, which you know will satisfy your largest possible allocation. Now allocate a list of these buffers, say 16 or 32. Each time you need a chunk of RAM to hold something, take it from the free list. When you're done with it, put it back. This avoids the fragmentation problem. (This is sometimes called a slab allocator.)

This is a little inefficient, because you might not need to use the entire chunk each time -- you might allocate a 256 byte chunk but only use 32 bytes of it. This is expected:  it's a design trade-off. You're trading a little bit of memory efficiency for reliability.

-Bill

[Garelli]

Hi Bill, Thank you very much for your great explanation, I´ll test it and check how my code behaves.