Hi,
The Zephyr LwM2M client seems to leak pending objects that are used to track retransmissions. When this happens it enters a loop where the lwm2m_rd_client stops sending notifications to the app. This happens regularly between 5 - 26 hours under our network conditions.
See the log excerpt below.
BR / Björn
...
uart:~$ uart:~$ [10:45:12.855,499] <dbg> net_lwm2m_rd_client.sm_send_registration: registration sent [52.59.45.238]
uart:~$ [10:45:13.062,347] <err> net_lwm2m_engine: Unknown IP address family:0
uart:~$ [10:45:13.062,347] <dbg> net_lwm2m_engine.lwm2m_udp_receive: checking for reply from [unk]
uart:~$ [10:45:13.062,377] <inf> net_lwm2m_rd_client: Update callback (code:2.4)
uart:~$ [10:45:13.062,377] <dbg> app.rd_client_event: Registration update complete
uart:~$ [10:45:13.062,408] <dbg> app_lwm2m_temp.lwm2m_temp_set_value: Temp = 24.135307
uart:~$ [10:45:13.062,469] <dbg> net_lwm2m_engine.lwm2m_engine_set: path:3303/0/5700, value:0x20022594, len:8
uart:~$ [10:45:13.062,530] <dbg> net_lwm2m_engine.lwm2m_notify_observer: NOTIFY EVENT 3303/0/5700
uart:~$ [10:45:13.062,561] <dbg> net_lwm2m_engine.lwm2m_engine_get: path:3/0/13, buf:0x20033578, buflen:4
uart:~$ [10:45:13.062,652] <dbg> net_lwm2m_engine.lwm2m_engine_set: path:3303/0/5518, value:0x2003357c, len:4
uart:~$ [10:45:13.062,744] <dbg> net_lwm2m_engine.lwm2m_notify_observer: NOTIFY EVENT 3303/0/5518
uart:~$ [10:45:13.062,744] <dbg> app_lwm2m_humid.lwm2m_humid_set_value: Humid = 24.680086
uart:~$ [10:45:13.062,835] <dbg> net_lwm2m_engine.lwm2m_engine_set: path:3304/0/5700, value:0x20022570, len:8
uart:~$ [10:45:13.062,866] <dbg> net_lwm2m_engine.lwm2m_notify_observer: NOTIFY EVENT 3304/0/5700
uart:~$ [10:45:13.062,896] <dbg> net_lwm2m_engine.lwm2m_engine_get: path:3/0/13, buf:0x20033578, buflen:4
uart:~$ [10:45:13.062,988] <dbg> net_lwm2m_engine.lwm2m_engine_set: path:3304/0/5518, value:0x2003357c, len:4
uart:~$ [10:45:13.063,049] <dbg> net_lwm2m_engine.lwm2m_notify_observer: NOTIFY EVENT 3304/0/5518
uart:~$ [10:45:13.063,049] <dbg> app_lwm2m_airpress.lwm2m_airpress_set_value: AirPress = 101.458000
uart:~$ [10:45:13.063,140] <dbg> net_lwm2m_engine.lwm2m_engine_set: path:3315/0/5700, value:0x20022528, len:8
uart:~$ [10:45:13.063,171] <dbg> net_lwm2m_engine.lwm2m_notify_observer: NOTIFY EVENT 3315/0/5700
uart:~$ [10:45:13.063,201] <dbg> net_lwm2m_engine.lwm2m_engine_get: path:3/0/13, buf:0x20033578, buflen:4
uart:~$ [10:45:13.063,293] <dbg> net_lwm2m_engine.lwm2m_engine_set: path:3315/0/5518, value:0x2003357c, len:4
uart:~$ [10:45:13.063,354] <dbg> net_lwm2m_engine.lwm2m_notify_observer: NOTIFY EVENT 3315/0/5518
uart:~$ [10:45:13.063,354] <dbg> app_lwm2m_airqual.lwm2m_airqual_set_value: AirQual = 205.464889
uart:~$ [10:45:13.063,446] <dbg> net_lwm2m_engine.lwm2m_engine_set: path:3300/0/5700, value:0x2002254c, len:8
uart:~$ [10:45:13.063,476] <dbg> net_lwm2m_engine.lwm2m_notify_observer: NOTIFY EVENT 3300/0/5700
uart:~$ [10:45:13.063,507] <dbg> net_lwm2m_engine.lwm2m_engine_get: path:3/0/13, buf:0x20033578, buflen:4
uart:~$ [10:45:13.063,598] <dbg> net_lwm2m_engine.lwm2m_engine_set: path:3300/0/5518, value:0x2003357c, len:4
uart:~$ [10:45:13.063,659] <dbg> net_lwm2m_engine.lwm2m_notify_observer: NOTIFY EVENT 3300/0/5518
uart:~$ [10:45:13.063,659] <dbg> app_lwm2m_netw_conn.lwm2m_netw_conn_set_rssi: RSSI = -99
uart:~$ [10:45:13.063,690] <dbg> net_lwm2m_engine.lwm2m_engine_set: path:26243/0/7, value:0x2003357e, len:2
uart:~$ [10:45:13.063,720] <dbg> app_lwm2m_conn_mon.lwm2m_conn_mon_set_rssi: RSSI = -99
uart:~$ [10:45:13.063,720] <dbg> net_lwm2m_engine.lwm2m_engine_set: path:4/0/2, value:0x2003357f, len:1
uart:~$ [10:45:13.063,751] <inf> net_lwm2m_rd_client: Update Done
uart:~$ [10:45:13.063,812] <dbg> net_lwm2m_engine.lwm2m_udp_receive: reply 0x20022458 handled and removed
uart:~$ [10:45:13.063,873] <dbg> net_lwm2m_engine.generate_notify_message: [MANUAL] NOTIFY MSG START: 3303/0/0(2) token:'7bc20f00441e91b9' [52.59.45.238] ERR
uart:~$ [10:45:13.065,582] <dbg> net_lwm2m_engine.generate_notify_message: NOTIFY MSG: SENT
uart:~$ [10:45:13.065,643] <dbg> net_lwm2m_engine.generate_notify_message: [MANUAL] NOTIFY MSG START: 3304/0/0(2) token:'4fe344d8dbfa4d9b' [52.59.45.238] ERR
uart:~$ [10:45:13.067,016] <dbg> net_lwm2m_engine.generate_notify_message: NOTIFY MSG: SENT
uart:~$ [10:45:13.067,077] <dbg> net_lwm2m_engine.generate_notify_message: [MANUAL] NOTIFY MSG START: 3315/0/0(2) token:'513758d9ae6f9c0c' [52.59.45.238] ERR
uart:~$ [10:45:13.068,725] <dbg> net_lwm2m_engine.generate_notify_message: NOTIFY MSG: SENT
uart:~$ [10:45:13.068,786] <dbg> net_lwm2m_engine.generate_notify_message: [MANUAL] NOTIFY MSG START: 3300/0/0(2) token:'411b775b59e60126' [52.59.45.238] ERR
uart:~$ [10:45:13.070,373] <dbg> net_lwm2m_engine.generate_notify_message: NOTIFY MSG: SENT
uart:~$ [10:45:13.240,264] <err> net_lwm2m_engine: Unknown IP address family:0
uart:~$ [10:45:13.240,295] <dbg> net_lwm2m_engine.lwm2m_udp_receive: checking for reply from [unk]
uart:~$ [10:45:13.240,325] <dbg> net_lwm2m_engine.notify_message_reply_cb: NOTIFY ACK type:2 code:0.0 reply_token:'7bc20f00441e91b9'
uart:~$ [10:45:13.240,386] <dbg> net_lwm2m_engine.lwm2m_udp_receive: reply 0x20022458 handled and removed
uart:~$ [10:45:13.248,260] <err> net_lwm2m_engine: Unknown IP address family:0
uart:~$ [10:45:13.248,291] <dbg> net_lwm2m_engine.lwm2m_udp_receive: checking for reply from [unk]
uart:~$ [10:45:13.248,321] <dbg> net_lwm2m_engine.notify_message_reply_cb: NOTIFY ACK type:2 code:0.0 reply_token:'4fe344d8dbfa4d9b'
uart:~$ [10:45:13.248,382] <dbg> net_lwm2m_engine.lwm2m_udp_receive: reply 0x20022470 handled and removed
uart:~$ [10:45:13.256,256] <err> net_lwm2m_engine: Unknown IP address family:0
uart:~$ [10:45:13.256,286] <dbg> net_lwm2m_engine.lwm2m_udp_receive: checking for reply from [unk]
uart:~$ [10:45:13.256,317] <dbg> net_lwm2m_engine.notify_message_reply_cb: NOTIFY ACK type:2 code:0.0 reply_token:'513758d9ae6f9c0c'
uart:~$ [10:45:13.256,378] <dbg> net_lwm2m_engine.lwm2m_udp_receive: reply 0x20022488 handled and removed
uart:~$ [10:45:13.264,251] <err> net_lwm2m_engine: Unknown IP address family:0
uart:~$ [10:45:13.264,282] <dbg> net_lwm2m_engine.lwm2m_udp_receive: checking for reply from [unk]
uart:~$ [10:45:13.264,312] <dbg> net_lwm2m_engine.notify_message_reply_cb: NOTIFY ACK type:2 code:0.0 reply_token:'411b775b59e60126'
uart:~$ [10:45:13.264,373] <dbg> net_lwm2m_engine.lwm2m_udp_receive: reply 0x200224a0 handled and removed
uart:~$ uart:~$ uart:~$ [10:45:21.815,643] <dbg> net_lwm2m_engine.generate_notify_message: [AUTO] NOTIFY MSG START: 26241/0/1(3) token:'bf561777a04ddd30' [52.59.45.238] ERR
uart:~$ [10:45:21.825,378] <dbg> net_lwm2m_engine.generate_notify_message: NOTIFY MSG: SENT
uart:~$ [10:45:21.825,439] <dbg> net_lwm2m_engine.generate_notify_message: [AUTO] NOTIFY MSG START: 26243/0/7(3) token:'e3cd088612e339a7' [52.59.45.238] ERR
uart:~$ [10:45:21.826,690] <dbg> net_lwm2m_engine.generate_notify_message: NOTIFY MSG: SENT
uart:~$ [10:45:21.826,751] <dbg> net_lwm2m_engine.generate_notify_message: [AUTO] NOTIFY MSG START: 6/0/0(2) token:'9118f2ca4437c567' [52.59.45.238] ERR
uart:~$ [10:45:21.828,094] <dbg> net_lwm2m_engine.generate_notify_message: NOTIFY MSG: SENT
uart:~$ [10:45:21.828,186] <dbg> net_lwm2m_engine.generate_notify_message: [AUTO] NOTIFY MSG START: 3200/0/0(2) token:'9502f37c527f292e' [52.59.45.238] ERR
uart:~$ [10:45:21.829,498] <dbg> net_lwm2m_engine.generate_notify_message: NOTIFY MSG: SENT
uart:~$ [10:45:21.829,559] <dbg> net_lwm2m_engine.generate_notify_message: [AUTO] NOTIFY MSG START: 3347/0/0(2) token:'a9b45889907ea536' [52.59.45.238] ERR
uart:~$ [10:45:21.830,902] <dbg> net_lwm2m_engine.generate_notify_message: NOTIFY MSG: SENT
uart:~$ uart:~$ [10:45:21.849,121] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:21.849,243] <err> net_lwm2m_rd_client: Registration update err: -12
uart:~$ [10:45:22.349,548] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:22.349,609] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ uart:~$ [10:45:22.997,222] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:22.997,344] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:22.998,321] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:22.998,352] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ [10:45:23.643,341] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:23.643,463] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:23.644,470] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:23.644,500] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ uart:~$ [10:45:24.283,355] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:24.283,477] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:24.284,484] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:24.284,515] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ [10:45:24.923,370] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:24.923,461] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:24.924,468] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:24.924,530] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ uart:~$ [10:45:25.550,140] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:25.550,262] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:25.551,177] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:25.551,239] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ [10:45:26.200,225] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:26.200,317] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:26.201,293] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:26.201,354] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ uart:~$ [10:45:26.840,332] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:26.840,454] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:26.841,461] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:26.841,522] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ [10:45:27.470,153] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:27.470,245] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:27.471,191] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:27.471,252] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ uart:~$ [10:45:28.120,361] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:28.120,452] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:28.121,459] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:28.121,520] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ [10:45:28.757,263] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:28.757,354] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:28.758,239] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:28.758,300] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ uart:~$ [10:45:29.232,238] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:29.232,360] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:29.258,300] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:29.258,361] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ [10:45:29.872,161] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:29.872,253] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:29.873,199] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:29.873,260] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ uart:~$ [10:45:30.669,372] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
uart:~$ [10:45:30.669,494] <err> net_lwm2m_rd_client: Registration err: -12
uart:~$ [10:45:30.670,501] <inf> net_lwm2m_rd_client: RD Client started with endpoint 'ard-352656100441446' with client lifetime 15
uart:~$ [10:45:30.670,562] <dbg> net_lwm2m_engine.lwm2m_parse_peerinfo: Parse url: coaps://52.59.45.238:5684
uart:~$ [10:45:31.640,228] <err> net_lwm2m_engine: Unable to find a free pending to track retransmissions.
...
