LwM2M client: registration SOMETIMES fails; observers not set.

We're developing an LwM2M application based on the lwm2m client sample with the goal of sending data to ThingsBoard as our main telemetry platform.

The Application in based on the v2.4.2 SDK, running modem firmware mfw_nrf9160_1.3.4 on the nRF9160 SICA B1 on a custom board. Thingsboard has recently implemented a fix which allowed us to start testing on ThingsBoard instead of Coiote 1-2 weeks ago and we now have a lot more logging available to us (ThingsBoard transport log and wireshark for network traffic).

Registration generally works well; all attributes are exchanged and observers are set and later regularly updated. However, SOMETIMES (in ~15% of registrations) registration fails and the device stops sending responses to the server's requests and fails to set the defined observers.

There is no change in the logs of the client device between successful and unsuccessful attempts, except of course for the observers not being set, although outgoing and incoming coap packages are currently not being logged on our lwm2m client device (I haven't looked into this yet).

Is this a configuration issue or some compatibility problem with thingsboard?

Here is the .config file of the build followed by screenshots of the server and network logs. (CONFIG_LWM2M_ENGINE_MAX_MESSAGES, MAX_PENDING, MAX_REPLIES and MAX OBSERVER were only recently increased from 15 to 30, and I've since run into other problems, but I think it wasn't the issue?)

Here you can see the wireshark log of a failed attempt:

And here a successful attempt:

The content of the messages is identical, up to the point where the failed registration stops answering.

1. client device sends initial message with object versions etc. to the server

2. Server answers with the registration ID, confirming registration and follows up with read requests for all the specified attributes (here 26).

3a. In the failed attempt, 16 requests are answered, then the device stops reacting

3b. In the successful attempt, 18 requests are answered, before the server repeats a few of the unanswered requests, then after a back and forth of requesting 3-4 resources and them being answered, the 4 observe requests are sent, the client confirms, the server sends pmin and pmax, the client sends a 204 response and the registration is complete.

Server-side logs:

Unsuccessful attempt:

Last logs of a successful attempt:

I hope I have provided enough details and thank you for your help

Best Regards

Alan

Parents
  • Hello,

    do you have a modem trace from this?

  • Hello Hakon, thanks for looking into this issue

    I currently do not have a modem trace. I could  look into it and make one by the end of the week. Would this be helpful considering the other comments?

  • AlanIoT said:
    Would this be helpful considering the other comments?

    Yes, it would be helpful to identify the issue.

  • I managed to get the trace file. I had to power cycle the device 5-6 times until the registration failed. We currently have a watchdog in place that reboots the device if the registration fails (no observers set 30 seconds after connection to the lwm2m server).

    You should be able to Identify the boots from the file; the second to last boot is the one that failed.

    Thanks for any insight you can provide Slight smile

    trace-2024-04-12T11-18-44.792Z.mtrace

    edit: Actually, I am not sure if it worked.. I saw the file size increase on each reboot, but I can't seem to successfully open the file with wireshark. let me know if it's corrupted or something.

  • AlanIoT said:
    Actually, I am not sure if it worked.. I saw the file size increase on each reboot, but I can't seem to successfully open the file with wireshark. let me know if it's corrupted or something.

    Yes, this file seems to be broken somehow, as I am also unable to open the file in Cellular monitor or Wireshark. Try to follow the procedure described here.

Reply
  • AlanIoT said:
    Actually, I am not sure if it worked.. I saw the file size increase on each reboot, but I can't seem to successfully open the file with wireshark. let me know if it's corrupted or something.

    Yes, this file seems to be broken somehow, as I am also unable to open the file in Cellular monitor or Wireshark. Try to follow the procedure described here.

Children
  • Though, and not only in this case, it seems, that gathering the modem traces in "non-simple" cases isn't that easy, what's about some statements about the "grants" for that lwm2m-client?

    Is it a "supported product"? Or rather an example without the intention to be used for products?   

  • Achim Kraus said:
    Is it a "supported product"? Or rather an example without the intention to be used for products?   

    Yes, it is a "supported product" on Nordic equipment, as is all Nordic developed samples as far as I understand.

  • Okay, I'll start trying to get a working modem trace later today. I'll assume that if I'm unable to open the file, so will you.

  • Yes, if it doesn't open in Cellular Monitor or Wireshark, then it's no good.