Commissioner not responding to join request

RE: Commissioner not responding to join request

Mary — Mon, 06 May 2024 20:27:59 GMT

It seems that there is a conversation between the Commissioner and the Leader before Joining is fully enabled. The signal that this has completed is a Thread state change with the OT_CHANGED_THREAD_NETDATA flag set.

So, I implemented a work around.

When the commissioner allows joining, start a timer (joinerstart_timer) with a 2 second timeout.

// Called in workqueue joining_work
static void activate_joining(struct k_work *item)
{
	ARG_UNUSED(item);

	struct openthread_context *ot_context = openthread_get_default_context();
	otError err;

	otInstance *instance = ot_context->instance;

	otCommissionerState state;
	if ((state = otCommissionerGetState(instance)) == OT_COMMISSIONER_STATE_ACTIVE)
	{
		openthread_api_mutex_lock(ot_context);
		err = otCommissionerAddJoiner(instance, NULL, pskd, JOINER_TIMEOUT); 
		openthread_api_mutex_unlock(ot_context);
		if (err != OT_ERROR_NONE)
		{
			printk("\r\notCommissionerAddJoiner() failed [%d]\r\n", err);
		}
		else
		{
			g_joining_allowed = true;

			printk("\r\nCommissioner joiner started for %d seconds...\r\n", JOINER_TIMEOUT);
			printk("Commissioner SessionID: [%d]\r\n\n", otCommissionerGetSessionId(instance));
			k_timer_start(&joinerstart_timer, K_MSEC(2000), K_MSEC(2000));
		}
	}
	else
	{
		printk("\r\nCommissioner state = %d. Restarting...", state);

		openthread_api_mutex_lock(ot_context);
		err = otCommissionerStart(ot_context->instance, commissioner_state_cb, commissioner_joiner_cb, ot_context);
		openthread_api_mutex_unlock(ot_context);
		if (err != OT_ERROR_NONE)
			printk("otCommissionerStart() failed [%d]\r\n", err);
		else
			g_start_joiner = true;		// trigger AddJoiner when "commissioner active" happens
	}
}

If the Thread state changes with OT_CHANGED_THREAD_NETDATA, stop the timer. Everything is ok.

static void on_thread_state_changed(otChangedFlags flags, struct openthread_context *ot_context, void *user_data)
{
	otError err;

	printk("\r\nThread state changed: role: %d  flags: 0x%x\r\n", otThreadGetDeviceRole(ot_context->instance), flags);

	if (flags & OT_CHANGED_THREAD_ROLE) 
	{
		switch (otThreadGetDeviceRole(ot_context->instance)) {
		case OT_DEVICE_ROLE_CHILD:
		case OT_DEVICE_ROLE_ROUTER:
		case OT_DEVICE_ROLE_LEADER:
			ot_is_connected = true;

			if (otCommissionerGetState(ot_context->instance) == OT_COMMISSIONER_STATE_DISABLED)
			{
				openthread_api_mutex_lock(ot_context);
				err = otCommissionerStart(ot_context->instance, commissioner_state_cb, commissioner_joiner_cb, ot_context);
				openthread_api_mutex_unlock(ot_context);
				if (err != OT_ERROR_NONE)
					printk("otCommissionerStart() failed [%d]\r\n", err);
			}
			break;

		case OT_DEVICE_ROLE_DISABLED:
		case OT_DEVICE_ROLE_DETACHED:
		default:
			ot_is_connected = false;
			break;

		} // endswitch: role
	} // endif: role changed

	if (flags & OT_CHANGED_THREAD_NETDATA)
	{
		k_timer_stop(&joinerstart_timer);
	}
}

But, if the timer expires first. Stop the commissioner.

void on_joinerstart_timeout(struct k_work *work)
{
	otError err;
	struct openthread_context *ot_context = openthread_get_default_context();

	printk("\r\nJoiner Start timed out. Restarting commissioner.");

	// Net Data Changed was not received, restart commissioner
	openthread_api_mutex_lock(ot_context);
	err = otCommissionerStop(ot_context->instance);
	openthread_api_mutex_unlock(ot_context);

	if (err != OT_ERROR_NONE)
		printk("otCommissionerStop() failed [%d]\r\n", err);
}

K_WORK_DEFINE(joinerstarttimout_work, on_joinerstart_timeout);

void joinerstart_timer_handler(struct k_timer *p_timer)
{
	k_timer_stop(&joinerstart_timer);
	k_work_submit(&joinerstarttimout_work);		// stops commissioner
}

The user will have to re-initiate joining (button press in our case), which will restart the Commissioner, then call activate_joining().

I still don't know why the process fails occasionally, but the workaround has succeeded several times.

Mary

RE: Commissioner not responding to join request

Mary — Wed, 03 Jan 2024 22:34:54 GMT

When joining fails, I do not see this line in the log:

[00:09:06.992,126] <inf> [I] JoinerRouter--: Joiner Router: start

The code that creates this log message is in

<sdk>/modules/lib/openthread/src/core/meshcop/joiner_router.cpp, line 67

void JoinerRouter::Start(void)
{
    VerifyOrExit(Get<Mle::MleRouter>().IsFullThreadDevice());

    if (Get<NetworkData::Leader>().IsJoiningEnabled())
    {
        uint16_t port = GetJoinerUdpPort();

        VerifyOrExit(!mSocket.IsBound());

        IgnoreError(mSocket.Open(&JoinerRouter::HandleUdpReceive, this));
        IgnoreError(mSocket.Bind(port));
        IgnoreError(Get<Ip6::Filter>().AddUnsecurePort(port));
        LogInfo("Joiner Router: start");
    }
    else
    {
        VerifyOrExit(mSocket.IsBound());

        IgnoreError(Get<Ip6::Filter>().RemoveUnsecurePort(mSocket.GetSockName().mPort));

        IgnoreError(mSocket.Close());
    }

exit:
    return;
}

My main question is why wasn't joining enabled?

How can I detect and correct this situation?

Mary

RE: Commissioner not responding to join request

Mary — Tue, 02 Jan 2024 14:28:30 GMT

As noted in my reply on Nov 9, the project was migrated to NCS 2.4.1

Mary

RE: Commissioner not responding to join request

Amanda Hsieh — Thu, 21 Dec 2023 13:34:00 GMT

Hi,

This function was introduced in Openthread API somewhere around November this year. Thus this function is not available at NCS 1.9.1. You will have to move to the latest version of NCS.

-Amanda H.

RE: Commissioner not responding to join request

Mary — Wed, 20 Dec 2023 19:16:30 GMT

I'm trying to look at the Commissioning data (which includes steering data) with this call:

otNetDataGetCommissioningDataset()

described here: https://openthread.io/reference/group/api-thread-general#otnetdatagetcommissioningdataset

In prj.conf, I have

CONFIG_OPENTHREAD_FTD=y

CONFIG_OPENTHREAD_NETDATA_PUBLISHER=y

And, in the compiler output I see:

- OT_NETDATA_PUBLISHER=ON --> OPENTHREAD_CONFIG_NETDATA_PUBLISHER_ENABLE=1

But, the end result of compilation is this error:

../src/coap_server.c:387: undefined reference to `otNetDataGetCommissioningDataset'

How do I include this function?

Mary

RE: Commissioner not responding to join request

Mary — Tue, 19 Dec 2023 21:47:19 GMT

Here is another log, when joining succeeded.

Joining is allowed at time [00:09:06.215,393], line 287

The Joiner Router is started at time [00:09:06.992,126], line 319. This event is missing in the failed example above.

The Discovery Request is at time [00:09:10.185,638, line 335. And the joining process follows.

devzone.nordicsemi.com/.../Joining_5F00_when_5F00_not_5F00_leader_5F00_ok.txt

It seems that

bool LeaderBase::IsJoiningEnabled(void) const om network_data_leader.cpp is returning false. I think the steering data is not getting set. Why would that be?

Mary

RE: Commissioner not responding to join request

Mary — Tue, 19 Dec 2023 20:55:10 GMT

It failed again today. Here is a log with more debug information.

CONFIG_OPENTHREAD_LOG_LEVEL_INFO=y

Joining is allowed at time [01:21:36.502,502], line 44

A Discovery request is received at time [01:21:53.936,798], line 84

and again at time [01:22:25.932,647] line 586

devzone.nordicsemi.com/.../joining_5F00_failed_5F00_security.txt

Do you see anything helpful?

Mary

RE: Commissioner not responding to join request

Mary — Fri, 01 Dec 2023 16:01:42 GMT

I added the mutex lock/unlock around the calls to otCommissionerAddJoiner() and any other calls that change something.

Some days later, the problem reappeared. I tested on two units running identical code. I verified that using nrfjprog --verify. One unit allowed a device to join, and the other didn't. I did a factory reset on the one that failed, and then joining succeeded.

Mary

RE: Commissioner not responding to join request

Amanda Hsieh — Tue, 14 Nov 2023 13:53:51 GMT

Hi,

I'm just now noticing that you don't use

void openthread_api_mutex_lock(struct openthread_context *ot_context);

and

void openthread_api_mutex_unlock(struct openthread_context *ot_context);

when you're calling the openthread API directly. Not sure if this is the cause of the problem. Please verify the problem still exists after fixing this.

-Amanda H.

RE: Commissioner not responding to join request

Mary — Thu, 09 Nov 2023 17:40:25 GMT

I have migrated the project to NCS 2.4.1, and I saw the same problem today.

The server/commissioner has been running overnight, ~16 hours.

Here is the Wireshark file:

devzone.nordicsemi.com/.../Client_5F00_join_5F00_failed.pcapng

Mary

RE: Commissioner not responding to join request

Amanda Hsieh — Mon, 28 Aug 2023 14:50:45 GMT

Hi,

Certifications are listed in the nRF52840 nRF Connect SDK Thread CIDs table.

Certification for NCS 1.9.X has been deprecated due to the security issue and we didn't release a patch for this version.

It is recommended to use the 2.4.1 release (certification we have already received, but infocenter update is pending).

-Amanda H.

RE: Commissioner not responding to join request

Mary — Thu, 24 Aug 2023 18:03:23 GMT

That is not our actual PSKd, I edited over it to keep ours confidential.

What NCS versions can be certified?

Mary

RE: Commissioner not responding to join request

Amanda Hsieh — Thu, 24 Aug 2023 14:14:57 GMT

Hi,

From what can be seen in logs, we're surprised the "OK" case passes - PSKd that is set there is invalid, as it can't contain characters "O" and "I". Other than this, the provided code looks fine and it's difficult to pinpoint the problem from the logs. Would it be possible for the client to share the wireshark pcap file?

One other thing to note is that this is an old version of NCS, and you won't be able to certify devices based on it (because of the Thread security vulnerability).

-Amanda H.

RE: Commissioner not responding to join request

Mary — Wed, 23 Aug 2023 19:09:14 GMT

Sorry for the delay. I increased the log level for openthread to 'Info'.

CONFIG_OPENTHREAD_LOG_LEVEL_INFO=y

I have been attempting joining for days, and it succeeds. I finally had one failure today, though it looks different from what I describe above. The joining process begins but does not complete. After the the Coap Server's commissioner joiner timed out, I restarted it and the joining process completed successfully.

I've attached logs for both success and failure.

devzone.nordicsemi.com/.../Joiner_5F00_log_5F00_ok.txt devzone.nordicsemi.com/.../Joiner_5F00_log_5F00_fail.txt

Mary

RE: Commissioner not responding to join request

Amanda Hsieh — Wed, 16 Aug 2023 12:50:40 GMT

Hi,

Would it be possible for you to collect more logs from both the commissioner and the joiner?

-Amanda H.

RE: Commissioner not responding to join request

Mary — Mon, 14 Aug 2023 14:34:02 GMT

I recompiled with this change to the prf.conf:

CONFIG_OPENTHREAD_DEBUG=y

#CONFIG_OPENTHREAD_DEBUG=n

CONFIG_OPENTHREAD_LOG_LEVEL_NOTE=y

When a client is trying to join, I saw this message on the Server:

W: [WARN]-MLE: Failed to process Discovery Request: Security

I see this message before I press the "Allow Joining" button, and again when it tries to join after I press the button.

otCommissionerAddJoiner(instance, NULL, pskd, JOINER_TIMEOUT); is not returning an error, but it doesn't seem like it worked.

Mry

RE: Commissioner not responding to join request

Amanda Hsieh — Fri, 11 Aug 2023 15:45:05 GMT

Hi,

I am looking into your case and will update it next week.

Regards,
Amanda H.