Enabling use of Azure IoT Edge Transparent Gateway between device and Azure IoT Hub

[brown27]

I am using a nrf52840-based custom board with Nordic Connect SDK v2.3 and Openthread. A Raspberry Pi is my Thread Border Router and NAT64 translator (Thread is ipv6 while Azure IoT Hub is ipv4-only). The nrf52840-based device can successfully provision itself using the Azure Device Provisioning Service and then connect to its assigned Azure IoT Hub. I am now attempting to place an Azure IoT Edge "transparent gateway" between the NAT64 translator and Azure IoT Hub.

I have configured my Raspberry Pi as a transparent gateway using these instructions - its IoT Edge device name in IoT Hub is "my-pi-edge-device". I provisioned another device called "my-sensor" and assigned "my-pi-edge-device" as its parent device (see picture below).

However, "my-sensor" does not attempt to reach "my-pi-edge-device" and instead goes straight to "my-azure-iot-hub" after provisioning. How can I point "my-sensor" to use "my-pi-edge-device" as a transparent gateway?

What I am trying to do seems similar to this Microsoft sample, where GatewayHostName in the connectionString equals the IoT Edge device's hostname. However, I do not know how (or if) a GatewayHostName and connectionString are used in the Connect SDK.

[AHaug]

Hi

For clarification, could you explain/clarify the following:

1. nRF -> RPi OTBR+NAT64 -> Azure hub: works and you're able to see communication between the devices?
   
   Which of the following are you attempting? Is it
2. nRF -> RPi OTBR+NAT64 ->  Azure Transparent Gateway -> Azure hub 
   Or is it
3. nRF -> RPi w NAT 64 and Azure Transparent Gateway -> Azure hub

Our initial thoughts is that the translator is not properly set up, but if you're able to see communication between the devices in item 1, it should be working properly. Nonetheless we recommend you to have a look at this similar case https://devzone.nordicsemi.com/f/nordic-q-a/95695/access-coap-server-using-nat64 which links to some other relevant issues as well.

Also, have you had any luck on raising the issue regarding how to use the Transparent Gateway on any Microsoft developer support pages?

Kind regards,
Andreas

[brown27]

Hello,

#1 - nRF -> RPi OTBR+NAT64 -> Azure hub - Correct, this works successfully and I can see communication between the device and IoT Hub.

So far, I have been attempting #3. However, I plan to try #2 today and tomorrow with a different NAT64 that we know is more reliable. This will also help me avoid having a route and/or IP addresses set incorrectly when trying to run the OTBR, NAT64, and Azure transparent gateway on the same RasPi. 

I have been working with our organization's Microsoft contacts for guidance as well. This example from their SDK seems similar to what I am attempting. However, I am unable to find references to the GatewayHostName (or similar) in the Nordic SDK. It seems that the information returned from the Device Provisioning Service is stored in the "dps_reg_ctx" struct in the Connect SDK's azure_iot_hub_dps_private.h. I would expect this struct to also contain the gateway hostname (as a parallel to connectionString in Microsoft's sample), but could easily be mistaken.

I will be sure to read through the links you provided - thank you for the assistance and I will report back!

[AHaug]

brown27 said:

So far, I have been attempting #3. However, I plan to try #2 today and tomorrow with a different NAT64 that we know is more reliable. This will also help me avoid having a route and/or IP addresses set incorrectly when trying to run the OTBR, NAT64, and Azure transparent gateway on the same RasPi. 

Noted, let me know if you have any success with the different translator or if you find any other solutions to the issue.

brown27 said:

I have been working with our organization's Microsoft contacts for guidance as well. This example from their SDK seems similar to what I am attempting. However, I am unable to find references to the GatewayHostName (or similar) in the Nordic SDK. It seems that the information returned from the Device Provisioning Service is stored in the "dps_reg_ctx" struct in the Connect SDK's azure_iot_hub_dps_private.h. I would expect this struct to also contain the gateway hostname (as a parallel to connectionString in Microsoft's sample), but could easily be mistaken.

I've brought this question up to discussion with the Azure IoT team and I will return back to you when and if we land on something constructive that may help you find a solution to this issue

Kind regards,
Andreas

[AHaug]

Hi again,

I have any update to you from the discussion with the Azure team:

We don't have support for using transparent gateway, unfortunately. Under the hood we use https://github.com/Azure/azure-sdk-for-c, which is for embedded devices, while the sample the you refer to is in https://github.com/Azure/azure-iot-sdk-c/, which is a general C library for Azure IoT Hub. The latter has support for transparent gateway, while the embedded SDK that we use does not.

This might be some of the reasons for why you can't get the devices to communicate through the "proper path". Nonetheless we're very interested to know if you're able to make it work.

There is a GitHub issue in the embedded SDK requesting pretty much what the you want (to our understanding of what you want): Requesting a sample program for iotedge downstream device (ESPRESSIF ESP32) · Issue #2500 · Azure/azure-sdk-for-c (github.com)

But unfortunately, we would not recommend you to wait for any immediate solutions to this issue.

Kind regards,
Andreas

[brown27]

Thank you for the clarification and update. That helps me plan a work-around. If your developers and Azure contacts could comment on the work-around that I am considering below, it would be greatly appreciated.

See the "Troubleshoot the gateway connection" section in Microsoft's "How to connect a downstream device" documentation. Based on that documentation, it seems that the device gets the gateway domain name from Device Provisioning Service. Then the device resolves that into an IP address via DNS and connects. If I manually overwrite/replace the IP address returned by getaddrinfo() in the Connect SDK v2.3's mqtt_helper.c broker_init() function, I could point the device to my transparent gateway.

Here is an example of what I am talking about. I have modified the broker_init() function to support Openthread's use of ipv6 and the route to my NAT64. I have overwritten the returned IP to 192.168.1.1 as discussed above. I have also modified mqtt_helper.c's certificates_provision() elsewhere to trust the IoT Edge device's root CA. I know that the code will run and the device will try to connect to 192.168.1.1, but there might be other complications with the transparent gateway connection that I am not aware of.

static int broker_init(struct sockaddr_storage *broker,
		       struct mqtt_helper_conn_params *conn_params)
#if IS_ENABLED(CONFIG_AZURE_OPENTHREAD)
{
	int err;
    struct addrinfo *result;
    struct addrinfo hints = {.ai_family = AF_INET, .ai_socktype = SOCK_STREAM};

	err = getaddrinfo(conn_params->hostname.ptr, NULL, &hints, &result);
	if (err) {
		LOG_ERR("getaddrinfo() failed, error %d", err);
		return -err;
	}

	LOG_DBG("IPv4 addr byte 1: 0x%02x",*(result->ai_addr->data+2));
	LOG_DBG("IPv4 addr byte 2: 0x%02x",*(result->ai_addr->data+3));
	LOG_DBG("IPv4 addr byte 3: 0x%02x",*(result->ai_addr->data+4));
	LOG_DBG("IPv4 addr byte 4: 0x%02x",*(result->ai_addr->data+5));

	char buf1[50];
	net_addr_ntop(AF_INET,result->ai_addr->data+2,buf1,sizeof(buf1));
	LOG_DBG("IPv4 Address: %s", buf1);

	char buf[50];
	sprintf(buf,"64:ff9b::c0a8:0101"); //192.168.1.1
	/*
	sprintf(buf,"64:ff9b::%02x%02x:%02x%02x",
			*(result->ai_addr->data+2),
			*(result->ai_addr->data+3),
			*(result->ai_addr->data+4),
			*(result->ai_addr->data+5));
	*/

	LOG_ERR("Routed IPv6 Address: %s", buf);

	struct sockaddr_in6 *broker6 =
		((struct sockaddr_in6 *)broker);

	inet_pton(AF_INET6, buf, &broker6->sin6_addr);
	broker6->sin6_family = AF_INET6;
	broker6->sin6_port = htons(CONFIG_MQTT_HELPER_PORT);

    freeaddrinfo(result);

    return err;
}
#else
//Use the Connect SDK v2.3's normal broker_init() code

If this is successful, I am confident that I can manage our DNS servers and transparent gateway IP address to make this work in production.