Enabling use of Azure IoT Edge Transparent Gateway between device and Azure IoT Hub

[brown27]

I am using a nrf52840-based custom board with Nordic Connect SDK v2.3 and Openthread. A Raspberry Pi is my Thread Border Router and NAT64 translator (Thread is ipv6 while Azure IoT Hub is ipv4-only). The nrf52840-based device can successfully provision itself using the Azure Device Provisioning Service and then connect to its assigned Azure IoT Hub. I am now attempting to place an Azure IoT Edge "transparent gateway" between the NAT64 translator and Azure IoT Hub.

I have configured my Raspberry Pi as a transparent gateway using these instructions - its IoT Edge device name in IoT Hub is "my-pi-edge-device". I provisioned another device called "my-sensor" and assigned "my-pi-edge-device" as its parent device (see picture below).

However, "my-sensor" does not attempt to reach "my-pi-edge-device" and instead goes straight to "my-azure-iot-hub" after provisioning. How can I point "my-sensor" to use "my-pi-edge-device" as a transparent gateway?

What I am trying to do seems similar to this Microsoft sample, where GatewayHostName in the connectionString equals the IoT Edge device's hostname. However, I do not know how (or if) a GatewayHostName and connectionString are used in the Connect SDK.

[AHaug]

Hi

For clarification, could you explain/clarify the following:

1. nRF -> RPi OTBR+NAT64 -> Azure hub: works and you're able to see communication between the devices?
   
   Which of the following are you attempting? Is it
2. nRF -> RPi OTBR+NAT64 ->  Azure Transparent Gateway -> Azure hub 
   Or is it
3. nRF -> RPi w NAT 64 and Azure Transparent Gateway -> Azure hub

Our initial thoughts is that the translator is not properly set up, but if you're able to see communication between the devices in item 1, it should be working properly. Nonetheless we recommend you to have a look at this similar case https://devzone.nordicsemi.com/f/nordic-q-a/95695/access-coap-server-using-nat64 which links to some other relevant issues as well.

Also, have you had any luck on raising the issue regarding how to use the Transparent Gateway on any Microsoft developer support pages?

Kind regards,
Andreas

[brown27]

Hello,

#1 - nRF -> RPi OTBR+NAT64 -> Azure hub - Correct, this works successfully and I can see communication between the device and IoT Hub.

So far, I have been attempting #3. However, I plan to try #2 today and tomorrow with a different NAT64 that we know is more reliable. This will also help me avoid having a route and/or IP addresses set incorrectly when trying to run the OTBR, NAT64, and Azure transparent gateway on the same RasPi. 

I have been working with our organization's Microsoft contacts for guidance as well. This example from their SDK seems similar to what I am attempting. However, I am unable to find references to the GatewayHostName (or similar) in the Nordic SDK. It seems that the information returned from the Device Provisioning Service is stored in the "dps_reg_ctx" struct in the Connect SDK's azure_iot_hub_dps_private.h. I would expect this struct to also contain the gateway hostname (as a parallel to connectionString in Microsoft's sample), but could easily be mistaken.

I will be sure to read through the links you provided - thank you for the assistance and I will report back!

[brown27]

Thank you for the clarification and update. That helps me plan a work-around. If your developers and Azure contacts could comment on the work-around that I am considering below, it would be greatly appreciated.

See the "Troubleshoot the gateway connection" section in Microsoft's "How to connect a downstream device" documentation. Based on that documentation, it seems that the device gets the gateway domain name from Device Provisioning Service. Then the device resolves that into an IP address via DNS and connects. If I manually overwrite/replace the IP address returned by getaddrinfo() in the Connect SDK v2.3's mqtt_helper.c broker_init() function, I could point the device to my transparent gateway.

Here is an example of what I am talking about. I have modified the broker_init() function to support Openthread's use of ipv6 and the route to my NAT64. I have overwritten the returned IP to 192.168.1.1 as discussed above. I have also modified mqtt_helper.c's certificates_provision() elsewhere to trust the IoT Edge device's root CA. I know that the code will run and the device will try to connect to 192.168.1.1, but there might be other complications with the transparent gateway connection that I am not aware of.

static int broker_init(struct sockaddr_storage *broker,
		       struct mqtt_helper_conn_params *conn_params)
#if IS_ENABLED(CONFIG_AZURE_OPENTHREAD)
{
	int err;
    struct addrinfo *result;
    struct addrinfo hints = {.ai_family = AF_INET, .ai_socktype = SOCK_STREAM};

	err = getaddrinfo(conn_params->hostname.ptr, NULL, &hints, &result);
	if (err) {
		LOG_ERR("getaddrinfo() failed, error %d", err);
		return -err;
	}

	LOG_DBG("IPv4 addr byte 1: 0x%02x",*(result->ai_addr->data+2));
	LOG_DBG("IPv4 addr byte 2: 0x%02x",*(result->ai_addr->data+3));
	LOG_DBG("IPv4 addr byte 3: 0x%02x",*(result->ai_addr->data+4));
	LOG_DBG("IPv4 addr byte 4: 0x%02x",*(result->ai_addr->data+5));

	char buf1[50];
	net_addr_ntop(AF_INET,result->ai_addr->data+2,buf1,sizeof(buf1));
	LOG_DBG("IPv4 Address: %s", buf1);

	char buf[50];
	sprintf(buf,"64:ff9b::c0a8:0101"); //192.168.1.1
	/*
	sprintf(buf,"64:ff9b::%02x%02x:%02x%02x",
			*(result->ai_addr->data+2),
			*(result->ai_addr->data+3),
			*(result->ai_addr->data+4),
			*(result->ai_addr->data+5));
	*/

	LOG_ERR("Routed IPv6 Address: %s", buf);

	struct sockaddr_in6 *broker6 =
		((struct sockaddr_in6 *)broker);

	inet_pton(AF_INET6, buf, &broker6->sin6_addr);
	broker6->sin6_family = AF_INET6;
	broker6->sin6_port = htons(CONFIG_MQTT_HELPER_PORT);

    freeaddrinfo(result);

    return err;
}
#else
//Use the Connect SDK v2.3's normal broker_init() code

If this is successful, I am confident that I can manage our DNS servers and transparent gateway IP address to make this work in production.

[AHaug]

Hi,

This is just an update to let you know I've queried for any input if you're on the right track with your work around and I will let you know when I hear back from them. 

Kind regards,
Andreas

[AHaug]

Hi,

After going back and forth in the Azure team we've landed on that we can't really give you any other pointers than relatively vague speculation, but to us it seems like you're on the correct path with your design and approach. It seems that there is a lacking DNS64 support on the OTBR side (which should create the artificial address in the reply for NAT64), if that's the case, then generating the NAT64 IPv6 address from IPv4 on the app side seems ok. But it might also be a matter of configuring the DNS64 properly on the OTBR side.

Again to clarify, this information is just some input and pointers and hopefully it does not confuse more than it verifies the process you're already on.

Kind regards;
Andreas

[brown27]

Ok, thank you for the update. I was out last week but I am resuming my work today.

[brown27]

After extensive development and debugging, I was able to connect my device to an Azure IoT Edge transparent gateway instance running on a Raspberry Pi. This same Raspberry Pi is also running my border router and NAT64. Since the Azure library within the Nordic Connect SDK does not use a connection string, I instead pointed my device to the IoT Edge's IP address (see my "broker_init()" function in an earlier comment, to which I have made slight changes). This meets all of our needs and will be a good solution for us.