Attachments (1) Download All
Nordic Case Info
  • Case ID: 321621
Options

aws_iot example without network offloading

GilDev

Hello,

I would like to know how to build the aws_iot example without network offloading to the modem. I need to make AWS connections through both the nRF9160’s LTE-M connection as well as through an Ethernet ENC424J600 controller (PHYTEC link board ETH or equivalent), only one of these two interfaces will be enabled at a time.

As I understand it, if I want to use more than one interface in my app, I have to disable sockets offloading (CONFIG_NET_SOCKETS_OFFLOAD=n), but doing that prevents enabling LTE connectivity (CONFIG_LTE_CONNECTIVITY=y). Hence my issues of having everything somehow use MbedTLS and being transparent to the AWS IoT library.

I have spent weeks trying to make the examples work independently and together (aws_iot, aws_iot_mqtt (which I cannot get to compile for the nRF9160DK because of build errors with TF-M), https_client and others) and cannot get anywhere close to where I need… I can make the dns_resolve work over Ethernet for example, by disabling the sockets offloading so that getaddrinfo() properly calls dns_get_addr_info() and such but putting everything together is a nightmare because of configuration incompatibilities with nRF Modem’s libraries…

I hope someone has any idea to help me here, thanks!

Parents
  • 0

    I’ve been experimenting a bit more with this sample and using only Ethernet as a first step, I’m now having errors when doing the TLS handshake. I attach the project (my Ethernet configuration is in the “prj-ethernet.overlay.conf" file), here is the log I’m getting with MbedTLS debug enabled:

    Increasing MBEDTLS_SSL_MAX_CONTENT_LEN to 6000 (from the default MTU size 1500) gives this:

    Is there an issue with my certs? They use the same format as in the https_client example for example, it works fine there. It seems I can’t enable CONFIG_MBEDTLS_PEM_CERTIFICATE_FORMAT because it required MBEDTLS_BUILTIN and a bunch of other stuff that again, conflicts with each other.

     aws-sample.zip

  • 0 in reply to GilDev

    Hey,

    it looks like there are some DNS resolve issues;

    Maybe it's not related to your problem, but you should check if CONFIG_NET_IPV6 is enabled in the config.

  • 0 in reply to Hakon

    The -100 means the DNS request is in progress (DNS_EAI_ALLDONE), the -103 means the DNS request completed (DNS_EAI_ALLDONE). All looks fine on that part, it’s the following that doesn’t work for some reason.

    I have successfully published to my device’s shadow through Ethernet now. But what I would like to do, is disable sockets offloading while using LTE-M as first asked in this thread. The aws-sample’s README says this:

    Instead of offloading the TLS sockets into the modem, you can use the Mbed TLS library from Zephyr.
    Using the Zephyr Mbed TLS, you can still use the offloaded sockets.

    What does this mean? I want to use the Mbed TLS library instead of offloading the TLS sockets into the modem. But if I disable CONFIG_NET_SOCKETS_OFFLOAD, CONFIG_LTE_CONNECTIVITY cannot be enabled.

  • +1 in reply to GilDev

    I have made some advancements on the matter. Turns out you can indeed keep the offloaded sockets enabled but still use the native implementation by using the SOCK_NATIVE flag when opening a socket. SOCK_NATIVE_TLS gives issues but that makes some sense considering the documentation:

    The thing is that this doesn’t seem to work for some functions like getaddrinfo() that is used kind of everywhere to resolve host names. Instead those only statically rely on the definition of CONFIG_NET_SOCKETS_OFFLOAD. I had to patch zsock_getaddrinfo() and zsock_freeaddrinfo() with a global “ethernet” flag like this to make it changeable at runtime:

    I also patched mqtt_client_tls_connect() so the type given to the socket is SOCK_NATIVE instead of SOCK_NATIVE_TLS, making the entire socket going through MbedTLS:

    And finally this flag is enabled when Ethernet is needed by patching connect_client() of the aws_iot library:

    This way, by changing the value of this ethernet flag on startup, I can up one interface or the other (LTE-M or Ethernet), start the DHCPv4 client on the Ethernet interface in case it is the selected one and the rest of the sample works properly by updating the AWS shadow. This is not made to have both interfaces up and working at the same time, in my case I just enable one or the other on startup depending on some local conditions (Ethernet cable plugged in or SIM card inserted).

    This is not a clean fix, it will probably require some adaptations to be made as I go along in my development (I’m not sure it will work as is with aws_fota for example). I’ll try to post the whole project with the SDK’s patch at some point.

    It seems crazy to me that I have to go through that much and alter the Zephyr networking libraries in this way, so I hope someone will be able to look at this and give some opinion, I may have missed something…

  • 0 in reply to GilDev

    I have reported this internally now.

  • 0 in reply to Hakon

    From what I have understood, what I did is a bit like  ’s solution mentioned here.

  • 0 in reply to GilDev

    Comment from modem team;

    This is not a nRF Connect SDK specific problem. I would advice seeking help on Zephyr Discord channel on how to route traffic between multiple network interfaces.

    First, I recommend the author should to study how Socket Offloading is implemented on Zephyr side and also study Network Interface APIs. In nRF91 platform, socket offloading is the interface we use to feed the traffic to the modem. It cannot be disabled as there is no alternative way. Socket offloading should be enabled and socket dispatcher might be the solution.

Reply Children
  • 0 in reply to Hakon

    Thanks for the insights. Indeed this seems to be a Zephyr limitation. Socket dispatcher sadly won’t help for things like getaddrinfo() or for Nordic’s libraries like AWS IoT because sockets are opened deep down in these ones.

    We are now facing issues with MBEDTLS using too much RAM of the nRF9160 despite following all the optimization guidelines, and no external RAM can be used to extend this. Maybe there aren’t any Nordic product that can suit our project right now, we may need to add a second microcontroller just to handle Ethernet communication, or to completely replace the nRF9160, we’ll see…

  • 0 in reply to GilDev

    Has there been any developments on these issues? I too am facing the issue of not being able to use things like getaddrinfo() for DNS hostname resolution.

    Really unfortunate there seems to be no way to take advantage of the power of Zephyr sockets and interface related libraries because the modem REQUIRES socket offloading. Not sure why the nRF Connect SDK and modem lib have to be setup this way. 

  • 0 in reply to dheadrick

    For us, it’s really the worst thing in our project. We had to implement ugly workarounds in Zephyr after weeks of work to get something that only partially works (we will have to do the same thing to get AWS FOTA updates through Ethernet soon) and leaves us at ~95 % usage on both flash and RAM, which causes crashes and lockups after a while (Nordic’s “AT modem” library complains about not being able to allocate memory after a few weeks, and our app gets stuck for 13h straight, no more explanation yet).

    Not being able to use the modem’s secure stack is annoying, along with the lack of examples using MbedTLS with the modem (and the AWS IoT sample for example). There’s no obligation in providing such things of course, but it’s just a big time hog if you’re not an expert of the platform and the products, and you initially think it’s a modular platform so you’ll be able to achieve what you want without too much hassle… It’s not easy, there are contradictory configuration options between Zephyr and the NCS libs and not much samples to help.

  • 0 in reply to GilDev

    Thank you very much GilDev for the prompt response! This is extremely dissapointing to hear... The nRF9151 was marketted to myself and my team as capable of allowing our project to use both Cellular and Ethernet interfaces in a single application. 

    I have been looking around everywhere trying to find a way to avoid doing ugly hacks and modifying the Zephyr code directly - as we do not have a large enough team to justify maintaining our own fork following the necessary modifications you mention. 

    I haven't even begun to dive into the TLS side of things to keep things simple, but now this has me worried and wondering if we will need to use a seperate microcontroller to interface with our ethernet controller, as  this seems simpler than having to hack apart Zephyr code directly. 

    We cannot be the first to come across this issue, and will likely not be the last. I hope sincerely the nordic team can help us find a better solution soon.

  • 0 in reply to dheadrick

    We have the exact same challenges as you, couldn’t justify to maintain a fork but here we are anyway, scared by any SDK update we do as we have to patch things manually again every time. If we were to do it again, we would probably have put another MCU to handle the Ethernet part indeed. We have had great success with STM32 and Ethernet controllers in the past, but it adds another completely different development environment and other problems along the way (even more for secure communications with AWS, I’m sure).

    If you have some time to try this out with the nRF9160 devkit and some Ethernet controller dev kits like we did, maybe it’s a good way to try things out and see if it’s a good fit for your application. I don’t know if things have changed since my issues…

Terms of service