Multi instance backend subscribing to MQTT topic

[Rasmus K]

Hi, I am wondering about the best way to handle the case where my backend is subscribing to MQTT topics in NFR cloud, when I am running multiple instances of it, for example in Kubernetes.

The requirement is that one of the instances should get one message, not all - in a load balanced fashion.

I have read about shared subscriptions and tried to use it. Seems to be working, however I am facing an issue where all of my instances keeps disconnecting and reconnecting. I suspect this is because the MQTT broker only allows one connection per client id, and I am using the same for all my instances (btw I am using a device account to subscribe to the topics from my backend).

My initial thoughts for a solution is:

1. Some form of leader election mechanism, to give one of my instances a "leader" role, hence subscribing to the topics and the other instances are not (works but complicated)
2. Generating as many device accounts as I plan to have replicas and use different device account per replica (I don't particularly like this because I don't want to create another account when scaling up, and the logic for grabbing an account feels non trivial)

Any other thoughts for a solution, that might be more straight forward?

Best regards,

Rasmus

[Charlie]

Hi Rasmus,

We do not have much experience on multiple instances MQTT client implementation since our focus is on single MQTT client on IoT device mostly.

You may reach to MQTT or Kubernetes forum to get more professional advice.

Best regards,
Charlie

[Rasmus K]

Hi Charlie,

Thanks I understand. I was thinking there must be others wanting to subscribe to the MQTT topics from device accounts to process the data in their applications. And nowadays it's just so common to run multiple replicas/instances of your application.

Do you think we should keep this open for a while or close it?

Best regards,

Rasmus

[Charlie]

Hi Rasmus,

We can keep this ticket open to see if there are any other developers will share their solutions.

Feel free to give an update if you solve this challenge by yourself.

Best regards,

Charlie

[Rasmus K]

Yes, for now I am using a simple leader election mechanism (backed by postgresql) - which means one of my replicas will be elected as a leader, and the leader subscribes to the MQTT topics. In the future I might want to use compute across my replicas to receive MQTT messages, then I need to change this approach.

Best regards,

Rasmus

[Rasmus K]

Hi again Charlie,

I revised my solution to enable all my replicas to subscribe to MQTT topics of devices which are connected to my API and should be provided with live location updates. However I have encountered an issue, namely that I can only create one account device. As the MQTT broker prevents me from connecting with the same client ID, I need to use different ones (and different account devices, right?) so my thought was to have multiple, pooled account devices which my replicas can pick from when connecting to the MQTT broker. Which leads me to some questions:

1. Is there no way to create more account devices?
2. Is there another, recommended way for my backend to subscribe to live location events instead of subscribing to appropriate MQTT topics?

Thanks in advance,

Rasmus

[Charlie]

Have you thought about using EMQX which already supports K8s deployment?

Running MQTT on Kubernetes: A Practical Guide | EMQ (emqx.com)
EMQX is the leading open source MQTT broker. It provides a Docker Official Image which is available on Docker Hub and an open source Kubernetes operator, which supports the free and open source version of EMQX.

[Rasmus K]

Yes, we have thought about self hosting our own MQTT broker. However we wanted to explore a managed solution first. FOTA etc is supported out of the box for most managed solutions AFAIK.

Is it possible to provide answers to my questions? AWS IoT (which NRF cloud is based on AFAIK) does not have a limitation on how many "things" can be created (which I guess is the account device in NRF cloud). So another solution would be to use our own AWS IoT infra, bypassing NRF cloud, right?