MQTT to Google Cloud PubSub republishing solution design approach / pattern - google-cloud-platform

Context
I am designing a solution where I need to ingest sensor data from different types of constrained devices into Google Cloud PubSub.
Pre-Conditions
These constrained devices only MQTT and only support MQTT over TCP. They do not support MQTT over TLS and can not do any sort of certificate based connection.
I do have the flexibility of picking the MQTT broker, however.
Given the context, pre-conditions, kindly suggest an approach where I can republish messages from MQTT to Google PubSub.
Thanks in advance!

Pretty much all MQTT brokers support bridging to other brokers.
So the devices would connect to your broker using raw MQTT, it would then create a single TLS authenticated connection to the Google IoT Broker using MQTT over TLS and forward all messages over that connection.
This means there is no need for a separate client to connect to both brokers and forward the messages.

Related

SIM800 MQTT publish to Google IoT Core

I'm trying to publish some data to Google Cloud using the SIM800 module together with ESP32. Google IoT Core provides its documentation on connecting MQTT clients here. It says:
Associate the MQTT client with MQTT server certificates.
However, in the SIM800 manufacturer's documentation on AT commands usage (here), there is nothing about associating the tls certificate. Given information concern making simple connections providing only the host, username and the password.
How can I make a connection that would take the certificate into account?

Proxy in between device and Google IoT Core using MQTT?

I have a situation where I want to use Google IoT Core to support bi-directional communication between my devices and existing GCP stack. The trouble is, some of my devices cannot connect to GCP's MQTT bridge because they are blocked from reaching it directly. The communication must instead go through my own hosted server. In fact, some devices will not be allowed to trust traffic either inbound or outbound to anything but my own hosted server, and this is completely out of my control.
Basically all suggested solutions that I have found propose the use of MQTT over WebSockets. WebSockets consume too many system resources for the server I have available, and so MQTT proxy over WebSockets is extremely undesirable and likely is not even feasible for my use case. It also defeats the purpose of using a lightweight, low-bandwidth protocol like MQTT in the first place.
To make matters more complicated, Google IoT Core documentation explicitly says that it does not support bridging MQTT brokers with their MQTT bridge. So hosting my own MQTT server seems to be out of the question.
Is it even possible to create a proxy -- either forward or reverse -- for this use case that allows for native, encrypted, full-duplex MQTT traffic? If so, what would be the recommended way to achieve this?
If you have hybrid set-up, meaning you have on-premise servers and a cloud server and you want to bridge them using Google IoT by using MQTT.
You can try in this github link, upon checking this MQTT broker has been tested to Google IoT. Since Google IoT is not supporting 3rd paryt MQTT broker.

Amazon MQ RabbitMQ connection

I came across this article for migrating rabbitmq to amazon mq (rabbitmq).
https://aws.amazon.com/blogs/compute/migrating-message-driven-applications-to-amazon-mq-for-rabbitmq/
I see this sentence:
"In Amazon MQ for RabbitMQ, we only support the secure version of AMQP using TLS. The code snippet below demonstrates AMQPS connection using Pika library. Please note that we do not support peer verification on server side."
Does it mean its mandatory to always have a secure connection for amazon mq (rabbitmq) OR its simply saying that if we decide to go for secure connections we must use TLS
from the documentation Encryption in transit
By default, Amazon MQ brokers use the recommended TLS 1.2 to encrypt data. Amazon MQ does not currently support configuring brokers to use different TLS versions.
Key Concepts Security and authentication
Connections to Amazon MQ brokers use Transport Layer Security (TLS)
Supported wire-level protocols
You can access your brokers by using any programming language that ActiveMQ supports and by enabling TLS explicitly for the following protocols:
Infrastructure security in Amazon MQ
You use AWS published API calls to access Amazon MQ through the network. Clients must support Transport Layer Security (TLS) 1.0 or later. We recommend TLS 1.2 or later. Clients must also support cipher suites with perfect forward secrecy (PFS) such as Ephemeral Diffie-Hellman (DHE) or Elliptic Curve Ephemeral Diffie-Hellman (ECDHE). Most modern systems such as Java 7 and later support these modes.
You have to use TLS.

Choosing AWS service for MQTT broker

I need to build IOT MQTT broker that should work on secure MQTT protocol. I also need to manage users that connects to this service and manage subscription access control. Idon't need MQTT via web socket.
At first glance I was planning to use EC2 service in order to create Ubuntu virtual machine and install Mosquitto service in it. But later I found Internet of Things section that contains set of services.
Is it possible to construct MQTT service according my requirements by using Internet of Things. By choosing Internet of Things I hope to get more specialized functionality.
You can use AWS IoT for this instead, they have a managed MQTT endpoint that you can add 'things' to it.
https://docs.aws.amazon.com/iot/latest/developerguide/mqtt.html
You'll be able to easily connect the endpoint to other services as this is part of their cloud solutions.
https://docs.aws.amazon.com/iot/latest/developerguide/iot-gs.html

Is AWS IoT Device Gateway an MQTT broker?

I am trying to understand Amazon AWS IoT and I see there is a Device Gateway connected between Sensors/Actuators and AWS (Rules Engine). I understand Sensors/Actuators are MQTT Clients which can Publish/Subscribe to the topics.
Device Gateway also supports Publish/Subscribe pattern is that means
Device Gateway is also a MQTT Client ?
Then where is the MQTT broker?
OR Device Gateway is Broker + communication interface to AWS-IoTservices?
I have attached an Image Amazon presentation on AWS IoT for the reference:
The gateway is a MQTT broker, but not a fully compliant one. Some of the differences are:
no retained message support
no QOS2 support
no message persistence
Full details can be found here http://docs.aws.amazon.com/iot/latest/developerguide/protocols.html