I am new to Google Clout Iot core. I just tried the iot core, registered the device and published some data on a topic.
Now i am wondering is only Google functions are used to get/subscribe data from it? or can i also used the data published from a device directly into my applications e.g. node.js or python? like it is usually subscribed through a normal MQTT topic? I dont need a serverless architecture i want to use the data from google iot into my server and store/use it from there..
thanks a lot!
When a message arrives over MQTT or HTTP from a device over IoT Core, that message is published on a GCP Pub/Sub topic using the GCP Pub/Sub infrastructure. What this means is that you can develop a consuming application however you like as long as that application consumes the message from Pub/Sub.
The diagram shown here from the documentation is a good reference.
GCP exposes the API for Pub/Sub to be a subscriber. There are language bindings for a variety of languages.
Cloud Functions is typically seen as a serverless consumer of Pub/Sub messages as it scales to zero and can scale up if there is bursty load. However, there is nothing to prevent you from writing Compute Engine hosted applications or Container hosted applications which can act as subscribers.
References:
Cloud Pub/Sub documentation
Cloud Pub/Sub Client Libraries
I think your question is more on the architectural side of things. You can use IoT Core with something like Pub/Sub and write the data to Cloud Storage in order to later consume it with an application.
I would recommend you to check some of the diagrams from this page if you are interested in knowing which products you could use to best fit your needs.
I hope you find this useful!
Related
I am publishing ESP data to Google Cloud IOT via MQTT to a particular topic. But If I have many devices data, then is it necessary to create different topic for each device? If Yes, then how can I trigger a single cloud function for multiple topics.
I couldn't find it anywhere on the Google Cloud Platform Documentation. So please help me!.
No, the message contain the attributes of your device. Like this, you can know the context of the message and process all the messages by the same Cloud Functions.
In my project multiple IoT devices need to publish to a single topic and cloud functions listening to that topic need to process data published by IoT devices. Multiple devices should be able to publish at the same time.A MQTT broker logic is needed. But in Google IoT Core "The MQTT bridge allows only a single connection for a given device ID. If a second device connects with the ID of a device that is already connected, the older device connection will be closed automatically.".
I am new to google cloud IoT and trying to understand basic concepts. I have looked into gateways but I am not sure that gateways work as broker. Can multiple devices publish through a gateway to a same single topic? or should I implement/add a MQTT broker(mosquitto I guess) myself or is there an another way to use google Cloud IoT in my case? Could you guys direct me to the right direction. Opening sub/pub topics for each device doesn't sound right because in my project there can be thousands of IoT devices.
Let all the devices in the registry publish messages to the common topic given to the registry.Then trigger a cloud function with this topic.
You can process the messages with cloud function based on the deviceId
I'm new to the GCP and currently have a microservice architecture using GKE and gRPC. The microservices are publishing events to Google Cloud Pub/Sub. My Web-UI is
using Google Cloud Endpoints to send requests to the Microservices.
I want to have a lot of live/push updates on the website (such as live updating user statistics etc) and now wonder how this is done best. Is it a bad practice to let the Web-UI subscribe to a topic in Google Cloud Pub/Sub? Are there other technologies in the GCP that may be better for this case?
Cloud Pub/Sub is intended for `torrents` use cases, where you have a lot of data communicated between relatively few publishers/subscribers.
Firebase Cloud Messaging might better fit the particular `trickles` use case you describe, where you are sending smaller updates to many, possibly transient, subscribers.
I'm developing an app that has chat between users. Does it make sense to use Pub/Sub for this purpose? I couldn't find in the documentation as a possible use case. Are there any good reasons why it's not a good fit?
Google Cloud Pub/Sub would not be a good fit for communicating with end user devices in a chat product. Cloud Pub/Sub is designed for torrents: a relatively small number of long-lived streams that require high throughput. In a chat app, you need trickles: a very large number of ephemeral streams that have a low throughput.
Cloud Pub/Sub quotas only allow for 10,000 topics project and 10,000 subscriptions per topic or per project. When setting up a new streaming pipeline, one generally creates the topic and its subscriptions once at the beginning and then publishes and subscribes from that point on. Therefore, Cloud Pub/Sub is a better fit for server-to-server communication or for large-scale streaming event ingestion, often in connection with Dataflow and/or BigQuery.
The permission model for Cloud Pub/Sub doesn't make it a good choice for a chat app, either. Permission to write to or read from a topic or subscription requires the user to be authenticated. Authenticating each individual user with permissions in your project is not really feasible, so you are left with the option of using common credentials for all of your users, which could be insecure.
That's not to say that Cloud Pub/Sub couldn't be part of a chat service, it just wouldn't be used to deliver messages to specific users. In fact, Cloud Pub/Sub can be used as the transport mechanism for messages sent to Hangouts Chat Bots. In this situation, though, the response to the user goes directly to the Hangouts Chat API, not back through Pub/Sub. Therefore, if you were implementing a chat service, you could potentially have your users send messages to a frontend server you design that then publishes those messages to Cloud Pub/Sub. Another server you build could be a subscriber to those messages and then deliver the appropriate messages to the appropriate individual users. In this situation, Cloud Pub/Sub is used for ingesting all of the messages from your frontend server (torrent) and delivery of messages to individual users falls to some other mechanism (trickles). The product in Google Cloud designed for this individual device delivery is Firebase Cloud Messaging,
Can Google Pub/Sub be used for chat messaging?
Yes. I would not use Pub/Sub for person-to-person chat messaging. Pub/Sub is designed for systems to de-couple themselves to support distributed systems.
Why not use Pub/Sub:
Permissions. You will need to create service account credentials and distribute this Json file to each subscriber. This is not secure.
Each chat person will need a subscription to a Pub/Sub Topic and then poll that subscription. This means one subscription per chat person. This is neither a good design nor inexpensive.
There are much better solutions for chat.
Pub/Sub for chat? No. As it is rightly mentioned above, Pub/sub has 2 major use cases, for data ingestion and for decoupling the system.
I would recommend you search a bit about web sockets if are still interested making a chat app.
I'm looking for something like AWS's device management functionality (https://aws.amazon.com/iot-device-management/) that exists on Google's Firebase service.
I found this: https://cloud.google.com/community/tutorials/cloud-iot-firestore-config
But I'm not sure if I can achieve the same functionality as with AWS IoT Device Management (I'm still new to Firebase).
Is there a Firebase equivalent to AWS IoT Device Management, specifically regarding onboarding and organization?
There is no equivalent in Firebase.
As has been mentioned, no, there's nothing directly in Firebase.
Have you looked at Cloud IoT Core? It has a lot of the features you're probably looking for in a device manager. From there, there are data pipeline tools (like Google Cloud Functions, or Cloud Dataflow) which can ferry the telemetry data from your devices into Firebase (or, potentially a better option, Firestore).
I did a session at the Google Cloud Next conference this year (2018) which talked about the end-to-end IoT application, and we published the code we built on stage which includes IoT device management, and used Cloud Functions to push the data into Firestore if you're interested. Link can be found here. The Cloud Functions that moved the data into Firestore are in the gcf folder. The device folder has the Python code we used on device to send telemetry up to IoT Core.