I have a need to poll for a close-to-real time reading from a serial device (using ESP32) from a web application. I am currently doing this using Particle Photons and the Particle Cloud API, and am wondering if there is a way to achieve similar using Google Cloud IoT.
From reading the documentation, it seems a common way to do this is via PubSub and then to publish to BigQuery via DataFlow or Firebase via Cloud Functions. However, to reduce pricing overhead, I am hoping to only trigger a data exchange(s) when the device receives an external request.
It looks like there is a way to send commands to the IoT device - am I on the right track with this? I can't seem to find the documentation here, but after receiving a command it would use PubSub to publish to a Topic, which can trigger a Cloud Function to update Firebase?
Lastly, it also looks like there is a way to do a GET request to the device's DeviceState, but this can only be updated once per second (which might also work, though it sounds like they generally discourage using state for this purpose).
If there is another low-latency, low-cost way to allow a client to poll for a real-time value from the IoT device that I've missed, please let me know. Thank you!
Espressif has integrated Google's Cloud IoT Device SDK which creates an authenticated bidirectional MQTT pipe between the device and IoT Core. As you've already discovered, you can send anything from the cloud to the device (it's called a "command" but it's just an MQTT payload so you can put almost anything you want in it) and vice versa (it's called "telemetry" but again it's just an MQTT payload). Once incoming messages from devices reach the cloud, pubsub can route them wherever you want. I don't know if I'd call it real-time, but latencies in a good WiFi network tend to be under a second.
Related
I am a beginner at cloud computing, and I'm hoping to get some guidance or advice as to how I can set up a cloud connected to IoT devices and a running application to control the behavior of these devices.
Firstly, there are 5 devices that have to connected via 3G or LTE because of the distance among the devices, so the way I am thinking of is connecting them to the internet using dynamic public ip addresses and using a dynamic DNS server. It seems like I should be using AWS-IoT service to manage these devices. How should I go about doing that, or is there a better approach? The devices all use MQTT and/or REST API.
The next step is to write an application and I was suggested to use AWS Lambda, am I heading towards the correct direction? How do I link the connected devices on AWS-IoT to AWS Lambda?
I know the question may sound vague but I am still new and exploring different solutions. Any guidance or recommendations for the right step forward is appreciated.
I assume your devices (or, one of them) has 64-bit CPU (x86 or Arm) that run Linux.
It's a kind of 70:30 balance where:
- 70% of the work needs to focus on building and testing edge-logic.
- 30% of the work on the rest (IoT Cloud, Lambda etc).
Here is what I suggest.
1/ Code your edge-logic first! (the piece of code that you want to execute ultimately on your devices).
2/ Test it on-the-edge by logging on to the devices (if you can) via SSH and running it.
3/ Once you have that done, 70% of the job is over.
4/ Rest 30% is to complete the jigsaw in cloud. Best place to start: Lambda and Greengrass.
5/ To summarize it all, you will create greengrass components on cloud, install AWS Greengrass Core software on your device, followed by deploying your configuration on your device over-the-air (OTA).
Now, you can use any MQTT client (or) biult-in MQTTTester of AWS IoT -> Test wizard to send a message to your topic to trigger your edge-logic on the device!
Good luck!
cheers,
ram
I am following this tutorial
https://codelabs.developers.google.com/codelabs/iotcore-heartrate/index.html?index=..%2F..index#0
Now i am able to send heart rate sensor data to Google Cloud BigQuery, Cloud storage etc, as described in the tutorial clearly and I am able to visualise it as well
But my next question is, how do we get access to data in real-time. For example, say if the heart rate data from Raspberry Pi (3B+) goes up over 75, i want to trigger and turn on the LED of the ESP32 that is connected at the receiving end.
In a nutshell, I want to do some actuation (like LED blinking as I told earlier) on ESP32, based on the sensor data from Raspberry Pi that goes to Google Cloud. I am only successful in sending, storing, and visualising sensor data in Google Cloud. Your help in enabling me to complete the actuation step is so valuable as I am pretty much clueless, how it can be done
Thanking you
There's a couple options here. The easiest to stand up, is Cloud Functions. The function can be triggered by Pub/Sub messages. It can also be authenticated with the IoT Core Admin SDK (via service accounts) to then send a configuration/command back down to the device you want to light up with the LED.
I wrote a blog post about setting up the Cloud to device communication piece:
https://medium.com/google-cloud/cloud-iot-step-by-step-cloud-to-device-communication-655a92d548ca
It covers how to setup the function to do it, although the function code itself in the example is an HTTP function, which means it triggers by hitting a URL endpoint instead of Pub/Sub, but that part's easy enough.
The big piece you'll need to investigate is pulling the Pub/Sub message in the function that triggered it. There's good docs on that here:
https://cloud.google.com/functions/docs/calling/pubsub
If you have super high throughput, then Cloud Functions can get expensive, and at that point you'd want to switch over to using something like Dataflow (https://cloud.google.com/dataflow/docs/). Then either having that job when it runs react to telemetry and hit an endpoint Function when it hits the target condition, or go through authenticating the job itself with the IoT Admin SDK. I haven't done that before, so I actually don't know how easy/hard that might be to do.
I'm trying to play with AWS IoT to communicate with multiple identical devices.
So far, so good, all my devices are connected to it, and the only difference between them could be a single device ID (like a mac address or a serial number)
Now I'd like to send a message to a single specific device using its device ID and I don't know if there is a good way to do that?
I could make each device to subscribe to a topic like /<DEVICE_ID>, however it doesn't seem like a good practice especially if I have thousands of devices.
Plus, AWS discourages it as stated in the AWS IoT documentation:
Note
We do not recommend using personally identifiable information in your
topics.
Is there a good way to handle this use case? Or is AWS IoT only useful to manage multiple devices at once?
Here is the best practice for creating an MQTT topic.
https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/
Talking about your specific case
Each device needs to have a unique identity to send a command to a particular device. In this case, you need to have device_id into your MQTT topic.
You can use the following pattern to send a message for a destination device
protocol_prefix / type_of_message / dest_id / message_id
hexaiot/controldevice/d12345/x123
Use wild card character at the time of device subscription to subscribe to the topic
Upon using the Google Cloud IoT Core platform, it seems to be built around the idea of sending configurations down to the device and receiving states back from it.
Google's own documentation suggests using that approach instead of building around sending commands down (as a config) and getting responses back (as a state).
However in the very end of the documentation they show an example of exactly that.
I am struggling to understand how does one support both approaches? I can see the benefit of how it was designed but I am also struggling to understand how would one be able to talk to the device using such an idiom of values and results as the config.
Has anybody implemented a command/response flow? Is it possible to subscribe to the state topic to retrieve the state of the device in my own application?
Edit based on clarifying comment below:
We've got a beta feature we're calling "Commands" which will do the reboot you're talking about. So the combination of config messages (for persistent configuration that you want to send a device on startup/connect to IoT Core) and Commands for fire and forget like a reboot message can do what your'e talking about. Current state is a bit trickier, in that you could either have a callback mechanism where you send a command to ask, and listen on the events/ channel for a response, or have the device report state (/state/ MQTT topic) and just ask IoT Core's admin SDK rather than the device.
Commands just went open beta, you should have access to it now. If you're using the gcloud SDK from command line, you'll need to do a gcloud components update and then gcloud beta iot devices --help will show the commands group. If you're using the console, when you drill down to a single device, you should now see "Send Command" next to "Update Configuration" on the top bar.
Old Answer: As a stab at answering, it sounds like rather than using the state topic, you could/should just use the standard /events/ topic and subscribe to the Pub/Sub topic the devices go into instead?
It really depends on the volume and number of devices we're talking about in terms of keeping that state machine in sync.
Without knowing what specifically you're implementing, I'd probably do something like send configs down, respond from device on the /events/ topic, and have a Cloud Function that tracks the Pub/Sub topic and updates something like a Firestore instance with the state of the devices, rather than using the /state/ topic. Especially if you're doing something in response directly to the state reporting of the device.
Send command to device
To send a command to a device, you will need to use the sendCommandToDevice API call.
Receive command from device
To receive a command from a device, subscribe to the /devices/<your-device-d>/commands/# topic.
Full examples will eventually be published to the Google Cloud IoT Core samples repos:
Java
NodeJS
Python
I am trying to send chunks of a data by using MQTT to aws IoT and by rule engine, the data will be streamed to a web app in real time (or near to real time )manner.
I am trying to find a way by which I can get some kind of acknowledgement. the role of acknowledgement is important for my use case as I am trying to send some kind of medical equipment data (lets say blood pressure data during operation).
thus, In a nut shell, I need a quickest way to transfer data(just like MQTT) between the device and AWS IoT which should have some kind acknowledgement mechanism.
Also I would like to add that ,'if suppose the device or web server could not get the message sent by MQTT due to some internet issues, then it will be lost right. I need to add some sort of mechanism by which the mqtt messages could be buffered and queued for some time so that once the device or web app comes online it can get the queued data. I also know that there is something called device shadow but we have thought using it differently. can you suggest about it ?'
I am sure that some of you have faced this problem and also found an alternate of MQTT in data transferring to AWS IoT.
Kindly share your thoughts.
Thanks.