I am currently working on an IoT device using AWS IoT core. I am new to working with IoT device. What is the standard/best way for determining whether the device is online and connected to the internet?
Thanks you!
Since you have been using AWS IoT Core, I would recommend that you stay in fully managed services provided by AWS IoT suite. No need to reinvent the wheel such as provisioning a separate database for a basic requirement of pretty much every IoT-enabled solution.
What I understand is that you want to monitor your IoT device fleets for state changes or failures in operation, and to trigger actions when such events occur. To address this challenge, I'd suggest using AWS IoT Events. It accepts inputs from many different IoT telemetry data sources including smart sensors, edge devices, management applications, and other AWS IoT services. You can easily push any telemetry data input to AWS IoT Events by using a standard API interface.
In specific to device heartbeat, please take a look at this sample detector model. A detector model simply represents your equipment or process. On the console, you can find some other pre-made detector model templates which you can customize based on your use-case.
One way to know if a device is online is to check for a heartbeat.
A device heartbeat is a small mqtt message to a topic that the device sends every 5 minutes.
In IoT Core, you would configure a rule that would update a Dynamodb table with a timestamp each time a message is sent to the heartbeat topic.
By checking this timestamp in Dynamodb, you can confirm if your device is currently online.
You can follow this Developer Guide to get connect disconnect events. it works on MQTT topics so we can use rules to trigger Lambda or other services.
Related
In AWS IOT we can make device subscribe to a topic. When a message is received on a topic, the device can be programmed to execute some code.
AWS IOT Jobs seems similar in that the device listens on the job and executes certain code when job is received.
How are AWS IOT Jobs different to Topic subscription?
The primary purpose of jobs is to notify devices of a software or
firmware update.
AWS IOT Job Doc
AWS IOT Events activities (like subscribing to a topic) would be the generic implementation for doing stuff when a device gets a message. IOT jobs are more of a managed workflow for doing a specific activity- like notifying devices of a firmware update and using CodeSigning.
Just want to add an important point to what #Bobshark wrote.
Yes, Amazon engineers implemented a set of endpoints to manage a whole job lifecycle on a single device and the process of gradually rolling out jobs over a fleet of devices.
However, IoT jobs are not tied down to using MQTT as the transport protocol. As the AWS docs [1] mention:
Devices can communicate with the AWS IoT Jobs service through these methods:
MQTT
HTTP Signature Version 4
HTTP TLS
My personal advice: Use jobs if you would have to implement your own update procedure (such as progress reporting, gradual rollouts, etc.) otherwise.
[1] https://docs.aws.amazon.com/iot/latest/developerguide/jobs-devices.html
What is the best practice to check if AWS IoT Core thing is still offline?
Being able to query the state of an AWS IoT thing will for many be an essential part of their application. Lucky AWS has a best practise on how to get lifecycle events here:
https://docs.aws.amazon.com/iot/latest/developerguide/life-cycle-events.html
In the documentation there is a section describing how to deal with lifecycle events an here they mention that after receiving a disconnect life cycle event you should wait some time before checking if the device is still offline.
When that message becomes available and is processed (by Lambda or another service), you can first check if the device is still offline before taking further action.
The question here is how do i check if the device is still offline?
The suggested infrastructure in the documentation is a IoT Rule subscribed to topic '$aws/events/presence/disconnected/clientId' will create a message on a SQS queue. The SQS queue will delay the message using the deliveryDelay attribute, before a lambda function will consume the message. It is in this lambda function that we should check if the device is still offline.
I could simply implement a dynamodb table with a device state and a timestamp which will be updated every time there is a connect or disconnect event without delay. But there must be a smarter way to determine if the device is still offline?
You already have the answer. Once the disconnect event is fired, you will assume the client is staying offline unless IoT Core sends update for the same client on the topic '$aws/events/presence/connected/clientId'.
Obviously you need to subscribe both topics.
Thing Indexing is a bad idea in most cases if you think about why you need IoT from beginning. It's only good for Amazon but won't give you any value.
You could use Thing Indexing. It has the option to index the connectivity status of the device (thingConnectivityIndexingMode).
I am using google IoT core and pubsub services for my IoT devices. I am publishing data using pubsub to the database. but I think its quite expensive to store every data into the database. I have some data like if the device is on or off and a configuration file which has some parameter which I need to process my IoT payload. Now I am not able to understand if configuration and state topic in IoT is expensive or not? and how long the data is stored in the config topic and is it feasible that whenever the parameter is changed in the config file it publish that data into config topic? and what if I publish my state of a device that if it is online or not every 3 seconds or more into the state topic?
You are mixing different things. There is Cloud IoT, where you have a device registry, with metadata, configuration and states. You also have PubSub topic in which you can publish message about IoT payload that can contain configuration data (I assume that is that you means in this sentence: "it publish that data into config topic").
In definitive it's simple.
All the management operations on Cloud IoT are free (device registration, configuration, metadata,...). There is no limitation and no duration limit. The only one which exists in the quotas for rate limit and configuration size.
The inbound and outbound traffic from and to the IoT devices is billed as described here
If you use PubSub for pushing your messages, Cloud Functions (or Cloud Run, or other compute option), a database (Cloud SQL or Datastore/Firestore), all these services are billed as usual, there is no relation with Cloud IoT service & billing. The constraints of each services are applied as a regular usage. For example, a PubSub message live up to 7 days (by default) in a subscription and until it hasn't acknowledged.
EDIT
Ok, got it, I took time to understood what you wanted to achieve.
The state is designed for getting the internal representation of the devices, but the current limitation doesn't allow you to update it automatically when you received message.
You have 2 solutions:
Either you can update your devices and send an update message only when its state changes (it's for this kind of use case that the feature is designed!)
Or, let the device published the messages every 3 seconds, but in the event PubSub topic. Get the events in a function which get the state list, get the first one (the most recent) and compare the value with the PubSub message. If different, update the state. This workflow also work with external database like Datastore or Firestore.
I am using AWS IoT. I want to throttle the connections and messages from a particular device.
( mainly to prevent costs )
Is there any way to achieve this?
AWS IoT device defender can be used for addressing security vulnerabilities, detect anamolies, etc.
But I wan to set up some threshold ( e.g. 100 messages per day), after which the messages from the same device should be rejected.
Configuring the behavior(rule) and threshold for AWS IoT Device Defender metrics generated by IoT devices is feasible. This shall help in invocation of appropriate action once the violation occurs. Behaviors(rules) convey the AWS IoT Device Defender on the normal device behavior using which it shall recognize when a device is doing something abnormal. A behavior is generally defined using a metric.
The below link can be a good starting point
https://aws.amazon.com/blogs/iot/use-aws-iot-device-defender-to-detect-statistical-anomalies-and-to-visualize-your-device-security-metrics/
AWS IoT Device Defender can detect abnormal device behavior and take actions. The below link configures two behaviors which can be modified for your requirement. First behavior - “msgReceive”, verifies that every five minutes the number of messages received from the device is less than 100. Second behavior - “bytesOut”, verifies that every five minutes the number of bytes sent out by the device is less than 10,000 (approximately 10 K).
https://aws.amazon.com/blogs/iot/detect-anomalies-connected-devices/?nc1=b_rp
Once detection and alerting is done, mitigation is feasible using AWS IoT Device Defender that helps in investigation of issues by providing contextual and historical information about the device such as device metadata, device statistics, and historical alerts for the device. You can also use AWS IoT Device Management tools to perform mitigation steps such as revoking permissions, rebooting a device, resetting factory defaults, or pushing security fixes.
With Rules engine, the AWS IoT rules are analyzed and actions are performed based on the MQTT topic stream a message is received on. The Rules Engine enables evaluation of inbound messages published into AWS IoT Core and transforms and delivers them to another device or a cloud service(AWS services like Lambda, S3, Kinesis, SQS, SNS and 3rd party external endpoints via lambda and SNS), based on business rules you define to process and transform data. This is the place where decisions can be made about a device’s messages (for example, message filtering, routing messages to other services, route messages to AWS endpoints and even a direct processing of messages). In this case, you may need to have Rules engine that blocks(message filtering) the device based on device id & threshold using your application of interest. So, here the rule can trigger a Lambda function that will compare the threshold value with the collected data and act upon accordingly like push notification to mobile as intimation to user via SNS service and rejecting the device.
You can author rules within the management console or write rules using a SQL-like syntax. Rules can also trigger the execution of your Java, Node.js or Python code in AWS Lambda, giving you maximum flexibility and power to process device data. The below link has related information on AWS IoT Rules https://docs.aws.amazon.com/iot/latest/developerguide/iot-rules.html
I'd like to use AWS IoT to manage a grid of devices. Data by device must be sent to a queue service (RabbitMQ) hosted on an EC2 instance that is the starting point for a real time control application. I read how to make a rule to write data to other Service: Here
However there isn't an example for EC2. Using the AWS IoT service, how can I connect to a service on EC2?
Edit:
I have a real time application developed with storm that consume data from RabbitMQ and puts the result of computation in another RabbitMQ queue. RabbitMQ and storm are on EC2. I have devices producing data and connected to IoT. Data produced by devices must be redirected to the queue on EC2 that is the starting point of my application.
I'm sorry if I was not clear.
The AWS IoT supports pushing the data directly to other AWS services. As you have probably figured out by now publishing to third party APIs isn't directly supported.
From the choices AWS offers Lambda, SQS, SNS and Kinesis would probably work best for you.
With Lambda you could directly forward the incoming message using the one of Rabbit MQs APIs.
With SQS you would put it into an AWS queue first and than poll this queue transfering it to RabbitMQ.
Kinesis would allow more sophisticated processing, but is probably too complex.
I suggest you program a Lamba with the programming language of your choice using one of the numerous RabbitMQ APIs.