I'm trying to create a collaborative web application where multiple users can work together on various (shared) projects. So far I have a JavaScript client and one local jWebSocket server.
To remain scalable upon deployment, I thought of two options:
Option 1
I can use AWS IoT instead of multiple jWebSocket servers. Publishing changes of a project is easy, I would just need to publish to e.g. /project/{project-id}. But how would the traditional request-response mechanism work?
The Problem: EC2 instances handling requests would be reachable by publishing to distinct topics (e.g. /server/1). But when the JS client connects to AWS IoT, it does not know of any EC2 instance to send requests to. How could I assign each client to an instance/topic?
Option 2
Run jWebSocket servers on multiple EC2 instances behind an AWS Application Load Balancer. The balancer would simply assign each client to a server and the traditional request-response flow would not be a problem. But what about pushing changes?
The Problem: Because each server has its own set of connected clients, it can not push changes to clients connected to another server.
Remarks
Mixing jWebSocket to send requests to and AWS IoT to receive events from seems like a sloppy solution.
I assume I can programmatically adapt the IoT policies per cognito identity to allow/deny the subscription to specific projects.
Using AWS Lambda and relinquishing servers altogether is not an option due to the high latency introduced by Lambda (if you've made different experiences, please share).
Related posts
IoT request response protocol
Thanks for any thought you could give me on this issue.
I've got it. The first suggestion in this question pointed me into the right direction. The solution allows all clients to maintain a direct WebSocket connection to the server they originally connected to, without subscribing to specific topics.
It works as follows:
When a client connects to a server, the server subscribes to the client's channel
If a server needs to send a message to a client that is not connected, it publishes that message to the client's channel
(you guessed it) The server that is subscribed to the channel can then process the message on the first server's behalf
"Pusher" in the diagram describes this SaaS, but can of course be replaced by any other messaging service.
Related
I am working on a project wherein a mobile app (Flutter app) should create a unique identity (IOT thing) of my IOT device machine (on AWS) as soon as the user logins onto the app and also it should establish a connection (and subsequent communications) with the isolated IOT device after its configuration (Somewhat similar to Google Home configuration process).
The process starts with user logging onto the app (using AWS Amplify/Congnito to automatically add the user account in the cloud). Subsequently, the app needs to configure the IOT device (ESP32) and send Wifi credentials of home network by connecting to ESP's WiFi AP. However, since both the app and IOT device are isolated from each other (except during the configuration process), I don't have much idea on how exactly will mobile app connect (creating a unique IOT thing automatically) and communicate with the isolated IOT device over AWS IOT cloud.
I have worked with some of the AWS services like AWS Lambda, DynamoDB, AWS IOT for quite a while, hence I am aware of the basics. One of the possible solutions for the above maybe Fleet Provisioning by a Trusted User feature provided by AWS IOT (I did read it's documentation, but still confused). We can also use ESP32's Bluetooth feature for the app to scan nearby devices with which it needs to connect with, but I am not sure how will the connection establish through AWS cloud.
Since I am a beginner to AWS cloud services, it'd be great if someone can provide a possible solution for the above in detail. Also if possible, please provide a solution which would work incase of a large-scale deployment of the above project.
The parts you may be missing are missing are 1) MQTT messaging, which is the IoT message broker service provided by IoT Core, and 2) the need for an application program interface to handle creating and activating devices.
MQTT is a lightweight and widely adopted messaging protocol that is designed for constrained devices.
Your devices (things) will need to communicate back to AWS via MQTT messages, which are captured and processed by the IoT Rules service.
These rules typically trigger Lambda functions, which implement the process logic you need for your application.
See https://mqtt.org/ and https://docs.aws.amazon.com/iot/latest/developerguide/what-is-aws-iot.html
2} you will want to implement a simple API to handle device activation. The AWS API Gateway service makes it easy to implement APIs. For example, you can implement an endpoint for POST /device/ to create a device. That endpoint can trigger a Lambda handles device provisionsing.
I have developed an electronic device that connects to AWS IoT Core service through MQTT protocol. Now I can connect, publish and subscribe topics.
The device has some inputs (i.e. buttons) and outputs (i.e. relays).
Now I want to develop a mobile or web application to let an authorized user read inputs status and read/write outputs. More than one user could be authorized to control a single device.
What it's not clear to me, but I have no knowledge of backend technologies, is how to manage users, i.e. mobile/web app.
One strategy is to create a new Thing for each User, with its own certificate. The web/mobile app uses MQTT protocol as the electronic device. Topics can be used as a chat channel: the user asks for inputs status and the device answers with updated status.
Another strategy could be to create a HTTP API (maybe with AWS Gateway API) that web/mobile apps use. The API is programmed to publish and subscribe topics. The user should have a IAM access.
I know I could user Device Shadow, but I don't think it's a good soltion in my use case. The status of inputs can change frequently (maybe 1000 times a day), but the user wants to retrieve the status through the mobile/web app only sometimes (one a day or less frequently). It's seems it's useless to send so many messages to keep the shadow updated.
you can use to connect your WebApp with AWS IoT via MQTT(publish and subscribe methods are also available in the system).
Front end library documentation
Please Refer the link and see if this solves your problem https://medium.com/#serverlessguru/serverless-real-time-reactjs-app-aws-iot-mqtt-17d023954045.
One of the core features of AWS IOT is to provide bi-directional communication through the device gateway. Knowing that the implementation is private, can anyone conceptually explain how the gateway is able to push messages to a device on my local network? Does the service maintain a persistent connection to the device?
To add context, in my experience prior to AWS IOT, if I wanted to communicate with my embedded app (using HTTP) I would need to set up port-forwarding, so it's a mystery to me why device gateway could work without this step.
AWS IOT is implemented with MQTT, an MQTT connection is permanent TCP connection from the device to the gateway (broker), meaning that when there is a need to send a message to the device this connection is used.
Because the connection is initiated from the device to the gateway there is no need to set up any port forwarding (apart from that which is automatically handled by a home router doing NAT)
AWS IoT, uses the Pub/Sub pattern to provide communication between devices and message broker.
Publish–subscribe is a messaging pattern where senders of messages,
called publishers, do not program the messages to be sent directly to
specific receivers, called subscribers, but instead categorize
published messages into classes without knowledge of which
subscribers, if any, there may be. Similarly, subscribers express
interest in one or more classes and only receive messages that are of
interest, without knowledge of which publishers, if any, there are.
More about this pattern here: https://msdn.microsoft.com/en-us/library/ff649664.aspx
Brief Solution:
I am storing IOT Device location and Service Provider Cars location in Dynamo DB via AWS Kinesis.
I am mapping One User with a specific service providers Cars and dispatching the Car to user’s shared location.
Problem: I need to track these mapped (service provider car and user location) on real time on service provider dashboard.
1. Does DynamoDB offers any direct API to publish and track these location real time?
2. Do I need to expose these mapped location and track then on dashboard via AWS Kinesis? Does AWS Kinesis offer such APIs?
Any suggestions?
If you are using MQTT IoT Core from AWS you have other options.
Subscribe to MQTT topic directly via WebSockets by AWS JS library.
You can Use Cognito to manage users and to have some acces to AWS services, this way you can get credentials to subscribe to the topic in MQTT using WebSockets like this link describe.
https://docs.aws.amazon.com/iot/latest/developerguide/protocols.html#mqtt-ws
Having own Users manager and using a EC2 Instance.
You can create an EC2 instance and put an AppServer with NodeJs for example and use SocketIO to comunicate to the WebBrowser of users with your own manager, this way they can receive realtime info, whatever you want. You can then use an AWS MQTT IoT NodeJS client and subscribe to the topics you want and just send it by WebSockets of SocketIO server (All this in the same WebApp). This way you need to consider number of users and data in traffic. You can have topics by users for example topics like this "car/tracking/mycarId" where mycarId is an unique Id that identifies this user/car and you canonly subscribe to these you want at the same time, this way you are not subscribed to all topics of your cars and you are not receiving innecesary data and you dont need so much proccess.
I have implemented the second one but now Im migrating my realtime block to Cognito and MQTT via WebSockets.
Regards, Héctor
I am working on DJANGO server, creating a REST service. The purpose of the service is to send a push notification to the phone number mentioned in the request. I am using AWS SNS service for push notifications. I am establishing connection between my server and AWS every time there is a request to the server. My question is, can we establish a connection once the server is up, like opening a port as we do in a chat application? Or I should establish a connection every time?
I am using Boto package and below is my code
connection = SNSConnection(AWS_ACCESS_KEY,AWS_SECRET_KEY)
connection.publish(message=jsonMessage, subject=title, target_arn=endPoint, message_structure=structure)
The Client classes in the AWS SDK are just wrappers around other lower-level HTTP clients that interface with the HTTP REST methods of the AWS APIs. They are very light-weight so instantiating and destroying a Client instance on each HTTP request to your DJANGO app should be just fine -- you will incur little overhead. I wouldn't worry about shared state clients like you might utilize in Java or other multi-threaded application designs.