I am building real-time dashboard using aws services, currently my application using MySQL database(RDS), which service and how would be designed real-time dashboard using Amazon web service, currently my approach is to use kinesis with redshift and connect my application via JDBC connector or use kinesis with s3 and use Athena to show real-time aggregators.
please help.
Thanks in advance.
Although the Amazon Kinesis Docs mention that Kinesis streams can be used to send updates to dashboards, Kinesis has no native mechanism to do this alone. For some very good security reasons it's unwise to allow clients (i.e. from a webpage you serve) to access backend services like Kinesis directly.
Instead, you'll want to set up your application layer (Java, in this case) to listen to the Kinesis streams and expose any relevant events to your client, storing any changes you need to keep track of in your database (RDS, in this case). For a real-time dashboard I'd recommend using something like WebSockets to send events in real time to your webpage from your server, as they're widely supported and easy to use. Heres a tutorial on how to implement WebSockets on GlassFish
Related
So I know this question probably gets asked a lot but this is kind of a specific one.
I am currently working as a webdev for my company and we are currently developing a webapp for our customers to use (energy management system and monitoring app).
My boss asked me to take on the DevOps role for this application (Which in this case is just basically setting up the aws services). The thing is I am pretty new to setting up AWS services and i am not sure where i should start. I am hoping some of the more experienced users could suggest a workflow after I describe what the app will be doing and the stacks we are using.
App/Product description:
Our app comes with an IoT module (ESP 32) that is placed into a customers Electrical Cabinet. We connect several devices to this module so we can read the incoming data.
The module sends this data to an MQTT protocol. After this we read the data from the MQTT protocol using telegraf. Telegraf sends this data to influxDB so we can use it in our application. All the devices (from each customer) data is saved to influxDB this way.
The application itself is written in NodeJS (backend) and VueJS/Nuxt (frontend). It's a simple dashboard application that shows the customer the specific data for his/her device. This authorization is done with Nuxt-auth on the frontend and Nest Js authentication on the backend. All the customer data is saved to a SQL database.
So this is a simple overview of the app. All technologies used are:
Telegraf
InfluxDB
NodeJS (NestJs)
VueJS (NuxtJs)
MySQL
IoT
MQTT protocol
What would be the best way to deploy this app using aws?
Currently our dev team was thinking of this:
AWS Fargate: Docking the application and using Fargate to deploy
AWS RDS: SQL database
InfluxDb
Telegraf
AWS IoT Core
The application should be online 24/7 and we are thinking our userbase will be around 500 users (recurring logins on the platform)
I have been looking for similar questions that match our needs but didn't really find any. If someone does find one please feel free to share them.
Should you need more info, I will be happy to supply more information.
This apparently is so simple question but I have been trying hard and came accross many questions and articles but none of them really answers this.
There are many ways to design the architecture after the log is ingested and transferred to AWS. We can use ES, Kinesis and other services.
My problem is how to transfer the application client logs to AWS securly, anywhere, could be S3, Lambda, Kindesis, ElasticSearch. We can surely have an endpoint exposed but that will be open. How to authenticate this to make sure malicious users don't spam our logs? what is the best way to ingest and transfer logs to AWS from mobile applicatios?
Creat an API that collect logs and ingest to kinesis expose it with API gateway use API keys aur authentication API to generate bearer token.
In application create functionality that make API call.
We did this in one of our application successfully
One way is to use Amplify SDK on the client application to log
securely.
Have a look at this link:
https://medium.com/swlh/add-monitoring-to-your-amplify-app-by-using-amplify-framework-d4c43b2bb84b
Amazon EventBridge enables developers to connect 3rd party event-driven applications with Amazon services. Amazon AppFlow offers event-driven integration with 3rd party apps as well.
What is the difference between the two services for the event-driven scenarios and when to use one over the other?
The AppFlow FAQ contains this info:
Amazon EventBridge enables developers to build event driven
applications that interact with SaaS applications and AWS services.
SaaS applications that have integrated with EventBridge emit events to
the customer’s event bus, which can then be routed to targets such as
Amazon EC2 instances or Lambda functions for processing. AppFlow
supports bi-directional transfer of data between SaaS applications and
AWS services that may be initiated by humans using a UI, a schedule,
or events - all with a point and click interface.
AppFlow is great for anyone who wants to connect together their applications without writing any code. EventBridge works with Step Functions and Lambda (and various other AWS services), and so is well suited for developers who need the extra flexibility that those services provide. Right now EventBridge is only one way, whereas AppFlow allows you to send data back to SaaS apps. The list of supported partners for each service is also different.
For some of AppFlow's partners, data is received via API polling (from the FAQ: "AppFlow is a fully managed API integration service that replaces custom connectors"), which is slightly different from EventBridge which receives data in an event-driven manner, where an event is sent via an HTTP call from the partner as soon as a change occurs. The pricing of the two services also differs: EventBridge charges $1/million events, and AppFlow charges $0.001 per flow (with an additional charge per GB of data processed).
I am currently working on a web portal for a foundation. Applicants for a grant will receive access data in advance independently of this portal. New applications will then be created and processed in the portal itself. Once an application is complete, it is sent off. Later the application will be approved or rejected.
There are a number of technical specifications on which I have no influence. The frontend should be implemented using Html+Javascript. The backend should use the Amazon Web Services (AWS). If there is a need to program something for the backend - then C# should be used.
I know how to implement the classic client-server solution. At the moment, however, AWS offers me an unmanageable set of services. And here I'm hoping for suggestions as to which of the services I should take a closer look at. Ideally, no complete 'server solution' should run on a virtual server. Instead, Lambda functions are mentioned again and again. So would Amazon RDS and AWS Lambda be a sensible and sufficient combination? Did I miss something?
Thank you very much for your suggestions.
One solution would be to use AWS S3 to server HTML, CSS, JS, Images and other static content. You could use AWS Lambda via AWS API Gateway to serve as a backend. AWS Lambda would then connect to AWS RDS or AWS DynamoDB if you would prefer a NoSQL solution.
Image taken from AWS Github repo
You can get a more detailed description of how to set this up at
https://github.com/aws-samples/aws-serverless-workshops/tree/master/WebApplication/
I'm trying to create push notifications for my mobile application.
Is it possible integrate Delphi FireMonkey with Amazon SNS services?
Thanks.
(Using Delphi 10 Seattle)
The Amazon APIs are REST based. You can simply use the INDY component library or maybe even the Rest Client that shipped recently with Delphi. Also, Delphi has built in classes for talking to many of the AMAZON services like S3. You can use those as an example.
Amazon Integrator from n/software makes it easy to access Amazon Web Services from Delphi
Easy-to-use components can be used to add, modify and delete objects stored on S3 or SimpleDB, add or remove messages from SQS, integrate with ASW e-commerce services, or control EC2 instances. As well as SNS
https://www.nsoftware.com/in/amazon/
Short answer is Yes.
Its possible to use Amazon Cloud API with Delphi. Extending Cloud API, Amazon Simple Notification Service (SNS) let you send notifications using different protocols including email, http and push notifications. (TAmazonSNS Service)
There are 3 provided classes to access Amazon Web Services with Cloud API:
Simple Storage Service (S3)
Simple Database Service (SimpleDB)
Simple Queue Service (SQS)
Also you can check Paweł Głowacki's Amazon Web Services Delphi CodeRage X demos from here.