I am new to AWS and have a question about an application I'm trying to write. I have a bunch of data that sits within Amazon RDS. On a periodic basis, I would like a small snippet of code to run against this data and in certain situations have notifications sent where appropriate. Of all the AWS services, what is the best architecture for this?
Thanks
You could use a simple cron job running on an EC2 instance. The cron job could run a script (PHP, Perl, whatever) to go fetch the data and then do something with it (notify people, generate reports etc)
Does that help?
See here for details on getting started with a Linux instance: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html
You could achieve the same results using a Windows machine and Scheduled Tasks. Here's the getting started guide for Windows instances: http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/EC2Win_GetStarted.html
You can use Scheduled Lambda service driven by CloudWatch events. It acts as a resilient cron.
Related
I am trying to create a simple C++ program that sends something like a real-time status message to AWS CloudWatch to inform that it is up and running, and the status goes offline when it's closed (real-time online/offline status). The C++ program will be installed at multiple users' computers, so there will be like a dashboard on CloudWatch. Is this even possible? I'm lost on AWS between Alarms/Logs/Metrics/Events..etc.
I also want to send some stats from each PC where the program is installed, like CPU usage for example, is it possible to make a dashboard on CloudWatch to monitor this as well? Am I free to create dashboard with whatever data I want? All the tutorials I found talk about integrating CloudWatch with other AWS services (Like Lambda and EC2) which isn't my case.
Thank you in advance.
The best way to monitor a process will be using AWS CloudWatch procstat plugin. First, create a CloudWatch configuration file with PID file location from EC2 and monitor the memory_rss parameter of the process. You can read here more.
For stats you can install CloudWatch Agent on each machine and collect necessary metrics. You can read here more.
I have a daily process that needs to digest a tremendous amount of data from two external sources. It normally requires around 28GB or RAM, and a decent amount of processing power. Due to this, an AWS Lambda won't work.
In the meantime, I've been running the process on an EC2 instance. In order to save resources, I've attempted to start the instance using a CloudWatch event. Since no event exists for "StartEC2," I'm kicking off a AWS Lambda instead, which in turn starts the EC2 isntance using Amazon support libraries.
All of this is extremely cumbersome, and I've been looking for a library or pattern that can do what I want. Essentially, I need to start an EC2 instance on a cron/event, deliver a unit of work to it (Shell Script, Java App, whatever), have it run it, then shutdown.
I'd love any suggestions for accomplishing this.
Look into AWS Systems Manager (SSM), you can create an Automation document that will launch the instance, run any custom scripts or tasks, and shut it down again when you're done. You can trigger the SSM Automation with a cron schedule via CloudWatch Events.
You may also want to consider AWS Batch for this type of workload.
Sorry i am very new to AWS and looking for the correct solution to implement following.
I need to build a job ( C# preferbly, since rest of the code is developed in c#) that will run nightly ( once in 24 hours hopefully ) to get some records from a postgress database table(s) and update the status based on a pre-defined condition.
What is the best way to implement this in AWS with postgress.
You will need to trigger an SQL Client (running somewhere) that will connect to the PostgreSQL database and run the desired queries.
This could be done from:
An Amazon EC2 instance
A computer anywhere on the Internet
An AWS Lambda function
If you have a Windows instance running somewhere, feel free to use it (be it on EC2 or elsewhere).
Alternatively, you could create an AWS Lambda function that connects to the database and runs the desired commands. The Lambda function can be assigned a schedule to run on a regular basis.
See: Schedule Expressions Using Rate or Cron - AWS Lambda
AWS Lambda functions can be written in a variety of languages, including .Net core.
I've setup my State Machine in the AWS console (I'm using it as a asynchronous workflow platform to trigger a few lambdas). Anyway, I want to trigger state machine with some input from a java service running on an EC2 instance. But I can't find anyway to do this. In fact it looks like the only triggering mechanism is via cloudwatch? Is that right? That doesn't seem right?
Thanks any help would help
In the AWS SDK for Java you have both AWSStepFunctionsClient.startExecution() and AWSStepFunctionsAsyncClient.startExecutionAsync()
I'm moving stuff from Azure to AWS, and the only thing I'm really gonna miss is the webjobs, where I can schedule command line jobs.
I know I can achieve somewhat the same with task scheduler or windows services, but I do also like the way webjobs shows logs and that stuff...
Do anybody know a tool like that, that can run windows command line apps on AWS?
Checkout AWS Lambda. It is a new service from AWS.
AWS Lambda, compute service that runs your code in response to events and automatically manages the compute resources for you, making it easy to build applications that respond quickly to new information.
Lambda vs WebJobs