I currently have an ec2 instance and an rds instance running in AWS. I wish to know of the times that certain functions/routes/db changes were initiated/activated over the past couple of days. Is there any way to do this?
Related
Just wondering what is the best practice here.
I have these dev environments, dev/QA/UAT/ab/monkey and so on, which are used only during the daytime. We would like to save some cost here, by shutting them down during nighttime.
Each environment consists of frontend/API/caching/queueing/DB servers/Docker images.
Is using Terraform's create/destroy daily the right approach here?
First thing I noticed is the IP address change on removing EC2 instances. Every day on destroying the env, I will have to re-map the DNS. But this can be solved using EC2 elastic IP. But then I read somewhere:
if you’re using an EIP to just provide a public IP and not to rapidly and seamlessly distribute traffic in the event of an outage while keeping DNS records the same, it’s best to just use the AWS non-EIP pub IP and DNS records for pub access
Does AWS give a public DNS that doesn't go away if I shut down the EC2 instance?
Next is of course the data back-ups that I have to do. I have to back up all DBs, assets like images and videos, logs are not a concern since I will be pushing them off to another server using a log collector agent but all other data needs to be backed up before removal using Terraform destroy. I will also have tones of ECR images, I guess I need to back up them as well.
This feels like a lot of work. What is the best practice?
Just to add, almost all environments will run through-out the year.
You definitely could destroy these environments every day, depending on where your infrastructure as code lives, you could do this in a number of ways. For example if it's in a github repo, using github actions and workflows, you could create a task that runs a little while after you finish each day that would destroy everything. Other options would be gitlab which has it's own way of doing this, or something like Jenkins/TeamCity/Bamboo/CircleCI which could automate the job for you.
In theory you could set up another job that applies them again each weekday morning, so you can save money and you don't waste time each morning setting up your dev envs.
With regards to your DNS issues, if you are managing your DNS records with route53, you can add a resource for your records which point to the public IP of your instance (that would be an A record), or the public DNS of your instance (for example). Then when you create the new resources each morning the records will be updated to point at your new instances.
Simply shutting down the instances isn't always going to cut all of your costs, as you will still be paying for some resources like the EBS volumes and if you have elastic IPs which are not in use you get charged for that, load balancers generate charges even when not in use etc
I am trying to create an API that runs on AWS EC2 t2.micro. The problem I'm having is that my instance keeps shut down automatically every ~3 hours, which could be because of the "session time" of my AWS Educate account (screenshot attached)
Is there any way to keep my instance running constantly (for days and even months)?
I am using "tmux", which does seem to keep my API and the EC2 instance running even after my ssh connection is terminated, but the EC2 instance itself still shuts down automatically.
EDIT: If it is not possible to keep an EC2 instance of an AWS Educate account running constantly. Is there a way to start a new session automatically when the old instance's session has expired?
(Maybe using a script/using some tools offered by AWS? I'm new to AWS so I don't know if this is possible)
Sadly you can't change that. It is explicitly stated in AWS educate docs:
When your session ends, your resources will be “stopped.” You will be required to re-start your resources when you start a new session.
Instead of Using AWS Educate you can Create Regular AWS account which provides some services for free for one year. It includes the EC2 instance as well so you don't have to pay anything and you can run for months and year it will never gone down until you manually stop it.
I have a small single-instance deployment running on an EC2 instance which hosts both a web application and its database (MySQL). I've been looking to separate the deployment out into an EC2 instace for the web app and an RDS cluster for the database, and wanted to take advantage of the new AWS Savings Plans for both if possible.
My questions the are:
AWS Savings Plans seem to only apply to 'pure' compute EC2 instances, not to RDS instances as well. Can someone confirm or disprove this?
If Savings Plans did apply to RDS instances, is there a reason to not use them, and instead just use an Instance Reservation?
Since August 2020, AWS Savings Plans includes:
Amazon EC2
AWS Lambda
AWS Fargate
They do not apply to Amazon RDS db instances. For those, you can continue to use Amazon RDS Reserved Instances.
I want to clarify that even though Savings Plans do not cover RDS instances, they do cover EC2 instances that are part of EMR, ECS and EKS Clusters. Based on this link:
"Both plan types apply to EC2 instances that are a part of Amazon EMR, Amazon EKS, and Amazon ECS clusters. Amazon EKS charges will not be covered by Savings Plans, but the underlying EC2 instances will be. "
Also, Compute Savings Plans also apply to your Fargate and Lambda usage.
We moved to RDS from EC2 instances running self installed MySQL years ago. For me, at has been great. All of the RDS features work flawlessly, point and click, without the mundane work of spinning up, replicating, backing up, and failing over databases. It simply works great. Use reserved instances if you plan on keeping for at least a year. At 30% savings the cost is awash even if you bail on the server after about 9 months and don't use the entire year. Plus you can sell the unused remaining on the marketplace.
Downsides?
You do NOT get command line OS access to the MySQL server. You get an admin login to mySQL. The only way to manage it is through the AWS UI and the mysql client command line or managing client (like MySQL Workbench or Heidi).
You may want to run a mysqldump script on a separate EC2 to dump databases separately/additionally. AWS does SNAPSHOTS which require an entire restore of a sandbox server just to get a single table someone botched up, for example. I go to the MySQLdump files all the time. Never have needed the SNAPSHOT unless I am spinning up a sandbox copy of the entire instance for some reason.
In a nutshell, mySQL on RDS is great.
One other side note. We migrated an app using MySQL5.7 to Aurora MySQL with absolutely zero issues. Complete drop-in replacement (in our case).
I have approximately 40 Windows EC2 instances running at the moment. This number will start to grow substantially in the next few months. Each one is a t2.small Windows 2016 Server instance. Cost is starting to become an issue as the number increases. Each instance has its own Elastic IP address because when user Tom wants to access his machine he will use the DNS tom.mydomain.com.
tom.mydomain.com is registered in a Route53 hosted zone pointing to Elastic IP 22.33.44.55 which has been associated with Tom's EC2 instance.
Problem is that Tom only needs to use his machine 4 hours per day. When not using it he simply shuts the machine down. But... An Elastic IP that is pointing to a stopped instance costs almost as much per hour as a t1.micro instance in a running state.
So what I want to do is when Tom logs into AWS console and starts his EC2 instance, it will automatically register itself with Route53 against the DNS "tom.mydomain.com".
In short I want to do away with the need for Elastic IPs which are fast becoming a very substantial cost.
The tutorial Auto-Register EC2 Instance in AWS Route 53
looks like it does exactly what I want to do. The problem is the scripting is for Linux. I want to get it working for Windows. I have everything done down to step 6 in the tutorial but am stuck there. Any one get something similar to this working on Windows?
I would recommend:
Create a web-based front-end where your users can authenticate and request access to their Amazon EC2 instance
You could use Amazon Cognito for authentication and DynamoDB for data storage
Once the user authenticates, the service can:
Start their EC2 instance (if it was previously stopped)
Associate the random public IP address to the customer's domain name
Tell the user that the instance is now available
Users login to the instance and perform their work function
You then have some mechanism (I'm not sure what) that detects that they no longer need the instance, and then Stops the instance to save costs
The above process avoids assigning IAM credentials to your users. While IAM credentials are important for staff members who work on your AWS infrastructure, they should not be assigned to end-users of your service.
The process also avoids assigning IAM permissions to each EC2 instance. While the instances themselves could call Route 53 to update a record for their domain name, this requires an IAM Role to be assigned to the EC2 instance. If your users have access to the instance itself, this would potentially open a security hole where they could call Route 53 with incorrect data, such as assigning other users' domain names to their own instance.
It's worth mentioning that the above recommendations mirror the way that Amazon WorkSpaces operates — users authenticate, their instance is started and after a period of non-use the instance is stopped.
I will recommend use of cloudformation template. Cloudformation can create EC2 and then attach it to route53 url. So when tom like to use the EC2 instance, he have to run the stack in Cloudformation. Once he finished he have to go back to cloudformation and destroy the stack.
Yes Cloudformation would be a recommended approach. You can try cloudkast which is an online cloudformation template generator. It will make your task of creating cloudformation template very easy and effortless
I am trying to wrap my head around EC2 instances, and I am having a bit of an issue. I heard from a friend of mine that Amazon will kill EC2 instances, and then they restart the image (thus losing all state). Unless it uses EBS as a backing store, you get no persistence.
But I have been looking into Xen and it seems like instances should easily migrate instead of being killed/restarted.
So, do Amazon EC2 instances randomly stop/start an image with all state being managed by something external like EBS?
Amazon EC2 instances will not be stopped/started/restarted unless you issue a command to do so.
In some situations (eg hardware maintenance), you might receive a request from Amazon asking you to stop & start your instance (which moves it to a different host). Such requests are typically issued with two weeks notice.
One AWS customer told me that their instance had been running continuously for over three years.
Yes it is quite possible that an EC2 instance dies and is replaced. Depending upon your data, you may need to use EBS, EFS or S3 to prevent data loss in such cases.