I have approximately 40 Windows EC2 instances running at the moment. This number will start to grow substantially in the next few months. Each one is a t2.small Windows 2016 Server instance. Cost is starting to become an issue as the number increases. Each instance has its own Elastic IP address because when user Tom wants to access his machine he will use the DNS tom.mydomain.com.
tom.mydomain.com is registered in a Route53 hosted zone pointing to Elastic IP 22.33.44.55 which has been associated with Tom's EC2 instance.
Problem is that Tom only needs to use his machine 4 hours per day. When not using it he simply shuts the machine down. But... An Elastic IP that is pointing to a stopped instance costs almost as much per hour as a t1.micro instance in a running state.
So what I want to do is when Tom logs into AWS console and starts his EC2 instance, it will automatically register itself with Route53 against the DNS "tom.mydomain.com".
In short I want to do away with the need for Elastic IPs which are fast becoming a very substantial cost.
The tutorial Auto-Register EC2 Instance in AWS Route 53
looks like it does exactly what I want to do. The problem is the scripting is for Linux. I want to get it working for Windows. I have everything done down to step 6 in the tutorial but am stuck there. Any one get something similar to this working on Windows?
I would recommend:
Create a web-based front-end where your users can authenticate and request access to their Amazon EC2 instance
You could use Amazon Cognito for authentication and DynamoDB for data storage
Once the user authenticates, the service can:
Start their EC2 instance (if it was previously stopped)
Associate the random public IP address to the customer's domain name
Tell the user that the instance is now available
Users login to the instance and perform their work function
You then have some mechanism (I'm not sure what) that detects that they no longer need the instance, and then Stops the instance to save costs
The above process avoids assigning IAM credentials to your users. While IAM credentials are important for staff members who work on your AWS infrastructure, they should not be assigned to end-users of your service.
The process also avoids assigning IAM permissions to each EC2 instance. While the instances themselves could call Route 53 to update a record for their domain name, this requires an IAM Role to be assigned to the EC2 instance. If your users have access to the instance itself, this would potentially open a security hole where they could call Route 53 with incorrect data, such as assigning other users' domain names to their own instance.
It's worth mentioning that the above recommendations mirror the way that Amazon WorkSpaces operates — users authenticate, their instance is started and after a period of non-use the instance is stopped.
I will recommend use of cloudformation template. Cloudformation can create EC2 and then attach it to route53 url. So when tom like to use the EC2 instance, he have to run the stack in Cloudformation. Once he finished he have to go back to cloudformation and destroy the stack.
Yes Cloudformation would be a recommended approach. You can try cloudkast which is an online cloudformation template generator. It will make your task of creating cloudformation template very easy and effortless
Related
I am trying to create an API that runs on AWS EC2 t2.micro. The problem I'm having is that my instance keeps shut down automatically every ~3 hours, which could be because of the "session time" of my AWS Educate account (screenshot attached)
Is there any way to keep my instance running constantly (for days and even months)?
I am using "tmux", which does seem to keep my API and the EC2 instance running even after my ssh connection is terminated, but the EC2 instance itself still shuts down automatically.
EDIT: If it is not possible to keep an EC2 instance of an AWS Educate account running constantly. Is there a way to start a new session automatically when the old instance's session has expired?
(Maybe using a script/using some tools offered by AWS? I'm new to AWS so I don't know if this is possible)
Sadly you can't change that. It is explicitly stated in AWS educate docs:
When your session ends, your resources will be “stopped.” You will be required to re-start your resources when you start a new session.
Instead of Using AWS Educate you can Create Regular AWS account which provides some services for free for one year. It includes the EC2 instance as well so you don't have to pay anything and you can run for months and year it will never gone down until you manually stop it.
I would like to try to setup AWS Launch Template, or just Spot request (persistance) and I need automatically attach my specific volume.
The main idea - spot instance will be process data and store it in a separate volume. When Spot will die, another Spot should be requested automatically (which will be built from an image with predefined software) and data should continue processing automatically (and again, storing in my second volume).
But, I can`t setup it in AWS console, so, looks like it is not possible. Am I wrong? Is it possible in some another way?
The same according IP address - I would like to have the same IP address for any of "versions" of Spot (after recreating for example)
It is possible to attach and detach Elastic IP using AWS CLI to achieve what you want.
However there are other possible workarounds in case you do not want to script AWS CLI :
Using Route53 you can define an A record with TTL of 60 seconds if you can accept few minutes of transition. That way you get to use a domain to access your underlying instance instead.
Setup a ALB and forward the request to the EC2 fleet
I got the mail from Amazon regarding my EC2 as follows:
EC2 has detected degradation of the underlying hardware hosting your Amazon EC2 instance (instance-ID: xxxxxxxx) associated with your AWS account (AWS Account ID: xxxxxx) in the ap-south-1 region. Due to this degradation your instance could already be unreachable. We will stop your instance after 2020-12-29 22:00:00 UTC. Please take appropriate action before this time.
The affected instances are listed below:
xxxxxxx
So what would I do now to keep my data safe ?
AWS provides guides on what to do when an instance is about to be retired:
What do I need to know when my Amazon EC2 instance is scheduled for retirement?
The easiest way would be to stop/start the instance:
You are required to stop and then start the instance at your preferred time before the instance retirement date. Stopping and starting the instance moves the instance to another healthy host.
However, exact details depend on your instance type (e.g. EBS based or instance store based), thus please read the guide in the link provided to understand your options which are mostly determined on your actual EC2 instance setup.
Is it possible to do AutoScaling with Static IPs in AWS ? The newly created instances should either have a pre-defined IP or pick from a pool of pre-defined IPs.
We are trying to setup ZooKeeper in production, with 5 zooKeeper instances. Each one should have a static-IP which are to hard-coded in the Kafka's AMI/Databag that we use. It should also support AutoScaling, so that if one of the zooKeeper node goes down, a new one is spawned with the same IP or from a pool of IPs. For this we have decided to go with 1 zoo-keeper instance per AutoScaling group, but the problem is with the IP.
If this is the wrong way, please suggest the right way. Thanks in advance !
One method would be to maintain a user data script on each instance, and have each instance assign itself an elastic IPs from a set of EIPs assigned for this purpose. This user data script would be referenced in the ASGs Launch Configuration, and would run on launch.
Say the user script is called "/scripts/assignEIP.sh", using the AWS CLI you would have it consult the pool to see which ones are available and which ones are not (already in use). Then it would assign itself one of the available EIPS.
For ease of IP management, you could keep the pool of IPs in a simple text properties file on S3, and have the instance download and consult that list when the instance starts.
Keep in mind that each instance will need an to be assigned IAM instance profile that will allow each instance to consult and assign EIPs to itself.
Building off of this question:
https://unix.stackexchange.com/questions/24355/is-there-a-way-to-get-the-public-dns-address-of-an-instance
I know how to get an ec2 instance's own public DNS address. What I need is a way for this instance to get the public DNS instance of a second ec2 instance.
The idea is that I will have ~50 instances running, one or two of which will be a spot instance that is constantly running. All of the other worker instances need to know the spot, or master, instance's public DNS name to connect to it within my application. How can I do this?
On another note, is there a way I can create a backup of my spot master instance? In case it fails, I would like to have another spot instance that immediately takes its place, but my worker ec2 instances would have to update their information about the spot instance's public dns address.
I think the only way to get the public DNS of your other instance is by using the command line interface or Web API provided by amazon.
The concrete command you need is ec2-describe-instances which provides data about public DNS settings for each instance.
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DescribeInstances.html
Of course you can do the same through the Web API:
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeInstances.html
Regarding the backup you can map the spot instance to EBS (which is preferrable) and then make snapshot backups. The snapshot backups are still triggered manually in the amazon console (or again through command line tools and Web API). Snapshots should be good for regular backups.
You can also use a service like http://www.skeddly.com/ to automate your EC2 snapshot backups.
If you want to backup the full AMI image of your spot instance, so you can re-create it from scratch at a later time, or create multiple instances from the same image etc. go to the management console and do the following:
Click on Instances
Select the instance you want to create an AMI from
Click on "Actions" and select "Create Image"
Set the Image name and other info and save
An alternative is to use S3. When a spot instance comes up it will read its own public address and write it to a bucket in S3. The other instances will look up the bucket the first time they need it and use this value. If the spot instance goes down, the workers will poll the bucket periodically until a new spot instance comes up and updates the bucket.
Make sure to set the bucket to only allow authenticated access so only your applications can modify it.
This approach has a security advantage, as the VMs do not need access to your EC2 credentials. They only need access to a specific S3 bucket.