I am trying to create an API that runs on AWS EC2 t2.micro. The problem I'm having is that my instance keeps shut down automatically every ~3 hours, which could be because of the "session time" of my AWS Educate account (screenshot attached)
Is there any way to keep my instance running constantly (for days and even months)?
I am using "tmux", which does seem to keep my API and the EC2 instance running even after my ssh connection is terminated, but the EC2 instance itself still shuts down automatically.
EDIT: If it is not possible to keep an EC2 instance of an AWS Educate account running constantly. Is there a way to start a new session automatically when the old instance's session has expired?
(Maybe using a script/using some tools offered by AWS? I'm new to AWS so I don't know if this is possible)
Sadly you can't change that. It is explicitly stated in AWS educate docs:
When your session ends, your resources will be “stopped.” You will be required to re-start your resources when you start a new session.
Instead of Using AWS Educate you can Create Regular AWS account which provides some services for free for one year. It includes the EC2 instance as well so you don't have to pay anything and you can run for months and year it will never gone down until you manually stop it.
Related
Yesterday I had a nightmarish experience with my AWS account.
In short:
1. My only running instance was taken out of service in response to an EC2 instance status checks failure at 2021-09-23T19:00:46Z.
2. A new instance was started in response to a difference between desired and actual capacity, increasing the capacity from 0 to 1 at 2021-09-23T19:01:06Z.
3. Volume attached to my running instance was gone along with all data, software etc. nowhere to be found.
4. All the weekly snap shots of this volume are also gone, so I cannot recreate volume.
I am using AWS services for many many years and it never happened to me. I am very surprised that this can happen on AWS.
I have posted the question on AWS EC2 discussion forum. But I noticed that they rarely answer any question.
Should I buy there "Developer Support" for $29 per month but I am not sure even that will solve the problem. And how they will ensure that it does not happen again.
Any advice what should I do in this situation?
PS: Only now I realized the importance of good customer support when choosing cloud service provider.
The message that "A new instance was started in response to a difference between desired and actual capacity" indicates that you were using an Amazon EC2 Auto Scaling Group.
Auto Scaling groups will automatically launch new instances when capacity is required and terminate instances when there is too much capacity. Also, if an instance fails health checks, it will be automatically terminated and replaced with a new instance.
When Auto Scaling terminates an instance, the EC2 instance is 'killed'. Any Amazon EBS volumes attached to the instance will, by default, also be deleted unless Delete on Termination is intentionally turned off.
When operating under an Auto Scaling group, it is not a good idea to store data on the instance itself because instances can be launched and terminated. Data should be stored in a database, or in Amazon S3 or Amazon EFS.
AWS might be able to recover that data if the volumes were recently deleted. Yes, you would need to subscribe to AWS Support to request assistance.
It is unfortunate that your instance was launched under Auto Scaling many years ago, which led to this problem. On the plus side, it shows the reliability of Amazon EC2 since was running without issue for all those years (until now).
We have an EC2 instance running in AWS EC2 instance. We have our ML algorithms and data that. We have also hosted a web-based interface also in that machine.
Now there are no new developments happening in that EC2 instance. We would like to terminate AWS subscription for a short period of time (for the purpose of cost-reduction and exploring new cloud services). Most importantly, we want to be in a position where we can purchase a new EC2 instance with a fresh AWS subscription, use the backup which we take now, and resume all operations (web-backend, SMS services for our app which is hosted in AWS, etc.).
What is the best way to do it? Is temporary termination of AWS subscription advisable?
There is no concept of an "AWS Subscription". AWS is charged on-demand, which means you only pay when you use resources.
If you temporarily do not want the Amazon EC2 instance, you could:
Stop the instance, which is like turning off the power. You will not be charged for the instance, but you will still pay for the disk storage attached to the instance. You can simply Start the instance again when you wish to use it. You will only be charged while the instance is running. OR
Create an image of the instance, then terminate the instance. This will create an Amazon Machine Image (AMI), which contains a copy of the disks. You can then launch a new Amazon EC2 instance from the AMI when you wish to use it again. This is a lower-cost option compared to simply stopping the instance, but it takes more effort to stop/start.
It is quite common for companies to stop Amazon EC2 instances at night or over the weekend to reduce costs while they are not needed.
EDIT: Just thought of a third option. Will test it and be back. Not worth it; it would involve creating an image from the EC2 instance and then convert that image to a VM image, storing the VM image in S3. There may be some advantages to this, but I do not see them.
I think you have two options, both of them very reasonably priced. If you can separate the data from the operating system, then your best option would be to use an S3 bucket as a file system within the EC2 instance. Your EC2 instance would use this bucket to store all your "ML algorithms and data" and, possibly, even your "web-based interface". Whenever you decide that you no longer need the processing capacity of the EC2, you would unmount the S3 bucket file system from the EC2 instance and terminate that instance. After configuring an appropriate lifecycle rule for the S3 bucket, it would transition to Glacier, or even Glacier Deep Archive [you must considerer the different options of long term storage]. In the future, whenever you want to work with your data again, you would move your data from Glacier back to S3, create a new EC2 instance, install your applications, mount your S3 bucket as a file system and you would have access to all your data. I think this is your least expensive and shortest recovery time objective option. To implement this option, look at my answer to this question; everything you need to use an S3 bucket as a regular folder inside the EC2 instance is there.
The second option provides an integrated solution, meaning the operating system and the data stay together, and allows you to restore everything as it was the day you stopped processing your data. It's made up of the following cycle:
Shutdown your EC2 and make a note of all the specs [you need them further down].
Export your instance to a virtual image, vmdk for example, and store it in your S3 bucket. Something like this:
aws ec2 create-instance-export-task --instance-id i-0d54b0682aa3998a0
--target-environment vmware --export-to-s3-task DiskImageFormat=VMDK,ContainerFormat=ova,S3Bucket=sm-vm-backup,S3Prefix=vms
Configure an appropriate lifecycle rule for the S3 bucket so that it transitions to Glacier, or even Glacier Deep Archive.
Terminate the EC2 instance.
In the future you will need to implement the inverse, so you will need to restore the archived S3 Object [make sure you you can live with the time needed by AWS to do this]
Import the virtual image as an EC2 AMI, something like this [this is not complete - you will need some more options that you saved above]:
aws ec2 import-image --disk-containers
Format=ova,UserBucket="{S3Bucket=sm-vm-backup,S3Key=vmsexport-i-0a1c382e740f8b0ee.ova}"
Create an EC2 instance based on the image and you're back in business.
Obviously you should do some trial runs and even automate the entire process if it's something that will be done frequently. I have a feeling, based on what you said, that the first option is a better option, provided you can easily install whatever applications they use.
I'm assuming that you launched an EC2 instance from a base Amazon Machine Image and then added your own software and models to it. As opposed to launched an EC2 instance from an AWS Marketplace offering.
The simplest thing to do is to create an Amazon Machine Image (AMI) from your running EC2 instance. That will capture the current state of the instance and persist it in your AWS account. Then you can terminate the instance. Later, when you want to recreate it, launch a new instance, selecting the saved AMI instead of a standard AMI.
An alternative is to avoid the need to capture machine state at all, by using standard DevOps practices to revision-control everything you need to recreate the state of a running machine.
Note that there are costs associated with an AMI, though they are minimal ($0.05 per GB-month of data stored, for example).
I had contacted AWS customer care regarding this issue. Given below is the response I received. Please add your comments on which option might be good for me.
Note: I acknowledge the AWS customer care team for their help.
I understand that you require some information on cost saving for your
Instance since you will not be utilizing the service for a while.
To assist you with this I would recommend checking out the Instance
Stop/Start link here:
==>https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Stop_Start.html .
When you stop an Instance, you do not lose any data & you are not
charged for the resources any further. However please keep in mind
that you will still be charged for any EBS Storage Volumes attached to
the stopped Instance(s).
I also recommend checking out the below links on how you can reduce
your costs.
==>https://aws.amazon.com/premiumsupport/knowledge-center/reduce-aws-bill/
==>https://aws.amazon.com/blogs/compute/10-things-you-can-do-today-to-reduce-aws-costs/
That being said, please note that as I am in the billing department,
for the best assistance with the various plans you will require the
assistance of our Sales Team.
The Sales Team will be able to assist with ways to save while
maintaining your configurations.
You will be able to reach the Sales Team here:
==>https://aws.amazon.com/websites/contact-us/.
Once you have completed the details in the link, a member of the team
will be in touch with you at their soonest.
I have approximately 40 Windows EC2 instances running at the moment. This number will start to grow substantially in the next few months. Each one is a t2.small Windows 2016 Server instance. Cost is starting to become an issue as the number increases. Each instance has its own Elastic IP address because when user Tom wants to access his machine he will use the DNS tom.mydomain.com.
tom.mydomain.com is registered in a Route53 hosted zone pointing to Elastic IP 22.33.44.55 which has been associated with Tom's EC2 instance.
Problem is that Tom only needs to use his machine 4 hours per day. When not using it he simply shuts the machine down. But... An Elastic IP that is pointing to a stopped instance costs almost as much per hour as a t1.micro instance in a running state.
So what I want to do is when Tom logs into AWS console and starts his EC2 instance, it will automatically register itself with Route53 against the DNS "tom.mydomain.com".
In short I want to do away with the need for Elastic IPs which are fast becoming a very substantial cost.
The tutorial Auto-Register EC2 Instance in AWS Route 53
looks like it does exactly what I want to do. The problem is the scripting is for Linux. I want to get it working for Windows. I have everything done down to step 6 in the tutorial but am stuck there. Any one get something similar to this working on Windows?
I would recommend:
Create a web-based front-end where your users can authenticate and request access to their Amazon EC2 instance
You could use Amazon Cognito for authentication and DynamoDB for data storage
Once the user authenticates, the service can:
Start their EC2 instance (if it was previously stopped)
Associate the random public IP address to the customer's domain name
Tell the user that the instance is now available
Users login to the instance and perform their work function
You then have some mechanism (I'm not sure what) that detects that they no longer need the instance, and then Stops the instance to save costs
The above process avoids assigning IAM credentials to your users. While IAM credentials are important for staff members who work on your AWS infrastructure, they should not be assigned to end-users of your service.
The process also avoids assigning IAM permissions to each EC2 instance. While the instances themselves could call Route 53 to update a record for their domain name, this requires an IAM Role to be assigned to the EC2 instance. If your users have access to the instance itself, this would potentially open a security hole where they could call Route 53 with incorrect data, such as assigning other users' domain names to their own instance.
It's worth mentioning that the above recommendations mirror the way that Amazon WorkSpaces operates — users authenticate, their instance is started and after a period of non-use the instance is stopped.
I will recommend use of cloudformation template. Cloudformation can create EC2 and then attach it to route53 url. So when tom like to use the EC2 instance, he have to run the stack in Cloudformation. Once he finished he have to go back to cloudformation and destroy the stack.
Yes Cloudformation would be a recommended approach. You can try cloudkast which is an online cloudformation template generator. It will make your task of creating cloudformation template very easy and effortless
I am using AWS free tier, and running Elastic Beanstalk which is running EC2 and RDS, and I am looking for a way to stop and start the environment only when needed should i stop EC2 and RDS individually from the dashboard or what would be a good way of doing it?
One of the post i found was When I stop and start an ec2 cents os instance , what data do I loose and it says data will not be lost but how do I stop and start the EBS when needed?
If you are using Beanstalk and try to stop particular EC2 instance from EC2 console than Beanstalk will bring it back automatically. If you want to stop whole EBS environment than you can use Terminate option that will terminate it, obviously. You will be able to bring it back for 40 days and after that it will be lost. Remember that you will see terminated environment for ~1h in the EBS console and after that time you will be able to bring it back only using eb tool so remember to write down your environment's ID so you can restore it later with $ eb restore ENV_ID
As far as the EC2 instance is concerned, if you have a load-balanced, auto-scaling setup, then you can use the scheduled autoscaling feature to shrink your desired number of instances to zero on whatever schedule you like. To do this, go to the capacity section of the environment dashboard in the console, and scroll to the bottom ("Time-based Scaling"). Here you can enter two cron expressions, one for scale out and one for scale in, for a recurring pattern which will shut down the EC2 instance when you like.
The RDS instance is a bit trickier. You could write a lambda function that would take a snapshot, shut it down, and later restore the snapshot to a new instance, scheduled using a cron expression or similar in CloudWatch Events. A similar approach could work for the EC2 instance and its EBS volume.
After I initiated an Amazon ECS following their tutorial (But I don't recall there is one step that asks me for the key-pair information.)
After I set it up, I found that there is an extra EC2 in my EC2 instance list that starts to charge me money. I wonder what that EC2 is doing.
Is it the EC2 that is associated with ECS that I can start to build my own server on?
If so, how can I log into it? (There is no key-pair information for me to log in. It says I need to log in via valid username-password pair, but I don't even know my username.)
If not, how can I kill it? (Directly terminating it in EC2 service is not helping since it seems ECS will just start another one)
The username will be ec2-user. ECS creates a launch configuration in which you can set key pair
ECS creates an autoscaling group you can find it under ec2/autoscaling/home you can edit this group and set min and desired to 0 this will shut down the instance automatically.