2 months ago I started using AWS and now I am having an issue with the Free Tier.
I installed a system with the elastic beanstalk tool (1 EC2 and 1 RDS).
Now I am seeing in Management Dashboard that I will be exceeding my monthly limit for EBS Snapshots. The forecast is 194.94% and I am just starting the month.
I reviewed all the settings and I couldn't find anything strange.
EC2
1 Running instances
1 volumes
2 Key Pairs
1 Elastic Ips
1 Snapshots
By looking the volume menu in AWS, the Snapshot id in the only volume that I have is not the same as the snapshot that I have. This is strange.
volume information:
Volume ID --> vol-0cff610c
Size --> 8 Gib
Volume type --> gp2
IOPS --> 100 / 3000
Snapshot --> snap-04786
State --> in-use
And in the snapshot menu, the volume id is not the same as the volume that I have in the other view.
Snapshot Information
Snapshot ID --> snap-0d2988
Size --> 8Gib
Volume --> vol-05f5fee
Status --> completed
Does anybody have an idea of what is causing this problem with the EBS?
Thanks!
Firstly, don't panic. EBS Snapshots are 5c/GB/month. Worst case, 8GB = 40c/month.
Secondly, if you don't want a snapshot, just delete it. Especially if you don't remember making it, and don't need to keep it.
Elastic Beanstalk can create AWS resources and terminate them automatically for you, depending on your workload.
If your server was under heavy workload at some point, Elastic Beanstalk could have provisioned auto scaling group with Elastic Load Balancer, and the number of EC2 instances can increase to cater for the increased workload.
As the workload drops, it can terminate the instances for you as well. Resulting in left-over snapshots, especially if you code writes to the local EC2 hdd.
Elastic Beanstalk is the "one-click" solution for deploying scalable clusters of servers.
Similar services are as follow :
Elastic Beanstalk (easiest)
OpsWorks
Cloud Formation (hardest)
LightSail is not included here as it doesn't really scale automatically. Lambda is serverless so it is fundamentally different.
today I reviewed again the information and AWS console.
My problem was that I have an unused AMI and the snapshot was related to that AMI.
I thought that the only AMI I had was being used by the instance. But I was wrong, the AMI used in the instance was a different one. Probably because I started my instance with elastic beanstalk.
So the solution was to deregister this unused AMI, then delete the Snapshot.
Thanks to all
Related
I require a fair bit of RAM and disk for a Docker container that will run infrequently as a task on ECS. My workflow:
Start EC2 instance for ECS
Run task on ECS
Terminate EC2 instance
I terminate the instance between runs because these resources are relatively expensive, and I don't want them running when not in use. Fargate is not appropriate due to its resource limitations, so I'm running ECS on EC2.
I cannot get more than 30GB disk for the image without a lot of human intervention. I can attach arbitrary EBS data volumes (/dev/xvdcz), but AWS still always creates a 30GB root volume /dev/xvda which is used for the container itself.
How do I use a larger than 30GB volume for the Docker container itself?
What I've tried:
Creating an Auto Scaling Group with a launch configuration where the root volume is larger. This does create an instance with a larger root volume, but there is no way to attach this group to a cluster, or link its created EC2 instance with the cluster. Cluster creation seems to be tied to an auto scaling group and instance.
Using an instance with a large dedicated SSD rather than an EBS volume, again the 30GB partition is created for the container.
Mounting dev/xvdcz to the container. This does add the space, but requires me to rewrite my code to only use this folder.
Using the AWS ECS cli to modify disk after creation. This is described in a similar issue. However, as my EC2 instance terminates after task completion, the ID does not persist between runs, and aws ecs describe-clusters does not specify the underlying EC2 instance, so this cannot be automated. A human needs to boot up the instance, look at the ID, and then the volume size can be modified via the CLI.
This issue was brought up on Github back in 2016 but marked as not important and closed, the discussion there is not very helpful.
Under EC2 > Auto Scaling, create a new Auto Scaling Group with a launch config that has your chosen root volume size. This will boot up an EC2 instance by default, leave it for now.
Go to your cluster's auto-created Auto Scaling Group, note the name down (need it later), then click Launch Template > Edit. If it says 'launch configuration,' press 'switch to launch template.'
Select your previously created Auto Scaling Group's template (note if you created multiple versions, select the latest, version 1 is selected by default). Select 'adhere to launch template' and click 'update.'
Delete the Auto Scaling group you first created. This will shut down its related EC2 instance.
Reboot your cluster's ec2 instance:
aws autoscaling set-desired-capacity --auto-scaling-group-name NAME_OF_GROUP --desired-capacity 0
Wait for it to shut down, you can see instance state in the EC2 console.
aws autoscaling set-desired-capacity --auto-scaling-group-name NAME_OF_GROUP --desired-capacity 1
Again wait for it to boot up. Once booted, and for any subsequent boots, /dev/xvda will be the size you specified.
I have one r5.xlarge windows ec2 instance which is attached to 6TB EBS volume and the backup for EBS is taken every week.
Now I want a better solution with zero downtime.
if ec2 instance fails I want new ec2 instance to be to created and it should be attached to EBS volume automatically.
if EBS volume crashes I want backup snapshot to be available and gets attached to ec2 instance with zero downtime and volume content available immediately.
Is there a way to implement solution for this and how?
If you want to improve the availability of the large drive, I would recommend FSx. It's not so trivial - you need to set up Active Directory, and your EC2 needs to join AD - but once you have all these components (and there is additional cost as well) - FSx provides significantly higher availability than EBS assuming you're using a multi-AZ setup.
More information here
Note that load balancer tags are irrelevant to your question - although to address your first situation you probably need load balancer balancing two EC2 instances
You should try and use Elastic BeanStalk, this will handle the autoscaling on its own. You just have to configure the settings while creating it.
This should be the nearest solution that you need, as explained by Marcin above there is no zero downtime for 2 single point of failures. So Elastic BeanStalk seems to be the best solution for you.
Updated:
So in this case what you can do is add a load balance and add auto scaling, so that the first point of your questions is taken care and if the ec2 instance goes down through autoscaling a new instance will be create and in the shell script you can add commands to map to the EBS.
And for EBS, there isn't much you can do, you can take regular backups and if the EBS fails then replace it with the latest backup or snapshot, also check the below link:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-restoring-volume.html
It seems that my EC2 usage limit is being reached rather quickly. I have deleted all of my EC2 instances and most of my S3 buckets, and none of my EC2 instances even exist in the terminated state. Are there any other services other than EC2 that use the EBS storage? Thanks in advance.
Amazon EBS is only used by Amazon EC2 instances. (Well, it is also used by Amazon RDS, but it shows up as an RDS charge, not EBS.)
We are currently about a third of the way through the month, so you'd want to be around 30% of usage.
The Amazon EBS snapshot usage is ahead of that (58%). If this worries you, then you can delete snapshots under the Snapshots section in the EC2 console. Amazon Machines Images (AMIs) also use EBS snapshots, so check the Images section too.
The amounts are "growing" because they are based on a month of usage. So, 1GB for 1 day is ~ 3% of the month's total.
However, there is little need to panic — EBS Snapshots are charged at 5c/GB/month, so at the current rate of usage you might be charged 10c.
There can be another possibility where you run an instance with EBS volumes 'not deleted' even after the termination of instance, it can accrue storage charges also. So delete those ones also in the 'Volumes' section of EC2 instance if its still unused.
I upgraded an AWS EC2 machine 10 days ago (from t2-medium to t2-large). Again, I just upgraded today the same AWS EC2 machine (from t2-large to t2-xlarge) and after rebooting I noticed a massive loss of data (10 days of work). In fact, after changing instance type I found out it reverted to the same "state" of the machine just after doing the upgrade 10 days ago. Not less important, I have been doing snapshots periodically (twice a day), and when I try to recover from the last one, each snapshot is an exact copy of the machine after the first upgrade.
This is not the first project I have in AWS, and everytime I did a snapshot, it would always keep the very last copy of the machine. But NOT in this case. So I am really dazzled.
Can you give some hint on what could happen? Is it normal that AWS deletes data (DDBB + folder and files) when changing instance type? Do you think it might have to do with IAM policies?
Thank you
The EC2 user guide has information about the EC2 Storage options. To summarise:
The EC2 instance store volume is erased when the instance is stopped or terminated.
In contrast, EBS (Elastic Block Storage) can be mounted to a running EC2 instance (c.f. a physical hard drive). EBS volumes can be detached from one instance and attached to another instance. Moreover, you can create a snapshot of an EBS volume that is stored in S3 and you can recreate EBS volumes from that snapshot that can be attached to other EC2 instances.
The EC2 user guide provides step by step instructions:
Restoring an Amazon EBS Volume from a Snapshot
Attaching an Amazon EBS Volume to an Instance
Lastly, no, this behaviour is not related to IAM policies.
I am using the free tier of AWS. I am experimenting with ECS and am following the article http://docs.aws.amazon.com/AmazonECS/latest/developerguide/launch_container_instance.html to create an ECS instance. One this I noticed is that using the community image amzn-ami-2016.03.e-amazon-ecs-optimized adds an EBS volume which cuts into my free tier usage. My question is, is this EBS volume required and can I do it without the EBS volume?
Any EC2 instance would need a Root volume at the very least to start the OS. All volumes in AWS are EBS volumes. So if you were wondering if you can have an EC2 instance without EBS, I don't think that is possible.
However, you can still reduce your EBS cost. It costs 10 cents per GB per month for an EBS volume. If you would notice, all Amazon ECS optimized EC2 instances are configured to use 30GB of EBS volume storage. That means you pay $3.00 extra per EC2 instance for a month! 8 out of that 30 GB is for Root, and 22 out of 30GB is for docker use.
Source:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-ami-storage-config.html
By default, the Amazon ECS-optimized Amazon Linux AMI ships with 30
GiB of total storage. You can modify this value at launch time to
increase or decrease the available storage on your container instance.
This storage is used for the operating system and for Docker images
and metadata. The sections below describe the storage configuration of
the Amazon ECS-optimized Amazon Linux AMI, based on the AMI version.
Of course, you don't need the full 8GB for Root and full 22GB for docker. So you can lower your cost by reducing the size of those volumes to say 2GB for Root and 2GB for docker use. Then you would be paying 40 cents per month, and not $3.00.
Reducing volume size, as far as I know, is not easy.
Since that is out of scope for this question, I will just provide this link for interested parties.
Now that you are aware that there are 2 volumes used by ecs optimized instances, there is a way for you to NOT use the 22GB volume at all, and simply use the Root volume for docker storage. This too is not easy but can be done by creating your own AMI with docker and ecs agent installed. Then you will have to configure your docker to use the Root volume instead of the other one. Here is a thread which slightly discussed this issue.
For AWS ECS there is no additional charge for Amazon EC2 Container Service. You pay for AWS resources (e.g. EC2 instances or EBS volumes) you create to store and run your application. Free tier in AWS https://aws.amazon.com/free/ only Amazon EC2 Container Registry is part of free tier which offers 500 MB for storage.
And also if you are creating ECS containers from amzn-ami-2016.03.e-amazon-ecs-optimized AMI the volumes will be EBS so you will have to pay for EBS volumes.