I've only just starting using ecs and it seems like ebs is sky rocking.I wanna stay within the free tier without worrying about this My question is how do i lower ebs?
I'm using terraform ecs module. I changed the root_block_device_size to 10
module "ecs_cluster" {
cluster_instance_root_block_device_size = 10
}
This is the library I'm using Ecs module link(Terraform)
It mentions this
cluster_instance_root_block_device_size number
Description: The size in GB of the root block device on cluster instances.
Default: 30
cluster_instance_root_block_device_type string
Description: The type of the root block device on cluster instances ('standard', 'gp2', or 'io1').
Default: "standard"
For my ecs i have changed the block device size to 10 not sure if i should mess around with gp2 or io1 to lower it
Thanks!
Update using this https://aws.amazon.com/ebs/pricing/ and by
putting my configuration and it seems like gp3 and lowering the gigabytes does lower the price....However it seems like doing 10gb cannot be initialized with Terraform so i started deploying it with 30gb and lowered it to 10gb and it seemed to have worked...
That screenshot is for I/Os. Those are reads/writes to the disk volume(s). That's not related to the size of the volumes. Changing it from 30GB to 10GB is not going to impact the I/Os metric at all.
I/Os are only charged on magnetic EBS volume types. If you switched to an SSD based EBS volume type, like gp2, you would not be charged for the I/Os.
The AWS Free Tier includes "30 GB of Amazon EBS: any combination of General Purpose (SSD) or Magnetic". So you could have up to 30GB of free gp2 volumes, which would be much faster than the standard magnetic volumes you are using now, and you would also not be charged for I/Os on those volumes.
Related
what does a "2" refers in SSD gp2 AWS storage?
I am refering to General Purpose(GP) Solid State Disk(SSD) in AWS,but why a "2" in gp2.I understand EC2 stands for Elastic Cloud Compute, but in GP2 there are no two P's.
It's just a name. Don't get too hung-up on it.
Yes, while the numbers in EC2 and S3 are due to duplicated letters (Elastic Compute Cloud, Simple Storage Service), the EBS volume types are generational names, like the EC2 instance types (eg t1, m4, c5).
The EBS volumes types are:
General Purpose SSD (gp2)
Provisioned IOPS SSD (io1)
Throughput Optimized HDD (st1)
Cold HDD (sc1)
Magnetic (standard)
Cant Modify or Resize Amazon EBS Volume. In us-east-1d N. Virginia. The instance it's connected to is t2.medium CentOS 7.
Any help would be appreciated.
Update: the answer below was correct when written, but was subsequently superceded by this announcement on June 28, 2018:
Starting today, Elastic Volumes extends support to Amazon Elastic Block Store (EBS) magnetic (standard) volume type. You can now dynamically increase capacity or change the type of magnetic (standard) volumes with no downtime or performance impact using a simple API call or a few console clicks.
https://aws.amazon.com/about-aws/whats-new/2018/06/amazon-ebs-extends-elastic-volumes-to-support-ebs-magnetic--standard--volume-type/
The issue that originally triggered this question should no longer occur.
In the screen shot, the volume type shows standard.
That's a previous generation magnetic volume.
The previous generation Magnetic volume type is not supported by the volume modification methods [...]
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/limitations.html
Those can't be resized, so the option is grayed out. Your new instance from the AMI probably has a gp2 SSD volume, which does support resize.
Stop the instance that the EBS volume is attached to.
Then the modify option should be available for use.
If that doesn't work then try detaching the volume.
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-expand-volume.html
As #michael-sqlbot says, you are using magnetic "standard" EBS volume, you can convert to ssd "gp2" volume following these steps:
Create snapshot of your EBS volume
Create new PIOPS/General Purpose SSD volume from your EBS
snapshot
Detaching existing attached magnetic volume from the
instance
Attach new PIOPS/General Purpose SSD volume
More info: https://n2ws.com/blog/how-to-guides/how-to-change-your-ebs-volume-type
I am using the free tier of AWS. I am experimenting with ECS and am following the article http://docs.aws.amazon.com/AmazonECS/latest/developerguide/launch_container_instance.html to create an ECS instance. One this I noticed is that using the community image amzn-ami-2016.03.e-amazon-ecs-optimized adds an EBS volume which cuts into my free tier usage. My question is, is this EBS volume required and can I do it without the EBS volume?
Any EC2 instance would need a Root volume at the very least to start the OS. All volumes in AWS are EBS volumes. So if you were wondering if you can have an EC2 instance without EBS, I don't think that is possible.
However, you can still reduce your EBS cost. It costs 10 cents per GB per month for an EBS volume. If you would notice, all Amazon ECS optimized EC2 instances are configured to use 30GB of EBS volume storage. That means you pay $3.00 extra per EC2 instance for a month! 8 out of that 30 GB is for Root, and 22 out of 30GB is for docker use.
Source:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-ami-storage-config.html
By default, the Amazon ECS-optimized Amazon Linux AMI ships with 30
GiB of total storage. You can modify this value at launch time to
increase or decrease the available storage on your container instance.
This storage is used for the operating system and for Docker images
and metadata. The sections below describe the storage configuration of
the Amazon ECS-optimized Amazon Linux AMI, based on the AMI version.
Of course, you don't need the full 8GB for Root and full 22GB for docker. So you can lower your cost by reducing the size of those volumes to say 2GB for Root and 2GB for docker use. Then you would be paying 40 cents per month, and not $3.00.
Reducing volume size, as far as I know, is not easy.
Since that is out of scope for this question, I will just provide this link for interested parties.
Now that you are aware that there are 2 volumes used by ecs optimized instances, there is a way for you to NOT use the 22GB volume at all, and simply use the Root volume for docker storage. This too is not easy but can be done by creating your own AMI with docker and ecs agent installed. Then you will have to configure your docker to use the Root volume instead of the other one. Here is a thread which slightly discussed this issue.
For AWS ECS there is no additional charge for Amazon EC2 Container Service. You pay for AWS resources (e.g. EC2 instances or EBS volumes) you create to store and run your application. Free tier in AWS https://aws.amazon.com/free/ only Amazon EC2 Container Registry is part of free tier which offers 500 MB for storage.
And also if you are creating ECS containers from amzn-ami-2016.03.e-amazon-ecs-optimized AMI the volumes will be EBS so you will have to pay for EBS volumes.
AWS launched sc1 and st1 HDD EBS volume types recently, I can't seem to use these as root volumes while launching new EC2 instances or launching from already created AMI's (tried both).
I chose an m4 machine, in any case, the root volume is EBS itself, below is a screenshot, the second volume that I add gets the new options, however the first one I can't choose the same. Is this by design AWS people?
If you look from http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html
under the main table for volume type as Throughput Optimized HDD (st1) and Cold HDD (sc1) it says
Cannot be a boot volume
and below
Throughput Optimized HDD (st1) volumes provide low-cost magnetic
storage that defines performance in terms of throughput rather than
IOPS. This volume type is a good fit for large, sequential workloads
such as Amazon EMR, ETL, data warehouses, and log processing. Bootable
st1 volumes are not supported.
and
Cold HDD (sc1) volumes provide low-cost magnetic storage that defines
performance in terms of throughput rather than IOPS. With a lower
throughput limit than st1, sc1 is a good fit ideal for large,
sequential cold-data workloads. If you require infrequent access to
your data and are looking to save costs, sc1 provides inexpensive
block storage. Bootable sc1 volumes are not supported.
Because the customer experience would be awful. Boot volumes use small, random I/O; these volumes aren't designed for small I/O. Just use GP2 for boot volumes.
We have a replication setup on Amazon EC2 that used Magnetic disks (15GB) that we swapped with SSD disks (15GB) for each the replication servers. We noticed that the slaves would fall behind the master and never catch up with these new SSD disks. This is something that never happened with the Magnetic disks but happened on each and every SSD disk.
We decided to try and move the databases back to Magnetic disks after the SSD disks fell more than 2 days behind. Within 2 hours the slave completely caught up.
I thought that SSD disks were more efficient, and all around better than Magnetic disks and that is why Amazon decided to make them standard.
Another bit of information is that we are using Micro instances, but the only changes we made was with the attached disk.
Anyone have any ideas?
I think you might be bumping up against the max IOPS for your 15GB SSD drive.. Amazon only allows an average of 45 IOPS for that disk size. On a magnetic drive, they have an extra charge when you use more IOPS, it does not seem to be throttled in the same way:
From Amazon's Pop Up on IOPS:
The number of input-output operations per second. For Provisioned IOPS (SSD) volumes, you can specify the IOPS rate when you create the volume. The ratio of IOPS provisioned and the volume size requested can be a maximum of 30 (in other words, a volume with 3000 IOPS must be at least 100 GB). General Purpose (SSD) volume types have a baseline IOPS of volume size X 3 and can burst up to 3000 IOPS for 30 minutes.