Is there a way to generate EC2 utilization report which contains below selections (I need all of them in one report)
EC2 instance name
Current status (stopped/running)
Availability Zone
Region
Cost incurred for the current month.
There is no single report that would contain that information.
If you want the "current" information about an instance, you'll need to make an API call, or use the AWS Command-Line Interface (CLI) aws ec2 describe-instances command. That will provide you with everything except cost.
You can use the Cost and Usage Report to identify costs associated with particular instances (by Instance ID, not name).
Related
I am trying to differentiate costs among my AWS EC2 instances.
I've tried everything mentioned via forums and AWS, I've created Tags to ID each EC2 instance, and I still cannot seem to get billing report telling me how much each instance is accumulating cost wise. They are all lumped together.
I created and enabled a tag for each, but still Cost Explorer is virtually useless, and only chunks these costs into instance types.
One other similar solution mentioned to create separate AWS accounts for each customer/client? Which in my case, is too late, if this is the only way to differentiate costs among the running EC2 instances.
You need to manually enable cost allocation tags in the console.
https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/activating-tags.html
To activate your tags
Sign in to the AWS Management Console and open the Billing and Cost Management console at https://console.aws.amazon.com/billing/.
In the navigation pane, choose Cost Allocation Tags.
Select the tags that you want to activate.
Choose Activate.
Good luck!
We have an EC2 instance running in AWS EC2 instance. We have our ML algorithms and data that. We have also hosted a web-based interface also in that machine.
Now there are no new developments happening in that EC2 instance. We would like to terminate AWS subscription for a short period of time (for the purpose of cost-reduction and exploring new cloud services). Most importantly, we want to be in a position where we can purchase a new EC2 instance with a fresh AWS subscription, use the backup which we take now, and resume all operations (web-backend, SMS services for our app which is hosted in AWS, etc.).
What is the best way to do it? Is temporary termination of AWS subscription advisable?
There is no concept of an "AWS Subscription". AWS is charged on-demand, which means you only pay when you use resources.
If you temporarily do not want the Amazon EC2 instance, you could:
Stop the instance, which is like turning off the power. You will not be charged for the instance, but you will still pay for the disk storage attached to the instance. You can simply Start the instance again when you wish to use it. You will only be charged while the instance is running. OR
Create an image of the instance, then terminate the instance. This will create an Amazon Machine Image (AMI), which contains a copy of the disks. You can then launch a new Amazon EC2 instance from the AMI when you wish to use it again. This is a lower-cost option compared to simply stopping the instance, but it takes more effort to stop/start.
It is quite common for companies to stop Amazon EC2 instances at night or over the weekend to reduce costs while they are not needed.
EDIT: Just thought of a third option. Will test it and be back. Not worth it; it would involve creating an image from the EC2 instance and then convert that image to a VM image, storing the VM image in S3. There may be some advantages to this, but I do not see them.
I think you have two options, both of them very reasonably priced. If you can separate the data from the operating system, then your best option would be to use an S3 bucket as a file system within the EC2 instance. Your EC2 instance would use this bucket to store all your "ML algorithms and data" and, possibly, even your "web-based interface". Whenever you decide that you no longer need the processing capacity of the EC2, you would unmount the S3 bucket file system from the EC2 instance and terminate that instance. After configuring an appropriate lifecycle rule for the S3 bucket, it would transition to Glacier, or even Glacier Deep Archive [you must considerer the different options of long term storage]. In the future, whenever you want to work with your data again, you would move your data from Glacier back to S3, create a new EC2 instance, install your applications, mount your S3 bucket as a file system and you would have access to all your data. I think this is your least expensive and shortest recovery time objective option. To implement this option, look at my answer to this question; everything you need to use an S3 bucket as a regular folder inside the EC2 instance is there.
The second option provides an integrated solution, meaning the operating system and the data stay together, and allows you to restore everything as it was the day you stopped processing your data. It's made up of the following cycle:
Shutdown your EC2 and make a note of all the specs [you need them further down].
Export your instance to a virtual image, vmdk for example, and store it in your S3 bucket. Something like this:
aws ec2 create-instance-export-task --instance-id i-0d54b0682aa3998a0
--target-environment vmware --export-to-s3-task DiskImageFormat=VMDK,ContainerFormat=ova,S3Bucket=sm-vm-backup,S3Prefix=vms
Configure an appropriate lifecycle rule for the S3 bucket so that it transitions to Glacier, or even Glacier Deep Archive.
Terminate the EC2 instance.
In the future you will need to implement the inverse, so you will need to restore the archived S3 Object [make sure you you can live with the time needed by AWS to do this]
Import the virtual image as an EC2 AMI, something like this [this is not complete - you will need some more options that you saved above]:
aws ec2 import-image --disk-containers
Format=ova,UserBucket="{S3Bucket=sm-vm-backup,S3Key=vmsexport-i-0a1c382e740f8b0ee.ova}"
Create an EC2 instance based on the image and you're back in business.
Obviously you should do some trial runs and even automate the entire process if it's something that will be done frequently. I have a feeling, based on what you said, that the first option is a better option, provided you can easily install whatever applications they use.
I'm assuming that you launched an EC2 instance from a base Amazon Machine Image and then added your own software and models to it. As opposed to launched an EC2 instance from an AWS Marketplace offering.
The simplest thing to do is to create an Amazon Machine Image (AMI) from your running EC2 instance. That will capture the current state of the instance and persist it in your AWS account. Then you can terminate the instance. Later, when you want to recreate it, launch a new instance, selecting the saved AMI instead of a standard AMI.
An alternative is to avoid the need to capture machine state at all, by using standard DevOps practices to revision-control everything you need to recreate the state of a running machine.
Note that there are costs associated with an AMI, though they are minimal ($0.05 per GB-month of data stored, for example).
I had contacted AWS customer care regarding this issue. Given below is the response I received. Please add your comments on which option might be good for me.
Note: I acknowledge the AWS customer care team for their help.
I understand that you require some information on cost saving for your
Instance since you will not be utilizing the service for a while.
To assist you with this I would recommend checking out the Instance
Stop/Start link here:
==>https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Stop_Start.html .
When you stop an Instance, you do not lose any data & you are not
charged for the resources any further. However please keep in mind
that you will still be charged for any EBS Storage Volumes attached to
the stopped Instance(s).
I also recommend checking out the below links on how you can reduce
your costs.
==>https://aws.amazon.com/premiumsupport/knowledge-center/reduce-aws-bill/
==>https://aws.amazon.com/blogs/compute/10-things-you-can-do-today-to-reduce-aws-costs/
That being said, please note that as I am in the billing department,
for the best assistance with the various plans you will require the
assistance of our Sales Team.
The Sales Team will be able to assist with ways to save while
maintaining your configurations.
You will be able to reach the Sales Team here:
==>https://aws.amazon.com/websites/contact-us/.
Once you have completed the details in the link, a member of the team
will be in touch with you at their soonest.
I am looking to build a POC for my current gig. It is an AWS ec2 with an application that needs to talk with internal AWS private IP's and several outside AWS public IP's.
After benchmarking, I am able to get all the costs for an hour of running this application except for the network data transfer cost. (All from the command line)
Does anyone have an idea on how to get current network costs from either the command line or web interface?
The only potential cost for data transfer between Amazon EC2 instances in the same region would be 1c/GB in & 1c/GB out if using Public IP addresses or going across AZs.
Traffic between instances via private IP addresses in the same AZ would be zero.
For reference, see https://www.duckbillgroup.com/resources/:
https://aws.amazon.com/blogs/mt/using-aws-cost-explorer-to-analyze-data-transfer-costs/
In Cost Explorer using filters you can analyze data transfer costs:
After your cost allocation tags have been activated, and your workloads have run for at least a day, you can use filters in Cost Explorer to analyze your costs over that period.
Sign in to AWS Cost Explorer at https://console.aws.amazon.com/cost-reports/home?#/
Choose Explore in the navigation pane, and then choose Cost and Usage.
Choose the date range for the period for which you want to see the costs, and choose the Apply button.
Choose Filters – Service, then EC2-instances, and then EC2-ELB.Next choose Apply filters.
To see the total EC2 data transfer cost:
Choose Filters, Usage Type Group, EC2: Data Transfer – inter-Availability Zone, Internet (Out), and Region to Region (Out), then choose Apply filters.
You can also choose each individual data transfer type by checking only the box for that type.
I have reservation of type t2.micro and us-ease-1d availability zone, also I have multiple instances running, including the one with mentioned type and zone.
As a result I expect billing for this one instance will take into account the reservation. The problem is that I haven't found any link between reservation and actual instances.
I've used aws ec2 describe-instances and aws ec2 describe-reserved-instances CLI commands but I wasn't able to find any link.
Is it possible to realise which billing approach will be used for each instance using Amazon SDK?
So f.e. I will see that some instance is linked to some reserved instance (reservation)
Is it possible to realise which billing approach will be used for each instance using Amazon SDK?
No, it isn't.
The "link" between reserved instances and running instances is not something the EC2 operational infrastructure knows about. It's all done after the fact, in billing.
Each hour, for a given instance type and availability zone placement, you're billed for your reserved instances (depending on the terms of the reservation, this happens whether you have this many instances running or not, though in some cases, the amount billed here is $0 since you've already paid). Then, if the number of running on-demand instances for that type and placement exceeds exceeds the number of reserved instances for that type and placement, the difference for that hour is billed at the on-demand rate.
So if you had purchased one reserved instance matching a certain spec and during a given hour you had two such instances running, it's not really the case that one of your instances "is the reserved instance" and the other one isn't. If you stop either one, then the next hour, the reserved instance pricing applies to the one that remains running... but EC2 can't tell you which is which and, in fact, the billing logic is such that it doesn't matter.
There isn't really a link between reservations and specific instances. Think of it more like a discount that gets applied to your bill, after you have incurred some instance charges.
You can use the Reserved Instance Utilization Report to see how your reservations have been applied to the instance hours you have been charged for.
I've been searching for details of this all day and cannot find any information. I'm using AWS and wanted to know if it is possible, with an amazon service or external, to monitor and log the EC2 compute usage of individual directories?
No, it is not possible to allocate Amazon EC2 usage costs to a fine-grained level, such as individual directory, application or user.
Amazon EC2 is charged on an hourly basis, with the cost varying by:
Instance Type (CPU, RAM, chipset)
Operating System (eg Windows instances include an hourly charge for Windows)
Billing type (On-Demand, Reserved Instance, Spot)
While the AWS billing system can allocate EC2 costs based on tags (eg Department), this is only done for a whole instance on an hourly basis.
Some ideas for fine-grained allocation of costs:
Record usage in CloudWatch via a Custom Metric, then extract data for billing purposes
Store usage information in log files and send them to CloudWatch Logs, then extra data for billing purposes
Track usage in a file, send it to Amazon S3 and then aggregate information each billing cycle
Each of these options would require custom coding and processing.