Hi we have about hundred instances in AWS but we don't have a clue which uses what.
We'd need to be able to view costs per instance.
We've tried tags etc everything possible in settings but without any luck. Do we need to make linked account to or something to each instance? Or should we just move away from aws? Or is there some external service for this?
You can calculate cost using AWS Cost Calculator: https://calculator.s3.amazonaws.com/index.html
Related
I go into billing and filter by project & group by SKU. This gives me the individual services that I'm being charged for (view below). I can see that Compute Engine is costing me £5.59 per month. If I go into Compute Engine I don't have any instances. In this particular example I discovered that had a Serverless VPC Connector running and I think that is using the Service but it is just a guess.
How do I find out what specific product instances that I have enabled are costing me money? How, in this example, would I find out from Billing that it is the Serverless VPC Connector that is costing me money and linked to that Compute Engine instance?
(I've had other situations where I'm being billed for an IP address but not the name of that IP address or what products / instances of products are consuming the IP address).
Interesting...
It looks like indeed looking at https://cloud.google.com/vpc/pricing#serverless-vpc-pricing
serverless VPC connectors could be billed as e2-micro
While in many cases you will see billing report actually try to associate your billing to human understandable service like so (under service and/or SKU columns)
it doesn't seem to be always the case.
One way to try to resolve that I guess would be to follow this path
https://cloud.google.com/blog/topics/cost-management/use-labels-to-gain-visibility-into-gcp-resource-usage-and-spending
Following instructions and once you get your billing exported to big query you could run the following:
select
sku.id,
l.key
from
YOUR_PROJECT.YOUR_BILLING_EXPORT_DATASET.YOUR_BILLING_TABLE
,UNNEST(labels) as l
where
sku.id = '9FE0-8F60-A9FO' # your SKU id in question
and labels should give you a better idea of whats what.
This should give you an overall direction, perhaps there is a way to parse this out without exporting to BQ (first) and querying it (second) as this may also cost you money.
PS.
Labels are always a good idea when you create services/resources explicitly.
I would like to use spot instances in my cluster. I was trying to create a node group with a spot priced launch template when I discovered it's not possible to specify instance market options like maximum price.
https://github.com/hashicorp/terraform-provider-aws/issues/15118
This seems like a pretty major limitation seeing as spot instances can spike up in price to orders of magnitude more than on-demand.
https://devops.stackexchange.com/questions/893/why-is-the-aws-ec2s-spot-price-greater-than-the-on-demand-price
Is there really no way around this? Is no one using spot instances in EKS?
I had a quick look for you in the Python SDK documentation. I didn't find any mention about what you are looking for in the API https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/eks.html#EKS.Client.create_nodegroup I guess the same applies in terms of IaC. I would suggest you reach out to your Technical Account Manager (TAM) or AWS Support and let them know about your needs. They will be able to create a feature request for you.
I created my AWS account and got 12 months free plan. Then I went to the teg editor to check all my running services and there were 165 unnecessary running services. Maybe someone had the same problem? Is this ok and I don't have to pay for it?)
Screenshot
Just because they exist, does not mean they are running.
Those look like to be the default VPCs that are created by AWS in every region by default for every AWS account.
If you didn't create them, don't worry - you aren't being charged for them.
AWS does not provide any default resources that charge you money.
I would like to know a system by which I can keep track of multiple aws accounts, somewhere around 130+ accounts with each account containing around 200+ servers.
I wanna know methods to keep track of machine failure, service failure etc.
I also wanna know methods by which I can automatically turn up a machine if the underlying hardware failed or the machine terminated while on spot.
I'm open to all solutions including chef/terraform automation, healing scripts etc.
You guys will be saving me a lot of sleepless nights :)
Thanks in advance!!
This is purely my take on implementing your problem statement.
1) Well.. for managing and keeping track of multiple aws accounts you can use AWS Organization. This will help you manage centrally with one root account all the other 130+ accounts. You can enable consolidated billing as well.
2) As far as keeping track of failures... you may need to customize this according to your requirements. For example: You can build a micro service on top of docker containers or ecs whose sole purpose is to keep track of failures, generate a report and push to s3 on a daily basis.You can further create a dashboard using AWS quicksight out of this reports in S3.
There can be another micro service which will rectify the failures. It just depends on how exhaustive and fine grained you want your implementation to be.
3) For spawning instances when spot instances are terminated, it can be achieved through you simple autoscaling configurations. Here are some of the articles you may want to go through which will give you some ideas:
Using Spot Instances with On-Demand instances
Optimizing Spot Fleet+Docker with High Availability
AWS Organisations are useful for management. You can also look at multiple account billing strategy and security strategy. A shared services account with your IAM users will make things easier.
Regarding tracking failures you can set up automatic instance recovery using CloudWatch. CloudWatch can also have alerts defined that will email you when something happens you don't expect, though setting them up individually could be time consuming. At your scale I think you should look into third party tools.
As amazon charges for an hour even when I use it for minutes. It is getting little expensive to do my school projects or play around with EMR. As there are micro instances free I want to make use of these to run my mapreduce jobs, there seem to be no option doing so any help in this regard would be great.
Also if that is totally not posibble I wanna know how do I pick any running instance (probably small instance which EMR gives an option to select via console) for mapreduce job? I am basically planning to run few small instances and get all my small mapreduce jobs use these instances this way I can make most of the money I pay.
Thanks in advance :)
I myself fired-up some EC2 instances and tried to run a map reduce job using Elastic MapReduce console and I was not given any options to use the instances already up. Amazon charging on per hour basis is turning out to be bad thing to me at-least.
Please add more info if I am missing or wrong in any way.
PS: I chose to answer my own question as I did not see any help coming and thought this might someday be helpful to someone experimenting with AWS.