I am looking to build a POC for my current gig. It is an AWS ec2 with an application that needs to talk with internal AWS private IP's and several outside AWS public IP's.
After benchmarking, I am able to get all the costs for an hour of running this application except for the network data transfer cost. (All from the command line)
Does anyone have an idea on how to get current network costs from either the command line or web interface?
The only potential cost for data transfer between Amazon EC2 instances in the same region would be 1c/GB in & 1c/GB out if using Public IP addresses or going across AZs.
Traffic between instances via private IP addresses in the same AZ would be zero.
For reference, see https://www.duckbillgroup.com/resources/:
https://aws.amazon.com/blogs/mt/using-aws-cost-explorer-to-analyze-data-transfer-costs/
In Cost Explorer using filters you can analyze data transfer costs:
After your cost allocation tags have been activated, and your workloads have run for at least a day, you can use filters in Cost Explorer to analyze your costs over that period.
Sign in to AWS Cost Explorer at https://console.aws.amazon.com/cost-reports/home?#/
Choose Explore in the navigation pane, and then choose Cost and Usage.
Choose the date range for the period for which you want to see the costs, and choose the Apply button.
Choose Filters – Service, then EC2-instances, and then EC2-ELB.Next choose Apply filters.
To see the total EC2 data transfer cost:
Choose Filters, Usage Type Group, EC2: Data Transfer – inter-Availability Zone, Internet (Out), and Region to Region (Out), then choose Apply filters.
You can also choose each individual data transfer type by checking only the box for that type.
Related
I am trying to differentiate costs among my AWS EC2 instances.
I've tried everything mentioned via forums and AWS, I've created Tags to ID each EC2 instance, and I still cannot seem to get billing report telling me how much each instance is accumulating cost wise. They are all lumped together.
I created and enabled a tag for each, but still Cost Explorer is virtually useless, and only chunks these costs into instance types.
One other similar solution mentioned to create separate AWS accounts for each customer/client? Which in my case, is too late, if this is the only way to differentiate costs among the running EC2 instances.
You need to manually enable cost allocation tags in the console.
https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/activating-tags.html
To activate your tags
Sign in to the AWS Management Console and open the Billing and Cost Management console at https://console.aws.amazon.com/billing/.
In the navigation pane, choose Cost Allocation Tags.
Select the tags that you want to activate.
Choose Activate.
Good luck!
Is there a way to find the IP addresses of all connections to AWS RDS instances.
You can get connected Host IP from information_schema using MySQL Query.
select host from information_schema.processlist WHERE ID=connection_id();
Because in basic monitoring only these metrics are available.
For Monitoring, choose the option for how you want to view your metrics from these:
CloudWatch:
Shows a summary of DB instance metrics available from Amazon
CloudWatch. Each metric includes a graph showing the metric monitored
over a specific time span.
Enhanced monitoring:
Shows a summary of OS metrics available for a DB instance with
Enhanced Monitoring enabled. Each metric includes a graph showing the
metric monitored over a specific time span.
OS Process list:
Shows details for each process running in the selected instance.
So the option that can help you in this regards is Performance Insights.
Performance Insights:
If you are interested in detail matrics like HOST IPs, the number of connections, Slow queries and many more which can eliminate the need of DBA I believe as very good experience with Using Amazon RDS Performance Insights
Top activity can list any of the dimensions indicated at the top of the list. For Aurora PostgreSQL, Performance Insights currently supports listing top SQL, waits, hosts, and users.
analyzing-amazon-rds-database-workload-with-performance-insights
If you are using MariaDB:
select id, user, host, db, command, time, state, info, progress from information_schema.processlist;
Other options specific to AWS.
Turn on VPC Flowlogs , then use loginishgts to query historical
traffic to the instances.
Inspect the security group of the RDS, it may list the client(s) which is allowed to connect to it.
Turn on the general query log https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.MariaDB.html
I want help on understanding the AWS cost explorer graph to track the huge data transfer usage.
I have noticed the AWS account bills for jan, Feb and March (till current date) where it is showing a huge data transfer charge as a bill line item (image attached AWS Bill line Item)
regional data transfer - in/out/between EC2 AZs or using elastic IPs
or ELB
. Further i checked it in AWS Cost Explorer reports by applying Group by filter Region wise and can see that it has data transfer for each region but also for
No Region
, i am not able able to understand this bar graph (please see the image attached and yellow graph AWS Cost Explorer Reports Region Wise) with level "No Region".
A good starting point would be to enable VPC Flow Logs. VPC Flow Logs will show you where the source and destination of all the traffic within your VPC. After you've analysed the logs, you should have a good indication of where to begin investigating.
Out of context but adding it here as it might help you: for some services such as S3, you can enable object-level logging to help you understand what is accessing your objects, which could help you further understand why you're paying for data transfers.
You can avoid paying for data transfer charges between AWS services by using VPC Endpoints. VPC endpoints allow you to connect directly to the service rather than over the internet, which will avoid incurring extra data charges. More on VPC Endpoints here.
Is there a way to generate EC2 utilization report which contains below selections (I need all of them in one report)
EC2 instance name
Current status (stopped/running)
Availability Zone
Region
Cost incurred for the current month.
There is no single report that would contain that information.
If you want the "current" information about an instance, you'll need to make an API call, or use the AWS Command-Line Interface (CLI) aws ec2 describe-instances command. That will provide you with everything except cost.
You can use the Cost and Usage Report to identify costs associated with particular instances (by Instance ID, not name).
I've been searching for details of this all day and cannot find any information. I'm using AWS and wanted to know if it is possible, with an amazon service or external, to monitor and log the EC2 compute usage of individual directories?
No, it is not possible to allocate Amazon EC2 usage costs to a fine-grained level, such as individual directory, application or user.
Amazon EC2 is charged on an hourly basis, with the cost varying by:
Instance Type (CPU, RAM, chipset)
Operating System (eg Windows instances include an hourly charge for Windows)
Billing type (On-Demand, Reserved Instance, Spot)
While the AWS billing system can allocate EC2 costs based on tags (eg Department), this is only done for a whole instance on an hourly basis.
Some ideas for fine-grained allocation of costs:
Record usage in CloudWatch via a Custom Metric, then extract data for billing purposes
Store usage information in log files and send them to CloudWatch Logs, then extra data for billing purposes
Track usage in a file, send it to Amazon S3 and then aggregate information each billing cycle
Each of these options would require custom coding and processing.