I'm starting with Alexa development and AWS in general. I've subscribed for the free tier, created my skill, set a AWS Lambda function and done some little testing. I got nothing more running on AWS.
What I've noticed that except for AWS Lambda and Cloudwatch usage I got requests to AWS Key Management Service on my Billing Dashboard. I'm not using any environment variables as this was one of the reasons for KMS requests suggested by Google.
From my billing management report I got 3 times more KMS requests than to my Lambda (30 vs 9). I know this is small number but KMS got 20k requests in the free tier and Lambda got 1000000 and I just don't understand how this connects to each other.
Is AWS KMS required for Lambda operation? What is it used for?
Many AWS services are using KMS to manage keys and access to keys while keeping them under your control.
The full list is documented here https://docs.aws.amazon.com/kms/latest/developerguide/service-integration.html
Pricing of KMS is per keys that you create and manage. https://aws.amazon.com/kms/pricing/
Keys automatically created by AWS Services are for free.
I just checked my bill and I am not charged for KMS at all.
I do suggest you to enable CloudTrail logs on your account to understand where the KMS calls you're seeing are originated from.
To query Cloudtrail logs, you can make a simple SQL query on Athena.
Doc to setup Athena for Cloudtrail : https://docs.aws.amazon.com/athena/latest/ug/cloudtrail-logs.html
SQL Query to analyze kms calls :
SELECT eventtime,
useridentity.type,
eventsource,
eventname,
sourceipaddress,
eventtime
FROM "default"."cloudtrail_logs_logs_sst_cloudtrail"
WHERE eventsource = 'kms.amazonaws.com' AND
eventtime BETWEEN '2018-07-01' AND '2018-07-31' ;
Related
I'm the administrator of an AWS account that has 4 users. One of the users is racking up higher-than-expected costs.
I checked the Cost Explorer, but could not seem to configure it to view individual users.
As an administrator in AWS, how do I see all of the services this particular user has been using during a given period of time (e.g. the last 12 months)?
Thanks!
AWS resources are associated with an AWS Account, not a specific user.
If a user has the necessary permissions to create resources (eg an Amazon EC2 instance), then the instance is launched in the AWS Account, but there is no link back to the user that requested the resource.
You can, however, use AWS CloudTrail:
AWS CloudTrail is an AWS service that helps you enable operational and risk auditing, governance, and compliance of your AWS account. Actions taken by a user, role, or an AWS service are recorded as events in CloudTrail. Events include actions taken in the AWS Management Console, AWS Command Line Interface, and AWS SDKs and APIs.
It will show all API calls made by the user, including the Action ('launch an EC2 instance'), their IP address, timestamp, etc. Operations in the AWS management console will also be shown, since it makes API calls on behalf of the user.
CloudTrail keeps a history of the past 90 days, but you can create specific 'Trails' that retain data permanently. If you have not done this, then it will not be possible to see what they did prior to 90 days ago.
I wanted to know if there was a way to track alerts or audit anything that happens with the AWS account like who changed what and why. I did find this https://docs.aws.amazon.com/opensearch-service/latest/developerguide/audit-logs.html where they use a comand line for enabling audit logs on an existing domain: aws opensearch update-domain-config --domain-name my-domain --log-publishing-options "AUDIT_LOGS={CloudWatchLogsLogGroupArn=arn:aws:logs:us-east-1:123456789012:log-group:my-log-group,Enabled=true}" but this is in regard to Amazon OpenSearch Service which I believe is only free for 12 months if you haven't used already. AWS Audit Manager. I am aware there are services that can do this but require a fee and I wanted to know if there were any free options
From the AWS documentation:
With AWS CloudTrail, you can monitor your AWS deployments in the cloud by getting a history of AWS API calls for your account, including API calls made by using the AWS Management Console, the AWS SDKs, the command line tools, and higher-level AWS services. You can also identify which users and accounts called AWS APIs for services that support CloudTrail, the source IP address from which the calls were made, and when the calls occurred. You can integrate CloudTrail into applications using the API, automate trail creation for your organization, check the status of your trails, and control how administrators turn CloudTrail logging on and off.
AWS Config provides a detailed view of the resources associated with your AWS account, including how they are configured, how they are related to one another, and how the configurations and their relationships have changed over time.
Basically, AWS CloudTrail keeps a log of API calls (requests to AWS to do/change stuff), while AWS Config tracks how individual configurations have changed over time (for a limited range of resources, such as Security Group rule being changed).
Is there a way to find the number of EC2 instances which were launched in last 1/2/3/4/5 or 6 months in all regions? (running and terminated).
From a similar question as below, I can only get the current status (running|stopped|terminated) but not anything from past months.
How to see all running Amazon EC2 instances across all regions?
Please advise. This is purely for audit purpose.
Thanks in advance.
AWS CloudTrail makes it easier to ensure compliance with internal policies and regulatory standards by providing a history of activity in your AWS account.
AWS have an option to view Event History if you have CloudTrail enabled. Please go through this AWS page to view clear instructions.
If you like to use AWS CLI then this documentation provides all the details.
I would recommend a combination of CloudTrail logs stored in S3 and Athena to do the query. The problem with CloudTrail alone is that you have a three month window before logs roll off. Your requirements include as far out as six months.
To deliver log files to an S3 bucket, CloudTrail must have the required permissions, and it cannot be configured as a Requester Pays bucket. CloudTrail automatically attaches the required permissions to a bucket when you create an Amazon S3 bucket as part of creating or updating a trail in the CloudTrail console.
To setup Athena you can configure through the CloudTrail Console:
Open the CloudTrail console at https://console.aws.amazon.com/cloudtrail/
In the navigation pane, choose Event history.
Choose Create Athena table.
For Storage location, use the down arrow to select the Amazon S3 bucket where log files are stored for the trail to query.
Choose Create table. The table is created with a default name that includes the name of the Amazon S3 bucket.
Then you can run a query similar to this in Athena:
SELECT eventname,
useridentity.principalid,
awsregion,
eventtime
FROM cloudtrail_logs
WHERE eventtime >= '2021-02-01T00:00:00Z'
AND eventtime < '2021-08-30T00:00:00Z'
AND (eventname ='RunInstances')
References
Create S3 Bucket Policy for CloudTrail
Query CloudTrail logs with Athena
Athena Search CloudTrail Logs
Is there a way to track or log what AWS KMS is using which key for which purposes.
I noticed that in some accounts that we do not use KMS, there are still KMS activities in the billing. In fact, there are activities in every region.
While these activities are in the free tier, it would be transparent to know what are these activities. Especially so since "keys" are involved.
You can log all KMS activites with CloudTrail.
Logging AWS KMS API Calls Using AWS CloudTrail
I am trying to see which user was responsible for changes in S3 (at buckets level).
I could not find a audit trail for actions done at S3 bucket level or EC2 who created instances. Beanstalk has a log of the actions the machine performed, but not which user.
Is there a way around AWS that we can see this information in IAM or any other location ?
P.S: I am not interested to know about S3 log buckets which provide access logs
Update
AWS has just announced AWS CloudTrail, finally making auditing API calls available as of today (and for free), see the introductory post AWS CloudTrail - Capture AWS API Activity for details:
Do you have the need to track the API calls for one or more AWS
accounts? If so, the new AWS CloudTrail service is for you.
Once enabled, AWS CloudTrail records the calls made to the AWS APIs
using the AWS Management Console, the AWS Command Line Interface
(CLI), your own applications, and third-party software and publishes
the resulting log files to the Amazon S3 bucket of your choice.
CloudTrail can also issue a notification to an Amazon SNS topic of
your choice each time a file is published. Each call is logged in JSON
format for easy parsing and processing.
Please note the following (temporary) constraints:
Not all services are covered yet, though the most important ones are included in the initial release already and AWS plans to add support for additional services over time.
Update: AWS has recently added Seven New Services, and another one today, see below.
More importantly, not all regions are supported yet (right now the US East (Northern Virginia), and US West (Oregon) Regions only), though AWS will be adding support for additional Regions as quickly as possible.
Update: AWS has just added More Locations and Services, quickly approaching coverage of their entire Global Infrastructure indeed.
Initial Answer
This is a long standing feature request, but unfortunately AWS does not provide (public) audit trails as of today - the most reasonable way to add this feature would probably be a respective extension to AWS Identity and Access Management (IAM), which is the increasingly ubiquitous authentication and authorization layer for access to AWS resources across all existing (and almost certainly future) Products & Services.
Accordingly there are a few respective answers provided within the IAM FAQs along these lines:
Will AWS Identity and Access Management administrative actions be logged to an audit trail?:
No. This is planned for a future release.
Will user actions in AWS services be logged to an audit trail?
No. This is planned for a future release.
Current pricing for a single CloudTrail is free.
1. Enable CloudTrail
Use the CloudTrail dashboard and send all events to an S3 bucket, e.g. my-cloudtrail
2. Go Through the Results
The CloudTrail dashboard let's you do some cursory searches, but if you have many thousands of events, it's a pain to use.
Let's say I want actions for user foo_user, I just use the CLI tool:
mkdir -p /tmp/cloudtrail
cd /tmp/cloudtrail
aws s3 sync s3://mc10-cloudtrail .
cd AWSLogs
zcat `find . -type f` | jq '.Records[] | "\(.eventName) \(.userIdentity.userName)"' | grep food_user | sort | uniq
Example Output:
"CreateGrant foo_user"
"DescribeInstances foo_user"
"GetConsoleOutput foo_user"
"ModifyInstanceAttribute foo_user"
"StartInstances foo_user"
"StopInstances foo_user"
Note: S3 data events are billed differently in CloutTrail, but this is somewhat redundant, because you can just enable logging on your S3 bucket and grep those logs, or point them at Logstash/Kibana.