I used Terraform to create a new S3 bucket for getting logs automatically from three different existing S3 bucket, next step I want to make the most use of these logs, getting various notification, e.g if someone created/deleted/modified a S3 bucket, relevant users will get notified about the events. Because at the moment all the logs are a mess, and the filename of the logs are all meaningless, I've been messing around with CloudWatch and CloudTrail, not sure what is the right way to do this.Can someone help me, many thanks.
Related
I'm trying to trigger AWS Step Function whenever a new file is uploaded on S3 bucket. I'm using Cloudwatch rules to do this but I'm getting this warning
I tried to follow AWS documentation link "https://docs.aws.amazon.com/step-functions/latest/dg/tutorial-cloudwatch-events-s3.html#tutorial-cloudwatch-events-s3-step-1" but state machine did not invoked.
Can anyone tell me what exactly I'm doing wrong?
EDIT
I created this trail and region is Ohio
I found the issue, we need to enable data events as well to get API calls for S3. It was not mentioned in above AWS Document.
Scenario:
My client have 80+ S3 Buckets and 1000+ applications is running in their AWS account. I want to get the list of IAM users/roles who are accessing the objects in all the S3 Buckets.
Method 1: Initially I tried to fetch it from CloudTrail Event History, but no luck.
From the above image, you can see CloudTrail is failing to log the object level logging.
Method 2: I created a CloudTrail Trails to log the activities. But it captures all management level activities happening through out the account which makes me hard to find the S3 logs alone(I already mentioned that there is 80+ Buckets & 1000+ applications in the account).
Method 3: S3 Server Access Log: If I enable this option, it creates log entry for every action happening to the objects. (that is: When I attempt to read a log file, it creates an another log. It keeps on doubling the count of logs)
If anyone have a solution to find the list of IAM users/roles who are accessing the S3 bucket objects and in an effective way, please help me.
Thanks in advance.
For each bucket, configure object-level logging.
Once that is complete, you can use the CloudTrail API to filter events and extract IAM identities making the requests.
aws cloudtrail lookup-events --lookup-attributes AttributeKey=ResourceType,AttributeValue=AWS::S3::Object --query Events[*].Username
I have a usecase where I want to put data into an S3 bucket, for it to read later, by another account. I only want the other account to be able to read the file in S3, and once they have read it, I will then delete the file myself.
I have been reading the S3 documentation, and cannot see they cover this usecase: of sending a notification when a file in an S3 bucket is read ?
Can anyone help, or suggest an alternative workflow ? I have been looking at AWS SNS and was wondering if that would be a better solution ?
You could use CloudTrail and CloudWatch Events to enable this workflow.
By default S3 API calls are not logged so you'd want to enable that following the instructions here.
Then enable a CloudWatch event rule for the Simple Storage Service where the "GetObject" operation occurs.
Have this event invoke a Lambda function that will remove the object.
More information available here.
I'm new to AWS Connect and I tried to save recordings + trigger lambda in an incoming flow.
Here's the diagram:
However, I can't see any files in the bucket or even any logs in lambda. Is there something wrong in the diagram? I also enabled saving recording to s3 with the default bucket AWS Connect suggested. I can see the bucket but it's always empty.
Thanks!
****EDIT****
The logs of /aws/connect/personal is also empty
To enable your recordings from the dashboard in the Set block pull Set logging behavior onto your contact flow and touch enable. The lambda follow the instructions laid out here: https://docs.aws.amazon.com/connect/latest/adminguide/connect-lambda-functions.html. You need to be sure that you generate the permissions for Connect to access the lambda in your terminal utilizing aws-cli an example is given review the parameters that you need to change from the code provided.
Lastly be sure to save, and then save and publish and in a few minutes your changes should take effect.
I am new to AWS CloudTrail. I have gone through number of aws docs and unable to figure out how to read cloudtrails last 7 days logs through program without configuring trail or without getting charged.
I want to write a java program which will read audit logs from aws and process those logs. I know we can create trail and we can read logs from aws s3 bucket using program, but I don't know how to read logs using aws sdk api for last 7 days like how we get logs on aws console ( we can read last 7 days audit logs free of cost.).
We can get this done using - cloudtrail-processing-library, but the properties/conf file for this lib requires sqs url as argument which i don't have, rather I don't know.
Please assist me so that I can write java program.
Regards,
Sachin
You can use the lookupEvents API in cloudtrail for getting the list of events (any create/update/delete operations).
http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/cloudtrail/AWSCloudTrail.html#lookupEvents-com.amazonaws.services.cloudtrail.model.LookupEventsRequest
The logs are stored in a S3 bucket and you can use AWS athena to process and query the logs if you want, so you don't have to write a Java program. If you do then that program will need IAM privileges to read from the S3 bucket that stores the logs.
AWS Athena
How to find your Cloud trail logs
Java code examples on S3 bucket objects
Java Cloudtrail SDK reference