I have a bucket called "my-bucket" in account A and I'm trying to determine it's location from account B.
I have the following bucket policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::11122233344:root"
},
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::my-bucket/*"
},
{
"Sid": "2",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::11122233344:root"
},
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::my-bucket"
}
]}
I've also stumbled across this documentation which contains the same policy: https://docs.aws.amazon.com/AmazonS3/latest/dev/example-walkthroughs-managing-access-example2.html
Now using account B, I can successfully execute the command aws s3 ls s3://my-bucket, however with aws s3api get-bucket-location --bucket my-bucket I get the access denied message.
According to the documentation: https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETlocation.html
To use this implementation of the operation, you must be the bucket
owner.
When in doubt always refer to the AWS API documentation it usually specifies the permission needed for the given action.
Related
I am trying to export logs from one of my CloudWatch log groups into Amazon S3, using AWS console.
I followed the guide from AWS documentation but with little success. My organization does not allow me to manage IAM roles/policies, however I was able to find out that my role is allowed all log-related operations (logs:* on all resources within the account).
Currently, I am stuck on the following error message:
Could not create export task. PutObject call on the given bucket failed. Please check if CloudWatch Logs has been granted permission to perform this operation.
My bucket policy is set in the following way:
{
[
...
{
"Sid": "Cloudwatch Log Export 1",
"Effect": "Allow",
"Principal": {
"Service": "logs.eu-central-1.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::my-bucket"
},
{
"Sid": "Cloudwatch Log Export 2",
"Effect": "Allow",
"Principal": {
"Service": "logs.eu-central-1.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::my-bucket/*"
}
]
}
Prior to editing bucket policy, my error message had been
Could not create export task. GetBucketAcl call on the given bucket failed. Please check if CloudWatch Logs has been granted permission to perform this operation.
but editing the bucket policy fixed that. I would expect allowing PutObject to do the same, but this has not been the case.
Thank you for help.
Ensure when exporting the data you configure the following aptly
S3 bucket prefix - optional This would be the object name you want to use to store the logs.
While creating the policy for PutBucket, you must ensure the object/prefix is captured adequately. See the diff for the PutBucket statement Resource:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "s3:GetBucketAcl",
"Effect": "Allow",
"Resource": "arn:aws:s3:::my-exported-logs",
"Principal": { "Service": "logs.us-east-2.amazonaws.com" }
},
{
"Action": "s3:PutObject" ,
"Effect": "Allow",
- "Resource": "arn:aws:s3:::my-exported-logs/*",
+ "Resource": "arn:aws:s3:::my-exported-logs/**_where_i_want_to_store_my_logs_***",
"Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } },
"Principal": { "Service": "logs.us-east-2.amazonaws.com" }
}
]
}
Please check this guide Export log data to Amazon S3 using the AWS CLI
Policy's looks like the document that you share but slight different.
Assuming that you are doing this in same account and same region, please check that you are placing the right region ( in this example is us-east-2)
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "s3:GetBucketAcl",
"Effect": "Allow",
"Resource": "arn:aws:s3:::my-exported-logs",
"Principal": { "Service": "logs.us-east-2.amazonaws.com" }
},
{
"Action": "s3:PutObject" ,
"Effect": "Allow",
"Resource": "arn:aws:s3:::my-exported-logs/*",
"Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } },
"Principal": { "Service": "logs.us-east-2.amazonaws.com" }
}
]
}
I think that bucket owner full control is not the problem here, the only chance is the region.
Anyway, take a look to the other two examples in case that you were in different accounts/ using role instead user.
This solved my issue, that was the same that you mention.
One thing to check is your encryption settings. According to https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3ExportTasksConsole.html
Exporting log data to Amazon S3 buckets that are encrypted by AWS KMS is not supported.
Amazon S3-managed keys (SSE-S3) bucket encryption might solve your problem. If you use SSE-KMS, Cloudwatch can't access your encryption key in order to properly encrypt the objects as they are put into the bucket.
I had the same situation and what worked for me is to add the bucket name itself as a resource in the Allow PutObject Sid, like:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowLogsExportGetBucketAcl",
"Effect": "Allow",
"Principal": {
"Service": "logs.eu-west-1.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::my-bucket"
},
{
"Sid": "AllowLogsExportPutObject",
"Effect": "Allow",
"Principal": {
"Service": "logs.eu-west-1.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": [
"my-bucket",
"my-bucket/*"
]
}
]
}
I also believe that all the other answers are relevant, especially using the time in milliseconds.
I have two buckets:
"source-bucket" (in AWS account 88888888).
"destination-bucket" (in AWS account 99999999)
Both buckets have versioning enabled and are located in the same region (eu-west-1).
In the source bucket, I've created a Replication-rule with the following settings:
I opted for automatic role creation, which created a role with the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:ListBucket",
"s3:GetReplicationConfiguration",
"s3:GetObjectVersionForReplication",
"s3:GetObjectVersionAcl",
"s3:GetObjectVersionTagging",
"s3:GetObjectRetention",
"s3:GetObjectLegalHold"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::source-bucket",
"arn:aws:s3:::source-bucket/*",
"arn:aws:s3:::destination-bucket",
"arn:aws:s3:::destination-bucket/*"
]
},
{
"Action": [
"s3:ReplicateObject",
"s3:ReplicateDelete",
"s3:ReplicateTags",
"s3:ObjectOwnerOverrideToBucketOwner"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::source-bucket/*",
"arn:aws:s3:::destination-bucket/*"
]
}
]
}
According to the documentation found here https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication-walkthrough-2.html
, I've added a bucket policy to "destination-bucket", which looks as follows:
{
"Version": "2012-10-17",
"Id": "",
"Statement": [
{
"Sid": "Set permissions for objects",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::88888888:role/service-role/auto-created-role"
},
"Action": [
"s3:ReplicateObject",
"s3:ReplicateDelete"
],
"Resource": "arn:aws:s3:::destination-bucket/*"
},
{
"Sid": "Set permissions on bucket",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::88888888:role/service-role/auto-created-role"
},
"Action": [
"s3:GetBucketVersioning",
"s3:PutBucketVersioning"
],
"Resource": "arn:aws:s3:::destination-bucket"
}
]
}
But, when I add a file to the source bucket, nothing seems happens.
Does anyone have any idea what could be wrong here?
The AWS docs aren't the best here. From your pictures I see you have enabled the setting "change object ownership to destination bucket owner" (as most people would).
However, this requires an extra permission on the destination side give them ownership. s3:ObjectOwnerOverrideToBucketOwner
The following policy should work for you
{
"Version": "2012-10-17",
"Id": "",
"Statement": [
{
"Sid": "Set permissions for objects",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::88888888:role/service-role/auto-created-role"
},
"Action": [
"s3:ReplicateObject",
"s3:ReplicateDelete",
"s3:ObjectOwnerOverrideToBucketOwner"
],
"Resource": "arn:aws:s3:::destination-bucket/*"
},
{
"Sid": "Set permissions on bucket",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::88888888:role/service-role/auto-created-role"
},
"Action": [
"s3:GetBucketVersioning",
"s3:PutBucketVersioning"
],
"Resource": "arn:aws:s3:::destination-bucket"
}
]
}
To debug this issue I used aws s3api head-object --bucket <bucket> --key <prefix> --query ReplicationStatus to see the replication failed and then I added s3:* permission on the destination side to see if it was a permission issue. Which in this case it was.
Check if this helps.
By default, Amazon S3 doesn't replicate objects that are stored at rest using server-side encryption with AWS Key Management Service (AWS KMS) customer master keys (CMKs). To replicate encrypted objects, you modify the bucket replication configuration to tell Amazon S3 to replicate these objects.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication-walkthrough-4.html
I'm trying to allow a set of users in a group access to a role through which they can upload objects to an s3 bucket.
The group as the policy:
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::ACCOUNTID:role/Clinic_Sync"
}
}
The role "Clinic_Sync" has the policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "SyncReqs",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::*/*"
},
{
"Sid": "SyncReqs2",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::*"
}
]
}
The bucket has the policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTID:role/Clinic_Sync"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::mydata"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTID:role/Clinic_Sync"
},
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::mydata/*"
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::mydata",
"arn:aws:s3:::mydata/*"
],
"Condition": {
"StringNotLike": {
"aws:userId": [
"ADMINUSERID:*",
"ACCOUNTNO"
]
}
}
}
]
}
The idea being that no one can access the bucket except through assuming this role (other than the admin). I have created the credentials files as follows:
[default]
aws_access_key_id = ACCESSID1
aws_secret_access_key = SECRETKEY1
[csync]
role_arn = arn:aws:iam::ACCOUNTID:role/Clinic_Sync
source_profile = default
And the config file:
[default]
output = json
region = eu-west-2
[profile csync]
role_arn = arn:aws:iam::ACCOUNTID:role/Clinic_Sync
source_profile = default
The bucket policy seems to work, as running the command "aws s3 cp hello.txt s3://mydata" gives the error: Upload failed. An error occured when calling the PutObject operation: Access Denied.
But when I try to use the role, using the command "aws s3 cp hello.txt s3://run3d-data --profile csync", it gives this error:
upload failed: .\hello.txt to s3://mydata/hello.txt An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:iam::ACCOUNTID:user/TestAcc2 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::ACCOUNTID:role/Clinic_Sync
I've been searching the web for an answer for ages and can't find any answers. The aws documentation is frankly unintelligible to me. If anyone can help me find a solution to this I'd be much appreciated as I'm tearing my hair out here.
To reiterate, I just want the users in a particular group to have access to a role that grants them permission to use an s3 bucket, but block all other access to the bucket.
Your bucket policy seems to say: "Deny access to the bucket unless aws:userId is a given Admin User ID or Account Number. It does not reference the Role.
Therefore, accessing the bucket via the Role will be denied. This is because Deny always overrides Allow.
Writing policies with Deny can be quite difficult, as seen in this situation.
If you really want to keep a bucket secure, it is easier to put the bucket in a separate AWS Account and only grant cross-account access to the entities that should have access. This way, no Deny policy is required.
If you receive a not authorised to perform sts:AssumeRole error, make sure the Trust Policy grants access to users by selecting the Another AWS account option when creating the role. The policy should look similar to:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111111111111:root"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
I want to use Glue Crawler to crawl data from an S3 bucket. This S3 bucket is in another AWS account. Let's call is Account A. My Glue Crawler is in Account B.
I have created a Role in Account B and called it AWSGlueServiceRole-Reporting
I have attached the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BucketAccess",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::AccountAbucketname"
]
},
{
"Sid": "ObjectAccess",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": [
"arn:aws:s3:::AccountABucketName/Foldername/*"
]
}
]
}
And also AWSGlueServiceRole policy.
In Account A that has the S3 bucket, I've attached the following bucket policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountB:role/AWSGlueServiceRoleReporting”
},
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::AccountABucketName"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountB:role/AWSGlueServiceRoleReporting”
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::AccountABucketName/FolderName/*"
}
]
}
I'm able to run a Glue Crawler in Account B on this S3 bucket and it created Glue Tables. But when I try to query them in Athena, I get Access Denied.
Can anybody help me how to query it in Athena??
When Amazon Athena queries run, they use the permissions of the user that is running the query.
Therefore, you will need to modify the Bucket Policy on the bucket in Account A to permit access by whoever is running the query in Amazon Athena:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::AccountB:role/AWSGlueServiceRoleReporting",
"arn:aws:iam::AccountB:user/username"
]
},
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::AccountABucketName"
},
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::AccountB:role/AWSGlueServiceRoleReporting",
"arn:aws:iam::AccountB:user/username"
]
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::AccountABucketName/FolderName/*"
}
]
}
The user will also need sufficient S3 permissions (on their IAM User) to access that S3 bucket. (For example, having s3:ListBucket and s3:GetObject on S3 buckets. They likely already have this, but it is worth mentioning.)
This is different to AWS Glue, which uses an IAM Role. Athena does not accept an IAM Role for running queries.
I try to copy a file from my local to S3 bucket using AWS - CLI.
this is my command:
aws s3 cp /path/to/local/file s3://bucket-name
but the message I get is :
An error occured (AccessDenied) when calling the PutObject operation: Access Denied
I've set the policy in my s3 bucket to accept everything, like this:
{
"Id": "BucketPolicy",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllAccess",
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::MyBucket",
"arn:aws:s3:::MyBucket/*"
],
"Principal": "*"
}
]
}
What could be the reason I'm getting this error?
Any help would be appreciated!
Worth checking your command, For copy, you need to specify the source file. The destination file can be current directory.aws cli
aws s3 cp test.txt s3://mybucket/test2.txt
Basic bucket policy aws s3 docs
This policy allows all users to retrieve any object in MyBucket. It also grants put and delete permission to the root user of the AWS account 1234-5678-9012.
{
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::MyBucket/*"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:root"
},
"Action": [
"s3:DeleteObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::MyBucket/*"
}
]
}