I am trying to export logs from one of my CloudWatch log groups into Amazon S3, using AWS console.
I followed the guide from AWS documentation but with little success. My organization does not allow me to manage IAM roles/policies, however I was able to find out that my role is allowed all log-related operations (logs:* on all resources within the account).
Currently, I am stuck on the following error message:
Could not create export task. PutObject call on the given bucket failed. Please check if CloudWatch Logs has been granted permission to perform this operation.
My bucket policy is set in the following way:
{
[
...
{
"Sid": "Cloudwatch Log Export 1",
"Effect": "Allow",
"Principal": {
"Service": "logs.eu-central-1.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::my-bucket"
},
{
"Sid": "Cloudwatch Log Export 2",
"Effect": "Allow",
"Principal": {
"Service": "logs.eu-central-1.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::my-bucket/*"
}
]
}
Prior to editing bucket policy, my error message had been
Could not create export task. GetBucketAcl call on the given bucket failed. Please check if CloudWatch Logs has been granted permission to perform this operation.
but editing the bucket policy fixed that. I would expect allowing PutObject to do the same, but this has not been the case.
Thank you for help.
Ensure when exporting the data you configure the following aptly
S3 bucket prefix - optional This would be the object name you want to use to store the logs.
While creating the policy for PutBucket, you must ensure the object/prefix is captured adequately. See the diff for the PutBucket statement Resource:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "s3:GetBucketAcl",
"Effect": "Allow",
"Resource": "arn:aws:s3:::my-exported-logs",
"Principal": { "Service": "logs.us-east-2.amazonaws.com" }
},
{
"Action": "s3:PutObject" ,
"Effect": "Allow",
- "Resource": "arn:aws:s3:::my-exported-logs/*",
+ "Resource": "arn:aws:s3:::my-exported-logs/**_where_i_want_to_store_my_logs_***",
"Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } },
"Principal": { "Service": "logs.us-east-2.amazonaws.com" }
}
]
}
Please check this guide Export log data to Amazon S3 using the AWS CLI
Policy's looks like the document that you share but slight different.
Assuming that you are doing this in same account and same region, please check that you are placing the right region ( in this example is us-east-2)
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "s3:GetBucketAcl",
"Effect": "Allow",
"Resource": "arn:aws:s3:::my-exported-logs",
"Principal": { "Service": "logs.us-east-2.amazonaws.com" }
},
{
"Action": "s3:PutObject" ,
"Effect": "Allow",
"Resource": "arn:aws:s3:::my-exported-logs/*",
"Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } },
"Principal": { "Service": "logs.us-east-2.amazonaws.com" }
}
]
}
I think that bucket owner full control is not the problem here, the only chance is the region.
Anyway, take a look to the other two examples in case that you were in different accounts/ using role instead user.
This solved my issue, that was the same that you mention.
One thing to check is your encryption settings. According to https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3ExportTasksConsole.html
Exporting log data to Amazon S3 buckets that are encrypted by AWS KMS is not supported.
Amazon S3-managed keys (SSE-S3) bucket encryption might solve your problem. If you use SSE-KMS, Cloudwatch can't access your encryption key in order to properly encrypt the objects as they are put into the bucket.
I had the same situation and what worked for me is to add the bucket name itself as a resource in the Allow PutObject Sid, like:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowLogsExportGetBucketAcl",
"Effect": "Allow",
"Principal": {
"Service": "logs.eu-west-1.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::my-bucket"
},
{
"Sid": "AllowLogsExportPutObject",
"Effect": "Allow",
"Principal": {
"Service": "logs.eu-west-1.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": [
"my-bucket",
"my-bucket/*"
]
}
]
}
I also believe that all the other answers are relevant, especially using the time in milliseconds.
I would like to create the AWS Config access grant to the Amazon S3 Bucket and the policy is provided below that I write according to the link https://docs.aws.amazon.com/config/latest/developerguide/s3-bucket-policy.html:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSConfigBucketPermissionsCheck",
"Effect": "Allow",
"Principal": {
"Service": [
"config.amazonaws.com"
]
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::targetBucketName"
},
{
"Sid": "AWSConfigBucketExistenceCheck",
"Effect": "Allow",
"Principal": {
"Service": [
"config.amazonaws.com"
]
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::targetBucketName"
},
{
"Sid": "AWSConfigBucketDelivery",
"Effect": "Allow",
"Principal": {
"Service": [
"config.amazonaws.com"
]
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::targetBucketName/[optional] prefix/AWSLogs/sourceAccountID-WithoutHyphens/Config/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
I want to know about the part provided below:
"Resource": "arn:aws:s3:::targetBucketName/[optional] prefix/AWSLogs/sourceAccountID-WithoutHyphens/Config/*",
Whats the meaning of the prefix and how do I fill the prefix/AWSLogs/sourceAccountID-WithoutHyphens/Config/* part of the policy?
Thanks.
The prefix is what you define when you configure logging to S3. This is optional. Config writes the logs to S3 bucket using a standard path/key format which is "prefix/AWSLogs/sourceAccountID-WithoutHyphens/Config/*".
If you configure Config logging to S3 from console, you won't have to worry about the bucket policy as it will be created automatically. You simply give the bucket name and optional prefix.
I am the owner of AWS AccountC and need List and Get Permissions to BucketName owned by another person/team.
The bucket policy created is attached below. Policy for AccountA and AccountB were already existing and I added the policy for AccountC as given below
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AccessA",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::AccountA:root",
"arn:aws:iam::AccountA:user/ABC-Prod"
]
},
"Action": [
"s3:GetObject",
"s3:List*"
],
"Resource": [
"arn:aws:s3:::BucketName/*",
"arn:aws:s3:::BucketName"
]
},
{
"Sid": "AccessB",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::AccountB:user/service-user",
"arn:aws:iam::AccountB:role/BatchUserRole"
]
},
"Action": "*",
"Resource": [
"arn:aws:s3:::BucketName/*",
"arn:aws:s3:::BucketName"
]
},
{
"Sid": "AccessC",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountC:root"
},
"Action": "s3:List*",
"Resource": "arn:aws:s3:::BucketName"
},
{
"Sid": "AccessD",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountC:root"
},
"Action": "s3:Get*",
"Resource": "arn:aws:s3:::BucketName/*"
}
]
}
I am able to list contents of BucketName using
aws s3 ls BucketName.
However, when I try
aws s3 cp --recursive BucketName/folderName/ ., it gives me an Access Denied error
An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
Block public access is enabled on the bucket, however I believe it should not affect since the Bucket policy is added
Tried multiple way to write the policy but the error persists. Can someone please help me understand what I might be missing here? Would be really grateful
I'm following the instructions from this answer to generate the follow S3 bucket policy:
{
"Id": "Policy1495981680273",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1495981517155",
"Action": [
"s3:GetObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::surplace-audio",
"Principal": "*"
}
]
}
I get back the following error:
Action does not apply to any resource(s) in statement
What am I missing from my policy?
From IAM docs, http://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements.html#Action
Some services do not let you specify actions for individual resources; instead, any actions that you list in the Action or NotAction element apply to all resources in that service. In these cases, you use the wildcard * in the Resource element.
With this information, resource should have a value like below:
"Resource": "arn:aws:s3:::surplace-audio/*"
Just removing the s3:ListBucket permission wasn't really a good enough solution for me, and probably isn't for many others.
If you want the s3:ListBucket permission, you need to just have the plain arn of the bucket (without the /* at the end) as this permission applies to the bucket itself and not items within the bucket.
As shown below, you have to have the s3:ListBucket permission as a separate statement from the permissions pertaining to items within the bucket like s3:GetObject and s3:PutObject:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Principal": {
"AWS": "[IAM ARN HERE]"
},
"Resource": "arn:aws:s3:::my-bucket-name"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Principal": {
"AWS": "[IAM ARN HERE]"
},
"Resource": "arn:aws:s3:::my-bucket-name/*"
}
]
}
Error Action does not apply to any resource(s) in statement
Simply it means that the action (you wrote in policy) doesn't apply to the resource. I was trying to make public my bucket so that anybody can download from my bucket. I was getting error until I remove ( "s3:ListBucket") from my statement.
{
"Id": "Policyxxxx961",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmtxxxxx4365",
"Action": [
"s3:GetObject",
"s3:ListBucket",
"s3:PutObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::bucket-name/*",
"Principal": "*"
}
]
}
Because list bucket doesn't apply inside the bucket, thus by deleting this action policy worked fine.
Just ran into this issue and found a shorter solution for those that want to have ListBucket and GetObject in the same policy. The important thing is to list both the bucket-name and bucket-name/* under Resource.
{
"Id": "Policyxxxx961",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmtxxxxx4365",
"Action": [
"s3:GetObject",
"s3:ListBucket",
"s3:PutObject"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::bucket-name",
"arn:aws:s3:::bucket-name/*"
],
"Principal": "*"
}
]
}
To fix this issue, what you need to do in policy rule, locate the Resource, and add your arn bucket in array, one with * and the second on without * at the end. This will fix the error.
{
"Version": "2012-10-17",
"Id": "Policy3783783783738",
"Statement": [
{
"Sid": "Stmt1615891730703",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::76367367633:user/magazine-demo-root-user"
},
"Action": [
"s3:ListBucket",
"s3:ListBucketVersions",
"s3:GetBucketLocation",
"s3:DeleteObject",
"s3:AbortMultipartUpload",
"s3:Get*",
"s3:Put*"
],
"Resource": [
"arn:aws:s3:::magazine-demo",
"arn:aws:s3:::magazine-demo/*"
]
}
]
}
Just do one change in json policy resource.
"Resource": ["arn:aws:s3:::bucket-name/*"]
Note : Add /* after bucket-name
Ref Docs :
https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements.html
I have also faced the similar issue while creating the bucket
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::mrt9949"
]
}
]
}
I have changed the above code to
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::mrt9949/*"
]
}
]
}
add /* to your bucket name it will solve the issue
Here my bucket name is mrt9949
In my case the solution to this error was trying to remove some of Actions that I was applying. Some of them are not relevant to, or cannot work with this resource.
In this case it wouldn't let me include these:
GetBucketAcl
ListBucket
ListBucketMultipartUploads
Whenever you are trying to apply use bucket policies. Remember this thing, If you are using actions like "s3:ListBucket", "s3:GetBucketPolicy", "s3:GetBucketAcl" etc. which are related to bucket, the resource attribute in policy should be mentioned as <"Resource": "arn:aws:s3:::bucket_name">.
Ex.
{
"Version": "2012-10-17",
"Id": "Policy1608224885249",
"Statement": [
{
"Sid": "Stmt1608226298927",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetBucketPolicy",
"s3:GetBucketAcl",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::bucket_name"
}
]
}
If you are using actions like "s3:GetObject", "s3:DeleteObject", "s3:GetObject" etc. which are related to object, the resource attribute in policy should be mentioned as <"Resource": "arn:aws:s3:::bucket_name/*">.
ex.
{
"Id": "Policy1608228066771",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1608228057071",
"Action": [
"s3:DeleteObject",
"s3:GetObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::bucket_name/*",
"Principal": "*"
}
]
}
Finally if you are using actions like "s3:ListBucket", "s3:GetObject" etc. these actions are related to both bucket and object then the resource attribute in policy should be mentioned as <"Resource": ["arn:aws:s3:::bucket_name/*", "Resource": "arn:aws:s3:::bucket_name">.
ex.
{
"Version": "2012-10-17",
"Id": "Policy1608224885249",
"Statement": [
{
"Sid": "Stmt1608226298927",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::bucket_name",
"arn:aws:s3:::bucket_name/*"
]
}
] }
Go to Amazon S3 in your instance.
Go to Permissions -> Public Access tab.
Select Edit and uncheck Block all public access and save.
You will see 'Public' tag in Permission tab and Access Control List.
You might have several policy statements and this error is a very generic one. Best is to comment all other statements except any one (like just GetObject, or ListBuckets, Or PutObject) and execute the code and see. If it works fine, it means the ARN path is right. Else, the ARN should include the bucket name alone or a bucketname with /*.
Some resources like ListBucket accept ARN with the full name like "arn:aws:s3:::bucket_name", while GetObject or PutObject expects a /* after the bucket_name. Change the ARNs according to the service and it should work now!
You have to check the pattern of the arn defined under the Resource tag for the Policy-
"Resource": "arn:aws:s3:::s3mybucketname/*"
With the addition of "/*" at the end would help to resolve the issue if you face it even after having your Public Access Policy Unblocked for your Bucket.
From AWS > Documentation > AWS Identity and Access Management > User Guide
https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_resource.html
It is clearly defined in a note, Some services do not let you specify actions for individual resources.
you use the wildcard * in the Resource element
"Resource": "arn:aws:s3:::surplace-audio/*"
You can also configure ListBuckets for each folder, like so
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowSESPuts-1521238702575",
"Effect": "Allow",
"Principal": {
"Service": "ses.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::buckets.email/*",
"Condition": {
"StringEquals": {
"aws:Referer": "[red]"
}
}
},
{
"Sid": "Stmt1586754972129",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::596322993031:user/[red]"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::buckets.email",
"Condition": {
"StringEquals": {
"s3:delimiter": "/",
"s3:prefix": [
"",
"domain.co",
"domain.co/user"
]
}
}
},
{
"Sid": "Stmt1586754972129",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::596322993031:user/[red]"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::buckets.email",
"Condition": {
"StringLike": {
"s3:prefix": "domain.co/user/*"
}
}
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::596322993031:user/[red]"
},
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::buckets.email/domain.co/user/*"
}
]
}
These rules are used together with SES to receive an email, but allows an external user to view the files that were put in the bucket by SES.
I followed the instructions from here: https://aws.amazon.com/blogs/security/writing-iam-policies-grant-access-to-user-specific-folders-in-an-amazon-s3-bucket/
Also, you must specify prefix as domain.co/user/ WITH slash at the end when using the SDK, otherwise you'll get access denied. hope it helps anyone
This is so simple just add "/*" at the end. You have given only the root directory link, but the action needs to be applied to the object.
Type,
"Resource": "arn:aws:s3:::surplace-audio/*"
I found that my ListBuckets was not working because the IAM Principle did not have ListAllMyBuckets permission.
Is it possible to create an S3 bucket policy that allows read and write access from aws machine learning ? I tried below bucket policy, but not work.
bucket policy:
{
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "machinelearning.amazonaws.com"
},
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::cxra/*"
},
{
"Effect": "Allow",
"Principal": {
"Service": "machinelearning.amazonaws.com"
},
"Action": "s3:PutObjectAcl",
"Resource": "arn:aws:s3:::cxra/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
},
{
"Effect": "Allow",
"Principal": {
"Service": "machinelearning.amazonaws.com"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::cxra"
},
{
"Effect": "Allow",
"Principal": {
"Service": "machinelearning.amazonaws.com"
},
"Action": [
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::cxra"
}
]
}
Error screen
Double check your policy against Granting Amazon ML Permissions to Output Predictions to Amazon S3 and Granting Amazon ML Permissions to Read Your Data from Amazon S3
As they are documented as two separate policies, I suggest you follow that, rather than joining the two policies in one single document. It will be easier to troubleshoot.