can't copy file to S3 - amazon-web-services

I try to copy a file from my local to S3 bucket using AWS - CLI.
this is my command:
aws s3 cp /path/to/local/file s3://bucket-name
but the message I get is :
An error occured (AccessDenied) when calling the PutObject operation: Access Denied
I've set the policy in my s3 bucket to accept everything, like this:
{
"Id": "BucketPolicy",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllAccess",
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::MyBucket",
"arn:aws:s3:::MyBucket/*"
],
"Principal": "*"
}
]
}
What could be the reason I'm getting this error?
Any help would be appreciated!

Worth checking your command, For copy, you need to specify the source file. The destination file can be current directory.aws cli
aws s3 cp test.txt s3://mybucket/test2.txt
Basic bucket policy aws s3 docs
This policy allows all users to retrieve any object in MyBucket. It also grants put and delete permission to the root user of the AWS account 1234-5678-9012.
{
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::MyBucket/*"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:root"
},
"Action": [
"s3:DeleteObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::MyBucket/*"
}
]
}

Related

How can i download a local copy of an S3 snapshot of an AWS Postgres DB?

There is a snapshot in S3 of a Postgres DB, but the download button is grayed out... if i navigate to each file in each table, i am able to download the .gz.parquet files individually, but that is crazy.
So I installed the aws cli, configured a default user, tried to run aws s3 cp s3://<your-bucket-name>/<your-snapshot-name> <local-path> but always get:
fatal error: An error occurred (404) when calling the HeadObject operation: Key <your-snapshot-name> does not exist
But it does exist, and I can see it in the aws website and see the root folder if i run aws s3 ls.
So i tried aws s3 cp --recursive s3://<your-bucket-name>/<your-snapshot-name> <local-path> and it goes through all the folders, copies them to my computer, but theyre all empty folders, and i get the following error for every folder its going through:
An error occurred (AccessDenied) when calling the GetObject operation: The ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access.
The permissions I'm using are a generic (what i thought was) all access to S3:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*",
"s3-object-lambda:*"
],
"Resource": "*"
}
]
}
Plus two from here:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListAllMyBuckets"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::snapshots"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::snapshots/*"
]
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListAllMyBuckets"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::snapshots"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::snapshots/*"
]
}
]
}
What am I missing here?
The first error you are experiencing is probably because the aws s3 cp command works on objects, not directories. A good way to copy a whole directory (including subdirectories) is to use aws s3 sync.
The second error mentions "customer master key". This is probably referring to a KMS key that was used to encrypt the file when it was created by Amazon RDS. Try giving yourself kms:* permissions (although you probably only need kms:Decrypt) and it should be able to read the file.

fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

I am trying to publish AWS VPC Flow Logs from account A to S3 bucket in another account B. I am able to do so but when i try to download the logs from account A, i am getting the error
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
My account A has IAM userA that i am using.
I am using the below command to download:
aws s3 cp s3://vpcflowlogscross/VPCLogs/{subscribe_account_id}/vpcflowlogs/{region_code}/2021/06/14/xyz.log.gz .
I have tried adding the region as well in the above command but no luck. Also while investigating this i came across documentation to add ListBucket permission which i already have.
My bucket policy in account B:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSLogDeliveryWrite",
"Effect": "Allow",
"Principal": {
"Service": "delivery.logs.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::vpcflowlogs-cross/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
},
{
"Sid": "AWSLogDeliveryCheck",
"Effect": "Allow",
"Principal": {
"Service": "delivery.logs.amazonaws.com"
},
"Action": [
"s3:GetBucketAcl",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::vpcflowlogs-cross"
},
{
"Sid": "DownloadObjects",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountA:user/userA"
},
"Action": [
"s3:GetObject",
"s3:GetObjectAcl",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::vpcflowlogs-cross",
"arn:aws:s3:::vpcflowlogs-cross/*"
]
]
}
I am using the below IAM user policy in account A to download the objects that are in account B S3 bucket.
My IAM user policy in account A:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "IAMAllowDownload",
"Effect": "Allow",
"Action": [
"s3:GetObjectAcl",
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::vpcflowlogs-cross",
"arn:aws:s3:::vpcflowlogs-cross/*"
]
}
]
}
I have tried changing the bucket and IAM policy but still not luck. I understand that the log objects getting published to the account B are owned by it but i am using the s3 "getobject" permission in bucket policy for the account A Principal userA shouldn't this allow me to download the objects.

AWS S3 GetObject error on cross account access

I am the owner of AWS AccountC and need List and Get Permissions to BucketName owned by another person/team.
The bucket policy created is attached below. Policy for AccountA and AccountB were already existing and I added the policy for AccountC as given below
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AccessA",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::AccountA:root",
"arn:aws:iam::AccountA:user/ABC-Prod"
]
},
"Action": [
"s3:GetObject",
"s3:List*"
],
"Resource": [
"arn:aws:s3:::BucketName/*",
"arn:aws:s3:::BucketName"
]
},
{
"Sid": "AccessB",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::AccountB:user/service-user",
"arn:aws:iam::AccountB:role/BatchUserRole"
]
},
"Action": "*",
"Resource": [
"arn:aws:s3:::BucketName/*",
"arn:aws:s3:::BucketName"
]
},
{
"Sid": "AccessC",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountC:root"
},
"Action": "s3:List*",
"Resource": "arn:aws:s3:::BucketName"
},
{
"Sid": "AccessD",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountC:root"
},
"Action": "s3:Get*",
"Resource": "arn:aws:s3:::BucketName/*"
}
]
}
I am able to list contents of BucketName using
aws s3 ls BucketName.
However, when I try
aws s3 cp --recursive BucketName/folderName/ ., it gives me an Access Denied error
An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
Block public access is enabled on the bucket, however I believe it should not affect since the Bucket policy is added
Tried multiple way to write the policy but the error persists. Can someone please help me understand what I might be missing here? Would be really grateful

Access Denied when querying in Athena for data in S3 bucket in another AWS account

I want to use Glue Crawler to crawl data from an S3 bucket. This S3 bucket is in another AWS account. Let's call is Account A. My Glue Crawler is in Account B.
I have created a Role in Account B and called it AWSGlueServiceRole-Reporting
I have attached the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BucketAccess",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::AccountAbucketname"
]
},
{
"Sid": "ObjectAccess",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": [
"arn:aws:s3:::AccountABucketName/Foldername/*"
]
}
]
}
And also AWSGlueServiceRole policy.
In Account A that has the S3 bucket, I've attached the following bucket policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountB:role/AWSGlueServiceRoleReporting”
},
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::AccountABucketName"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountB:role/AWSGlueServiceRoleReporting”
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::AccountABucketName/FolderName/*"
}
]
}
I'm able to run a Glue Crawler in Account B on this S3 bucket and it created Glue Tables. But when I try to query them in Athena, I get Access Denied.
Can anybody help me how to query it in Athena??
When Amazon Athena queries run, they use the permissions of the user that is running the query.
Therefore, you will need to modify the Bucket Policy on the bucket in Account A to permit access by whoever is running the query in Amazon Athena:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::AccountB:role/AWSGlueServiceRoleReporting",
"arn:aws:iam::AccountB:user/username"
]
},
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::AccountABucketName"
},
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::AccountB:role/AWSGlueServiceRoleReporting",
"arn:aws:iam::AccountB:user/username"
]
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::AccountABucketName/FolderName/*"
}
]
}
The user will also need sufficient S3 permissions (on their IAM User) to access that S3 bucket. (For example, having s3:ListBucket and s3:GetObject on S3 buckets. They likely already have this, but it is worth mentioning.)
This is different to AWS Glue, which uses an IAM Role. Athena does not accept an IAM Role for running queries.

AWS S3 - Access denied when getting bucket location

I have a bucket called "my-bucket" in account A and I'm trying to determine it's location from account B.
I have the following bucket policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::11122233344:root"
},
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::my-bucket/*"
},
{
"Sid": "2",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::11122233344:root"
},
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::my-bucket"
}
]}
I've also stumbled across this documentation which contains the same policy: https://docs.aws.amazon.com/AmazonS3/latest/dev/example-walkthroughs-managing-access-example2.html
Now using account B, I can successfully execute the command aws s3 ls s3://my-bucket, however with aws s3api get-bucket-location --bucket my-bucket I get the access denied message.
According to the documentation: https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETlocation.html
To use this implementation of the operation, you must be the bucket
owner.
When in doubt always refer to the AWS API documentation it usually specifies the permission needed for the given action.