After using aws configure to setup access I can list all the files in an s3 bucket with:
aws s3api list-objects --bucket my_bucket_name
but when I run:
aws s3 cp s3://s3.amazonaws.com/my_bucket_name/ . --recursive
or
aws s3 sync s3://s3.amazonaws.com/my_bucket_name/ .
I get the following error:
fatal error: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied
Any ideas?
The following worked - ie without amazon domain:
aws s3 sync s3://my_bucket_name .
Related
Our client shared a S3 bucket to us. The following command to list the objects works.
AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... aws s3 ls s3://bucket/company4/obj1
But it got 403 error when downloading?
AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... aws s3 cp s3://bucket/company4/obj1 .
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
The issue was caused by missing proxy. It works after setting the proxy export http_proxy=.... Why aws s3 ls doesn't require proxy?
Trying to learn terraform and running through some examples from their documentation. I am trying to move a file from my pc to a S3 bucket. I create this bucket with the command:
aws s3api create-bucket --bucket=terraform-serverless-example --region=us-east-1
If I then check on the console or call the s3api list-buckets I can see it exists and is functional. I can also upload using the console. However when I try to run this command:
aws s3 cp example.zip s3://terraform-serverless-example/v1.0.0/example.zip
It returns this error:
upload failed: ./code.zip to s3://test-verson1/v1.0.0/code.zip An error occurred (NoSuchBucket) when calling the PutObject operation: The specified bucket does not exist
I have tried to configure the region by adding it as a flag, have made sure permissions are correct and so on and it's all set up as should be. I cant figure out why this maybe and would appreciate any help.
aws s3 cp s3://arxiv/pdf/arXiv_pdf_0001_001.tar s3://bucket --request-payer requester
fails with
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
But if I do
aws s3 cp s3://arxiv/pdf/arXiv_pdf_0001_001.tar . --request-payer requester
it works fine
Additionally this also works, but it only copies one file at a time:
aws s3api copy-object --copy-source arxiv/pdf/arXiv_pdf_0001_001.tar --request-payer requester --key arXiv_pdf_0001_001.tar --bucket arxivmanifest
Whats going on?
When I ran the first command, it gave the error:
An error occurred (AccessDenied) when calling the GetObjectTagging operation: Access Denied
This is because the aws s3 cp command does more than just copy the file by also attempting to copy tagging (it seems). It would also seem that the bucket has not granted permissions for the GetObjectTagging API call.
In contrast, the aws s3api copy-object command issues a single API call. In fact, all s3api commands map to a specific API call. The aws s3 commands are 'higher level' commands that do more, such as enabling a --recursive copy).
I am trying to upload/download to S3 with transfer acceleration endpoint in AWS CLI from an EC2.
aws s3 cp /home/centos/<FOLDER_NAME> s3://<BUCKET_NAME>/<KEY_NAME> --region
ap-south-1 --endpoint-url http://<S3-Transfer-Acc-endpoint> [--recursive]
I am able to upload files and folders successfully and also the download using recursive for folders works.
But when i download a single large file from the bucket, using the following command
aws s3 cp s3://<BUCKET_NAME>/<KEY_NAME> /home/centos/<KEY_NAME> --region
ap-south-1 --endpoint-url http://<S3-Transfer-Acc-endpoint>
I face the below issue
fatal error: An error occurred (400) when calling the HeadObject operation:
Bad Request
I've got a user with permission with the AmazonS3FullAccess policy, my bucketname is abcxyzchristest.
I've configured the AWS cli with the above user.
When running aws s3 cp --region eu s3://abcxyzchristest/ abcxyz I get:
fatal error: An error occurred (InvalidRequest) when calling the ListObjects operation: Missing required header for this request: x-amz-content-sha256
What is the cause of the error?
It was due the incorrect region specified.
I had to do:
aws s3 cp --region eu-central-1 s3://abcxyzchristest/ bobby
There are two steps to solve this problem:
1) Configure your command line aws
[root#ip-172-31-18-132 ec2-user]# aws configure
AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [ap-south-1a]: ap-south-1 <-- give your region here
Default output format [None]:
2) Specify your region in the s3 cp command
aws s3 cp --recursive s3://buckername /home/ec2-user --region ap-south-1