I have created a static website in an S3 bucket AWS. I have created two files in bucket one in index.html and 2nd is error.html. When I open index.html and click on object URL in AWS it gives below error:
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
<Code>InvalidArgument</Code>
<Message>Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4.</Message>
<ArgumentName>Authorization</ArgumentName>
<ArgumentValue>null</ArgumentValue>
<RequestId>R69TKNDJTYZ8E0SW</RequestId>
<HostId>OAOZKRsA6ATOgH6jBr5jO1fS0zi+GSh4at34nLq8V/Ug8Icvuy8c6NOlCoNqqjpBcORg8bDlzJ0=</HostId>
</Error>
I have checked every possible solution but nothing works. My bucket has public access like below my bucket name it is written in red Publicly accessible. But still I could not find what is issue.
I have created a S3 mock service in my code base.
// Create a S3 Client
S3Client client = S3Client.builder()
.serviceConfiguration(configuration)
.credentialsProvider(AnonymousCredentialsProvider.create())
.region(Region.of("us-west-2"))
.endpointOverride(new URI("http://localhost:8001"))
.build();
// Create a S3 bucket with a bucket name i.e.
CreateBucketResponse createBucketResponse = client.createBucket(CreateBucketRequest.builder().
bucket(<BucketName>).build());
// Verify if the bucket has been created or not.
// If the bucket is not created then the following lines will throw no such bucket
exception.
HeadBucketRequest headBucketRequest = HeadBucketRequest.builder()
.bucket(**<BucketName>**)
.build();
HeadBucketResponse headBucketResponse = client.headBucket(headBucketRequest);
// Update the Versioning status of the created bucket to Enabled.
PutBucketVersioningRequest versioningRequest = PutBucketVersioningRequest.builder().bucket(**<BucketName>**)
.versioningConfiguration(
VersioningConfiguration.builder().status(BucketVersioningStatus.ENABLED).build()
).build();
PutBucketVersioningResponse result = client.putBucketVersioning(versioningRequest);
// Check if the bucket versioning status is enabled or not.
BucketVersioningStatus bucketVersioningStatus = client.getBucketVersioning(GetBucketVersioningRequest.builder().bucket(**<BucketName>**).build()).status();
Note : The bucketVersioningStatus is null/empty for the above call. In the real time, when I am creating the bucket in the cloud and setting the versioning status gives me proper results but in the mock s3 client I am not getting the same expected outcome.
I am not sure if I have used the PutBucketVersioningRequest appropriately.
NOTE: I am using the S3 Client V2 and not the AmazonS3Client
Kindly help me in identifying the root cause of the issue.
Have following code to generate pre-signed URL:
params = {'Bucket': bucket_name, 'Key': object_name}
response = s3_client.generate_presigned_url('get_object',
Params=params,
ExpiresIn=expiration)
that works fine on old one bucket I am using for last year:
https://old-bucket.s3.amazonaws.com/test_image.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIxxxxxxxxxxE%2F20210917%2Feu-north-1%2Fs3%2Faws4_request&X-Amz-Date=20210917T210448Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=54e173601fec5f140dd901b0eae1dafbcd8d7ee8b8f311fdc1b120ca447cdd0c
I can paste this URL to browser and download file. File is AWS-KMS encrypted.
But same AWS-KMS encrypted file uploaded to new one created bucket returns following URL:
https://new-bucket.s3.amazonaws.com/test_image.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIxxxxxxxxxxE%2F20210917%2Feu-north-1%2Fs3%2Faws4_request&X-Amz-Date=20210917T210500Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=2313e0131d4251f9fba522fc8e9880d960f674f3449e141848bd38ca19e1b528
returns SignatureDoesNotMatch error:
The request signature we calculated does not match the signature you provided. Check your key and signing method.
No any changes in source code - but just bucket name provided to generate_presigned_url function.
The IAM user I am providing to boto3.client has write/read permissions for both buckets.
Comparing properties and permissions for both buckets and for files I am requesting from buckets - everything looks the same.
GetObject and PutObject works fine for both buckets in a case of dealing with file directly. The issue is only in a case of using pre-signed URL.
So is any settings/permissions/rules/anything else need to be configured/enabled to make pre-signed URLs working with certain S3 bucket?
I am following a tutorial here and if I take this s3 URL from the tutorial, https://s3.eu-central-1.amazonaws.com/deepset.ai-farm-qa/datasets/documents/wiki_gameofthrones_txt.zip, I am able to directly download the zip file to local.
When I subsistute my own zip file URL, I get an error that BadZipFile: File is not a zip file, and if I try my URL for zip file, I get permission denied instead of being able to download.
I also confirmed the zip files are formated correctly using terminal: unzip -t zipfile.zip
What permissions do I need to change in s3 or on the s3 object to allow download of zip file directly from URL?
Still very new to IAM s3 permissions and current permission are the standard ones when creating bucket.
Objects in Amazon S3 are private by default. This means that they cannot be accessed by an anonymous URL (like you have shown).
If you want a specific object to be publicly available (meaning that anyone with the URL can access it), then use the Make Public option in the S3 management console. This can also be configured at the time that the object is uploaded by specifying ACL=public-read.
If you want a whole bucket, or particular paths within a bucket, to be public, then you can create a Bucket Policy that grants access to the bucket. This requires S3 Block Public Access to be disabled.
You can also generate n Amazon S3 pre-signed URL, which provides time-limited access to a private object. The pre-signed URL has additional information added that grants permission to access the private object. This is how web applications provide access to private objects to authorized users, such as photo websites.
If an object is accessed via an AWS API call or the AWS Command-Line Interface (CLI), then AWS credentials are used to identify the user. If the user has permission to access the object, then they can download it. This method uses an API call rather than a URL.
Two solutions:
Make your bucket/file public. Check this ( Not recommended)
{
"Version":"2012-10-17",
"Statement":[
{
"Sid":"PublicRead",
"Effect":"Allow",
"Principal": "*",
"Action":["s3:GetObject","s3:GetObjectVersion"],
"Resource":["arn:aws:s3:::DOC-EXAMPLE-BUCKET/*"]
}
]
}
Use pre-signed URLs with SDK .. check this
var params = {Bucket: bucketname , Key: keyfile , Expires: 3600 , ResponseContentDisposition : `attachment; filename="filename.ext"` };
var url = s3.getSignedUrl('getObject', params);
How can we get the URL for the amazon S3 bucket files. I have tried to get the file by below format
http://s3-REGION-.amazonaws.com/BUCKET-NAME/KEY
This format will be helpful to download the file if it has public access and server side encryption is disabled.
Purpose of URL generation is to share with internal teams in my organization. This file might have exceptions of any applications.
I have to make the file or the bucket to be restricted to my organization (not for public). The bucket what ever I have server side encryption is enabled. How can we generate the file url which has server side encryption is enabled ?
You can generate a presigned URL for an S3 object: https://docs.aws.amazon.com/cli/latest/reference/s3/presign.html
Presigned URLs can be generated programmatically as well with all AWS SDKs.
For example in Java: https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURLJavaSDK.html
if you are using .net aws sdk to generate predesign url
var s3Client = new AmazonS3Client();
var request1 = new GetPreSignedUrlRequest
{
BucketName = "bucketName",
Key = "filename(original one and no coding)",
Expires = DateTime.Now.AddMinutes(5)
};
var urlString = s3Client.GetPreSignedURL(request1);