My setup:
I have a Cloudfront Origin Group where Bucket A is a primary bucket and Bucket B is a secondary bucket. Lambda#Edge is added on origin-request to do a certain process.
Whenever a request comes to Cloudfront, my Lambda#Edge modifies it to match the folder structure of my bucket and returns file accordingly.
If Bucket A doesn't have a certain file it throws an error and Cloudfront failover requests the file from Bucket B. Bucket B doesn't have the same structure as Bucket it, it should return the file from unmodified file path in the bucket.
Example:
My Original Request: /somefile.html
Lambda#Edge modifies this request to get the file from Bucket A to: /en/somefile.html
If Bucket A doesn't have this somefile.html then this request goes to Bucket B. It should return file from the originally requested path: /somefile.html and not /en/somefile.html
The above scenario is very simple, my original scenario is much complex. Basically Bucket A file path is processed path while Bucket B should return file from an originally requested path.
What I want:
Using Lambda#Edge how can I detect if the request is on Bucket A or bucket B?
What I have tried:
I tried adding certain header in request headers and check if the header exists then its request to Bucket B. But this doesn't seem to be working.
The hostname of the origin that CloudFront will try to contact after an origin-request trigger returns control can be found in one of two places.
If you are using what CloudFront calls an S3 origin (the REST interface to S3) it will be here:
event.Records[0].cf.request.origin.s3.domainName
If you are using what CloudFront calls a custom origin -- which includes S3 website hosting endpoints as well as any other origin server that isn't S3 REST, it's here:
event.Records[0].cf.request.origin.custom.domainName
These can be used to determine which of two origins in an origin group will receive the request, next.
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-event-structure.html#lambda-event-structure-request
Related
I have a s3 bucket that I created and I am adding objects and reading them in a client that calls the bucket via my_bucket.s3.ap-southeast-2.amazonaws.com endpoint. Is there any way to add a header (eg: Strict-Transport-Security) to the responses from S3 (using aws console or cloudformation).
I am aware that I can do this by using cloudfront to route the request. But I want to know if I can achieve this directly from S3.
I have a a CloudFront Distribution with an S3 bucket as the origin (at its root).
If the origin has a path pattern of test/*, but I don't have a test directory in my S3 bucket, will requests such as www.domain.com/test/t-shirt fail?
I can't tell if that request would go to bucket/test/t-shirt or to bucket/t-shirt.
The short answer is no. That said, you may need to use Lambda#Edge to rewrite the request back to the S3 bucket to match up the CloudFront request with the S3 request.
Yes, www.domain.com/test/t-shirt will fail since the CloudFront tries to fetch the object stored in the S3 bucket bucket using the key test/t-shirt.
I would like to configure Cloudfront so first it looks for an object in S3 bucket and if it doesn't exist in the S3 bucket Cloudfront will use the next origin, which is Load Balancer, to retrieve the file. Is that possible?
Yes, It is possible using new origin Failover feature of CloudFront.
You can allow CloudFront to have List bucket access on S3 bucket so S3 can return 404 status code and you can choose to contact ELB if 404 occurs from S3.
Secondly, you can leverage lambda#edge Origin response function to make connection to ELB in case S3 returns 404 error.
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-and-origin-failover.html
aws s3 sync does not seem to copy the website redirect metadata by default.
There is this option:
--website-redirect (string) If the bucket is configured as a website, redirects requests for this object to another object in the same
bucket or to an external URL. Amazon S3 stores the value of this
header in the object metadata.
But I'm looking for some kind of directive to get sync to copy the redirect of each file to the sync target. Is there any way to do that?
aws s3 cp has the same option. I'm not sure how sync would do this since it is the whole directory, cp only does the single file unless you are using sync with specific files and not the whole dir.
It looks like the redirect is just metadata injected in the file and that is what the --website-redirect is setting.
The following Amazon S3 API actions support the x-amz-website-redirect-location header in the request. Amazon S3 stores the header value in the object metadata as x-amz-website-redirect-location.
https://docs.aws.amazon.com/AmazonS3/latest/dev/how-to-page-redirect.html
x-amz-website-redirect-location
If the bucket is configured as a website, redirects requests for this object to another object in the same bucket or to an external URL. Amazon S3 stores the value of this header in the object metadata. For information about object metadata, see Object Key and Metadata.
In the following example, the request header sets the redirect to an object (anotherPage.html) in the same bucket:
x-amz-website-redirect-location: /anotherPage.html
In the following example, the request header sets the object redirect to another website:
x-amz-website-redirect-location: http://www.example.com/
For more information about website hosting in Amazon S3, see Hosting Websites on Amazon S3 and How to Configure Website Page Redirects in the Amazon Simple Storage Service Developer Guide.
Type: String
Default: None
Constraints: The value must be prefixed by, "/", "http://" or "https://". The length of the value is limited to 2 K.
https://docs.aws.amazon.com/AmazonS3/latest/dev/how-to-page-redirect.html
I have a Cloudfront distribution which takes one of my s3 buckets as its origin server. The files are uploaded to s3 by a third party attachment uploader.
When I try to access the file in s3 via cloudfront I am getting a 403 Forbidden error with an Access Denied XML (as below). But when I manually upload files to the s3 bucket I am able to access the file via cloudfront.
The permission for both the files are same except the owner of the file. For the file uploaded by me manually the owner, of the file is my account and for the file uploaded by the uploader, it is the uploader. The third party attachment uploader gives full access of the object to the bucket owner. Also, I have restricted bucket access but not viewer access.
What are the reasons which can cause this error? How do I go about debugging this?
When a second AWS account uploads content to an S3 bucket serving content via CloudFront with OAI, the uploaded file needs to have the OAI canonical ID added with the --grant read=id="OAI-canonical-ID" when the file is uploade; also add the S3 bucket owner as a grant full=id="BucketOwnerID". The aws cli was used to perform the uploaded. Adjust according for the method that is used.
When the file is viewed in the S3 bucket, the permissions will have CloudFront listed as a grantee. The file should be readable via CloudFront.
It is possible that the "Access Denied" response you are receiving is a response from S3 that is cached by CloudFront from before the object was available in S3.
For example, if you try the CloudFront URL and the file does not exist in S3, then you'll get the "Access Denied" response. Even after uploading the file, CloudFront will have the "Access Denied" response cached until the end of the TTL. During this time, you will continue to receive the "Access Denied" response.
Try invalidating the distribution. After that, request the file and see if you get the correct response.
If this solves the problem, then you need to figure out how to avoid requesting the object from CloudFront before it exists in S3.