This is a simple question that possibly applies to all CDNs but I have not been able to find an answer to on the web or on the AWS site (http://aws.amazon.com/cloudfront/). Hopefully it is a simple answer for anyone familiar with CDNs or CloudFront and this might help others with understanding how this works.
If I were to use CloudFront for whole site delivery and set up an origin server (e.g. origin is www.myexample.com) then if I have an html file (example1.html) being served at www.myexample.com/example1.html and example1.html were to contain element that include an img tag with a src http://www.anothersite.com/anotherExample.jpg or even a S3 bucket source, then does this jpg from another source get cached in the cdn too?
You should connect the S3 bucket to a CloudFront distribution and use that links in the html code itself. I may be wrong, but I don't see how a CDN might be able to cache those links, as the client browser themselves request these resources based on the downloaded html code from the CDN.
Hence, in your example, only requests for myexample.com would go through your CloudFront distribution without any additional origin settings.
Edit: see #Skill M2 comment regarding adding multiple origin's for the same CloudFront distribution
Related
I have an S3 bucket that is served via a CloudFront distribution, and I want to have some simple route logic to send different paths to specific S3 files. So for example, domain.com/colors/* would go to the domain.com/rainbow, while domain.com/numbers/* would go somewhere else, let's say domain.com/math.
Currently I've achieved something kind of like this by setting the 404 error page to the page I want served, but there is no logic here; all 404s go to the same place. So I essentially want to have more granular control over the error logic in my CloudFront distribution.
I've read about this being achieved by setting different paths to different origins, but I only have (and only want to have) one single origin, which is the S3 bucket.
How can I achieve this?
I ended up using Lambda#Edge to get this functionality. This sample from AWS is very similar code to what I ended up using: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-examples.html#lambda-examples-a-b-testing
NOTE: I'm providing details of my setup, but really this is a "how is this possible" question, not a "please debug my setup" question.
I have a "singe page application" (ie. an HTML file that uses the History API to simulate URLs). I'm serving this app on AWS S3, behind an AWS Cloudfront ... front.
I had successfully configured things so that if someone went to www.example.com/foo (let's pretend I own example.com), Cloudfront would serve an "error page" of my index.html. My index.html would then see the URL, and use its routing to show the user the correct page.
That all worked great ... until it didn't. Now for some reason when I go to www.example.com/foo, I get redirected to www.example.com. I'm trying to debug things, but what I can't understand is how I'm going from /foo to the main page.
When I look in the Network panel of my developer tools, I can see the request made to the original (/foo). Then I can see the chain of requests (for images, css files, etc.), and they all have a referrer of www.example.com/foo.
Then all of the sudden I see a request for React Developer tools (why it needs to make a request is beyond me) ... and it's from referrer www.example.com. After that I get one last image request from /foo, and then all subsequent requests come from www.example.com.
Can anyone explain how this could be working? I know that if a server returns a redirect (either type) that could change my URL ... but every request has a 200 status (ie. no server redirects).
I know Javascript could "push" a new URL to my browser ... but that would leave a history entry right? When I go "back" (either with my browser or history.back()) I go to the page before; I don't go "back" to /foo.
So somehow I'm not making a history entry, but I am switching my URL, and the URL I make requests from, and this all happens within milliseconds on page load ... without any redirects. How?
P.S. When I use my dev tools to add an beforeunload breakpoint, then try to navigate from example.com to example.com/foo I don't hit that break point (either for going to /foo, or when I'm "redirected" back to example.com).
When I check the box for any Load event, I do see some happen ... after my URL has already switched. In other words, I type example.com/foo, hit enter, and by the time any event fires I'm back on example.com. Whatever mechanism is doing the "redirection" here ... it doesn't trigger any load events.
I figured out my (AWS-specific) problem, thanks to a bit of Gatsby documentation. I'll include the details below in case it helps others, but I won't accept this answer, as I still don't understand how AWS did what it did (and I'd still welcome an answer for that).
What happened was that I had my Cloudfront "Origin Domain Name and Path" pointing to:
example.com.s3.amazonaws.com
However, as explained on https://www.gatsbyjs.com/docs/deploying-to-s3-cloudfront/:
There are two ways that you can connect CloudFront to an S3 origin. The most obvious way, which the AWS Console will suggest, is to type the bucket name in the Origin Domain Name field. This sets up an S3 origin, and allows you to configure CloudFront to use IAM to access your bucket. Unfortunately, it also makes it impossible to perform serverside (301/302) redirects, and it also means that directory indexes (having index.html be served when someone tries to access a directory) will only work in the root directory. You might not initially notice these issues, because Gatsby’s clientside JavaScript compensates for the latter and plugins such as gatsby-plugin-meta-redirect can compensate for the former. But just because you can’t see these issues, doesn’t mean they won’t affect search engines.
In order for all the features of your site to work correctly, you must instead use your S3 bucket’s Static Website Hosting Endpoint as the CloudFront origin. This does (sadly) mean that your bucket will have to be configured for public-read, because when CloudFront is using an S3 Static Website Hosting Endpoint address as the Origin, it’s incapable of authenticating via IAM.
Once I changed my Cloudfront "Origin Domain Name and Path" to the bucket's static hosting URL:
http://example.com.s3-website-us-west-1.amazonaws.com
Everything worked!
But again, I still don't understand how AWS did what it did when I mis-set my "Origin Domain Name and Path". It redirected me to my root domain, seemingly without either a redirect response OR a client-side redirect, and I'd love to hear how that was accomplished.
I am trying to understand how Cloudfront works. Assume static website is static.com and dynamic website is dynamic.com. static.com has thousands of html files containing img tags referencing images coming from static.com.
dynamic.com is Java based dynamically generating HTML and img tags and images comes from dynamic.com
Assume images are not manually copied to s3. No modifications are made in both sites in regards to Cloudfront other than DNS settings.
Assume Cloudfront url setup for static.com is mystaticxyzz.cloudfront.net and for dynamic.com it is mydynamicxyz.cloudfront.net
CloudFront works as a CDN sitting in front of what are called Origins.
These origins are the endpoints that CloudFront forwards traffic to, to retrieve the response and content. This could be a single server, a load balancer or any other resolvable hostname that is publicly accessible.
If you want to split between static and dynamic content you would create an origin for each type of content within the same distribution. One would be the default origin whilst the other would be matched based on a file path (/css or /images).
Each of these origins can include their own cache behaviours which enable you to define whether they should be cached and how long.
When a user accesses the CloudFront domain dependant on the path it will route to the appropriate origin or retrieve a response from the edge cache where possible.
I know this is rather late, but I am just going to add this here for those struggling to cache dynamic and static content.
Firstly, you need to understand your application your application.
Client Side Rendering
if you have a reactjs you don't need to worry too much about your caching behavior as you will be rendering , the data which will be fetched from an api client side.
none of the static files/content will be changing which are being delivered to the end user.
Since the APIs requests will be coming from a different domain , that data won't be cached by the cdn . Moreover, the data being rendered will update the html via javascript. If your javascript files are continously updating then you can use invalidations for them.
If you have content that is not stored on the origin and your CSR app is fetching the content from using a separate domain from your website domain, you will need to set up a separate cdn and point the domain name to that cdn. You wont need to make any changes to your application as the domain name stays the same for that.
However, if you static content that exists in the same origin (e.g. s3) then you would just request the content using the domain name of the cdn from which the request will come from client to cdn to origin (if not cached / expired)
lastly, assume we have separate origins like an s3 bucket for react app and s3 bucket for images . We can set up a single cdn with multiple origins . This means we can use cloudfront as an aggregator , you will then be able to cache content from different origins by using special paths.
This means , where ever you make calls to those origins previously. i.e. using the the s3 domain names, you would need to update them to that single domain name as the caching behaviors will handle the requests to the respective domains
example:
www.example.com(react-app react s3 bucket)
www.example.com/images (some s3 image bucket)
<img source={{url:"www.example.com/images/example.jpg"}} />
cloud front will make a request to that server based on that origin for the behavior configured on "/images"
Server Side Rendered
for serverside rendered apps , ideally the default cahcing behavior on the origin should allow all the different http methods , because you will have post and put http requests which you will want cloudfront to forward to the origin .
Make sure that you forward all query strings and cookies to the origin using a request policy. You can fine tune it with white listing query strings or cookies but this will make life easier. Also, the default caching behavior should use a caching policy that disables cache i.e. min,default,max ttls = 0secs . this is because the content is dynamic in nature and gets rendered on the server and not client side thus you will encounter unexpected behaviors in your application depending on how it is set up.
if you have static content on different paths like "/img", "/css" , or "/web/pages/information" cache those independently from the default behavior the respective ttls on them.
you could do some cool stuff using the cache-control header which can by pass the cache if you dont want to configure a 101 behaviors.
https://aws.amazon.com/blogs/networking-and-content-delivery/improve-your-website-performance-with-amazon-cloudfront/
Just understand your application and you will be able to leverage cdn properly
if you have a webserver that does a mixture of server side and client side rendering
just identify which paths are client-side rendered and cache those static files.
Any thing that is dynamic in nature that requires the application to make requests to the origin , make use of the caching disabled policy within a behavior.
Moreover, any of those patterns(of using a single cdn with a single/multiple origins or multiple cdns with differing origins ) mentioned earlier is applicable to serverside rendering if some content gets rendered clients side such as images
Many people have received 100s of links to PoCs that are on an internal facing bucket and the links are in this structure.
https://s3.amazonaws.com/bucket_name/
I added a redirect using AWS's Static website hosting section in Properties and it ONLY redirects when the domain is formatted like this:
https://bucket_name.s3-website-us-east-1.amazonaws.com
Is this a bug with S3?
For now, how do I make it redirect using both types of links? My current workaround is to add a meta redirect tag in each html file.
The s3-website is the only endpoint that supports redirects unfortunately. Using the s3.amazonaws.com supposes that you will be using S3 as a storage layer, instead of a website. If the link is to a specific object, you can place an HTML file at that url with a JS redirect, but other than that there is really no way to achieve what you are trying to do.
In the future, i would recommend always setting up a Cloudfront distribution for those kinds of usecases, as that will allow you to change the origin later on.
I'm trying to set a Content-Security-Policy header for an html file I'm serving via s3/cloudfront. I'm using the web-based AWS console. Whenever I try to add the header:
it doesn't seem to respect it. What can I do to make sure this header is served?
I'm having the same problem (using S3/CloudFront) and it appears there is currently no way to set this up easily.
S3 has a whitelist of the headers permitted, and Content-Security-Policy is not on it. Whilst it is true you can use the prefixed x-amz-meta-Content-Security-Policy, this is unhelpful as there is no browser support for it.
There are two options I can see.
1) you can serve the html content from a webserver on an EC2 instance and set that up as another CloudFront origin. Not really a great solution.
2) include the CSP as a meta tag within your html document:
<!doctype html>
<html>
<head>
<meta http-equiv="Content-Security-Policy" content="default-src http://*.foobar.com 'self'">
...
This option is not as widely supported by browsers, but it appears to work with both Webkit and Firefox, so the current Chrome, Firefox, Safari (and IOS 7 Safari) seem to support it.
I chose 2 as it was the simpler/cheaper/faster solution and I hope AWS will add the CSP header in the future.
S3/CloudFront takes any headers that the origin set and forward those to the client, but you can't set custom headers on you response directly.
You can use Lambda#Edge function that can inject security headers through CloudFront.
Here is how the process works: (reference aws blog)
Viewer navigates to website.
Before CloudFront serves content from the cache it will trigger any
Lambda function associated with the Viewer Request trigger for that
behavior.
CloudFront serves content from the cache if available, otherwise it
goes to step 4.
Only after CloudFront cache ‘Miss’, Origin Request trigger is fired
for that behavior.
S3 Origin returns content.
After content is returned from S3 but before being cached in
CloudFront, Origin Response trigger is fired.
After content is cached in CloudFront, Viewer Response trigger is
fired and is the final step before viewer receives content.
Viewer receives content.
Below is the blog from aws on how to do this step by step.
https://aws.amazon.com/blogs/networking-and-content-delivery/adding-http-security-headers-using-lambdaedge-and-amazon-cloudfront/
If you are testing through CloudFront, have you made sure you have invalidated the cached objects? Can you try to upload a completely new file and then try accessing it via CF and see if the header is still not there?
Update
Seems like custom metadata will not work as expected as per DOC. Any metadata other than the ones supported by S3 (the ones displayed in the dropdown) will have to be prefixed with x-amz-meta-