I am using CloudFront to access my S3 bucket.
I perform both GET and PUT operations to retrieve and update the data. The problem is that after i send PUT request with new data, GET request still returns the older data. I do see that the file is updated in S3 bucket.
I am performing both GET and PUT from iOS application. However, i tried performing GET request using regular browsers and i still receive older data.
Do i need to do anything in addition to make CloudFront refresh its data?
Cloudfront caches your data. How long depends on the headers the origin serves content with and the distribution settings.
Amazon has a document with the full results of how they interac, but if you haven't set your cache control headers, not changed any cloudfront settings, then by default data is cached for upto 24 hours.
You can either:
set headers indicating how long to cache content for (e.g. Cache-Control: max-age=300 to allow caching for up to 5 minutes). How exactly you do this depends on how you are uploading the content, at a pinch you can use the console
Use the console / api to invalidate content. Beware that only the first 1000 invalidations a month a free - beyond that amazon charges. In addition, invalidations take 10-15 minutes to process.
Change the naming strategy for your s3 data so that new data is served under a different name (perhaps less relevant in your case)
When you PUT an object into S3 by sending it through Cloudfront, Cloudfront proxies the PUT request back to S3, without interpreting it within Cloudfront... so the PUT request changes the S3 object, but the old version of the object, if cached, would have no reason to be evicted from the Cloudfront cache, and would continue to be served until it is expired, evicted, or invalidated.
"The" Cloudfront cache is not a single thing. Cloudfront has over 50 global edge locations (reqests are routed to what should be the closest one, using geolocating DNS), and objects are only cached in locations through which they have been requested. Sending an invalidation request to purge an object from cache causes a background process at AWS to contact all of the edge locations and request the object be purged, if it exists.
What's the point of uploading this way, then? The point has to do with the impact of packet loss, latency, and overall network performance on the throughput of a TCP connection.
The Cloudfront edge locations are connected to the S3 regions by high bandwidth, low loss, low latency (within the bounds of the laws of physics) connections... so the connection from the "back side" of Cloudfront towards S3 may be a connection of higher quality than the browser would be able to establish.
Since the Cloudfront edge location is also likely to be closer to the browser than S3 is, the browser connection is likely to be of higher quality and more resilient... thereby improving the net quality of the end-to-end logical connection, by splitting it into two connections. This feature is solely about performance:
http://aws.amazon.com/blogs/aws/amazon-cloudfront-content-uploads-post-put-other-methods/
If you don't have any issues sending directly to S3, then uploads "through" Cloudfront serve little purpose.
Related
We are building a website which contains a lot of images and data. We have optimized a lot to make the website faster. Then we decided to use AWS CloudFront also to make it faster for all regions around the world. The app works faster after the integration of CloudFront.
But later we found that the data will load to CloudFront cache only when the website asks for it. So we are afraid that the initial load will take the same time as it used to take without the CDN because it loads from S3 to CDN first and then to the user.
Also, we used the default TTL values (ie., 24 hours). In our case, a user may log in once or twice per week to this website. So in that case also, the advantage of caching won't work here as well because the caching expires after 24 hours. Will raising the time of TTL (Maximum TTL) to a larger value solve the issue? Does it cost more money? And I also read that, increasing to a longer TTL is not a good idea as it has some disadvantages also for updating the data in s3.
Cloudfront will cache the response only after the first user requests for it. So it will be slow for the first user, but it will be significantly faster for every other user after the first user. So it does make sense to use Cloudfront.
Using the default TTL value is okay. Since most users will see the same content and the website has a lot of static components as well. Every user except the first user will see a fast response from your website. You could even reduce this to 10-12 hours depending on how often you expect your data to change.
There is no additional cost to increasing your TTL. However invalidation requests are charged. So if you want to remove a cache, there will be a cost added to it. So I would prefer to keep a short TTL as short as your data is expected to change, so you dont have to invalidate existing caches when your data changes. At the same time, maximum number of users can benefit from your CDN.
No additional charge for the first 1,000 paths requested for invalidation each month. Thereafter, $0.005 per path requested for invalidation.
UPDATE: In the event that you only have 1 user using the website over a long period of time (1 week or so), it might not be of much benefit to use CloudFront at all. CloudFront and all caching services are only effective when there are multiple users requesting for the same resources.
However you might still have a marginal benefit using CloudFront, as the requests will be routed from the edge location to S3 over AWS's backbone network which is much faster than the internet. But whether this is cost effective for you or not depends on how many users are using the website and how slow it is.
Aside from using CloudFront, you could also try S3 Cross Region Replication to increase your overall speed. Cross Region Replication can replicate your buckets to a different region as and when they are added in one region. This can help to minimize latency for users from other regions.
We have deployed our website on GCP VM, and enabled GCP CDN in front of the VM. When we browse website in most of the cases GCP CDN making requests to the Origin VM.
I am using below stack driver query to check the cache hits.
resource.type="http_load_balancer"
resource.labels.forwarding_rule_name="rule_name"
httpRequest.serverIp="gcpvmip"
httpRequest.requestUrl="request_url"
httpRequest.cacheFillBytes > 0
Based on your latest comment, it sounds like you're expecting all requests to your site to be served from Cloud CDN's caches without contacting your origin server. However, it's normal to see cache misses when using a CDN. Each CDN operates numerous caches, not one big global cache. The fact the content for one URL has been inserted into one cache does not mean it will be present in all caches everywhere. Further, unpopular cache entries are routinely evicted from cache to make room for more popular content.
Here are some relevant excerpts from the Cloud CDN docs:
Cloud CDN uses caches in numerous locations around the world. Caching
is reactive in that an object is stored in a particular cache if a
request goes through that cache and if the response is cacheable. An
object stored in one cache does not automatically replicate into other
caches; cache-to-cache fill happens only in response to a
client-initiated request.
https://cloud.google.com/cdn/docs/overview
Note that the expiration time is an upper bound on how long a cache
entry remains valid. There is no guarantee that a cache entry will
remain in the cache until it expires.
https://cloud.google.com/cdn/docs/caching
Note, though, that Cloud CDN operates numerous caches around the
world, and old cache entries are routinely evicted to make room for
new content. As a result, multiple cache fills per resource are
expected as part of normal operation.
https://cloud.google.com/cdn/docs/support#low-hit-rate
If you're seeing low cache hit rates for popular content, that last link has suggestions that should help.
I know exactly what the problem is... GCP CDN does not have Origin Shield feature. Even worse, with GCP almost every request comes from a different one of its massive number of CDN PoPs around the world. Without Origin Shield, your app server is the origin server and it has to fill the cache of every CDN edge point.
In my experience you should use GCP CDN only for DOS protection & caching and improving the TTFB performance of HTML requests (specially to offload SSL handshake). Use another CDN for caching other assets with better Cache/Hit ratio.
Some CDN providers have Origin Shield which helps with the cache hit ratio. E.g. create cdn.yourdomain.com with a CDN provider that has Origin Shield Feature and serve all other static content from there.
I know it may sound crazy to put a CDN in front of your CDN, but trust me it works amazing and you can even save money if you go with a CDN that charges less for bandwidth. Also, GCP CDN only caches content up to 10MB.
I know cloudfront updates it's servers ~24 hours [source].
My question is why does sometimes it take less than 24 hours? Like sometimes I update s3 and bam the new content is available from XXdomain.com immediately. Other times it seems like it takes the full 24 hours.
How can anyone explain the variation? Why does it seem like a non-standard amount of time to update?
It depends upon whether the request is cached or not. If a request did not a POP (Point of Presence) it need to fetch from origin. If that is the first request, then it will need to contact the origin and serve whatever is there.
In other cases, if already cached, then it will serve whatever is cached. We don't modify contents for longer time. If you purge, usually it takes longer time like 24 hours varies based on the pop location, network availability, cacheable size for a domain, etc.,
You can use either cache headers or set you cache configuration to your desired time.
Hope it helps.
I know cloudfront updates it's servers ~24 hours
That isn't really an accurate description of what happens.
More correctly, we can say that by default, an object cached in a CloudFront edge will be evicted after 24 hours.
There is no active update process. The CloudFront cache is a passive, pull-through cache. When a request arrives, it is served from cache if a cached copy is available and not stale, otherwise a request is sent to the origin server, the object is stored in the cache, and returned to the viewer.
If the cache does not contain a fresh copy of the object, it fetches it immediately from the origin upon request. Thus, the timing of requests made by you and others will determine how frequently it appears that CloudFront is "updating," even though "updating" isn't really an accurate term for what is occurring.
The CloudFront cache is also not monolithic. If you are in the Eastern U.S. and your user base is in Western Europe, you would potentially see the update sooner, because the edge that is handling your request handles less traffic and is this less likely to have a handled a recent request and have a cached copy available.
After updating your content in S3, create a CloudFront invalidation request for /*. This marks everything cached previously as expired so that all subsequent requests will be sent to the origin server and all viewers will see fresh content. Each AWS account can create 1,000 invalidation requests per month (across all distributions combined), at no cost.
I have a script that I install on a page and it will load some more JS and CSS from an S3 bucket.
I have versions, so when I do a release on Github for say 1.1.9 it will get deployed to /my-bucket/1.1.9/ on S3.
Question, if I want to have something like a symbolic link /my-bucket/v1 -> /my-bucket/1.1.9, how can I achieve this with AWS or CloudFlare?
The idea is that I want to release a new version by deploying it, to my bucket or whatever CDN, and than when I am ready I want to switch v1 to the latest 1.x.y version released. I want all websites to point to /v1 and get the latest when there is new release.
Is there a CDN or AWS service or configuration that will allow me to create a sort of a linux-like symbolic link like that?
A simple solution with CloudFront requires a slight change in your path design:
Bucket:
/1.1.9/v1/foo
Browser:
/v1/foo
CloudFront Origin Path (on the Origin tab)
/1.1.9
Whatever you configure as the Origin Path is added to the beginning of whatever the browser requested before sending the request to the Origin server.
Note that changing this means you also need to do a cache invalidation, because responses are cached based on what was requested, not what was fetched.
There is a potential race condition here, between the time you change the config and invalidate -- there is no correlation in the order of operations between configuration changes and invalidation requests -- a config change followed by an invalidation may be completed after,¹ so will probably need to invalidate, update config, invalidate, verify that the distribution had progressed a stable state, then invalidate once more. You don't need to invalidate objects individually, just /* or /v1*. It would be best if only the resource directly requested is subject to the rewrite, and not it's dependencies. Remember, also, that browser caching is a big cost-saver that you can't leverage as fully if you use the same request URI to represent a different object over time.
More complicated path rewriting in CloudFront requires a Lambda#Edge Origin Request trigger (or you could use Viewer Request, but these run more often and thus cost more and add to overall latency).
¹ Invalidation requests -- though this is not documented and is strictly anecdotal -- appear to involve a bit of time travel. Invalidations are timestamped, and it appears that they invalidate anything cached before their timestamp, rather than before the time they propagate to the edge locations. Architecturally, it would make sense if CloudFront is designed such that invalidations don't actively purge content, but only serve as directives for the cache to consider any cached object as stale if it pre-dates the timestamp on the invalidation request, allowing the actual purge to take place in the background. Invalidations seem to complete too rapidly for any other explanation. This means creating an invalidation request after the distribution returns to the stable Deployed state would assure that everything old is really purged, and that another invalidation request when the change is initially submitted would catch most of the stragglers that might be served from cache before the change is propagated. Changes and invalidations do appear to propagate to the edges via independent pipelines, based on observed completion timing.
We are using AWS cloudfront to render static contents on our site with origin as S3 BUCKET. Now as next steps, the user can dynamically upload images which we want to push to CDN. But we would require different sizes of it so that we can use it later in in the site. One option is to actually do preprocessing of images before pushing to S3 BUCKET . This ends up creating multiple images based on sizes. Can we do post processing something like http://imageprocessor.org/imageprocessor-web/ does but still use cloudfront. Any feedback would be helpful.
Regards
Raghav
Well, yes, it is possible to do post-processing and use CloudFront but you need an intermediate layer between CloudFront and S3. I designed a system using the following high-level implementation:
Request arrives at CloudFront, which serves the image from cache if available; otherwise CloudFront sends the request to the origin server.
The origin server is not S3. The origin server is Varnish, on EC2.
Varnish sends the request to S3, where all the resized image results are stored. If S3 returns 200 OK, the image is returned to CloudFront and to the requesting browser and the process is complete. Since the Varnish machine runs in the same AWS region as the S3 bucket, the performance is essentially indistinguishble between CloudFront >> S3 and CloudFront >> Varnish >> S3.
Otherwise, Varnish is configured to retry the failed request by sending it to the resizer platform, which also runs in EC2.
The resizer examines the request to determine what image is being requested, and what size. In my application, the desired size is in the last few characters of the filename, so xxxxx_300_300_.jpg means 300 x 300. The resizer fetches the source image... resizes it... stores the result in S3... and returns the new image to Varnish, which returns it to CloudFront and to the requester. The resizer itself is Imagemagick wrapped in Mojolicious and uses a MySQL database to identify the source URI where the original image can be fetched.
Storing the results in a backing store, like S3, and checking there, first, on each request, is a critical part of this process, because CloudFront does not work like many people seem to assume. Check your assumptions against the following assertions:
CloudFront has 50+ edge locations. Requests are routed to the edge that optimal for (usually, geographically close to) the viewer. The edge caches are all independent. If I request an object through CloudFront, and you request the same object, and our requests arrive at different edge locations, then neither of us will be served from cache. If you are generating content on demand, you want to save your results to S3 so that you do not have to repeat the processing effort.
CloudFront honors your Cache-Control: header (or overridden values in configuration) for expiration purposes, but does not guarantee to retain objects in cache until they expire. Caches are volatile and CloudFront is no exception. For this reason, too, your results need to be stored in S3 to avoid duplicate processing.
This is a much more complex solution than pre-processing.
I have a pool of millions of images, a large percentage of which would have a very low probability of being viewed, and this is an appropriate solution, here. It was originally designed as a parallel solution to make up for deficiencies in a poorly-architected preprocessor that sometimes "forgot" to process everything correctly, but it worked so well that it is now the only service providing images.
However, if your motivation revolves around avoiding the storage cost of the preprocessed results, this solution won't entirely solve that.