How should I go about disabling the cache of non 200 OK responses in API Gateway.
For one of our API endpoints we implemented our complementary throttle mechanism and we're sending a 429 HTTP response.
The intention is for the client to retry the request after a short time when the server is ready to fulfil it but what happens now is that the API Gateway caches the initial response and keeps sending that from cache instead.
As per the response to Can AWS API Gateway cache invalidate specific entries based on the response content?, the API Gateway cache doesn't appear to have the functionality to just "sometimes" cache a result. The documentation shows a way to have the client make a request that will ignore the existing cache (by setting Cache-Control: max-age=0), but doesn't show a way for the server to say that "this is a single-use response that shouldn't be cached.
The first thing that I think is worth trying is specifying a header like Cache-Control: max-age=0 in your error response just to try it to see if it works. The AWS API Gateway uses CloudFront under the hood for its distribution, so it may just work.
If that doesn't work, other options include:
Turn off the AWS API Gateway cache. If you need a cache, set up your own caching using CloudFront or another service that allows for more fine-grained control over which responses get cached.
Try to move your throttling earlier in the process (I'm not sure if you're using the built-in API Throttling features), but since you've said you "implemented" your mechanism I'm guessing you are doing it yourself in your back-end handling the requests. If you can do throttling before your caching layer (whether it's the built-in API Gateway caching or some other system), that may end up solving your problem and put less strain on your back-end request handler.
After sending 429 responses to the client, when the service is free to handle further requests, send your own "cache invalidation" request with Cache-Control: max-age=0 to get the "real" value cached. Obviously, this would be a bit tricky as you'd need to know when the service is up and available to handle more requests without getting bogged down again with adding a bunch more requests as soon as it's "free" again.
Depending on your exact caching needs, just have a low-enough TTL in your caching settings. For example, if once throttling kicks in, it's likely to not be available again for at least 60 seconds, then having a 60 second TTL means that the 429 response will get served from the cache for that time. But, since you were just throttling anyway and thus your service is "overloaded", it may be acceptable for your situation to continue serving that 429 until the TTL expires. This would need to be the same short TTL for both "success" and "failure" responses, though.
Related
I'm using Google Cloud CDN to cache an HTML page.
I've configured all the correct headers as per the docs, and the page is caching fine. Now, I want to change it so that it only caches when the request has no cookies, i.e. no cookie header set.
My understanding was that this was simply a case of changing my origin server to add a vary: cookie header to all responses for the page, then only adding the caching headers Cache-Control: public and Cache-Control: max-age=300 when no cookie header is set on the request.
However, this doesn't work. Using curl I can see that all caching headers, the vary: cookie header, are set as expected when I send requests with and without cookies, but I never get cache hits on the requests without cookies.
Digging into the Cloud CDN logs, I see that every request with no cookie header has cacheFillBytes populated with the same number as the response size - whereas it's not for the requests with a cookie header set with a value (as expected).
So it appears like Cloud CDN is attempting to populate the cache as expected for requests with no cookies, it's just that I never get a cache hit - i.e. it's just cacheFillBytes every time, cacheHit: true never appears in the logs.
Has anyone come across anything similar? I've triple-checked all my headers for typos, and indeed just removing the vary: cookie header makes caching work as expected, so I'm almost certain my configuration is right in terms of headers and what Cloud CDN considers cacheable.
Should Cloud CDN handle vary: cookie like I'm expecting it to? The docs suggest it handles arbitrary vary headers. And if so, why would I see cacheFillBytes on every request, with Cache-Control: public and Cache-Control: max-age=300 set on the response, but then never see a cacheHit: true on any subsequent request (I've tried firing hundreds with curl in a loop, it really never hits, it's not just that I'm populating a few different edge caches)?
I filed a bug with Google and it turns out that, indeed, the documentation was wrong.
vary: cookie is not supported by Cloud CDN
The docs have been updated - the only headers that can be used with vary are Accept, Accept-Encoding and Origin.
As per the GCP documentation[1], it is informed that Cloud CDN respects any Vary headers that origin servers include in responses. As per this information it looks like vary:cookie is supported by GCP Cloud CDN since any Vary header that the origin serves will be respected by Cloud CDN. Keep in mind though that this will negatively impact caching because the Vary header indicates that the response varies depending on the client's request headers. Therefore, if a request for an object has request header Cookie: abc, then a subsequent request for the same object with request header Cookie: xyz would not be served from the cache.So, yes it is supported and respected but will impact caching (https://cloud.google.com/cdn/docs/troubleshooting-steps?hl=en#low-hit-rate).
[1]https://cloud.google.com/cdn/docs/caching#vary_headers
I have a CloudFront distribution set up so that <domain>/api redirects me to <api-gateway-url>/<env>/api. However I find that sometimes CloudFront caches responses to GET requests and the browser does not redirect to the API Gateway endpoint and returns the cached response.
Example: /api/getNumber redirects to <api-gateway-url>/<env>/api/getNumber and returns me 2. I change the response so that it should return the number 300, but when I make a request through my browser now there is no redirect and I still get back the number two. The x-cache response header says cache hit from CloudFront.
AWS CloudFront is often used for caching, thus decreasing the number of requests that will hit the back-end resources. Therefore you shouldn't use CloudFront on your testing environment if you want to imediately see changes.
In your case it seems, that your endpoint doesn't have any parameters (Path/Query), so essentially what CloudFront sees is the same request every time, naturally in this case you will hit the cache.
You have a couple of options to "fix" that:
Diversify your API requests (using parameters for example)
Use CloudFront's TTL options, to make CloudFront keep the cached objects less time
NOTE: This is not advisable if this is production environment, because it might eliminate the whole point of caching and disrupt expected behavior
Disable CloudFront's caching for those paths that don't take parameters and/or whose response will change often, thus keeping caching on for the rest of your distribution:
https://aws.amazon.com/premiumsupport/knowledge-center/prevent-cloudfront-from-caching-files/
And lastly if this is just your test environment, disable CloudFront, but the things above might later on apply to your production environment
I'm looking for a service that allows me to proxy/modify incoming requests inside AWS.
Currently I am using cloudfront, but that has limited functions.
I need to be able to see user agent strings and make proxy decisions based on that - like reverse proxying to another domain, or routing all requests to /index.html.
Anyone know of a service that within AWS - or outside of AWS.
It sounds like you are describing Lambda#Edge, which is a CloudFront enhancement that allows you to define Lambda functions that will fire at any of 4 hook points in the CloudFront signal flow, and modify the request or generate a dynamic response.
Viewer Request triggers allow inspection/modification of requests and dynamic generation of small responses before the cache lookup.
Origin Request triggers are similar, but fire after the cache is checked. They allow you to inspect and modify the request, including changing the origin server, path, and/or query string, or to generate a response instead of allowing CloudFront to proceed with the connection to the origin.
If the request goes to the origin, then once it returns, an Origin Response trigger can fire to modify the response headers or replace the response body with a different body you generate. The response after this trigger is finished with it is what gets stored in the cache, if cacheable.
Once a reaponse is cached, further firing of the Origin Request and Origin Response triggers doesn't occur for subsequent requests that can be served from the cache.
Finally, when the response is ready, whether it came from the cache or the origin, a Viewer Response trigger can modify it further, if desired.
Response triggers can also inspect many of the headers from the original request.
Lambda#Edge functions are written in Node.js, and are presented with the request or responses as simple structured objects that you inspect and/or modify.
I am trying to set-up API Gateway + Lambda Proxy + Cache and having issues with it.
Firstly it doesn't seem to be hitting the cache. Oddly when I use chrome dev tools to hit it and click 'disable' cache, it somehow does hit the cache! The difference is it sets the cache-control to 'none'. A normal requests it sets cache-control to 'max-age=0'.
Secondly when it does cache, its ignoring the URL parameter as a key so with different querystring values, it still returns the same response.
I have enabled the API Cache on the stage settings, I have also entered the 'querystrings' in the Method request settings as well and turned on the 'caching' checkbox for each querystring parameter.
Any thoughts on this? If this does not work I may have to resort to using Elasticache internally but it would have been great if API Gateway could handle this directly.
My API request are 'GET' requests with querystring parameters.
Let's say I have two files: one for safari and one for Firefox.
I want to check User-Agent and return file based on the User-Agent.
How do I do this without adding external server?
You can't do this without adding an extra server.
S3 supports static content. It does not¹ vary its response based on request headers.
CloudFront relies on the origin server if content needs to vary based on request headers. Note that by default, CloudFront doesn't forward most headers to the origin, but this can be changed in the cache behavior configuration. If you forward the User-Agent header to the origin, your cache hit rate drops dramatically, since CloudFront has no choice but to assume any and every change in the user agent string could trigger a change in the response, so an object in the cache that was requested by a specific user agent string will only be served to a future browser with an identical user agent string. It will cache each different copy, but this still hurts your hit rate. If you only want to know the general type of browser, CloudFront can inject special headers to tell the origin whether the user agent is desktop, smart-tv, mobile, or tablet, without actually forwarding the user agent string and causing the same negative impact on the cache hit ratio.
So CloudFront will correctly cache the appropriate version of a page for each unique user agent... but the origin server must implement the actual content selection logic. And when the origin is S3, that isn't supported -- unless you have a server between CloudFront and S3. This is a perfectly valid configuration -- I have such a setup, with a server that rewrites the request path received from CloudFront before sending the request to S3, then returns the content from S3 back to CloudFront, which returns the content to the browser.
AWS Lambda would be a potential candidate for an application like this, acting as the necessary server (a serverless server, if you will) between CloudFront and S3... but it does not yet suport binary data, so for anything other than text, that isn't an option, either.
¹At least, not in any sense that is relevant, here. Exceptions exist for CORS and when access is granted or denied based on a limited subset of request headers.