This page provides public keys to decrypt headers from Google's Identity Aware Proxy. Making a request to the page provides its own set of headers, one of which is Expires (it contains a datetime).
What does the expiration date actually mean? I have noticed it fluctuating occasionally, and have not noticed the public keys changing at the expiry time.
I have read about Securing Your App With Signed Headers, and it goes over how to fetch the keys after every key ID mismatch, but I am looking to make a more efficient cache that can fetch the keys less often based on the expiry time.
Here are all the headers from the public keys page:
Accept-Ranges →bytes
Age →1358
Alt-Svc →quic=":443"; ma=2592000; v="39,38,37,36,35"
Cache-Control →public, max-age=3000
Content-Encoding →gzip
Content-Length →519
Content-Type →text/html
Date →Thu, 29 Jun 2017 14:46:55 GMT
Expires →Thu, 29 Jun 2017 15:36:55 GMT
Last-Modified →Thu, 29 Jun 2017 04:46:21 GMT
Server →sffe
Vary →Accept-Encoding
X-Content-Type-Options →nosniff
X-XSS-Protection →1; mode=block
The Expires header controls how long HTTP caches are supposed to hold onto that page. We didn't bother giving Google's content-serving infrastructure any special instructions for the keyfile, so whatever you're seeing there is the default value.
Is there a reason the "refresh the keyfile on lookup failure" approach isn't a good fit for your application? I'm not sure you'll be able to do any better than that, since:
Unless there's a bug or problem, you should never get a key lookup failure.
Even if you did have some scheduled key fetch, it'd probably still be advisable to refresh the keyfile on lookup failure as a fail-safe.
We don't currently rotate the keys super-frequently, though that could change in the future (which is why we don't publish the rotation interval), so it shouldn't be a significant source of load. Are you observing that refreshing the keys is impacting you?
--Matthew, Google Cloud IAP engineer
Related
Cloudfront is configured to cache the images from our app. I found that the images were evicted from the cache really quickly. Since the images are generated dynamically on the fly, this is pretty intense for our server. In order to solve the issue I set up a testcase.
Origin headers
The image is served from our origin server with correct Last-Modified and Expires headers.
Cloudfront cache behaviour
Since the site is HTTPS only I set the Viewer Protocol Policy to HTTPS. Forward Headers is set to None and Object Caching to Use Origin Cache Headers.
The initial image request
I requested an image at 11:25:11. This returned the following status and headers:
Code: 200 (OK)
Cached: No
Expires: Thu, 29 Sep 2016 09:24:31 GMT
Last-Modified: Wed, 30 Sep 2015 09:24:31 GMT
X-Cache: Miss from cloudfront
A subsequent request
A reload a little while later (11:25:43) returned the image with:
Code: 304 (Not Modified)
Cached: Yes
Expires: Thu, 29 Sep 2016 09:24:31 GMT
X-Cache: Hit from cloudfront
A request a few hours later
Nearly three hours later (at 14:16:11) I went to the same page and the image loaded with:
Code: 200 (OK)
Cached: Yes
Expires: Thu, 29 Sep 2016 09:24:31 GMT
Last-Modified: Wed, 30 Sep 2015 09:24:31 GMT
X-Cache: Miss from cloud front
Since the image was still cached by the browser it loaded quickly. But I cannot understand how the Cloudfront could not return the cached image. Therefor the app had to generate the image again.
I read that Cloudfront evicts files from its cache after a few days of being inactive. This is not the case as demonstrated above. How could this be?
I read that Cloudfront evicts files from its cache after a few days of being inactive.
Do you have an official source for that?
Here's the official answer:
If an object in an edge location isn't frequently requested, CloudFront might evict the object—remove the object before its expiration date—to make room for objects that have been requested more recently.
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Expiration.html
There is no guaranteed retention time for cached objects, and objects with low demand are more likely to be evicted... but that isn't the only factor you may not have considered. Eviction may not be the issue, or the only issue.
Objects cached by CloudFront are like Schrödinger's cat. It's a loose analogy, but I'm running with it: whether an object is "in the cloudfront cache" at any given instant is not a yes-or-no question.
CloudFront has somewhere around 53 edge locations (where your browser connects and the content is physically stored) in 37 cities. Some major cities have 2 or 3. Each request that hits cloudfront is routed (via DNS) to the most theoretically optimal location -- for simplicity, we'll call it the "closest" edge to where you are.
The internal workings of Cloudfront are not public information, but the general consensus based on observations and presumably authoritative sources is that these edge locations are all independent. They don't share caches.
If, for example, your are in Texas (US) and your request routed through and was cached in Dallas/Fort Worth, TX, and if the odds are equal that you any request from you could hit either of the Dallas edge locations, then until you get two misses of the same object, the odds are about 50/50 that your next request will be a miss. If I request that same object from my location, which I know from experience tends to route through South Bend, IN, then the odds of my first request being a miss are 100%, even though it's cached in Dallas.
So an object is not either in, or not in, the cache because there is no "the" (single, global) cache.
It is also possible that CloudFront's determination of the "closest" edge to your browser will change over time.
CloudFront's mechanism for determining the closest edge appears to be dynamic and adaptive. Changes in the topology of the Internet at large can change shift which edge location will tend to receive requests sent from a given IP address, so it is possible that over the course of a few hours, that the edge you are connecting to will change. Maintenance or outages or other issues impacting a particular edge could also cause requests from a given source IP address to be sent to a different edge than the typical one, and this could also give you the impression of objects being evicted, since the new edge's cache would be different from the old.
Looking at the response headers, it isn't possible to determine which edge location handled each request. However, this information is provided in the CloudFront access logs.
I have a fetch-and-resize image service that handles around 750,000 images per day. It's behind CloudFront, and my hit/miss ratio is about 50/50. That is certainly not all CloudFront's fault, since my pool of images exceeds 8 million, the viewers are all over the world, and my max-age directive is shorter than yours. It has been quite some time since I last analyzed the logs to determine which and how "misses" seemed unexpected (though when I did, there definitely were some, but their number was not unreasonable), but that is done easily enough, since the logs tell you whether each response was a hit or a miss, as well as identifying the edge location... so you could analyze that to see if there's really a pattern here.
My service stores all of its output content in S3, and when a new request comes in, it first sends a quick request to the S3 bucket to see if there is work that can be avoided. If a result is returned by S3, then that result is returned to CloudFront instead of doing all the fetching and resizing work, again. Mind you, I did not implement that capability because of the number of CloudFront misses... I designed that in from the beginning, before I ever even tested it behind CloudFront, because -- after all -- CloudFront is a cache, and the contents of a cache are pretty much volatile and ephemeral, by definition.
Update: I stated above that it does not appear possible to identify the edge location forwarding a particular request by examining the request headers from CloudFront... however, it appears that it is possible with some degree of accuracy by examining the source IP address of the incoming request.
For example, a test request sent to one of my origin servers through CloudFront arrives from 54.240.144.13 if I hit my site from home, or 205.251.252.153 when I hit the site from my office -- the locations are only a few miles apart, but on opposite sides of a state boundary and using two different ISPs. A reverse DNS lookup of these addresses shows these hostnames:
server-54-240-144-13.iad12.r.cloudfront.net.
server-205-251-252-153.ind6.r.cloudfront.net.
CloudFront edge locations are named after the nearest major airport, plus an arbitrarily chosen number. For iad12 ... "IAD" is the International Air Transport Association (IATA) code for Washington, DC Dulles airport, so this is likely to be one of the edge locations in Ashburn, VA (which has three, presumably with different numerical codes at the end, but I can't confirm that from just this data). For ind6, "IND" matches the airport at Indianapolis, Indiana, so this strongly suggests that this request comes through the South Bend, IN, edge location. The reliability of this test would depend on the consistency with which CloudFront maintains its reverse DNS entries. It is not documented how many independent caches might be at any given edge location; the assumption is that there's only one, but there might be more than one, having the effect of increasing the miss ratio for very small numbers of requests, but disappearing into the mix for large numbers of requests.
Is X-Amz-Expires a required header/parameter? Official documentation is inconsistent and uses it in some examples, while not in others.
If it is not required, what is the default expiration value of a signed request? Does it equal to the maximum possible value for X-Amz-Expires parameter, which is 604800 (seven days)?
The documentation (see above links) talks about X-Amz-Expires parameter only in context of passing signing parameters in a query string. If X-Amz-Expires parameter is required, is it only required for passing signing parametes in query string (as opposed to passing them with Authorization header)?
Update:
Introduction to AWS Security Processes paper, on page 17 says
A request must reach AWS within 15 minutes of the
time stamp in the request. Otherwise, AWS denies the request.
Now what time stamp are we talking about here? My guess is that it is X-Amz-Date. If I am correct, then another question crops up:
How do X-Amz-Date and X-Amz-Expires parameters relate to each other? To me it sounds like request expiration algorithm falls back to 15 mins from X-Amz-Date timestamp, if X-Amz-Expire is not present.
Is X-Amz-Expires a required header/parameter?
X-Amz-Expires is only used with query string authentication, not with the Authorization: header.
There is no default value with query string authentication. It is a required parameter, and the service will reject a request if X-Amz-Algorithm=AWS4-HMAC-SHA256 is present in the query string but X-Amz-Expires=... is not.
<Error>
<Code>AuthorizationQueryParametersError</Code>
...
Now what time stamp are we talking about here?
This refers to X-Amz-Date: when used with the Authorization: header. Because X-Amz-Date: is part of the input to the signing algorithm, a change in the date or time also changes the signature. An otherwise-identical request signed 1 second earlier or later has an entirely different signature. AWS essentially allows your server clock to be wrong by up to 15 minutes without breaking your ability to authenticate requests. It is not a fallback or a default. It is a fixed window.
The X-Amz-Date: of Authorization: header-based requests is compared by AWS to their system time, which is of course synched to UTC, and the request is rejected out if hand if this value is more than 15 minutes skewed from UTC when the request arrives. No other validation related to authentication occurs prior to the time check.
Validation of Query String authentication expiration involves different logic:
X-Amz-Expires must not be a value larger than 604800 or smaller than 0; otherwise the request is immediately denied without further processing, including a message similar to the one above.
X-Amz-Date must not be more than 15 minutes in the future, according to the AWS system clock. The error is Request is not yet valid.
X-Amz-Date must not be more than X-Amz-Expires number of seconds in the past, relative to the AWS system clock, and no 15 minute tolerance applies. The error is Request has expired.
If any of these conditions occur, no further validation is done on the signature, so these messages will not change depending on the validity of the signature. This is checked first.
Also, the leftmost 8 characters of your X-Amz-Date: must match the date portion of your Credential component of the Authorization: header. The date itself has zero tolerance for discrepancy against the credential (so, when signing, don't read your system time twice, else you risk generating an occasional invalid signature around midnight UTC).
Finally, requests do not expire while in the middle of processing. If you send a request using either signing method that is deemed valid when it arrives but would have expired very soon thereafter, it is always allowed to run to completion -- for example, a large S3 download or an EBS snapshot creation request will not start, then fail to continue, because the expiration timer struck while the request had already started on the AWS side. If the action was authorized when requested, then it continues and succeeds as normal.
I'm having a problem and tried to follow answers here in forum, but with no success whatsoever.
In order to generate thumbnails, I have set up the following schema:
S3 Account for original images
Ubuntu Server using NGINX and Thumbor
Cloudfront
The user uploads original images to S3, which will be pulled through Ubuntu Server with Cloudfront in front of the request:
http://cloudfront.account/thumbor-server/http://s3.aws...
The big deal is, that we often loose objects in Cloudfront, I want them to stay 360 days in cache.
I get following response through Cloudfront URL:
Cache-Control:max-age=31536000
Connection:keep-alive
Content-Length:4362
Content-Type:image/jpeg
Date:Sun, 26 Oct 2014 09:18:31 GMT
ETag:"cc095261a9340535996fad26a9a882e9fdfc6b47"
Expires:Mon, 26 Oct 2015 09:18:31 GMT
Server:nginx/1.4.6 (Ubuntu)
Via:1.1 5e0a3a528dab62c5edfcdd8b8e4af060.cloudfront.net (CloudFront)
X-Amz-Cf-Id:B43x2w80SzQqvH-pDmLAmCZl2CY1AjBtHLjN4kG0_XmEIPk4AdiIOw==
X-Cache:Miss from cloudfront
After a new refresh, I get:
Age:50
Cache-Control:max-age=31536000
Connection:keep-alive
Date:Sun, 26 Oct 2014 09:19:21 GMT
ETag:"cc095261a9340535996fad26a9a882e9fdfc6b47"
Expires:Mon, 26 Oct 2015 09:18:31 GMT
Server:nginx/1.4.6 (Ubuntu)
Via:1.1 5e0a3a528dab62c5edfcdd8b8e4af060.cloudfront.net (CloudFront)
X-Amz-Cf-Id:slWyJ95Cw2F5LQr7hQFhgonG6oEsu4jdIo1KBkTjM5fitj-4kCtL3w==
X-Cache:Hit from cloudfront
My Nginx responses as following:
Cache-Control:max-age=31536000
Content-Length:4362
Content-Type:image/jpeg
Date:Sun, 26 Oct 2014 09:18:11 GMT
Etag:"cc095261a9340535996fad26a9a882e9fdfc6b47"
Expires:Mon, 26 Oct 2015 09:18:11 GMT
Server:nginx/1.4.6 (Ubuntu)
Why does Cloudfront not store my objects as indicated? Max-Age is set?
Many thanks in advance.
Your second request shows that the object was indeed cached. I assume you see that, but the question doesn't make it clear.
The Cache-Control: max-age only specifies the maximum age of your objects in the Cloudfront Cache at any particular edge location. There is no minimum time interval for which your objects are guaranteed to persist... after all, Cloudfront is a cache, which is volatile by definition.
If an object in an edge location isn't frequently requested, CloudFront might evict the object—remove the object before its expiration date—to make room for objects that are more popular.
— http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Expiration.html
Additionally, there is no concept of Cloudfront as a whole having a copy of your object. Each edge location's cache appears to operate independently of the others, so it's not uncommon to see multiple requests for relatively popular objects coming from different Cloudfront edge locations.
If you are trying to mediate the load on your back-end server, it might make sense to place some kind of cache that you control, in front of it, like varnish, squid, another nginx or a custom solution, which is how I'm accomplishing this in my systems.
Alternately, you could store every result in S3 after processing, and then configure your existing server to check S3, first, before attempting the work of resizing the object again.
Then why is there a documented "minimum" TTL?
On the same page quoted above, you'll also find this:
For web distributions, if you add Cache-Control or Expires headers to your objects, you can also specify the minimum amount of time that CloudFront keeps an object in the cache before forwarding another request to the origin.
I can see why this, and the tip phrase cited on the comment, below...
The minimum amount of time (in seconds) that an object is in a CloudFront cache before CloudFront forwards another request to your origin to determine whether an updated version is available.
...would seem to contradict my answer. There is no contradiction, however.
The minimum ttl, in simple terms, establishes a lower boundary for the internal interpretation of Cache-Control: max-age, overriding -- within Cloudfront -- any smaller value sent by the origin server. Server says cache it for 1 day, max, but configured minimum ttl is 2 days? Cloudfront forgets about what it saw in the max-age header and may not check the origin again on subsequent requests for the next 2 days, rather than checking again after 1 day.
The nature of a cache dictates the correct interpretation of all of the apparent ambiguity:
Your configuration limits how long Cloudfront MAY serve up cached copies of an object, and the point after which it SHOULD NOT continue to return the object from its cache. They do not mandate how long Cloudfront MUST maintain the cached copy, because Cloudfront MAY evict an object at any time.
If you set the Cache-Control: header correctly, Cloudfront will consider the larger of max-age or your Minimum TTL as the longest amount of time you want them to serve up the cached copy without consulting the origin server again.
As your site traffic increases, this should become less of an issue, since your objects will be more "popular," but fundamentally there is no way to mandate that Cloudfront maintain a copy of an object.
<RequestTime>Mon, 14 Mar 2011 10:09:28 GMT</RequestTime>
<ServerTime>2011-03-14T09:09:29Z</ServerTime></Error>
reason: The reason of this problem is that Amazon S3 allows only a small time stamp variation up to 15 minutes between the server and its requesting client (user pc). As Amazon is a big backup server of large number of users, security does matter a lot.
solution: I installed ntp on my ubuntu machine and try to sync it with s3. But still throwing same error.
How can I solved it.
My project is in Django
Make sure you use UTC time for your requests. From the AWS docs:
Request Elements
Time stamp—Each request must contain the date and time the request
was created, represented as a string
in UTC.
I had the same problem: Update your date with the following:
rdate -s ntp.xs4all.nl
substitute with whatever ntp server you require.
I am thinking of a rest web service that ensure for every request sent to him that :
The request was generated by the user who claim it ;
The request has not been modified by someone else (uri/method/content/date);
For GET requests, it should be possible to generate a URI with enough information in it to check the signature and set a date of expiration. That way a user can delegate temporary READ permissions to a collaborator for a limited time period on a ressource with a generated URI.
Clients are authenticated with id and a content-signature based on their password.
There should be no session at all, and so server state ! The server and the client share a secret key (a password)
After thinking about it and talking with some really nice folks, it seems there is no rest service existing to do that as simple as it should be for my use case. (HTTP Digest and OAuth can do this with server state and are very chatty)
So I Imagined one, and I'm asking your greats comments on how it should be designed (I will release it OpenSource and Hope it can help others).
The service use a custom "Content-signature" header to store credentials. An authenticated request should contains this header :
Content-signature: <METHOD>-<USERID>-<SIGNATURE>
<METHOD> is the sign method used, in our case SRAS.
<USERID> stands for the user ID mentioned earlier.
<SIGNATURE> = SHA2(SHA2(<PASSWORD>):SHA2(<REQUEST_HASH>));
<REQUEST_HASH> = <HTTP_METHOD>\n
<HTTP_URI>\n
<REQUEST_DATE>\n
<BODY_CONTENT>;
A request is invalidated 10 minutes after it has been created.
For example a typical HTTP REQUEST would be :
POST /ressource HTTP/1.1
Host: www.elphia.fr
Date: Sun, 06 Nov 1994 08:49:37 GMT
Content-signature: SRAS-62ABCD651FD52614BC42FD-760FA9826BC654BC42FD
{ test: "yes" }
The server will answer :
401 Unauthorized
OR
200 OK
Variables would be :
<USERID> = 62ABCD651FD52614BC42FD
<REQUEST_HASH> = POST\n
/ressource\n
Sun, 06 Nov 1994 08:49:37 GMT\n
{ test: "yes" }\n
URI Parameters
Some parameters can be added to the URI (they overload the headers informations) :
_sras.content-signature=<METHOD>-<USERID>-<SIGNATURE> : PUT the credentials in the URI, not in the HTTP header. This allow a user to share a signed request ;
_sras.date=Sun, 06 Nov 1994 08:49:37 GMT (request date*) : The date when the request was created.
_sras.expires=Sun, 06 Nov 1994 08:49:37 GMT (expire date*) : Tell the server the request should not expire before the specified date
*date format : http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.18
Thanks for your comments.
There are several issues that you need to consider when designing a signature protocol. Some of these issues might not apply to your particular service:
1- It is customary to add an "X-Namespace-" prefix to non-standard headers, in your case you could name your header something like: "X-SRAS-Content-Signature".
2- The Date header might not provide enough resolution for the nonce value, I would therefore advise for a timestamp having at least 1 millisecond of resolution.
3- If you do not store at least the last nonce, one could still replay a message in the 10 minutes window, which is probably unacceptable on a POST request (could create multiple instances with same values in your REST web service). This should not be a problem for GET PUT or DELETE verbs.
However, on a PUT, this could be used for a denial of service attack by forcing to update many times the same object within the proposed 10 minutes window. On a GET or DELETE a similar problem exists.
You therefore probably need to store at least the last used nonce associated with each user id and share this state between all your authentication servers in real-time.
4- This method also requires that the client and servers be clock synchronized with less than 10 minutes skew. This can be tricky to debug, or impossible to enforce if you have AJAX clients for which you do not control the clock. This also requires to set all timestamps in UTC.
An alternative is to drop the 10 minutes window requirement but verify that timestamps increase monotonically, which again requires to store the last nonce. This is still a problem if the client's clock is updated to a date prior to the last used nonce. Access would be denied until the client's clock pass the last nonce or the server nonce state is reset.
A monotonically increasing counter is not an option for clients that cannot store a state, unless the client could request the last used nonce to the server. This would be done once at the beginning of each session and then the counter would be incremented at each request.
5- You also need to pay attention to retransmissions due to networks errors. You cannot assume that the server has not received the last message for which a TCP Ack has not been received by the client before the TCP connection dropped. Therefore the nonce needs to be incremented between each retransmission above the TCP level and the signature re-calculated with the new nonce. Yet a message number needs to be added to prevent double execution on the server: a double POST would result in 2 object being created.
6- You also need to sign the userid, otherwise, an attacker might be able to replay the same message for all users which nonces have not yet reached that of the replayed message.
7- Your method does not guaranty the client that the server is authentic and has not been DNS-hijacked. Server authentication is usually considered important for secure communications. This service could be provided by signing responses from the server, using the same nonce as that of the request.
I would note that you can accomplish this with OAuth, most notably "2-legged OAuth" where client and server share a secret. See https://www.rfc-editor.org/rfc/rfc5849#page-14. In your case, you want to omit the oauth_token parameter and probably use the HMAC-SHA1 signature method. There's nothing particularly chatty about this; you don't need to go through the OAuth token acquisition flows to do things this way. This has the advantage of being able to use any of several existing open source OAuth libraries.
As far as server-side state, you do need to keep track of what secrets go with which clients, as well as which nonces have been used recently (to prevent replay attacks). You can skip the nonce checking / lifetimes if you run things over HTTPS, but if you're going to do that, then HTTPS + Basic Auth gives you everything you described without having to write new software.