from the docs here,
CloudFront adds the header to the viewer request before forwarding the request to your origin. The header value contains an encrypted string that uniquely identifies the request.
Now, when inspecting my appsync response on the client (postman), I find x-amz-cf-id as a response header. But I am sure my system has nothing to do with CloudFront,
My questions,
Does appsync uses cloudfront, (by default somewhere internally) ?
In what scenario(s) is this header added to my appsync response ?
What does it tells me(client) with respect to appsync (other than the cloudfront-req-id ofcourse) ?
appsync, api gateway, cognito and other lots of services use cloudFront in the backend/internally by AWS for high performance and to reduce latency etc purposes. CloudFront is playing just a role of reverse proxy , nothing else.
Because CloudFront always serves x-amz-cf-id header in response which can be used to track individual request(the string is unique), There is no use of this for appsync other then if AWS asks you provide his to track some failed requests if you contact them.
When you resolve the appsync endpoint and perform a reverse dns on that IP address, you'll see it actually use cloudfront.
Related
I have an application where the client is hosted on S3 with a CloudFront distribution. The API is behind an API Gateway with a WAF, and the client makes http requests to the API to fetch and post data.
I want to restrict the access to the API such that it's only available from the client, and it should return an error when someone tries to access the API directly.
The trick is that the API is exposed to a 3rd party, so I cannot use API Gateway authorizers, because they must have direct access.
I set up a Custom Origin Header (My-Secret-Header: 1234567890qwertyuiop) in CloudFront, and I thought that I could create a rule in WAF to allow requests with this header (plus the 3rd party based on other criteria, but that part is working well, and it's not an issue), and block everything else.
The problem is that My-Secret-Header never makes it to the WAF, and it doesn't get added to the http requests originated from the client application.
I also tried to add the custom header with Lambda#Edge, no success. I created heaps of logs in with Lambda#Edge, and the event.Records[0].cf.request.origin.s3.customHeaders shows My-Secret-Header (which is expected).
What is the best way to add a custom header to the client request, so that it would be possible to create a rule in WAF?
I want to restrict the access to the API such that it's only available from the client, and it should return an error when someone tries to access the API directly.
The short answer is: there is no way to do this. There is no way to tell if a request originates from JavaScript in the browser, a Postman call, a user typing the URL in the address bar, etc.
Custom Headers in CloudFront are not headers that are added onto API requests that the user makes from the served files. They are headers that CloudFront uses to retrieve the static source that it is serving. In the case that the source is in an S3 bucket, these custom headers are on the request that CloudFront uses to retrieve files from your S3 bucket.
Once a user has the files that CloudFront serves (HTML, CSS, JavaScript, assets, etc.), CloudFront is no longer a part of the process. Any API calls made on the frontend do not go through CloudFront.
There are a few very weak ways to do what you are trying to do, but all are easily bypassable and absolutely cannot be used when security is in any way necessary. For instance, you can make an API Key and hard-code it into the application, but it is completely exposed to anyone who can access the page. Same for hard-coded access key ID and secret access key.
Ultimately what you need is an authentication system, some way to make sure that users are allowed to make the API calls that they are making. I don't know if this fits your use case, but Amazon Cognito is an excellent service that handles user authentication and federation.
I'm using Apigateway with a Cloudfront distribution. I've setup a custom Authorizer. The distribution is configured to forward the whitelisted header Authorization.
As an example, i've got 2 urls:
http://domain.tld/me (the logged in user informations)
http://domain.tld/products (the products collection)
The goal is to cache /products but not based on Authorization header. On the other side, I would like /me to be cached with Authorization header (because I don't want user B get cached information from user A). The both urls are protected with the authorizer.
How can i do that ?
If you need to authenticate a request based on the Authorization header before allowing them to see /products yet you don't want the object to be cached based on the Authorization header, then you can't use CloudFront with an API Gateway custom authorizer, because there is no way for CloudFront to pass a header through without caching on it -- such a configuration doesn't make sense, because CloudFront wouldn't know whether the request should be authorized at all.
One workaround is to use a Lambda#Edge Viewer Request trigger instead of a custom authorizer, to allow/disallow access to CloudFront on the /products path based on Authorization. If a Viewer Request trigger returns control to CloudFront (rather that emitting its own internally-generated response) then the cache is checked and the response served from the cache (hit) or fetched from the origin (miss). The header doesn't need to be whitelisted in order to be accessible to a viewer request trigger.
If the backend requires a generic authentication token for a path that doesn't vary based on user, then CloudFront can inject that using Custom Origin Headers.
I would like to use AWS's Server Side Encryption (SSE) with the AWS Key Management Service (KMS) to encrypt data at rest in S3. (See this AWS blog post detailing SSE-KMS.)
However, I also have the requirement that I use Cloudfront Presigned URLs.
How can I set up a Cloudfront distribution to use a key in AWS KMS to decrypt and use S3 objects encrypted at rest?
(This Boto3 issue seems to be from someone looking for the same answers as me, but with no results).
This was previously not possible because CloudFront didn't support it and because (as I mentioned in comments on John's answer -- which was on the right track) there was no way to roll-your-own solution with Lambda#Edge because the X-Amz-Cf-Id request header --generated on the back side of CloudFront and visible only to S3, not to the trigger invocation -- would invalidate any signature you tried to add to the request inside a Lambda#Edge trigger, because signing of all X-Amz-* headers is mandatory.
But the X-Amz-Cf-Id header value is now exposed to a Lambda#Edge trigger function in the event structure -- not with the other request headers, but as a simple string attribute -- at event.Records[0].cf.config.requestId.
With that value in hand, you can use the execution role credentials and the built-in SDK in the Lambda#Edge environment to generate a signature and and add the necessary headers (including an Authorization header with the derived credential identifier and freshly-generated signature) to the request.
This setup does not use an Origin Access Identifier (OAI) because the Lambda#Edge trigger's IAM Execution Role is used instead of an OAI to persuade S3 that the request is authorized.
Achraf Souk has published an official AWS blog post explaining the solution from start to finish.
https://aws.amazon.com/blogs/networking-and-content-delivery/serving-sse-kms-encrypted-content-from-s3-using-cloudfront/
Use S3 Presigned URLs. This AWS article discusses how to generate urls using Java, but this is easily ported to another language.
Server-Side Encryption with AWS Key Management Service (SSE-KMS)
The following setup works for us:
In your application, generate a signed URL that Cloudfront can validate (https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html).
Instead of using OAI, you create a Lambda#Edge origin request function as per: https://aws.amazon.com/blogs/networking-and-content-delivery/serving-sse-kms-encrypted-content-from-s3-using-cloudfront/
Please note that if your bucket contains an '.' (ours did), there's a bug in the JS code that can be mitigated with something like:
// Infer the region from the host header
// const region = options.host.split('.')[2];
const hostArr = options.host.split('.');
const region = hostArr [hostArr.length - 3];
Lastly, we added an origin response Lambda#Edge to wash away headers that we do not want exposed. Esp the X-Amz-Server-Side-Encryption-Aws-Kms-Key-Id that includes the AWS Account ID.
Lastly I'd like to comment on the above statement/comment that Lamda#Edge response bodies are limited to 1 MB, this only applies to content generated (or modified if you include the body) by the lambda function.
When using the Lambda#Edge function above, the response from the S3 origin has no such limit, we are serving objects >> 1MB (normally 100+ MB).
I'm a bit confused by how API Gateway and CloudFront work together. Ultimately, I want to be able to have a custom header and value be considered part of my cache key. I know this can be done by whitelisting (if I'm using CloudFront).
So when I make the following request:
GET /pagesRead/4
Some-Header: fizz
This returns, for instance, '29 pages'
Then there's a post that updates id 4 to '45 pages'
If I make this request
GET /pagesRead/4
Some-Header: buzz
It will now return '45 pages'
But I'm using API Gateway, which obviously has it's own CloudFront behind the scenes. Is there a way I can configure API Gateway to use its 'behind-the-scenes' CloudFront to whitelist my custom header? Does this even need to be done?
According to this documentation: AWS-API-Gatway, It seems like I can just enable API caching in API Gateway, and it will consider my headers as part of the cache key.
Am I understanding this correctly? If all I want is for my headers to be a part of the cache key, what's the difference between 'Enabling API Caching' in API Gateway and adding a CloudFront instance on top of API Gateway and white-listing in CloudFront?
UPDATE:
I've added a header like this in API Gateway:
But on GET, I am getting stale data from the cache.
GET /pagesRead/4 test-header: buzz
The difference is that API Gateway doesn't actually use the CloudFront cache. CloudFront does provide some front-end services for all API Gateway APIs edge-optimized API endpoints¹, but caching does not appear to be one of them, based on the following:
API Gateway enables caching by creating a dedicated cache instance.
...and...
You should not use the X-Cache header from the CloudFront response to determine if your API is being served from your API Gateway cache instance.
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-caching.html
It is possible to cascade an Edge Optimized API Gateway endpoint behind a CloudFront distribution that you create, but it's not without certain inconveniences. Latency increases somewhat, since you're passing through more systems. Given that configuration, the CloudFront-Is-*-Viewer and CloudFront-Viewer-Country headers, and probably any notion of the client IP will be invalid, because the API Gateway deployment will see attributes of the additional CloudFront distribution that is in front of it, rather than of the real client. X-Forwarded-For will still be right, but will have to be handled with care, because it will contain one extra hop that will have to be correctly handled.
For an application where you want to put API Gateway behind your own CloudFront distribution, use one of the new Regional endpoints to deploy your API stage.
it will consider my headers as part of the cache key.
You do have to configure the cache key explicitly, based on the document you cited, but yes, the API Gateway cache will then cache responses based on the value of that header, and other attributes in the cache key.
¹ edge optimized endpoints. API Gateway now has two different kinds of endpoints. The original design is now called edge-optimized, and the new option is called regional. Regional endpoints do not use front-end services from CloudFront, and may offer lower latency when accessed from EC2 within the same AWS region. All existing endpoints were categorized as edge-optimized when the new regional capability was rolled out. With a regional endpoint, the CloudFront-* headers are not present in the request, unless you use your own CloudFront distribution and whitelist those headers for forwarding to the origin.
When you enable caching in API Gateway,
You can also optionally add,
RequestPath
QueryStringParameters
Http Headers
E.g.,
http://example.com/api/{feature}/?queryparam=queryanswer [ with header customheader=value1 ]
Above url gives you option to cache based on,
Just the URL without PathParameters: http://example.com/api/
Optionally include PathParameter: http://example.com/api/{feature}/
Optionally include QueryStrings: http://example.com/api/{feature}/?queryparam=queryanswer
Optionally include Http Headers: You can either include regular header like User-Agent or Custom headers
Whatever the caching mode you have in API-Gateway, you can also have it under CloudFront as well.
Also to clear up the cache, in your http response send Cache-Control: max-age=0
Hope it helps.
Is it possible to enable/disable caching a request through the AWS API Gateway in the response of the request?
According to this document: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-caching.html It appears that the most granular one can get in defining cache settings is enabling/disabling caching for a specific API function. What I am wanting to do is allow the response for the API request to dictate whether or not it is to be cached. (i.e. I want my end API program to be able to determine if a response for a given request should be cached).
Is this possible, and if so how can it be accomplished?
Configure your own CloudFront distribution, with the API Gateway endpoint as the origin server. CloudFront web distributions respect Cache-Control headers from the origin server. If you customize that response, this should accomplish your objective.
API Gateway, as you may already know, runs behind some of the CloudFront infrastructure already, so this might seem redundant, but this appears to be the only way to take control of the caching behavior.