I would like to add a query string parameter to my Cloudfront Url to be able to get some additional info into the Cloudfront log. I have two distributions, one is signed and one is not signed, pointing to two different S3 buckets (one with audio, one with images).
Access to both distributions works fine without added query strings, but if I add a query parameter like the test one below:
https://x.cloudfront.net/audio.m4a?li=...62&Expires=1544430879&Signature=...QTQ__&Key-Pair-Id=xxx&test=fail
https://y.cloudfront.net/image.jpg?test=allgood
The first one fails (Access Denied) but the second one works fine.
Neither one of the distributions forwards the query string to S3.
The signed audio distribution has logging enabled while the image distribution doesn't have logging. Besides this, their setups are the same.
What do I need to do in order to get the audio distribution to accept my custom query parameter?
Thanks
/o
One of the core concepts behind signed URLs is that they are not vulnerable to tampering -- you can't change a signed URL and have it remain valid.
CloudFront uses the public key to validate the signature and confirm that the URL hasn't been tampered with. If the signature is invalid, the request is rejected.
...
Signed CloudFront URLs cannot contain extra query string arguments. If you add a query string to a signed URL after you create it, the URL returns an HTTP 403 status.
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html
To add a query string parameter to a CloudFront signed URL, you need to add it before signing the URL... because the addition will change the signature.
Additional/Optional parameters can be allowed for a signed cloudfront URL by appending an asterisk (*) to the resource in the signature policy.
For example, if you sign the URL: https://example.com/images/funny/cat.png* then you will be able to add optional parameters to the URL.
If the parameters should not be optional you could also sign it like this: https://example.com/images/funny/cat.png?w=*&h=*. This way the values of the w and h params may be anything but both parameters must be set.
Related
When I create a presigned url - it generates a X-Amz-Signature.
Using this signature in the query param along with the Range query param causes the signatur to then be invalid. Is there anyway to get around this?
The request signature we calculated does not match the signature you provided. Check your key and signing method.
The signature will no longer be valid if you change the URL's query parameters or headers after creating it, which will lead to the error
The request signature we calculated does not match the signature you provided. Check your key and signing method.
Your question doesn't make clear which query parameter you are updating, however, it's possible that you are doing so after creating the pre-signed URL if you are changing the "Range" query parameter. If this is the case, the changed Range header would render the signature invalid because it is used in the calculation of the signature.
Solution : Create a fresh pre-signed URL with the amended Range header to get around this. If the Range header needs to be changed frequently, write a function that produces a fresh pre-signed URL each time the Range header is changed.
I have an API Gateway REST API with an integration to S3. We don't want to directly expose the buckets and object keys so there is an authorizer that is taking a path parameter and returning an object key. This object key may contain a number of path parameters, eg, path/to/object. I want to be able to rewrite the integration path to include this.
When I try using a path variable the value is replaced but it's URL encoded, ie, path%2Fto%2Fobject so of course the S3 request fails. I don't see any way of replacing the path variable with a value that contains multiple path parameters. I've also tried the same thing using #context.requestOverride.path.XXX in a mapping template and it also ends up URL encoded. I've tried using $utils.urlDecode() but again the result is encoded.
Is there any way to replace a path variable with multiple path segments? Otherwise, is there a way to completely replace the path in a mapping template (not just specific path variables)? Or is there another way to achieve this eg can the object key be passed in a header?
We are using an onPrem S3 compatible storage server in an intranet network and we want to expose this intranet url to internet so we used a ReverseProxy with a mapping to the intranet url. When we test the intranet url it works perfectly but when we test the internet url we get the 403 error:
The request signature we calculated does not match the signature you provided. Check your Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details. (Service: Amazon S3; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: 0a440c7f:15cc604b1e2:12d3af:24d; S3 Extended Request ID: null), S3 Extended Request ID: null
After debugging, we found that the proxy modifies the host header used to calculate the signature in order to redirect the request to the intranet url...
So my question is how to supress some headers from the V4 signature calculation using AWS SDK or Boto3 client. or is there a better architecture to expose an onPrem S3 service.
Thanks in advance.
Amir.
There are essentially two solutions to this.
The first one is easier: sign the request for the internal URL, then just use simple string prefix replacement to rewrite the host part of the signed URL to point it to the hostname of the external proxy. When the proxy rewrites the Host header, it will end up rewriting it back to exactly what you signed.
It is, I assume, common knowledge that signed URLs are immune to tampering, for all practical purposes: you can't change anything about a signed URL without invalidating it... but that's not what this is. The change is temporary, and the proxy's net effect is to undo the change.
The alternate solution requires the proxy or another service in the chain (before the storage service) to know the signing keys and secrets, so that it can first validate the incoming request, and if valid, modify the request and then generate a new signature that the service will accept. I once wrote a service to do this so that when a request was for HEAD, the proxy would use the same key and secret (which it knew) to generate a signature for the same request, but with GET. If it matched the signature in the incoming request, the proxy would replace the existing signature with a signature for a HEAD request -- thus allowing the client to use a URL originally signed for a GET request, to make either a GET or a HEAD request -- something S3 does not natively support, since a GET and a HEAD for the same object require two different signed URLs. The concept is the same, though -- generate a signature in the proxy for what the client is requesting, to validate the incoming signature, and then re-sign the request as needed. The solution I built used HAProxy's Lua integration to examine and modify the request in flight.
I have a static SPA page that is using S3 as it's origin with CloudFront. If I visit www.domain.com/page, I will get the CloudFront path prefixed bucket-directory/prod/page/ which is expected.
Is it possible to capture the path in AWS Lambda and append the trailing slash to a request, so it becomes, www.domain.com/page > [Lambda] > www.domain.com/page/
I've been looking and trying the following resources to little avail:
http://blog.rowanudell.com/redirects-in-serverless/
http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html
Getting a perfect setup for a SPA or static page on Cloudfront is not trivial. In short, you will need to use (at least) an origin request lambda function setting for your CF distro. You need to handle a few edge cases like:
hashes
redirects to urls with trailing slashes (that you mentioned)
forwarding of query parameters when redirecting
For a quick starting function you can check this article https://tinyendian.com/articles/better-origin-response-function-for-cloudfront-hosted-static-pages explaining the actual code that you can copy from here https://gist.github.com/karolmajta/6aad229b415be43f5e0ec519b144c26e
Of course it is likely that as your app changes, you will need to modify this snippet here and there to match your needs.
You can do that via two ways.
In the CloudFront pattern, you can check for bucket-directory/prod/page and do a redirect with a lambda to bucket-directory/prod/page/.
Also you need to make sure the pattern to be in the following order,
bucket-directory/prod/page <-- this will get to lambda to perform redirect
bucket-directory/prod/page/
page will a regex based on your naming convention.
Or You can write a Lambda Function that can take the url and modify the url to append a slash if it is not there and forward the request to the origin.
http://docs.aws.amazon.com/lambda/latest/dg/lambda-edge.html
Lambda Edge will make it much easier and avoids a redirect.
With Lambdda Edge you can change from bucket-directory/prod/page to bucket-directory/something/someotherpage as well.
The documentation link should help.
The title more or less says it all. Judging from the fact that "restrict viewer access" section of the Cloudfront management console (picture below) uses radio buttons, It appears that its an either/or situation.
QUESTION: Are there other ways to parse one's S3 bucket such that it can have both public (i.e., viewable by anyone) and private (i.e., signed urls) content in it?
I can answer my own question. Yes, you can have signed and unsigned urls at the same Cloudfront distribution. In the example below, my distribution is called blahblah.cloudfront.com. The key is to specify two "origin server"s. One for the signed urls and another for the unsigned urls.
->AWS Management Console->Cloudfront->Distribution settings of selected Distribution
->Origins: create both origins (e.g., pointing to 2 different S3 buckets)
->Behaviors: create "path patterns" for each origin such that Cloudfront can
distinguish which origin the Cloudfront url points to. E.g., in my S3 bucket named
"tim-UNsigned-bucket" I set the path pattern to `uploads/*`. For my signed url bucket,
I gave it the more general path pattern of `*` and prioritized it 2nd.
As you can see below, this means that my unsigned url includes the path, uploads/,
whereas my signed does not.
Resulting Cloudfront url structure:
Signed urls: https:// or http://blahblah.cloudfront.com/file1.jpg?Policy=asf...
UNsigned urls: https:// or http://blahblah.cloudfront.com/uploads/file2.jpg
Note: Making any of these changes can take Cloudfront several minutes (5-15 minutes) to update. Therefore, do your best to make all the changes in my directions with the fewest mistakes possible otherwise it will take you forever.