Is it possible to create a policy with multiple statements when using a CloudFront custom policy for signed cookies (not signed URLs)?
I have read the documentation, and although all the examples just have one statement, I cannot see an explicit rule regarding the number of statements allowed.
If it's not possible to have multiple policy statements, it will be difficult to give a particular user signed-cookie access to say, five random files using only the CloudFront security. Any tips on how to do that would be appreciated.
This question is cross-posted here: https://forums.aws.amazon.com/thread.jspa?threadID=223440&tstart=0
FYI
I have faced with the same problem, and contacted with the official AWS support team.
Hello, thanks for offering us a great service.
I am an software engineer from Japan.
Can we have multiple custom policies, like below syntax?
{
"Statement": [
{ ... },
{ ... },
{ ... },
]
}
I have searched on the web, and found ones who are trying to
do the same thing and forums/Q&A as well.
However we found no answer from AWS official support teams
nor documents saying about that.
JSON syntax is array, so it seems to work with
multiple statements but do not work.
So, if it does not work, would you add a sentence
about that on the official document?
And then, I got the answer yesterday:
I just heard back this morning.
You're correct, adding more than one statement
to a custom policy is not supported.
I'm updating the documentation now.
So, I think in few days the documentation will be updated that you can not set multiple policy statements for CF Custom Policy for Pre-Signed Cookies.
It's upsetting there is nothing in the docs that says you can only have one item in the Statement array, but that's AWS docs for ya!
Anyways, a way around this limitation, is to set multiple cookies at different path levels. You'll need to generate a signed cookie for each path you want and set each cookie in whatever app you are using. You can imagine an endpoint in your api that generates all of the necessary cookies, sets them all in the header, and your front end then sets all of those cookies.
More specifically you'll want to create one CloudFront-Key-Pair-Id cookie with your cloudfront access key id, and scope that cookie path to the highest level that you're policies will be set to.
Use the AWS CloudFront SDK to sign a cookie for each Resource. Create a pair of CloudFront-Policy and CloudFront-Signature cookie for each path that corresponds to the Resource path.
Say I have the following two Resources and want to give access to both of them:
https://cfsub.cloudfront.net/animals/dogs/* https://cfsub.cloudfront.net/animals/cats/*
I'd create:
1 CloudFront-Key-Pair-Id cookie with a path of /animals
1 CloudFront-Policy cookie with the base64 policy generated from running the dogs custom policy through the cloudfront signer. This cookie should have a path of /animals/dogs.
1 CloudFront-Policy same thing for cats
1 CloudFront-Signature cookie with the signature generated from running the dogs custom policy through the cloudfront signer. This cookie should have a path of /animals/cats
1 CloudFront-Signature same thing for cats
All of these cookies should have a domain set to your cloudfront domain cfsub.cloudfront.net
Send all those up to your web app or mobile app.
I can't give definite information on this subject, it is an explicit question which someone at Amazon can give relevant information on.
That said, I believe CloudFront policies may include multiple statements. Their schema is similar to IAM policies but I don't think it'll work exactly how you're expecting.
With IAM policies, you're able to attach multiple statements to one policy but they are OR'd across the statements:
Generally, each statement in a policy includes information about a single permission. If your policy includes multiple statements, a logical OR is applied across the statements at evaluation time. Similarly, if multiple policies are applicable to a request, a logical OR is applied across the policies at evaluation time... IAM Policy Documentation
In the documentation you linked to, the Statement key's value is an array which you can include multiple statements in but they'll be OR'd across them. There is further information on how the policies are evaluated which will help in limiting access to the files you're working on.
Giving access to five random files will be a challenge which I do not believe is accomplishable with CloudFront access policies alone. The conditions available aren't designed with this use case in mind.
As Rodrigo M pointed out, using the AWS API from a script can accomplish what you're attempting to do. Unfortunately that is the only route I can imagine which will accomplish what you're attempting.
If you find a way to accomplish this task using only CloudFront policies (without other AWS services), I'll be quite interested in the solution. It'd be a creative policy and quite useful.
I have similar requirement and tested AWS CloudFront with canned policy include multiple resources to restrict access to different urls.
The policy is a valid json object, it looks like below:
{
"Statement":[
{
"Resource":"https://qssnkgp0nnr9vo.cloudfront.net/foo/*",
"Condition":{
"DateLessThan":{
"AWS:EpochTime":1492666203
}
}
},
{
"Resource":"https://qssnkgp0nnr9vo.cloudfront.net/bar/*",
"Condition":{
"DateLessThan":{
"AWS:EpochTime":1492666203
}
}
}
]
}
After I signed policy and send request to CloudFront it turned out AWS CloudFront does not support it. It got a 403 response said it was a Malformed Policy.
HTTP/1.1 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?><Error><Code>MalformedPolicy</Code><Message>Malformed Policy</Message></Error>
AWS officially supports only one statement in one singed policy. However, There is a workaround if you need 4 or less statements. For each statement you can create a separate pair of a CloudFront-Policy cookie and a CloudFront-Signature with its own path. The size of this pair of cookies would be around 600-900 bytes. Since the Cookie header has a limit of around 4Kb, you definitely can't use more than 5 pairs. Using 5 pairs has a high change to reach the header limit.
Related
So I got access to SimilarWeb ranking API from AWS(https://aws.amazon.com/marketplace/pp/prodview-clsj5k4afj4ma?sr=0-1&ref_=beagle&applicationId=AWSMPContessa).
I'm not able to figure out how to pass the authentication or how to give a request to retrieve the ranks for domains.
For ex. how will you pass the request for this URL in python?
URL: https://api-fulfill.dataexchange.us-east-1.amazonaws.com/v1/v1/similar-rank/amazon.com/rank
This particular product does not seem to be available any longer. Generally speaking, an AWS IAM principal with correct IAM permissions, can make API calls against AWS Data Exchange for APIs endpoints. The payload of the API call needs to adhere to the OpenAPI spec defined within the DataSet of the product used. The specific API call is 'SendApiAsset'. The easiest way to think about is to read the boto3 documentation for it, here: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dataexchange.html#DataExchange.Client.send_api_asset
Other AWS SDKs have the same call, idiomatic to the specific language.
The managed policy that describes the IAM permissions needed is named AWSDataExchangeSubscriberFullAccess, the dataexchange specific permission needed is 'dataexchange:SendApiAsset'.
The awscli way of making the call is described here: https://docs.aws.amazon.com/cli/latest/reference/dataexchange/send-api-asset.html
The required parameters are: asset-id, data-set-id, revision-id. You will likely also need to provide values for: method and body (and perhaps others also depending on the specific API you are calling.
The content of the 'body' parameter needs to adhere to the OpenAPI spec of the actual dataset provided as part of the product.
You can obtain the values for asset-id, data-set-id and revision-id from the AWS Data Exchange service web console describing the product/dataset.
I am using the Serverless framework to do AWS development.
I have an S3 bucket with different assets inside (videos, images, etc.)
I am serving the contents of that bucket through Cloudfront.
My goal is to let some files free in the internet (images) and protect others (videos) through signed URLs without having two buckets (one for private assets and one for public ones).
At the Cloudfront level I have set TrustedSigners to self:
TrustedSigners:
- self
This is how I am thinking of achieving this goal:
Use custom policies like:
{
"Statement":[
{
"Resource":"base URL or stream name",
"Condition":{
"DateLessThan":{
"AWS:EpochTime":ending date and time in Unix time format and UTC
}
}
}
]
}
I could use wildcards maybe for the image resources. The problem is I am not sure this is possible and I have no idea where to put this policy in the serverless.yml file.
Is this policy set at the sdk level?
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudfront-distribution.html
I do not see anywhere where I can declare the custom policy.
Have two Cloudfront distributions and somehow set filter the content of the files they are serving. One cloudfront would serve the images from the S3 bucket while the other the videos. I'm not sure this is possible also.
How would you guys do it?
Is there a chance?
Thank you!
You can work this out with Cache behaviours, Trusted signers are used in Cache behaviours, you can have different cache behaviours based on path such as /images , /video or even *.jpg etc
Path pattern doesn't support regex yet, it supports wildcards currently:
* matches 0 or more characters.
? matches exactly 1 character.
CloudFront cache behaviour
In AWS I can find under:
Cloudfront >> Reports & Analytics >> Top Referrers (CloudFront Top Referrers Report)
There I get the top25 items. How can I get ALL of them?
I have turned on logging in my bucket, but it seems that the referrer is not part of the log-file. Any idea how amazon collects its top25 and how I can according to that get the whole list?
Thanks for your help, in advance.
Amazon's built in analytics are, as you've noticed, rather basic. The data you're looking for all lives in the logfiles that you can set cloudfront up to export (in the cs(Referer) field). If you know what you're looking for, you can set up a little pipeline to download logs, pull out the numbers you care about and generate reports.
Amazon also makes it easy[1] to set up Athena or Redshift to look directly at Cloudfront or S3 logfiles in their target bucket. After a one-time setup, you could query them directly for the numbers you need.
There are also paid services built to fill in the holes in Amazon's default reports. S3stat (https://www.s3stat.com/), for example, will give you a Top 200 Referrer list in its reports, with the ability to export complete lists.
[1] "easy", using Amazon's definition of the word, meaning really really hard.
TLDR: We have to trick CloudFront 307 redirect caching by creating new cache behavior for responses coming from our Lambda function.
You will not believe how close we are to achieve this. We have stucked so badly in the last step.
Business case:
Our application stores images in S3 and serves them with CloudFront in order to avoid any geographic slow downs around the globe.
Now, we want to be really flexible with the design and to be able to request new image dimentions directly in the CouldFront URL!
Each new image size will be created on demand and then stored in S3, so the second time it is requested it will be
served really quickly as it will exist in S3 and also will be cached in CloudFront.
Lets say the user had uploaded the image chucknorris.jpg.
Only the original image will be stored in S3 and wil be served on our page like this:
//xxxxx.cloudfront.net/chucknorris.jpg
We have calculated that we now need to display a thumbnail of 200x200 pixels.
Therefore we put the image src to be in our template:
//xxxxx.cloudfront.net/chucknorris-200x200.jpg
When this new size is requested, the amazon web services have to provide it on the fly in the same bucket and with the requested key.
This way the image will be directly loaded in the same URL of CloudFront.
I made an ugly drawing with the architecture overview and the workflow on how we are doing this in AWS:
Here is how Python Lambda ends:
return {
'statusCode': '301',
'headers': {'location': redirect_url},
'body': ''
}
The problem:
If we make the Lambda function redirect to S3, it works like a charm.
If we redirect to CloudFront, it goes into redirect loop because CloudFront caches 307 (as well as 301, 302 and 303).
As soon as our Lambda function redirects to CloudFront, CloudFront calls the API Getaway URL instead of fetching the image from S3:
I would like to create new cache behavior in CloudFront's Behaviors settings tab.
This behavior should not cache responses from Lambda or S3 (don't know what exactly is happening internally there), but should still cache any followed requests to this very same resized image.
I am trying to set path pattern -\d+x\d+\..+$, add the ARN of the Lambda function in add "Lambda Function Association"
and set Event Type Origin Response.
Next to that, I am setting the "Default TTL" to 0.
But I cannot save the behavior due to some error:
Are we on the right way, or is the idea of this "Lambda Function Association" totally different?
Finally I was able to solve it. Although this is not really a structural solution, it does what we need.
First, thanks to the answer of Michael, I have used path patterns to match all media types. Second, the Cache Behavior page was a bit misleading to me: indeed the Lambda association is for Lambda#Edge, although I did not see this anywhere in all the tooltips of the cache behavior: all you see is just Lambda. This feature cannot help us as we do not want to extend our AWS service scope with Lambda#Edge just because of that particular problem.
Here is the solution approach:
I have defined multiple cache behaviors, one per media type that we support:
For each cache behavior I set the Default TTL to be 0.
And the most important part: In the Lambda function, I have added a Cache-Control header to the resized images when putting them in S3:
s3_resource.Bucket(BUCKET).put_object(Key=new_key,
Body=edited_image_obj,
CacheControl='max-age=12312312',
ContentType=content_type)
To validate that everything works, I see now that the new image dimention is served with the cache header in CloudFront:
You're on the right track... maybe... but there are at least two problems.
The "Lambda Function Association" that you're configuring here is called Lambda#Edge, and it's not yet available. The only users who can access it is users who have applied to be included in the limited preview. The "maximum allowed is 0" error means you are not a preview participant. I have not seen any announcements related to when this will be live for all accounts.
But even once it is available, it's not going to help you, here, in the way you seem to expect, because I don't believe an Origin Response trigger allows you to do anything to trigger CloudFront to try a different destination and follow the redirect. If you see documentation that contradicts this assertion, please bring it to my attention.
However... Lambda#Edge will be useful for setting Cache-Control: no-cache on the 307 so CloudFront won't cache it, but the redirect itself will still need to go all the way back to the browser.
Note also, Lambda#Edge only supports Node, not Python... so maybe this isn't even part of your plan, yet. I can't really tell, from the question.
Read about the Lambda#Edge limited preview.
The second problem:
I am trying to set path pattern -\d+x\d+\..+$
You can't do that. Path patterns are string matches supporting * wildcards. They are not regular expressions. You might get away with /*-*x*.jpg, though, since multiple wildcards appear to be supported.
I am working with a team that is using S3 to host content and they moved from a single bucket for all brands to one bucket for each brand and now we are having trouble when linking to the content from within salesforce site.com page. When I copy the link from S3 as HTTPS, I get a >"Your connection is >not private, Attackers might be trying to steal your information from >spiritxpress.s3.varsity.s3.amazonaws.com (for example, passwords, messages, or credit cards)."
I have asked them to compare the settings from the one that is working, and I don't have access to dig into it myself, and we are pretty new to this as well so thought I would see if there were any known paths to walk down. The ID and Key have not changed and I can access the content via CyberDuck, it just is not loading when reached via a link.
Let me know if additional information is needed and I will provide as quickly as I can.
[EDIT] the bucket naming convention they are using is all lowercase and meets convention guidelines as well, but it seems strange to me they way it is structured as they have named the bucket "brandname.s3.companyname" and when copying the link it comes across as "https://brandname.s3.company.s3.amazonaws.com/directory/filename" where the other bucket was being rendered as "https://s3.amazonaws.com/bucketname/......
Whoever made this change has failed to account for the way wildcard certificates work in HTTPS.
Requests to S3 using HTTPS are greeted with a certificate identifying itself as "*.s3[-region].amazonaws.com" and in order for the browser to consider this to be valid when compared to the link you're hitting, there cannot be any dots in the part of the hostname that matches the * offered by the cert. Bucket names with dots are valid, but they cannot be used on the left side of "s3[-region].amazonaws.com" in the hostname unless you are willing and able to accept a certificate that is deemed invalid... they can only be used as the first element of the path.
The only way to make dotted bucket names and S3 native wildcard SSL to work together is the other format: https://s3[-region].amazonaws.com/example.dotted.bucket.name/....
If your bucket isn't in us-standard, you likely need to use the region in the hostname, so that the request goes to the correct endpoint, e.g. https://s3-us-west-2.amazonaws.com/example.dotted.bucket.name/path... for a bucket in us-west-2 (Oregon). Otherwise S3 may return an error telling you that you need to use a different endpoint (and the endpoint they provide in the error message will be valid, but probably not the one you're wanting for SSL).
This is a limitation on how SSL certificates work, not a limitation in S3.
Okay, it appears it did boil down to some permissions that were missed and we were able to get the file to display as expected. Other issues are present, but the present one is resolved so marking as answered.