This has also been posted in the AWS forum. But it's languishing a bit (and I'm in a hurry to solve this problem).
This question is with regard to a 'download' distribution, not 'streaming'.
I've recently signed up and created an Amazon S3 bucket and then created an Amazon CloudFront (CF) distribution out of that bucket. Here's the relevant info:
Bucket Name: stella_media
Folder In Bucket: visia
Which results in a working URL (with public-read access) like this: http://s3.amazonaws.com/stella_media/visia/720_125M_Zero_Dark_Thirty.mp4
So you'll see that if you use the above URL (and you're not using Firefox) it loads the MP4 video. All the media in my bucket is set to public-read.
My Problem Is With Access To The Same Content Via Cloudfront:
And my CF distrubution has the following properties:
Delivery Method: download
Distribution Status: deployed
Price Class: US & Europe
State: Enabled
Domain Name: d2322fq9z81lph.cloudfront.net
However, when I use the URL provided to me when I setup my CF distribution on that bucket I get a "noSuchKey" error: http://d2322fq9z81lph.cloudfront.net/stella_media/visia/720_125M_Zero_Dark_Thirty.mp4
<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>stella_media/visia/720_125M_Zero_Dark_Thirty.mp4</Key>
<RequestId>5E2FA36884444757</RequestId>
<HostId>
HiphTLuv4P2tiJemBRNvIGsq8DRbSCMocdAvm4oto9NVnnKNHuDZWuFHz+xyCt6B
</HostId>
</Error>
So ... exactly what URL am I supposed to be using to point to my video OR is there some sort of permissions setting that I've overlooked to make the content in my CF distribution public?
Thanks for any help.
For the benefit of any others that may come across this, I've figured this out.
Apparently CloudFront URL's DO NOT INCLUDE THE BUCKET NAME. So it would play out like so:
S3 URL
http://s3.amazonaws.com/stella_media/visia/720_125M_Zero_Dark_Thirty.mp4
CloudFront URL
http://d2322fq9z81lph.cloudfront.net/visia/720_125M_Zero_Dark_Thirty.mp4
Hopefully that helps anyone else from going out of their mind trying to figure out what's wrong.
Related
I am using Ansible Playbook to push to an S3 bucket, and then use Cloudfront distribution to serve the site over https.
I would like to implement this using Ansible Playbook, however I haven't been able to configure the task correctly. I'm not sure where to specify the S3 Origin of the bucket?
- cloudfront_distribution:
comment: 'description'
enabled: yes
state: present
default_root_object: index.html
validate_certs: yes
viewer_certificate:
acm_certificate_arn: {arn of certificate}
alias: {www.mydomain.com}
origins:
- id: 'id'
s3_origin_access_identity_enabled: yes
Where can I specific the S3 bucket that I would like to serve through Cloudfront?
I've tried using the origins to specify the URL of the S3 bucket. But so far I have only been getting errors.
Would anyone be able to help me to specifiy the S3 bucket location so that I can serve the site through S3?
It appears that the module is expecting domain_name: to be of the form my-awesome-bucket.s3.amazonaws.com which triggers the magic s3 sauce, and that should cause your s3_origin_access_identity_enabled: yes to take effect.
If you find that to be the case, it might be worth filing an issue to have them update the documentation to be a little clearer.
Following is what I'm doing. I'm generating a pre-signed URL using a custom domain for my s3 bucket resources which are not public.
https://files.customdomain.com/file123?AWSAccessKeyId=XXX&Expires=1541220685&Signature=XXXX
Also to add the certificate I've created a cloudfront distribution for the bucket having following origin settings
Origin Domain Name: bucket-name.s3.amazonaws.com
Origin Id : s3.bucket-name
Restrict Bucket Access: No
Yet I'm unable to access my resources. Throws access denied error. Any help would be appreciated.
There are two cases:
If your bucket has regular name.
In this case you should use CloudFront to access your bucket.
And like mentioned above URL looks like in this answer:
https://cloudfront-url/file123?AWSAccessKeyId=XXX&Expires=1541220685&Signature=XXXX
If your bucket has s3 static website name.
In this case your bucket name looks like files.customdomain.com and you can generate pre-signed url for this bucket:
https://files.customdomain.com/file123?AWSAccessKeyId=XXX&Expires=1541220685&Signature=XXXX
In your DNS you will have CNAME files.customdomain.com points to files.customdomain.com.s3.[bucket-region].amazonaws.com.
NOTICE
When I generate pre-signed URL via aws-cli:
aws s3 presign s3://files.customdomain.com/file123 --endpoint-url https://files.customdomain.com
I get URL with duplicate bucket name in the path:
https://files.customdomain.com/files.customdomain.com/file123?AWSAccessKeyId=XXX&Expires=1541220685&Signature=XXXX
instead of:
https://files.customdomain.com/file123?AWSAccessKeyId=XXX&Expires=1541220685&Signature=XXXX
I don't know if it has the same behavior via SDK.
Have you tried initializing S3 with the custom url var S3 = new AWS.S3({endpoint: 'media.domain.com', s3BucketEndpoint: true});
More info https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html
Also, make sure signature is correct as well https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingAWSSDK.html#specify-signature-version
Ref : https://github.com/aws/aws-sdk-js/issues/891
When using S3 with CloudFront, you don't want an S3 signed URL... you want a CloudFront signed URL.
Read Configuring Security and Limiting Access to Content in the CloudFront developer guide.
I found a solution for this question. The signed url needs to be generated for cloudfront url endpoint from s3 bucket. Therefore instead of
https://files.customdomain.com/file123?AWSAccessKeyId=XXX&Expires=1541220685&Signature=XXXX
it needs to be
https://cloudfront-url/file123?AWSAccessKeyId=XXX&Expires=1541220685&Signature=XXXX
and DNS records had to resolve custom domain to cloudfront url.
I have two S3 buckets under two different accounts. Permissions and CORS setting of both buckets are same.
Regions of two buckets are as following (First one working)
Region: Asia Pacific (Singapore) (ap-southeast-1) works
Region: US East (Ohio) (us-east-2) does not work
I created Upload script with Node.js and supplied region plus following
Key : __XXXX__
secret: __XXXXX____,
bucket: _____XXXX__
'x-amz-acl': 'public-read',
ACL: 'public-read'
Code works fine with first, uploaded files is also accessible publicly. But with 2nd account(Region: us-east-2), script runs successfully and return URL also, but when I look in bucket there is no upload and url is saying permission denied which means resource is not available. Strange things are
Why URL is returned if file is not uploaded in bucket?
Why same code does not working for other account,
I tried AWS documentation also but that seems like it's not written for human like me. Help will be highly appreciated.
script runs successfully and return URL also, but when I look in bucket there is no upload
If you can really see no resources in the bucket, then the upload really failed (I've seen too many scripts just ignoring any error response) or the upload is executed to different place than you expect. Care to share the script?
and url is saying permission denied which means resource is not available
Unfortunatelly that's something you have find out yourself. If the object is in the bucket, it has public access, correct cors settings, then maybe the url is not correct.
I'm configuring an environment with a Amazon S3 Bucket for storage of media files and Amazon CloudFront for restricted distribution purposes.
The access to those media files needs to be private and should be done via a signed URL. So I created the S3 Bucket on South America (São Paulo) region and uploaded some test files. Then I created a CloudFront Distribution with that previous bucket as Origin and it's Bucket Access is restricted. I created a new OAI (Origin Access Identity) and also selected the option Yes, Update Bucket Policy so that it auto-configures the S3 Bucket Policies.
I'm only using the default Behavior and it's configured with HTTP and HTTPS viewer protocol policy and GET, HEAD allowed methods. Restrict Viewer Access (Use Signed URLs or Signed Cookies) is set and the Trusted Signer is set to Self.
Here's some images to clarify the setup:
S3 Bucket Policy
Distribution's Origin
Distribution's Behavior
I'm getting a HTTP 403 while trying to access the signed URL generated with either awscli or cfsign.pl
<Error>
<Code>AccessDenied</Code>
<Message>Access denied</Message>
</Error>
Is there something missing that I don't know? It looks like I made everything the docs said to do.
I received the same Access Denied error and spent the last couple hours trying to figure out what was going on. I finally realized that the Expires parameter was set in the past since I was using my local time instead of UTC. Make sure to set the Expires in the future according to UTC.
In my case the problem was with URL I was passing to URL signing code (I was using AWS SDK for Node.js).
cloudFront.getSignedUrl({
url: `${distributionUrl}/${encodeURI(key)}`,
expires: Math.floor(new Date().getTime() / 1000) + 60 * 60
})
Note encodeURI. I was not doing that. The resulting signed URL would still have URI components encoded, BUT would have invalid signature, thus causing 403 error.
EDIT: ...And you have to wrap it into url.format() like this:
cloudFront.getSignedUrl({
url: url.format(`${distributionUrl}/${encodeURI(key)}`),
expires: Math.floor(new Date().getTime() / 1000) + 60 * 60
})
I guess they should be doing that in SDK.
After recreating both the Amazon S3 Bucket and Amazon CloudFront Distribution I was still experiencing the issue. After a session with my rubber duck I found out that the Private Key file that I was using belongs to a deleted CloudFront Key-pair.
Now that I'm using the correct key to encrypt things everything is working fine. That doesn't explain why the first bucket and distribution weren't working because in that specific case I was using the same set of configurations and the right Private Key file.
I also encountered the same issue. Probably, we have to re-generate Clouf Front key-pair.
I have created a CloudFront distribution to serve the static website. S3 is origin server.
Now if we access CloudFront URL, it redirects to S3 location.
d2s18t7gwlicql.cloudfront.net
or
test.telekha.in
In the browser it is showing
https://telekha-test-www.s3.ap-south-1.amazonaws.com/index.html#/dashboard
I am expecting https://test.telekha.in/#/dashboard
If I access https://test.telekha.in through curl it returns my index.html document
If I access http://test.telekha.in through curl it returns
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>CloudFront</center>
</body>
</html>
But in browser both HTTP and HTTPS are redirecting to https://telekha-test-www.s3.ap-south-1.amazonaws.com/index.html#/
Please let me know how to resolve this issue.
Quick Solution
Use the regional domain name of your S3 bucket to configure the CloudFront distribution's origin, e.g.: {bucket-name}.s3.{region}.amazonaws.com.
Explanation
According to the discussion on AWS Developer Forums: Cloudfront domain redirects to S3 Origin URL (via archive.org), it takes time for DNS records to be created and propagated for newly created S3 buckets. The issue is not visible for buckets created in US East (N. Virginia) region, because this region is the default one (fallback).
Each S3 bucket has two domain names, one global and one regional, i.e:
global — {bucket-name}.s3.amazonaws.com
regional — {bucket-name}.s3.{region}.amazonaws.com
If you configure your CloudFront distribution to use the global domain name, you will probably encounter this issue, due to the fact that DNS configuration takes time.
However, you could use the regional domain name in your origin configuration to escape this DNS issue in the first place.
CloudFormation Template
If you are using CloudFormation, you can use the RegionalDomainName output attribute of the AWS::S3::Bucket resource:
S3Bucket:
Type: AWS::S3::Bucket
CloudFrontDistribution:
Type: AWS::CloudFront::Distribution
Properties:
DistributionConfig:
Origins:
- DomainName: !GetAtt S3Bucket.RegionalDomainName
More information
As well, I would highly recommend to read this blog post on the future of S3 different path formats:
Amazon S3 Path Deprecation Plan – The Rest of the Story
I found the issue. It is with cloudfront configuration.
This blog helped me.
While defining the origin I have directly selected S3 bucket. We should enter the domain of the S3 bucket like telekha-test-www.s3-website.ap-south-1.amazonaws.com
The first thing to check if you think you are seeing this is to run the curl command below. If it returns HTTP/1.1 307 Temporary Redirect, then you are seeing this issue.
$ curl -I https://YOUR_CF_DOMAINNAME.cloudfront.net/
HTTP/1.1 307 Temporary Redirect
Content-Type: application/xml
Content-Length: 0
Connection: keep-alive
x-amz-bucket-region: ap-southeast-2
Location: http://yourS3bucketname.s3-ap-southeast-2.amazonaws.com/
Date: Wed, 12 Jul 2017 00:20:27 GMT
Server: AmazonS3
Age: 1775
X-Cache: Hit from cloudfront
Via: 1.1 someid.cloudfront.net (CloudFront)
X-Amz-Cf-Id: someguid==
The best description I found of this issue is:
S3 updates the DNS for the global REST endpoint hierarchy *.s3.amazonaws.com with a record sending requests to the right region for the bucket within a short time after bucket creation, and CloudFront appears rely on this for sending the requests to the right place. Before that initial update is complete, S3 will return a redirect and CloudFront returns that redirect to the browser. ~ michael-sqlbot
Given this issue is actually due to the internal DNS propagation of the S3 bucket name (which is not 100% clear, but seems highly likely) that occurs when you configure the bucket in S3, then it should be possible to avoid this issue by configuring a public web site in S3 prior to configuring the Cloudfront distro, and per the doco, configure the S3 public web name as the cloudfront origin rather than the s3 bucketname.
For reference, I have both S3 bucket names and S3 website names configured as Cloudfront origins and I can say that they do both work! (eventually?)
References:
AWS official setup guide
AWS Forum topic
Turns out this is just a timing issue which fixes itself after a while if everything is configured correctly. More information can be found in this AWS forum thread.
Current accepted answer here and linked blog article suggest enabling static website for your S3 bucket and then changing CF origin to point to that static website. This solution does solve the redirect problem but with the side effect that you website is now available using both CF URL or your custom CNAME as well as using S3 URL.
To expand on the accepted answer, this part at the end of the referenced blog post in particular is helpful:
I found a subtle “bug” some days ago: when using URLs like
www.example.com/about/, Amazon S3 will in fact return the “index.html”
file inside the folder (because it is configured as a static website
bucket).
The funny thing is that if you omit the trailing slash
(www.example.com/about), S3 will first check if an object called
“about” exists. If it does not, it will consider that about is a
folder, and will issue a 301 redirection to about/. When using
CloudFront, this means that CloudFront will in fact cache… the
redirection instead of the file itself! Therefore, you must make sure
that all your URLs end by a trailing slash to avoid a useless
redirect.