I have users across the world and they GET object from Amazon S3 buckets. So I'd like to set up 4 different buckets in different regions (3 of them are replications). Cloudfront is not my choice since the first-time request latency is still high. Is there any service that can receive the GET request from the user, automatically decide which bucket it should get files from with the fastest speed, GET files from that bucket and then send back the file to the user?
I have looked up Route 53, but it's all about hosting a website. My case is a mobile app. I just want to get the files from the server.
Thank you so much!
You can deploy your application in the specific regions you want and make sure in code that the files are come from the same region bucket.
Then you can use Route53 GeoLocation/Weighted routing policy and then based on your users location the request will go to that region and be processed accordingly.
Related
I'm trying to serve static web content (HTML, CSS, and JS files) from S3 buckets. I know I can go to the bucket's properties tab and choose the item Use this bucket to host a website from the Static website hosting box. And I'm sure this step will still be part of the solution I'm looking for but it won't be all.
Here's what I'm trying to accomplish:
Deploying the same content to multiple regions and based on availability and/or latency, provide the service to the client.
As for the API Gateway, I know how to do this. I should create the same API Gateway (alongside underlying lambda functions) and Custom Domain Names in all the regions. And then creating the same domain on Route 53 (of type CNAME) and choose Latency as Routing Policy. One can also set up a Health Check for the Record Set so availability of the API Gateway and lambda functions are checked periodically.
Now I want to do the same for the S3 bucket and my static content. i.e. I want to deploy the same content to different regions and somehow make Route 53 to route the request to the closest available bucket. Previously, I was using CloudFront but it seems to me in this setup, I can only introduce one bucket.
Does anyone know how can I serve my static content from multiple buckets? If you are going to suggest CouldFront, please tell me how you plan to use multiple buckets.
You can generate a certificate, setup a CloudFront distribution to grab the content from your bucket and then point your domain to your distribution using Route53. You get free https and you can also add several S3 buckets as origins for your distribution.
From AWS Docs:
After you configure CloudFront to deliver your content, here's what happens when users request your objects:
1. A user accesses your website or application and requests one or more objects, such as an image file and an HTML file.
2. DNS routes the request to the CloudFront edge location that can best serve the request—typically the nearest CloudFront edge location in terms of latency—and routes the request to that edge location.
3. In the edge location, CloudFront checks its cache for the requested files. If the files are in the cache, CloudFront returns them to the user. If the files are not in the cache, it does the following:
3a. CloudFront compares the request with the specifications in your distribution and forwards the request for the files to the applicable origin server for the corresponding file type—for example, to your Amazon S3 bucket for image files and to your HTTP server for the HTML files.
3b. The origin servers send the files back to the CloudFront edge location.
3c. As soon as the first byte arrives from the origin, CloudFront begins to forward the files to the user. CloudFront also adds the files to the cache in the edge location for the next time someone requests those files.
P.D. Keep in mind this is for static content only!
This is possible with CloudFront using Lambda#Edge to change origin based on answer from Route 53.
Please refer this blog for a sample Lambda#Edge code to do this -
https://aws.amazon.com/blogs/apn/using-amazon-cloudfront-with-multi-region-amazon-s3-origins/
I want to host my website in S3 (because it seems cheaper and i don't have server side script). I have a domain, and i want my domain link to my S3 website. So far, what i do is enabling Static website hosting in my S3 website bucket, and set Route53 record set's Alias Target to my S3 website. it's working. But it's not good enough, i want it to deal with multi regions.
I know that Transfer acceleration can auto sync files to other regions so it's faster for other regions. But i don't know how to make it work with Route53. I hear that some people uses CloudFront to do that but i don't quite understand how. And i don't want to manually create buckets in several regions and manually set up for each region
do you have any suggestion for me?
If your goal is to reduce latency for users worldwide, then Amazon CloudFront is definitely the way to go.
Amazon CloudFront has over 100 edge locations globally, so it has more coverage than merely using AWS regions.
You simply create a CloudFront distribution, point it to your S3 bucket and then point your domain name to CloudFront.
Whenever somebody accesses your content, CloudFront will retrieve it from S3 and cache it in the edge location closest to that user. Then, other users who want the data will receive it from the local cache. Thus, your website appears fast for many users.
See also: Amazon CloudFront pricing
I have a few EC2 instances (t2.micro) behind a load balancer on the us-east-1 region (N. Virginia) and my users are accessing the application from South America. This is my current setup mainly because costs are about 50% of what I would pay for the same services here in Brasil.
My uploads all go to S3 buckets, also in the us-east-1 region.
When a user requests a file from my app, I check for permission because the buckets are not public (hence why I need all data to go through EC2 instances) and I stream the file from S3 to the user. The download speeds for the users are fine and usually reach the maximum the user connection can handle, since I have transfer acceleration enabled for my buckets.
My issue is uploading files through the EC2 instances. The upload speeds suffer a lot and, in this case, having transfer acceleration enabled on S3 does not help in any way. It feels like I'm being throttled by AWS, because the maximum speed is capped around 1Mb/s.
I could maybe transfer files directly from the user to S3, then update my databases, but that would introduce a few issues to my main workflow.
So, I have two questions:
1) Is it normal for upload speeds to EC2 instances to suffer like that?
2) What options do I have, other than moving all services to South America, closer to my users?
Thanks in advance!
There is no need to 'stream' data from Amazon S3 via an Amazon EC2 instance. Nor is there any need to 'upload' via Amazon EC2.
Instead, you should be using Pre-signed URLs. These are URLs that grant time-limited access to upload to, or download from, Amazon S3.
The way it works is:
Your application verifies whether the user is permitted to upload/download a file
The application then generates a Pre-signed URL with an expiry time (eg 5 minutes)
The application supplied the URL to the client (eg a mobile app) or includes it in an HTML page (as a link for downloads or as a form for uploads)
The user then uploads/downloads the file directly to Amazon S3
The result is a highly scalable system because your EC2 system does not need to be involved in the actual data transfer.
See:
Share an Object with Others - Amazon Simple Storage Service
Uploading Objects Using Pre-Signed URLs - Amazon Simple Storage Service
We want to host images on our application as fast as possible. As we already have an AWS setup we prefer to host our images on S3 buckets (but are open for alternatives).
The challenge is routing the request to the closest S3 bucket.
Right now we use Amazon Route 53 with geolocation routing policy to the closes EC2 instance wich redirects to the respective bucket. We find this inefficent as the request goes:
origin->DNS->EC2->S3 and would prefer
origin->DNS->S3. Is it possible to bind two static website S3 buckets to the same domain where request are routed based on Geolocation?
Ps: We have looked into cloudfront, but since many of the images are dynamic and are only viewed once we would like the origin to be as close to the user as possible.
It's not possible to do this.
In order for an S3 bucket to serve files as a static website, the bucket name must match the domain that is being browsed. Due to this restriction, it's not possible to have more than one bucket serve files for the same domain because you cannot create more than one bucket with the same name, even in different regions.
CloudFront can be used to serve files from S3 buckets, and those S3 buckets don't need to have their names match the domain. So at first glance, this could be a workaround. However, CloudFront does not allow you to create more than one distribution for the same domain.
So unfortunately, as of this writing, geolocating is not possible from S3 buckets.
Edit for a deeper explanation:
Whether the DNS entry for your domain is a CNAME, an A record, or an ALIAS is irrelevant. The limitation is on the S3 side and has nothing to do with DNS.
A CNAME record will resolve example.com to s3.amazonaws.com to x.x.x.x and the connection will be made to S3. But your browser will still send example.com in the Host header.
When S3 serves files for webpages, it uses the Host header in the HTTP request to determine from which bucket the files should be served. This is because there is a single HTTP endpoint for S3. So, just like when your own web server is hosting multiple websites from the same server, it uses the Host header to determine which website you actually want.
Once S3 has the Host that you want, it compares it against the buckets available. It decided that the bucket name would be used to match against the Host header.
So after a lot of research we did not find an answer to the problem. We did however update our setup. The scenario is that a user clicks a button and will view some images in an IOS app. The request when the user pushes the button is geo rerouted to the nearest EC2 instance for faster performance. Instead of returning the same imagelinks in EU and US we updated it so when clicking in US you get links to an American S3 bucket and the same for Europe. We also put up two cloud front distributions, one in front of each S3 bucket, to increase speed.
I have an application which is a static website builder.Users can create their websites and publish them to their custom domains.I am using Amazon S3 to host these sites and a proxy server nginx to route the requests to the S3 bucket hosting sites.
I am facing a load time issue.As S3 specifically is not associated with any region and the content being entirely HTML there shouldn't ideally be any delay.I have a few css and js files which are not too heavy.
What can be the optimization techniques for better performance? eg: Will setting headers ? or Leverage caching help? I have added an image of pingdom analysis for reference.
Also i cannot use cloudfront as when the user updates an image the edge locations have a delay of few minutes before the new image is reflected.It is not instant update,hence restricting the use for me. Any suggestions on improving it?
S3 HTTPS access from a different region is extremely slow especially TLS handshake. To solve the problem we invented Nginx S3 proxy which can be find over the web. S3 is the best as origin source but not as a transport endpoint.
By the way try to avoid your "folder" as a subdomain but specify only S3 regional(!) endpoint URL instead with the long version of endpoint URL, never use https://s3.amazonaws.com
One the good example that reduces number of DNS calls is the following below:
https://s3-eu-west-1.amazonaws.com/folder/file.jpg
Your S3 buckets are associated with a specific region that you can choose when you create them. They are not geographically distributed. Please see AWS doc about S3 regions: https://aws.amazon.com/s3/faqs/
As we can see in your screenshot, it looks like your bucket is located in Singapore (ap-southeast-1).
Are your clients located in Asia? If they are not, you should try to create buckets nearer, in order to reduce data access latency.
About cloudfront, it should be possible to use it if you invalide your objects, or just use new filenames for each modification, as tedder42 suggested.