I am in the process of setting up a medium sized AWS infrastructure for a web project. I may be overthinking a couple things and therefore wanted to ask the community for opinions. Any input is appreciated.
Please see the graphic:
here
Explanation (from left to right):
My domain is hosted on GoDaddy and will simply route to Cloudfront in order to globally cache static content.
Cloudfront will point to Route53 which is responsible for routing the user to the closest region based on Geoprximity and/or Latency
Each region will have an availability load balancer pointing to an EC2 instance (different availability zones for disasters fallback)
From there, each EC2 instance writes to a single MySQL database. Static content is loaded from a S3 bucket.
This MySQL database replicates/synchronizes itself across availability zones and regions and creates read-replicas
If an EC2 instance has a read-request, it contacts another Route53 router that forwards the read-request to a load-balancer (in each region) based on where the request is coming from (geoproximity/latency). The only alternative I see here would be to directly point read-requests from a European EC2 instance to a European load-balancer. (vice versa for US)
The load-balancer in each region will then decide from which database to read based on health or amount of requests
Each EC2 instance can also trigger a LAMBDA function through an API Gateway.
What am I missing? Is this too much? What are the ups and downs of this construct?
Thank you all so much!
Things look reasonable up to step 6. There's no need to find the nearest MySQL database, because your instances already know where it is -- it's the one in the local region.
Step 7 is problematic because ELBs can't be used to balance RDS instances. However, with Aurora/MySQL, you get a single cluster "reader" endpoint hostname with a short TTL that load balances across your Aurora replicas. If a replica dies, it's removed from DNS automatically.
Step 8 it's not strictly necessary to use API Gateway -- instances can directly invoke Lambda functions through the Lambda API.
Additionally, there's Lambda#Edge that allows triggering Lambda functions directly from CloudFront -- although if the Lambda function you need is large in size (dependencies) or needs to run inside a VPC, you have to cascade two of them -- the edge function (not in VPC) invokes the regional function (large, or in a VPC) -- but this is still typically less expensive than API Gateway. Edge functions automatically replicate globally and run in the region closest to the CloudFront edge handling the individual request, and within any given function invocation this can be identified by inspecting process.env.AWS_REGION. Edge functions can also be used to change the origin serving the content -- so, e.g. if your function sees that it's been invoked in an EU region, it can rewrite the request so that CloudFront will send S3 requests to an EU bucket.
If your site is at the apex of a domain, e.g. example.com rather than, say, www.example.com, your domain will need to be hosted in Route 53, not Go Daddy, because constraints in the DNS standards do not allow the dynamic behavior required by CloudFront at the apex. You can still have your domain registered with them, but not hosted by them.
Related
I have a custom origin i.e. a web app on an EC2 instance. How do I decide whether I should go for:
a Cloudfront CDN
or,
deploy multiple instances in different regions and configure a Geolocation/proximity based routing policy
The confusion arises from the fact that both aim at routing the request to the nearest location (edge location in case of Cloudfront and region specific EC2 instance when it comes to multi-region deployments with Geolocation based policy with Route 53) based on where the request originates from.
There is no reason why you can't do both.
CloudFront automatically routes requests to an edge location nearest the viewer, and when a request can't be served from that location or the nearest regional cache, CloudFront does a DNS lookup for the origin domain name and fetches the content from the origin.
So far, I've only really stated the obvious. But up next is a subtle but important detail:
CloudFront does that origin server DNS lookup from a location that is near the viewer -- which means that if the origin domain name is a latency-based record set in Route 53, pointing to deployments in two or more EC2 regions, then the request CloudFront makes to "find" the origin will be routed to the origin deployment nearest the edge, which is also by definition going to be near to the viewer.
So a single, global CloudFront deployment can automatically and transparently select the best origin, using latency-based configuration for the backend's DNS configuration.
If the caching and transport optimizations provided by CloudFront do not give you the global performance you require, then you can deploy in multiple regions, behind CloudFront... being mindful, always, that a multi-region deployment is almost always a more complex environment, depending on the databases that are backing your application and how they are equipped to handle cross-region replication for reads and/or writes.
Including CloudFront as the front-end is also a better solution for fault tolerance among multiple regional deployments, because CloudFront correctly honors the DNS TTL on your origin server's DNS record, and if you have Route 53 health checks configured to take an unhealthy region out of the DNS response on the origin domain name, CloudFront will quickly stop sending further requests to it. Browsers are notoriously untrustworthy in this regard, sometimes caching a DNS answer until all tabs/windows are closed.
And if CloudFront is your front-end, you can offload portions of your logic to Lambda#Edge if desired.
You can use multi region for lot reasons mainly,
Proximity
Failover (incase if first region fails, requests can be sent to another region)
Multi region lambda deployment is clearly documented here. You can apply the same logic to all of the AWS Resources too. (DynamoDB, S3)
https://aws.amazon.com/blogs/compute/building-a-multi-region-serverless-application-with-amazon-api-gateway-and-aws-lambda/
You can also run Lambda#Edge to force all your requests / splits to one region on the edge.
Hope it helps.
I have attended an AWS training, and they explained to us that a good practice is to have cache all dynamic content via Cloudfront, setting TTL to 0, even if you have an LB in front on the Load Balancer. So it could be like:
Route 53 -> CloudFront -> Application LB
I can not see any advantage of this architecture, instead of having directly (only for dynamic content):
Route 53 -> Application LB
I do not see the point since Cloudfront will send all traffic always to the LB, so you will have:
Two HTTPS negotiation (client <-> Cloudfront, and Cloudfront <-> LB)
No caching at all (it is dynamic content, it should not be cached, since that is the meaning of "dynamic")
You will not have the client IP since your LB will see only the Cloudfront IP (I know this can be fixed, to have the client IP, but then you will have issues with the next bullet).
As an extra work, you need to be able to update your LB security groups often, to match the CloudFront IPs (for this region), as I guess you want to get traffic only from your Cloudfront, and not directly from the LB public endpoint.
So, probably, I am missing something important about this Route 53 -> CloudFront -> Application LB architecture.
Any ideas?
Thanks!
Here are some of the benefits of having cloudfront on top of your ALB
For a web application or other content that's served by an ALB in Elastic Load Balancing, CloudFront can cache objects and serve them
directly to users (viewers), reducing the load on your ALB.
CloudFront can also help to reduce latency and even absorb some distributed denial of service (DDoS) attacks. However, if users can
bypass CloudFront and access your ALB directly, you don't get these
benefits. But you can configure Amazon CloudFront and your Application
Load Balancer to prevent users from directly accessing the Application
Load Balancer (Doc).
Outbound data transfer charges from AWS services to CloudFront is $0/GB. The cost coming out of CloudFront is typically half a cent less
per GB than data transfer for the same tier and Region. What this
means is that you can take advantage of the additional performance and
security of CloudFront by putting it in front of your ALB, AWS Elastic
Beanstalk, S3, and other AWS resources delivering HTTP(S) objects for
next to no additional cost (Doc).
The CloudFront global network, which consists of over 100 points of presence (POP), reduces the time to establish viewer-facing
connections because the physical distance to the viewer is shortened.
This reduces overall latency for serving both static and dynamic
content (Doc).
CloudFront maintains a pool of persistent connections to the origin, thus reducing the overhead of repeatedly establishing new
connections to the origin. Over these connections, traffic between
CloudFront and AWS origins are routed over a private backbone network
for reliability and performance. This reduces overall latency for
serving both static and dynamic content (Doc).
You can use geo restriction, also known as geo blocking, to prevent users in specific geographic locations from accessing content that
you're distributing through a CloudFront distribution (Doc).
In other words you can use the benefits of ClodFront to add new features to your source (ALB, Elastic Beanstalk, S3, EC2) but if you don't need these features it is better not to do this configuration in your architecture.
Cloudfront enables you deliver content faster because Cloudfront Edge locations closer to the user requesting and are connected to the AWS Regions through the AWS network backbone.
You can terminate SSL at cloudfront and make the load balancer listen at port 80
Cloudfront allows to apply geo location restriction easily in 2 clicks.
I think another reason you may want to use CF in front of an ALB is that you could have a better experience with WAF (if you are already using (or planning to) WAF, of course).
Even though WAF is available for both ALB and CF, ALB and CF use different services for WAF. The reason is that Cloudfront is a global service and ALB is one per region.
That may bring more complex management and duplication of ACL (and probably more costs).
Cloudfront is really an amazing CDN content delivery network service like Akamai etc . Now if your web applications having lots of dynamic content like media files even you static code you can put them into a S3 bucket another object storage service by AWS .
Once you have your dynamic content to you S3 bucket you can create a Cloudfront distribution by considering that bucket as a origin this phenomena will cached your dynamic content across AWS multiple edge locations. And it will become fast to deliver on client side.
Now if we talk Load balancer . So it have it’s own purpose to serve image you are using a Application where you get an unpredictable traffic so here your Load balancer which we are considering an Application or classic Load balancer which is accepting request from Route 53 and passing it to your servers.
For high availability and scalability we consider such architecture of Application.
we create a autoscaling group of our ec2 instances and put them behind a load balancer and as per our scaling policy example: if my cpu or memory utilization goes more that 70% launch another instance or similar.
You can set a request policy as well on load balancer to serve traffic to your ec2 server maybe in round Robbin or on availability.
I just shared the best practices recommended of AWS fault tolerant and highly available architecture . I hope this may help you to get a better idea to decide now .
Please let me know if I can help you with more suggestions on it.
Thanks and Happy Leanings!!
If all my endpoints are AWS services like ELB or S3 "Evaluate Target Health" can be used instead of failover records correct? I can use multiple weighted, geo, or latency records and if I enabled "Evaluate Target Health" it also servers the purpose of failover if one of the resources a record is pointing to is not healthly route53 will not send traffic to it.
The only use I see for failover records with custom healthchecks is for non-aws resources OR if maybe you have a more complex decision you want DNS to make instead of just ELB/S3/etc service health.
EDIT: so it seems while I can get active-active with "Evaluate Target Health" (on alias endpoints) if I want active-passive I have to use a failover policy- is this correct?
Essentially, yes. Evaluating target health makes the records viable candidates for generating responses, only when healthy. Without a failover policy, they're all viable when they're all healthy.
If you do something like latency-based routing and you had two targets, let's say Ohio and London, then you'd essentially have a dual active/passive configuration with reversed roles -- Ohio active and London passive for viewers in North America, and the roles reversed for viewers in Europe. But if you want global active/passive, you'd need a a failover policy.
Note that if you are configuring any kind of high-availability design using Route 53 and target health, your best bet is to do all of this behind CloudFront -- where the viewer always connects to CloudFront and CloudFront does the DNS lookup against Route 53 to find the correct origin based on whatever rules you've created. The reason for this is that CloudFront always respects the DNS TTL values. Browsers, for performance reasons, do not. Your viewers can find themselves stuck with DNS records for a dead target because their browsers don't flush their cached DNS lookups until all tabs in all windows are closed. For users like me, that almost never happens.
This also works with latency-based routes in Route 53 behind CloudFront, because CloudFront has already routed the viewer to its optimal edge, and when that edge does a lookup on a latency-based route in Route 53, it receives the answer that has the lowest latency from the CloudFront edge that's handling the request... so both viewer to CloudFront and CloudFront to origin routes are thus optimal.
Note also that failover routing to S3 with only DNS is not possible, because S3 expects the hostname to match the bucket name, and bucket names are global. An S3 failure is a rare event, but it has happened at least once. When it happened, the impact was limited to a single region, as designed. For a site to survive an S3 regional failure requires additional heroics involving either CloudFront and Lambda#Edge triggers, or EC2-based proxies that can modify the request as needed and send it to the alternate bucket in an alternate region.
Latency-based routing to buckets with Route 53 is also not possible, for the same reason, but can be accomplished by Lambda#Edge origin request triggers. These triggers are aware of the AWS region where a given invocation is running, and thus can swap origin servers based on location.
Amazon released new feature - to support regional api endpoints
Does it mean I can deploy my same API code in two regions which sends request to Lambda micro-services? (It will be two different Https endpoints)
And have CloudFront distribute the traffic for me?
Any code snippets?
Does it mean I can deploy my same API code in two regions which sends request to Lambda micro-services? (It will be two different Https endpoints)
This was already possible. You can already deploy the same API code in multiple regions and create different HTTPS endpoints using API Gateway.
What you couldn't do, before, was configure API Gateway API endpoints in different regions to expect the same hostname -- and this is a critical capability that was previously unavailable, if you wanted to have a geo-routing or failover scenario using API Gateway.
With the previous setup -- which has now been renamed "Edge-Optimized Endpoints" -- every API Gateway API had a regional endpoint hostname but was automatically provisioned behind CloudFront. Accessing your API from anywhere meant you were accessing it through the CloudFront, which meant optimized connections and transport from the API client -- anywhere on the globe -- back to your API's home region via the AWS Edge Network, which is the network that powers CloudFront, Route 53, and S3 Transfer Acceleration.
Overall, this was good, but in some cases, it can be better.
The new configuration offering, called a Regional API Endpoint, does not use CloudFront or the Edge Network... but your API is still only in one region (but keep reading).
Regional API Endpoints are advantageous in cases like these:
If your traffic is from EC2 within the region, this avoids the necessity of jumping onto the Edge Network and back off again, which will optimize performance of API requests from inside the same EC2 region.
If you wanted to deploy an API Gateway endpoint behind a CloudFront distribution that you control (for example, to avoid cross-origin complications, or otherwise integrate API Gateway into a larger site), this previously required that you point your CloudFront distribution to the CloudFront distribution managed by API Gateway, thus looping through CloudFront twice, which meant transport latency and some loss of flexibility.
Creating a Regional API Endpoint allows you to then point your own CloudFront distribution directly at the API endpoint.
If you have a single API in a single region, and it's being accessed from points all over the globe, and you aren't using CloudFront yourself, the Edge-Optimized endpoint is still almost certainly the best way to go.
But Regional API Endpoints get interesting when it comes to custom domain names. Creating APIs with the same custom domain name (e.g. api.example.com) in multiple AWS regions was not previously possible, because of API Gateway's dependency on CloudFront. CloudFront is a global service, so the hostname namespace is also global -- only one CloudFront distribution, worldwide, can respond to a specific incoming request hostname. Since Regional API Endpoints don't depend on CloudFront, provisioning APIs with the same custom domain name in multiple AWS regions becomes possible.
So, assuming you wanted to serve api.example.com out of both us-east-2 and us-west-2, you'd deploy your individual APIs and then in each region, create a custom domain name configuration in each region for api.example.com with a Regional API Endpoint, selecting an ACM certificate for each deployment. (This requires ACM certs in the same region as the API, rather than always in us-east-1.)
This gives you two different hostnames, one in each region, that you use for your DNS routing. They look like this:
d-aaaaaaaaaa.execute-api.us-east-2.amazonaws.com
d-bbbbbbbbbb.execute-api.us-west-2.amazonaws.com
So, what next?
You use Route 53 Latency-Based routing to create a CNAME record for api.example.com with two targets -- one from us-east-2, one from us-west-2 -- pointing to the two respective names, along with health checks on the targets. Route 53 will automatically resolve DNS queries to whichever regional endpoint is closer to the requester. If, for example, you try to reach the API from us-east-1, your DNS query goes to Route 53 and there's no record there for us-east-1, so Route 53 determines that us-east-2 is the closer of the two regions, and -- assuming the us-east-2 endpoint has passed its healthcheck -- Route 53 returns the DNS record pointing to d-aaaaaaaaaa.execute-api.us-east-2.amazonaws.com.
So, this feature creates the ability to deploy an API in multiple AWS regions that will respond to the same hostname, which is not possible with Edge Optimized API Endpoints (as all endpoints were, prior to the announcement of this new feature).
And have CloudFront distribute the traffic for me?
Not exactly. Or, at least, not directly. CloudFront doesn't make origin determinations based on the requester's region, but Lambda#Edge dynamic origin selection could be used to modify the origin server based on the requester's general location (by evaluating which API region is nearest to the CloudFront edge that happens to be serving a specific request).
However, as you can see, above, Route 53 Latency-Based routing can do that for you. Yet, there's still a compelling reason to put this configuration behind a CloudFront distribution, anyway... two reasons, actually...
This is in essence a DNS failover configuration, and that is notoriously unreliable when the access is being made by a browser or by a Java programmer who hasn't heard that Java seems to cache DNS lookups indefinitely. Browsers are bad about that, too. With CloudFront in front of your DNS failover configuration, you don't have to worry about clients caching your DNS lookup, because CloudFront does it correctly. The TTL of your Route 53 records -- which used as an origin server behind CloudFront -- behaves as expected, so regional failover occurs correctly.
The second reason to place this configuration behind CloudFront would be if you want the traffic to be transported on the Edge Network. If the requests are only coming from the two AWS regions where the APIs are hosted, this might not be helpful, but otherwise it should improve responsiveness overall.
Note that geo-redundancy across regions is not something that can be done entirely transparently with API Gateway in every scenario -- it depends on how you are using it. One problematic case that comes to mind would involve a setup where you require IAM authentication against the incoming requests. The X-Amz-Credential includes the target region, and the signature of course would differ because the signing keys in Signature V4 are based on the secret/date/region/service/signing-key paradigm (which is a brilliant design, but I digress). This would complicate the setup since the caller would not know the target region. There may be other complications. Cognito may have similar complications. But for a straightforward API where the authentication is done by your own mechanism of application tokens, cookies, etc., this new capability is very much a big deal.
Somewhat amusingly, before this new capability was announced, I was actually working on the design of a managed service that would handle failover and geo-routing of requests to redundant deployments of API Gateway across regions, including a mechanism that had the capability to compensate for the differing region required in the signature. The future prospects of what I was working on are a bit less clear at the moment.
It means you can deploy your API based on the region which reduces latency.
One use case would be, Say you have a Lambda function which is invoking an API request. If both Lambda and API is in the same region then you can expect high performance
Please have a look on https://docs.aws.amazon.com/apigateway/latest/developerguide/create-regional-api.html
I have an ELB which balances some EC2 instances.
The ELB exposes the endpoints of the entire system.
Now I am creating a CloudFront distribution over this ELB.
Is there a way to allow users to connect ONLY using CloudFront endpoint and refuse direct connections to ELB?
Thanks
You would have to restrict the security group to the list of IP address ranges used by CloudFront. This is a subset of the list published here.
Unfortunately that list is subject to change, so you can't just set it once and forget it. Amazon has published a tutorial here that walks you through setting up a Lambda function that will automatically update your security group when Amazon publishes an updated IP list.
Unfortunately there is no straight forward way to do that right now.
ELB access can only be limited by IP ranges. You could try to limit the ELB to CloudFront's IP ranges, but this is rather brittle and changes frequently. If a new IP range is introduced, you may end up accidentally blocking CloudFront. I would say that this approach is not advisable, but I've seen it done when the requirement was mandatory. And it did break a few times.
You can set up a automated security group that only allows Cloudfront IP's and let a Lambda function to update it when Cloudfront IP ranges change. On my blog post, you can find a complete Cloudformation template that will set this up for you:
https://medium.com/cagataygurturk/restricting-elb-access-to-cloudfront-8b0990dea69f
If there is no record in R53 that uses your Load Balancer, and only cloudfront defines Alternate Domain Names (CNAMEs) used by your Load Balancer, then you can associate a WAF ACL with your Load Balancer that drops any request that does not match the Alternate Domain Names.
In that case, you force using the CloudFront Distribution for your Load Balancer.
AWS blogs have a solution for this scenario.
What it does is basically creating a lambda function that subscribes to a SNS topic which receives notifications for AWS IP address range changes (this topic is owned by AWS). This lambda then updates the ELB/ALB security group dynamically. Lambda code is available here.
Starting 2022 AWS finally provides a solution for this problem with managed prefix-lists.
You can create an inbound security rule and under source directly specify the prefix list, instead of manually providing IP-Addresses:
To make your server reachable only from Cloudfront Servers follow these steps:
Go to https://console.aws.amazon.com/vpc/home#ManagedPrefixLists
Choose your region (The region of your Load Balancer) and search for "com.amazonaws.global.cloudfront.origin-facing" and copy the id (e.g. "pl-a3a144ca" for europe-central-1)
Edit your security group for the Load-Balancer and add a new Entry with Type: HTTP and as source paste the prefix-list-id from step 2
Now your security group will automatically always use the current IP-Addresses from Cloudfront, now updating necessary. - A caveat: The prefix list counts as ~50 rules against the rules-limit for a security group. If you have a lot of other custom rules, you will likely have to create a second security group with the other rules if this one is full.
As of February 2022 there is a simpler solution. AWS now manages a prefix list for Cloudfront which auto updates.
For details: https://aws.amazon.com/about-aws/whats-new/2022/02/amazon-cloudfront-managed-prefix-list/