I have an application that needs to handle huge traffic. The previous version of the application hits nearly 2,000,000 requests in 15 mins. That version does not have a CDN so that I need to deploy nearly 50 containers each for frontend and backend. So now I have added a CDN in front of my application. I have chosen AWS Cloudfront as CDN because the application is hosted on AWS.
Right now, I need to do the load test for this new application. If I do the load test with the Cloudfront URL, will it show the exact result as it will be served by Cloudfront?
If I load test with the Load Balancer URL and find out the required number of servers for handling the required load, will that be an over provision? As the Cloudfront will serve my application from nearly 189 edge locations (from AWS docs), is that much servers are required?
How can I find a relation between the amount of traffic that can be handled with and without Cloudfront?
Load testing Cloudfront itself is not the best idea, according to Cloudfront main page
The Amazon CloudFront content delivery network (CDN) is massively scaled and globally distributed.
However you could test the performance of your website with and without the CDN to see if there is a benefit/ROI of using Cloudfront as it doesn't come for free and you need to ensure that it makes sense to use it as it might be the case your application performance will be sufficient without the CDN integration.
Check out 6 CDN Load Testing Best Practices for more details.
Also make sure to add DNS Cache Manager to your test plan to ensure that each JMeter thread (virtual user) independently resolves the underlying server address for the ELB as it might be the case all the threads will be hitting the same IP address.
You can conduct the load test with cloudfront url as this is the real user scenario.
Please check the auto-scaling is enabled for the server. Also you need to monitor the load balancer while test execution to validate the traffic.
Also you need to check the security software/filters setting for compression and caching headers for the requests. Sometime this security patches/Filer ignore the header and it will impact on application performance in AWS cloud.
Use AWS cloud watch to monitor the servers.
Related
Our current setup: corporate network is connected via VPN with AWS, Route53 entry is pointing to ELB which points to ECS service (both inside a private VPC subnet).
=> When you request the URL (from inside the corporate network) you see the web application. ✅
Now, what we want is, that when the ECS service is not running (maintenance, error, ...), we want to directly provide the users a maintenance page.
At the moment you will see the default AWS 503 error page. We want to provide a simple static HTML page with some maintenance information.
What we tried so far:
Using Route53 with Failover to CloudFront distributing an S3 bucket with the HTML
This does work, but:
the Route53 will not failover very fast => Until it switches to CloudFront, the users will still see the default AWS 503 page.
as this is a DNS failover and browsers (and proxies, local dns caches, ...) are caching once resolved entries, the users will still see the default AWS 503 page after Route53 switched, because of the caching. Only after the new IP address is resolved (may take some minutes or up until browser or os restart) will the user see the maintenance page.
as the two before, but the other way around: when the service is back running, the users will see the maintenance page way longer, than they should.
As this is not what we were looking for, we next tried:
Using CloudFront with two origins (our ELB and the failover S3 bucket) with a custom error page for 503.
This is not working, as CloudFront needs the origins to be publicly available and our ELB is in a private VPC subnet ❌
We could reconfigure our complete network environment to make it public and restrict the access to CloudFront IPs. While this will probably work, we see the following drawbacks:
The security is decreased: Someone else could setup a CloudFront distribution with our web application as the target and will have full access to it outside of our corporate network.
To overcome this security issue, we would have to implement a secure header (which will be sent from CloudFront to the application), which results in having security code inside our application => Why should our application handle that security? What if the code has a bug or anything?
Our current environment is already up and running. We would have to change a lot for just an error page which comes with reduced security overall!
Use a second ECS service (e.g. HAProxy, nginx, apache, ...) with our application as target and an errorfile for our maintenance page.
While this will work like expected, it also comes with some drawbacks:
The service is a single point of failure: When it is down, you can not access the web application. To overcome this, you have to put it behind an ELB, put it in at least two AZs and (optional) make it horizontally scalable to handle bigger request amounts.
The service will cost money! Maybe you only need one small instance with little memory and CPU, but it (probably) has to scale together with your web application when you have a lot of requests!
It feels like we are back in 2000s and not in a cloud environment.
So, long story short: Are there any other ways to implement a f*****g simple maintenance page while keeping our web application private and secure?
I want to use a GCP load balancer to terminate HTTPS and auto manage HTTPS cert renewal with Lets Encrypt.
The pricing calculator gives me $21.90/month for a single rule. Is this how much it would cost to do HTTPS termination for a single domain? Are there cheaper managed options on GCP?
Before looking at the price, and to another solution, look at what you need. Are you aware of Global Load balancer capabilities?
It offers you a unique IP reachable all over the globe and route the request to the region the closest to your user for reducing the latency. If the region is off, or the capacity of your backend full (health check KO), the request is routed to the next closest region.
It allows you to rewrite your URL, to manage SSL certificates, to cache your file into CDN, to scale with your traffic, to deploy security layer on top of it, like IAP, to absorb the DDoS attack without impacting your backend.
And the price is for 5 forwarding rules, not only one.
Now, of course, you can do differently.
You can use regional solution. This solution is often free or affordable. But you don't have all the Global load balancer feature.
If your backend is on Cloud Run or App Engine. Cloud Endpoint is a solution for Cloud Function (and other API endpoints).
You can deploy and set up your own nginx with your SSL certificate on a compute engine.
If you want to serve only static file, you can have a look to Firebase hosting.
I have attended an AWS training, and they explained to us that a good practice is to have cache all dynamic content via Cloudfront, setting TTL to 0, even if you have an LB in front on the Load Balancer. So it could be like:
Route 53 -> CloudFront -> Application LB
I can not see any advantage of this architecture, instead of having directly (only for dynamic content):
Route 53 -> Application LB
I do not see the point since Cloudfront will send all traffic always to the LB, so you will have:
Two HTTPS negotiation (client <-> Cloudfront, and Cloudfront <-> LB)
No caching at all (it is dynamic content, it should not be cached, since that is the meaning of "dynamic")
You will not have the client IP since your LB will see only the Cloudfront IP (I know this can be fixed, to have the client IP, but then you will have issues with the next bullet).
As an extra work, you need to be able to update your LB security groups often, to match the CloudFront IPs (for this region), as I guess you want to get traffic only from your Cloudfront, and not directly from the LB public endpoint.
So, probably, I am missing something important about this Route 53 -> CloudFront -> Application LB architecture.
Any ideas?
Thanks!
Here are some of the benefits of having cloudfront on top of your ALB
For a web application or other content that's served by an ALB in Elastic Load Balancing, CloudFront can cache objects and serve them
directly to users (viewers), reducing the load on your ALB.
CloudFront can also help to reduce latency and even absorb some distributed denial of service (DDoS) attacks. However, if users can
bypass CloudFront and access your ALB directly, you don't get these
benefits. But you can configure Amazon CloudFront and your Application
Load Balancer to prevent users from directly accessing the Application
Load Balancer (Doc).
Outbound data transfer charges from AWS services to CloudFront is $0/GB. The cost coming out of CloudFront is typically half a cent less
per GB than data transfer for the same tier and Region. What this
means is that you can take advantage of the additional performance and
security of CloudFront by putting it in front of your ALB, AWS Elastic
Beanstalk, S3, and other AWS resources delivering HTTP(S) objects for
next to no additional cost (Doc).
The CloudFront global network, which consists of over 100 points of presence (POP), reduces the time to establish viewer-facing
connections because the physical distance to the viewer is shortened.
This reduces overall latency for serving both static and dynamic
content (Doc).
CloudFront maintains a pool of persistent connections to the origin, thus reducing the overhead of repeatedly establishing new
connections to the origin. Over these connections, traffic between
CloudFront and AWS origins are routed over a private backbone network
for reliability and performance. This reduces overall latency for
serving both static and dynamic content (Doc).
You can use geo restriction, also known as geo blocking, to prevent users in specific geographic locations from accessing content that
you're distributing through a CloudFront distribution (Doc).
In other words you can use the benefits of ClodFront to add new features to your source (ALB, Elastic Beanstalk, S3, EC2) but if you don't need these features it is better not to do this configuration in your architecture.
Cloudfront enables you deliver content faster because Cloudfront Edge locations closer to the user requesting and are connected to the AWS Regions through the AWS network backbone.
You can terminate SSL at cloudfront and make the load balancer listen at port 80
Cloudfront allows to apply geo location restriction easily in 2 clicks.
I think another reason you may want to use CF in front of an ALB is that you could have a better experience with WAF (if you are already using (or planning to) WAF, of course).
Even though WAF is available for both ALB and CF, ALB and CF use different services for WAF. The reason is that Cloudfront is a global service and ALB is one per region.
That may bring more complex management and duplication of ACL (and probably more costs).
Cloudfront is really an amazing CDN content delivery network service like Akamai etc . Now if your web applications having lots of dynamic content like media files even you static code you can put them into a S3 bucket another object storage service by AWS .
Once you have your dynamic content to you S3 bucket you can create a Cloudfront distribution by considering that bucket as a origin this phenomena will cached your dynamic content across AWS multiple edge locations. And it will become fast to deliver on client side.
Now if we talk Load balancer . So it have it’s own purpose to serve image you are using a Application where you get an unpredictable traffic so here your Load balancer which we are considering an Application or classic Load balancer which is accepting request from Route 53 and passing it to your servers.
For high availability and scalability we consider such architecture of Application.
we create a autoscaling group of our ec2 instances and put them behind a load balancer and as per our scaling policy example: if my cpu or memory utilization goes more that 70% launch another instance or similar.
You can set a request policy as well on load balancer to serve traffic to your ec2 server maybe in round Robbin or on availability.
I just shared the best practices recommended of AWS fault tolerant and highly available architecture . I hope this may help you to get a better idea to decide now .
Please let me know if I can help you with more suggestions on it.
Thanks and Happy Leanings!!
I have a website which I serve using express running in an EC2 instance. But this EC2 serves only the static content (html, js, css) and the dynamic part comes from API Gateway. Right now, these two have different IPs (and domains) which means that I have to deal with CORS problems accessing API Gateway from the web pages. If I could somehow serve the static content and dynamic one through the same address, that would be much better.
The way I see it, this can be done in two ways. I can serve both of them on the same host but different ports which I'm not sure if it's going to solve the same CORS problem or not. But another way which I'm sure it will not face the CORS problem is serving API Gateway under some specific sub-folder. Like http://example.com/api while the static content is served from any url except that.
Does anyone know how can I do this? Is CloudFront what I need? Or Elastic Load Balancer?
Yes, CloudFront is what you need for this scenario.
Application Load Balancer can also do path-based routing, but it doesn't support API Gateway as a target.
By default, CloudFront can route requests under a single domain to the correct choice from up to 25 destinations, using up to 25 path (matching) patterns (both of these limits can be increased by request, but it sounds like for now, you only need 2 of each, /api/* to the API, and the default * route to EC2). You can also leverage this setup to put some static content in an S3 bucket and take some load off of the servers in EC2.
For this configuration, you will want to configure your API Gateway deployment with a regional endpoint, not an edge optimized endpoint. This is because edge optimized endpoints already use part of the CloudFront infrastructure (a part to which you have no ability to configure), so using an edge-optimized endpoint behind your own CloudFront distribution sends each request and response through the CloudFront network twice, increasing latency.
I'm building a globally distributed streaming platform built with Wowza server. Reading this article on
How could I determine which AWS location is best for serving customers from a particular region? I'd like to use this method to pick the ingest server based on the geolocation or lowest latency.
On the other site our CDN needs to pull from the server that is being streamed to. Is there a way for the Route53 to select the server that doesn't return 404 for the requested content?
You can do this, yes, using Route53 health checks. That way, R53 can determine the appropriate endpoint that is "healthy" and serving up your content. I'm not sure what time gap would be involved though -- probably 30-90 seconds at least to detect an unhealthy endpoint and switch over.