I have previously seen it done by having one EC2 instance running HAProxy, configured via a json file/lambda function, that in turn controlled the traffic with sticky sessions, into two separate elasticbeanstalk applications. So we have two layers of load balancing.
However, this has a few issues, one being: Testing several releases becomes expensive, requires more and more EB applications.
By canary release, I mean, being able to release to only a percentage of traffic, to figure out any errors that escaped the devs, the review process, and the QA process, without affecting all traffic.
What would be the best way to handle such a setup with AWS resources and not break the bank? :)
I found this Medium article that explain the usage of passive autoscaling group where you deploy the canary version into it and monitor for statistics. Once you are satisfied with the result, you can change the desired count for the canary autoscaling group to 0, and perform rolling upgrade to the active autoscaling group.
Here is the link to the article: https://engineering.klarna.com/simple-canary-releases-in-aws-how-and-why-bf051a47fb3f
The way you would achieve canary testing with elastic beanstalk is by
Create a 2nd beanstalk environment to which you deploy the canary release
Use a Route53 Weighted routing policy to send a percentage of the DNS requests to your canary environment.
If you're happy with the performance of the canary you can then route 100% of the traffic to the canary env, etc.
Something to keep in mind with DNS routing is, that the weighted routing is not an exact science since clients cache DNS based on the TTL you set in Route53. In the extreme scenario where you would have e.g. only one single client calling your beanstalk environment (such as a a single web server) and the TTL is set to 5 minutes, it could happen that the switching between environments only happens every 5 minutes.
Therefore for weighted routing it is recommended to use a fairly low TTL value. Additionally having many clients (e.g. mobile phones) works better in conjunction with DNS routing.
Alternatively it might be possible to create a separate LB in front of the two beanstalk environments that balances requests between the beanstalk environments. However I'm not 100% sure if a LB can sit in front other (beanstalk) LBs. I suspect the answer is not but I haven tried yet.
Modifying the autoscaling group in elastic beanstalk is not possible, since the LB is managed by beanstalk and beanstalk can decide to revert the changes you did manually on the LB. Additionally beanstalk does not allow you to deploy to a subset of instances while keeping the older version on another subset.
Hope this helps.
Traffic splitting is supported natively by Elastic Beanstalk.
Be sure to select a "high availability" config preset when creating your application environment (by clicking on "configure more options"), as this will configure a load balancer for your env:
Then edit the "Rolling updates and deployments" section of your environment and choose "Traffic splitting" as your deployment strategy.
Related
We are looking to separate our blog platform to a separate ec2 server (In Nginx) for better performance and scalability.
Scenario is:
Web request (www.example.com) -> Load Balancer/Route -> Current EC2 Server
Blog request (www.example.com/blog) -> Load Balancer/Route -> New Separate EC2 Server for blog
Please help in this case what is the best option to use:
Haproxy
ALB - AWS
Any other solution?
Also, is it possible to have the load balancer or routing mechanism in a different AWS region? We are currently hosted in AWS.
Haproxy
You would have to set this up on an EC2 server and manage everything yourself. You would be responsible for scaling this correctly to handle all the traffic it gets. You would be responsible for deploying it to multiple availability zones to provide high availability. You would be responsible for installing all security updates on the operating system.
ALB - AWS
Amazon will automatically scale this out to handle any amount of traffic you get. Amazon will handle all security patches of the underlying system. Amazon provides free SSL certificates for ALBs. Amazon will deploy this automatically across multiple availability zones to provide high availability.
Any other solution?
I think AWS Global Accelerator would work here as well, but you would have to weigh the differences between Global Accelerator and ALB to decide which fits your use case and budget the best.
You could also look at placing a CDN in front of everything, like CloudFront or Cloudflare.
Also, is it possible to have the load balancer or routing mechanism in
a different AWS region?
AWS Global Accelerator would be the thing to look at if load balancing in different regions is a concern for you. Given the details you have provided I'm not sure why you would want this however.
Probably what you really need is a CDN in front of your websites, with or without the ALB.
Scenario is:
Web request (www.example.com) -> Load Balancer/Route -> Current EC2
Server Blog request (www.example.com/blog) -> Load Balancer/Route ->
New Separate EC2 Server for blog
In my view you can use ALB deployed in multi AZ for high availability for the following reasons :-
aws alb allows us to route traffic based on various attributes and path in URL is one of them them.
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html#rule-condition-types
With aws ALB you can have two target groups with instance handling traffic one for first path (www.example.com) and second target group for another path (www.example.com/blog).
ALB allows something called SNI (which allows to handle multiple certications behind a single alb for multiple domains), so all you need to do is set up single https listener and upload your certificates https://aws.amazon.com/blogs/aws/new-application-load-balancer-sni/
i have answered on [something similar] it might help you also
This is my opinion, take it as that. I am sure a lot of people wont agree.
If your project is small or personal, you can go with HAProxy (Cheap USD4 or less if you get a t3a as a spot instance) Or free if you place it inside another EC2 of yours may be using docker.
If your project is not personal or not small, go with ALB (Expensive but simpler and better integrated to other AWS stuff)
HAProxy can handle tons of connections, but you have to do more things by yourself. ALB can also handle tons of connections and AWS will do most of the work.
I think HAProxy is more suitable for personal/small projects because if your project doesnt grow, then you dont have to touch HAProxy. It is set and forget the same as ALB but cost less.
You usually wont mind about Availability zones or disaster tolerance in a personal project, so HAProxy should be easy to config.
Another consideration: AWS offers a free tier on ALB, so if your project will run for less than a year ALB is the way to go.
If you are learning, then ALB should be considered because real clients usually love to stick to AWS in all aspects, and HAProxy is your call and also your risk (just to reduce cost for a company that usually pays a lot more for your salary, so not worth the risk).
I am using AWS Elastic Beanstalk. In there, I selected a Traffic Splitting deploy strategy, with a 100% split (so that 100% of new instances will have the new version and have their health evaluated).
Here's how (according to their documentation) that is supposed to work:
During a traffic-splitting deployment, Elastic Beanstalk creates a new set of instances in a separate temporary Auto Scaling group. Elastic Beanstalk then instructs the load balancer to direct a certain percentage of your environment's incoming traffic to the new instances. Then, for a configured amount of time, Elastic Beanstalk tracks the health of the new set of instances. If all is well, Elastic Beanstalk shifts remaining traffic to the new instances and attaches them to the environment's original Auto Scaling group, replacing the old instances. Then Elastic Beanstalk cleans up—terminates the old instances and removes the temporary Auto Scaling group.
And more specifically:
Rolling back the deployment to the previous application version is quick and doesn't impact service to client traffic. If the new instances don't pass health checks, or if you choose to abort the deployment, Elastic Beanstalk moves traffic back to the old instances and terminates the new ones.
However, it seems silly that it only looks at my internal /health health checks, and not the overall health status of the environment, from the HTTP status codes, that it already has information on.
I tried the following scenario:
Deploy a new version.
As soon as the "health evaluation period" begins, flood the server with error 500s (from an endpoint I made specifically for this purpose).
AWS then moves all my instances into "degraded" state, and "unhealthy", but then seems to ignore it, and goes on anyway.
See the following two log dump screenshots (they are oldest-first).
Is there any way that I can make AWS respect the HTTP status based health checks that it already performs, during a traffic split? Or am I bound to only rely on custom-developed health checks entirely?
Update 1: Even weirder, I tried making my own healthchecks fail always too, but it still decides to deploy the new version with the failed healthcheck!
Update 2: I noticed that the temporary auto scaling group that it creates while assessing health does only have an "EC2" type health check, and not "ELB". I think that might be the root cause. If I could only get it to use "ELB" instead.
That is interesting! I do not know if setting the health check type to "ELB" may do the job because we use CodeDeploy, which has far better rollback capabilities than AWS Elastic Beanstalk.
However, there is a well-document way in the docs [1] to apply the setting you are looking for:
[...] By default, the Auto Scaling group, created for your environment uses Amazon EC2 status checks. If an instance in your environment fails an Amazon EC2 status check, Auto Scaling takes it down and replaces it.
Amazon EC2 status checks only cover an instance's health, not the health of your application, server, or any Docker containers running on the instance. If your application crashes, but the instance that it runs on is still healthy, it may be kicked out of the load balancer, but Auto Scaling won't replace it automatically. [...]
If you want Auto Scaling to replace instances whose application has stopped responding, you can use a configuration file to configure the Auto Scaling group to use Elastic Load Balancing health checks. The following example sets the group to use the load balancer's health checks, in addition to the Amazon EC2 status check, to determine an instance's health.
Example .ebextensions/autoscaling.config
Resources:
AWSEBAutoScalingGroup:
Type: "AWS::AutoScaling::AutoScalingGroup"
Properties:
HealthCheckType: ELB
HealthCheckGracePeriod: 300
It does not mention the new traffic splitting deployment feature, though.
Thus, I cannot confirm this is the actual solution, but at least you can give it a shot.
[1] https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environmentconfig-autoscaling-healthchecktype.html
Once upon a time I thought that the Immutable Deployment option in Elastic Beanstalk was a holy panacea -- but it only works when a deployment involves no changes to the application's database schema.
We've now resorted to blue-green deployments. However, this only works if you control the DNS. If you are a SaaS solution and you allow customers to create a CNAME then B/G is often a spectacular failure as the enterprise: a) sets a very high TTL, and/or b) their internal DNS or firewalls caches the underlaying IP addresses of the ALB (which are dynamic and, of course, replaced when you swap the URL of the blue and green environments).
Traffic splitting is written as an option in the Elastic Beanstalk documentation.
But it's not actually an option in the configuration section in the console.
This wouldn't be the first time I've seen Elastic Beanstalk's docs are out of date so it could be AWS have removed that feature.
Since AWS introduced CodeStar I suspect Elastic Beanstalk is getting the cold shoulder.
I have an application running on AWS EC2. In order to get high availability, we are planning to set up AWS Load balancer and host the application in a minimum of 2 EC2 instances.
We also heard about Docker swarm, in which we can create a service with 2 managers on 2 separate EC2 instances and the swarm will take care of the rest (instead of using ALB with 2 EC2). It will balance the load to all the containers inside the cluster and it also restarts the container if anything makes this go down.
So I want to know which is the best choice for my case. The application won't get heavy load/traffic. Why I choose load balancer is for high availability. If 1 instance goes down the other will be taken care of this.
If there are any other choices to fulfill my requirements, it will be highly appreciated.
Thanks.
I presume it's a stateless application.
A combination of both Swarm & ALB is what you can go for but you will need to incorporate autoscaling etc sooner or later which means you will need to manage the swarm cluster & maintain it.
With ALB you will get real good metrics which you will surely miss while using Swarm.
BUT, you do have a few better options which will manage the cluster for you. You just will have to manage & maintain the docker images -
Use ECS.
Use EKS (only in us-east-1 as if now)
Use ElasticBeanstalk MultiContainer.
Ref -
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_ecs.html
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html
https://aws.amazon.com/eks/
I have an AWS environment with one ALB, two api nodes and an RDS. I perform in place deployments today. I read about canary releases and was wondering how I could transform my current environment to use canary deployments. My environment is entirely on AWS. The main question I have is about routing, what service/component is needed to dial up/down?
Route53's Weighted Routing, The big disadvantage of this approach is that propagating DNS changes can take some time, another approach is to use autoscaling groups(ASG) to upscale the new version and then downscale the old version once you are confident.
Weighted routing lets you associate multiple resources with a single
domain name (example.com) or subdomain name (acme.example.com) and
choose how much traffic is routed to each resource. This can be useful
for a variety of purposes, including load balancing and testing new
versions of software.
How do you put up a maintenance page in AWS when you want to deploy new versions of your application behind an ELB? We want to have the ELB route traffic to the maintenance instance while the new auto-scaled instances are coming up, and only "flip over" to the new instances once they're fully up. We use auto-scaling to bring existing instances down and new instances, which have the new code, up.
The scenario we're trying to avoid is having the ELB serve both traffic to new EC2 instances while also serving up the maintenance page. Since we dont have sticky sessions enabled, we want to prevent the user from being flipped back and forth between the maintenance-mode page and the application deployed in an EC2 instance. We also can't just scale up (say from 2 to 4 instances and then back to 2) to introduce the new instances because the code changes might involve database changes which would be breaking changes for the old code.
I realise this is an old question but after facing the same problem today (December 2018), it looks like there is another way to solve this problem.
Earlier this year, AWS introduced support for redirects and fixed responses to Application Load Balancers. In a nutshell:
Locate your ELB in the console.
View the rules for the appropriate listener.
Add a fixed 503 response rule for your application's host name.
Optionally provide a text/plain or text/html response (i.e. your maintenance page HTML).
Save changes.
Once the rule propagates to the ELB (took ~30 seconds for me), when you try to visit your host in your browser, you'll be shown the 503 maintenance page.
When your deployment completes, simply remove the rule you added.
The simplest way on AWS is to use Route 53, their DNS service.
You can use the feature of Weighted Round Robin.
"You can use WRR to bring servers into production, perform A/B testing,
or balance your traffic across regions or data centers of varying
sizes."
More information in AWS documentations on this feature
EDIT: Route 53 recently added a new feature that allows DNS Failover to S3. Check their documentation for more details: http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover.html
Came up with another solution that's working great for us. Here are the required steps to get a simple 503 http response:
Replicate your EB environment to create another one, call it something like app-environment-maintenance, for instance.
Change the configuration for autoscaling and set the min and max servers both to zero. This won't cost you any EC2 servers and the environment will turn grey and sit in your list.
Finally, you can use the AWS CLI to now swap the environment CNAME to take your main environment into maintenance mode. For instance:
aws elasticbeanstalk swap-environment-cnames \
--profile "$awsProfile" \
--region "$awsRegion" \
--output text \
--source-environment-name app-prod \
--destination-environment-name app-prod-maintenance
This would swap your app-prod environment into maintenance mode. It would cause the ELB to throw a 503 since there aren't any running EC2 instances and then Cloudfront can catch the 503 and return your custom 503 error page, should you wish, as described below.
Bonus configuration for custom error pages using Cloudfront:
We use Cloudfront, as many people will for HTTPS, etc. Cloudfront has error pages. This is a requirement.
Create a new S3 website hosting bucket with your error pages. Consider creating separate files for response codes, 503, etc. See #6 for directory requirements and routes.
Add the S3 bucket to your Cloudfront distribution.
Add a new behavior to your Cloudfront distribution for a route like /error/*.
Setup an error pages in Cloudfront to handle 503 response codes and point it to your S3 bucket route, like /error/503-error.html
Now, when your ELB thorws a 503, your custom error page will be displayed.
And that's it. I know there are quite a few steps to get the custom error pages and I tried a lot of the suggested options out there including Route53, etc. But all of these have issues with how they work with ELBs and Cloudfront, etc.
Note that after you swap the hostnames for the environments, it takes about a minute or so to propagate.
Route53 is not a good solution for this problem. It takes a significant amount of time for DNS entries to expire before the maintenance page shows up (and then it takes that same amount of time before they update after maintenance is complete). I realize that Lambda and CodeDeploy triggers did not exist at the time this question was asked, but I wanted to let others know that Lambda can be used to create a relatively clean solution for this, which I have detailed in a blog post:
http://blog.ajhodges.com/2016/04/aws-lambda-setting-temporary.html
The jist of the solution is to subscribe a Lambda function to CodeDeploy events, which replaces your ASG with a micro instance serving a static page in your load balancer during deployments.
As far as I could see, we were in a situation where the above answers didn't apply or weren't ideal.
We have a Rails application running the Puma with Ruby 2.3 running on 64bit Amazon Linux/2.9.0 that seems to come with a (classic) ELB.
So ALB 503 handling wasn't an option.
We also have a variety hardware clients that I wouldn't trust to always respect DNS TTL, so Route53 is risky.
What did seem to work nicely is a secondary port on the nginx that comes with the platform.
I added this as .ebextensions/maintenance.config
files:
"/etc/nginx/conf.d/maintenance.conf":
content: |
server {
listen 81;
server_name _ localhost;
root /var/app/current/public/maintenance;
}
container_commands:
restart_nginx:
command: service nginx restart
And dropped a copy of https://gist.github.com/pitch-gist/2999707 into public/maintenance/index.html
Now to set maintenance I just switch my ELB listeners to point to port 81 instead of the default 80. No extra instances, s3 buckets or waiting for clients to fresh DNS.
Only takes maybe ~15s or so for beanstalk (probably mostly waiting for cloudformation in the back-end) to apply.
Our deployment process first runs a cloudformation to spun up a ec2 micro instance (Maintenance instance) which copies pre-defined static page from s3 onto the ec2. Cloudformation is supplied with elb's to which micro ec2 instance is attached. Then a script (powershell or cli) is run to remove web instances (ec2) from elb's leaving Maintenance instance.
This way we switch to maintenance instance during deployment process.
In our case, we have two elb's, one for external and the other internal. Our internal elb's will not be updated during this process and is how we have post prod deployment smoke test is done.
Once testing is done, we run another script to attach web instances back to elb's and delete the Maintenance stack.