I have a system set up on AWS where I have a set of ec2 insatnces (as an application server from an elastic beanstalk) running in an auto-scaling load-balanced environment. All this works fine.
I would like to load test this instance in order to obtain results that help me to figure out what more needs to be done to the system in order for it to handle, potentially, millions of users. I have used a tool called Locust (http://locust.io) so far to do this. This allows me to send requests to my instance(s?) through a proxy as desired. However, I cannot tell whether the requests are being routed to multiple instances or the same one constantly; and if they are being load balanced appropriately I can't see how many requests each of the ec2 instances are receiving or their health under load. (I have a feeling that the requests are not being properly load balanced as the failure rate always seems to increase drastically at a similar point every test run.)
Is there a way to get this information inside from the AWS ec2 or elastic beanstalk consoles, or is there a better distributed web based load testing tool that can provide the data I need?
There are two ways to get this information
1) Create S3 Bucket and save ELB logs. You can filter these logs to check which instance is serving your request
2) Retrieve application level logs : If apache/nginx installed on your EC2 instances to serve the request. Filter apache/nginx logs in every machine
Hope it helps !!
There is a way to get this data from the AWS console.
Inside the elastic beanstalk console there is a tab titled health. This tab (in the enhanced health overview) shows the number of requests per second, the response for the requests, the latency, the load average and the CPU utilisation for each ec2 instance being run by the elastic beanstalk.
An example of this data is shown in the following image.
This data allows the system manager to see which of their back-end instances are receiving requests and how many they are each being sent through a load-balancer and a proxy.
This can also be attained from the AWS CLI using:
eb health environment_name
Related
I've been digging in the AWS docs for ages and am at my wits end trying to find non AWS official examples.
How do I decide if I should have failover and latency routing or should I have both? I currently have the site on Elastic beanstalk with both a dev and production version, but I get a 500 or 502 errors at least a couple times a month where if you refresh the page, it eventually loads but then the CSS is missing or the page doesn’t load and sometimes the page is just slow to load even with caching. How am I supposed to know if it’s a need for failover or latency routing, or should I have both? The AWS notifications only say “Environment health has transitioned from Degraded to Severe”. How do I log where/which AWS server Route 53 had serve the page?
Are you supposed to have multiple EC2 instances for latency based routing? I’m confused why the docs say to create a latency record for each of my EC2 instances.
https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/TutorialTransitionToLBR.html
I currently have Codepipeline connected to my Github, so that changes are automatically deployed to the dev site, and then I manually approve changes to production. If I have multiple EC2 instances, do I need to set up the code pipeline for each EC2 instance such that it’s connected to my Github and then manually approve changes for all instances—ie would I just have multiple copies of the site hosted in diff regions in this situation? How do people manage this? I’m assuming there’s some way to approve production launch for all at once if this is what is done but I don't know what to google
I have simple server now (some xeon cpu hosted somewhere), running apache/php/mysql (no docker, but its a possibility) and Im expecting some heavy traffic and I need my server to handle that.
Currently the server can handle about 100 users at once, I need it to handle couple thousands possibly.
What would be easiest and fastest solution to move my app to some scalable hosting?
I have no experience with AWS or something like that.
I was reading about AWS and similar, but Im mostly confused and not sure what should I choose.
The basic choice is:
Scale vertically by using a bigger computer. However, you will eventually hit a limit and you will have a single-point of failure (one server!), or
Scale horizontally by adding more servers and spreading the traffic across the servers. This has the added advantage of handling failure because, if one server fails, the others can continue serving traffic.
A benefit of doing horizontal scaling in the cloud is the ability to add/remove servers based on workload. When things are busy, add more servers. When things are quiet, remove servers. This also allows you to lower costs when things are quiet (which is not possible on-premises when you own your own equipment).
The architecture involves putting multiple servers behind a Load Balancer:
Traffic comes into a Load Balancer
The Load Balancer sends the request to a server (often based upon some measure of how "busy" each server is)
The server processes the request and sends a response back to the Load Balancer
The Load Balancer sends the response to the original requester
AWS has several Load Balancers available, which vary by need. If you are simply sending traffic to a single application that is installed on all servers, a Network Load Balancer should be sufficient. For situations where different parts of the application are on different servers (eg mobile interface vs web interface), you could use a Application Load Balancer.
AWS also assists with horizontal scaling by providing the Amazon EC2 Auto Scaling service. This allows you to specify details of the servers to launch (disk image, instance type, network settings) and Auto Scaling can then automatically launch new servers when required and terminate ones that aren't required. (Note that they launch and terminate, not start and stop.)
You can further define scaling policies that tell Auto Scaling when to launch/terminate instances by measuring metrics such as CPU Utilization. This way, the number of servers can approximately match the volume of traffic.
It should be mentioned that if you have a database, it should be stored separately to the application servers so that it does not get terminated. You could use the Amazon Relational Database Service (RDS) to run a database for you, or you could run one on a separate Amazon EC2 instance.
If you want to find out more about any of the above technologies, there are plenty of talks on YouTube or blog posts that can explain and demonstrate their use.
I have a monitor installed into with my application, JavaMelody. The application is running on 7 different instances in AWS in an auto scaling group behind a load balancer in AWS. When I go to myapp.com/monitoring, I get statistics from JavaMelody. However, it is only giving me specifics for the node that the load balancer happens to direct me. Is there a way I can specify which node I am browsing to in a web browser?
The Load Balancer will send you to an Amazon EC2 instance based upon a least open connections algorithm.
It is not possible to specify which instance you wish to sent to.
You will need to connect specifically to each instance, or have the instances push their data to some central store.
You should use CloudWatch Custom Metrics to write data from your instances and their monitoring agent, and then use CloudWatch Dimensions to aggregate this data for the relevant instances
I have not tried this myself but you may create several listeners in your load balancer with a different listening port and a different target server for each listener. So the monitoring reports of the instance #1 may be available at http://...:81/monitoring etc for #2, #n
Otherwise, I think that there are other solutions such as:
host or path based load balancing rules (path based rules would need to add net.bull.javamelody.ReportServlet in your webapp to listen on different paths)
use a javamelody collector server to collect the data in a separate server and to have monitoring reports for each instance or aggregated for all instances
send some of the javamelody metrics to AWS CloudWatch or to Graphite
I have created a LBCookieStickinessPolicy for my ELB.
But I can't seem to find on any AWS documentation a command that retrieves the instances that are currently 'sticked' (I mean, the actual instance that the ELB is sending load now).
I only find the commands that create the policy itself (create-lb-cookie-stickiness-policy & create-app-cookie-stickiness-policy) ...Any ideas?
Sticky sessions mean that a single user's web browser gets stuck to a single server instance (unless the server goes down or the user clears cookies). The ELB still distributes load across all the servers attached to it. The ELB would distribute multiple users across multiple server instances.
So there is no way to see what you are looking for because the ELB is always using all instances. Now if you just had a single user on your website, you could look at the server logs of each web server to determine which server that user is "stuck" to. In general you would need to look at the web server logs to see which servers are currently receiving traffic.
As I understand I can use EC2 as web server for my application. But how load balancer is working?
For example I have one EC2 instance. In this way load balancer will not work. Am I right?
For example I have few EC2 instances. In this way I can configure load balancer to balance between all my EC2 servers.
Am I right?
Application files at all instances should be synced? Is there is any Amazon tool to sync? Or I should use something like rsync or post commit hooks to sync files between EC2 instances?
Is it possible to use one EC2 instance for web application (php + nginx) and for CDN (Cloud Front)? Or what is the better way to reach this: I need to store static files but I should access them from web application (php scripts) through file system. So I am going to use EC2 and Clod Front. But how can I get access?
Thanks for your time.
Technically, the load belancer will work, it's just that it'll only balance the traffic to one instance.
Correct. You register the instances with the elastic load balancer, and whilst those instances are healthy - it will respond to them.
There's many different ways to sync files - it all depends on what you want to sync. Cloud Architecture is a little different to traditional architecture. For example, rather than loading the images onto the EBS volume, you'd try and offload them (and serve them) from S3. Therefore the only things you'd need to "sync" would be the webserver files themselves. You could use CloudFormation to roll out updates, post commit hooks and rsync are also good options. The challenge is to remember that it can scale / fail almost at will - so you need to ensure that each instance knows how to get the information and keep itself updated in isolation.
Yes. It's called a custom origin. What you want to do though is put a url rewrite on the outbound server that rewrites the local urls to cloudfront domains.
Hope that helps