I am developing an application service based on WSO2 AS. my intention is that the application should be deployed in an AS-cluster in order to cope with the high volume traffic.
the cluster should be a dynamic one in order to scale up or down as per the traffic changes.
also, a user's service might persist in one of the instances for quite some time; in case of failure, a user's service should be restored in a peer instance by the backup and restore mechanism of an object archive(database).
So, the challenge is:
I need to tell the load balancer something about the instance in which the user service persists. so that the load balancer will always route the same user's requests to the same instance in the cluster. and in case of failure, I could update the load balancer with the new instance in which the user's service had been restored.
preferably it could be something that could be generated dynamically by a application server instance; accessible in the program environment; understood and used by the load balancer to route request...
anyone has any idea?
thanks a lot
After googling around for some time. I found an alternative which WSO2 claimed supporting(http://wso2.com/products/elastic-load-balancer/).
NGinx Plus comes with a feature named Session Persistence (https://www.nginx.com/products/session-persistence/) which provides methods directing load balancer of its routing of incoming requests to a specific back end server
Related
This question is for the infrastructure pros, hope anyone reaches this text.
I’m currently using a setup with one EC2 instance behind a classic load balancer on AWS running a websocket express based server. I always planed to scale my application so I started it behind a LB.
Now I’m on time to startup another instance, but I have this major problem: My websocket leaves a program running on the server - even when the user is out of the website - and return to show the program log to the user when he comes back to the website.
Of course if the user connects to another instance on the load balancer, he will not be able to access a program running on another instance. So the only solution is to connect a user to the same EC2 instance, always.
I searched a lot but I didn’t find anything related, besides sticky sessions based on cookies. The problem of this solution is that it expires after sometime, and I want my user to access the program log again no matter how much time he spent without doing it.
So my question is: Is there a way to sticky a user connection with the same EC2 instance using a AWS classic load balancer?
In a way that new users follow the standard algorithm, going to be connected to the lower used instance, and old users keeps going to the same EC2 every new connection. Is that possible?
Otherwise I’ll not be able to scale my application delivering, because the main purpose of this server is to connect this running program with a specific user.
I don't think you can customize CLB for that. But ALB just recently introduced Application Cookie Stickiness:
Application Load Balancer (ALB) now supports Application-based cookie stickiness. This new feature helps customers ensure that clients connect to the same load balancer target for the duration of their session using application cookies. This enables customers to achieve a consistent client-server experience with greater controls such as the flexibility to set custom cookie names and criteria for client-target stickiness within a target group.
Thus maybe, if you can migrate from CLB into ALB, the application-level cookies could be solution to your issue.
There is one thing of scaling that I yet do not understand. Assume a simple scenario ELB -> EC2 front-end -> EC2 back-end
When there is high traffic new front-end instances are created, but, how is the connection to the back-end established?
How does the back-end application keep track of which EC2 it is receiving from, so that it can respond to the right end-user?
Moreover, what happen if a connection was established from one of the automatically created instances, and then the traffic is low again and the instance is removed.. the connection to the end-user is lost?
FWIW, the connection between the servers is through WebSocket.
Assuming that, for example, your ec2 'front-ends' are web-servers, and your back-end is a database server, when new front-end instances are spun up they must either be created from a 'gold' AMI that you previously setup with all the required software and configuration information, OR as part of the the machine starting up it must install all of your customizations (either approach is valid). with either approach they will know how to find the back-end server, either by ip address or perhaps a DNS record from the configuration information on the newly started machine.
You don't need to worry about the backend keeping track of the clients - every client talking to the back-end will have an IP address and TCPIP will take care of that handshaking for you.
As far as shutting down instances, you can enable connection draining to make sure existing conversations/connections are not lost:
When Connection Draining is enabled and configured, the process of
deregistering an instance from an Elastic Load Balancer gains an
additional step. For the duration of the configured timeout, the load
balancer will allow existing, in-flight requests made to an instance
to complete, but it will not send any new requests to the instance.
During this time, the API will report the status of the instance as
InService, along with a message stating that “Instance deregistration
currently in progress.” Once the timeout is reached, any remaining
connections will be forcibly closed.
https://aws.amazon.com/blogs/aws/elb-connection-draining-remove-instances-from-service-with-care/
I have created a LBCookieStickinessPolicy for my ELB.
But I can't seem to find on any AWS documentation a command that retrieves the instances that are currently 'sticked' (I mean, the actual instance that the ELB is sending load now).
I only find the commands that create the policy itself (create-lb-cookie-stickiness-policy & create-app-cookie-stickiness-policy) ...Any ideas?
Sticky sessions mean that a single user's web browser gets stuck to a single server instance (unless the server goes down or the user clears cookies). The ELB still distributes load across all the servers attached to it. The ELB would distribute multiple users across multiple server instances.
So there is no way to see what you are looking for because the ELB is always using all instances. Now if you just had a single user on your website, you could look at the server logs of each web server to determine which server that user is "stuck" to. In general you would need to look at the web server logs to see which servers are currently receiving traffic.
I am looking to build a scalable REST webservice on the Google Cloud Compute Engine but have a couple of requirements that I am not sure how best to implement.
Structure so far:
2 Instances running a REST webservice connected to a MySQL Cloud database.
(number of instances to scale up in the future)
Load balancer to split request between the two or more Instances.
this part is fine.
What I need next is that the traffic (POST requests from instances to an external webservice) must come from a single IP address. I assume these requests can not route back through the public IP of the load balancer?
I get the impression the solution to this is to route all requests from instances though a 3rd instance running squid. Is this the best way to do this? (side question)
Now to my main question:
I have been reading about ApiAxle which sounds like a nice proxy for Web Services, giving some good access control, throttling and reporting capabilities.
Can I have an instance running ApiAxle followed by a google cloud Load Balancer which shares the request from the proxy to the backend instances that do the leg work and feed the response back through the ApiAxle proxy, thus having everything though a single IP visible to clients using the API? (letting me add new instances to the pool to add capacity.)
and Would the proxy be much of a bottle neck?
Thanks in advance.
/Dave
(new to this, so sorry if its a stupid question because I cant find anything like this on the web)
Sounds like you need to NAT on your outbound traffic so it appears to come from one IP address. You need to do that via a third instance since Google LB stack doesn't provide this. GCLB works only with inbound connections on the load-balanced IP.
You can setup source-NAT using advanced routing, or you can use a proxy as you suggested.
I have an application on an Windows server EC2 with an SQL server for our database.
What I would like to do is an load balancer so the application won't fail due to overload.
I have a couple of questions that Im not certain about it.
I believe that i need to create an image of my current instance and duplicate it. my problem is that my database is based on my current instance so it would duplicate my database as well.
Do I need another instance just for my database?
If yes, then it means that I need a total of 3 instances. 2 for the application and 1 for the database.
In this case I need to change my application to connect to the new instance database instead of the current database.
After all that happens I need to add a load balancer.
I hope I made myself clear.
I would recommend using RDS (http://aws.amazon.com/rds/) for this. This way you don't have to worry about the database server and just host your application server on EC2 instances. Your AMI will then contain just the application server and thus when you scale up you will be launching additional app servers only and not database servers.
Since you are deploying a .NET application, I would also recommend taking a look at Elastic Beanstalk (http://aws.amazon.com/elasticbeanstalk/) since it will really help to make auto scaling much easier and your solution will scale up/down as well as self-heal itself.
As far the load balancer is concerned, you can either manually update your load balancer will the new instances of your application server or you can let your auto scale script do it for you. If you go for ElasticBeanstalk, then Elastic Beanstalk will take care of adding/removing instances to/from your Elastic Load Balancer for you on its own.