We have a C++ (SOAP-based) web service deployed Using Systinet C++ Server, that has a single port for all the incoming connections from Java front-end.
However recently in production environment when it was tested with around 150 connections, the service went down and hence I wonder how to achieve load-balancing in a C++ SOAP-based web service?
The service is accessed as SOAP/HTTP?
Then you create several instances of you services and put some kind of router between your clients and the web service to distribute the requests across the instances. Often people use dedicated hardware routers for that purpose.
Note that this is often not truly load "balancing", in that the router can be pretty dumb, for example just using a simple round-robin alrgorithm. Such simple appraoches can be pretty effective.
I hope that your services are stateless, that simplifies things. If indiviual clients must maintain affinity to a particualr instance thing get a little tricker.
Related
Is there a common way to establish a network connection from a CloudFoundry-Service to a CloudFoundry App which the service is bound to.
In typical fashion apps receive their bind credentials and establish network connections to provisioned service for example databases.
It would be very handy to establish a connection from a service to an app, so the service could scrape endpoints that are provided by the app.
Any thoughts on this, why is it / or isn't it possible, why could it be a bad idea.
Normally, you have your service and the application receives credentials from the service through the service binding (i.e. VCAP_SERVICES).
You want to reverse this arrangement, which is fine, but the service will need to have some way to know how to reach the applications. The way to do this would be through routes bound to your application.
I have seen something like this done before, this is roughly the process. I'm sure you can adapt it to your requirements.
Create a service broker. The broker is responsible for managing service instances and service credentials. The broker is notified when an instance is created and when a binding occurs. Your broker will need to handle these requests.
The broker, in addition to its normal responsibilities, is going to need to maintain state indicating which applications have instances & bindings. In addition, the broker is going to need to use the org/space/app guids it's provided through the service broker API and talk to the CloudFoundry API to fetch the routes for the applications that are bound to it. You don't usually get these through the service broker API, but since you want to talk to the applications from the service, you need this information. It gives the service a way to communicate with the application.
Your broker may also provide the service in question (i.e. talking to applications), or it can delegate to some other process/container/VM to provide the service. If your service does the latter, then you need a way to a.) create the process/container/VM and b.) pass along the information it requires to talk to your application.
Obviously, you need to code the logic that will take the routes for applications that have created instances and bindings and communicate with them.
There can be some limitations with using the routes. First, not all routes are public. For internal routes, it would be kind of complicated to allow the broker/service to talk to the app. The broker/service would need to be an application on CF and you would need to specifically allow that communication (would require more API calls). Second, some apps just don't have routes. Perhaps this won't happen in your case, but it's worth considering. Lastly, not all routes are HTTP, some can be TCP as well. Your broker/service would need to handle both of those.
A variation on the above process, instead of using routes or talking to the API, you could have your broker/service provide some mechanism through the credentials to the application such that it registers itself with the broker/service. Thus when your applications start, they'll read the service info, register with the service and then go about their business. In this way, the application would have some additional flexibility about what information it provides when it registers with the broker/service. The downside is that the app has to do some work to be compatible.
Following is a point mentioned in a presentation slide related to SOA, and it confuses me with the concepts of service orchestration and service choreography. To enable service choreography, shouldn't a web service be able to call another web service?
SOA builds applications out of software services. Services comprise intrinsically
unassociated, loosely coupled units of functionality that have no calls to
each other embedded in them.
In theory, a service can do anything it needs to do to accomplish its job. So there doesn't seem to be a good reason to forbid using a second service to do your work. Why reinvent the wheel?
In practice, the issue is more complicated. If you start calling other services on your own web server, then you'll eventually starve it of resources. At best, "real" clients will have to wait a bit longer for their answers while your web service server plays with itself.
Another issue is recursive loops: Service A calls B calls C calls A calls B ... you get the idea. A small change in one service can introduce such a loop without anyone noticing and it can sit there for a long time until it suddenly kills your server.
That is why you should build micro services in a hierarchy inside the server (i.e. below the web service layer - this is not exposed to clients). Those micro services can use each other in a top-down manner (to avoid the loops). Unit tests then make sure they behave properly.
Lastly, such reuse is very slow. Each HTTP request takes a lot of resources to create, send, parse and process. Calling an internal method directly can be 10 - 10000 times faster.
These are the main reasons why the services exposed by a single server shouldn't reuse each other via the "public client API".
Note: There are web services which build new services by using existing ones. IFTTT - "If This Then That" is one such beast.
You could adopt every concept according to your needs. In my current project we have a separate module that is responsible for the Orchestration. This is required since in real life usage, scenarios can be very complicated. So in order to be close to the actual management of your system, you need to have such one.
Another advantage of this approach is that the Separation_of_concerns is kept. Also aligns the business request with the applications, data, and infrastructure that you have. It defines policies and service levels through automated workflows, provisioning etc.
Orchestration is critical in the delivery of Cloud services too. As they are networked to allow sharing of data-processing tasks, centralized data storage, and online access to services or resources.
I've noticed that most architectures that acts as a web service client uses a proxy to communicate with the rest server? While it is possible to access a rest service without a proxy server, one example I've read is this where it uses a proxy server to communicate with its rest server are there any advantages of using a proxy to access a rest service?
Using a proxy is usually not necessary for small local application web services. It depends mostly on your server load (number of clients, frequency of requests), and on the network area where your services are accessed : back-office server-to-server, front-office LAN, WAN or on the whole internet).
The REST webservices are mostly online resources, identified in a unique way by an URL, and generally served in a classic HTTP way. From the client side, he does not know if the data he gets is static, dynamic or cached. He simply gets the data as if it's static.
On large scale applications, with the increase of clients, resources and web services requests, you need technical components to handle problematics like user balancing, usage tracking of your web services as your application evolves. You'll also want to deliver the best performance you can to the clients. This can be achieved efficiently with a proxy solution.
Advantages of NOT using a proxy:
Simplicity
Advantages of using a proxy-based solution:
Rewrite URLs from a single centralized entry point (instead of setting it heterogeneously on each server/app/ws configuration).
Track the usage of your webservices (globally)
Enhance performance capabilities (caching, balancing to dedicated servers)
Managing API versions (switching gobally /myAPI from /myAPI-V1 to /myAPI-V2 easily done, and go back fingers in the nose)
Modifying some API calls on-the-fly (compatibility between versions, do preliminary input data validation, or add technical information to calls).
Manage webservices security globally (control IPs, quota per user, etc).
Hope this answers your question.
Edit (in answer to comment)
The proxy can act as a cache. For frequently asked resources (REST services), it can serve the same response to several users. Your service will be called juste once, even if there is 100 requests on this resource.
But this depend on how your services are really used, so you need to track requests to know if caching is helpful or not in your case.
How many users do you have ?
How many web services ?
Whar kind of data/resources are served ?
How fast are your services (individually) ?
What is the network performance ? (LAN? WAN? Internet? Mobile?)
How many servers and applications serve your users ?
Do you encounter any network load problems ?
A proxy cannot "accelerate" your existing services, but it can enhance the way you serve the resources to your clients.
Do not use a proxy if you do not know if you need it. You must know what is your actual system architecture and what are the weaknesses and bottlenecks.
I always read that one reason to chose a RESTful architecture is (among others) better scalability for Webapplications with a high load.
Why is that? One reason I can think of is that because of the defined resources which are the same for every client, caching is made easier. After the first request, subsequent requests are served from a memcached instance which also scales well horizontally.
But couldn't you also accomplish this with a traditional approach where actions are encoded in the url, e.g. (booking.php/userid=123&travelid=456&foobar=789).
A part of REST is indeed the URL part (it's the R in REST) but the S is more important for scaling: state.
The server end of REST is stateless, which means that the server doesn't have to store anything across requests. This means that there doesn't have to be (much) communication between servers, making it horizontally scalable.
Of course, there's a small bonus in the R (representational) in that a load balancer can easily route the request to the right server if you have nice URLs, and GET could go to a slave while POSTs go to masters.
I think what Tom said is very accurate, however another problem with scalability is the barrier to change upon scaling. So, one of the biggest tenants of REST as it was intended is HyperMedia. Basically, the server will own the paths and pass them to the client at runtime. This allows you to change your code without breaking existing clients. However, you will find most implementations of REST to simply be RPC hiding behind the guise of REST...which is not scalable.
"Scalable" or "web scale" is one of the most abused terms when it comes to the web, the cloud and REST, and mainly used to convince management to get their support for moving their development team on board the REST train.
It is a buzzword that holds no value. If you search the web for "REST scalability" you'll find a lot of people parroting each other without any concrete evidence.
A REST service is exactly equally scalable as a service exposed over a SOAP interface. Both are just HTTP interfaces to an application service. How well this service actually scales depends entirely on how this service was actually implemented. It's possible to write a service that cannot scale as all in both REST and SOAP.
Yes, you can do things with SOAP that makes it scale worse, like rely on state and sessions. SOAP out of the box does not do this. This requires you to use a smarter load balancer, which you want anyway if you're really concerned with whatever form of scaling.
One thing that REST allows that SOAP doesn't, and that some other answers here address, is caching cacheable responses through an HTTP caching proxy or at the client side. This may make a REST service somewhat more lightly loaded than a SOAP service when a lot of operations' responses are cacheable. All this means is that fewer requests end up in your service.
The main reason behind saying a rest application is scalable is, Its built upon a HTTP protocol. Because HTTP is stateless. Stateless means it wont share anything between other request. So any request can go to any Server in a load balanced cluster. There is nothing forcing this user request go to this server. We can overcome this by using token.
Because of this statelessness,All REST application are very easy to scale. But if you want get high throughput(number of request capable in one second) in each server, then you should optimize blocking things from the application. Follow the following tips
Make each REST resource is a small entity. Don't read data from join of many tables.
Read data from near by databases
Use caches (Redis) instead of databases(You can save DISK I/O)
Always keep data sources as much as near by because these blocks will make server resources (CPU) ideal and it no other request can use that resource while it is ideal.
A reason (perhaps not the reason) is that RESTful services are sessionless. This means you can easily use a load balancer to direct requests to various web servers without having to replicate session state among all of your web servers or making sure all requests from a single session go to the same web server.
I'm setting up a project and one of the main questions is how to implement a simple message queueing system (something along the line of a messenger chat system). I would like to avoid polling, but there will most likely be a lot of concurrent connections (tens of thousands). These will be HTTP+SSL connections, started from an application not a browser.
One solution I found would be DNS Load Balancing: distribute these persistent connections across a bunch of nginx webservers.
What do you think? Any other possible solutions?
For load balancing, keeping the application server stateless will open up the field significantly. Once you've got that, you're free to use almost any generic load balancer. From something protocol specific like HTTP load balancers to the generic TCP level load balancers.
Keep it stateless, the rest will be trivial in comparison.
If you are planning on using web services (XML message passing ), you can use gsoap, which has an included web server sample application, which uses thread pools. I've run a server using this and mysql ( for persistent state ). I agree with Ryan on reducing/eliminating the statefulness of the application.
DNS load balancing will allow you to distribute queries between multiple IP addresses, which could be multiple servers. Keep in mind that your clients could get different servers from one request to another, so your applicaiton can't use local state management. Your applicaiton will have to store its state in a centralized location such as a database.
Have you considered peer-to-peer? The state of the art in punching through firewalls is actually very effective especially since you're running your own client software in each instance, and you have servers to start the connection.
More work, but significantly less server resources.
Also, write your own server software - make sure it can handle a lot of connections and is extraordinarily lightweight and you should be able to handle thousands of connections per server before you do load balancing.
-Adam