We are building an OLTP gateway, were this gateway has the following (partly) requirements:
High-throughput and low latency, no single point of failure, and fully scalable solution with initial capability for 500 tps per instance and scalable to 5000 tps.
(but of course :-))
The gateway will also have to integrate with third party services (like - acquiring banks, alternative payment methods and other implementations).
We have many system zones (and modules within) so for the sake of clarity I will mention two:
Core Gateway zone – encapsulates many modules consisting together the OLTP engine.
And Back-end Integration zone - for these third party services implementations.
Our system zones separation considered the ability to scale module independently in a clustered environment when the module overloads.
We are working with java and we plan on using AKKA and Camel platforms as the basis for the gateway OLTP architecture.
We chose these platforms for their well known advanced features.
We are new to AKKA and Camel platforms so we did a lot of research and learning over the web and we feel very confident with its capabilities.
So right now our main concern is with AKKA's routing and load balancing messages/requests/work between our modules and system zones.
We try to find out if using a regular NLB between our system zones will be a better choice to go with.
We try to measure our decision according to the following parameters and order:
1. Performance
2. Known and successive implementation
3. Statistics
4. Fault-tolerant
5. Routing strategies
6. Configuration
7. Debugging on (near) real time
We believe that the sections 4-6 in the above list can easily satisfied by AKKA and we wonder about the first three and section 7…
We have researched over the web for information about AKKA's routing and load balancing and came out with many good articles.
http://prochera.com | use case for akka (stackoverflow) (also akka's docs and I have some more links)
But we did not found much of information that will help us to decide which is the better approach (AKKA's routing and load balancing VS regular NLB)
So finally here are banch of questions concerning our dilemma:
1. Can you provide some statistics on AKKA's routing and load balancing performance in distributed systems?
2. Can you provide comparison between AKKA's routing and load balancing VS regular NLB?
3. Can you provide known and successive implementation in the market?
4. Is there known problems/cons using AKKA's routing and load balancing?
5. We wonder on a solution that implements AKKA yet let external load balancing do the inter-machine communications - which can be replaced by all AKKA mechanism at a later phase.
Can you think of one (high level)?
6. Can AKKA perform sticky based load balancing onto multiple back-end servers?
7. Is it easy to debug / log trace on (near) real time the AKKA's routing and load balancing?
Would really appreciate some feedbacks.
Tzvika
Related
I have an application with 3 modules and 25 endpoints (between modules). Modules: Users, CRM, PQR.
I want to optimize AWS costs and generally respect the architecture best practices.
Should I build a lambda function for each endpoint?
Does using many functions cost more than using only one?
The link in Gustavos' answer provides a decent starting point. I'll elaborate on that based on the criteria you mentioned in the comments.
You mentioned that you want to optimize for cost and architecture best practices, let's start with the cost component.
Lambda pricing is fairly straightforward and you can check it out on the pricing page. Basically you pay for how long your code runs in 1MS increments. How much each millisecond costs depends on how many resources you provision for your Lambda function. Lambda is typically not the most expensive item on your bill, so I'd start optimizing it, once it becomes a problem.
From a pricing perspective it doesn't really matter if you have fewer or more Lambda functions.
In terms of architecture best practices, there is no single one-size-fits-all reference architecture, but the post Gustavo mentioned is a good starting point: Best practices for organizing larger serverless applications. How you structure your application can depend on many factors:
Development team size
Development team maturity/experience (in terms of AWS technologies)
Load patterns in the application
Development process
[...]
You mention three main components/modules with 25 endpoints in total:
Users
CRM
PQR
Since you didn't tell us much about the technology stack, I'm going to assume you're trying to build a REST API that serves as the backend for some frontend application.
In that case you could think of the three modules as three microservices, which implement specific functionality for the application. Each of them implements a few endpoints (combination of HTTP-Method and path). If you start with an API Gateway as the entry point for your architecture, you can use that as an abstraction of the internal architecture for your clients.
The API Gateway can route requests to different Lambda functions based on the HTTP method and path. You can now choose how to implement the backend. I'd probably start off with a common codebase from which multiple Lambdas are built and use the API gateway to map each endpoint to a Lambda function. You can also start with larger multi-purpose Lambdas and refactor them in time to extract specific endpoints and then use the API Gateway to route to the more specialized Lambdas.
You might have noticed, that this is a bit vague and that's on purpose. I think you're going to end up with roughly as many Lambdas as you'll have endpoints, but it doesn't mean you have to start that way. If you're just getting started with AWS, managing a bunch of Lambdas and there interaction can seem daunting. Start with more familiar architectures and then refactor them to be more cloud native over time.
It depends on your architecture and how decoupled you want it to be. Here is a good starting point for you to take a look into best practices:
https://aws.amazon.com/blogs/compute/best-practices-for-organizing-larger-serverless-applications/
I have gone through several links regarding the difference between Clustering and Network Load Balancing. What I got :
Clustering: A cluster is a group of resources that are trying to achieve a common objective, and are aware of one another. Clustering usually involves setting up the resources (servers usually) to exchange details on a particular channel (port) and keep exchanging their states, so a resource’s state is replicated at other places as well.
Load Balancing: serving a large number of requests (web, VPN connections, ...) by having multiple "copies" of a server.
Now I am not sure What is the difference between ColdFusion clustering and Network load balancing?
What are the benefits of creating multiple instances in a single CF server,Cluster them and host my webapp ?
In terms of CF the main difference between "clustering" and "load balancing" is with clustering the session scope variables, caches, etc... will be shared around the cluster so that all servers know about the sessions, etc... and can therefore answer any request but with load balancing you are simply splitting the traffic to different servers and they are not aware of each other, so things like session variables are not shared and if used for say a login process will cause the user to logged out again if they move from say Server A to Server B when making subsequent requests. In that situation you will need to implement something like "sticky sessions" on the load balancer to stop people from moving between servers.
Clustering is a broad, imprecise term. Anything involving more than one computer is called "clustering"; even the most trivial approaches. Which makes this a great marketing term. And there also is a data mining technique, clustering (alias of cluster-analysis), that has nothing to do with server clustering.
Try to use more precise terms:
load-balancing for example indicates that every host is essentially doing the same thing.
failover goes a step beyond load-balancing: whereas in load balancing one could split the users (say, by the hash code of their username) and every node is only responsible for part of the users, the term failover indicates that any host may fail, and there is a node that can replace its functionality.
sharding is the opposite. It is also a type of load-balancing, but one where you split the load by some kind of key. You can of course have sharded scenarios that still support failover (at reduced performance) when you do the sharding mostly to improve caching.
cluster-computing commonly refers to distributed computation, such as computing weather models by parallel-processing. This will require interaction between hosts, whereas load-balancing of web sites usually involves only one frontend node.
Most likely, ColdFusion only supports failover style load-balancing. Every node will be able to serve every request (so any node may fail), and will only benefit mildly from sharding.
I'm trying to install and configure a highly availability setup for the WSO2 API Manager. I've been reading through this document: http://docs.wso2.org/wiki/display/Cluster/Clustering+API+Manager and in there it explains to break up the 4 components of the application into separate folders and that these 4 components can run on a single server. I'm not sure why this is needed. All I really want to do is take 2 servers, install the full application on both of them (without breaking the application up into 4 different pieces) and cluster them together between two servers with an Elastic Load Balancer in front of them.
What is the purpose of splitting up the multiple components on the same server if they all run out of a single installation? I'm looking for the simplest way to provide fail over capability to this application if one server goes down. Any insight into their methodology would be greatly appreciated.
Thanks.
The article you've linked describes on distributing different components of API Manager. If you look at the very end of that article there's a link to clustering configuration doc. In a production deployment usually it is encouraged that the 4 components are run on different nodes rather than having everything in a node and having multiple such nodes. That's why it goes on explaining breaking it down to separate components. The official AM doc below has a page on different deployment patterns.
You can go through the following articles to get a better understanding on clustering API Manager.
http://docs.wso2.org/wiki/display/AM140/Clustered+Deployment
http://sanjeewamalalgoda.blogspot.com/2012/09/how-do-clustering-and-enable-replicate.html
My 2cts:
The documentation mentioned in the remarks, explains how WSO2 sees the world of clustering. Spread the different functionality over different JVM's. This sounds logical from architectural point of view. A dis-advantages is that the diffent applications need to me administrated as well by operations. This makes the technical architecture rather complex.
In our situation, we defined 2 different servers with extra CPU and memory, on these servers we have installed the full WSO2 API Manager and defined the cluster configuration. Everything provisioned via Puppet.
Just a straightforward install, all data-source pointing to one schema in an Oracle database.
And...it is working; Our Developers happy, Operations happy, Architect department happy
The project I'm currently involved requires that business logic must be implemented in Web Service that will be consumed by the Presentation Tier Components (i.e. Web Applications).
The company has an Enterprise Service Bus, and up-to-date almost every Web Service developed is exposed through this bus. I asked some colleagues around about when to expose Service through ESB and I got this answers:
If there's an ESB, expose everything through it: There are several benefits like Load-Balancing and location transparency
If the ESB will only act as a Proxy -i.e no message transformation- just don't use it: You'll overload the ESB and lose performance. You'll better do a point-to-point connection.
You should expose a component through ESB if there's a protocol transformation (like exposing a Stored Procedure as a SOAP Service). If this isn't present you better go Point-to-Point.
So I'm curious if there's a general agreement or best-practice of when to expose a Web Service through it or not. Any reading/reference would be a great help.
From my point of view and after 4 years of experience with SOA technologies, using an ESB will always overload the system since you are adding a new layer and making all your communications go through it. Transformation (either messaging or protocol) and routing aren't to hard to accomplish without an ESB and point to point communication will have a bit higher throughput. Same happens also with business process automation, there are ways to get there without the need of an ESB.
In the other hand, the use of an ESB has several benefits in the scope of a corporation but it must be within a vision and strategy. One of the best examples is a company that has been working for a long time with a wide range of tools, each of them for a specific purpose and that made the company be distributed in teams which work in silos, ones isolated from the others. After a long time that makes interaction between teams complex and slow. A well planned SOA strategy will help to integrate all those tools and start replacing them for more meaningful lightweight items.
So, IMHO, Using an ESB just to solve a couple of "issues" in a single project without a corporate strategy isn't a good idea and, eventually, the word SOA will be banned in your company, when the problem isn't SOA by itself by rather the lack of vision and corporate strategy.
The only rule of thumb that I found regarding the use of ESBs is: The requirement of transformation, routing, business process automation (with or without human interaction), etc. in a single project is not a symptom of going SOA (almost every project has to perform transformations, routing and business process automation), but when those needs are the ones for a whole corporation then it's worth to think about it from a business point of view, never a technical one. If there isn't a business perspective, then SOA will fail.
This is a really wide topic and discussion can last for ages, I will suggest you a couple of links for further reading:
Some SOA Case Studies
Top 10 Reasons why SOA fails
I think I know the answer to this one, but I have just spotted someone doing exactly the opposite - who is a very skilled developer, were they just having a bad day?
Q: Is it OK to call a service from within a service in an SOA architected system?
I suspect not, but i'll ask anyway...
I can't think of any reason why not. I can think of multiple reasons why a service could call another service. I've architected and implemented multiple services that do this. I'm also aware of other architectures that set up this sort of system.
Definitely yes. This SOA pattern is commonly called choreography where one web service processes and then passes the message to another down a processing pipeline. Google it and you will find some good references.
Another case may be more technical reason like routing, where you have frontend webservices that routes your messaged to different backend services based on security policy, content, bridge different technology or transport protocol etc.
The answer to this is as always "it depends..." let me explain what I mean.
Calling another service in a SOA is of course a totally acceptable thing to do, it is at the heart of SOA to be able to compose new things out of existing services.
The more important part is how you call the services, consider a SOA system where multiple services collaborate in a call chain call each one enlisting the other in the transactional scope. Doing this sort of thing without careful planning will have a massive impact on your systems performance. The same call chain designed using well partitioned services that are scoped at the correct unit of work suffers less.
Consider system robustness, in a typical architecture one service tends to become more popular than others and lands up having many other services calling it. A failure of this one service brings down the entire system due to all the other services being dependent on the call to this one service.
Consider the difference between synchronous and asynchronous calls, when do you use what? That is the impact of each?
How do you design and partition services to limit the cost of crossing the service boundary on each call?
Lots of questions, but if you look around there are many places to find answers, I suggest starting with these.
I would suggest you read articles by Thomas Erl and Roger Sessions, this will give you a firm handle on what SOA is all about.
Building a SOA
SOA Design Pattern
Achieving integrity in a SOA
Why your SOA should be like a VW Beetle
SOA explained for your boss
WCF Service Performance
think about the architectural goal of "separation of concerns". instead of each service knowing how to do everything, it can rely on other specialized services for shared pieces of functionality
I have some different scenario. What if you want to implement service level clustering in your SOA? For example, say persistence service is resides on one machine and is responsible for handling all persistence tasks in the cluster. So, on other machine which needs persistence functionality, need to connect to the that machine only(ignore fail over factor).
Now while login, if User service has instance of Persistence service directly, then you will not be able to implement service level clustering.
We have our SOA middleware and I called service from each other directly. But when we implemented service level clustering using JMS/ActiveMQ, we faced the problem for those interconnected services.