I want to make a cluster of Data Services Servers(DSS), and use an Enterprise Service Bus (ESB) as load balancer. In this deployment, what is the purpose of having a manager DSS in the cluster, and if there is a manager, is it a single point of failure?
These are the references which I used for load balancing and DSS clustering:
Dynamic load balancing between 3 nodes
How to install WSO2 Carbon cluster management feature?
The dynamic load balancing mechanism in WSO2 ESB, discovers the DSS members in an application group using a group communication framework and shares the load in runtime.
Load balancer is not bound or coupled to any cluster manager - it will simply distribute the load among nodes in applicationDomain.
So - in runtime - cluster manager doesn't create any single point of failure.
If you want you can setup a DSS cluster even without a cluster manager and distribute the load among the nodes via ESB.
The cluster manager - which is a component installed only to manage your cluster...
This is an extension to Prabath's answer.
DSS can be configured to work in a cluster. So that all DSS nodes act as members in a single cluster. This facilitates sharing session among each of the nodes.
Or else, you can have all DSS nodes running in isolation (using the same configuration), fronted by a load balancer (LB). Unlike the previous approach, this method does not support share sessions between DSS nodes. Thus only supports stateless services.
WSO2 ESB can act as a LB. But having a single instance of LB will make it a SPoF. And, LB can be configured to run in a cluster as well.
I don't know what's behind the decision of using an ESB instead of an ELB for LB, but it's up to you which one to use.
The manager is not a single point of failure, it's just a way to manage the entire cluster from a single management console (with limitations), and can be configured to be a worker at the same time.
Regarding the LB layer, you can use keepalived to avoid having a SPoF in the ESB acting as a LB, the same way it's done for WSO2 ELB's.
Take a look on that Failover for ELB with keepalived
Related
With the Cloud Foundry Feature, "Polyglot" for integrated Service Discovery and direct communication between service containers through the internal routes, How does the Load Balancing work? Is Cloud Foundry taking care of the Load Balancing? Is there a way to utilize Client Side Load Balancing, something like Ribbon on top of this Polyglot enabled communication?
When you are using container to container networking...
If you connect directly to IP addresses, no load balancing is done.
If you use the platform's DNS based polyglot service discovery, then you will get limited load balancing via round-robin DNS.
With the polyglot service discovery feature, DNS responses are rotated so that IPs are listed in different orders in the response. You can observe/validate this by doing the following:
Map an internal route to an app
Scale the same app up to have two or more instances
Run cf ssh into any app container
Inside the container, run dig <internal-route>
Repeat the last step any number of times. You should see the response from DNS come back with IP addresses in a different order (they are rotated).
That said, there is nothing to stop you from using a different form of load balancing be that a reverse proxy app you have deployed or something client side like Ribbon.
I have been working with spring and now would like to learn spring boot and microservices. I understand what microservice is all about and how it works. While going through docs i came across many things used to develop microservices along with spring boot which i am very much confused.
I have listed the systems below.and the questions:
Netflix Eureka - I understand this is service discovery platform.
All services will be registered to eureka server and all
microservices are eureka clients. Now my doubt is , without having
an API gateway is there any use with this service registry ? This is
to understand the actual use of service registry.
ZUULApi gateway- I understand ZUUL can be used as API gateway which is basically a load balancer , that calls appropriate
microservice corresponding to request URL. iS that assumption
correct? will the api gateway interact with Eureka for getting the
appropriate microservice?
NGINX - I have read NGINX can also be used as API gateway? Is that possible? Also i read some where else like NGINX can be used as a service registry , that is as an alternate for Eureka ! Thus which is right? Api gateway or service registry or both? I know nginx is a webserver and reverse proxies can be powerfully configured.
AWS api gateway - Is this can also be used as an alternate for ZUUL?
RIBBON - for what ribbon is used? I didn't understand !
AWS ALB- This can also be used for load balancing. Thus do we need ZUUL if we have AWS ALB?
Please help
without having an API gateway is there any use with this service registry ?
Yes. For example you can use it to locate (IP and port) of all your microservices. This comes in handy for devops type work. For example, at one project I worked on, we used Eureka to find all instances of our microservices and ping them for their status (/health, /info).
I understand ZUUL can be used as API gateway which is basically a load balancer , that calls appropriate microservice corresponding to request URL. iS that assumption correct?
Yes but it can do a lot more. Essentially because Zuul is more of a framework/library that you turn into a microservice, you can code it to implement any kind of routing logic you can come up with. It is very powerful in that sense. For example, lets say you want to change how you route based on time of day or any other external factors, with Zuul you can do it.
will the api gateway interact with Eureka for getting the appropriate microservice?
Yes. You configure Zuul to point to Eureka. It becomes a client to Eureka and even subscribes to Eureka for realtime updates (which instances have joined or left).
I have read NGINX can also be used as API gateway? Also i read some where else like NGINX can be used as a service registry , that is as an alternate for Eureka ! Thus which is right? Api gateway or service registry or both?
Nginx is pretty powerful and can do API gateway type work. But there are some major differences. AFAIK, microservices cannot dynamically register with Nginx, please correct me if I am wrong... as they can with Eureka. Second, while I know Nginx is highly (very highly) configurable, I suspect its configuration abilities do not come close to Zuul's routing capabilities (due to having the whole Java language at your disposal within Zuul to code your routing logic). It could be the case that there are service discovery solutions that work with Nginx. So Nginx will take care of the routing and such, but service discovery will still require a solution.
Is this can also be used as an alternate for ZUUL?
Yes AWS API Gateway can be used as a Zuul replacement of sorts. The issue here, just like Nginx, is service discovery. AWS API Gateway lets you apply logic to your routing... though not as open ended as Zuul.
for what ribbon is used?
While you can use the Ribbon library directly, for the most part consider it as an internal dependency of Zuul. It helps Zuul do the simple load balancing that it does. Please note that this project is in maintenance mode and not recommended any more.
This can also be used for load balancing. Thus do we need ZUUL if we have AWS ALB?
You can use ALB with ECS (elastic container service) to replace Eureka/Zuul. ECS will take care of the service discover for you and will map all instances of a particular service to a Target Group. Your ALB routing table can then route to Target Groups based on simple routing rules. The routing rules in ALB are very simple though, but improving over time.
Different systems which can be used for the working of microservices, that comes along with spring boot:
Eureka:
Probably the first microservice to be UP. Eureka is a service registry, means , it knows which ever microservices are running and in which port. Eureka is deploying as a sperate application and we can use #EnableEurekaServer annotation along with #SpringBootAPplication to make that app a eureka server. So our eureka service registery is UP and running. From now on all microservices will be registered in this eureka server by using #EnableDiscoveryClient annotation along with #SpringBootAPplication in all deployed microservices.
Zuul: ZUUL is a load balancer , routing application and reverse proxy server as well. That is before we were using apache for reverse proxy things , now , for microservices we can use ZUUL. Advantage is, in ZUUL we can programatically set configurations, like if /customer/* comes go to this microservice like that. Also ZUUL can act as a load balancer as well , which will pick the appropriate microservice in a round robin fashion. SO how does the ZUUL knows the details of microservices, the answer is eureka. It will work along with eureka to get microservice details. And in fact this ZUUL is also a Eureka client where we should mark using #EnableDiscoveryClient, thats how these 2 apps(Eureka and zuul) linked.
Ribbbon:
Ribbon use for load balancing. This is already available inside ZUUL, in which zuul is using Ribbon for load balancing stuff. Microservices are identified by service-name in properties file. IF we run 2 instances of one microservices in different port, this will be identified by Eureka and along with Ribbon(Inside zuul), requests will be redirected in a balanced way.
Aws ALB , NGINX , AWS Api gateway etc: There are alternatives for all the above mentioned things. Aws is having own load balancer, service discovery , api gateway etc . Not only AWS all cloud platofrms ,like Azure, have these. Its depends which one to use.
Adding a general question as well , How these microservices communicate each other: Using Resttemplate or Feignclient actual rest API can be called or Message queues like Rabbit MQ etc can be used .
Eureka can be used in conjunction with NGINX, which leads to very powerful combination.
I am using it on AWS EC2 environment. Previously instead of NGINX I was using Spring Cloud Gateway and before that Zuul. Depending of the load Spring Cloud Gateway was running on AWS t3.medium or t3.large instances. After moving to NGINX I am using t3.micro (8 times less memory) instance. I am almost sure that I can do the trick and with t3.nano (16 times less memory) instance, but I wanted to be sure that there will be no surprises.
Below are the high level steps what you have to do in order to plug NGINX in the Eureka ecosystem. More details you can find in NGINX With Eureka Instead of Spring Cloud Gateway or Zuul article.
Create a service which can read the configuration of all applications from Eureka and to 'translate' it to NGINX configuration.
Create a cronjob entry which at certain period will read the configuration from the above service and will call the NGINX hot reload
NGINX which will consume the configuration produced from the service and the cronjob and will work as API Gateway
I'm trying to understand WSO2 APIM components and deployment scenarios but the terminology is confusing/vague for me. Clustering vs distributed deployments, profiles, and Port Offsets.
Basically I'd like to deploy a minimal 5 node setup where:
Node # (Location) Purpose
(DMZ) the GW (worker=True right?) and KeyManager
(DMZ) 2nd GW node (as above) for GW & KeyManager
(non-dmz) the Management Console, MySQL master
(non-dmz) the Publisher UI,TrafficManager, MySQL slave
(DMZ) the Store
Questions:
Should I use -DportOffset=0 on all nodes?
What -Dprofile=?? do I need to use on each of the 5 nodes?
The 2 gateway nodes will be load-balanced by an F5 load balancer
for incoming api-traffic. What port is used there, 9443 or 9763?
What ports need to be accessible on the DMZ hosts for this to work?
I assume 3306,9443,9763,8280,8243,7711, and 9999,11111 if JMX reqd
Please don't point me to the documentation, that's what is confusing me.
Running the key manager nodes, Store node in the DMZ is not recommended as they need db access. If you are using multi tenancy, you cannot host gateway worker nodes in the DMZ as well due to db access. What you can do is host those nodes in LAN and have a reverse proxy in the DMZ to expose the endpoints on the Gateway and Store. If you do not use multi tenancy, then you can run gateway worker nodes in the DMZ as dbs are not used.
As you are running multiple WSO2 servers in a single server you need to use port offsets to avoid conflicts. Default port offset is 0. You can run one WSO2 server with default port offset. For the other server you need to use port offset 1 or any value other than 0. You can start the server by giving the -DportOffset=1 at the startup. Best way is to change the value offset to 1 in /repository/conf/carbon.xml so that you do not need to provide the -DportOffset value at the startup.
-Dprofile is denote the profile which API Manager should start. If you start with -Dprofile=api-publisher, it would only starts the front end/backend features relevant to the API Publisher. Running product profiles are mostly recommended as it would only load relevant features of the profile. You can use profiles in your deployment as you are running 6 profiles of API Manager.
I think you are referring gateway worker nodes which serve API traffic. If so, it will use passthrough ports that are 8280(http) and 8243(https). Requests can serve using both. 9443 and 9763 are servlet ports are those will not used in gateway worker nodes and only in gateway manager node for service calls.
My recommendation is you should revise this setup as you are running nodes in DMZ which have db access.
Should I use -DportOffset=0 on all nodes?
It depends on how do you set up those nodes. If all of these servers in the same node (machine), you must use different port offset as all the API Manager servers use those port, so, there will be port conflicts.
What -Dprofile=?? do I need to use on each of the 5 nodes?
It will adjust the ports used by API Manager so that, there won't be any port conflicts between them if you are running on same node.
The 2 gateway nodes will be load-balanced by an F5 load balancer for
incoming api-traffic. What port is used there, 9443 or 9763?
For API requests/responses handling, you need 9763.
What ports need to be accessible on the DMZ hosts for this to work? I
assume 3306,9443,9763,8280,8243,7711, and 9999,11111 if JMX reqd
Yes, it's correct.
Further, you can use WSO2 support any issues you encountered.
I am using Amazon cloud server (AWS) to create Mule server nodes. Issue with AWS is it doesn't support multicasts, but MuleSoft requires all the nodes are in same network and multicasts enabled for clustering.
Amazon FAQ:
https://aws.amazon.com/vpc/faqs/
Q. Does Amazon VPC support multicast or broadcast?
Ans:No.
Mule cluster doesn't show proper heartbeat without multicasts enabled, mule_ee.log file should show as
Cluster OK
Members [2] {
Member [<IP-Node1>]:5701 this
Member [<IP-Node2>]:5701
}
but my cluster shows as:
Members [1] {
Member [<IP-Node1>]:5701 this
}
which is wrong according to MuleSoft standards. I created a sample Poll scheduler application and deployed in Mule cluster which runs in both nodes due to improper handling of Mule cluster.
But my organization needs AWS to continue with server configuration.
Question
1) is there any other approach instead of using Mule cluster, I can use both Mule server nodes and make it HA cluster configuration(Active-Active).
2) Is it possible to make one server up and running(active) and another one passive mode instead of Mule HA(ACtive-Active) mode?
3) CloudHub and AnypointMQ is deployed in AWS, how did MuleSoft handle multicasts issues with AWS?
According to Mulesoft support team, they don't advise managing Mule HA in AWS , it doesnt matter if we aree managing with ARM or MMC.
The Mule instances communicate with each other and guarantee HA as well as not processing a single request more than once but that does not work on AWS because latency may cause the instances to disconnect from one another. We need to have the servers on-prem to have HA model
Multicast and Unicast are just used for the nodes to be discoverable automatically and further more as explained in the documentation.
Mule cluster config
AWS know limitation: here
We are planning to try out WSO2 Greg and wanted to know if we use external hardware load balancer, do we still need to configure manager + worker node configuration or we can start two standalone instances connecting to same back end i.e. oracle database and load balance it via front end external load balancer?
Thanks
You can use a third-party load balancer for WSO2 GREG cluster. You do not have to use manager/worker configuration for Axis2 clustering (subDomain property) as we use with WSO2 ELB.
You will still have to configure Axis2 clustering without subDomain property. I would recommend you to use wka membership scheme and nominate few nodes as wka members in the cluster.