Do you have any examples of how to cluster docker containers for wso2 api manager and DAS? I currently have a set up where I have separated and running each APIM product profile (manager,worker,store,keymanager and publisher) in separate docker containers. I will be particularly interested in information on how to scale services (for ex. how to add more workers, more stores to the cluster) for high availability and load balancing.
What is the container management system you are using? If its k8s 1 you can refer to below documentations.
Please follow the below documentation which will help you to get an idea on this. In k8s you can achieve the scalability trough increasing the number of pods.
[1] http://kubernetes.io/docs/whatisk8s/
[2] https://docs.wso2.com/display/KA100/Clustered+Deployment+of+WSO2+Products+on+Kubernetes
[3] https://docs.wso2.com/display/CLUSTER44x/Clustering+API+Manager+1.10.0
[4] https://github.com/wso2/dockerfiles/tree/v1.2.0/wso2am
Related
Now I am working with our company product developed with spring boot , angular and PostgreSQL technologies where front end angular is communicating with 138 back end ReST API end points. And these 138 end points are from 35 different spring boot project. And all these end points need to separately deploy for 5 different tenant. Actually end point working is same.But databases are different for different tenant. And we decided to go with AWS cloud. And we are looking for cost effective deployment method from AWS.
Our Current Development/Test strategy - Current we are developing application(final stage of development) and testing our application using our On-premise server. Here we are using 5 ubuntu machines. And we created kubernetes cluster with 2 master nodes and 3 worker nodes.And from our SVN repository and Jenkins server we implemented CI/CD pipeline deployment to this 5 machines.
Proposed Cloud Solution - Now we are thinking with to use either EKS deployment method or any of CodeDeploy/CodePipeline method to implement this big project.
So by considering cost and control over infrastructure management which solution is better for my product? Now I am not that much experienced as solution architect and still in cloud learning curve. So can any one suggest/guide me to think properly to achieve my goal please?
Company consideration
Control over infrastructure
Cost effective
Easy management of aws services for multi-tenant deployment
Data security ( Installing database on ec2/ RDS)
Management of load balances
Control over infrastructure
it would be better to manage it on Github, Gitlab, and or AWS code build, or cloud build.
indeed AWS code build, and repo is great tools but again consider the limitation of extra users it allows only 5 users if your team is very big you might have to pay to compare to managing projects at the Github & GitLab level.
Cost effective
EKS would be a good option compared to ECS or others as it has limitations of we can not run the Daemon set or Privilege PODs.
If you are looking for running everything On POD and auto-scalable with little less flexibility and don't want to manage much ECS also a good idea, but again you have to derive the capacity and compare both pricing ECS vs EKS.
Note : EKS will also charge the per hour charges $0.10 for each cluster + worker nodes. it's not just worker nodes like in on-prem we run.
Data security ( Installing database on ec2/ RDS)
RDS would be better as it's managed service compare to managing the EC2 and database performance and encryption etc.
it would be better to use RDS and EKS so the K8s service can connect to RDS easily on a private network.
RDS would be a cost-effective option considering the management of DB over EC2.
Management of load balances
NLB or ALB will take care of that you can use any of them as per the requirement with EKS.
Cloud front could be also a great option with cloud storage to serve static assets, which will reduce calls, improve performance and be cost-effective also.
I have to create and configure a two node WSO2 EI cluster. In particular I have to cluster an ESB profile and MB profile.
I have some architectural doubts about this:
CLUSTERING ESB PROFILE DOUBTS:
I based my assumptions on this documentation: https://docs.wso2.com/display/EI640/Clustering+the+ESB+Profile
I found this section:
Note that some production environments do not support multicast.
However, if your environment supports multicast, there are no issues
in using this as your membership scheme
What could be the reason for not supporting multicast? (so I can inform about possible issues with it). Looking into the table (inside the previous link) it seems to me that possible problem could be related to the following points:
All nodes should be in the same subnet
All nodes should be in the same multicast domain
Multicasting should not be blocked
Is obtaining this information from system\network engineers enough to decide whether to proceed with the multicast option?
Using multicast instead of WKA, would I need to do the same configuration steps listed in the first deployment scenario (the WKA based one) related to the "mounting registry" and "creating\connecting to databases" (as shown in the first documentation link)?
Does using Multicast instead of WKA allow me to not stop the service when I add a new node to the cluster?
CLUSTERING MB PROFILE:
From what I understand, MB profile cluster can use only WKA as membership scheme.
Does using WKA mean that I have to stop the service when I add a new node to the cluster?
So at the end can we consider the ESB cluster and the MB cluster two different clusters? Does the ESB cluster (if it is configured using multicast) need the service to be stopped when a new node is added while the MB cluster is stopped to add a new one?
Many virtual private cloud networks, including Google Cloud Platform,
Microsoft Azure, Amazon Web Services, and the public Internet do not
support multicast. Because such a platform does not support multicast.
If you configure wso2 products with multicast as the membership shceam it will not work as expected. That is the main reason for the warning in the official documentation.
You can consider the platform capability and chose any of the following membership schemes when configuring Hazalcast clustering in WSO2 Products.
WKA
Multicast
AWS
Kubernetes
Other than WKA the rest of the options for membership schema does not require you to include all the IPs of the member's in the configuration. So newly introduced nodes can join the cluster with ease.
Even in the WKA membership scheme if you have at least one known member active you can join a new member to the cluster then follow the configuration change and restart the other services without any service interruption.
Please note with all the above membership scheme usages the rest of
the configurations related to each product are needed to successfully
complete the cluster.
Regarding your concern about Clustering the MB Profile,
You can use any of the above-mentioned membership schemas which matches your deployment environment.
Regarding the adding new members to WKA, You can maintain service availability and apply the changes to servers one by one. You only need at least one WKA member running to introduce a new member to the cluster.
WSO2 MB Profile introduces cluster coordination through an RDBMS. With this new feature by default, cluster coordination is not handled by hazelcast engine. When the cluster coordination through an RDBMS is dissabled is allow the hazelcast engine to manage cluster coordination
Please note when the RDMS coordination is used there are no server restarts required.
I hope this was helpfull.
I have gone through the cloudbreak documentation and I am still not sure what is the exact purpose of this component.
Is it actually useful only for deploying the cluster in any cloud services and if so can we customise the components that needs to be installed in the cluster.
If it is only for maintaining the deployment of a cluster then is there any cost involved in using cloudbreak?
Cloudbreak main purpose is Hdp or Hdf cluster management. It provides an UI and api to access, create and edit the cluster. It also provides access control management for the clusters. Yes you can customize the components installation via ambari blueprint.
One additional benefit was from its component periscope, which provides autoscaling based on ambari alerts.
I have an Active-Active Deployment of WSO2 API Manager. I don't know if I should enable Hazelcast Clustering, because:
A) On one hand, in the link of official documentation that I followed to deploy, Hazelcast doesn't appear.
B) On the other hand, this link of official documentation says that backend throttling limits will not be shared across the cluster when Hazelcast clustering is disabled (and I of course want that backend throttling limits are shared across the cluster!). But that link is under section "Distributed Deployment", and I haven't a "Distributed Deployment". As I said, I have a "Active-Active Deployment", so I don't know if I should follow that link and install Hazelcast.
If you need backend throttling, then you have to enable clustering in the nodes. Although it is mentioned under distributed deployment, for Active-Active deployment also needs clustering if you require backend service throttling.
The idea here is that two nodes serve the requests while they are in a cluster and enable backend service throttling.
if I should follow that link and install Hazelcast
You don't need to install anything, just enable the clustering and setup the IP addresses if wka membership scheme is used (please not many cloud providers or native docker don't support multicast)
The hazelcast cluster is used to broadcast the token invalidation messages and throttling limits. You don't need to enable the cluster at all, but then you may miss the messages between nodes.
I wanted to get List of containers and their details running on GCP Kubernets,
From API page https://developers.google.com/apis-explorer/#p/container/v1/
We could get Cluster and Node details, but I'm looking for more granular levels like Pods and Container.
Is there any way to get those.
Pod and container details are accessible through the kubernetes API, not using the Google Cloud SDK.
Unfortunately, this means getting that information on a per cluster basis.
Alternatively, the Cloud Console can be used by going to Kubernetes Engine > Workloads which will list all replica sets and individual pods (if not controlled by a replicaset).
You might be able to configure Stackdriver Monitoring to create a group consisting of all the pod and container resources in use in the project and then use the Monitoring API to call on that group, but I haven't tested that.