Has anyone successfully created AS and ESB cluster together?
I am trying to create test cluster with 1 ELB fronting 3 ESB nodes (1 mgt & 2 workers) and 3 AS nodes (1 mgt & 2 workers).
I followed 4.1.0 cluster creation guide here http://docs.wso2.org/display/Cluster/Clustering+WSO2+Products and had ESB cluster working successfully.
However when I try to add AS cluster to LB, following instructions on above link it, whole thing fell-a-part. My suspicion is on proxyPort mapping for AS.
I was wondering if any one had successfully created cluster setup as above and can share the over all steps.
I have resolved my issue by adding new axis2 transports on ELB.
My set up as follows: ELB ports 8280 and 8243 going ESB clusters. ELB ports 80 and 443 going to AS cluster.
How ELB loadbalancer.conf works is not clear. Hope great people of WOS2 improve ELB documentation.
Related
I built a new cluster with Terraform for a AWS EKS, single node group with a single node.
This cluster is using 1.22 and cant seem to get anything to work correctly.
So Istio will install fine, i have installed versions 1.12.1, 1.13.2, 1.13.3 & 1.13.4 and all seem to have the same issue with auto injecting the sidecar.
Error creating: Internal error occurred: failed calling webhook "namespace.sidecar-injector.istio.io": failed to call webhook: Post "https://istiod.istio-system.svc:443/inject?timeout=10s": context deadline exceeded
But there are also other issues with the cluster, even without using Istio. My application is pulled and the pod will build fine but can not connect to the database. This is an external DB to the cluster - no other build (running on Azure) have any issues connecting to the DB
I am not sure if this is the issue with the application not connecting to the ext. DB but the sidecar issue could have something to do with BoundServiceAccountTokenVolume?
There is a warming about it being enabled on all clusters from 1.21 - a little odd as i have another applications with istio, running on another cluster with 1.21 on AWS EKS!
I also have this application running with istio without any issues in Azure on 1.22
I seem to have fix it :)
It seems to be a port issue with the security groups. I was letting terraform build its own group.
When I opened all the ports up in the 'inbound' section it seemed to work.
I then closed them all again and only opened 80 and 443 - which again stopped Istio from auto-injecting its sidecar
My app was requesting to talk to Istio on port 15017, so i opened just that port, along sided ports 80 and 443.
Once that port was opened, my app started to work and got the sidecar from Istio without any issue.
So it seems like the security group stops pod-to-pod communication... unless i have completely messed up my terraform build in some way
This question already has answers here:
Does Kafka support ELB in front of broker cluster?
(2 answers)
Closed 2 years ago.
I have 3 node kafka cluster (zookeeper also installed on the same 3 nodes). Im not sure if I deploy the AWS NLB in front of my broker. I have 3 producers, even though if the evenly go to all 3 brokers, they will decide where to partition it and etc. I don't know what benefit I can get from AWS NLB and what is the cons of it.
I also researched this and didn't find much help out there. I ended up putting a NLB, with a TCP target group, in front of my broker and this is why:
Save some DNS headache. I have a CNAME on the NLB A record and that's what I use for my bootstrap server value. I can scale horizontally seamlessly by just adding the new broker to the NLB target group (via Cloudformation). I'm not tied down to any IPs in our AWS environment now because of DNS records. I also use a Route53 private zone for the Zookeeper nodes so the brokers only point to the overall A record that all those nodes share.
Easy to monitor broker health with built in CW monitoring.
I read about the benefit of SSL offloading with an ELB, but I don't really consider that a benefit because the client to broker comms will still be non-SSL. I'm not doing that, but I thought I'd list it.
I haven't done any benchmark testing with the NLB yet, but I'm not too concerned. IMO, the simplified DNS made it worth it.
Cheers
EDIT: Proxy protocol will not work with Kafka, so if you want the ability to restrict traffic via source IP, in your security groups, you'll have to use type 'instance' vs type 'ip' for your NLB target group targets.
https://aws.amazon.com/premiumsupport/knowledge-center/security-group-load-balancer/
A lesson learned using the NLB name from a target:
https://aws.amazon.com/premiumsupport/knowledge-center/target-connection-fails-load-balancer/
For this issue, I just switched my --bootstrap-server to 'localhost' on any broker target.
I'm trying to understand WSO2 APIM components and deployment scenarios but the terminology is confusing/vague for me. Clustering vs distributed deployments, profiles, and Port Offsets.
Basically I'd like to deploy a minimal 5 node setup where:
Node # (Location) Purpose
(DMZ) the GW (worker=True right?) and KeyManager
(DMZ) 2nd GW node (as above) for GW & KeyManager
(non-dmz) the Management Console, MySQL master
(non-dmz) the Publisher UI,TrafficManager, MySQL slave
(DMZ) the Store
Questions:
Should I use -DportOffset=0 on all nodes?
What -Dprofile=?? do I need to use on each of the 5 nodes?
The 2 gateway nodes will be load-balanced by an F5 load balancer
for incoming api-traffic. What port is used there, 9443 or 9763?
What ports need to be accessible on the DMZ hosts for this to work?
I assume 3306,9443,9763,8280,8243,7711, and 9999,11111 if JMX reqd
Please don't point me to the documentation, that's what is confusing me.
Running the key manager nodes, Store node in the DMZ is not recommended as they need db access. If you are using multi tenancy, you cannot host gateway worker nodes in the DMZ as well due to db access. What you can do is host those nodes in LAN and have a reverse proxy in the DMZ to expose the endpoints on the Gateway and Store. If you do not use multi tenancy, then you can run gateway worker nodes in the DMZ as dbs are not used.
As you are running multiple WSO2 servers in a single server you need to use port offsets to avoid conflicts. Default port offset is 0. You can run one WSO2 server with default port offset. For the other server you need to use port offset 1 or any value other than 0. You can start the server by giving the -DportOffset=1 at the startup. Best way is to change the value offset to 1 in /repository/conf/carbon.xml so that you do not need to provide the -DportOffset value at the startup.
-Dprofile is denote the profile which API Manager should start. If you start with -Dprofile=api-publisher, it would only starts the front end/backend features relevant to the API Publisher. Running product profiles are mostly recommended as it would only load relevant features of the profile. You can use profiles in your deployment as you are running 6 profiles of API Manager.
I think you are referring gateway worker nodes which serve API traffic. If so, it will use passthrough ports that are 8280(http) and 8243(https). Requests can serve using both. 9443 and 9763 are servlet ports are those will not used in gateway worker nodes and only in gateway manager node for service calls.
My recommendation is you should revise this setup as you are running nodes in DMZ which have db access.
Should I use -DportOffset=0 on all nodes?
It depends on how do you set up those nodes. If all of these servers in the same node (machine), you must use different port offset as all the API Manager servers use those port, so, there will be port conflicts.
What -Dprofile=?? do I need to use on each of the 5 nodes?
It will adjust the ports used by API Manager so that, there won't be any port conflicts between them if you are running on same node.
The 2 gateway nodes will be load-balanced by an F5 load balancer for
incoming api-traffic. What port is used there, 9443 or 9763?
For API requests/responses handling, you need 9763.
What ports need to be accessible on the DMZ hosts for this to work? I
assume 3306,9443,9763,8280,8243,7711, and 9999,11111 if JMX reqd
Yes, it's correct.
Further, you can use WSO2 support any issues you encountered.
Here is the issue I am encountering.
I am trying to deploy the WSO2 API Manager which is open source.
Can find the documenation on how to do this here:
https://github.com/wso2/kubernetes-artifacts/tree/master/wso2am
Dockerfiles:
https://github.com/wso2/dockerfiles/tree/master/wso2am
What I did was take the build the docker images which is required for kuberenetes.
I than take these docker images and deploy them to EC2 Container Service.
I than update the wso2 kuberenetes spec files (controllers) to use the image I pushed to EC2 Container Service.
I then go into kubernetes:
kubernetes-artifacts/wso2am and run "./deploy -d"
It than runs the wait for launch script but it just keeps looping and never "finds" that it is up.
root#aw-kubernetes:~/wso2kubernetes/kubernetes-artifacts/wso2am# ./deploy.sh -d
Deploying MySQL Governance DB Service...
service "mysql-govdb" created
Deploying MySQL Governance DB Replication Controller...
replicationcontroller "mysql-govdb" created
Deploying MySQL User DB Service...
service "mysql-userdb" created
Deploying MySQL User DB Replication Controller...
replicationcontroller "mysql-userdb" created
Deploying APIM database Service...
service "mysql-apim-db" created
Deploying APIM database Replication Controller...
replicationcontroller "mysql-apim-db" created
Deploying wso2am api-key-manager Service...
You have exposed your service on an external port on all nodes in your
cluster. If you want to expose this service to the external internet, you may
need to set up firewall rules for the service port(s) (tcp:32013,tcp:32014,tcp:32015) to serve traffic.
See http://releases.k8s.io/release-1.3/docs/user-guide/services-firewalls.md for more details.
service "wso2am-api-key-manager" created
Deploying wso2am api-store Service...
You have exposed your service on an external port on all nodes in your
cluster. If you want to expose this service to the external internet, you may
need to set up firewall rules for the service port(s) (tcp:32018,tcp:32019) to serve traffic.
See http://releases.k8s.io/release-1.3/docs/user-guide/services-firewalls.md for more details.
service "wso2am-api-store" created
Deploying wso2am api-publisher Service...
You have exposed your service on an external port on all nodes in your
cluster. If you want to expose this service to the external internet, you may
need to set up firewall rules for the service port(s) (tcp:32016,tcp:32017) to serve traffic.
See http://releases.k8s.io/release-1.3/docs/user-guide/services-firewalls.md for more details.
service "wso2am-api-publisher" created
Deploying wso2am gateway-manager Service...
You have exposed your service on an external port on all nodes in your
cluster. If you want to expose this service to the external internet, you may
need to set up firewall rules for the service port(s) (tcp:32005,tcp:32006,tcp:32007,tcp:32008) to serve traffic.
See http://releases.k8s.io/release-1.3/docs/user-guide/services-firewalls.md for more details.
service "wso2am-gateway-manager" created
Deploying wso2am api-key-manager Replication Controller...
replicationcontroller "wso2am-api-key-manager" created
Waiting wso2am to launch on http://172.20.0.30:32013
.......
I tried to comment out the "/wait-until-server-starts.sh" script and have it just start everything. But still not able to access the API Manager.
Could really use some insight on this as I am completely stuck.
I have tried everything I can think of.
If anyone on the WSO2 team or that has done this could help out it would really be appreciated.
My theory right now is maybe this was never tested deploying this to AWS but only to a local setup? but I could be wrong.
Any help would be greatly appreciated!
EDIT:
Adding some outputs from kubectl logs etc while it is in the loop waiting for server to come up I see these things:
root#aw-kubernetes:~# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-apim-db-b6b0u 1/1 Running 0 11m
mysql-govdb-0b0ud 1/1 Running 0 11m
mysql-userdb-fimc6 1/1 Running 0 11m
wso2am-api-key-manager-0pse8 1/1 Running 0 11m
Also doing a kubectl logs shows that everything started properly:
[2016-07-21 18:46:59,049] INFO - StartupFinalizerServiceComponent Server : WSO2 API Manager-1.10.0
[2016-07-21 18:46:59,049] INFO - StartupFinalizerServiceComponent WSO2 Carbon started in 34 sec
[2016-07-21 18:46:59,262] INFO - CarbonUIServiceComponent Mgt Console URL : https://wso2am-api-key-manager:32014/carbon/
[2016-07-21 18:46:59,262] INFO - CarbonUIServiceComponent API Publisher Default Context : http://wso2am-api-key-manager:32014/publisher
[2016-07-21 18:46:59,263] INFO - CarbonUIServiceComponent API Store Default Context : http://wso2am-api-key-manager:32014/store
#Alex This was an issue in WSO2 Kubernetes Artifacts v1.0.0 release. We have fixed this in the master branch [1].
The problem was that the deployment process was trying to verify WSO2 API-M server sockets using private IP addresses of the Kubernetes nodes. We updated the scripts to use the public/external IP address if they are available via the Kubernetes CLI. For this to work, you may need to setup Kubernetes on AWS according to [2].
[1] https://github.com/wso2/kubernetes-artifacts/commit/53cc6979965ebed8800b803bb3454f3b758b8c05
[2] http://kubernetes.io/docs/getting-started-guides/aws/
How to check if WSO2 ELB is working properly?
I have a ELB and 2 ELB(1 manager and 1 worker) running, I want to check if ELB is doing its work or not.
I want to check it using a SOAP request, SOAP endpoint should point to ELB or ESB?
I have configured ELB according to what is there in WSO2's documentation.
Thanks.
The WSO2 Elastic Load Balancer has been discontinued. You can download NGinx Plus [1] - the load balancer by NGinx - for which we provide support.
If you are currently using WSO2 ELB and need guidance, please visit our documentation page, Spacially Auto-Scaling in Load Balancer
In order to set up the WSO2 Elastic Load Balancer with one manager and one worker please refer document [1]
In order to check if WSO2 ELB is working properly, you can check it with autoscaling facilities in WSO2 ELB.
Please refer to document [2] for more information on autoscaling.
If you need to send a request to the ESB first you need to point it to ELB.
[1] https://www.nginx.com/resources/admin-guide/
[2] http://blog.afkham.org/2011/09/how-to-setup-wso2-elastic-load-balancer.html