I would like to have a cluster of gateway in api manager. I saw the documents how to split the 4 components with mysql, I saw also the document explaining how to cluster the api gateway (Mgt / Worker) but I miss some information: Is it necessary that both clusters connect to the same database or each cluster can/must have is own database?
A image is better than words:
So what I would like to do is to have 1 KeyManager, 1 store and 1 publisher and 2 Gateways. SO when I publish 1 endpoint it is available on both gateways (I will use host file the specify hostnames instead of ip). I don't know if it is possible that kind of configuration.
Another possibility is to have a shared keyManager between both wso2 servers. When the user has his authorization key he can use both wso2 servers.
You should share databases among clusters. Basically there are 3 databases used. UserDB, registryDB, APImanager DB.
If you want to allow same users to login/publish APIs (for example) you must share UserDB among clusters. Same thing applies to other DBs too.
When you cluster the gateway you should consider following
if your are using muti-tenancy
then point all gateway nodes should be pointed to usermgt db and registry db
if no muti-tenancy
all gaetway nodes should be pointed to registry db(because there you will have tier.xml ,so when adding custom tier.xml it should be synced with all gateway nodes)
gateway doesn't use apimgt db , but you need to have the config of apimgt db which points to default h2 db,other wise you will get error when start the gateway node.
Related
I'm using aws OpenSearch in a private vpc.
I've about 10000 entries under some index.
For local development i'm running an local OpeanSearch container and i'd like to export all the entries from the OpenSearch service into my local container.
I can get all the entries from the OpeanSerch API but the format of the response is different then the format that should be when doing _bulk operation.
Can someone please tell me how should i do it?
Anna,
There are different strategies you can take to accomplish this, considering the fact that your domain is running in a private VPC.
Option 1: Exporting and Importing Snapshots
From the security standpoint, this is the recommended option, as you are moving entire indices out of the service without exposing the data. Please follow the AWS official documentation about how to create custom index snapshots. Once you complete the steps, you will have an index snapshot stored on an Amazon S3 bucket. After this, you can securely download the index snapshot to your local machine, then follow the instructions on the official OpenSearch documentation about how to restore the index snapshots.
Option 2: Using VPC Endpoints
Another way for you to export the data from your OpenSearch domain is accessing the data via a alternate endpoint using the VPC Endpoints feature from AWS OpenSearch. It allows you to to expose additional endpoints running on public or private subnets within the same VPC, different VPC, or different AWS accounts. In this case, you are essentially create a venue to access the OpenSearch REST APIs outside of the private VPC, to which you need to take care of who other than you will be able to do so as well. Please follow the best practices related to secure endpoints if you follow this option.
Option 3: Using the ElasticDump Open Source Utility
The ElasticDump utility allows you to retrieve data from Elasticsearch/OpenSearch clusters in a format of your preference, and then import that data back to another cluster. It is a very flexible way for you to move data around—but it requires the utility to access the REST API endpoints from the cluster. Run this utility in a bastion server that has ingress access to your OpenSearch domain in the private VPC. Keep in mind, though, that AWS doesn't provide any support to this utility, and you must use it at your own risk.
I hope that helps with your question. Let us know if you need any more help on this. đŸ™‚
Suppose we have a kubernetes stack running on AWS and we would like to config our component directly via kubernetes chart files to be able to provision some AWS services (for example a DynamoDB table).
What would be the best practice if we want to achieve this and hopefully also allow our kubernetes component can connect to the provisioned services via IAM way (not just using simple key secret pairs)
So once achieve that, one can let a kubernetes component has a char file such as:
apiVersion: alpha1/v1
Kind: Database
type: dynamodb
table_name: a_test_table
hash_key: some_id
range_key: some_order
Then when deploying this component onto Kubernetes stacks, it will provision a DynamoDB table with name a_test_table for this service to be successfully connected onto.
What you are describing here is more or less what the AWS Service Broker is trying to achieve. Using the Open Service Broker API, the AWS broker would react to the creation of specific kubernetes objects, creating the required resources on AWS. Here you can find an AWS post explaining how this works.
There are different brokers for different cloud providers, not just AWS.
Try for example on Pivotal the Elefant SQL or Redis services: You can create multiple service keys with different names, but they contain the same secrets.
I would have thought that in creating different keys I would be able to revoke them independently. Is that only possible for some services?
The behavior depends on the service broker implementation. The recommended approach is to generate unique values for every binding and service key. Not all service brokers do that though.
For reference, see the open service broker API here -> https://github.com/openservicebrokerapi/servicebroker/blob/v2.13/spec.md#credentials
I know, for example, that the Pivotal MySQL Service Broker follows this advice and generates unique credentials for each binding and service key.
https://github.com/cloudfoundry/cf-mysql-broker
Hope that helps!
In the multi-tenanted environment (create and work with tenants), Registry Database is required to share the information in this database between the Gateway and Key Manager components.
So in the above line Which information need to be shared and where it need to be shared?
There is no special DB configurations for Multi-tenancy, Tenant related information will be stored in same DBs. And when you are clustering APIM you need to share databases among the nodes irrespective of tenant usage. You can refer this guide for clustering.
What is basically says is you need to do step 8.c and 11 in Installing and configuring the databases section in API Manager clustering guide for gateway and keymanager nodes as well.
I have two instances of wso2 api manager running on two different servers.Both of them are referring to same UM_DB . I created a role by logging with admin credentials on one server .After that i checked for the role on other server by logging with admin credentials again.I found that there was role existing on other server but permission that i provided for that role does not exist on another server.Is that a bug with wso2 api manager or I missed something in configuration..?
You want to deploy two APIM instances in a cluster. It is better to refer the APIM clustering guide to setup it properly. There are tow things you need to understand.. when your deploying APIM in cluster
You must point both instance in to same database. There are can be three logical databases i.e UM, Registry and AM database. These three can be an one physical DB. However must pointed to same by the both instance.
You must configure the Hazelcast based clustering using axis2.xml file. This is required because, APIM uses Hazelcast based implementation to distribute the data in the caches. Sometime, In your scenario, i guess you have not configured this. Therefore permission tree has not been distributed between two nodes. Therefore lot of data that is stored in the caches for high performance. therefore please make sure to configure this properly.
I guess this would help you.