I'm performing WSO2 API manager + Analytics 2.0 POC now. When i change datasource from H2 to Oracle, in wso2am-2.0.1-SNAPSHOT, there are 2 data source config files:
master-datasources.xml & metrics-datasources.xml, according Installing and configuring the databases, there should be WSO2AM_DB, WSO2UM_DB and the WSO2REG_DB datasource configurations, but i just find WSO2_CARBON_DB & WSO2AM_DB, so my questions are
Is WSO2_CARBON_DB = WSO2UM_DB + WSO2REG_DB?
for WSO2_METRICS_DB, according Enabling Metrics and Storage Types, if we enable JDBC storage, can we store all components metrics information in one shared db or it needs one db per component(local)?
What's WSO2_MB_STORE_DB used for? from the scripts, it's for Message Store and Andes Context Store. Can we keep to use H2 in prod. cluster env.?
When i config wso2am-analytics-2.0.0-SNAPSHOT, i have below questions:
Can we share WSO2_CARBON_DB setting for both APIMGRT related components and analytics? or it's better to not share?
For WSO2AM_STATS_DB, is analytics resposible to aggregate and write to it, APIMGRT responsible to read? Which APIMGRT components need to read it?
For analytics related store, it supports RDBMS, Cassandra, HBase, but it does not support mongodb, right?
for GEO_LOCATION_DATA, What's this used for? Can we just use H2 in prod. env.?
APIM:
1) In default pack, yes. But in a production environment, it is recommended to separate them as WSO2_CARBON_DB, WSO2UM_DB and WSO2REG_DB (Please note you need WSO2_CARBON_DB too, to store local data. And this can be an h2 database)
2) You can have a shared DB
3) WSO2_MB_STORE_DB is required only if you use Advanced Throttling. Tables for this are created by APIM itself. So you don't need to run any scripts on it.
APIM Analytics:
1) You can share WSO2UM_DB and WSO2REG_DB. But don't share (local) WSO2_CARBON_DB.
2) Store and Publisher
3) See WSO2 DAS with MongoDB
4) GEO_LOCATION_DATA is used for Geolocation Based Statistics. H2 is not recommended.
Related
I am installing a WSO2 EI two nodes cluster and I have some doubts about the number of databases that have to be created.
I will try to explain my situation as best that I can: I am using WSO2 EI 6.5.0.
I have two nodes cluster. Each node contains the ESB and the MB profile. I will create the needed databases on a Microsoft SQL Server cluster DB.
So I have this situation:
NODE 1: ESB AND MB
NODE 2: ESB AND MB
Reading on the official documentation:
CLUSTERING WSO2 ESB PROFILE: https://docs.wso2.com/display/EI650/Clustered+Deployment
CLUSTERING WSO2 MB PROFILE: https://docs.wso2.com/display/EI650/Clustering+the+Message+Broker+Profile
I can found information of the DBs needed by ESB and MB profiles.
In particular ESB profile need the creation of these 4 databases:
WSO2_USER_DB: JDBC user store and authorization manager.
REGISTRY_DB: Shared database for config and governance registry mounts in the product's nodes.
REGISTRY_LOCAL1: Local registry space in Node 1.
REGISTRY_LOCAL2: Local registry space in Node 2.
while MB profile need the following 5 databases:
WSO2_USER_DB: JDBC user store and authorization manager.
REGISTRY_DB: Shared database for config and governance registry mounts in the product's nodes.
REGISTRY_LOCAL1: Local registry space in Node 1.
REGISTRY_LOCAL2: Local registry space in Node 2.
MB_DB: Stores instance data that are specific to the message broker profile.
So it appears that the first 4 databases are the same between the ESB and MB profiles and that the MB profiles uses an addictional database named MB_DB.
And here my doubt: in a situation like the one that I have (2 nodes cluster that both contains ESB and MB profiles) how many databases have do I have to create?
My original idea was 5 databases because I thought that the first 4 databases are shared between ESB and MB profiles but now I have the following doubt: in theory (correct me if I am doing wrong assertion) both ESB and MB may use same user database and registry but checking the default H2 setup we have that:
ESB profile use one database placed in /usr/lib/wso2/wso2ei/6.5.0/repository/database/
MB profile use one database placed in /usr/lib/wso2/wso2ei/6.5.0/wso2/broker/repository/database/
So in theory it should mean that it is better to do not mix databases between profile/products.
So what exactly I should do? How many and what databases I have to create?
I am now thinking that maybe I have to completly separate the ESB and MB profiles databases using prefix, something like this:
ESB profile databases: I have to create these 4 databases:
ESB_WSO2_USER_DB
ESB_REGISTRY_DB
ESB_REGISTRY_LOCAL1
ESB_REGISTRY_LOCAL2
MB profiles databases: I have to create these 5 databases;
MB_WSO2_USER_DB
MB_REGISTRY_DB
MB_REGISTRY_LOCAL1
MB_REGISTRY_LOCAL2
WSO2_MB (specific for MB profile so I am not using a reduntant suffix)
Could this be a good solution? But in this way I will have completlely separate dbs and not shared information for users and registry.
What is the best practice in a case like a two nodes cluster like mine?
Thank you
As you said, it is fine to keep separate databases. And what usually does is creating databases with a prefix as you mentioned.
However, let's say if its only one set of users that will be accessing both ESB and MB, then you can go ahead and point a single userdb from the both products (means all the four instances running in 2 nodes).
Regarding the registry database, let say if you have a configuration file that needs to be access from both ESB and MB, then you have to share a single registry_db (and also have to follow the proper registry mounting / see the docs for that).
If there is no such case, it is best to keep separate databases.
Bottom line: As long as there is nothing to be shared between ESB and MB internally, it is best to maintain a separate set of databases. You only need to follow clustering and share databases only between the instances of the same product.
Cheers!
I am trying to run a stored procedure in WSO2 CEP with SQL Server 2016. Is this possible? I already looked through the documentation on this which is scarce. there is only one sample with connection to SQL Server.
Although, it looks like it is limited to selects, inserts and deletes.
I was not able to find any documentation on other driver functionality through CEP.
Also, I did find some information on other WSO2 Services, not wso2 CEP, which might allow me to run the stored procedures. If you have any information on that and who to plug CEP with those services It would help as well.
WSO2CEP is a real time analytics product which provides few SQL operations with RDBMS publisher to publish events as mentioned in the documentation [1].
Only work around would be modifying existing query defined output-event-adapters.xml file located in <CEP-Location>/repository/conf direcory. Inside the file locate <adapterConfig type="rdbms"> element and you will find properties of SQL operations such as update into like below.
<property key="updateTableRow">UPDATE $TABLE_NAME SET $COLUMN_VALUES WHERE $CONDITION</property>
You can change above like below to execute stored procedure
<property key="updateTableRow">EXEC $TABLE_NAME $COLUMN_VALUES</property>
Then in RDBMS publisher you need to change execution.mode as update.
[1] https://docs.wso2.com/display/CEP400/RDBMS+Event+Publisher
In wso2am/repository/conf/datasources/*.xml, I can see 5 datasources :
<name>WSO2_CARBON_DB</name>
<name>WSO2AM_DB</name>
<name>WSO2AM_STATS_DB</name>
<name>WSO2_MB_STORE_DB</name>
<name>WSO2_METRICS_DB</name>
But in wso2am/dbscripts, I can find only 4 scripts for 4 databases (no script for WSO2AM_STATS_DB).
Is WSO2AM_DB supposed to stay an H2 database in production ? or should it point to an existing database ?
You don't need to create tables in WSO2AM_STATS_DB manually. Just creating a database is enough. Tables are automatically created by analytics scripts.
Table creation of the statistics database is handled by the Analytics
scripts when you configure APIM Analytics, so you will create the
statistics database in this step but will not specify a source script.
Ref: https://docs.wso2.com/display/CLUSTER44x/Clustering+API+Manager+2.0.0
I have configured API Manager 2.0.0 & API Manager Analytics Pack to use MySQL databases.
For each server, there exists a WSO2AM_STATS_DB. I have given these differing names on my MySQL server. I have also pointed my datasources in master-datasources.xml(for APIM) & stats-datasources.xml(for Analytics) to the relevant databases.
I couldn't find any relevant schema(dbscripts) for these databases in their respective packs.
On running, the Analytics database is populated but the APIM database isn't and throws an exception. The Analytics database not only gets the schema but also the invocation details of my API.
I am unable to get the stats on my dashboard though.
Previously, I (unwittingly) configured the h2-repository stats database to be the same for both servers (due to the folder structure) and was able to get all the statistics on my dashboard in the publisher.
Other configurations I have tried :
On the MySQL Server, pointed it to the same database (the Analytics one with the schema) but with no results on my dashboard (after waiting for a while).
Both datasources (WSO2AM_STATS_DB) in 2 servers should be pointed to the same database. There are no database scripts for this. Tables are created automatically.
By default in both servers, Stats DB path comes like this. (note ../ part)
<url>jdbc:h2:../tmpStatDB/WSO2AM_STATS_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000;AUTO_SERVER=TRUE</url>
So if you extract both servers to the same directory as mentioned in this doc, both datasources will be pointing to the same database (inside tmpStatDB) like this.
/parent_dir
|__wso2am-2.0.0/
|__wso2am-analytics-2.0.0/
|__tmpStatDB/
So, what happens here is, wso2am-analytics writes stats data to shared database, then apim reads it and shows data on its databases.
I'm able to configure WSO2 BAM data source WSO2_CARBON_DB to work with Oracle DB, but I'm not able to do the same with other data sources.
Is it possible to disable Cassandra and make WSO2 BAM works only with Oracle DB, including all stored data (configuration / input data / analyzed data and so on)?
For the stat store, we use cassandra. It gives high read/write performance than RDBMS. You can not configure RDBMS instead of Cassandra.
For other DB related operations you can use any RDBMS( mssql/mysql..).
Eg: registry/user store