How to connect to the WSO2 AM database thorugh client? - wso2

I am newbie to WSO2 AM and researching to understand it's good features. It seems that "The default database of user manager is the H2 database that comes with WSO2 products. We can configure it to point to databases by other vendors such as IBM DB2, Oracle, MySQL using the scripts provided by WSO2 for installing and configuring relational databases".
How we can connect WSO2 AM to MySQL DB? What files needs to be
changed? (I'm interested to use MySQL and not H2 db)
How we can use external client to see all tables?
Requesting you to please guide me on this.

Following sample works better with a fresh install(prior to initial start-up) :
Put the Mysql jar connector/driver in wso2am/repository/components/lib directory.
MySql you will need 2 databases :
mysql> create database reg_db;
mysql> create database am_db;
mysql> grant all on reg_db.* to dba#localhost identified by 'dba';
mysql> grant all on am_db.* to dba#localhost identified by 'dba';
mysql> flush privileges;
Overwrite the following 2 data sources in wso2am/repository/conf/datasources/master-datasources.xml
<datasource>
<name>WSO2_CARBON_DB</name>
<description>The datasource used for registry and user manager</description>
<jndiConfig>
<name>jdbc/WSO2CarbonDB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration> <url>jdbc:mysql://localhost:3306/reg_db?autoReconnect=true&relaxAutoCommit=true&</url>
<username>dba</username>
<password>dba</password>
<driverClassName>com.mysql.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
<defaultAutoCommit>false</defaultAutoCommit>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2AM_DB</name>
<description>The datasource used for API Manager database</description>
<jndiConfig>
<name>jdbc/WSO2AM_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration> <url>jdbc:mysql://localhost:3306/am_db?autoReconnect=true&relaxAutoCommit=true&</url>
<username>dba</username>
<password>dba</password>
<driverClassName>com.mysql.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
<defaultAutoCommit>false</defaultAutoCommit>
</configuration>
</definition>
</datasource>
Start the server with wso2server.sh -Dsetup or wso2server.bat -Dsetup

Related

WSO2 enterprise integrator. - datasource on server

It is possible deploy datasource in WSO2 EI on server without GUI? I try find something on internet but i found only create datasource in GUI.
Thanks
You may use Integration Studio for this purpose. Please refer https://docs.wso2.com/display/EI650/Managing+Data+Integration+Artifacts+via+Tooling#ManagingDataIntegrationArtifactsviaTooling-Step2:Creatingthedatasourceconnection.
You can create a carbon application in the tooling and deploy it to the server without using GUI.
You can add your datasource into master-datasources.xml that is located in EI_HOME\conf\datasources\master-datasources.xml
here is the example code:
<datasource>
<name>WSO2_ESB_LOGS_DB</name>
<description/>
<jndiConfig>
<name>jdbc/WSO2EsbLogsDB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:postgresql://localhost:5432/mydb</url>
<username>myusername</username>
<password>mypassword</password>
<driverClassName>org.postgresql.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<defaultAutoCommit>false</defaultAutoCommit>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>

wso2 esb how to connect and query to HBASE

In Wso2 ESB how to connect to HBase and query. As wso2 ESB we have dblookup mediator but it is for sql databases. So is there any solution for connection with HBASE.
You need to first provide the datasource details in wso2 esb, either you can do from front end or backend. i have mentioned the path WSO2_Home/repository/conf/datasources/update below in datasource.xml file.
<datasource>
<name>WSO2_ANALYTICS_RS_DB_HBASE</name>
<description>The datasource used for analytics file system</description>
<jndiConfig>
<name>jdbc/WSO2HBaseDB</name>
</jndiConfig>
<definition type="HBASE">
<configuration>
<property>
<name>hbase.zookeeper.quorum</name>
<value>localhost</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>
<property>
<name>fs.hdfs.impl</name>
<value>org.apache.hadoop.hdfs.DistributedFileSystem</value>
</property>
<property>
<name>fs.file.impl</name>
<value>org.apache.hadoop.fs.LocalFileSystem</value>
</property>
</configuration>
</definition>
</datasource>
After the changes try restarting the server and see if you are able to connect from front end.

WSO2-AM: API calls in store fail after restart

We've setup the most recent versions of
API Manager
Identity Server
Data Analytics Server
in separate pods on a kubernetes cluster (we've used the existing scripts as a basis for creating the docker images), all sharing relevant MySQL databases hosted in another pod on the same cluster. After a lot of configuring everything works as expected, we can add users/roles, create/publish/invoke APIs and collect analytics on these calls. The last issue we're facing however is, that once an API got published, it's possible to invoke it in the store. The problem appears once the API-manager pod gets restarted (for example by scaling the relevant deployment down/up, not by using the carbon dashboard), once the API-Manager is up and running again after such a (hard) restart, the published APIs are still visible in the store, however, if one tries to make a call to one of the (previously working) endpoints, an 404 message is returned and the API-manager logs the following error:
INFO {org.apache.synapse.mediators.builtin.LogMediator} - STATUS =
Message dispatched to the main sequence. Invalid URL, RESOURCE =
/"context"/"endpoint being called"
where "context" is the context set for a given API while publishing it and "endpoint being called" the REST endpoint of the service which should receive the call. Further, if we try to make a change to the API in the publisher (i.e. re-publish it), there seems to be a 50/50 chance that it works and fixes all issues till the next restart of the pod or an error is thrown when clicking the "Next: Implement" button on the first step, which has been recorded here already (although it says that it has been resolved in a previous version already).
The remaining setup is the following:
Users/Roles
User: prod, used for creating and publishing an API in the publisher
Roles: prod-role (empty role with no priviledges), internal/creator,
internal/publisher
User: cons, used for invoking an API in the store
Roles: cons-role (empty role, no priviledges, just for providing
access to the API in the store), internal/subscriber
Databases
WSO2_CARBON_DB referees to a database on the MySQL pod, used only by the API-Manager
WSO2REG_DB is supposed to be the shared registry database between the three modules (am, das, is)
excerpt from master-datasources.xml from the am:
<datasource>
<name>WSO2_CARBON_DB</name>
<description>The datasource used by the registry</description>
<jndiConfig>
<name>jdbc/WSO2CarbonDB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:mysql://mysql-apimdb:3306/amcarbondb?autoReconnect=true&useSSL=false</url>
<username>wso2carbon</username>
<password>abc</password>
<driverClassName>com.mysql.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2AM_DB</name>
<description>The datasource used for API Manager database</description>
<jndiConfig>
<name>jdbc/WSO2AM_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:mysql://mysql-apimdb:3306/apimgtdb?autoReconnect=true&useSSL=false</url>
<username>wso2carbon</username>
<password>abc</password>
<defaultAutoCommit>false</defaultAutoCommit>
<driverClassName>com.mysql.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2UM_DB</name>
<description>The datasource used by user manager</description>
<jndiConfig>
<name>jdbc/WSO2UM_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:mysql://mysql-apimdb:3306/userdb?autoReconnect=true&useSSL=false</url>
<username>wso2carbon</username>
<password>abc</password>
<driverClassName>com.mysql.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2REG_DB</name>
<description>The datasource used for registry</description>
<jndiConfig>
<name>jdbc/WSO2REG_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:mysql://mysql-apimdb:3306/regdb?autoReconnect=true&useSSL=false</url>
<username>wso2carbon</username>
<password>abc</password>
<driverClassName>com.mysql.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
All of these databases get populated upon usage and are initialized using the DBscripts from the different products for the initial setup. Further, the registry.xml is configured as follows (showing just the changes, the remainder of the file is as it is configured by default):
<currentDBConfig>wso2registry</currentDBConfig>
<readOnly>false</readOnly>
<enableCache>false</enableCache>
<registryRoot>/</registryRoot>
<dbConfig name="wso2registry">
<dataSource>jdbc/WSO2CarbonDB</dataSource>
</dbConfig>
<dbConfig name="govregistry">
<dataSource>jdbc/WSO2REG_DB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9943/registry">
<id>gov</id>
<dbConfig>govregistry</dbConfig>
<cacheId>wso2carbon#jdbc:mysql://mysql-apimdb:3306/regdb</cacheId>
<readOnly>false</readOnly>
<enableCache>false</enableCache>
<registryRoot>/</registryRoot>
</remoteInstance>
<mount path="/_system/governance" overwrite="true">
<instanceId>gov</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>
<mount path="/_system/config" overwrite="true">
<instanceId>gov</instanceId>
<targetPath>/_system/config</targetPath>
</mount>
In the end it seems to be an issue with some data being lost upon restart as the memory on the am pod is not persistent, since we've moved all databases to a persistent storage on another pod however, this shouldn't be a problem -- unless we missed something. Is there another registry/data source that need to be added/changed? All other default data sources not shown above (STATS, MB_STORE, METRICS) are setup to use the MySQL server as well, so unless there is something else than the data sources provided in the datasource folder, all data should remain on the corresponding MySQL databases.
I guess you haven't used any persistent storage for API Manager. When you create an API, the API artifact is created in the file system and stored in wso2am-2.1.0/repository/deployment/server/synapse-configs/de‌​fault/api location. You need to have a persistent storage for 'wso2am-2.1.0/repository/deployment/server/'. In your case, during the restart, you don't have relevant data as you have a fresh pack.
this can be fixed by move to persistent volume following folders:
as was suggested:
<WSO2_CARBON_HOME>/repository/deployment/server
but, In case of multitenant installation, need to make persistent also:
<WSO2_CARBON_HOME>/repository/tenants

WSO2 Carbon database setup

Having to troubleshoot a time out issue when calling our WSO2 carbon setup. I know practically nothing about WSO2 but have read that the H2 database supplied is not suitable for production. Although a H2 database is still used for the local registry.
Taking a look at repository\conf\datasources\master-datasources.xml it appears that we are still using the local H2 database for the WSO2_CARBON_DB.
<datasource>
<name>WSO2_CARBON_DB</name>
H2
</datasource>
<datasource>
<name>WSO2AM_DB</name>
SQL Server
</datasource>
<datasource>
<name>WSO2UM_DB</name>
SQL Server
</datasource>
<datasource>
<name>WSO2REG_DB</name>
SQL Server
</datasource>
<datasource>
<name>WSO2AM_STATS_DB</name>
H2
</datasource>
<datasource>
<name>JAGH2</name>
h2
</datasource>
Can anyone tell me if this set up is incorrect and I need to set up another SQL server DB for WSO2_CARBON_DB, and also tell em which one of these is the "local registry"
Thanks
Having a local h2 db for WSO2_CARBON_DB is OK. But in production you need to integrate a more reliable db engine such as mysql/sql server/oracle etc.. for databases other than the WSO2_CARBON_DB

User Management Tables are not shared between WSO2 products

I have followed the this guide on sharing registry between WSO2 products using G-Reg. But the problem is that User Management Tables are not shared.
For example, I have created a new user "test" in G-Reg, but it is not valid in WSO2 ESB product (only native user "admin" is possible).
So is it possible to share users stored in G-Reg? Or are they stored in local H2 databases?
So is it possible to share users stored in G-Red? Or are they stored
in local H2 databases?
Yes, You can share users between all WSO2 products. By default, H2 used which is shipped with all the products.
If you are going to share the user among multiple products, You need to go for a production ready databases like mysql, postgresql, etc.
Create a database named userdb
Add the below entry in repository/conf/datasources/master-datasources.xml
<datasource>
<name>WSO2_CARBON_USER</name>
<description>The datasource used for registry and user manager</description>
<jndiConfig>
<name>jdbc/WSO2CarbonDB_User</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:mysql://localhost:3306/userdb</url>
<username>wso2carbon</username>
<password>wso2carbon</password>
<driverClassName>com.mysql.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
Change the datasource JNDI name in repository/conf/user-mgt.xml file like following in both G-REG and ESB
<Property name="dataSource">jdbc/WSO2CarbonDB_User</Property>
Start the G-REG or ESB with -Dset up option to create the required database.This is required only in the first time.
For this you'll need to configure the products to point to the same user store by configuring it through repository/conf/user-mgt.xml. Have a look at this doc to get an idea how to do it.