WSO2 Identity Server pattern clustering - wso2

We are in production with WSO2 Identity Server 5.8.0
We configured the system according to the pattern 1 as described here
When we checked the symlink of config and governance we noticed that only config has a symlink icon as showed in this image
Moreover we registered some Service Providers but in the management interface they are not listed although they are present in the table. This seems to be related to the fact that we changed the WSO2 IS admin username after we created the Service Providers and in the table sp_app there is the old username (I can't understand the reason)
Her our configurations about pattern-1. We are using postgreSQL DB.
What we did is the following:
creation of db username_db. Here we executed the following SQL scripts
dbScripts/bps/bpel/create/postgresql.sql
dbScripts/content/postgresql.sql
dbScripts/identity/postgresql.sql
dbScripts/identity/uma/postgresql.sql
dbScripts/metrics/postgresql.sql
creation of db wso2sharedregistry. Here we executed the sql contained in
dbScripts/postgresql.sql
We configigured the following datasource in the master-datasources.xml file
<datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration">
<providers>
<provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider>
</providers>
<datasources>
<datasource>
<name>WSO2_CARBON_DB</name>
<description>The datasource used for registry and user manager</description>
<jndiConfig>
<name>jdbc/usernameDB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:postgresql://xxxxx:yyyy/username_db</url>
<username>username</username>
<password>username</password>
<driverClassName>org.postgresql.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationInterval>30000</validationInterval>
<defaultAutoCommit>true</defaultAutoCommit>
<!--<validationQuery>SELECT 1</validationQuery>-->
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2_SHAREDREGISTRY_DB</name>
<description>The datasource used for registry and user manager</description>
<jndiConfig>
<name>jdbc/WSO2sharedbd</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:postgresql://xxxxx:yyyy/wso2sharedregistry</url>
<username>username</username>
<password>username</password>
<driverClassName>org.postgresql.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationInterval>30000</validationInterval>
<defaultAutoCommit>true</defaultAutoCommit>
<!--<validationQuery>SELECT 1</validationQuery>-->
</configuration>
</definition>
</datasource>
</datasources>
</datasources-configuration>
We configured the registry.xml in this way
<?xml version="1.0" encoding="ISO-8859-1"?>
<wso2registry>
<currentDBConfig>wso2registry</currentDBConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<dbConfig name="wso2registry">
<dataSource>jdbc/WSO2CarbonDB</dataSource>
</dbConfig>
<dbConfig name="wso2sharedregistry">
<dataSource>jdbc/WSO2sharedbd</dataSource>
</dbConfig>
<handler class="org.wso2.carbon.identity.entitlement.policy.finder.registry.RegistryPolicyHandler">
<filter class="org.wso2.carbon.identity.entitlement.policy.finder.registry.RegistryPolicyMediaTypeMatcher">
<property name="mediaType">application/xacml-policy+xml</property>
</filter>
</handler>
<remoteInstance url="https://localhost:9443/registry">
<id>instanceid</id>
<dbConfig>wso2sharedregistry</dbConfig>
<readOnly>false</readOnly>
<enableCache>false</enableCache>
<registryRoot>/</registryRoot>
</remoteInstance>
<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>
<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/nodes</targetPath>
</mount>
<indexingConfiguration>
<startIndexing>false</startIndexing>
<startingDelayInSeconds>35</startingDelayInSeconds>
<indexingFrequencyInSeconds>5</indexingFrequencyInSeconds>
<!--number of resources submit for given indexing thread -->
<batchSize>40</batchSize>
<!--number of worker threads for indexing -->
<indexerPoolSize>40</indexerPoolSize>
<!-- location storing the time the indexing took place-->
<lastAccessTimeLocation>/_system/local/repository/components/org.wso2.carbon.registry/indexing/lastaccesstime</lastAccessTimeLocation>
<!-- the indexers that implement the indexer interface for a relevant media type/(s) -->
<indexers>
<indexer class="org.wso2.carbon.registry.indexing.indexer.MSExcelIndexer" mediaTypeRegEx="application/vnd.ms-excel"/>
<indexer class="org.wso2.carbon.registry.indexing.indexer.MSPowerpointIndexer" mediaTypeRegEx="application/vnd.ms-powerpoint"/>
<indexer class="org.wso2.carbon.registry.indexing.indexer.MSWordIndexer" mediaTypeRegEx="application/msword"/>
<indexer class="org.wso2.carbon.registry.indexing.indexer.PDFIndexer" mediaTypeRegEx="application/pdf"/>
<indexer class="org.wso2.carbon.registry.indexing.indexer.XMLIndexer" mediaTypeRegEx="application/xml"/>
<indexer class="org.wso2.carbon.registry.indexing.indexer.XMLIndexer" mediaTypeRegEx="application/(.)+\+xml"/>
<indexer class="org.wso2.carbon.registry.indexing.indexer.PlainTextIndexer" mediaTypeRegEx="application/swagger\+json"/>
<indexer class="org.wso2.carbon.registry.indexing.indexer.PlainTextIndexer" mediaTypeRegEx="application/(.)+\+json"/>
<indexer class="org.wso2.carbon.registry.indexing.indexer.PlainTextIndexer" mediaTypeRegEx="text/(.)+"/>
<indexer class="org.wso2.carbon.registry.indexing.indexer.PlainTextIndexer" mediaTypeRegEx="application/x-javascript"/>
</indexers>
<exclusions>
<exclusion pathRegEx="/_system/config/repository/dashboards/gadgets/swfobject1-5/.*[.]html"/>
<exclusion pathRegEx="/_system/local/repository/components/org[.]wso2[.]carbon[.]registry/mount/.*"/>
</exclusions>
</indexingConfiguration>
<versionResourcesOnChange>false</versionResourcesOnChange>
<!-- NOTE: You can edit the options under "StaticConfiguration" only before the
startup. -->
<staticConfiguration>
<versioningProperties>true</versioningProperties>
<versioningComments>true</versioningComments>
<versioningTags>true</versioningTags>
<versioningRatings>true</versioningRatings>
</staticConfiguration>
</wso2registry>
Any tips in how we can:
show the previously registered Service Providers
create the correct registry symlink

Related

Stackdriver Logging | Missing TraceId and SpanId

I am trying to integrate SpringBootApplication(microservice) with StackdriverLogging by using spring-cloud-gcp-starter-logging. I am able to see the logs in GCP but in the logs traceId and SpanId is missing.For this I tried to use Spring-cloud-sleuth also, but as I am using apache kafka in my microservice sleuth is not working properly.
Can anyone help me how Can I add traceId and SpanId information in logs??
POM.xml Configuration :
<spring-cloud.version>Hoxton.SR1</spring-cloud.version>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-logging</artifactId>
</dependency>
logback.xml :
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/cloud/gcp/autoconfigure/logging/logback-json-appender.xml" />
<appender name="CONSOLE_JSON" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="org.springframework.cloud.gcp.logging.StackdriverJsonLayout">
<includeTraceId>true</includeTraceId>
<includeSpanId>true</includeSpanId>
<includeThreadName>false</includeThreadName>
</layout>
</encoder>
</appender>
<logger name="org.apache.kafka" level="warn">
<appender-ref ref="CONSOLE_JSON" />
</logger>
<root level="INFO">
<appender-ref ref="CONSOLE_JSON" />
</root>
</configuration>
By default, Sleuth doesn’t send the trace data for every single request. You probably don’t want to trace every single request either. The trace sampling rate can be adjusted by configuring the spring.sleuth.sampler.percentage property. For more details please refer to "Spring Boot and Spring Cloud Sleuth" in this link and have a look to this video.

Replicate Secondary user store configuration in WSO2 Identity Server cluster

I have deployed WSO2 IS 5.1 cluster with 2 nodes. When I add secondary user store to one of the nodes, the configuration is stored on the filesystem of that node. The other node does not display any details related to the added secondary userstore. How do I configure such that all filesystem based configuration gets replicated across nodes in the cluster?
Do I need to use registry based depSync for this? I have tried to setup jdbc based registry for shared config and governance but I still do not see secondary userstore configuration replicating.
My registry.xml:
<wso2registry>
<currentDBConfig>wso2registry</currentDBConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<dbConfig name="wso2registry">
<dataSource>jdbc/WSO2LocalRegistry</dataSource>
</dbConfig>
<dbConfig name="sharedregistry">
<dataSource>jdbc/WSO2RegistryDB</dataSource>
</dbConfig>
<handler class="org.wso2.carbon.identity.entitlement.policy.finder.registry.RegistryPolicyHandler">
<filter class="org.wso2.carbon.identity.entitlement.policy.finder.registry.RegistryPolicyMediaTypeMatcher">
<property name="mediaType">application/xacml-policy+xml</property>
</filter>
</handler>
<remoteInstance url="https://localhost:9443/registry">
<id>instanceid</id>
<dbConfig>sharedregistry</dbConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/config</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>
<versionResourcesOnChange>false</versionResourcesOnChange>
<staticConfiguration>
<versioningProperties>true</versioningProperties>
<versioningComments>true</versioningComments>
<versioningTags>true</versioningTags>
<versioningRatings>true</versioningRatings>
</staticConfiguration>
</wso2registry>
In carbon.xml of each node:
<!-- Deployment Synchronizer Configuration. Uncomment the following section when running with "registry based" dep sync.
In master nodes you need to set both AutoCommit and AutoCheckout to true
and in worker nodes set only AutoCheckout to true.
-->
<DeploymentSynchronizer>
<Enabled>true</Enabled>
<AutoCommit>true</AutoCommit>
<AutoCheckout>true</AutoCheckout>
</DeploymentSynchronizer>
You need to sync files in the <IS_HOME>/repository/deployment/server/ folder (<IS_HOME>/repository/tenants/ folder if you are having multiple tenants) in a proper IS cluster.
IS have inbuilt mechanism to do that called deployment synchroniser. You can refer [1] on exact details to set that up.
[1] https://docs.wso2.com/display/Cluster/SVN-based+Deployment+Synchronizer
Regards,

Jetty MDC handler doesn't provide contextPath

Jetty 9.3.8.v20160314 running on AWS EC2 Linux machine with Java 1.8.0_51 has two web apps under two different dir+context.xml - webapps/app1 and webapps/app2 with an empty webapps/ROOT directory.
I'm trying to get each webapp to have a separate log with contextPath in its name.
Each app has the following webapps/appX.xml context defined -
<Configure class="org.eclipse.jetty.webapp.WebAppContext">
<Set name="contextPath">/appX</Set>
<Set name="war">
<SystemProperty name="jetty.home" default="."/>/webapps/appX
</Set>
</Configure>
Followed Example: Centralized Logging with Logback and installed webapp-logging module, app is using slf4j Logger and resources/logback.xml is -
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss} %t %c{0} [%p] %m%n</pattern>
</encoder>
</appender>
<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator>
<key>contextPath</key>
<defaultValue>unknown</defaultValue>
</discriminator>
<sift>
<appender name="FILE-${contextPath}" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${jetty.home}/logs/${contextPath}.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>${contextPath}_%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- keep 30 days' worth of history -->
<maxHistory>30</maxHistory>
<totalSizeCap>1GB</totalSizeCap>
</rollingPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss} %t %c{0} [%p] %m%n</pattern>
</encoder>
<append>true</append>
</appender>
</sift>
</appender>
<root level="INFO">
<appender-ref ref="STDOUT"/>
<appender-ref ref="SIFT"/>
</root>
</configuration>
But it all goes to unknown.log. Using servername instead works well and creates separate file for each domain being used to access the server.
So it would seem that contextPath isn't properly assigned by ContextLogHandler.
Why is it empty?

Why am I unable to view stats in the WSO2 API Manager Store?

Everyone please note that this question has been updated as I've recently had logs to work with. Below the log is the original posting with my configs, and the behavior I was seeing.
Today, WSO2 APIM continues to send to BAM receivers which update the Stats database. However when I click on any of the statistics link in the publisher I get:
TID: [0] [AM] [2014-03-04 13:43:18,815] ERROR {org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject} - Error while invoking APIUsageStatisticsClient for ProviderAPIUsage {org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject}
org.wso2.carbon.apimgt.usage.client.exception.APIMgtUsageQueryServiceClientException: Error occurred while querying from JDBC database
at org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.queryFirstAccess(APIUsageStatisticsClient.java:1747)
at org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.getFirstAccessTime(APIUsageStatisticsClient.java:1675)
at org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject.jsFunction_getFirstAccessTime(APIProviderHostObject.java:2911)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)
at org.mozilla.javascript.FunctionObject.call(FunctionObject.java:386)
at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42)
at org.jaggeryjs.rhino.publisher.modules.statistics.c1._c_getFirstAccessTime_13(/publisher/modules/statistics/usage.jag:351)
at org.jaggeryjs.rhino.publisher.modules.statistics.c1.call(/publisher/modules/statistics/usage.jag)
at org.mozilla.javascript.ScriptRuntime.applyOrCall(ScriptRuntime.java:2430)
at org.mozilla.javascript.BaseFunction.execIdCall(BaseFunction.java:269)
at org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:97)
at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42)
at org.jaggeryjs.rhino.publisher.modules.statistics.c0._c_anonymous_13(/publisher/modules/statistics/module.jag:29)
at org.jaggeryjs.rhino.publisher.modules.statistics.c0.call(/publisher/modules/statistics/module.jag)
at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0._c_anonymous_1(/publisher/site/blocks/stats/ajax/stats.jag:220)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.call(/publisher/site/blocks/stats/ajax/stats.jag)
at org.mozilla.javascript.optimizer.OptRuntime.call0(OptRuntime.java:23)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0._c_script_0(/publisher/site/blocks/stats/ajax/stats.jag:4)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.call(/publisher/site/blocks/stats/ajax/stats.jag)
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.call(/publisher/site/blocks/stats/ajax/stats.jag)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.exec(/publisher/site/blocks/stats/ajax/stats.jag)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:570)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.exec(RhinoEngine.java:273)
at org.jaggeryjs.jaggery.core.manager.WebAppManager.execute(WebAppManager.java:432)
at org.jaggeryjs.jaggery.core.JaggeryServlet.doPost(JaggeryServlet.java:29)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:379)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339)
at org.jaggeryjs.jaggery.core.JaggeryFilter.doFilter(JaggeryFilter.java:21)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.sql.SQLException: Incorrect syntax near 'limit'.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:372)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2988)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2421)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:671)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQLQuery(JtdsStatement.java:505)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeQuery(JtdsStatement.java:1427)
at org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.queryFirstAccess(APIUsageStatisticsClient.java:1729)
... 63 more
TID: [0] [AM] [2014-03-04 13:43:18,836] ERROR {JAGGERY.modules.statistics.usage:jag} - java.lang.NullPointerException: null {JAGGERY.modules.statistics.usage:jag}
I have configured WSo2 API Manager 1.6.0 and BAM 2.4.0 to both use the same datasource configuration for WSO2AM_STATS_DB.
For the store it is a single WSO2AM_STATS_DB entry that matches the entry on the BAM server (below) and API-Manager.xml is updated to:
<APIUsageTracking>
<Enabled>true</Enabled>
<PublisherClass>org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher</PublisherClass>
<ThriftPort>7612</ThriftPort>
<BAMServerURL>tcp://myBAMserver:7612/</BAMServerURL>
<BAMUsername>user</BAMUsername>
<BAMPassword>pwd</BAMPassword>
<DataSourceName>jdbc/WSO2AM_STATS_DB</DataSourceName>
<GoogleAnalyticsTracking>
<Enabled>false</Enabled>
<TrackingID>UA-XXXXXXXX-X</TrackingID>
</GoogleAnalyticsTracking>
</APIUsageTracking>
and now the datasource for APIM:
<datasource>
<name>WSO2AM_STATS_DB</name>
<description>The datasource used for getting statistics to API Manager</description>
<jndiConfig>
<name>jdbc/WSO2AM_STATS_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<defaultAutoCommit>false</defaultAutoCommit>
<url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_apiStatsdb</url>
<username>wso2storeuser</username>
<password>storepwd</password>
<driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
For the BAM server here are my datasources:
<datasource>
<name>WSO2_CARBON_DB</name>
<description>The datasource used for API Manager database</description>
<jndiConfig>
<name>jdbc/WSO2CarbonDB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<defaultAutoCommit>false</defaultAutoCommit>
<url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_carbondb</url>
<username>user</username>
<password>pwd</password>
<driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2AM_STATS_DB</name>
<description>The datasource used for getting statistics to API Manager</description>
<jndiConfig>
<name>jdbc/WSO2AM_STATS_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<defaultAutoCommit>false</defaultAutoCommit>
<url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_apiStatsdb</url>
<username>user</username>
<password>pwd</password>
<driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2BAM_DATASOURCE</name>
<description>The datasource used for analyzer data</description>
<definition type="RDBMS">
<configuration>
<defaultAutoCommit>false</defaultAutoCommit>
<url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_apiStatsdb</url>
<username>user</username>
<password>pwd</password>
<driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2BAM_CASSANDRA_DATASOURCE</name>
<description>The datasource used for Cassandra data</description>
<definition type="RDBMS">
<configuration>
<url>jdbc:cassandra://localhost:9161/EVENT_KS</url>
<username>admin</username>
<password>admin</password>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2BAM_UTIL_DATASOURCE</name>
<description>The datasource used for BAM utilities, such as message store etc..</description>
<definition type="RDBMS">
<configuration>
<url>jdbc:cassandra://localhost:9161/BAM_UTIL_KS</url>
<username>admin</username>
<password>admin</password>
</configuration>
</definition>
</datasource>
<!-- The URL configs are loaded from cassandra-component.xml -->
<datasource>
<name>WSO2BAM_HIVE_INCREMENTAL_DATASOURCE</name>
<definition type="RDBMS">
<configuration>
<username>admin</username>
<password>admin</password>
<dataSourceProps>
<property name="replicationFactor">1</property>
<property name="strategyClass">org.apache.cassandra.locator.SimpleStrategy</property>
<property name="readConsistencyLevel">QUORUM</property>
<property name="writeConsistencyLevel">QUORUM</property>
<property name="keyspaceName">HIVE_INCREMENTAL_KS</property>
</dataSourceProps>
</configuration>
</definition>
</datasource>
When I look in mydbserver/wso2_apiStatsdb I see that the following tables were created and have been updated with data:
API_REQUEST_SUMMARY, API_Resource_USAGE_SUMMARY
update: some additional tables have been added and updated...
API_FAULT_SUMMARY, API_RESPONSE_SUMMARY, API_VERSION_USAGE_SUMMARY
However when I go to the publisher page I get:
As far as I can tell I've updated the publisher/store and gateway/key manager to send data to the BAM server. The BAM server appears to process and put that summarized data in the sql server database. My entries for SQL server database match for the publisher/store config and the BAM server config. Does anyone have any ideas what I'm missing that will enable me to see the stats when in the publisher?
Can u please check if the EnableBillingAndUsage is set to false. This is available in conf/api-manager.xml
<EnableBillingAndUsage>false</EnableBillingAndUsage>
Gammonster, this has been figured out. It appears that the devil is indeed in the details - the runtime was trying to execute a sql query using the 'limit' keyword in a SQL Server database and SQL Server does not support 'limit'. It has 'top' instead.
Would have appreciated if the product documentation expressly identified SQL Server to not be a preferred DB instance for BAM.
Are you running the BAM on port offset 1? As far as I can see you have BAM running on a different server. Try running BAM with port offset 1. The default config provided in the site is for a usecase where API Manager and BAM runs on the same server with port offsets (API M -0 and BAM -1). Hence all the relevant thrift ports are also incremented by 1 in the default config. Hence I suggest you to run BAM on offset 1.
In the meantime pls check if the MQSQL DB that has summarized stats contain any data inside them.
This can also happen if you haven't configured the summarized database correctly in API Manager. From what you have mentioned, API Manager seems to publish data correctly, but it's not fetching summarized data. Please see if you have defined WSO2AM_STATS_DB in master-datasources.xml inside APIM directory.

Avoid use of deprecated methods in org.eclipse.jetty.server.ssl.SslSocketConnector with jetty-maven-plugin

Releated to
Deprecated Embedded Jetty 7.6 using SSL
Jetty SslConnector's deprecated methods
How to write correct Maven POM without using the deprecated methods / tags in connector like needClientAuth or keystore?
Example with deprecated method use:
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<!-- see http://docs.codehaus.org/display/JETTY/Maven+Jetty+Plugin -->
<version>8.0.4.v20111024</version>
<!-- see http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.mortbay.jetty%22%20AND%20a%3A%22jetty-maven-plugin%22 -->
<dependencies>
<!--[...]-->
</dependencies>
<configuration>
<webAppXml>src/main/resources/jetty-Login.xml</webAppXml>
<scanIntervalSeconds>5</scanIntervalSeconds>
<webAppConfig>
<contextPath>/MyApp</contextPath>
</webAppConfig>
<connectors>
<connector implementation="org.eclipse.jetty.server.bio.SocketConnector">
<port>8080</port>
</connector>
<connector implementation="org.eclipse.jetty.server.ssl.SslSocketConnector">
<port>8443</port>
<password>changeit</password>
<wantClientAuth>true</wantClientAuth><!-- deprecated! -->
<needClientAuth>false</needClientAuth><!-- deprecated! -->
<keystore>/my/path/to/java/keystore</keystore><!-- deprecated! -->
</connector>
</connectors>
</configuration>
</plugin>
</plugins>
A custom Ssl configuration via the maven configuration structure isn't possible.
This is due to the introduction of the SslContextFactory requirement on the SslSocketConnector constructor to tighten up some SSL security issues on the server side.
Maven can only construct objects from the default constructor when using the structures in the pom.xml.
You'll have to bridge the change via the <jettyXml> element.
Go grab a copy of the jetty-ssl.xml from the distribution and put it into your ${project.basedir}/src/main/config/jetty-ssl.xml and use the following configuration block.
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>${jetty-version}</version>
<configuration>
<scanIntervalSeconds>5</scanIntervalSeconds>
<webAppConfig>
<contextPath>/MyApp</contextPath>
</webAppConfig>
<jettyXml>src/main/config/jetty-ssl.xml</jettyXml>
<connectors>
<connector implementation="org.eclipse.jetty.server.bio.SocketConnector">
<port>8080</port>
</connector>
</connectors>
</configuration>
</plugin>