'Read Time Out' between wso2 APIM and APIM-Analytics - wso2

Canario:
APIM and APIM-Analytics (both in 2.6.0) at the same localhost machine.
Identity Server in other Machine
Use the doc to make configuration between APIM and Analytics.
Setup te Datasources for external Oracle DB instance:
IS strat Ok, Analytics Worker start ok, Analytics Dashboard Start Ok, Analytics Manager Start Ok
After default configuration, Apim start with connection issue:
...
ERROR{org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker} -
Error while trying to connect to the endpoint. Cannot borrow client for
ssl://localhost:7712.
{org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker}
org.wso2.carbon.databridge.agent.exception.DataEndpointLoginException:
Cannot borrow client for ssl://localhost:7712.
at
org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.connect(DataEndpointConnectionWorker.java:134)
at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.run(DataEndpointConnectionWorker.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.wso2.carbon.databridge.agent.exception.DataEndpointLoginException: Error while trying to login to the data receiver.
at org.wso2.carbon.databridge.agent.endpoint.thrift.ThriftDataEndpoint.login(ThriftDataEndpoint.java:54)
at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.connect(DataEndpointConnectionWorker.java:128)
... 6 more
Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
at org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:65)
at org.wso2.carbon.databridge.commons.thrift.service.secure.ThriftSecureEventTransmissionService$Client.send_connect(ThriftSecureEventTransmissionService.java:104)
at org.wso2.carbon.databridge.commons.thrift.service.secure.ThriftSecureEventTransmissionService$Client.connect(ThriftSecureEventTransmissionService.java:95)
at org.wso2.carbon.databridge.agent.endpoint.thrift.ThriftDataEndpoint.login(ThriftDataEndpoint.java:47)
... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:750)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:159)
... 11 more
...
When I had access the consoles Analytics (PUBLISHER, STORE or ADMIN), the API Usage analtyics interface become stucked.
I try to mak few changes inside api-manager.xml. Now the ANALYTICS part is lik follows:
<!-- Enable Analytics for API Manager -->
<Enabled>true</Enabled>
<StreamProcessorServerURL>{tcp://localhost:7612}</StreamProcessorServerURL>
<!--StreamProcessorAuthServerURL>{ssl://localhost:7712}</StreamProcessorAuthServerURL-->
<!-- Administrator username to login to the remote StreamProcessor server. -->
<StreamProcessorUsername>admin</StreamProcessorUsername>
<!-- Administrator password to login to the remote StreamProcessor server. -->
<StreamProcessorPassword>admin</StreamProcessorPassword>
<!-- For APIM implemented Statistic client for RDBMS -->
<StatsProviderImpl>org.wso2.carbon.apimgt.usage.client.impl.APIUsageStatisticsRestClientImpl</StatsProviderImpl>
<!-- StreamProcessor REST API configuration -->
<StreamProcessorRestApiURL>https://localhost:9444</StreamProcessorRestApiURL>
<StreamProcessorRestApiUsername>admin</StreamProcessorRestApiUsername>
<StreamProcessorRestApiPassword>admin</StreamProcessorRestApiPassword>
I expect to discovery why this is happen, If i follow de default documentation (https://docs.wso2.com/display/AM260/Configuring+APIM+Analytics)
thanks

This problem it was solved by import the Analytics certificate to wso2carbon.jks and client-truststore.jks.
In the beginning I just import to client-truststore.jks and miss wso2carbon.jks.
It's important to use the full qualified name to create the new certificates and keystore, and use to make correct link between tools at api-manager.xml
Remember to add the full qualified name at hosts file.
Thanks

Related

How to disable ThrottlingConfigurations from deployment.toml in WSO2 3.0.0

I am configuring Apache DS user store in WSO2 3.0.0. All the user store configuration I have given in deployment.toml file. But while starting the server it is giving the below error. After searching that looks like this error is coming as is enable in api-manager.xml, but I am not able to disable it from deployment.toml. Can anyone help me with the configuration I should provide in deployment.toml to make it disabled. Also from the error log it seems like server is also searching for JMS but not enable. Please help me with this also. Error given below:
[2020-01-28 18:39:57,261] WARN - DataEndpointGroup No receiver is reachable at reconnection, will try to reconnect every 30 sec
[2020-01-28 18:39:57,263] INFO - DataBridge user admin connected
[2020-01-28 18:39:57,269] ERROR - AuthenticationServiceImpl Invalid User : admin
[2020-01-28 18:39:57,270] ERROR - DataEndpointConnectionWorker Error while trying to connect to the endpoint. Cannot borrow client for ssl://<local_machine_ip>:9711.
org.wso2.carbon.databridge.agent.exception.DataEndpointLoginException: Cannot borrow client for ssl://<local_machine_ip>:9711.
at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.connect(DataEndpointConnectionWorker.java:145) ~[org.wso2.carbon.databridge.agent_5.2.12.jar:?]
at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.run(DataEndpointConnectionWorker.java:59) [org.wso2.carbon.databridge.agent_5.2.12.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_112]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_112]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: org.wso2.carbon.databridge.agent.exception.DataEndpointLoginException: Error while trying to login to data receiver :/<local_machine_ip>:9711
at org.wso2.carbon.databridge.agent.endpoint.binary.BinaryDataEndpoint.login(BinaryDataEndpoint.java:50) ~[org.wso2.carbon.databridge.agent_5.2.12.jar:?]
at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.connect(DataEndpointConnectionWorker.java:139) ~[org.wso2.carbon.databridge.agent_5.2.12.jar:?]
... 6 more
Caused by: org.wso2.carbon.databridge.commons.exception.AuthenticationException: org.wso2.carbon.identity.authentication.AuthenticationException: Invalid User : admin
at sun.reflect.GeneratedConstructorAccessor269.newInstance(Unknown Source) ~[?:?]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
at org.wso2.carbon.databridge.agent.endpoint.binary.BinaryEventSender.processResponse(BinaryEventSender.java:163) ~[org.wso2.carbon.databridge.agent_5.2.12.jar:?]
at org.wso2.carbon.databridge.agent.endpoint.binary.BinaryDataEndpoint.login(BinaryDataEndpoint.java:44) ~[org.wso2.carbon.databridge.agent_5.2.12.jar:?]
at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.connect(DataEndpointConnectionWorker.java:139) ~[org.wso2.carbon.databridge.agent_5.2.12.jar:?]
... 6 more
Please Note: I am running WSO2 3.0.0 in local windows machine for POC.
Thanks,
Surodip
This is the configuration to disable data publishing to the traffic manager, and the JMS listener.
[apim.throttling]
enable_data_publishing = false
enable_decision_connection = false

Distributed WSO2 APIM: Problems with KeyManager

Now I am testing the API-Manager doing a distributed install of pruduct.
When I start the Analytitcs and publisher (both in ditributed hosts), the analytic's Log donĀ“t stop to show the error messages:
[2018-04-12 15:00:18,770] ERROR {org.wso2.carbon.databridge.core.internal.queue.QueueWorker} - Dropping wrongly formatted event sent for -1234
org.wso2.carbon.databridge.core.exception.EventConversionException: Error when converting loganalyzer:1.0.0 of event bundle with events 1
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:181)
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.toEventList(ThriftEventConverter.java:90)
at org.wso2.carbon.databridge.core.internal.queue.QueueWorker.run(QueueWorker.java:73)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.wso2.carbon.databridge.core.exception.EventConversionException: No StreamDefinition for streamId loganalyzer:1.0.0 present in cache
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:166)
... 7 more
This means the APIM (or other product) is sending events with streamId loganalyzer:1.0.0 , however the analytics server has no such stream definition.
The analytics server is effectively a WSO2 DAS with preconfigured streams and analytics related to some other product. The log messages indicates, that the analytics application (org_wso2_carbon_analytics_apim-1.0.0.car) is not (yet) deployed.
It happens commonly when you start up the analytics server, it receives the product (APIM) events before the analytics app is deployed. Once the analytics app is deployed, the DAS should stop logging these messages
So in your case I'd try to have a look on the analytics server in the start of the log file why the analytics application is not properly deployed

WSO2 EI Analytics Profile Database configuration

I'm configuring WSO2 EI Analytics profile to use PostgreSQL instead of H2 database.
I have changed the following files:
analytics-datasources.xml,
master-datasources.xml,
metrics-datasources.xml
in \wso2\analytics\conf\datasources.
I have, also, executed the scripts to create the database in dbscripts. The scripts generate only tables for metrics and master, but they do not create tables for analytics.
Anyway when I run the analytics server i have some errors as shown below:
Failed to perform Category Drilldown on table: org_wso2_esb_analytics_stream_MediatorStatPerMinute: Error while connecting to the remote service. Connection refused (Connection refused) {JAGGERY.controllers.apis.eianalytics:jag}
TID: [-1234] [] [2017-11-06 16:43:00,262] ERROR {org.wso2.carbon.databridge.core.internal.queue.QueueWorker} - Dropping wrongly formatted event sent for -1234 {org.wso2.carbon.databridge.core.internal.queue.QueueWorker}
org.wso2.carbon.databridge.core.exception.EventConversionException: Error when converting org.wso2.esb.analytics.stream.FlowEntry:1.0.0 of event bundle with events 1
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:181)
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.toEventList(ThriftEventConverter.java:90)
at org.wso2.carbon.databridge.core.internal.queue.QueueWorker.run(QueueWorker.java:73)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.wso2.carbon.databridge.core.exception.EventConversionException: No StreamDefinition for streamId org.wso2.esb.analytics.stream.FlowEntry:1.0.0 present in cache
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:166)
... 7 more
It seems they are missing some database tables, but i don't know how to create them.
These errors are not present when i use H2 database with the default configuration.
Anyone can help me?
I solved the problem.
It was a JDBC driver problem.
With JDK 1.8 it is necessary to use the PostgreSQL JDBC 42.1.4.
I hope it will be useful for someone.

WSO2 DAS server configuration issue - Dropping wrongly formatted event sent for -1234

I have configured DAS with API manager server using REST client, but not able to push data to DAS server. Please see error logs in DAS server. Could you please help me to understand what is wring in configuration?
TID: [-1234] [] [2016-05-20 18:07:05,566] ERROR {org.wso2.carbon.databridge.core.internal.queue.QueueWorker} - Dropping wrongly formatted event sent for -1234 {org.wso2.carbon.databridge.core.internal.queue.QueueWorker}
org.wso2.carbon.databridge.core.exception.EventConversionException: Error when converting org.wso2.apimgt.statistics.throttle:1.0.0 of event bundle with events 1
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:181)
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.toEventList(ThriftEventConverter.java:90)
at org.wso2.carbon.databridge.core.internal.queue.QueueWorker.run(QueueWorker.java:73)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.wso2.carbon.databridge.core.exception.EventConversionException: No StreamDefinition for streamId org.wso2.apimgt.statistics.throttle:1.0.0 present in cache
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:166)
... 7 more
Can you try redeploying the car app again. For that first do following
Delete the .car application from /repository/deployment/server/carbonapps
Delete any existing streams defs (related to APIM stats) by login to DAS management console and going to Manage > Event > Streams
Re deploy car app by putting it in /repository/deployment/server/carbonapps
If everything goes well you would see two scripts in Manage > Batch Analytics > Scripts section. Try to execute each script and see if there is any error.

Username authentification with symmetric key working locally but not in remote server

I have a jax-ws web service developped using netbeans 7 and glassfish 3.1.2 .
I configured the web service to use Username authentication with symmetric keys security and default keystore.
When I test my web service in localhost everything work fine but when I delployed it in remote test server it didnt' work .
First I got exception complaining about Key used to decrypt EncryptedKey cannot be null so I uploaded the local keystore.jks and cacerts.jks to the remote server.
Now I'm gettinh those exceptions:
Server side:
WSITPVD0035: Error in Verifying Security in Inbound Message. com.sun.xml.wss.impl.PolicyViolationException: com.sun.xml.wss.impl.WssSoapFaultException: Invalid Security Header at
com.sun.xml.wss.impl.policy.verifier.MessagePolicyVerifier.verifyPolicy(MessagePolicyVerifier.java:151) at
com.sun.xml.ws.security.opt.impl.incoming.SecurityRecipient.createMessage(SecurityRecipient.java:1003) at
com.sun.xml.ws.security.opt.impl.incoming.SecurityRecipient.validateMessage(SecurityRecipient.java:248) at
com.sun.xml.wss.provider.wsit.WSITServerAuthContext.verifyInboundMessage(WSITServerAuthContext.java:588) at
com.sun.xml.wss.provider.wsit.WSITServerAuthContext.validateRequest(WSITServerAuthContext.java:361) at
com.sun.xml.wss.provider.wsit.WSITServerAuthContext.validateRequest(WSITServerAuthContext.java:264) at
com.sun.enterprise.security.webservices.CommonServerSecurityPipe.processRequest(CommonServerSecurityPipe.java:173) at
com.sun.enterprise.security.webservices.CommonServerSecurityPipe.process(CommonServerSecurityPipe.java:144) at
com.sun.xml.ws.api.pipe.helper.PipeAdapter.processRequest(PipeAdapter.java:119) at
com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:961) at
com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:910) at
com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:873) at
com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:775) at
com.sun.xml.ws.server.WSEndpointImpl$2.process(WSEndpointImpl.java:386) at
com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.handle(HttpAdapter.java:640) at
com.sun.xml.ws.transport.http.HttpAdapter.handle(HttpAdapter.java:263) at
com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(ServletAdapter.java:163) at
org.glassfish.webservices.Ejb3MessageDispatcher.handlePost(Ejb3MessageDispatcher.java:120) at
org.glassfish.webservices.Ejb3MessageDispatcher.invoke(Ejb3MessageDispatcher.java:91) at
org.glassfish.webservices.EjbWebServiceServlet.dispatchToEjbEndpoint(EjbWebServiceServlet.java:200) at
org.glassfish.webservices.EjbWebServiceServlet.service(EjbWebServiceServlet.java:131) at
javax.servlet.http.HttpServlet.service(HttpServlet.java:770) at
com.sun.grizzly.http.servlet.ServletAdapter$FilterChainImpl.doFilter(ServletAdapter.java:1059) at
com.sun.grizzly.http.servlet.ServletAdapter$FilterChainImpl.invokeFilterChain(ServletAdapter.java:999) at
com.sun.grizzly.http.servlet.ServletAdapter.doService(ServletAdapter.java:434) at
com.sun.grizzly.http.servlet.ServletAdapter.service(ServletAdapter.java:384) at
com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:179) at
com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:117) at
com.sun.enterprise.v3.services.impl.ContainerMapper$Hk2DispatcherCallable.call(ContainerMapper.java:354) at
com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195) at
com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:849) at
com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:746) at
com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1045) at
com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:228) at
com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137) at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104) at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90) at
com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79) at
com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54) at
com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59) at
com.sun.grizzly.ContextTask.run(ContextTask.java:71) at
com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532) at
com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513) at
java.lang.Thread.run(Thread.java:662) Caused by: com.sun.xml.wss.impl.WssSoapFaultException: Invalid Security Header at
com.sun.xml.ws.security.opt.impl.util.SOAPUtil.newSOAPFaultException(SOAPUtil.java:159) at
com.sun.xml.wss.impl.policy.verifier.MessagePolicyVerifier.processSecondaryPolicy(MessagePolicyVerifier.java:220) at
com.sun.xml.wss.impl.policy.verifier.MessagePolicyVerifier.verifyPolicy(MessagePolicyVerifier.java:144) ... 43 more
Client side:
Exception in thread "AWT-EventQueue-0" javax.xml.ws.soap.SOAPFaultException: Invalid Security Header
at com.sun.xml.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:193)
at com.sun.xml.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:126)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:123)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:93)
at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:144)
How can i fix this ?
Thanks .
The stack trace does not look familiar to me but maybe your problem is the certificate itself.
When you create a certificate you should set a 'server name'. This will be used by the client to check if the certificate's 'server name' match with the URL 'server name'. If the server names does not match the client should abort the connection because it can be an stolen certificate!!!. For more info, check this tutorial
Example with a java client: if you create a certificate for 'localhost' the client
verification will pass if and only if it access the web service using an URL
like 'https://localhost/stuff...'. So, if you tries to access the same application with the same client but using the IP like 'https://10.0.0.1/stuff...', a verification error should arise.
NOTE: The default configuration of a java web service client do this check, but If you want to bypass this client side verification please check this post
Try to check if this is the problem you are having. If it is the case you have several solutions:
Create a certificate for the server where you are deploying the
application
Disable client side verification
PS: I never did something special on the server-side to solve this kind of issues. Installing the correct certificate should be enough
In my case the problem was in CommandMap. Need to add "application/ciphervalue" handler in CommandMap, because WSIT does it only once while loading CVDataHandler class.
See com.sun.xml.ws.security.opt.impl.util.CVDataHandler static block for more details how to add.