We already have Active MQ running on the server. As a result we are getting port conflict when running WSO2. Where can we change the port to avoid this conflict.
So far we have done the following:
1. Searched for references for port 5672 under /repository/conf and changed 5672 to 5673. Files changed included broker and jndi.
Looks like the conflict is resolved but some services are still trying to connect to port 5672. I have not been able to find where to change this.
[2017-02-03 08:37:36,342] INFO - QpidServiceComponent WSO2 Message Broker is started.
[2017-02-03 08:37:36,374] INFO - JMSConnectionFactory JMS ConnectionFactory : Siddhi-JMS-Consumer initialized
[2017-02-03 08:37:36,379] INFO - JMSTransportHandler Starting jms topic consumer thread...
[2017-02-03 08:37:36,800] INFO - PassThroughHttpListener Starting Pass-through HTTP Listener...
[2017-02-03 08:37:36,811] INFO - PassThroughListeningIOReactorManager Pass-through HTTP Listener started on 0:0:0:0:0:0:0:0:8280
[2017-02-03 08:37:36,815] INFO - PassThroughHttpSSLListener Starting Pass-through HTTPS Listener...
[2017-02-03 08:37:36,825] INFO - PassThroughListeningIOReactorManager Pass-through HTTPS Listener started on 0:0:0:0:0:0:0:0:8243
[2017-02-03 08:37:37,669] INFO - AMQConnection Unable to connect to broker at tcp://172.17.51.124:5672
org.wso2.andes.transport.TransportException: Could not open connection
at org.wso2.andes.transport.network.mina.MinaNetworkTransport$IoConnectorCreator.connect(MinaNetworkTransport.java:216)
at org.wso2.andes.transport.network.mina.MinaNetworkTransport.connect(MinaNetworkTransport.java:74)
at org.wso2.andes.client.AMQConnectionDelegate_8_0.makeBrokerConnection(AMQConnectionDelegate_8_0.java:130)
at org.wso2.andes.client.AMQConnection$2.run(AMQConnection.java:631)
at org.wso2.andes.client.AMQConnection$2.run(AMQConnection.java:628)
at java.security.AccessController.doPrivileged(Native Method)
at org.wso2.andes.client.AMQConnection.makeBrokerConnection(AMQConnection.java:628)
at org.wso2.andes.client.AMQConnection.<init>(AMQConnection.java:409)
at org.wso2.andes.client.AMQConnectionFactory.createConnection(AMQConnectionFactory.java:351)
at org.wso2.andes.client.AMQConnectionFactory.createConnection(AMQConnectionFactory.java:40)
at org.wso2.carbon.apimgt.jms.listener.utils.JMSUtils.createConnection(JMSUtils.java:387)
at org.wso2.carbon.apimgt.jms.listener.utils.JMSListener.checkJMSConnection(JMSListener.java:137)
at org.wso2.carbon.apimgt.jms.listener.utils.JMSListener.start(JMSListener.java:79)
at org.wso2.carbon.apimgt.jms.listener.utils.JMSListener.run(JMSListener.java:186)
at java.lang.Thread.run(Thread.java:745)
org.wso2.andes.AMQConnectionFailureException: Could not open connection
at org.wso2.andes.client.AMQConnection.<init>(AMQConnection.java:486)
at org.wso2.andes.client.AMQConnectionFactory.createConnection(AMQConnectionFactory.java:351)
at org.wso2.andes.client.AMQConnectionFactory.createConnection(AMQConnectionFactory.java:40)
at org.wso2.carbon.apimgt.jms.listener.utils.JMSUtils.createConnection(JMSUtils.java:387)
at org.wso2.carbon.apimgt.jms.listener.utils.JMSListener.checkJMSConnection(JMSListener.java:137)
at org.wso2.carbon.apimgt.jms.listener.utils.JMSListener.start(JMSListener.java:79)
at org.wso2.carbon.apimgt.jms.listener.utils.JMSListener.run(JMSListener.java:186)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.wso2.andes.transport.TransportException: Could not open connection
at org.wso2.andes.transport.network.mina.MinaNetworkTransport$IoConnectorCreator.connect(MinaNetworkTransport.java:216)
at org.wso2.andes.transport.network.mina.MinaNetworkTransport.connect(MinaNetworkTransport.java:74)
at org.wso2.andes.client.AMQConnectionDelegate_8_0.makeBrokerConnection(AMQConnectionDelegate_8_0.java:130)
at org.wso2.andes.client.AMQConnection$2.run(AMQConnection.java:631)
at org.wso2.andes.client.AMQConnection$2.run(AMQConnection.java:628)
at java.security.AccessController.doPrivileged(Native Method)
at org.wso2.andes.client.AMQConnection.makeBrokerConnection(AMQConnection.java:628)
at org.wso2.andes.client.AMQConnection.<init>(AMQConnection.java:409)
... 7 more
[2017-02-03 08:37:37,750] ERROR - JMSListener Unable to continue server startup as it seems the JMS Provider is not yet started. Please start the JMS provider now.
[2017-02-03 08:37:37,756] ERROR - JMSListener Connection attempt : 1 for JMS Provider failed. Next retry in 20 seconds
[2017-02-03 08:37:39,373] INFO - CarbonEventManagementService Starting polling event receivers
[2017-02-03 08:37:43,411] INFO - ThriftDataReceiver Thrift Server started at 0.0.0.0
[2017-02-03 08:37:43,451] INFO - ThriftDataReceiver Thrift SSL port : 7711
[2017-02-03 08:37:43,468] INFO - ThriftDataReceiver Thrift port : 7611
We resolved this by using port offsets. No need to manually change ports.
https://docs.wso2.com/display/AM200/Changing+the+Default+Ports+with+Offset
Related
I'm trying to push incoming JSON payload to AWS SQS Queue in WSO2 EI 6.4.0. Facing Exception like java.io.IOException: Target Connection is stale intermittently.
We're unable to push payload to Queue.
Log:
[2022-08-27 03:08:49,801] [-1] [] [HTTPS-Sender I/O dispatcher-5] WARN {org.apache.synapse.transport.passthru.Targe
tHandler} - Connection closed by target host while sending the request Remote Address : proxy.abc.com/
10.0.x.x:3090
[2022-08-27 03:08:49,801] [-1234] [] [PassThroughMessageProcessor-29] ERROR {org.apache.synapse.transport.passthru.P
assThroughHttpSSLSender} - IO while building message
java.io.IOException: Target Connection is stale..
As per this wso2 link, do i need to disable this http.connection.stalecheck by making value as 1 in <ESB_Home>/repository/conf/nhttp.properties file?
Please suggest to resolve this issue.
The error java.io.IOException: Target Connection is stale is not the cause of your issue, it's just a by product of the actual issue. With the information provided I believe the actual issue is the following.
[2022-08-27 03:08:49,801] [-1] [] [HTTPS-Sender I/O dispatcher-5] WARN {org.apache.synapse.transport.passthru.Targe
tHandler} - Connection closed by target host while sending the request Remote Address : proxy.abc.com/
10.0.x.x:3090
Since the connection was closed by the target(Assume this is SQS) the collections are getting staled after some time. This is expected, so try to find out why the target is closing the connection. If you are going through a corporate Proxy, check the proxy first, then check SQS side to see whether there is any information useful to debug the issue.
I try to run Worker and dashbord in same machine.
the first tools is running coorectly, but when i start the second the error has been raised :
[2018-03-07 09:59:43,546] INFO
{org.wso2.msf4j.internal.websocket.EndpointsRegistryImpl} - Endpoint
Registered : /server-stats/{type}
[2018-03-07 09:59:43,636] INFO {org.wso2.carbon.data.provider.DataProviderAPI} - Data Provider
Service Component is activated
[2018-03-07 09:59:44,909] INFO {org.wso2.msf4j.internal.websocket.WebSocketServerSC} - All required
capabilities are available of WebSocket service component is
available.
[2018-03-07 09:59:45,049] INFO {org.wso2.msf4j.internal.MicroservicesServerSC} - All microservices
are available
[2018-03-07 09:59:45,346] INFO {org.wso2.transport.http.netty.listener.ServerConnectorBootstrap$HTTPServerConnector}
- HTTP(S) Interface starting on host 0.0.0.0 and port 9643
[2018-03-07 09:59:45,939] INFO {org.wso2.carbon.metrics.core.config.model.JmxReporterConfig} -
Creating JMX reporter for Metrics with domain
'org.wso2.carbon.metrics'
[2018-03-07 09:59:45,954] INFO {org.wso2.carbon.metrics.core.reporter.impl.AbstractReporter} -
Started JMX reporter for Metrics
[2018-03-07 09:59:45,954] INFO {org.wso2.msf4j.analytics.metrics.MetricsComponent} - Metrics
Component is activated
[2018-03-07 09:59:45,970] INFO {org.wso2.carbon.databridge.agent.internal.DataAgentDS} - Successfully
deployed Agent Server
[2018-03-07 09:59:52,914] ERROR {org.wso2.carbon.kernel.internal.startupresolver.StartupComponentManager}
- Runtime Exception occurred while calling onAllRequiredCapabilitiesAvailable of component
carbon-datasource-service
com.zaxxer.hikari.pool.PoolInitializationException: Exception during
pool initialization: Connection is broken:
"java.net.SocketTimeoutException: connect timed out:
169.254.235.125:59336" [90067-196]
at com.zaxxer.hikari.pool.HikariPool.initializeConnections(HikariPool.java:581)
at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:152)
at com.zaxxer.hikari.HikariDataSource.(HikariDataSource.java:73)
at org.wso2.carbon.datasource.rdbms.hikari.HikariRDBMSDataSource.getDataSource(HikariRDBMSDataSource.java:56)
at org.wso2.carbon.datasource.rdbms.hikari.HikariDataSourceReader.createDataSource(HikariDataSourceReader.java:74)
at org.wso2.carbon.datasource.core.DataSourceBuilder.buildDataSourceObject(DataSourceBuilder.java:79)
at org.wso2.carbon.datasource.core.DataSourceBuilder.buildDataSourceObject(DataSourceBuilder.java:60)
at org.wso2.carbon.datasource.core.DataSourceBuilder.buildCarbonDataSource(DataSourceBuilder.java:44)
at org.wso2.carbon.datasource.core.DataSourceManager.initDataSources(DataSourceManager.java:153)
at org.wso2.carbon.datasource.core.internal.DataSourceListenerComponent.onAllRequiredCapabilitiesAvailable(DataSourceListenerComponent.java:125)
at org.wso2.carbon.kernel.internal.startupresolver.StartupComponentManager.lambda$notifySatisfiableComponents$7(StartupComponentManager.java:266)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at org.wso2.carbon.kernel.internal.startupresolver.StartupComponentManager.notifySatisfiableComponents(StartupComponentManager.java:252)
at org.wso2.carbon.kernel.internal.startupresolver.StartupOrderResolver$1.run(StartupOrderResolver.java:204)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Caused by: org.h2.jdbc.JdbcSQLException: Connection is broken: "java.net.SocketTimeoutException: connect timed out:
169.254.235.125:59336" [90067-196]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:168)
at org.h2.engine.SessionRemote.connectServer(SessionRemote.java:457)
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:367)
at org.h2.jdbc.JdbcConnection.(JdbcConnection.java:116)
at org.h2.jdbc.JdbcConnection.(JdbcConnection.java:100)
at org.h2.Driver.connect(Driver.java:69)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:101)
at com.zaxxer.hikari.pool.HikariPool.addConnection(HikariPool.java:496)
at com.zaxxer.hikari.pool.HikariPool.initializeConnections(HikariPool.java:565)
... 15 more
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.h2.util.NetUtils.createSocket(NetUtils.java:103)
at org.h2.util.NetUtils.createSocket(NetUtils.java:83)
at org.h2.engine.SessionRemote.initTransfer(SessionRemote.java:115)
at org.h2.engine.SessionRemote.connectServer(SessionRemote.java:453)
... 23 more
can you please advise?
Thanks.
Can you share the WSO2 SP version you were using when you got this exception?
Also please check whether the AUTO_SERVER=TRUE
config is avaiable in the jdbc url of WSO2_METRICS_DB datasource configuration, which can be found in
{WSO2_SP_HOME}/conf/worker/deployment.yaml
eg :
jdbcUrl: 'jdbc:h2:${sys:carbon.home}/wso2/dashboard/database/metrics;AUTO_SERVER=TRUE'
I configured all datasource in mysql, and i can running all SP componnent.
the issue is related to H2 datasase, that not allowed to share connection with default configuration.
i will check default H2 connection parametrs, and test again.
finished WSO2 installation and first time I start the service and try to log into the management console it's not accessible (hosted on a Amazon EC2 unix instance).
Based on the log file the server starts normally:
[2017-03-31 22:40:29,622] [EI-Core] INFO - CarbonUIServiceComponent Mgt Console URL : https://XXX.XXX.XXX.XXX:8243/carbon/
When I try logging into mgt console the browser is hang up and won't finish loading the page. On wso2carbon.log I get the log entries below:
[2017-03-31 22:42:51,456] [EI-Core] WARN - EndpointContext Endpoint : AnonymousEndpoint with address https://XXX.XXX.XXX.XXX:37045 will be marked SUSPENDED as it failed
[2017-03-31 22:42:51,457] [EI-Core] WARN - EndpointContext Suspending endpoint : AnonymousEndpoint with address https://XXX.XXX.XXX.XXX:37045 - current suspend duration is : 30000ms - Next retry after : Fri Mar 31 22:43:21 UTC 2017
[2017-03-31 22:43:44,246] [EI-Core] INFO - SourceHandler Writer null when calling informWriterError
[2017-03-31 22:43:44,247] [EI-Core] WARN - SourceHandler Connection time out after request is read: http-incoming-1 Socket Timeout : 180000 Remote Address : YYY.YYY.YYY.YYY/179.212.90.90:4529
[2017-03-31 22:45:51,671] [EI-Core] WARN - ConnectCallback Connection refused or failed for : /XXX.XXX.XXX.XXX:37045
[2017-03-31 22:45:51,672] [EI-Core] WARN - EndpointContext Endpoint : AnonymousEndpoint with address https://XXX.XXX.XXX.XXX:37045 will be marked SUSPENDED as it failed
[2017-03-31 22:45:51,672] [EI-Core] WARN - EndpointContext Suspending endpoint : AnonymousEndpoint with address https://XXX.XXX.XXX.XXX:37045 - last suspend duration was : 30000ms and current suspend duration is : 30000ms - Next retry after : Fri Mar 31 22:46:21 UTC 2017
[2017-03-31 22:46:44,662] [EI-Core] INFO - SourceHandler Writer null when calling informWriterError
[2017-03-31 22:46:44,663] [EI-Core] WARN - SourceHandler Connection time out after request is read: http-incoming-2 Socket Timeout : 180000 Remote Address : 179.212.90.90/179.212.90.90:4592
[2017-03-31 22:48:52,151] [EI-Core] WARN - ConnectCallback Connection refused or failed for : /XXX.XXX.XXX.XXX:37045
[2017-03-31 22:48:52,151] [EI-Core] WARN - EndpointContext Endpoint : AnonymousEndpoint with address https://XXX.XXX.XXX.XXX:37045 will be marked SUSPENDED as it failed
[2017-03-31 22:48:52,151] [EI-Core] WARN - EndpointContext Suspending endpoint : AnonymousEndpoint with address https://XXX.XXX.XXX.XXX:37045 - last suspend duration was : 30000ms and current suspend duration is : 30000ms - Next retry after : Fri Mar 31 22:49:22 UTC 2017
XXX.XXX.XXX.XXX is the local host IP. YYY.YYY.YYY.YYY is my local IP from where I am trying to connect to the Amazon instance.
I am just trying to open mgt console but it seems the ESB is trying to process my request as a regular endpoint request.
The most confusing piece is that I can open the Analytics dashboard (on 9444) and BPEL mgt console (on 9445) that ships inside EI 6.0.0. Only the Enterprise Integrator mgt console accessible under 8243 port is not reachable.
Thanks in advance.
During login into EI console (ESB+DSS+AS console on 8243 port) EI trying to connect some other services. In my cases I've got (in some installations) Name Notification error at this point.
In your case you received timeout. Probably these port (37045) is closed for IP XXX.XXX.XXX.XXX
Try to configure AWS Security Group to open this port
The following ERROR is logged on the gateway worker nodes on start-up.
2016-08-23 12:32:42,344 [-] [Timer-5] ERROR KeyTemplateRetriever Exception when retrieving throttling data from remote endpoint
Unexpected character (<) at position 0.
at org.json.simple.parser.Yylex.yylex(Unknown Source)
at org.json.simple.parser.JSONParser.nextToken(Unknown Source)
at org.json.simple.parser.JSONParser.parse(Unknown Source)
at org.json.simple.parser.JSONParser.parse(Unknown Source)
at org.json.simple.parser.JSONParser.parse(Unknown Source)
at org.wso2.carbon.apimgt.gateway.throttling.util.KeyTemplateRetriever.retrieveKeyTemplateData(KeyTemplateRetriever.java:100)
at org.wso2.carbon.apimgt.gateway.throttling.util.KeyTemplateRetriever.loadKeyTemplatesFromWebService(KeyTemplateRetriever.java:111)
at org.wso2.carbon.apimgt.gateway.throttling.util.KeyTemplateRetriever.run(KeyTemplateRetriever.java:54)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Despite the error the gateway worker nodes startup and the environment can be used to successfully invoke a sample API.
All the apim nodes, bar the traffic manager, however report these warnings
2016-08-22 16:40:56,652 [-] [Timer-5] WARN KeyTemplateRetriever Failed retrieving throttling data from remote endpoint: Connection refused. Retrying after 15 seconds...
2016-08-22 16:40:56,653 [-] [Timer-4] WARN BlockingConditionRetriever Failed retrieving Blocking Conditions from remote endpoint: Connection refused. Retrying after 15 seconds...
Environment:
APIM 2.0.0 cluster
publisher (default profile)
store (default profile)
gw manager and 2 gw workers (default profiles)
traffic manager (using traffic-manager profile)
Database: MariaDB Server, wsrep_25.10.r4144
Userstore : Read/write LDAP
JVM: java version "1.8.0_92"
OS: CentOS Linux release 7.0.1406 (Core)
n.b. key manager un-configured using default pack settings
If you disable Advanced Throttling in api-manager.xml like below, that error will go away. If you enable that, it requires a key manager node.
<EnableAdvanceThrottling>false</EnableAdvanceThrottling>
I encountered the issue recently and the issue was throttle#data#v1.war (repository/deployment/server/webapps/throttle#data#v1.war) hasn't been deployed at the time worker starts up.
If you have a distributed AM 2.0 deployment make sure Keymanager is up and throttle#data#v1.war is deployed in keymanager before worker srartup..
Facing the following issue while trying to connect WSO2 ESB proxy services to queues (newly created) configured in IBM MQ.
The strange part is, able to connect to queues which were already existing in the same Queue Manager, successfully.
TID: [0] [ESB] [2016-01-14 12:05:12,705] INFO {org.apache.axis2.transport.jms.JMSListener} - Connection attempt: 1 for JMS Provider for service: UOFSInterface was successful! {org.apache.axis2.transport.jms.JMSListener}
TID: [0] [ESB] [2016-01-14 12:05:12,712] INFO {org.apache.axis2.transport.jms.ServiceTaskManager} - Task manager for service : UOFSPickedInterface [re-]initialized {org.apache.axis2.transport.jms.ServiceTaskManager}
TID: [0] [ESB] [2016-01-14 12:05:12,829] WARN {org.apache.axis2.transport.jms.JMSUtils} - Cannot locate destination : UOFS.ESB.IN {org.apache.axis2.transport.jms.JMSUtils}
javax.naming.NamingException: Unable to communicate with the queue manager while looking up object UOFS.ESB.IN
at com.ibm.mq.jms.context.MQContext.lookup(MQContext.java:947)
at javax.naming.InitialContext.lookup(InitialContext.java:411)
at org.apache.axis2.transport.jms.JMSUtils.lookup(JMSUtils.java:583)
at org.apache.axis2.transport.jms.JMSUtils.lookupDestination(JMSUtils.java:789)
at org.apache.axis2.transport.jms.ServiceTaskManager.getDestination(ServiceTaskManager.java:950)
at org.apache.axis2.transport.jms.ServiceTaskManager.access$2700(ServiceTaskManager.java:50)
at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.createConsumer(ServiceTaskManager.java:905)
at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.getMessageConsumer(ServiceTaskManager.java:766)
at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.receiveMessage(ServiceTaskManager.java:516)
at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.run(ServiceTaskManager.java:428)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
TID: [0] [ESB] [2016-01-14 12:05:12,833] ERROR {org.apache.axis2.transport.jms.ServiceTaskManager} - Error creating JMS consumer for service : UOFSInterface. JMSWMQ2008: Failed to open MQ queue 'UOFS.ESB.IN'. {org.apache.axis2.transport.jms.ServiceTaskManager}
The MQ client user profile used by the client system (WSO2) didn´t have the appropriate "authorities" like PUT, SET etc., to drop messages into the queue.
The issue was resolved once the profile was given appropriate authorities to PUT, SET & BROWSE on the MQ queue object.