how to share an secondary user store in multi tenant? - wso2-identity-server

I define 2 tenant in wso2 identity server 5.1, for example tenant TA, tenant TB.
Then login with TA and add an userStore in TA and login with TB and add same userstore in TB, that i want an userstore share in each two tenant. when IS is started and login with each tenant, when i want fetch user list, add user or any operation on user i get this error:
[2016-09-14 09:24:03,675] rasa-novin.admin#rayanen.com#rasa-novin.tenant [2] [IS]DEBUG {org.wso2.carbon.user.core.common.UserStoreDeploymentManager} - The deployment of enbcas_local.xml is not valid.
org.wso2.carbon.user.core.UserStoreException: Could not initialize new user store manager : enbcas.local Duplicate domain names not allowed.
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.callSecure(AbstractUserStoreManager.java:158)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.addSecondaryUserStoreManager(AbstractUserStoreManager.java:4078)
at org.wso2.carbon.user.core.common.UserStoreDeploymentManager.deploy(UserStoreDeploymentManager.java:68)
at org.wso2.carbon.identity.user.store.configuration.deployer.UserStoreConfigurationDeployer.deploy(UserStoreConfigurationDeployer.java:167)
at org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136)
at org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:807)
at org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:144)
at org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377)
at org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254)
at org.apache.axis2.deployment.DeploymentEngine.loadServices(DeploymentEngine.java:135)
at org.wso2.carbon.core.multitenancy.TenantAxisConfigurator.deployServices(TenantAxisConfigurator.java:599)
at org.wso2.carbon.core.multitenancy.utils.TenantAxisUtils.createTenantConfigurationContext(TenantAxisUtils.java:369)
at org.wso2.carbon.core.multitenancy.utils.TenantAxisUtils.getTenantConfigurationContext(TenantAxisUtils.java:148)
at org.wso2.carbon.core.services.util.CarbonAuthenticationUtil.onSuccessAdminLogin(CarbonAuthenticationUtil.java:134)
at org.wso2.carbon.core.services.authentication.AuthenticationAdmin.login(AuthenticationAdmin.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
at org.apache.axis2.rpc.receivers.RPCMessageReceiver.invokeBusinessLogic(RPCMessageReceiver.java:117)
at org.apache.axis2.receivers.AbstractInOutMessageReceiver.invokeBusinessLogic(AbstractInOutMessageReceiver.java:40)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.local.LocalTransportReceiver.processMessage(LocalTransportReceiver.java:169)
at org.apache.axis2.transport.local.LocalTransportReceiver.processMessage(LocalTransportReceiver.java:82)
at org.wso2.carbon.core.transports.local.CarbonLocalTransportSender.finalizeSendWithToAddress(CarbonLocalTransportSender.java:45)
at org.apache.axis2.transport.local.LocalTransportSender.invoke(LocalTransportSender.java:77)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:430)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.wso2.carbon.authenticator.stub.AuthenticationAdminStub.login(AuthenticationAdminStub.java:659)
at org.wso2.carbon.authenticator.proxy.AuthenticationAdminClient.login(AuthenticationAdminClient.java:64)
at org.wso2.carbon.ui.DefaultCarbonAuthenticator.doAuthentication(DefaultCarbonAuthenticator.java:119)
at org.wso2.carbon.ui.AbstractCarbonUIAuthenticator.handleSecurity(AbstractCarbonUIAuthenticator.java:218)
at org.wso2.carbon.ui.BasicAuthUIAuthenticator.authenticate(BasicAuthUIAuthenticator.java:83)
at org.wso2.carbon.ui.CarbonUILoginUtil.handleLogin(CarbonUILoginUtil.java:380)
at org.wso2.carbon.ui.CarbonSecuredHttpContext.handleSecurity(CarbonSecuredHttpContext.java:244)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:60)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.wso2.carbon.ui.filters.CRLFPreventionFilter.doFilter(CRLFPreventionFilter.java:59)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1739)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1698)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.security.PrivilegedActionException: java.lang.reflect.InvocationTargetException
at java.security.AccessController.doPrivileged(Native Method)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.callSecure(AbstractUserStoreManager.java:148)
... 77 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager$2.run(AbstractUserStoreManager.java:151)
... 79 more
Caused by: org.wso2.carbon.user.core.UserStoreException: Could not initialize new user store manager : enbcas.local Duplicate domain names not allowed.
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.addSecondaryUserStoreManager(AbstractUserStoreManager.java:4095)
... 84 more
what is my problem?
thank you

I have tried the same scenario with IS 5.1.0 (standalone setup) and it works fine. I have used org.wso2.carbon.user.core.jdbc.JDBCUserStoreManager as the secondary user store.
A secondary user store can be shared among tenants, but you need to configure them separately in all the tenants. And the user store domain name should be unique within the tenant. Once a new user store created, the respective XML configuration file is created at <IS_HOME>/repository/tenants/<TENANT_ID>/userstores directory.
Can you please verify the setup again? If the problem it still there please share the list of files in <IS_HOME>/repository/tenants/<TENANT_ID>/userstores/ directories?

Related

WSO2 IS Active Directory Primary Store SCIM 2 issues

I'm using WSO2 IS 5.8.0. I configured it by using Active Directory as primary user store.
In a first moment I didn't enable the SCIM for this store. All works pretty good. Then I needed to enable SCIM so I enabled it in the user.mgt.xml file
Then I configured the urn:ietf:params:scim:schemas:core:2.0 dialect as suggested here (I know it is related to the SCIM 1 but there is no documentation for SCIM 2)
From when I enabled the SCIM for this user store, I'm no more able in adding users from WSO2 console interface. When I add an user I have this error
TID: [-1234] [] [2019-07-17 16:28:30,073] DEBUG {org.wso2.carbon.identity.governance.listener.IdentityStoreEventListener} - doPreAddUser executed in the IdentityStoreEventListener for user: angelo.immediata11
TID: [-1234] [] [2019-07-17 16:28:30,073] ERROR {org.wso2.carbon.user.core.common.AbstractUserStoreManager} - Error occurred while accessing Java Security Manager Privilege Block when called by method addUser with 6 length of Objects and argTypes [class java.lang.String, class java.lang.Object, class [Ljava.lang.String;, interface java.util.Map, class java.lang.String, boolean]
TID: [-1234] [] [2019-07-17 16:28:30,077] ERROR {org.wso2.carbon.user.mgt.UserRealmProxy} - Error occurred while accessing Java Security Manager Privilege Block when called by method addUser with 6 length of Objects and argTypes [class java.lang.String, class java.lang.Object, class [Ljava.lang.String;, interface java.util.Map, class java.lang.String, boolean]
org.wso2.carbon.user.core.UserStoreException: Error occurred while accessing Java Security Manager Privilege Block when called by method addUser with 6 length of Objects and argTypes [class java.lang.String, class java.lang.Object, class [Ljava.lang.String;, interface java.util.Map, class java.lang.String, boolean]
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.callSecure(AbstractUserStoreManager.java:187)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.addUser(AbstractUserStoreManager.java:2708)
at org.wso2.carbon.user.mgt.UserRealmProxy.addUser(UserRealmProxy.java:775)
at org.wso2.carbon.user.mgt.UserAdmin.addUser(UserAdmin.java:200)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
at org.apache.axis2.rpc.receivers.RPCMessageReceiver.invokeBusinessLogic(RPCMessageReceiver.java:117)
at org.apache.axis2.receivers.AbstractInOutMessageReceiver.invokeBusinessLogic(AbstractInOutMessageReceiver.java:40)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.local.LocalTransportReceiver.processMessage(LocalTransportReceiver.java:170)
at org.apache.axis2.transport.local.LocalTransportReceiver.processMessage(LocalTransportReceiver.java:82)
at org.wso2.carbon.core.transports.local.CarbonLocalTransportSender.finalizeSendWithToAddress(CarbonLocalTransportSender.java:45)
at org.apache.axis2.transport.local.LocalTransportSender.invoke(LocalTransportSender.java:77)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:442)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:228)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.wso2.carbon.user.mgt.stub.UserAdminStub.addUser(UserAdminStub.java:1788)
at org.wso2.carbon.user.mgt.ui.UserAdminClient.addUser(UserAdminClient.java:94)
at org.apache.jsp.user.add_002dfinish_002dajaxprocessor_jsp._jspService(add_002dfinish_002dajaxprocessor_jsp.java:165)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:472)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:395)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:339)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.wso2.carbon.ui.JspServlet.service(JspServlet.java:155)
at org.wso2.carbon.ui.TilesJspServlet.service(TilesJspServlet.java:80)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:37)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.owasp.csrfguard.CsrfGuardFilter.doFilter(CsrfGuardFilter.java:88)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:65)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:494)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
at org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:80)
at org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:91)
at org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:65)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1137)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1780)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1739)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.security.PrivilegedActionException: java.lang.reflect.InvocationTargetException
at java.security.AccessController.doPrivileged(Native Method)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.callSecure(AbstractUserStoreManager.java:165)
... 81 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager$2.run(AbstractUserStoreManager.java:168)
... 83 more
Caused by: java.lang.NullPointerException
at org.wso2.carbon.identity.governance.listener.IdentityStoreEventListener.doPreAddUser(IdentityStoreEventListener.java:107)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.addUser(AbstractUserStoreManager.java:2823)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.addUser(AbstractUserStoreManager.java:2729)
... 88 more
TID: [-1234] [] [2019-07-17 16:28:30,081] ERROR {org.wso2.carbon.user.mgt.ui.UserAdminClient} - Error occurred while accessing Java Security Manager Privilege Block when called by method addUser with 6 length of Objects and argTypes [class java.lang.String, class java.lang.Object, class [Ljava.lang.String;, interface java.util.Map, class java.lang.String, boolean]
org.wso2.carbon.user.mgt.stub.UserAdminUserAdminException: UserAdminUserAdminException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.wso2.carbon.user.mgt.stub.UserAdminStub.addUser(UserAdminStub.java:1802)
at org.wso2.carbon.user.mgt.ui.UserAdminClient.addUser(UserAdminClient.java:94)
at org.apache.jsp.user.add_002dfinish_002dajaxprocessor_jsp._jspService(add_002dfinish_002dajaxprocessor_jsp.java:165)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:472)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:395)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:339)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.wso2.carbon.ui.JspServlet.service(JspServlet.java:155)
at org.wso2.carbon.ui.TilesJspServlet.service(TilesJspServlet.java:80)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:37)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.owasp.csrfguard.CsrfGuardFilter.doFilter(CsrfGuardFilter.java:88)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:65)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:494)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
at org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:80)
at org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:91)
at org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:65)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1137)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1780)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1739)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
I add also WSO2 console image
Do you have any suggestions? How can I solve this issue? This is really very frustrating.
Thank you
Angelo
EDIT
From the stack trace I arrived to the class
The exception is generated in class org.wso2.carbon.user.core.common.AbstractUserStoreManager in the method private Object callSecure(String methodName, final Object[] objects, Class[] argTypes) throws UserStoreException. More precisely here:
return AccessController.doPrivileged(new PrivilegedExceptionAction<Object>(){
public Object run() throws Exception {
return method.invoke(instance, objects);
}
});
For some reason when SCIM 2 is enabled the method.invoke instruction generates an error.
Do you have any ideas on why this happens?
I was wondering about WSO2 claims mapping toward AD; I mapped what I could and I deleted what I could.
The following pictures list my mapping:
Nothing changed.
This thing is really frustrating.
Do you have any suggestion?
This original issue coming from the NullPointerException that is shown in the stack trace. User store exception which is thrown saying Java Security Block is just a wrapper. The exception is occurring from the [1] line. (If you can debug the code you might able to find an answer.) However, my guess is this is due to a bad claim mapping. Enabling debug logs in the user.core component might also give more clues.
[1] https://github.com/wso2-extensions/identity-governance/blob/30d246b7a9c2d973bf30e6926bfb0a15003711bf/components/org.wso2.carbon.identity.governance/src/main/java/org/wso2/carbon/identity/governance/listener/IdentityStoreEventListener.java#L107
After long time and decompiling and debugging several classes I was able in solving the issue.
In WSO2 DB there was a dirty value for the createdTime claim (maybe related to some typo done by me or my colleagues). This generated a null key in claims map inside the IdentityStoreEventListener class. Once solved it we had to solve other issues related to the configuration.
Finally we had to configure all WSO2 claims with our customer AD instance (and we couldn't add any new attribute). We couldn't use a secondary user store so we had to delete some claims and map correctly others.
Here you can download a PDF where you can find all the mappings I used.
I hope this is useful
Angelo

WSO2 Tenant Updation fails v5.5.0

After migrating to v5.5.0, This error happens when this tenant is updated. Other tenants are working fine. And tenant is not able to login.
I have disabled org.wso2.carbon.identity.governance.listener.IdentityMgtEventListener with orderId 95. Then I can login into the tenant.
I also found a relative question and found tenant's Resident Identity Provider is throwing "Error loading Resident Identity Provider" and Keystore list page displays "Error occurred null".
The answer states,
In case others hit this - it seems the problem is if you use a
certificate alias other than 'wso2carbon' in the JKS keystore when you
change the SSL certificate.
As a novice in WSO2, I can't seem to get what it means. During the migration, I used the same wso2carbon.jks file from the old version(5.1.0). Also found old version also has the same Resident Identity Provider and Keystore list page errors.
How can I resolve this error? I can see that keystore for that tenant is not available. How do I create a keystore for a tenant?
{org.wso2.carbon.tenant.mgt.util.TenantMgtUtil} - Error in adding
claims to the user. org.wso2.carbon.user.core.UserStoreException:
Error when handling event : PRE_SET_USER_CLAIMS at
org.wso2.carbon.identity.governance.listener.IdentityMgtEventListener.handleEvent(IdentityMgtEventListener.java:619)
at
org.wso2.carbon.identity.governance.listener.IdentityMgtEventListener.handleEvent(IdentityMgtEventListener.java:572)
at
org.wso2.carbon.identity.governance.listener.IdentityMgtEventListener.doPreSetUserClaimValues(IdentityMgtEventListener.java:142)
at
org.wso2.carbon.user.core.common.AbstractUserStoreManager.setUserClaimValues(AbstractUserStoreManager.java:1430)
at
org.wso2.carbon.tenant.mgt.util.TenantMgtUtil.addClaimsToUserStoreManager(TenantMgtUtil.java:349)
at
org.wso2.carbon.tenant.mgt.services.TenantMgtAdminService.updateTenant(TenantMgtAdminService.java:417)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
at
org.apache.axis2.rpc.receivers.RPCMessageReceiver.invokeBusinessLogic(RPCMessageReceiver.java:117)
at
org.apache.axis2.receivers.AbstractInOutMessageReceiver.invokeBusinessLogic(AbstractInOutMessageReceiver.java:40)
at
org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at
org.apache.axis2.transport.local.LocalTransportReceiver.processMessage(LocalTransportReceiver.java:169)
at
org.apache.axis2.transport.local.LocalTransportReceiver.processMessage(LocalTransportReceiver.java:82)
at
org.wso2.carbon.core.transports.local.CarbonLocalTransportSender.finalizeSendWithToAddress(CarbonLocalTransportSender.java:45)
at
org.apache.axis2.transport.local.LocalTransportSender.invoke(LocalTransportSender.java:77)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442) at
org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:441)
at
org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:227)
at
org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at
org.wso2.carbon.tenant.mgt.stub.TenantMgtAdminServiceStub.updateTenant(TenantMgtAdminServiceStub.java:2362)
at
org.wso2.carbon.tenant.mgt.ui.clients.TenantServiceClient.updateTenant(TenantServiceClient.java:110)
at
org.wso2.carbon.tenant.mgt.ui.utils.TenantMgtUtil.updateTenantConfigBean(TenantMgtUtil.java:119)
at
org.apache.jsp.tenant_002dmgt.submit_005ftenant_005fajaxprocessor_jsp._jspService(submit_005ftenant_005fajaxprocessor_jsp.java:167)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:439)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:395)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:339)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
org.wso2.carbon.ui.JspServlet.service(JspServlet.java:155) at
org.wso2.carbon.ui.TilesJspServlet.service(TilesJspServlet.java:80)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:37)
at
org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at
org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at
org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:124)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:65)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:124)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:80)
at
org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:91)
at
org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:60)
at
org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at
org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at
org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at
org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at
org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at
org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
at
org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1115)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1775)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1734)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748) Caused by:
org.wso2.carbon.identity.event.IdentityEventException: Error while
getting connector configurations for property
:account.disable.handler.enable at
org.wso2.carbon.identity.handler.event.account.lock.util.AccountUtil.getConnectorConfig(AccountUtil.java:68)
at
org.wso2.carbon.identity.handler.event.account.lock.AccountDisableHandler.handleEvent(AccountDisableHandler.java:131)
at
org.wso2.carbon.identity.event.services.IdentityEventServiceImpl.handleEvent(IdentityEventServiceImpl.java:56)
at
org.wso2.carbon.identity.governance.listener.IdentityMgtEventListener.handleEvent(IdentityMgtEventListener.java:604)
... 80 more Caused by:
org.wso2.carbon.identity.governance.IdentityGovernanceException: Error
while retrieving resident Idp for test.com tenant. at
org.wso2.carbon.identity.governance.IdentityGovernanceServiceImpl.getConfiguration(IdentityGovernanceServiceImpl.java:108)
at
org.wso2.carbon.identity.governance.IdentityGovernanceServiceImpl.getConfiguration(IdentityGovernanceServiceImpl.java:131)
at
org.wso2.carbon.identity.handler.event.account.lock.util.AccountUtil.getConnectorConfig(AccountUtil.java:61)
... 83 more Caused by:
org.wso2.carbon.idp.mgt.IdentityProviderManagementException: Could not
find Resident Identity Provider for tenant test.com at
org.wso2.carbon.idp.mgt.IdentityProviderManager.getResidentIdP(IdentityProviderManager.java:311)
at
org.wso2.carbon.identity.governance.IdentityGovernanceServiceImpl.getConfiguration(IdentityGovernanceServiceImpl.java:105)
... 85 more TID: [-1234] [] [2019-04-09 15:46:18,571] ERROR
{org.wso2.carbon.tenant.mgt.ui.utils.TenantMgtUtil} - Failed to
update the tenant config. tenant-domain: test.com, tenant-admin:
admin.
org.wso2.carbon.tenant.mgt.stub.TenantMgtAdminServiceExceptionException:
TenantMgtAdminServiceExceptionException at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442) at
org.wso2.carbon.tenant.mgt.stub.TenantMgtAdminServiceStub.updateTenant(TenantMgtAdminServiceStub.java:2376)
at
org.wso2.carbon.tenant.mgt.ui.clients.TenantServiceClient.updateTenant(TenantServiceClient.java:110)
at
org.wso2.carbon.tenant.mgt.ui.utils.TenantMgtUtil.updateTenantConfigBean(TenantMgtUtil.java:119)
at
org.apache.jsp.tenant_002dmgt.submit_005ftenant_005fajaxprocessor_jsp._jspService(submit_005ftenant_005fajaxprocessor_jsp.java:167)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:439)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:395)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:339)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
org.wso2.carbon.ui.JspServlet.service(JspServlet.java:155) at
org.wso2.carbon.ui.TilesJspServlet.service(TilesJspServlet.java:80)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:37)
at
org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at
org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at
org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at
org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:124)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:65)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:124)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:80)
at
org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:91)
at
org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:60)
at
org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at
org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at
org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at
org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at
org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at
org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
at
org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1115)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1775)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1734)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748) ```
I have resolved this, although I'm not sure if this is the solution. I'm hoping this would help someone or someone could help me.
For tenant update throwing error Failed to update the tenant config. tenant-domain: test.com, tenant-admin: admin:
Disable the Identity Listener org.wso2.carbon.identity.governance.listener.IdentityMgtEventListener with orderId 95. Now in the admin user, you will be able to update the tenant.
The IdentityMgtEventListener with orderId 95 has to be disabled to login into the tenant for resolving keystore and Resident Identity Provider Issues.
For the keystore page throwing Error occurred null:
Create a keystore for the tenant and add the keystore. These links were helpful:
Changing the hostname
How to changing the primary keystore of a tenant in carbon products
For the Resident Identity Provider throwing Error loading Resident Identity Provider:
I found the tenant doesn't have an entry in the IDP table. So, I inserted a record for the tenant using the tenant id and this was resolved. Not sure if this is the right solution.
Sample MySQL query:
INSERT INTO database.IDP ('ID', 'TENANT_ID', 'NAME', 'IS_ENABLED', 'IS_PRIMARY', 'HOME_REALM_ID', 'INBOUND_PROV_ENABLED', 'IS_FEDERATION_HUB', 'IS_LOCAL_CLAIM_DIALECT') VALUES ('72', '10', 'LOCAL', '1', '1', 'localhost', '0', '0', '0');

Intermittent failure while publishing API in wso2 API manager

We are having WSO2 API Manager 2.1.0 setup in a clustered environment. API publication is failing intermittently. API publication request from Publisher to Gateway manager is served by AWS elastic load balancer. When the log file of publisher is analyzed, it shows the following error.
INFO {org.apache.axis2.transport.http.HTTPSender} - Unable to sendViaPost to url[https://LB-DNS:PORT/services/APIGatewayAdmin] {org.apache.axis2.transport.http.HTTPSender}
org.apache.axis2.AxisFault: Transport error: 504 Error: GATEWAY_TIMEOUT
at org.apache.axis2.transport.http.HTTPSender.handleResponse(HTTPSender.java:326)
at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:196)
at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:77)
at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:451)
at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:278)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:430)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.wso2.carbon.apimgt.gateway.stub.APIGatewayAdminStub.getApi(APIGatewayAdminStub.java:7012)
at org.wso2.carbon.apimgt.impl.utils.APIGatewayAdminClient.getApi(APIGatewayAdminClient.java:145)
at org.wso2.carbon.apimgt.impl.APIGatewayManager.publishToGateway(APIGatewayManager.java:102)
at org.wso2.carbon.apimgt.impl.APIProviderImpl.publishToGateway(APIProviderImpl.java:1519)
at org.wso2.carbon.apimgt.impl.APIProviderImpl.propergateAPIStatusChangeToGateways(APIProviderImpl.java:1328)
at org.wso2.carbon.apimgt.impl.UserAwareAPIProvider.propergateAPIStatusChangeToGateways(UserAwareAPIProvider.java:36)
at org.wso2.carbon.apimgt.impl.executors.APIExecutor.execute(APIExecutor.java:136)
at org.wso2.carbon.governance.registry.extensions.aspects.DefaultLifeCycle.runCustomExecutorsCode(DefaultLifeCycle.java:712)
at org.wso2.carbon.governance.registry.extensions.aspects.DefaultLifeCycle.invoke(DefaultLifeCycle.java:450)
at org.wso2.carbon.governance.registry.eventing.handlers.GovernanceEventingHandler.invokeAspect(GovernanceEventingHandler.java:338)
at org.wso2.carbon.registry.core.jdbc.handlers.HandlerManager.invokeAspect(HandlerManager.java:2792)
at org.wso2.carbon.registry.core.jdbc.handlers.HandlerLifecycleManager.invokeAspect(HandlerLifecycleManager.java:518)
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.invokeAspect(EmbeddedRegistry.java:2591)
at org.wso2.carbon.registry.core.caching.CacheBackedRegistry.invokeAspect(CacheBackedRegistry.java:510)
at org.wso2.carbon.registry.core.session.UserRegistry.invokeAspect(UserRegistry.java:1952)
at org.wso2.carbon.governance.api.common.dataobjects.GovernanceArtifactImpl.invokeAction(GovernanceArtifactImpl.java:829)
at org.wso2.carbon.governance.api.common.dataobjects.GovernanceArtifactImpl.invokeAction(GovernanceArtifactImpl.java:808)
at org.wso2.carbon.apimgt.impl.APIProviderImpl.changeLifeCycleStatus(APIProviderImpl.java:3755)
at org.wso2.carbon.apimgt.impl.UserAwareAPIProvider.changeLifeCycleStatus(UserAwareAPIProvider.java:148)
at org.wso2.carbon.apimgt.rest.api.publisher.impl.ApisApiServiceImpl.apisChangeLifecyclePost(ApisApiServiceImpl.java:303)
at org.wso2.carbon.apimgt.rest.api.publisher.ApisApi.apisChangeLifecyclePost(ApisApi.java:520)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:189)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:99)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:254)
at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160)
at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:180)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:299)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:218)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:274)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatSessionReplicationValve.invoke(CarbonTomcatSessionReplicationValve.java:45)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:958)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:333)
at org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(JvmRouteBinderValve.java:218)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1756)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1715)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
ERROR {org.wso2.carbon.apimgt.impl.APIGatewayManager} - Error occurred when publish to gateway Production and Sandbox {org.wso2.carbon.apimgt.impl.APIGatewayManager}
org.apache.axis2.AxisFault: Error while obtaining API information from gateway. Transport error: 504 Error: GATEWAY_TIMEOUT
at org.wso2.carbon.apimgt.impl.utils.APIGatewayAdminClient.getApi(APIGatewayAdminClient.java:149)
at org.wso2.carbon.apimgt.impl.APIGatewayManager.publishToGateway(APIGatewayManager.java:102)
at org.wso2.carbon.apimgt.impl.APIProviderImpl.publishToGateway(APIProviderImpl.java:1519)
at org.wso2.carbon.apimgt.impl.APIProviderImpl.propergateAPIStatusChangeToGateways(APIProviderImpl.java:1328)
at org.wso2.carbon.apimgt.impl.UserAwareAPIProvider.propergateAPIStatusChangeToGateways(UserAwareAPIProvider.java:36)
at org.wso2.carbon.apimgt.impl.executors.APIExecutor.execute(APIExecutor.java:136)
at org.wso2.carbon.governance.registry.extensions.aspects.DefaultLifeCycle.runCustomExecutorsCode(DefaultLifeCycle.java:712)
at org.wso2.carbon.governance.registry.extensions.aspects.DefaultLifeCycle.invoke(DefaultLifeCycle.java:450)
at org.wso2.carbon.governance.registry.eventing.handlers.GovernanceEventingHandler.invokeAspect(GovernanceEventingHandler.java:338)
at org.wso2.carbon.registry.core.jdbc.handlers.HandlerManager.invokeAspect(HandlerManager.java:2792)
at org.wso2.carbon.registry.core.jdbc.handlers.HandlerLifecycleManager.invokeAspect(HandlerLifecycleManager.java:518)
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.invokeAspect(EmbeddedRegistry.java:2591)
at org.wso2.carbon.registry.core.caching.CacheBackedRegistry.invokeAspect(CacheBackedRegistry.java:510)
at org.wso2.carbon.registry.core.session.UserRegistry.invokeAspect(UserRegistry.java:1952)
at org.wso2.carbon.governance.api.common.dataobjects.GovernanceArtifactImpl.invokeAction(GovernanceArtifactImpl.java:829)
at org.wso2.carbon.governance.api.common.dataobjects.GovernanceArtifactImpl.invokeAction(GovernanceArtifactImpl.java:808)
at org.wso2.carbon.apimgt.impl.APIProviderImpl.changeLifeCycleStatus(APIProviderImpl.java:3755)
at org.wso2.carbon.apimgt.impl.UserAwareAPIProvider.changeLifeCycleStatus(UserAwareAPIProvider.java:148)
at org.wso2.carbon.apimgt.rest.api.publisher.impl.ApisApiServiceImpl.apisChangeLifecyclePost(ApisApiServiceImpl.java:303)
at org.wso2.carbon.apimgt.rest.api.publisher.ApisApi.apisChangeLifecyclePost(ApisApi.java:520)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:189)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:99)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:254)
at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160)
at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:180)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:299)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:218)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:274)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatSessionReplicationValve.invoke(CarbonTomcatSessionReplicationValve.java:45)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:958)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:333)
at org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(JvmRouteBinderValve.java:218)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1756)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1715)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.axis2.AxisFault: Transport error: 504 Error: GATEWAY_TIMEOUT
at org.apache.axis2.transport.http.HTTPSender.handleResponse(HTTPSender.java:326)
at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:196)
at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:77)
at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:451)
at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:278)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:430)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.wso2.carbon.apimgt.gateway.stub.APIGatewayAdminStub.getApi(APIGatewayAdminStub.java:7012)
at org.wso2.carbon.apimgt.impl.utils.APIGatewayAdminClient.getApi(APIGatewayAdminClient.java:145)
... 71 more
Below error messages found in Gateway manager
ERROR {org.apache.synapse.transport.passthru.PassThroughHttpSSLSender} - IO Error sending response message {org.apache.synapse.transport.passthru.PassThroughHttpSSLSender}
org.apache.axis2.AxisFault: java.io.IOException: Broken pipe
at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
at org.apache.axis2.transport.http.SOAPMessageFormatter.writeTo(SOAPMessageFormatter.java:78)
at org.apache.synapse.transport.passthru.PassThroughHttpSender.sendUsingOutputStream(PassThroughHttpSender.java:307)
at org.apache.synapse.transport.passthru.PassThroughHttpSender.invoke(PassThroughHttpSender.java:271)
at org.apache.axis2.engine.AxisEngine.sendFault(AxisEngine.java:525)
at org.apache.axis2.transport.http.AxisServlet.handleFault(AxisServlet.java:433)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:216)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.owasp.csrfguard.CsrfGuardFilter.doFilter(CsrfGuardFilter.java:88)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:120)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:958)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1756)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1715)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.ctc.wstx.exc.WstxIOException: java.io.IOException: Broken pipe
at com.ctc.wstx.sw.BaseStreamWriter._finishDocument(BaseStreamWriter.java:1406)
at com.ctc.wstx.sw.BaseStreamWriter.close(BaseStreamWriter.java:247)
at org.apache.axiom.om.impl.MTOMXMLStreamWriter.close(MTOMXMLStreamWriter.java:188)
at org.apache.axiom.om.impl.llom.OMSerializableImpl.serializeAndConsume(OMSerializableImpl.java:197)
at org.apache.axis2.transport.http.SOAPMessageFormatter.writeTo(SOAPMessageFormatter.java:74)
... 50 more
Caused by: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:370)
at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:334)
at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:101)
at com.compuware.apm.agent.introspection.uem.impl.AgentInjectingServletOutputStream.flush(AgentInjectingServletOutputStream.java:168)
at com.ctc.wstx.io.UTF8Writer.flush(UTF8Writer.java:103)
at com.ctc.wstx.sw.BufferingXmlWriter.flush(BufferingXmlWriter.java:225)
at com.ctc.wstx.sw.BufferingXmlWriter.close(BufferingXmlWriter.java:198)
at com.ctc.wstx.sw.BaseStreamWriter._finishDocument(BaseStreamWriter.java:1404)
... 54 more
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:51)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
at org.apache.tomcat.util.net.SecureNioChannel.flush(SecureNioChannel.java:141)
at org.apache.tomcat.util.net.SecureNioChannel.write(SecureNioChannel.java:502)
at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:174)
at org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:164)
at org.apache.coyote.http11.InternalNioOutputBuffer.flushBuffer(InternalNioOutputBuffer.java:243)
at org.apache.coyote.http11.InternalNioOutputBuffer.flush(InternalNioOutputBuffer.java:95)
at org.apache.coyote.http11.AbstractHttp11Processor.action(AbstractHttp11Processor.java:800)
at org.apache.coyote.Response.action(Response.java:172)
at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:366)
at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:334)
at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:101)
at com.compuware.apm.agent.introspection.uem.impl.AgentInjectingServletOutputStream.flush(AgentInjectingServletOutputStream.java:168)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:182)
... 45 more
We were able to resolve this problem by analysing possible scenarios request can hang. We are using AWS EFS for sharing synapse files(APIM_HOME/repository/deployment/server/synapse-configs/default) between the gateway manager and gateway worker. While Publishing an API, Publisher will call APIGateway admin service which includes EFS file creation operation. We had created EFS with Burst mode. In burst mode, throughput is determined using burst credit. Since our EFS burst credit reduced to zero, EFS doesn't have enough throughput to do i/o operations in EFS. Changing the throughput mode to Provisioned mode solved the issue.
Useful links
https://docs.aws.amazon.com/efs/latest/ug/performance.html#throughput-modes
https://aws.amazon.com/premiumsupport/knowledge-center/efs-burst-credits/

Unable to delete user from Wso2 IS

I am not able to delete user from wso2 (intermediately).
For deleting user i have used RemoteUserStoreManagerServiceStub service in my application.
WSO2 Stack Trace Log:
TID: [-1234] [] [2018-05-26 16:30:35,971] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Error when handling event : POST_DELETE_USER
org.wso2.carbon.user.core.UserStoreException: Error when handling event : POST_DELETE_USER
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.callSecure(AbstractUserStoreManager.java:173)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.deleteUser(AbstractUserStoreManager.java:1273)
at org.wso2.carbon.um.ws.service.UserStoreManagerService.deleteUser(UserStoreManagerService.java:164)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:173)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:147)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:232)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.owasp.csrfguard.CsrfGuardFilter.doFilter(CsrfGuardFilter.java:88)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:124)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:65)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:124)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:80)
at org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:91)
at org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:60)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1115)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1775)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1734)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.security.PrivilegedActionException: java.lang.reflect.InvocationTargetException
at java.security.AccessController.doPrivileged(Native Method)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.callSecure(AbstractUserStoreManager.java:163)
... 62 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager$2.run(AbstractUserStoreManager.java:166)
... 64 more
Caused by: org.wso2.carbon.user.core.UserStoreException: Error when handling event : POST_DELETE_USER
at org.wso2.carbon.identity.governance.listener.IdentityMgtEventListener.handleEvent(IdentityMgtEventListener.java:619)
at org.wso2.carbon.identity.governance.listener.IdentityMgtEventListener.handleEvent(IdentityMgtEventListener.java:572)
at org.wso2.carbon.identity.governance.listener.IdentityMgtEventListener.doPostDeleteUser(IdentityMgtEventListener.java:298)
at org.wso2.carbon.user.core.common.AbstractUserStoreManager.deleteUser(AbstractUserStoreManager.java:1348)
... 69 more
Caused by: org.wso2.carbon.identity.event.IdentityEventException: Error while deleting consents for user abc.register#outlook.com
at org.wso2.carbon.identity.consent.mgt.handler.ConsentDeletionUserEventHandler.handleEvent(ConsentDeletionUserEventHandler.java:135)
at org.wso2.carbon.identity.event.services.IdentityEventServiceImpl.handleEvent(IdentityEventServiceImpl.java:56)
at org.wso2.carbon.identity.governance.listener.IdentityMgtEventListener.handleEvent(IdentityMgtEventListener.java:604)
... 72 more
Caused by: org.wso2.carbon.consent.mgt.core.exception.ConsentManagementServerException: Error while searching receipts.
at org.wso2.carbon.consent.mgt.core.util.ConsentUtils.handleServerException(ConsentUtils.java:64)
at org.wso2.carbon.consent.mgt.core.dao.impl.ReceiptDAOImpl.searchReceipts(ReceiptDAOImpl.java:303)
at org.wso2.carbon.consent.mgt.core.ConsentManagerImpl.searchReceipts(ConsentManagerImpl.java:589)
at org.wso2.carbon.consent.mgt.core.ConsentManagerImpl.searchReceipts(ConsentManagerImpl.java:562)
at org.wso2.carbon.consent.mgt.core.InterceptingConsentManager$21.execute(InterceptingConsentManager.java:546)
at org.wso2.carbon.consent.mgt.core.InterceptingConsentManager$21.execute(InterceptingConsentManager.java:542)
at org.wso2.carbon.consent.mgt.core.model.ConsentInterceptorTemplate.executeWith(ConsentInterceptorTemplate.java:56)
at org.wso2.carbon.consent.mgt.core.InterceptingConsentManager.searchReceipts(InterceptingConsentManager.java:542)
at org.wso2.carbon.identity.consent.mgt.handler.ConsentDeletionUserEventHandler.handleEvent(ConsentDeletionUserEventHandler.java:121)
... 74 more
Caused by: org.wso2.carbon.database.utils.jdbc.exceptions.DataAccessException: Error in performing Database query: '%s'SELECT CONSENT_RECEIPT_ID, LANGUAGE, PII_PRINCIPAL_ID, PRINCIPAL_TENANT_ID, STATE, SP_DISPLAY_NAME, SP_DESCRIPTION FROM (SELECT R.CONSENT_RECEIPT_ID, R.LANGUAGE, R.PII_PRINCIPAL_ID, R.PRINCIPAL_TENANT_ID, R.STATE, RS.SP_DISPLAY_NAME, RS.SP_DESCRIPTION, ROW_NUMBER() OVER ( ORDER BY R.CONSENT_RECEIPT_ID) AS ROWNUM FROM CM_RECEIPT R INNER JOIN CM_RECEIPT_SP_ASSOC RS ON R.CONSENT_RECEIPT_ID = RS.CONSENT_RECEIPT_ID WHERE PII_PRINCIPAL_ID LIKE ? AND PRINCIPAL_TENANT_ID=? AND SP_NAME LIKE ? AND STATE LIKE ?) AS RES WHERE RES.ROWNUM BETWEEN ? AND ?
at org.wso2.carbon.database.utils.jdbc.JdbcTemplate.executeQuery(JdbcTemplate.java:168)
at org.wso2.carbon.consent.mgt.core.dao.impl.ReceiptDAOImpl.searchWithoutSpTenant(ReceiptDAOImpl.java:382)
at org.wso2.carbon.consent.mgt.core.dao.impl.ReceiptDAOImpl.searchReceipts(ReceiptDAOImpl.java:293)
... 81 more
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Invalid object name 'CM_RECEIPT'.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:197)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1493)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:390)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:340)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1400)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:179)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:154)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQuery(SQLServerPreparedStatement.java:283)
at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tomcat.jdbc.pool.StatementFacade$StatementProxy.invoke(StatementFacade.java:114)
at com.sun.proxy.$Proxy19.executeQuery(Unknown Source)
at org.wso2.carbon.database.utils.jdbc.JdbcTemplate.executeQuery(JdbcTemplate.java:153)
... 83 more
Can any one tell me why it's happened (intermediate only)?
WSO2 IS Version: 5.5.0 with patch
Suggested link for patch (Reason for patch: remote claims not getting issues): Not getting remote claims from wso2

WSO2 IoT 3.1.0 failed to retreive oauth token using jwt error

I am installing WSO2 IOT Server 3.1.0 on a server. I just want to run it with default basic configurations. After Uploading to my server, I configured it to run with the IP using the script as mentioned in this link.
After running the script, I ran the broker and the iotserver profiles. I am able to log in the API store and publisher but when I tries to log in the devicemgt web console. I am getting the following error.
Error Snapshot:
Exception In Logs:
An exception thrown when executing the script '/app/modules/login.js'. {auth-module}
TID: [-1234] [] [2017-12-04 00:32:43,944] ERROR {auth-module} - org.wso2.carbon.apimgt.integration.client.exception.APIMClientOAuthException: failed to retrieve oauth token using jwt {auth-module}
org.mozilla.javascript.WrappedException: Wrapped org.wso2.carbon.apimgt.integration.client.exception.APIMClientOAuthException: failed to retrieve oauth token using jwt (/devicemgt/app/modules/login.js#31)
at org.mozilla.javascript.Context.throwAsScriptRuntimeEx(Context.java:1754)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:148)
at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:225)
at org.mozilla.javascript.optimizer.OptRuntime.callProp0(OptRuntime.java:85)
at org.jaggeryjs.rhino.devicemgt.app.modules.c3._c_anonymous_2(/devicemgt/app/modules/login.js:31)
at org.jaggeryjs.rhino.devicemgt.app.modules.c3.call(/devicemgt/app/modules/login.js)
at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c0._c_executeScript_10(/devicemgt/lib/modules/auth/auth.js:234)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c0.call(/devicemgt/lib/modules/auth/auth.js)
at org.mozilla.javascript.optimizer.OptRuntime.callName(OptRuntime.java:63)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c0._c_handleEvent_11(/devicemgt/lib/modules/auth/auth.js:259)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c0.call(/devicemgt/lib/modules/auth/auth.js)
at org.mozilla.javascript.optimizer.OptRuntime.callName(OptRuntime.java:63)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c0._c_anonymous_20(/devicemgt/lib/modules/auth/auth.js:547)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c0.call(/devicemgt/lib/modules/auth/auth.js)
at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c2._c_anonymous_1(/devicemgt/lib/modules/auth/acs.jag:34)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c2.call(/devicemgt/lib/modules/auth/acs.jag)
at org.mozilla.javascript.optimizer.OptRuntime.call0(OptRuntime.java:23)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c2._c_script_0(/devicemgt/lib/modules/auth/acs.jag:22)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c2.call(/devicemgt/lib/modules/auth/acs.jag)
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c2.call(/devicemgt/lib/modules/auth/acs.jag)
at org.jaggeryjs.rhino.devicemgt.lib.modules.auth.c2.exec(/devicemgt/lib/modules/auth/acs.jag)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:567)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.exec(RhinoEngine.java:273)
at org.jaggeryjs.jaggery.core.manager.WebAppManager.exec(WebAppManager.java:588)
at org.jaggeryjs.jaggery.core.manager.WebAppManager.execute(WebAppManager.java:508)
at org.jaggeryjs.jaggery.core.JaggeryServlet.doPost(JaggeryServlet.java:29)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:747)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:485)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:377)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:337)
at org.jaggeryjs.jaggery.core.JaggeryFilter.doFilter(JaggeryFilter.java:21)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.wso2.carbon.ui.filters.cache.ContentTypeBasedCachePreventionFilter.doFilter(ContentTypeBasedCachePreventionFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.owasp.csrfguard.CsrfGuardFilter.doFilter(CsrfGuardFilter.java:88)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:120)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at org.wso2.carbon.webapp.authenticator.framework.WebappAuthenticationValve.invoke(WebappAuthenticationValve.java:45)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.wso2.carbon.apimgt.integration.client.exception.APIMClientOAuthException: failed to retrieve oauth token using jwt
at org.wso2.carbon.apimgt.integration.client.OAuthRequestInterceptor.apply(OAuthRequestInterceptor.java:118)
at feign.SynchronousMethodHandler.targetRequest(SynchronousMethodHandler.java:158)
at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:88)
at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:76)
at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:103)
at com.sun.proxy.$Proxy38.tiersTierLevelTierNameGet(Unknown Source)
at org.wso2.carbon.apimgt.application.extension.APIManagementProviderServiceImpl.isTierLoaded(APIManagementProviderServiceImpl.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)
... 75 more
Caused by: org.wso2.carbon.identity.jwt.client.extension.exception.JWTClientException: Error when parsing the response <am:fault xmlns:am="http://wso2.org/apimanager"><am:code>101500</am:code><am:type>Status report</am:type><am:message>Runtime Error</am:message><am:description>Error in Sender</am:description></am:fault>
at org.wso2.carbon.identity.jwt.client.extension.JWTClient.getTokenInfo(JWTClient.java:169)
at org.wso2.carbon.identity.jwt.client.extension.JWTClient.getAccessToken(JWTClient.java:79)
at org.wso2.carbon.apimgt.integration.client.OAuthRequestInterceptor.apply(OAuthRequestInterceptor.java:99)
... 86 more
Caused by: Unexpected character (<) at position 0.
at org.json.simple.parser.Yylex.yylex(Unknown Source)
at org.json.simple.parser.JSONParser.nextToken(Unknown Source)
at org.json.simple.parser.JSONParser.parse(Unknown Source)
at org.json.simple.parser.JSONParser.parse(Unknown Source)
at org.json.simple.parser.JSONParser.parse(Unknown Source)
at org.wso2.carbon.identity.jwt.client.extension.JWTClient.getTokenInfo(JWTClient.java:153)
... 88 more
Can anybody help pointing out what is going wrong and what is happening?
Are you using the same IP as the one you entered when asked for the common name when running the change IP script, to access the server?
Appreciate if you could add here the entire log for server startup. That way we would be able to figure out what exactly is causing the problem.