I am running Apache Ignite in a Kubernetes environment as a stateful set. Everything is working as expected in Ignite 2.10. When I tried to upgrade the Ignite version to 2.11 and create a new Ignite cluster, the nodes are not starting up and I get the following error:
[20:47:02,014][SEVERE][main][IgniteKernal] Exception during start processors, node will be stopped and close connections
class org.apache.ignite.IgniteCheckedException: Failed to start processor: GridProcessorAdapter []
at org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:2006)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1270)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2141)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1787)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1172)
at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1066)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:952)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:851)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:721)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690)
at org.apache.ignite.Ignition.start(Ignition.java:353)
at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:367)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start Jetty HTTP server.
at org.apache.ignite.internal.processors.rest.protocols.http.jetty.GridJettyRestProtocol.loadJettyConfiguration(GridJettyRestProtocol.java:321)
at org.apache.ignite.internal.processors.rest.protocols.http.jetty.GridJettyRestProtocol.start(GridJettyRestProtocol.java:153)
at org.apache.ignite.internal.processors.rest.GridRestProcessor.startProtocol(GridRestProcessor.java:1012)
at org.apache.ignite.internal.processors.rest.GridRestProcessor.startHttpProtocol(GridRestProcessor.java:983)
at org.apache.ignite.internal.processors.rest.GridRestProcessor.start(GridRestProcessor.java:534)
at org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:2003)
... 11 more
Caused by: java.lang.IllegalStateException: No matching constructor class org.eclipse.jetty.server.Server in file:///opt/ignite/apache-ignite/config/custom-jetty.xml
at org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.configure(XmlConfiguration.java:454)
at org.eclipse.jetty.xml.XmlConfiguration.configure(XmlConfiguration.java:380)
at org.apache.ignite.internal.processors.rest.protocols.http.jetty.GridJettyRestProtocol.loadJettyConfiguration(GridJettyRestProtocol.java:318)
... 16 more
[20:47:02,017][SEVERE][main][IgniteKernal] Got exception while starting (will rollback startup routine).
class org.apache.ignite.IgniteCheckedException: Failed to start processor: GridProcessorAdapter []
at org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:2006)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1270)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2141)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1787)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1172)
at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1066)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:952)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:851)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:721)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690)
at org.apache.ignite.Ignition.start(Ignition.java:353)
at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:367)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start Jetty HTTP server.
at org.apache.ignite.internal.processors.rest.protocols.http.jetty.GridJettyRestProtocol.loadJettyConfiguration(GridJettyRestProtocol.java:321)
at org.apache.ignite.internal.processors.rest.protocols.http.jetty.GridJettyRestProtocol.start(GridJettyRestProtocol.java:153)
at org.apache.ignite.internal.processors.rest.GridRestProcessor.startProtocol(GridRestProcessor.java:1012)
at org.apache.ignite.internal.processors.rest.GridRestProcessor.startHttpProtocol(GridRestProcessor.java:983)
at org.apache.ignite.internal.processors.rest.GridRestProcessor.start(GridRestProcessor.java:534)
at org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:2003)
... 11 more
Caused by: java.lang.IllegalStateException: No matching constructor class org.eclipse.jetty.server.Server in file:///opt/ignite/apache-ignite/config/custom-jetty.xml
at org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.configure(XmlConfiguration.java:454)
at org.eclipse.jetty.xml.XmlConfiguration.configure(XmlConfiguration.java:380)
at org.apache.ignite.internal.processors.rest.protocols.http.jetty.GridJettyRestProtocol.loadJettyConfiguration(GridJettyRestProtocol.java:318)
... 16 more
I use a jetty.xml file which is included in the Ignite configuration as a connector configuration. The jetty.xml has the default settings as provided in the apache ignite webpage. Also the ignite-rest-http folder has been copied to lib folder; the OPTION_LIBS environment variable also has this module to enable it. None of the settings has changed, so not sure what is different in 2.11 to cause this issue. Everything works in Ignite 2.10.
You can fix it by changing threadPool argument -> threadpool in jetty XML configuration.
This is a bug in Ignite documentation. I've created a JIRA for this. See details here
Related
I am trying to deploy the war file on tomcat using jenkins on ubuntu machine.
Using:
jdk 1.8
tomcat 9
Can able to create .war file successfully inside webapps folder but while deploying, its failing. Also can able to see my .war context path in tomcat manager page but whaile accessing I'm getting 404.
ERROR: Build step failed with exception
org.codehaus.cargo.container.ContainerException: Failed to deploy [/var/lib/jenkins/workspace/First/target/InsuranceProvider.war]
at org.codehaus.cargo.container.tomcat.internal.AbstractTomcatManagerDeployer.deploy(AbstractTomcatManagerDeployer.java:104)
at org.codehaus.cargo.container.tomcat.internal.AbstractTomcatManagerDeployer.redeploy(AbstractTomcatManagerDeployer.java:172)
at hudson.plugins.deploy.CargoContainerAdapter.deploy(CargoContainerAdapter.java:81)
at hudson.plugins.deploy.CargoContainerAdapter$DeployCallable.invoke(CargoContainerAdapter.java:167)
at hudson.plugins.deploy.CargoContainerAdapter$DeployCallable.invoke(CargoContainerAdapter.java:136)
at hudson.FilePath.act(FilePath.java:1165)
at hudson.FilePath.act(FilePath.java:1148)
at hudson.plugins.deploy.CargoContainerAdapter.redeployFile(CargoContainerAdapter.java:133)
at hudson.plugins.deploy.PasswordProtectedAdapterCargo.redeployFile(PasswordProtectedAdapterCargo.java:95)
at hudson.plugins.deploy.DeployPublisher.perform(DeployPublisher.java:113)
at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:45)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:752)
at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.post2(MavenModuleSetBuild.java:1072)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:697)
at hudson.model.Run.execute(Run.java:1931)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: org.codehaus.cargo.container.tomcat.internal.TomcatManagerException: The Tomcat Manager responded "FAIL - Deployed application at context path [/InsuranceProvider] but context failed to start
" instead of the expected "OK" message
at org.codehaus.cargo.container.tomcat.internal.TomcatManager.invoke(TomcatManager.java:721)
at org.codehaus.cargo.container.tomcat.internal.TomcatManager.deployImpl(TomcatManager.java:767)
at org.codehaus.cargo.container.tomcat.internal.TomcatManager.deploy(TomcatManager.java:312)
at org.codehaus.cargo.container.tomcat.internal.AbstractTomcatManagerDeployer.deploy(AbstractTomcatManagerDeployer.java:99)
... 20 more
org.codehaus.cargo.container.tomcat.internal.TomcatManagerException: The **Tomcat Manager responded "FAIL - Deployed application at context path [/InsuranceProvider] but context failed to start
" instead of the expected "OK" message**
at org.codehaus.cargo.container.tomcat.internal.TomcatManager.invoke(TomcatManager.java:721)
at org.codehaus.cargo.container.tomcat.internal.TomcatManager.deployImpl(TomcatManager.java:767)
at org.codehaus.cargo.container.tomcat.internal.TomcatManager.deploy(TomcatManager.java:312)
at org.codehaus.cargo.container.tomcat.internal.AbstractTomcatManagerDeployer.deploy(AbstractTomcatManagerDeployer.java:99)
at org.codehaus.cargo.container.tomcat.internal.AbstractTomcatManagerDeployer.redeploy(AbstractTomcatManagerDeployer.java:172)
at hudson.plugins.deploy.CargoContainerAdapter.deploy(CargoContainerAdapter.java:81)
at hudson.plugins.deploy.CargoContainerAdapter$DeployCallable.invoke(CargoContainerAdapter.java:167)
at hudson.plugins.deploy.CargoContainerAdapter$DeployCallable.invoke(CargoContainerAdapter.java:136)
at hudson.FilePath.act(FilePath.java:1165)
at hudson.FilePath.act(FilePath.java:1148)
at hudson.plugins.deploy.CargoContainerAdapter.redeployFile(CargoContainerAdapter.java:133)
at hudson.plugins.deploy.PasswordProtectedAdapterCargo.redeployFile(PasswordProtectedAdapterCargo.java:95)
at hudson.plugins.deploy.DeployPublisher.perform(DeployPublisher.java:113)
at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:45)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:752)
at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.post2(MavenModuleSetBuild.java:1072)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:697)
at hudson.model.Run.execute(Run.java:1931)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Build step 'Deploy war/ear to a container' marked build as failure
Finished: FAILURE
I use Apache Karaf 4.2.8. I installed keycloak osgi features. I could configure authentication for karaf shell with keycloak using keycloak-jaas feature.
Now I want to configure authentication for my services under /cxf context.
I configured pax-web context so login-method is KEYCLOAK.
But I have to install keycloak-jetty-adapter as well, so it could be integrated. But when I try to install that feature I receive an error.
trun()> feature:install keycloak-jetty9-adapter
org.apache.felix.resolver.reason.ReasonException: Unable to resolve
root: missing requirement [root] osgi.identity;
osgi.identity=keycloak-jetty9-adapter; type=karaf.feature;
version="[8.0.1,8.0.1]";
filter:="(&(osgi.identity=keycloak-jetty9-adapter)(type=karaf.feature)(version>=8.0.1)(version<=8.0.1))" [caused by: Unable to resolve keycloak-jetty9-adapter/8.0.1: missing
requirement [keycloak-jetty9-adapter/8.0.1] osgi.identity;
osgi.identity=jetty; type=karaf.feature; version="[9.2.0,9.3.0)"] at
org.apache.felix.resolver.Candidates$MissingRequirementError.toException(Candidates.java:1343)
at
org.apache.felix.resolver.ResolverImpl.doResolve(ResolverImpl.java:392)
at
org.apache.felix.resolver.ResolverImpl.resolve(ResolverImpl.java:378)
at
org.apache.felix.resolver.ResolverImpl.resolve(ResolverImpl.java:332)
at
org.apache.karaf.features.internal.region.SubsystemResolver.resolve(SubsystemResolver.java:257)
at
org.apache.karaf.features.internal.service.Deployer.deploy(Deployer.java:388)
at
org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvision(FeaturesServiceImpl.java:1058)
at
org.apache.karaf.features.internal.service.FeaturesServiceImpl.lambda$doProvisionInThread$13(FeaturesServiceImpl.java:994)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) Caused by:
org.apache.felix.resolver.reason.ReasonException: Unable to resolve
keycloak-jetty9-adapter/8.0.1: missing requirement
[keycloak-jetty9-adapter/8.0.1] osgi.identity; osgi.identity=jetty;
type=karaf.feature; version="[9.2.0,9.3.0)" at
org.apache.felix.resolver.Candidates$MissingRequirementError.toException(Candidates.java:1343)
... 12 more Error executing command: Unable to resolve root: missing
requirement [root] osgi.identity;
osgi.identity=keycloak-jetty9-adapter; type=karaf.feature;
version="[8.0.1,8.0.1]";
filter:="(&(osgi.identity=keycloak-jetty9-adapter)(type=karaf.feature)(version>=8.0.1)(version<=8.0.1))" [caused by: Unable to resolve keycloak-jetty9-adapter/8.0.1: missing
requirement [keycloak-jetty9-adapter/8.0.1] osgi.identity;
osgi.identity=jetty; type=karaf.feature; version="[9.2.0,9.3.0)"]
If not to install that adapter, then I cannot reach my service (I receive 403 Forbidden) as it is protected via pax-web but authentication is impossible.
It seems that jetty within karaf is embedded and does not contain all features, which does not allow me to install jetty adapter.
According to the release notes for Apache Karaf 4.2.8, Karaf is using Jetty 9.4.22, which seems to be outside of your range in the error message version="[9.2.0,9.3.0)"
You'll either need to upgrade Keycloak to one that support Jetty 9.4.x
or downgrade Karaf to one that supports the same version of Jetty that Keycloak uses.
We are facing the issue while starting the informatica cluster service.
When starting Informatica cluster services, some scripts installing Ambari server on infabde, bdemaster and bdeslave.
The script is trying to install ambari on infabde again and again in loop, So the cluster service failed to start by saying that Ambari already installed in infabde. Its not trying to install to other two nodes.
Error Log:
2017-01-12 17:10:30,763 [localhost-startStop-1] INFO com.infa.products.ihs.service.ambari.ScriptLauncher- Waiting for Script's streams to end.
2017-01-12 17:10:41,210 [localhost-startStop-1] ERROR com.infa.products.ihs.beans.application.ClusterListener- [InfaHadoopServiceException_00047] The launch of Ambari server on host [infabde.lucidtechsol.com] failed because the host already has an installed Ambari server. You can add the host to another cluster.
com.infa.products.ihs.service.exception.InfaHadoopServiceException: [InfaHadoopServiceException_00047] The launch of Ambari server on host [infabde.hostname.com] failed because the host already has an installed Ambari server. You can add the host to another cluster.
Run the reset script
./ResetScript.sh true user#server.com user#server.com
./ResetScript.sh false user#server.com user#client.com
and enable IHS then.
ResetScript.sh can be found in services/Infahadoopserveice/binaries
I'm using the web client for web service. Client stub is generated with Axis 2. Everything is deployed on WebSphere 7.0.
When deployed on my developer machine (websphere instance run inside IBM RAD Eclipse) I'm connected with remote web service. However, on test machine, the error is thrown:
java.lang.NoClassDefFoundError: com.ibm.ws.wstx.handler.WSATGenerator (initialization failure)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:140)
at com.ibm.ws.wstx.WSTXClientTCMImpl.handleInbound(WSTXClientTCMImpl.java:100)
at com.ibm.ws.wstx.WSTXClientTCMImpl.cleanupContext(WSTXClientTCMImpl.java:81)
at org.apache.axis2.util.ThreadContextMigratorUtil.performContextCleanup(ThreadContextMigratorUtil.java:192)
at org.apache.axis2.jaxws.core.controller.impl.AxisInvocationController.postExecute(AxisInvocationController.java:657)
at org.apache.axis2.jaxws.core.controller.impl.AxisInvocationController.execute(AxisInvocationController.java:589)
at org.apache.axis2.jaxws.core.controller.impl.AxisInvocationController.doInvoke(AxisInvocationController.java:130)
at org.apache.axis2.jaxws.core.controller.impl.InvocationControllerImpl.invoke(InvocationControllerImpl.java:93)
at org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invokeSEIMethod(JAXWSProxyHandler.java:364)
at org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invoke(JAXWSProxyHandler.java:185)
at $Proxy94.getList(Unknown Source)
The ear that is deployed there is made by me on my development machine, so it's exactly the same code. So I suppose that it's some configuration issue. However, I have no idea, what part of configuration can be responsible for that.
So, I have the question, what is this WSATGenerator, and in which jar it should be available? Is it a standard library on WebSphere, or must it be manually configured? What configuration differences could cause that error thrown on test server and no by me?
I'm running Windows 7, the test machine is on Unix. Both machines are 64bit.
--edit--
Before NoClassDefFoundError, there is throws ExceptionInInitializerError:
java.lang.ExceptionInInitializerError
at java.lang.J9VMInternals.initialize(J9VMInternals.java:222)
at com.ibm.ws.wstx.handler.WSATGenerator.<clinit>(WSATGenerator.java:127)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:200)
at com.ibm.ws.wstx.WSTXClientTCMImpl.migrateThreadToContext(WSTXClientTCMImpl.java:61)
at org.apache.axis2.util.ThreadContextMigratorUtil.performMigrationToContext(ThreadContextMigratorUtil.java:163)
at org.apache.axis2.jaxws.core.controller.impl.AxisInvocationController.preExecute(AxisInvocationController.java:608)
at org.apache.axis2.jaxws.core.controller.impl.AxisInvocationController.execute(AxisInvocationController.java:570)
... 82 more
Caused by:
java.lang.ClassCastException: com.systinet.jaxrpc.rpc.ServiceFactoryImpl incompatible with com.ibm.wsspi.webservices.rpc.ServiceFactory
at com.ibm.ws.Transaction.wstx.WSATServices$1.run(WSATServices.java:83)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:63)
at com.ibm.ws.Transaction.wstx.WSATServices.<clinit>(WSATServices.java:74)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:200)
... 89 more
I am trying to get Django-Celery running on my Django App. I cannot get the worker server to run. When I try I get the message: No Connection could be made because the target machine actively refused it
Here is what I have done so far. First, I installed the django celery package: http://pypi.python.org/pypi/django-celery
I can load it into python without problems. I also installed the RabbitMQ server per the windows install instructions: http://www.rabbitmq.com/install.html#windows
Starting the tutorials in pytho on the RabbitMQ site I saw the need to install pika: http://pypi.python.org/pypi/pika. It imports without any problems.
From there I start the RabbitMQ server by running this at the command line: rabbitmq-service start
I get the message back that Service RabbitMQ started
Here is where I start to have problems.
I attempted the first steps in django-celery: http://packages.python.org/django-celery/getting-started/first-steps-with-django.html and the "hello world" example on the rabbitMQ site: http://www.rabbitmq.com/tutorials/tutorial-one-python.html
In both cases I get the message: No Connection could be made because the target machine actively refused it
My first thought was that this sounded like a firewall problem. So I went into the windows 7 firewall and added inbound and outbound rules to open the local and remote ports 5672 and 5673 to TCP protocol, but I still get the same error message.
When I run rabbitmqctl status i get the message:
Error: unable to connect to node 'rabbit#hostname': nodedown
diagnostics:
- nodes and their ports on hostname: [{rabbitmqctl18856, 505031}]
Does that mean it that it is trying to operate on those ports? what about the default 5672?
Any suggestions?
UPDATE: This was actually a problem resulting from several failed rabbitmq installs conflicting with the latest installation. If you have to remove rabbitmq use the 'rabbitmq-service remove' command and not SC DELETE, which cause a lot of problems for me and I had to go in and clean up my windows registry file.
The nodedown error indicated by rabbitmqctl suggests that the server isn't running on that machine.
Try going though the steps in RabbitMQ's troubleshooting guide. In particular, pay close attention to the logs. Has the server crashed for some reason? Could you post the logs somewhere?