I am trying to setup 3 Kafka brokers on ubuntu EC2 machines. But I am getting ConnectException while starting zookeeper. All the ports in the security group of my ec2 intsances are already open.
Below is the stack trace:
[2016-03-03 07:37:12,040] ERROR Exception while listening (org.apache.zookeeper.server.quorum.QuorumCnxManager)
java.net.BindException: Cannot assign requested address
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:330)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:507)
...
...
...
[2016-03-03 07:23:46,093] WARN Cannot open channel to 2 at election address /52.36.XXX.181:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager)
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
Below is the configuration of:
zookeeper.properties:
dataDir=/tmp/zookeeper
clientPort=2181
maxClientCnxns=0
server.1=52.37.XX.70:2888:3888
server.2=52.36.XXX.181:2888:3888
server.3=52.37.XX.42:2888:3888
initLimit=5
syncLimit=2
server.properties:
broker.id=1
port=9092
host.name=52.37.XX.70
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs
num.partitions=3
num.recovery.threads.per.data.dir=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
log.cleaner.enable=false
zookeeper.connect=52.37.XX.70:2181,52.36.XXX.181:2181,52.37.XX.42:2181
zookeeper.connection.timeout.ms=6000
I have added server's public IP in /etc/hosts of the instances. My modified /etc/hosts is as:
127.0.0.1 localhost localhost.localdomain ip-10-131-X-217
127.0.0.1 localhost localhost.localdomain 52.37.XX.70
127.0.0.1 localhost
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
The unique myid entries in /tmp/zookeper/myid are also entered correctly.
I have followed all the steps mentioned in: How to create a MultiNode - MultiBroker Cluster for Kafka on AWS
The issue was because I was using the Public IP of the server. Instead of that, use of Public DNS of ec2 instances fixed the issue.
Related
I am running a springboot app in docker using amazon ecs. The problem is the springboot application stops at connecting to datasource.
2022-04-13 01:35:25.473 INFO 1 --- [ main] com.query.QueryServiceApplication : Starting QueryServiceApplication v0.0.1-SNAPSHOT using Java 11.0.14.1 on 0ab5b5efda39 with PID 1 (/query-service.jar started by root in /)
2022-04-13 01:35:25.487 INFO 1 --- [ main] com.query.QueryServiceApplication : No active profile set, falling back to 1 default profile: "default"
2022-04-13 01:35:28.185 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2022-04-13 01:35:28.231 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 16 ms. Found 0 JPA repository interfaces.
2022-04-13 01:35:30.933 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2022-04-13 01:35:30.979 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2022-04-13 01:35:30.980 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.58]
2022-04-13 01:35:31.164 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2022-04-13 01:35:31.165 INFO 1 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 5477 ms
2022-04-13 01:35:32.967 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
After sometime i get the exception
java.sql.SQLNonTransientConnectionException: Got timeout reading communication packets
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110) ~[mysql-connector-java-8.0.28.jar!/:8.0.28]
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) ~[mysql-connector-java-8.0.28.jar!/:8.0.28]
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:829) ~[mysql-connector-java-8.0.28.jar!/:8.0.28]
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:449) ~[mysql-connector-java-8.0.28.jar!/:8.0.28]
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:242) ~[mysql-connector-java-8.0.28.jar!/:8.0.28]
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198) ~[mysql-connector-java-8.0.28.jar!/:8.0.28]
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138) ~[HikariCP-4.0.3.jar!/:na]
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364) ~[HikariCP-4.0.3.jar!/:na]
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206) ~[HikariCP-4.0.3.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476) ~[HikariCP-4.0.3.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561) ~[HikariCP-4.0.3.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115) ~[HikariCP-4.0.3.jar!/:na]
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112) ~[HikariCP-4.0.3.jar!/:na]
at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:159) ~[spring-jdbc-5.3.16.jar!/:5.3.16]
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:117) ~[spring-jdbc-5.3.16.jar!/:5.3.16]
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) ~[spring-jdbc-5.3.16.jar!/:5.3.16]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:330) ~[spring-jdbc-5.3.16.jar!/:5.3.16]
at org.springframework.boot.jdbc.EmbeddedDatabaseConnection.isEmbedded(EmbeddedDatabaseConnection.java:162) ~[spring-boot-2.6.4.jar!/:2.6.4]
at org.springframework.boot.autoconfigure.orm.jpa.HibernateDefaultDdlAutoProvider.getDefaultDdlAuto(HibernateDefaultDdlAutoProvider.java:42) ~[spring-boot-autoconfigure-2.6.4.jar!/:2.6.4]
at org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration.lambda$getVendorProperties$1(HibernateJpaConfiguration.java:130) ~[spring-boot-autoconfigure-2.6.4.jar!/:2.6.4]
at org.springframework.boot.autoconfigure.orm.jpa.HibernateSettings.getDdlAuto(HibernateSettings.java:41) ~[spring-boot-autoconfigure-2.6.4.jar!/:2.6.4]
at org.springframework.boot.autoconfigure.orm.jpa.HibernateProperties.determineDdlAuto(HibernateProperties.java:143) ~[spring-boot-autoconfigure-2.6.4.jar!/:2.6.4]
at org.springframework.boot.autoconfigure.orm.jpa.HibernateProperties.getAdditionalProperties(HibernateProperties.java:103) ~[spring-boot-autoconfigure-2.6.4.jar!/:2.6.4]
at org.springframework.boot.autoconfigure.orm.jpa.HibernateProperties.determineHibernateProperties(HibernateProperties.java:95) ~[spring-boot-autoconfigure-2.6.4.jar!/:2.6.4]
at org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration.getVendorProperties(HibernateJpaConfiguration.java:132) ~[spring-boot-autoconfigure-2.6.4.jar!/:2.6.4]
at org.springframework.boot.autoconfigure.orm.jpa.JpaBaseConfiguration.entityManagerFactory(JpaBaseConfiguration.java:132) ~[spring-boot-autoconfigure-2.6.4.jar!/:2.6.4]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.3.16.jar!/:5.3.16]
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653) ~[spring-beans-5.3.16.jar!/:5.3.16]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:638) ~[spring-beans-5.3.16.jar!/:5.3.16]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352) ~[spring-beans-5.3.16.jar!/:5.3.16]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195) ~[spring-beans-5.3.16.jar!/:5.3.16]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.16.jar!/:5.3.16]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.16.jar!/:5.3.16]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.16.jar!/:5.3.16]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.16.jar!/:5.3.16]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.16.jar!/:5.3.16]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.16.jar!/:5.3.16]
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1154) ~[spring-context-5.3.16.jar!/:5.3.16]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:908) ~[spring-context-5.3.16.jar!/:5.3.16]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) ~[spring-context-5.3.16.jar!/:5.3.16]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145) ~[spring-boot-2.6.4.jar!/:2.6.4]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:740) ~[spring-boot-2.6.4.jar!/:2.6.4]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:415) ~[spring-boot-2.6.4.jar!/:2.6.4]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:303) ~[spring-boot-2.6.4.jar!/:2.6.4]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1312) ~[spring-boot-2.6.4.jar!/:2.6.4]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1301) ~[spring-boot-2.6.4.jar!/:2.6.4]
at com.query.QueryServiceApplication.main(QueryServiceApplication.java:10) ~[classes!/:0.0.1-SNAPSHOT]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[query-service.jar:0.0.1-SNAPSHOT]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[query-service.jar:0.0.1-SNAPSHOT]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[query-service.jar:0.0.1-SNAPSHOT]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88) ~[query-service.jar:0.0.1-SNAPSHOT]
I check for security groups as instructed in post link. I made the RDS publicly accessible and allowed access from all IP's and all protocols in the security group. Both my RDS and ECS are in same VPC and same security group. I am not really sure what else to do here ? Will there be an issue connecting since its a docker container ?
I made the RDS publicly accessible
That could actually be a problem. The ECS task may be resolving the RDS endpoint to an Internet IP address, and then trying to connect to the database over the Internet, instead of keeping the traffic inside the VPC. If the ECS task doesn't have a public IP address, or it isn't running in a subnet with a route to a NAT gateway, then it wouldn't be able to connect.
Both my RDS and ECS are in same VPC and same security group.
Being in the same security group does not confer any connectivity by default. This is actually bad practice because it doesn't allow you to restrict traffic on specific ports, since you are mixing different services that use different ports in the same security group. Since you allowed access from all IPs and protocols this shouldn't be the cause of your current issue however.
Some steps to debug this would be to use ECS Exec, or spin up an EC2 instance in the same VPC subnet (with the same security group), and see if the RDS endpoint resolves to an internal or external IP, and see if you can connect with the MySQL command line.
This question about "aws sub-generator" (not "aws-containers sub-generator").
Question 1:
Is this generator supposed to update "application-prod.yml"? I can see the information I provided while running the sub-generator including the database name in ".yo-rc.json". But database user-name and password are neither there nor in the "application-prod.yml".
Question 2:
I generated a simple JHipster project with some entities. It was working fine locally on my PC. I used the "aws sub-generator" and everything went well. It created the Elastic Beanstalk application, an EC2 instance, the S3 bucket, the RDS database, the security groups...Here is the event log of the Elastic Beanstalk:
Successfully launched environment: alirezatodoapplicationapril-env
Environment health has been set to GREEN
Created CloudWatch alarm named: awseb-e-i2vaqsmdbh-stack-AWSEBCloudwatchAlarmHigh-18WDVCSH132VM
Created CloudWatch alarm named: awseb-e-i2vaqsmdbh-stack-AWSEBCloudwatchAlarmLow-EA2GT2HFN77U
Created Auto Scaling group policy named: arn:aws:autoscaling:us-east-1:827968211260:scalingPolicy:5cf0c635-40b7-4008-9ebe-57823a3f9b3c:autoScalingGroupName/awseb-e-i2vaqsmdbh-stack-AWSEBAutoScalingGroup-1SLCXWGKICYGM:policyName/awseb-e-i2vaqsmdbh-stack-AWSEBAutoScalingScaleDownPolicy-JHTI8C9VQTF1
Created Auto Scaling group policy named: arn:aws:autoscaling:us-east-1:827968211260:scalingPolicy:5e6f86c4-79f7-4504-a6d2-8a8a9c638ef5:autoScalingGroupName/awseb-e-i2vaqsmdbh-stack-AWSEBAutoScalingGroup-1SLCXWGKICYGM:policyName/awseb-e-i2vaqsmdbh-stack-AWSEBAutoScalingScaleUpPolicy-E1W03HEXTE6R
Waiting for EC2 instances to launch. This may take a few minutes.
Created Auto Scaling group named: awseb-e-i2vaqsmdbh-stack-AWSEBAutoScalingGroup-1SLCXWGKICYGM
Added EC2 instance 'i-074488994cedde8c7' to Auto Scaling Group 'awseb-e-i2vaqsmdbh-stack-AWSEBAutoScalingGroup-1SLCXWGKICYGM'.
Adding instance 'i-074488994cedde8c7' to your environment.
Created Auto Scaling launch configuration named: awseb-e-i2vaqsmdbh-stack-AWSEBAutoScalingLaunchConfiguration-1F3VDL1LKSD1B
Created load balancer named: awseb-e-i-AWSEBLoa-18MWGKA8O6FHP
Created security group named: awseb-e-i2vaqsmdbh-stack-AWSEBSecurityGroup-1BS14HUVJLFUD
Created security group named: sg-874777cd
Using elasticbeanstalk-us-east-1-827968211260 as Amazon S3 storage bucket for environment data.
createEnvironment is starting.
But, now when I use the link of the Beanstalk application it gives me a 404 error. What is missed? Here is the log of application startup (catalina.out):
██╗ ██╗ ██╗ ████████╗ ███████╗ ██████╗ ████████╗ ████████╗ ███████╗
██║ ██║ ██║ ╚══██╔══╝ ██╔═══██╗ ██╔════╝ ╚══██╔══╝ ██╔═════╝ ██╔═══██╗
██║ ████████║ ██║ ███████╔╝ ╚█████╗ ██║ ██████╗ ███████╔╝
██╗ ██║ ██╔═══██║ ██║ ██╔════╝ ╚═══██╗ ██║ ██╔═══╝ ██╔══██║
╚██████╔╝ ██║ ██║ ████████╗ ██║ ██████╔╝ ██║ ████████╗ ██║ ╚██╗
╚═════╝ ╚═╝ ╚═╝ ╚═══════╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══════╝ ╚═╝ ╚═╝
:: JHipster 🤓 :: Running Spring Boot 2.0.3.RELEASE ::
:: https://www.jhipster.tech ::
2018-07-31 20:18:22.679 ERROR 3398 --- [ost-startStop-1] o.s.boot.SpringApplication : Application run failed
java.lang.IllegalStateException: Could not locate PropertySource and the fail fast property is set, failing
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.locate(ConfigServicePropertySourceLocator.java:136)
at org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration.initialize(PropertySourceBootstrapConfiguration.java:94)
at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:633)
at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:373)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:325)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.run(SpringBootServletInitializer.java:155)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:135)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.onStartup(SpringBootServletInitializer.java:87)
at org.springframework.web.SpringServletContainerInitializer.onStartup(SpringServletContainerInitializer.java:172)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5204)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1141)
at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1875)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://localhost:8761/config/ToDo/prod/master": Connection refused (Connection refused); nested exception is java.net.ConnectException: Connection refused (Connection refused)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:732)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:680)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:600)
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.getRemoteEnvironment(ConfigServicePropertySourceLocator.java:218)
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.locate(ConfigServicePropertySourceLocator.java:96)
... 20 common frames omitted
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
at sun.net.www.http.HttpClient.New(HttpClient.java:339)
at sun.net.www.http.HttpClient.New(HttpClient.java:357)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984)
at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:76)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:723)
... 24 common frames omitted
31-Jul-2018 20:18:22.688 SEVERE [localhost-startStop-1] org.apache.catalina.core.ContainerBase.addChildInternal ContainerBase.addChild: start:
org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[]]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:167)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1141)
at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1875)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Could not locate PropertySource and the fail fast property is set, failing
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.locate(ConfigServicePropertySourceLocator.java:136)
at org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration.initialize(PropertySourceBootstrapConfiguration.java:94)
at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:633)
at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:373)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:325)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.run(SpringBootServletInitializer.java:155)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:135)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.onStartup(SpringBootServletInitializer.java:87)
at org.springframework.web.SpringServletContainerInitializer.onStartup(SpringServletContainerInitializer.java:172)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5204)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
... 10 more
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://localhost:8761/config/ToDo/prod/master": Connection refused (Connection refused); nested exception is java.net.ConnectException: Connection refused (Connection refused)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:732)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:680)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:600)
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.getRemoteEnvironment(ConfigServicePropertySourceLocator.java:218)
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.locate(ConfigServicePropertySourceLocator.java:96)
... 20 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
at sun.net.www.http.HttpClient.New(HttpClient.java:339)
at sun.net.www.http.HttpClient.New(HttpClient.java:357)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984)
at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:76)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:723)
... 24 more
31-Jul-2018 20:18:22.690 SEVERE [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Error deploying web application directory [/var/lib/tomcat8/webapps/ROOT]
java.lang.IllegalStateException: ContainerBase.addChild: start: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[]]
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:758)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1141)
at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1875)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
31-Jul-2018 20:18:22.691 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/var/lib/tomcat8/webapps/ROOT] has finished in [16,376] ms
31-Jul-2018 20:18:22.695 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
31-Jul-2018 20:18:22.727 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
31-Jul-2018 20:18:22.742 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 16499 ms
The JHipster AWS ElasticBeanstalk option only works with monoliths and won't deploy the JHipster Registry if that option is chosen when generating the project.
If you want to disable the JHipster Registry feature, you can set two config variables in your project. In bootstrap-prod.yml, set spring.cloud.config.enabled: false and disable Eureka in application-prod.yml with eureka.client.enabled: false.
In regards to the database connection settings, those are set as environment variables when creating the ElasticBeanstalk environment which Spring Boot uses instead of the hardcoded config in application-prod.yml
We are using neo4j-enterprise-3.0.4 in cluster on AWS with bolt protocol. HAproxy is used to we know who is master and who are the slaves in the HA cluster. The Java application is using the spring-data-neo4j framework and is deployed on a Tomcat 7.
Structure on AWS is: Neo4j HA Cluster (1 master and 2 slaves) ---> HAproxy ----> ELB
We are receiving this error. An error does not occur every time but occasionally
(yesterday 3 times).
org.neo4j.ogm.exception.ConnectionException: Error connecting to graph database using Bolt: N/A, Unable to close socket connection properly.Broken pipe
at org.neo4j.ogm.drivers.bolt.driver.BoltDriver.newSession(BoltDriver.java:97) ~[neo4j-ogm-bolt-driver-2.0.1.jar:?]
at org.neo4j.ogm.drivers.bolt.driver.BoltDriver.newTransaction(BoltDriver.java:71) ~[neo4j-ogm-bolt-driver-2.0.1.jar:?]
at org.neo4j.ogm.session.transaction.DefaultTransactionManager.openTransaction(DefaultTransactionManager.java:57) ~[neo4j-ogm-core-2.0.1.jar:?]
at org.neo4j.ogm.drivers.bolt.request.BoltRequest.executeRequest(BoltRequest.java:153) ~[neo4j-ogm-bolt-driver-2.0.1.jar:?]
at org.neo4j.ogm.drivers.bolt.request.BoltRequest.execute(BoltRequest.java:139) ~[neo4j-ogm-bolt-driver-2.0.1.jar:?]
at org.neo4j.ogm.session.delegates.ExecuteQueriesDelegate.query(ExecuteQueriesDelegate.java:93) ~[neo4j-ogm-core-2.0.1.jar:?]
at org.neo4j.ogm.session.delegates.ExecuteQueriesDelegate.query(ExecuteQueriesDelegate.java:73) ~[neo4j-ogm-core-2.0.1.jar:?]
at org.neo4j.ogm.session.Neo4jSession.query(Neo4jSession.java:313) ~[neo4j-ogm-core-2.0.1.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_92]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_92]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_92]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_92]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at com.sun.proxy.$Proxy73.query(Unknown Source) ~[?:?]
at org.springframework.data.neo4j.template.Neo4jTemplate.query(Neo4jTemplate.java:217) ~[spring-data-neo4j-4.1.1.RELEASE.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_92]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_92]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_92]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_92]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136) ~[spring-tx-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at com.sun.proxy.$Proxy75.query(Unknown Source) ~[?:?]
at com.kubid.booking.service.bookings.SnapCardBookingService.getBookingsInternal(SnapCardBookingService.java:216) ~[classes/:?]
at com.kubid.booking.service.bookings.SnapCardBookingService.getBookings(SnapCardBookingService.java:249) ~[classes/:?]
at com.kubid.booking.service.bookings.SnapCardBookingService$$FastClassBySpringCGLIB$$d05944f4.invoke(<generated>) ~[spring-core-4.2.0.RELEASE.jar:?]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:649) ~[spring-aop-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at com.kubid.booking.service.bookings.SnapCardBookingService$$EnhancerBySpringCGLIB$$bfd16c77.getBookings(<generated>) ~[spring-core-4.2.0.RELEASE.jar:?]
at com.kubid.booking.service.controller.BookingController.getBookings(BookingController.java:264) ~[classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_92]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_92]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_92]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_92]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) ~[spring-web-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137) ~[spring-web-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:111) ~[spring-webmvc-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:806) ~[spring-webmvc-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:729) ~[spring-webmvc-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959) [spring-webmvc-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) [spring-webmvc-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861) [spring-webmvc-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:624) [servlet-api.jar:?]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) [servlet-api.jar:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) [catalina.jar:7.0.65]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.65]
at org.apache.catalina.filters.CorsFilter.handleNonCORS(CorsFilter.java:436) [catalina.jar:7.0.65]
at org.apache.catalina.filters.CorsFilter.doFilter(CorsFilter.java:177) [catalina.jar:7.0.65]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.65]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.65]
at com.kubid.booking.service.config.RequestBodyFilter.doFilter(RequestBodyFilter.java:36) [classes/:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.65]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.65]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) [catalina.jar:7.0.65]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) [catalina.jar:7.0.65]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) [catalina.jar:7.0.65]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) [catalina.jar:7.0.65]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) [catalina.jar:7.0.65]
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) [catalina.jar:7.0.65]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) [catalina.jar:7.0.65]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423) [catalina.jar:7.0.65]
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079) [tomcat-coyote.jar:7.0.65]
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) [tomcat-coyote.jar:7.0.65]
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) [tomcat-coyote.jar:7.0.65]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-coyote.jar:7.0.65]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
Caused by: org.neo4j.driver.v1.exceptions.ClientException: Unable to close socket connection properly.Broken pipe
at org.neo4j.driver.internal.connector.socket.SocketClient.stop(SocketClient.java:157) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.connector.socket.SocketConnection.close(SocketConnection.java:181) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.connector.ConcurrencyGuardingConnection.close(ConcurrencyGuardingConnection.java:163) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.PooledConnection.dispose(PooledConnection.java:180) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.InternalConnectionPool$1.onDispose(InternalConnectionPool.java:196) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.InternalConnectionPool$1.onDispose(InternalConnectionPool.java:178) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.ThreadCachingPool.dispose(ThreadCachingPool.java:197) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.ThreadCachingPool.acquireFromGlobal(ThreadCachingPool.java:145) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.ThreadCachingPool.acquire(ThreadCachingPool.java:118) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.InternalConnectionPool.acquire(InternalConnectionPool.java:107) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.InternalDriver.session(InternalDriver.java:53) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.ogm.drivers.bolt.driver.BoltDriver.newSession(BoltDriver.java:95) ~[neo4j-ogm-bolt-driver-2.0.1.jar:?]
... 79 more
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:1.8.0_92]
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:1.8.0_92]
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[?:1.8.0_92]
at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[?:1.8.0_92]
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) ~[?:1.8.0_92]
at org.neo4j.driver.internal.connector.socket.TLSSocketChannel.close(TLSSocketChannel.java:452) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.connector.socket.SocketClient.stop(SocketClient.java:145) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.connector.socket.SocketConnection.close(SocketConnection.java:181) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.connector.ConcurrencyGuardingConnection.close(ConcurrencyGuardingConnection.java:163) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.PooledConnection.dispose(PooledConnection.java:180) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.InternalConnectionPool$1.onDispose(InternalConnectionPool.java:196) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.InternalConnectionPool$1.onDispose(InternalConnectionPool.java:178) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.ThreadCachingPool.dispose(ThreadCachingPool.java:197) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.ThreadCachingPool.acquireFromGlobal(ThreadCachingPool.java:145) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.ThreadCachingPool.acquire(ThreadCachingPool.java:118) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.pool.InternalConnectionPool.acquire(InternalConnectionPool.java:107) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.driver.internal.InternalDriver.session(InternalDriver.java:53) ~[neo4j-java-driver-1.0.0-RC1.jar:1.0.0-RC1-cc9a365a0ad5346cb6ba50f3b68a998b4d956b98]
at org.neo4j.ogm.drivers.bolt.driver.BoltDriver.newSession(BoltDriver.java:95) ~[neo4j-ogm-bolt-driver-2.0.1.jar:?]
This is our HAproxy .cfg:
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin
stats timeout 30s
user haproxy
group haproxy
daemon
maxconn 256
defaults
log global
mode tcp
option tcplog
option dontlognull
timeout connect 30s
timeout client 2h
timeout server 2h
frontend http-in
bind *:81
acl write_method method POST DELETE PUT
acl write_hdr hdr_val(X-Write) eq 1
acl write_payload payload(0,0) -m reg -i CREATE|MERGE|SET|DELETE|REMOVE
acl tx_cypher_endpoint path_beg /db/data/transaction
http-request set-var(txn.tx_cypher_endpoint) bool(true) if tx_cypher_endpoint
use_backend neo4j-master if write_hdr
use_backend neo4j-master if tx_cypher_endpoint write_payload
use_backend neo4j-all if tx_cypher_endpoint
use_backend neo4j-master if write_method
default_backend neo4j-all
frontend http-browse
bind *:83
mode http
default_backend neo4j-browser
backend neo4j-all
option httpchk GET /db/manage/server/ha/available HTTP/1.0\r\nAuthorization:\ Basic\ [code]
acl tx_cypher_endpoint var(txn.tx_cypher_endpoint),bool
stick-table type integer size 1k expire 70s
stick match path,word(4,/) if tx_cypher_endpoint
stick store-response hdr(Location),word(6,/) if tx_cypher_endpoint
server neo4j-1 192.0.0.250:7687 check port 7474
server neo4j-2 192.0.0.251:7687 check port 7474
server neo4j-3 192.0.0.252:7687 check port 7474
backend neo4j-master
option httpchk GET /db/manage/server/ha/master HTTP/1.0\r\nAuthorization:\ Basic\ [code]
server neo4j-1 192.0.0.250:7687 check port 7474
server neo4j-2 192.0.0.251:7687 check port 7474
server neo4j-3 192.0.0.252:7687 check port 7474
backend neo4j-browser
mode http
option prefer-last-server
option httpchk GET /db/manage/server/ha/master HTTP/1.0\r\nAuthorization:\ Basic\ [code]
server neo4j-1 192.0.0.250:7474 check
server neo4j-2 192.0.0.251:7474 check
server neo4j-3 192.0.0.252::7474 check
listen admin
bind *:82
mode http
stats enable
stats uri /haproxy?stats
stats realm Haproxy\ Statistics
stats auth admin:admin
We can not figure out where is a problem. Is it problem in:
1) org.neo4j.ogm.drivers.bolt.driver.BoltDriver
2) ELB on AWS
3) HAproxy .cfg
4) Bolt protocol on AWS
Before we were using Bolt as protocol, we used the HTTP endpoint, with the same settings on AWS and we did not had problem.
EDIT
We change configuration so this is what we know about our questions:
1) org.neo4j.ogm.drivers.bolt.driver.BoltDriver - error is only appear here on Tomcat 7.
2) ELB on AWS - we not using ELB any more we are connecting directly to HAproxy
3) HAproxy .cfg - our PHP team using port 83 to connect to Neo4j HA Cluster using http protocol and they do not have errors.
4) Bolt protocol on AWS - we add logs on ELB and EC2 and we did not receive any error about Bolt protocol.
Is there a problem with neo4j-enterprise-3.0.4 and neo4j-ogm-bolt-driver 2.0.4?
EDIT
There is a issue between neo4j-enterprise-3.0.4 or neo4j-ogm-bolt-driver 2.0.4 and AWS ELB or both.
We change our app settings and we are using Http driver with same structure on AWS instead of Bolt driver. We did not receive error.
We did not find a way to fix this issue.
I am using Hazelcast v3.6 on two amazon AWS virtual machines (not using the AWS specific settings for hazelcast). The connection is supposed to work via TCP/IP connection settings (not multicasting). I have opened 5701-5801 address for connection on the virtual machines.
I have tried using iperf on the two virtual machines using which I can see that the client on one VM connects to the server on another VM (and vice versa when I switch the client server setup for iperf).
When I launch two Hazelcast servers on different VM's, the connection is not established. The log statements and the hazelcast.xml config are given below (I am not using the programmatic settings for Hazelcast). I have changed the IP addresses below:
20160401-16:41:02.812 [cached2] InitConnectionTask INFO - [45.46.47.48]:5701 [dev] [3.6] Connecting to /22.23.24.25:5701, timeout: 0, bind-any: true
20160401-16:41:02.812 [cached3] InitConnectionTask INFO - [45.46.47.48]:5701 [dev] [3.6] Connecting to /22.23.24.25:5703, timeout: 0, bind-any: true
20160401-16:41:02.813 [cached1] InitConnectionTask INFO - [45.46.47.48]:5701 [dev] [3.6] Connecting to /22.23.24.25:5702, timeout: 0, bind-any: true
20160401-16:41:02.816 [cached1] InitConnectionTask INFO - [45.46.47.48]:5701 [dev] [3.6] Could not connect to: /22.23.24.25:5702. Reason: SocketException[Connection refused to address /22.23.24.25:570
2]
20160401-16:41:02.816 [cached1] TcpIpJoiner INFO - [45.46.47.48]:5701 [dev] [3.6] Address[22.23.24.25]:5702 is added to the blacklist.
20160401-16:41:02.817 [cached3] InitConnectionTask INFO - [45.46.47.48]:5701 [dev] [3.6] Could not connect to: /22.23.24.25:5703. Reason: SocketException[Connection refused to address /22.23.24.25:570
3]
20160401-16:41:02.817 [cached3] TcpIpJoiner INFO - [45.46.47.48]:5701 [dev] [3.6] Address[22.23.24.25]:5703 is added to the blacklist.
20160401-16:41:02.834 [cached2] TcpIpConnectionManager INFO - [45.46.47.48]:5701 [dev] [3.6] Established socket connection between /45.46.47.48:51965 and /22.23.24.25:5701
20160401-16:41:02.849 [hz._hzInstance_1_dev.IO.thread-in-0] TcpIpConnection INFO - [45.46.47.48]:5701 [dev] [3.6] Connection [Address[22.23.24.25]:5701] lost. Reason: java.io.EOFException[Remote socket
closed!]
20160401-16:41:02.851 [hz._hzInstance_1_dev.IO.thread-in-0] NonBlockingSocketReader WARN - [45.46.47.48]:5701 [dev] [3.6] hz._hzInstance_1_dev.IO.thread-in-0 Closing socket to endpoint Address[54.89.161.2
28]:5701, Cause:java.io.EOFException: Remote socket closed!
20160401-16:41:03.692 [cached2] InitConnectionTask INFO - [45.46.47.48]:5701 [dev] [3.6] Connecting to /22.23.24.25:5701, timeout: 0, bind-any: true
20160401-16:41:03.693 [cached2] TcpIpConnectionManager INFO - [45.46.47.48]:5701 [dev] [3.6] Established socket connection between /45.46.47.48:60733 and /22.23.24.25:5701
20160401-16:41:03.696 [hz._hzInstance_1_dev.IO.thread-in-1] TcpIpConnection INFO - [45.46.47.48]:5701 [dev] [3.6] Connection [Address[22.23.24.25]:5701] lost. Reason: java.io.EOFException[Remote socket
closed!]
Part of Hazelcast config
<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-3.6.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<group>
<name>abc</name>
<password>defg</password>
</group>
<network>
<port auto-increment="true" port-count="100">5701</port>
<outbound-ports>
<ports>0-5900</ports>
</outbound-ports>
<join>
<multicast enabled="false">
<!--<multicast-group>224.2.2.3</multicast-group>
<multicast-port>54327</multicast-port>-->
</multicast>
<tcp-ip enabled="true">
<member>22.23.24.25</member>
</tcp-ip>
</join>
<interfaces enabled="true">
<interface>45.46.47.48</interface>
</interfaces>
<ssl enabled="false" />
<socket-interceptor enabled="false" />
<symmetric-encryption enabled="false">
<algorithm>PBEWithMD5AndDES</algorithm>
<!-- salt value to use when generating the secret key -->
<salt>thesalt</salt>
<!-- pass phrase to use when generating the secret key -->
<password>thepass</password>
<!-- iteration count to use when generating the secret key -->
<iteration-count>19</iteration-count>
</symmetric-encryption>
</network>
<partition-group enabled="false"/>
iperf server and client log statements
Server listening on TCP port 5701
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 22.23.24.25, TCP port 5701
TCP window size: 1.33 MByte (default)
------------------------------------------------------------
[ 5] local 172.31.17.104 port 57398 connected with 22.23.24.25 port 5701
[ 4] local 172.31.17.104 port 5701 connected with 22.23.24.25 port 55589
[ ID] Interval Transfer Bandwidth
[ 5] 0.0-10.0 sec 662 MBytes 555 Mbits/sec
[ 4] 0.0-10.0 sec 797 MBytes 666 Mbits/sec
Server listening on TCP port 5701
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local xxx.xx.xxx.xx port 5701 connected with 22.23.24.25 port 57398
------------------------------------------------------------
Client connecting to 22.23.24.25, TCP port 5701
TCP window size: 1.62 MByte (default)
------------------------------------------------------------
[ 6] local 172.31.17.23 port 55589 connected with 22.23.24.25 port 5701
[ ID] Interval Transfer Bandwidth
[ 6] 0.0-10.0 sec 797 MBytes 669 Mbits/sec
[ 4] 0.0-10.0 sec 662 MBytes 553 Mbits/sec
Note:
I forgot to mention that I can connect from hazelcast client to server i.e. when I use a hazelcast client to connect to a single hazlecast server node, I am able to connect just fine
An outbound ports range which includes 0 is interpreted by hazelcast as "use ephemeral ports", so the <outbound-ports> element has actually no effect in your configuration. There is an associated test in hazelcast sources: https://github.com/hazelcast/hazelcast/blob/75251c4f01d131a9624fc3d0c4190de5cdf7d93a/hazelcast/src/test/java/com/hazelcast/nio/NodeIOServiceTest.java#L60
I'm trying to connect to a SOAP webserver, I was given a certificate but I get the below exception.
I did:
$openssl pkcs12 -in cert.pfx -out certificate.cer -nodes
$keytool -import -alias webserver -file certificate.cer
Certificate was added to keystore
I have a junit test that calls the remote server with these JVM options:
-ea -Djavax.net.debug=all -Dcom.sun.xml.ws.transport.http.client.HttpTransportPipe.dump=true
-Dcom.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.dump=true
-Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true
-Dcom.sun.xml.internal.ws.transport.http.HttpAdapter.dump=true
-Djavax.net.ssl.trustStore=certificate.cer
-Djavax.net.ssl.trustStoreType=PEM
-Djavax.net.ssl.trustStorePassword=pwd
-Djavax.net.ssl.keyStore=certificate.cer
-Djavax.net.ssl.keyStoreType=PEM
-Djavax.net.ssl.keyStorePassword=pwd
the remote call is done using:
authenticationService.getAuthenticationServiceSoap().authenticateWithPasswordAuth(clientId, username, password);
where the client I'm using has got these annotations:
#WebServiceClient
public class AuthenticationService extends Service {
and
#WebEndpoint(name = "AuthenticationServiceSoap")
public AuthenticationServiceSoap getAuthenticationServiceSoap()
I don't do anything else code wise, and I get the below Exception:
I don't even understand if java is trying to use https or no.
---[HTTP request - http://..../Soap/AuthenticationService.asmx]--- Accept: [text/xml, multipart/related] Content-Type: [text/xml;
charset=utf-8] SOAPAction:
["http://....../AuthenticateWithPasswordAuth"] User-Agent: [JAX-WS RI
2.2.4-b01]
Caused by: java.net.ConnectException: Connection refused at
java.net.PlainSocketImpl.socketConnect(Native Method) at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
java.net.Socket.connect(Socket.java:579) at
java.net.Socket.connect(Socket.java:528) at
sun.net.NetworkClient.doConnect(NetworkClient.java:180) at
sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at
sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at
sun.net.www.http.HttpClient.(HttpClient.java:211) at
sun.net.www.http.HttpClient.New(HttpClient.java:308) at
sun.net.www.http.HttpClient.New(HttpClient.java:326) at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:997)
at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:976)
at
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:851)
at
sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1092)
at
com.sun.xml.internal.ws.transport.http.client.HttpClientTransport.getOutput(HttpClientTransport.java:105)