I am trying to create the redshift connection using redshift jdbc driver which I downloaded from AWS redshift cluster console. Getting below exception
java.sql.SQLException: The connection attempt failed.
at com.amazon.redshift.util.RedshiftException.getSQLException(RedshiftException.java:48)
at com.amazon.redshift.Driver.connect(Driver.java:303)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)
at com.redshift.jdbc.test.ConnectionTest.main(ConnectionTest.java:27)
Caused by: java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.waitForConnect(Native Method)
at java.base/java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:107)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
at java.base/java.net.Socket.connect(Socket.java:591)
at com.amazon.redshift.core.RedshiftStream.<init>(RedshiftStream.java:86)
at com.amazon.redshift.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:101)
at com.amazon.redshift.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:214)
at com.amazon.redshift.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
at com.amazon.redshift.jdbc.RedshiftConnectionImpl.<init>(RedshiftConnectionImpl.java:288)
at com.amazon.redshift.Driver.makeConnection(Driver.java:466)
at com.amazon.redshift.Driver.connect(Driver.java:279)
... 3 more
code :
String jdbcURL = "jdbc:redshift://host:5439/database";
String user = "*****";
String password = "****";
String driver = "com.amazon.redshift.jdbc42.Driver";
Connection connection = null;
Statement statement = null;
try {
Class.forName(driver);
connection = DriverManager.getConnection(jdbcURL, user, password);
if (connection != null) {
System.out.println("Connection created!");
}
} catch (Exception ex) {
ex.printStackTrace();
} finally {
try {
if (connection != null)
connection.close();
} catch (Exception e) {
e.printStackTrace();
}
}
Can anyone please help me why I'm unable to make the connection?
I guess Caused by: java.net.SocketTimeoutException: connect timed out is a key, here's a few possibilities:
your ip is not whitelisted in redshift security group
if the code above is exactly what you're running then
String jdbcURL = "jdbc:redshift://host:5439/database"; doesn't contain a correct host and database, it myst something like
your-cluster-name.blahblahblah.eu-central-1.redshift.amazonaws.com:5439/your-database-name you can find this jobs url in redshift console -> cluster general info
UPDATE
Check your IP whitelist in security group
and add your IP if it's not here
Related
I have the following class written in Java using Eclipse on my Amazon EC2 instance.
import java.nio.ByteBuffer;
import com.amazonaws.auth.*;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.secretsmanager.*;
import com.amazonaws.services.secretsmanager.model.*;
public class SMtest {
public static void main(String[] args) {
String x = getSecret();
System.out.println(x);
}
public SMtest()
{
}
public static String getSecret() {
String secretName = "mysecret";
String endpoint = "secretsmanager.us-east-1.amazonaws.com";
String region = "us-east-1";
String result = "";
//BasicAWSCredentials awsCreds = new BasicAWSCredentials("mypublickey", "mysecretkey");
AwsClientBuilder.EndpointConfiguration config = new AwsClientBuilder.EndpointConfiguration(endpoint, region);
AWSSecretsManagerClientBuilder clientBuilder = AWSSecretsManagerClientBuilder.standard()
.withCredentials(new InstanceProfileCredentialsProvider(false));
//.withCredentials(new AWSStaticCredentialsProvider(awsCreds));
clientBuilder.setEndpointConfiguration(config);
AWSSecretsManager client = clientBuilder.build();
String secret;
ByteBuffer binarySecretData;
GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest()
.withSecretId(secretName).withVersionStage("AWSCURRENT");
GetSecretValueResult getSecretValueResult = null;
try {
getSecretValueResult = client.getSecretValue(getSecretValueRequest);
} catch(ResourceNotFoundException e) {
System.out.println("The requested secret " + secretName + " was not found");
} catch (InvalidRequestException e) {
System.out.println("The request was invalid due to: " + e.getMessage());
} catch (InvalidParameterException e) {
System.out.println("The request had invalid params: " + e.getMessage());
}
if(getSecretValueResult == null) {
result = "";
}
// Depending on whether the secret was a string or binary, one of these fields will be populated
if(getSecretValueResult.getSecretString() != null) {
secret = getSecretValueResult.getSecretString();
result = secret;
//System.out.println(secret);
}
else {
binarySecretData = getSecretValueResult.getSecretBinary();
result = binarySecretData.toString();
}
return result;
}
}
When I execute it from within Eclipse, it works just fine. When I compile the class to use it in ColdFusion (2021) from the same EC2 with the following code:
<cfscript>
obj = CreateObject("java","SMtest");
obj.init();
result = obj.getSecret();
</cfscript>
<cfoutput>#result#</cfoutput>
I get a "Failed to connect to service endpoint" error. I believe I have all the IAM credentials set up properly since it is working in straight Java. However, when I change the credentials to use the Basic Credentials with my AWS Public and Secret Key (shown in comments in the code above), it works in both Java and ColdFusion.
I created an AWS Policy that manages the Secrets Manager permissions. I also added AmazonEC2FullAccess. I also tried to create a VPC Endpoint, but this had no effect.
Why would it be working in Java and not in ColdFusion (which is based on Java) ? What roles/policies would I have to add to get it to work in ColdFusion when it is already working in Java?
STACK TRACE ADDED:
com.amazonaws.SdkClientException: Failed to connect to service endpoint:
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100)
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.getToken(InstanceMetadataServiceResourceFetcher.java:91)
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:69)
at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66)
at com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.getCredentialsEndpoint(InstanceMetadataServiceCredentialsFetcher.java:58)
at com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.getCredentialsResponse(InstanceMetadataServiceCredentialsFetcher.java:46)
at com.amazonaws.auth.BaseCredentialsFetcher.fetchCredentials(BaseCredentialsFetcher.java:112)
at com.amazonaws.auth.BaseCredentialsFetcher.getCredentials(BaseCredentialsFetcher.java:68)
at com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:165)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1266)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:842)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:792)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:779)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:753)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:713)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:695)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:539)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.doInvoke(AWSSecretsManagerClient.java:2454)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.invoke(AWSSecretsManagerClient.java:2421)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.invoke(AWSSecretsManagerClient.java:2410)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.executeGetSecretValue(AWSSecretsManagerClient.java:943)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.getSecretValue(AWSSecretsManagerClient.java:912)
at SMtest.getSecret(SMtest.java:52)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at coldfusion.runtime.java.JavaProxy.invoke(JavaProxy.java:106)
at coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:4254)
at coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:4217)
at cfsmtest2ecfm1508275519.runPage(D:\ColdFusion2021\cfusion\wwwroot\smtest.cfm:4)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:257)
at coldfusion.tagext.lang.IncludeTag.handlePageInvoke(IncludeTag.java:749)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:578)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:573)
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:43)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:162)
at coldfusion.filter.IpFilter.invoke(IpFilter.java:45)
at coldfusion.filter.LicenseFilter.invoke(LicenseFilter.java:30)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:97)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:81)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:60)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)
at coldfusion.CfmServlet.service(CfmServlet.java:231)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:311)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:228)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:46)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
at coldfusion.inspect.weinre.MobileDeviceDomInspectionFilter.doFilter(MobileDeviceDomInspectionFilter.java:57)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:373)
at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:462)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1723)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.net.MalformedURLException: Unknown protocol: http
at java.base/java.net.URL.<init>(URL.java:708)
at java.base/java.net.URL.fromURI(URL.java:748)
at java.base/java.net.URI.toURL(URI.java:1139)
at com.amazonaws.internal.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:83)
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:80)
... 80 more
Caused by: java.lang.IllegalStateException: Unknown protocol: http
at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:373)
at java.base/java.net.URL.<init>(URL.java:703)
... 84 more
After noticing that the bottom end of the Stack Trace says "unknown protocol: http" and "Illegal State Exception: Unknown Protocol", I changed the endpoint in my class above to:
String endpoint = "https://secretsmanager.us-east-1.amazonaws.com";
and I am running the web-site via https now. It still works with no problem running it in Eclpse, and I am still getting the same error when using ColdFusion (i.e., when using a browser)
Since the basic credentials are working and since the error is not access denied, it suggests there is no issue with your IAM setup. Instead, likely your process failed to fetch the EC2 instance metadata. For example, timeout when calling the metadata endpoint, hence the Failed to connect to service endpoint error.
One way around this is to retrieve the accessKey and secretKey manually by calling the instance metadata API. For example, using cfhttp to populate variables.
Example curl command from docs: retrieve /api/token and use it to retrieve /meta-data/iam/security-credentials/<rolename>.
[ec2-user ~]$ TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/iam/security-credentials/s3access
Output:
{
"Code" : "Success",
"LastUpdated" : "2012-04-26T16:39:16Z",
"Type" : "AWS-HMAC",
"AccessKeyId" : "ASIAIOSFODNN7EXAMPLE",
"SecretAccessKey" : "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"Token" : "token",
"Expiration" : "2017-05-17T15:09:54Z"
}
From here, if you still face any errors, at least it should be more explicit to you which step has failed and why.
Just a note, AWS recommends caching the credentials until near expiry instead of querying for every transaction to avoid throttling.
I'm using AWS CA to generate an SSL certificate for ElasticBeanstalk that I'm using in my Flutter app.
Based on the bets practices, I'm trying to pin HTTP client the Root certificate from Amazon Trusted Services
I face 2 issues with the Flutter HTTP client:
Whenever I try to load the certificate it crashes:
[VERBOSE-2:ui_dart_state.cc(209)] Unhandled Exception: TlsException: Failure trusting builtin roots (OS Error: BAD_PKCS12_DATA(pkcs8_x509.c:657), errno = 0)
This is the certificate I use (one of five, I add them all):
final _awsRoot1 = _decodePEM("""-----BEGIN CERTIFICATE-----
MIIDQTCCAimgAwIBAgITBmyfz5m/jAo54vB4ikPmljZbyjANBgkqhkiG9w0BAQsF
ADA5MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRkwFwYDVQQDExBBbWF6
b24gUm9vdCBDQSAxMB4XDTE1MDUyNjAwMDAwMFoXDTM4MDExNzAwMDAwMFowOTEL
MAkGA1UEBhMCVVMxDzANBgNVBAoTBkFtYXpvbjEZMBcGA1UEAxMQQW1hem9uIFJv
b3QgQ0EgMTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ4gHHKeNXj
ca9HgFB0fW7Y14h29Jlo91ghYPl0hAEvrAIthtOgQ3pOsqTQNroBvo3bSMgHFzZM
9O6II8c+6zf1tRn4SWiw3te5djgdYZ6k/oI2peVKVuRF4fn9tBb6dNqcmzU5L/qw
IFAGbHrQgLKm+a/sRxmPUDgH3KKHOVj4utWp+UhnMJbulHheb4mjUcAwhmahRWa6
VOujw5H5SNz/0egwLX0tdHA114gk957EWW67c4cX8jJGKLhD+rcdqsq08p8kDi1L
93FcXmn/6pUCyziKrlA4b9v7LWIbxcceVOF34GfID5yHI9Y/QCB/IIDEgEw+OyQm
jgSubJrIqg0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMC
AYYwHQYDVR0OBBYEFIQYzIU07LwMlJQuCFmcx7IQTgoIMA0GCSqGSIb3DQEBCwUA
A4IBAQCY8jdaQZChGsV2USggNiMOruYou6r4lK5IpDB/G/wkjUu0yKGX9rbxenDI
U5PMCCjjmCXPI6T53iHTfIUJrU6adTrCC2qJeHZERxhlbI1Bjjt/msv0tadQ1wUs
N+gDS63pYaACbvXy8MWy7Vu33PqUXHeeE6V/Uq2V8viTO96LXFvKWlJbYK8U90vv
o/ufQJVtMVT8QtPHRh8jrdkPSHCa2XV4cdFyQzR1bldZwgJcJmApzyMZFo6IQ6XU
5MsI+yMRQ+hDKXJioaldXgjUkK642M4UwtBV8ob2xJNDd2ZhwLnoQdeXeGADbkpy
rqXRfboQnoZsG4q5WTP468SQvvG5
-----END CERTIFICATE-----""");
And this is what I do to decode PEM :
Uint8List _decodePEM(pem) {
for (var s in _startsWith) {
if (pem.startsWith(s)) pem = pem.substring(s.length);
}
for (var s in _endsWith) {
if (pem.endsWith(s)) pem = pem.substring(0, pem.length - s.length);
}
pem = pem.replaceAll('\n', '');
pem = pem.replaceAll('\r', '');
return Uint8List.fromList(base64.decode(pem));
}
And add it via:
void certificatePinningPem(SecurityContext context) {
context.setTrustedCertificatesBytes(_awsRoot1);
context.setTrustedCertificatesBytes(_awsRoot2);
context.setTrustedCertificatesBytes(_awsRoot3);
context.setTrustedCertificatesBytes(_awsRoot4);
context.setTrustedCertificatesBytes(_awsRoot5);
}
I get the same exception when I read DER file :
final aws1 = await rootBundle.load('assets/cer/aws1.cer');
context.setTrustedCertificatesBytes(aws1.buffer.asUint8List());
PEM file from the connection is different
Despite that, when I use badCertificateCallback, pem file from the certificate doesn't match any of those five from repo So how to get Root certificate from connection?
httpClient.badCertificateCallback = (
X509Certificate certificate,
String host,
int port,
) {
print('Bad certificate: ${certificate.pem} for host $host:$port');
return false;
};
Any suggestion on how to properly do certificate pinning in Flutter with AWS Root certificates?
I found a few related threads but they don't really help, i.e. https://github.com/flutter/flutter/issues/39190#issuecomment-693315205
I'm unable to connect even after providing my service account key file (xxx.json) in my path.
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const getTheStuff = async () => {
try {
const results = await storage.getBuckets();
const [buckets] = results;
console.log('Buckets:');
buckets.forEach(bucket => {
console.log(bucket.name);
});
} catch (err) {
console.error('ERROR:', err);
}
}
I try to run this but I get the following in my console. I'm not sure if the authentication has happened and something is wrong with my config, or something else entirely. Looks like the API is being treated like it's running from the google cloud env, hence the connection to an internal hostname is not possible, but this library can be used externally to, so I'm confused.
[Error] A server with the specified hostname could not be found.
[Error] Fetch API cannot load http://metadata.google.internal./computeMetadata/v1/instance due to access control checks.
[Error] Failed to load resource: A server with the specified hostname could not be found. (instance, line 0)
[Log] Buckets: (main.chunk.js, line 83)
[Error] Failed to load resource: The request timed out. (instance, line 0)
I have configured redis using AWS Elasticache and connecting it through tomcat installed on AWS EC2.
Following is my code:
private JedisPool jedisPool = null;
#PostConstruct
private void initialiseJedisPools() {
try {
String redisHost = RESOURCES.getProperty("redis.master.host");
if(Objects.nonNull(redisHost)) {
Integer redisPort = Integer.valueOf(RESOURCES.getProperty("redis.master.port"));
jedisPool = new JedisPool(redisHost, redisPort);
}
} catch (NumberFormatException exception) {
logger.error("Exception occurred while initialising jedis pool.", exception);
}
}
public void addKey(String key, String value, Integer secondsToExpire) {
if (Objects.nonNull(jedisPool)) {
try (Jedis jedis = jedisPool.getResource()) {
jedis.set(key, value);
if (Objects.nonNull(secondsToExpire)) {
jedis.expire(key, secondsToExpire.intValue());
}
} catch (JedisException jedisException) {
logger.error("Exception thrown while adding key in cache.", jedisException);
}
}
}
Often a while I get the following error and I have to restart tomcat to make it work.
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
at redis.clients.util.Pool.getResource(Pool.java:53)
at redis.clients.jedis.JedisPool.getResource(JedisPool.java:226)
.
.
.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Too many open files
at redis.clients.jedis.Connection.connect(Connection.java:207)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93)
at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1767)
at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:106)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:868)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at redis.clients.util.Pool.getResource(Pool.java:49)
... 10 more
Caused by: java.net.SocketException: Too many open files
at java.net.Socket.createImpl(Socket.java:460)
at java.net.Socket.getImpl(Socket.java:520)
at java.net.Socket.setReuseAddress(Socket.java:1449)
at redis.clients.jedis.Connection.connect(Connection.java:174)
... 17 more
I have tried increasing the ulimit for open files, configuring a JedisPoolConfig for maxTotal, maxIdle, minIdle, etc, but no success.
Please suggest.
We have also faced similar issue in past for jedis connection with error "java.net.SocketException: Too many open files"
This happens due to jedis connection not getting closed.
call jedisPool.returnResourse(jedis) or jedis.close()
I am trying to connect a SOAP web service using apache http client. It is secured one (identified by https). I have added code for keystore file also.
Here is the code:
CloseableHttpClient httpclient = null;
try {
KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
FileInputStream instream = new FileInputStream(new File("C:\\SOAP\\qa.keystore"));
try {
trustStore.load(instream, "test1234".toCharArray());
}
catch(Exception e){
e.printStackTrace();
}
finally {
instream.close();
}
SSLContext sslcontext = SSLContexts.custom()
.loadTrustMaterial(trustStore, new TrustSelfSignedStrategy())
.build();
httpclient = HttpClients.custom()
.setSslcontext(sslcontext)
.build();
HttpPost httppost = new HttpPost("https://dev env hostname:7443/wsx/services/reiveFile_WS_VT_SECURED_INBOUND");
FileBody bin = new FileBody(new File("C:\\Payment Check8jan3.dat"));
StringBody fileName = new StringBody("Payment Check8jan3.dat", ContentType.TEXT_PLAIN);
StringBody fileType = new StringBody("111", ContentType.TEXT_PLAIN);
StringBody messageId = new StringBody("3454", ContentType.TEXT_PLAIN);
StringBody senderId = new StringBody("ekrjekrj", ContentType.TEXT_PLAIN);
StringBody checksum = new StringBody("b2ee8af554ab6933085d341b71765bc8", ContentType.TEXT_PLAIN);
StringBody timestamp = new StringBody("3434", ContentType.TEXT_PLAIN);
StringBody transportServiceVersion = new StringBody("4343", ContentType.TEXT_PLAIN);
HttpEntity reqEntity = MultipartEntityBuilder.create()
.addPart("FileName", fileName)
.addPart("FileType", fileType)
.addPart("messageId", messageId)
.addPart("senderId", senderId)
.addPart("checksum", checksum)
.addPart("timestamp", timestamp)
.addPart("transportServiceVersion", transportServiceVersion)
.addPart("payload", bin)
.build();
httppost.setEntity(reqEntity);
System.out.println("executing request " + httppost.getRequestLine());
CloseableHttpResponse response = httpclient.execute(httppost);
try {
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
HttpEntity resEntity = response.getEntity();
if (resEntity != null) {
System.out.println("Response content length: " + resEntity.getContentLength());
}
EntityUtils.consume(resEntity);
} finally {
response.close();
}
} finally {
httpclient.close();
}
I am getting the below exception :
Exception in thread "main" javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:275)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:254)
at org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:117)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:314)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at Test.test(Test.java:113)
at Test.main(Test.java:229)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at sun.security.ssl.InputRecord.read(Unknown Source)
... 18 more
Edit:
As suggested in one of the answer, I imported certificate into cacerts file also. But still getting the same exception.
Please suggest.
This is just a guess but given that the SSL handshake fails in its initial phase I would suspect a protocol version incompatibility. Try forcing the use of an older protocol version such as SSLv3 or SSLv2 and see if that makes any difference.
SSLContext sslcontext = SSLContexts.custom()
.loadTrustMaterial(trustStore)
.build();
SSLConnectionSocketFactory sslcsf = new SSLConnectionSocketFactory(
sslcontext, new String[] {"SSLv3"}, null, null);
CloseableHttpClient client = HttpClients.custom()
.setSSLSocketFactory(sslcsf)
.build();
In general when dealing with SSL issues of any sort a debug log of the SSL initial handshake is usually enough to pinpoint the cause of the problem. See this troubleshooting guide for details
Try importing certificates into cacerts file or create a new one, also point the same to your server where webservice is hosted/consumed.
Also check properly, if complete certificate chain is imported into truststore or not.
The same can be achieved programmatically or through importing into cacerts.
Better way to to import into trusstore, also check for public and private keys if imported properly.