I am trying to create a connection to AWS Redshift from the Airflow Web UI, but so far it does not work. I always get the error: ('communication error', ConnectionRefusedError(111, 'Connection refused').
My Redshift is in a subnet of a VPC, but I have allowed public access. In its security group I have furthermore added: 0.0.0.0/0 IPV4 incoming access and my own IP as incoming access allowed.
As The input parameters to the airflow connection template, I use:
As the connection-id: An arbitrary name
host: redshift-clusterxxxxxxxxxxxxxxxxxxxxxxxxxx.redshift.amazonaws.com
The other parameters are also correct.
I have build a little python using redshift_connector to test whether I can connect from there but I am always getting a timeout error.
Has anyone got further ideas on how to solve this issue?
Tableau cannot connect to the redshift server.
It displayed:
An error occurred while communicating with Amazon Redshift
Unable to connect to the server. Check that the server is running and that you have access privileges to the requested database.
Error Code: BC42EF73
could not connect to server: Operation timed out
Is the server running on host "redshift-cluster-1.cncrnka9xarv.us-east-2.redshift.amazonaws.com" (3.143.87.206) and accepting TCP/IP connections on port 5439?
I just followed the setup in AWS Redshift Connection with Tableau - YouTube but it fails. What's the reason?
The first thing you should check is the Security Group associated with the Amazon Redshift database, since the video did not reference it.
The Security Group should permit Inbound access from 0.0.0.0/0 on port 5439. Note that this makes your database accessible to anywhere on the Internet, which is not good from a security perspective. However, I couldn't find a reference to the range of IP addresses that Tableau Online uses.
If that doesn't help, then confirm that the Redshift database is in a public subnet. A public subnet is defined as having a Route Table entry pointing to an Internet Gateway.
Solving this error contains 2 steps:
download the AWS redshift ODBC drive from here
Follow what is given in this answer by #JohnRotenstein.
For doing that follow the following steps:
Go to the cluster with which you want to establish the connection.
Then go to "Properties" tab.
Then below there is a section named "Network and security settings" which has "VPC security group" tab and below that there is a link that will redirect you to Security group of the VPC.
Select the VPC and then select the "inbound rules" tab.
There will be a security group where under "source" there will be something starting with "sg-" which means all the connections to redshift from the internet is blocked and only the addressed with this origin will be able to connect with AWS which are EC2 instances.
There click on "Edit Inbound rules".
In the window that opened up, click the "Add rule" button.
Security group rule ID - This you do not need to set
Type - All traffic
Protocol - All
Port range - All
Source - Anywhere Ipv4(not recommended) so it will be set to "0.0.0.0/0".
Description[optional]
Here set the Source to the IPaddress from where the connection request is going to be made and not the above one.
For all other connection queries can be solved by following this link
Ive went through the whole start-up tutorial and connect to the tinkerpop3 server remotely from an EC2 that is in the same VPC and get the error
gremlin> g.addV('person').property(id, '1').property('name', 'marko')
Host did not respond in a timely fashion - check the server status and submit ag ain.
Type ':help' or ':h' for help.
Display stack trace? [yN]
any reason this might be happening?
Let's try a couple of things to get you started with debugging the issue here:
Have you tried hitting the /status endpoint? If this endpoint is working, then there is a problem with the console configuration. If it isn't, then there is an issue with the connectivity of the EC2 instance to the DB.
Can you ensure that the EC2 instance has been launched with the same security group for which you gave inbound access to port 8182 on the DB (during step#8 in the setting up instructions?
Please ensure that your cluster and instance status is "available" as observed from the Neptune console.
The recommended way to manage such connections is 2 have 2 security groups:
client - A security group that you attach to all clients, like Lambdas, EC2 instances etc. The default outbound rule gives you outbound access to every resource in the VPC. You can tighten that if you'd like.
db - A security group that you should attach to your Neptune cluster. In this security group, edit hte inbound rules, and explicitly add a TCP rule that allows inbound connections to your database port (8182 is the default port).
You can attach the db security group to your cluster either during creation or by modifying existing clusters.
I was able to connect to the amazon rds aurora database locally, and run queries.
But on production EC2 server, the connection returns 500 server error "SQLSTATE[HY000] [2002] Connection timed out".
I've added the same credentials for the database to production, and can see they are being used in the error log.
I enabled the 'allow public access' setting.
I added all the security groups I have to the database (this is probably the problem, I didnt create any special groups, just whatever amazon suggested I let them do).
How could it be working locally but not on production?
Can you check your production server security group outbound rules. if it is connecting from local and not connecting from production machine , so should be some outbound traffic timeout.
It was a security group issue.
The default rds-setup-wizard security group was applied to my database instance, and this gave me local access. I guess it had my ip address or similar as an inbound rule.
I had to add a new security group and add that new group to the database instance.
My new security group needed an inbound rule that looked like this:
Type: MYSQL/Aurora
Protocol: TCP
Port Range: 3306
Source: my EC2's private ipv4 address with /32. Eg: 13.14.15.16/32
Am following this tutorial
http://thoughtsandideas.wordpress.com/2012/05/17/monitoring-and-managing-amazon-rds-databases-using-mysql-workbench/
I am not able to Test DB Connection while creating new server instance to connect to the RDS database.
It says Bad_Authentication[allowed_types=public_key], i have a tried different .pem files and have done through a lot of forums.
I have also allowed access to my IP in the DB SecurityGroups for RDS and the Security Groups for EC2.
Please help me out. Thanks.
This worked for me. Assuming you have logged into AWS:
Go to RDS -> Databases -> Select your database
Make sure the Public Accessibility value is Yes
Check your IP address using http://checkip.amazonaws.com/. If it is not added to the Security Rule Groups, you should add it. this step is crucial
3.1. Click on the rule name. This will open up a new tab, then click in the name of the rule.
3.2. Click on Edit inbound rules
3.3. Click on Add new rule, and add a rule with the following values:
Type: MYSQL/Aurora.
Protocol: TCP (default)
Port range: 3306 (default)
Source: My IP -> This will be the IP Address you got at http://checkip.amazonaws.com/.
Fill information in MySQL Workbench
Hostname: Use the value you have for Endpoint at Connectivity and Security
Port: Use the value you have for Port at Connectivity and Security, by default it is 3306.
Username: Use the value you have for master username at Configuration
Password: Use the value you set when you created the database instance.
If you forgot the password, you can change it by clicking on Modify in your database instance)
I hope this also helps you.
In my scenario the problem was simple
I had to enable 3306 port for MYSQL in the related Security group in RDS > db instance section.
For RDS no need to use ssh. You can directly connect the RDS with MySql workbench.
Follow the following steps-
Make sure that in Security Group you should have provide access for MYSQL
Open MySQL workbench and setup New Connection.
Provide a Connection Name.
Connection Method is set to Standard (TCP/IP)
In the HostName provide your RDS end point
Port is 3306
Enter the username what you have given in the RDS.
Test Connection, and provide the password.
That's it.
For Reference GoTo This Link
If you are not using EC2 Instance to connect with RDS DB Instance then DB Instance should have public access to connect.
While launching DB Instance you need to set Publicly Accessible to yes. You will get Public IP to connect from outside network. That means outside of VPC.
You need to launch DB Instance in Public Subnet which has directly access to Internet. To check Subnet has Internet access, you need to check Route Table which was attached with the subnet. In Route table, check Internet Gateway is attached to the subnet.
You don't need to SSH into DB Instance. Though the option is not available also.
In MySQL WorkBench, click on Setup New Connection.
Give connection name. Choose Standard (TCP/IP) option. You are not trying to connect DB Instance over EC2 Instance so the option with SSH with not work out in this case. You have to provide MySQL hostname, username, password and port.
To verify connection, click on Test Connection button.
If you want to use EC2 Instance and connect DB Instance over EC2 Instance, you have to use Standard (TCP/IP) over SSH option. Check the following link
What would prevent me from connecting to a MySQL server on AWS RDS from an AWS EC2 VM?
I had the same issue. I think you need to enable inbound traffic to the VPC. Follow the below article and you'll be fine. Default VPC is not allowing users to access 3306 by default and you need to allow traffic.
RDS and MySQL Connectivity
Make sure Public accessibility should be yes.
Also, Inside security Group section, click on the security link and then add a new rule under inbound rule section which will whitelist your IP address with the below data :
Type: MYSQL/Aurora.
Protocol: TCP (default)
Port range: 3306 (default)
Source: My IP -> This will be the IP Address you got at http://checkip.amazonaws.com/.
Hope this will help to connect with AWS RDS successfully. Thanks!
Based on the details in your question, it seems that you are trying to SSH into the RDS instance. This is not supported on RDS, it only works for EC2 based MySQL servers.
You should instead connect directly to the MySQL server, on port 3306 (or the one you set manually, if this is the case) and using the username and password you created when provisioning the RDS instance. This means using the Connection Type "Standard TCP/IP" in WB.
Also, the public IP address of the machine you're using MySQL Workbench on should be allowed in the RDS security group.
Even I had the same issue, like after trying all the things out there. A thing worked is modifying the RDS database and making it publically accessible. That sorted things out.
All of the other answers are extremely helpful to debug the connection process but for me the literal fix was to not put in the password into the keychain when initializing the MySQL Connections. Instead, I pressed Test and put in the password when prompted to do so. Running on macOS Monterey V.12.3.1 with a MBP M1, 2020