I'm noticing a strange issue connecting to the AWS serverless Aurora endpoint. In our environment, there is an ECS task that creates a DB schema and DB users successfully then there is an application that tries to connect to that same DB and fails.
For some reason, the application that tries to connect to the DB uses an IP address that is not in any CIDR block in the VPC.
VPC's IP CIDR block is 10.100.0.0/16. Serverless RDS cluster uses the subnet present in the VPC (IP range is 10.100.x.x), ECS task and application also uses the same subnet.
java.sql.SQLNonTransientConnectionException: Could not connect to HostAddress{host='dev-db-cluster-serverlessdbcluster-abcd-us-west-2.rds.amazonaws.com', port=3306}. (conn=86949)
Access denied for user 'testservice'#'10.1.6.133'
I'm not able to figure out why an application that uses the same DB host translates the wrong IP 10.1.6.133 and where is this IP coming from.
Related
I am trying to connect my AWS aurora database with pgAdmin 4 and it throws this error. I have tried all the previous solutions provided by the stack overflow answers like add inbound my IP and update pg_hab.conf. It still not working for me. Thank you in advance.
Error facing with pgAdmin
Aurora serverless can be only accessed from within VPC. It has no public Ip address. From docs:
You can't give an Aurora Serverless v1 DB cluster a public IP address. You can access an Aurora Serverless v1 DB cluster only from within a VPC.
This means you either have to connect to it from an EC2 instance running in the same VPC, or setup ssh tunneling or VPN connection between your local computer and the aurora. How to setup ssh tunnel is explained here and here.
Alternatively, use DATA API to interact with your database from outside of a VPC.
I am having an Amazon RDS Postgres instance which resides in the default VPC.
To connect to it, i am using different EC2 instances (Java Spring Boot and NodeJs) running in ElasticBeanstalk. These instances also reside in the default VPC.
Do these EC2 instances connect to/query the RDS instance through the internet or the calls do not leave the AWS Network?
If they leave the AWS network and the calls go through the internet, is creating a VPC endpoint the right solution? Or my whole understanding is incorrect.
Thanks a lot for your help.
Do these EC2 instances connect to/query the RDS instance through the internet or the calls do not leave the AWS Network?
The DNS of the RDS endpoint will resolve to private IP address when used from within VPC. So communication is private, even if you use public subnets or set your RDS instance as publicly available. However, for connection from outside of AWS, the RDS endpoint will resolve to public IP address if the db instance is publicly available.
If they leave the AWS network and the calls go through the internet, is creating a VPC endpoint the right solution?
There is no VPC endpoint for RDS client connections, only for management actions (creating db-instance, termination, etc). In contrast, Aurora Serverless has Data API with corresponding VPC endpoint.
To secure your DB-Instances communications you need to be sure at least about the following:
locate your RD in private subnet (route table does not contain default outbound route to internet gateway).
RDS security group just accept traffic inbound only from instances security group/groups on TCP port for PostgreSQL which is usually 5432.
In this case Traffice to RDS will go localy in your vpc, for vpc endpoints it can be used to access RDS API operations privatly which is not your case (you just need to connect your app to DB using connection string)
What am I trying to do?
I want to migrate and automatically replicate data from SQL Server in my on-premises Windows Server to DB in AWS Cloud. I am using AWS DMS (Database Migration Service) for this.
What have I done/tried already?
I have set up a site-to-site VPN (between on-premises network and AWS VPC)
I am able to ping EC2 instance in VPN from Windows Server on-premises
I am able to ping Windows Server on-premises from EC2 instance in VPN
I have created a DMS Replication Instance. Its Private IP is within the allowed VPC CIDR of the VPN connection set already
I am able to ping the Private IP of DMS Replication Instance from EC2 instance
However, I am NOT able to ping the Private IP of DMS Replication Instance from Windows Server on-premises
I have set-up a DB Server in my on-premises Windows Server. I added this DB as a DMS source endpoint. When I tried to test the connection, it failed with the following error message:
I have linked a Security Group to the DMS Replication Instance. This is the same Security Group I used in the VPN connection set up
My DMS source DB endpoint configuration is as follows:
What do I want to know?
Why am I not able to ping the private IP of DMS Replication Instance while I am able to ping an EC2 instance by setting up VPN
Why the DMS endpoint test connection is failing?
Could you help me in doing this DB migration please?
Probably the following debugging method would help you.
As you have mentioned that you are able to ping the EC2 instance private IP from your on-premise network, it was clear that Site-Site VPN is successful.
You did not mention that you created the DMS instance in the same subnet as the other windows instance which you are able to ping from your on-premise network. If you are created DMS in a different subnet please make sure the route table associated with that subnet has route propagation enabled . Then please check in the security groups that in the inbound rules you are allowing the port numbers and IP addresses. This way we can make sure all the things are setup proper in AWS.
From your on-premise sites please make a telnet test with the following command.
Windows/Linux:
Open command prompt in windows or terminal in linux and try
telnet <<DMS IP>> <<Port Number>>
If it is successful connected then you have connectivity between on-premise to DMS host.
If it is not successfully connected or timed out then you need to contact your on-premise network manager or who is in-charge and tell them that you have an issue connecting with AWS Subnet x.x.x.x/x CIDR from on-premise network
I have production stacks inside a Production account and development stacks inside a Development account. The stacks are identical and are setup as follows:
Each stack as its own VPC.
Within the VPC are two public subnets spanning to AZs and two private subnets spanning to AZs.
The private Subnets contain the RDS instance.
The public Subnets contain a Bastion EC2 instance which can access the RDS instance.
To access the RDS instance, I either have to SSH into the Bastion machine and access it from there, or I create an SSH tunnel via the Bastion to access it through a Database client application such as PGAdmin.
Current DMS setup:
I would like to be able to use DMS (Database Migration Service) to replication an RDS instance from Production into Development. So far I am trying the following but cannot get it to work:
Create a VPC peering connection between Development VPC and Production VPC
Create a replication instance in the private subnet of the Development VPC
Update the private subnet route tables in the development VPC to route traffic to the CIDR of the production VPC through the VPC peering connection
Ensure the Security group for the replication instance can access both RDS instances.
Main Problem:
When creating the source endpoint in DMS, the wizard only shows RDS instances from the same account and the same region, and only allows RDS instances to be configured using server names and ports, however, the RDS instances in my stacks can only be accessed via Bastion machines using tunnelling. Therefore the test endpoint connection always fails.
Any ideas of how to achieve this cross account replication?
Any good step by step blogs that detail how to do this? I have found a few but they don't seem to have RDS instances sitting behind bastion machines and so they all assume the endpoint configuration wizard can be populated using server names and ports.
Many thanks.
Securing the RDS instances via the Bastion host is sound security practice, of course, for developer/operational access.
For DMS migration service however, you should expect to open security group for both the Target and Source RDS database instances to allow the migration instance to have access to both.
From Network Security for AWS Database Migration Service:
The replication instance must have access to the source and target endpoints. The security group for the replication instance must have network ACLs or rules that allow egress from the instance out on the database port to the database endpoints.
Database endpoints must include network ACLs and security group rules that allow incoming access from the replication instance. You can achieve this using the replication instance's security group, the private IP address, the public IP address, or the NAT gateway’s public address, depending on your configuration.
See
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Security.Network.html
For network addressing and to open the RDS private subnet, you'll need a NAT on both source and target. They can be added easily, and then terminated after the migration.
You can now use Network Address Translation (NAT) Gateway, a highly available AWS managed service that makes it easy to connect to the Internet from instances within a private subnet in an AWS Virtual Private Cloud (VPC).
See
https://aws.amazon.com/about-aws/whats-new/2015/12/introducing-amazon-vpc-nat-gateway-a-managed-nat-service/
I'm new and trying to explore AWS and creating a test app running on IIS (EC2 set in Public) and MSSQL RDS(set in Private). However my IIS is not able to connect on the RDS. Looking on my IIS EC2 it has a defined private IP too, same subnet used also from RDS, security group of RDS allows the subnet of the EC2, IAM of EC2 also has RDS Full access.
But still my ec2 wont connect on the RDS. May I ask on what I'm missing on my settings?
It appears that your configuration is:
One VPC
A Public Subnet containing an Amazon EC2 instance
A Private Subnet containing an Amazon RDS for SQL Server instance
You are attempting to connect from the EC2 instance to the RDS db instance
For this to work, you should configure:
A Security Group (let's call it EC2-SG) that is associated with the EC2 instance, permitting connections such that you can login to the EC2 instance
A Security Group (RDS-SG) associated with the RDS db instance, with an incoming connection configured for SQL Server (port 1433) with source set to EC2-SG
That is, the database security group should ALLOW an incoming connection with a source of the security group that is associated to the EC2 instance. The VPC will automatically figure out the IP addresses — you do not need to specify them.
Then, connect from the EC2 instance to the RDS db instance via the DNS Name of the RDS instance that is given in the RDS console.