I have a staging and a production EKS cluster on AWS, and they use different DBs.
I need to deploy a replica of the prod app deployment in a temp namespace inside the staging cluster.
Now, the temp deployment needs to be connected to the prod Aurora.
But, the staging and production clusters are in separate VPCs, but [unfortunately] with the same CIDRs. So I cannot peer the two VPCs.
Also, the Aurora cluster is deployed in private subnets.
One [temporary] solution that I am thinking of is, to essentially make public the private subnet the Aurora writer is deployed into, and have my app in the staging cluster reach the prod db over the internet.
I found the private subnet that the Aurora writer is deployed into
Found the routing table that it uses
Could I just change the routing rule from 0.0.0.0/0 -> NAT-12345 to 0.0.0.0/0 -> IGW-12345 so instead of NAT it will use the Internet Gateway?
Is this something viable, and if so, do I need to make something else in order for the db endpoint eg the-prod-aurora-postgres.cluster-something123.uk-west-45.rds.amazonaws.com to be reachable over the internet?
Yes that's correct you have to create a public subnet (with route to igw)
Change aurora connectivity to public access
Adjust the security group to allow access only to the eks cluster
https://aws.amazon.com/premiumsupport/knowledge-center/aurora-private-public-endpoints/
Related
Just created an VPC for EKS Cluster and started RDS PostgreSQL instance with custom VPC.
Custom VPC has Subnets.
My Custom VPC has Internet Gateway attached.
EKS and RDS is in same VPC so they have internal communication.
My problem is that i want to connect to RDS from my local machine and i am unable. Regarding the problem i have created a new Security Group with Inbound Rules for PostgreSQL.
PostgreSQL TCP 5432 0.0.0.0/0 –
Im still unable to connect
UPDATE
RDS is Publicly accessible
Security group allows access to RDS
In order to connect to RDS instances from the internet you need to do these 3 things
Deploy your RDS instance in a "public" subnet. This means the subnet must have an Internet Gateway attached to it so it can respond properly to outbound requests
In your RDS instance under Connectivity, extend the Additional configuration section, and then choose Publicly accessible.
Make sure the security group allows access to your RDS instance.
Note: exposing a database to public access is not secure. What I recommend you to do is create a proxy with haproxy or a VPN.
To be able to connect to the RDS database remotely you need to select "yes" option for the "Public Accessibility" setting for you database. Here are some additional configurations that need to be taken into account (form AWS docs):
If you want your DB instance in the VPC to be publicly accessible, you
must enable the VPC attributes DNS hostnames and DNS resolution.
Your VPC must have a VPC security group that allows access to the DB
instance.
The CIDR blocks in each of your subnets must be large enough to
accommodate spare IP addresses for Amazon RDS to use during
maintenance activities, including failover and compute scaling.
Best,
Stefan
I have production stacks inside a Production account and development stacks inside a Development account. The stacks are identical and are setup as follows:
Each stack as its own VPC.
Within the VPC are two public subnets spanning to AZs and two private subnets spanning to AZs.
The private Subnets contain the RDS instance.
The public Subnets contain a Bastion EC2 instance which can access the RDS instance.
To access the RDS instance, I either have to SSH into the Bastion machine and access it from there, or I create an SSH tunnel via the Bastion to access it through a Database client application such as PGAdmin.
Current DMS setup:
I would like to be able to use DMS (Database Migration Service) to replication an RDS instance from Production into Development. So far I am trying the following but cannot get it to work:
Create a VPC peering connection between Development VPC and Production VPC
Create a replication instance in the private subnet of the Development VPC
Update the private subnet route tables in the development VPC to route traffic to the CIDR of the production VPC through the VPC peering connection
Ensure the Security group for the replication instance can access both RDS instances.
Main Problem:
When creating the source endpoint in DMS, the wizard only shows RDS instances from the same account and the same region, and only allows RDS instances to be configured using server names and ports, however, the RDS instances in my stacks can only be accessed via Bastion machines using tunnelling. Therefore the test endpoint connection always fails.
Any ideas of how to achieve this cross account replication?
Any good step by step blogs that detail how to do this? I have found a few but they don't seem to have RDS instances sitting behind bastion machines and so they all assume the endpoint configuration wizard can be populated using server names and ports.
Many thanks.
Securing the RDS instances via the Bastion host is sound security practice, of course, for developer/operational access.
For DMS migration service however, you should expect to open security group for both the Target and Source RDS database instances to allow the migration instance to have access to both.
From Network Security for AWS Database Migration Service:
The replication instance must have access to the source and target endpoints. The security group for the replication instance must have network ACLs or rules that allow egress from the instance out on the database port to the database endpoints.
Database endpoints must include network ACLs and security group rules that allow incoming access from the replication instance. You can achieve this using the replication instance's security group, the private IP address, the public IP address, or the NAT gateway’s public address, depending on your configuration.
See
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Security.Network.html
For network addressing and to open the RDS private subnet, you'll need a NAT on both source and target. They can be added easily, and then terminated after the migration.
You can now use Network Address Translation (NAT) Gateway, a highly available AWS managed service that makes it easy to connect to the Internet from instances within a private subnet in an AWS Virtual Private Cloud (VPC).
See
https://aws.amazon.com/about-aws/whats-new/2015/12/introducing-amazon-vpc-nat-gateway-a-managed-nat-service/
Background: I have a kubernetes cluster set up in one AWS account that needs to access data in an RDS MySQL instance in a different account and I can't seem to get the settings correct to allow traffic to flow.
What I've tried so far:
Setup a peering connection between the two VPCs. They are in the same region, us-east-1.
Created Route table entries in each account to point traffic on the corresponding subnet to the peering connection.
Created a security group in the RDS VPC to allow traffic from the kubernetes subnets to access MySql.
Made sure DNS Resolution is enabled on both VPC's.
Kubernetes VPC details (Requester)
This contains 3 EC2's (looks like each has its own subnet) that house my kubernetes cluster. I used EKS to set this up.
The route table rules I set up have the 3 subnets associated, and point the RDS VPC CIDR block at the peering connection.
RDS VPC details (Accepter)
This VPC contains the mysql RDS instance, as well as some other resources. The RDS instance has quite a few VPC security groups assigned to it for access from our office IP's etc. It has Public Accessibility set to true.
I repeated the route table setup (in reverse) and pointed back to the K8s VPC subnet / peering connection.
Testing
To test the connection, I've tried 2 different ways. The application that needs to access mysql is written in node, so I just wrote a test connector and example query and it times out.
I also tried netcat from a terminal in the pod running in the kubernetes cluster.
nc -v {{myclustername}}.us-east-1.rds.amazonaws.com 3306
Which also times out. It seems to be trying to hit the correct mysql instance IP though so I'm not sure if that means my routing rules are working right from the k8s vpc side.
DNS fwd/rev mismatch: ec2-XXX.compute-1.amazonaws.com != ip-{{IP OF MY MYSQL}}.ec2.internal
I'm not sure what steps to take next. Any direction would be greatly appreciated.
Side Note: I've read thru this Kubernetes container connection to RDS instance in separate VPC
I think I understand what's going on there. My CIDR blocks do not conflict with the default K8s ips (10.0...) so my problem seems to be different.
I know this was asked a long time ago, but I just ran into this problem as well.
It turns out I was editing the wrong AWS routing table! When I ran kops to create my cluster, it created a new VPC with its own routing table but also another routing table! I needed to add the peer connection route to the cluster's routing table instead of the VPC's Main routing table.
I trying to setup Kubernetes cluster using kops,
having all of my nodes and master running on a private shabnets on my existing AWS VPC,
when passing the vpcid and network cidr to the create command, i'm enforced to have the EnableDNSHostnames=true,
I wonder of it's possible to setup a cluster with that option set to false
So all of the instances lunched in the private vpc wont have public address
Thanks
It's completely possible to run in private subnets, that's how I deploy my cluster (https://github.com/upmc-enterprises/kubernetes-on-aws), where all servers are in private subnets and access is granted via bastion boxes.
For kops specifically, looks like there's support (https://github.com/kubernetes/kops/issues/428), but I'm not a big user of it so can't speak 100% to how well it works.
I have a corporate setup that primarily has 2 peered VPCs at the moment. We want to provision Elasticache (for Redis) so that 2 EC2 IIS Web Servers (one in each of the two VPCs) can connect to the same Elasticache cluster, is this possible?
Currently I can successfully connect to the cluster from the EC2 instance that is in the same VPC that the Elasticache cluster was provisioned in, but the other EC2 instance in the peered VPC cannot connect.
I tried allowing all access through security groups, turned off firewalls, etc... but nothing works.
Any help you can provide would be greatly appreciated.
We encountered the same issue and it turned out that route table configuration was not properly set in our case.
Route table for elasticache subnet need to have a config below.
Destination -> IP address range for client EC2 instance subnet
Target -> peering connection ('pcx-xxxxx')