So I have 2 different VPCs in the same account.
In the first VPC (A), I have an instance which is a part of a private subnet, and all the data is routed to a NAT gateway (Working on previous configurations).
Currently I am trying to access an instance (telnet/ping/anything) in the other VPC(B) from this instance.
I setup VPC peering and changed the main route tables of both the VPCs to target the peering connection. (Did not work)
Then I tried changing the route table of the private subnet to directly route to the peering Connection. (Did not work)
There are many security groups in play, however when I changed the SG on instance in B to accept all Connections, I was able to connect from my local PC but still not from instance in A. So i don't think SG is an issue. I thought it might be routing tables but was unable to find the cause.
When I traceroute from the instance in A, it goes to the NAT gateway private IP, and then to some AWS instance (OWNED BY AWS, NOT ME) and then gets lost.
Where can the connection be possibly wrong?
It is difficult to debug the situation from what you have described.
So, instead, I have tried to recreate your situation and have documented all the steps I took. Please follow these steps to create two more VPCs so that you feel comfortable with the fact that it actually can work.
Then, once you have it working, you can compare this configuration with your existing configuration to figure out what might be wrong with your current VPC configuration.
Here's what I did. Follow along!
Created VPC-A with the VPC Wizard ("VPC with a Single Public Subnet") and a CIDR of 10.0.0.0/16 and a public subnet of 10.0.0.0/24
Manually created VPC-B with a CIDR of 10.5.0.0/16 and a private subnet 10.5.0.0/24
Launched EC2 Instance-A in VPC-A (publicly accessible, with a Security Group permitting SSH access from 0.0.0.0/0)
Launched EC2 Instance-B in VPC-B (in the private subnet, with a Security Group permitting SSH access from 0.0.0.0/0)
Created a VPC Peering connection from VPC-A to VPC-B
Accepted the Peering connection
Added a route to the main Route Table for VPC-A with destination of 10.5.0.0/16 (the range of VPC-B), pointing to the peering connection
Added a route to the main Route Table for VPC-B with destination of 10.0.0.0/16 (the range of VPC-A), pointing to the peering connection
Logged into Instance-A via SSH
From Instance-A, connected via SSH to the private IP address of Instance-B
I had to first paste my private key into a PEM file on Instance-A, use chmod to set permissions, then use:
ssh -i keypair.pem ec2-user#10.5.0.15
I used the private IP address of Instance-B (10.5.0.15). This was randomly assigned, so it would be slightly different when you try this yourself.
And the result was... I successfully connected via SSH from Instance-A in VPC-A to Instance-B in VPC-B via the Peering connection (as proven by the fact that I connected via a Private IP address and the fact that VPC-B has no Internet Gateway).
So, if you follow along with the above steps and get it working, you'll then be able to compare your existing setup and figure out what's different!
I followed all the steps mentioned above and went through all the documentation of AWS. When I added a route in the subnet route table (of VPC A) with the CIDR of the peered VPC (VPC B) pointing to the peering connection it worked.
In my case VPC B (the peered VPC) has a CIDR of 172.31.0.0/16
Related
I have two existing vpc's. One is shared services, and the other is the actual application servers. I have created a peer between the two vpc's, and added routes on each vpc, but still cannot ssh from bastian to app server from the shared services vpc.
Details:
shared services vpc cidr(172.31.0.0/16)
bastian server ip (172.31.5.84)
route added to main route table (10.2.0.0/16 -> vpc-peer-id)
app server vpc cidr (10.2.0.0/16)
ec2 subnet instance ip (10.2.60.4)
route added to main route table (172.17.0.0/16 -> vpc-peer-id)
added sg allow (22 tcp 172.31.0.0/16)
I also added the same route to the app server subnet but no change.
I am completely stumped atm for how to set this up or even work our where it is blocking. Any help would be appreciated.
To assist you, I did the following:
Started with an existing VPC-A with CIDR of 172.31.0.0/16
Created a new VPC-B with CIDR of 10.0.0.0/16
Created a subnet in VPC-B with CIDR of 10.0.0.0/24
Launched an Amazon Linux EC2 instance in the new subnet in VPC-B
Inbound Security Group: Allow SSH from 172.31.0.0/16
Created Peering connection:
Requester VPC: VPC-A
Acceptor VPC: VPC-B
Accepted peering connection (Did you do this on yours?)
Configured Route Tables:
The public Route Table in VPC-A: Route 10.0.0.0/16 to VPC-B
The private Route Table in VPC-B: Route 172.31.0.0/16 to VPC-A
Opened an SSH connection to an existing instance in VPC-A
From that instance, opened an SSH connection to the private IP address of the new instance (10.0.0.121)
Result: Instantly got a Permission denied (publickey) error because I didn't supply the private key. Getting an instant error messaged proved network connectivity (as opposed to hanging, which normally indicates a lack of network connectivity).
I then supplied the correct private key and tried to SSH again.
Result: Connected!
The complete flow is:
My laptop -> Instance in public subnet of `VPC-A` -> Instance in `VPC-B`
This had to use the peering connection because VPC-B has no Internet Gateway and I connected via the private IP address of the instance.
So, I recommend that you double-check that you have done each of the above steps to find where your configuration might differ (accepting the peering connection, configuring the security group, etc).
Our aim is to get our Elastic Beanstalk setups to route traffic through a NAT gateway as we require for certain traffic connecting to API's which require IP whitelisting. Rather than make modifications to the current setup, I have created a separate/isolated VPC & EC2 instance to familiarise and test the setup. However I am yet to get the setup working as desired.
Here is the setup
VPC (vpc-77049811) with CIDR of 10.0.0.0/16
Internet gateway (igw-4d4b212a) assigned to mentioned VPC
Subnet (subnet-096d8a53) with CIDR of 10.0.1.0/24
NAT Gateway (nat-00bb49204627de7e6) attached to mentioned subnet and assigned Elastic IP
Route table attached to mentioned VPC and associate with mentioned subnet
1x EC2 Instance assigned to VPC and its own Elastic IP and Disabled Source/Destination Check
Route Table Setup
10.0.0.0/16 local
0.0.0.0/0 igw-4d4b212a
With the above setup, and am able to log into the server and make a curl request to get the servers public IP address (curl icanhazip.com). As soon as I add a rule to the route table for the url's resolved IP's to route through the NAT gateway though, I am unable to ping or request the curl request as it will timeout.
Rules added to route table which do not work
45.63.64.111/32 nat-00bb49204627de7e6
144.202.71.30/32 nat-00bb49204627de7e6
Not sure if I've overlooked something here or maybe I have misunderstood the concept and use cases for the NAT gateway?
This is public IP 45.63.64.111. You need IGW to reach to this traffic.
You either do that by directly redirecting your traffic to IGW
OR
You do that by directing to NAT then further directing that traffic to IGW
Directing to IGW part is missing.
Nat gateway is used for EC2 in private subnets (which does not have IGW attached to it). In scenario above, EC2 is in public subnet so ideally it does not need NAT.
Here is what I would do to use NAT-
1. Place EC2 in private subent. and have a Route table where all outgoing traffic to nat-gateway.
2. Nat-gateway which is in public subnet will forward your traffic to IGW.
It seems like you have misunderstood the purpose of a NAT.
Its purpose is to provide outbound internet access to instances in a private subnet without allowing any inbound connectivity - i.e. a subnet where the routing table does NOT have an entry for:
0.0.0.0/0 igw-4d4b212a
If you want to restrict access from your EC2 instance to specific IP addresses, put your NAT in the public subnet, create a private subnet, and put your instance in the private subnet. Then add the two routes to the route table associated with the private subnet:
45.63.64.111/32 nat-00bb49204627de7e6
144.202.71.30/32 nat-00bb49204627de7e6
If you simply want to restrict access of your EC2 instance to a couple of IP addresses, you can only create routes for those addresses:
45.63.64.111/32 igw-4d4b212a
144.202.71.30/32 igw-4d4b212a
Be aware that with this last option, your instance can be reached from the internet if you have rules open in your security groups.
I am trying to setup a nice and secure VPC for my lambda and RDS work. Essentially, I want my lambda to hit a site, get some data, and shove it into a database.
In isolation the parts all work. However the second I go to harden everything it all falls apart. Here is what I do:
Disable "Publically Available" from the RDS instance
Change the RDS instance to only accept connections from inside the VPC using the security group
Associate the lambda with a VPC (this kills the internet access)
Following this tutorial I created a NAT gateway, deleted the internet gateway from the VPC subnet, and replaced it with the NAT. Now, as expected, nothing can talk inbound, but things can talk outbound.
At this point I knew I needed a bastion instance, so I fired up an EC2 instance.
The EC2 instance is set to the same subnet the RDS and Lambda are on, and unfortunately this means that I have a problem - the NAT gateway is currently soaking up all the traffic via 0.0.0.0/0, which means there's no room for the internet gateway. Without the internet gateway I (obviously) can't SSH into my bastion instance so I can jump to access my RDS database.
How can I configure this all correctly? My guess is that I need to split the subnet up somehow and make a private and public subnet, the public having the bastion and internet gateway in it. However, I'm not sure how this will all work so the bastion instance can still properly jump to the RDS.
I'm really quite new to setting up AWS services so I'm hoping I didn't mess anything up long the way.
Following this tutorial I created a NAT gateway, deleted the internet
gateway from the VPC subnet, and replaced it with the NAT. Now, as
expected, nothing can talk inbound, but things can talk outbound.
Short Answer
The short answer is you shouldn't have "Killed the Internet Gateway"; thats not a step in the link you provided :) Leave the internet gateway as is in your current subnet. You're going to need a public subnet and the one that was routing 0.0.0.0/0 to IGW is an example of one you can could use.
The work involved is placing your NAT gateway in the Public Subnet, placing your bastion host in the public subnet, placing your lambda function in the private subnet, routing traffic in the private subnet to the NAT gateway in the public subnet, and providing your lambda function with access to your security group by putting it in its own lambda security group and "white listing" the lambda security group in the inbound rules for the security group protecting your database.
Background
Below I have an expanded answer providing background as to public/private subnets, granting internet access to private subnets, and allowing lambda access through security groups. If you don't feel like reading the background then jump to very end where I give a bullet point summary of the steps you'll need.
Public Subnet
A public subnet is one in which traffic originating outside your VPC, or destined for a target outside your VPC (internet), is routed through an internet gateway (IGW). AWS gives you initial default public subnets configured this way; you can identify them in the console by looking at their route table and seeing that under "destination" you find "0.0.0.0/0" targeting an IGW. This means a public subnet is more of a design pattern for "internet accessible" subnet made possible by simply configuring its default route to point to an IGW. If you wish to create a new public subnet you can create a new route table as well that point internet traffic at an IGW and link that route table to your new subnet. This is fairly easy in the console.
Private Subnet
A private subnet is a subnet with no IGW and not directly reachable from the internet, meaning you cannot connect to a public IP address of a system on a private subnet. With the exception of the AWS pre-configured default subnets, this is how new subnets your manually create are setup, as black boxes till you specify otherwise.
Granting Internet Access to Private Subnet
When you want things in your private subnet to be able to reach out to external internet services you can do this by using an intermediary known as a NAT gateway. Configure a route table the same as in the public subnet with the only difference being traffic destined for 0.0.0.0/0(Internet) you target for a NAT gateway sitting inside the public subnet. That last part is critical. Your NAT gateway needs to be in the public subnet but your private subnet is using it as the target for external traffic.
Security Group Access for Lambda
One simple way to allow your lambda function through your security group/firewall is to create a security group just for your lambda function and configure the security group protecting your RDS so that it allows traffic from the lambda security group.
In other words, in security group settings you don't have to specify just IP addresses as sources, you can specify other security groups and this is a pretty neat way of grouping items without having to know their IP address. Your lambda functions can run in the "Lambda Security Group" and anything protected by a security group that you want them to access can be configured to accept traffic from the "Lambda Security Group". Just make sure you actually associate your lambda function with the lambda security group as well as place it in the private subnet.
Lambda VPC Steps in a Nutshetll
Create a new NAT gateway and place it in the public subnet. This
point is important, the NAT gateway goes in the public subnet ( a
subnet whose route table routes 0.0.0.0/0 to an IGW)
Create a new subnet, you can call it Private-Lambda-Subnet. Unlike
the
default pre-configured subnets AWS gives you, this new subnet is
immediately private out of the box.
Create a new route table and link it to your Private-Lambda-Subnet
In the new route table for your private subnet add an entry that
routes 0.0.0.0/0 to a target of the NAT gateway. This is how your
private subnet will indirectly access the internet, by forwarding
traffic to the NAT which will then forward it to the IGW.
Your bastion host and anything else you want to be be publicly
accessible will need to be in the public subnet. This is probably
where you already have your RDS instances, which is fine if they are
firewalled/security group protected.
Create a new security group for your lambda function(s). You can
call it LambdaSecurityGroup.
Configure the inbound rules of your RDS guarding security group to
allow traffic from the LambdaSecurityGroup. This is possible because
you can use other security groups as sources in the firewall
settings, not just ip addresses.
You need a public subnet (default route is the Internet Gateway) and a private subnet (default route is the NAT Gateway). The NAT Gateway, itself, goes on the public subnet, so that it can access the Internet on behalf of the other subnets for which it is providing services. The bastion also goes on the public subnet, but Lambda and RDS go on the private subnet.
Anything can talk to anything on any subnet within a VPC as long as security groups allow it (and Network ACLs, but don't change these unless you have a specific reason to -- if you aren't sure, then the default settings are sufficient).
Our AWS instances are created in a public subnet availability zone and are not able to connect to the internet and SSH. So all the resources are created on the public subnet and one AZ.
I have developed CF nginx template with single VPC and two public subnets, butsecond public subnet instances are unable to connect network and SSH even though I'm giving public IP of the instance in the browser it is not working.
The main issue is instances which are launching in the second public subnet are unable to connect internet system logs are:
Contact the upstream for the repository and get them to fix the
problem
Reconfigure the base URL/etc.
Disable the repository, so yum won't use it by default
Looking at the scenario generally in order to enable access to or from the Internet for instances in a VPC subnet, you must do the following:
Attach an Internet gateway to your VPC.
Ensure that your subnet's route table points to the Internet gateway.
Ensure that instances in your subnet have a globally unique IP
address (public IPv4 address, Elastic IP address, or IPv6 address).
Ensure that your network access control and security group rules
allow the relevant traffic to flow to and from your instance.
To use an Internet gateway, your subnet's route table must contain a route that directs Internet-bound traffic to the Internet gateway. You can scope the route to all destinations not explicitly known to the route table (0.0.0.0/0 for IPv4 or ::/0 for IPv6).
Kindly Refer this AWS Documentation and see what you are missing , as you must have skipped one of the above mentioned things.
I have the following issue:
I want to setup a NAT or WAF in the following scenario:
Internet->VPC->IGW->Rotuer->Subnet1->EB1
->Subnet2->EB2
->Subnet3->EB3
in short words, my scenario is: VPC, Internet Gateway, Route table (Default from VPC) and subnets with EB into.
I tried some steps where I have to create another route table, set NAT instanc as target to 0.0.0.0/0 in this second one and attach the EB subnets to this one; attach the NAT subnet to the main route table and this seems not work.
Tried to Amazon AWS scneario 1 example that is my current configuration without results.
What can be missing to setup correctly it?
Thanks in advance.
I'm not entirely clear about what you are wanting to achieve, but I think you want your subnets to access the Internet, and prevent the Internet from having a direct access to the instances the reside in your private subnet(s).
It's hard to visualize your workflow that way. But i think I follow:
(AWS VPC Internet Gateway) --> Internet
AWS VPC CIDR range contains:
a) public/private subnet #1
b) private subnet #2. private subnet #3.. etc.
A typical set up of achieving requires the following Amazon entities:
1) IGW - internet Gateway
2) VPC with CIDR range of how big or small you want.
for the VMs that you want to protect, while still having internet access, you need them to be assign subnets. The subnets have to be WITHIN the VPC CIDR range. But do not make the subnet to be the same CIDR of the VPC, if you do this, then it essentially makes your private subnet a public accessible one since you have consumed all of the IP range and in which case you must assign it to the IGW which is not what you want.
3) A bastion server - what is a bastion server? This is a server that you purposely put to take internet incoming requests, which protect your private subnets. This can be in the form of a server that acts as a router or NAT. This can be either
3a) a homebrew install of Ubuntu (my fave distro) with iptables and ipv4 packets forwarded, or you can do
3b) a Nat gateway in your amazon VPC, or
3c) a vpc-nat AMI that you can find in the AMI Community. So just chose what method you wish.
4)Create Security group #1 - create a security group for the Bastion Server. I usually call it "NAT SG" and have inbound 0.0.0.0/0 and outbound to TCP 80, 443, 22 and etc, according to your outbound port needs determined by whatever you are running.
5) Security group #2 - create a security group for the private subnet. This is for traffic that will allow all within the subnet(s) and to funnel all outbound non private data to 0.0.0.0/0 to the bastion server.
To implement this is a simplified conceptual order of operations you need to do:
1) Create a VPC with the CIDR range (let's say 10.0.0.0/16)
1.5) create an Internet Gateway and assign it to the VPC (10.0.0.0/16)
2) Create private subnets with CIDR range that's in the VPC CIDR (10.0.1.0/24, 10.0.2.0/24, 10.0.2.0/24, etc).
3) Create a public subnet that will contain the bastion server, or whatever you want the Internet to see. (10.0.254.0/24 (this is huge, allowing 256 servers in the subnet. You will most likely only have the bastion server consuming only 1 server, so it will probably auto assign 10.0.254.2)
4) create 2 routing tables - one called private and one called public. For the private routing table, edit, then add a new Route for 0.0.0.0/0 goes to the bastion server ID. For the public routing table, edit , then add a new route for 0.0.0.0/0 to the Internet Gateway ID (see step 1.5)
5) Either launch your own server and set it up to become a NAT router (if you know how to do that), or start up the vpc-nat AMI (amzn-ami-vpc-nat-hvm-2016.03.0.x86_64-ebs - ami-311a1a5b) is the latest one, so use that. When you create the instance, select the public subnet, the VPC, and assign it a public IP (don't wait and give it an Elastic IP, you will want the public IP),select the "NAT SG" security group, and then Launch it. After it is launched you will select it and select Network --> src/dest. check and DISABLE it. This allows all traffic to funnel in. You only need 1 bastion server, I'm not stating to do all 3.
6) Launch your AMI or custom VM that you want private, and be within the private subnets (10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24... etc). And assign it a private security group, and the same VPC. If it is linux or windows you may want to generate a new private key (pem) for future login. Convert the pem to a ppk using puttygen.
Testing, in order to ensure you have internet connectivity:
the bastion server must be able to reach the world
the VM instance in the private subnet (if the routing was done accurately), should reach the internet by flowing packets to the 0.0.0.0/0 rule to pass through the bastion server, and the bastion server will allow resolution and flow to the internet for outbound requests.
I usually do a ping to google since they do not block ICMP requests and are quite reliable. So ssh into the bastion server, ping google.com. If you get the IP but it cannot get a reply then it means your Internet gateway or the Security group did not all all outbound traffic 0.0.0.0/0, so double-check that.
Now SSH into the private server (assuming linux) and ping the bastion server (10.0.254.X) and you should be able to get a response. That's a good sign. now ping google.com and see if you get a reply. I hope you do. If you get DNS resolution but it is stuck at get any responses, then you forgot to disable Src/Dest checking on the bastion server. Then go to your ec2 console, locate the bastion server and select, then actions, networking, and select src/dest check and confirm DISABLE and save.
That should make your private network resolve and hit the world, while still being protected from the internet.
I hope this helps.
Richard.