I started to mess around with aws a couple days ago and I have a question about VPCs.
I'm considering two options to make "private" a subnet and I want to know your opinion:
OPTION 1:
A subnet with an ig attached (a public subnet really) with an ACL to only allow traffic from the VPC.
Pros: cheaper!
Cons: public ip for each instance (is this a real problem?), different public Ip for each instance (a kind of messy stuff for ip's whitelists)
OPTION 2:
A subnet without an ig attached using a Nat gateway.
Pros: more secure?, same public Ip for each instance.
Cons: more expensive.
I consider the Option 2 the best technical solution, only a bit more expensive. But, do you consider the option 1 a bad practice? Has more pitfalls that I'm not seeing?
It appears that your requirements are:
Run some Amazon EC2 instances
Keep them "private" from the Internet (no direct access from the Internet to the instances)
Allow the instances to access the Internet (eg to download updates, or to communicate with AWS API endpoints)
Your Option 1 is a public subnet, made private by modifying the Network ACLs to allow traffic only within the subnet. You are giving public IP addresses to the instances, presumably so they can access the Internet. This will not work. The Network ACLs that limit access to the VPC will also block traffic from the instances to the Internet.
Your Option 2 is a private subnet, with instances accessing the Internet via a NAT gateway. There is no need to assign public IP addresses to these instances, since they are in a private subnet. Yes, this will work but, as you note, there is the extra expense of the NAT Gateway (or a NAT instance).
An alternative is to use a Public Subnet with Security Groups:
Use a public subnet and give instances auto-assigned public IP addresses. This will allow them to access the Internet. (Note: This is different to an Elastic IP Address, which is limited to 5 per region per AWS account.)
Use a Security Group on each instance to block Inbound access. The instances will still be able to initiate outbound access to the Internet (and responses will come back because Security Groups are stateful)
See documentation: Amazon EC2 Security Groups for Linux Instances
Of course, since the instances now block inbound traffic, you will need a way to connect to them. This is typically done by launching a Bastion Server (Jump Box) in your public subnet. You can connect to the Bastion Server via public IP address, then connect to the private servers via their private IP address. To allow this, configure the Security Group on the private instances to permit inbound traffic from the IP range of the VPC, or from the Security Group associated with the Bastion Server.
Related
My Linux based EC2 instances have both public and private IPs. I am hosting two servers on it:
S3 server, which will be accessible over both internet/private network and
NFS server, which should be accessed only via private IPs/internal network.
I want to open NFS port in the Security Group but only allow the access over the EC2 instances's private IP. I don't think this can be achieved by using only security groups as when the request reaches EC2 node, the Internet Gateway has already replaced the public IP with the private IP of the EC2 instance and the instance is not even aware that there is an extra public IP associated with it.
Instead what can be done is to choose the sources and selectively allow NFS access only to the VPCs and on-premises network (in case there is a VPN connectivity) but that would also mean that every time a new VPC is created OR a new VPN endpoint is established, someone has to modify the Security Group to accommodate new IP addresses.
What I ideally need is a way to apply port and IP based rules on the Internet Gateway which is currently not available/exported. Looking for alternatives.
The simplest method would be to add an Inbound rule to the Security Group where the Source is the CIDR range of the VPC.
For example, if your VPC is 172.31.0.0/16, then use this as the Source in the security group. It would permit any inbound connection from that IP address range.
I have just started my career in the IT field and currently learning new technologies like AWS.
I have a small doubt and it will be really helpful for me if anyone can help me with it.
"Can we connect ec2 instance which is having only private IP with the other ec2 instance with a public IP"
In Putty, I have logged into an EC2 instance (With public IP) and tried to ssh into other EC2 instances (which is having only private IP) by mentioning its private IP Address, but I wasn't able to login to that.
Thank you in advance :)
You can do this (providing security groups and network configuration allows).
The first method (and more prefered) is through a private to private connection, just because an instance has a public IP does not mean it is inaccessible from private inbound connections. If they share the same VPC as long as your security groups/NACLs allow inbound from the private IP range you will be able to connect to the instance via its private IP.
If the instances share a different VPC (or even account) you can connect between using either transit gateway or a single peering connection, then whitelist the IP range of the source. A caveat is that these separate networks cannot be peered if they have a crossover in network address ranges (you would need to create a secondary range to deal with this).
The alternative is to use either a NAT Gateway or NAT instance and then bind this to the route table for your instance for the 0.0.0.0/0 route. You would then need to whitelist in the public instances security group the public EIP of the NAT. This is less preferable as it requires communication across the internet.
I am trying to setup a nice and secure VPC for my lambda and RDS work. Essentially, I want my lambda to hit a site, get some data, and shove it into a database.
In isolation the parts all work. However the second I go to harden everything it all falls apart. Here is what I do:
Disable "Publically Available" from the RDS instance
Change the RDS instance to only accept connections from inside the VPC using the security group
Associate the lambda with a VPC (this kills the internet access)
Following this tutorial I created a NAT gateway, deleted the internet gateway from the VPC subnet, and replaced it with the NAT. Now, as expected, nothing can talk inbound, but things can talk outbound.
At this point I knew I needed a bastion instance, so I fired up an EC2 instance.
The EC2 instance is set to the same subnet the RDS and Lambda are on, and unfortunately this means that I have a problem - the NAT gateway is currently soaking up all the traffic via 0.0.0.0/0, which means there's no room for the internet gateway. Without the internet gateway I (obviously) can't SSH into my bastion instance so I can jump to access my RDS database.
How can I configure this all correctly? My guess is that I need to split the subnet up somehow and make a private and public subnet, the public having the bastion and internet gateway in it. However, I'm not sure how this will all work so the bastion instance can still properly jump to the RDS.
I'm really quite new to setting up AWS services so I'm hoping I didn't mess anything up long the way.
Following this tutorial I created a NAT gateway, deleted the internet
gateway from the VPC subnet, and replaced it with the NAT. Now, as
expected, nothing can talk inbound, but things can talk outbound.
Short Answer
The short answer is you shouldn't have "Killed the Internet Gateway"; thats not a step in the link you provided :) Leave the internet gateway as is in your current subnet. You're going to need a public subnet and the one that was routing 0.0.0.0/0 to IGW is an example of one you can could use.
The work involved is placing your NAT gateway in the Public Subnet, placing your bastion host in the public subnet, placing your lambda function in the private subnet, routing traffic in the private subnet to the NAT gateway in the public subnet, and providing your lambda function with access to your security group by putting it in its own lambda security group and "white listing" the lambda security group in the inbound rules for the security group protecting your database.
Background
Below I have an expanded answer providing background as to public/private subnets, granting internet access to private subnets, and allowing lambda access through security groups. If you don't feel like reading the background then jump to very end where I give a bullet point summary of the steps you'll need.
Public Subnet
A public subnet is one in which traffic originating outside your VPC, or destined for a target outside your VPC (internet), is routed through an internet gateway (IGW). AWS gives you initial default public subnets configured this way; you can identify them in the console by looking at their route table and seeing that under "destination" you find "0.0.0.0/0" targeting an IGW. This means a public subnet is more of a design pattern for "internet accessible" subnet made possible by simply configuring its default route to point to an IGW. If you wish to create a new public subnet you can create a new route table as well that point internet traffic at an IGW and link that route table to your new subnet. This is fairly easy in the console.
Private Subnet
A private subnet is a subnet with no IGW and not directly reachable from the internet, meaning you cannot connect to a public IP address of a system on a private subnet. With the exception of the AWS pre-configured default subnets, this is how new subnets your manually create are setup, as black boxes till you specify otherwise.
Granting Internet Access to Private Subnet
When you want things in your private subnet to be able to reach out to external internet services you can do this by using an intermediary known as a NAT gateway. Configure a route table the same as in the public subnet with the only difference being traffic destined for 0.0.0.0/0(Internet) you target for a NAT gateway sitting inside the public subnet. That last part is critical. Your NAT gateway needs to be in the public subnet but your private subnet is using it as the target for external traffic.
Security Group Access for Lambda
One simple way to allow your lambda function through your security group/firewall is to create a security group just for your lambda function and configure the security group protecting your RDS so that it allows traffic from the lambda security group.
In other words, in security group settings you don't have to specify just IP addresses as sources, you can specify other security groups and this is a pretty neat way of grouping items without having to know their IP address. Your lambda functions can run in the "Lambda Security Group" and anything protected by a security group that you want them to access can be configured to accept traffic from the "Lambda Security Group". Just make sure you actually associate your lambda function with the lambda security group as well as place it in the private subnet.
Lambda VPC Steps in a Nutshetll
Create a new NAT gateway and place it in the public subnet. This
point is important, the NAT gateway goes in the public subnet ( a
subnet whose route table routes 0.0.0.0/0 to an IGW)
Create a new subnet, you can call it Private-Lambda-Subnet. Unlike
the
default pre-configured subnets AWS gives you, this new subnet is
immediately private out of the box.
Create a new route table and link it to your Private-Lambda-Subnet
In the new route table for your private subnet add an entry that
routes 0.0.0.0/0 to a target of the NAT gateway. This is how your
private subnet will indirectly access the internet, by forwarding
traffic to the NAT which will then forward it to the IGW.
Your bastion host and anything else you want to be be publicly
accessible will need to be in the public subnet. This is probably
where you already have your RDS instances, which is fine if they are
firewalled/security group protected.
Create a new security group for your lambda function(s). You can
call it LambdaSecurityGroup.
Configure the inbound rules of your RDS guarding security group to
allow traffic from the LambdaSecurityGroup. This is possible because
you can use other security groups as sources in the firewall
settings, not just ip addresses.
You need a public subnet (default route is the Internet Gateway) and a private subnet (default route is the NAT Gateway). The NAT Gateway, itself, goes on the public subnet, so that it can access the Internet on behalf of the other subnets for which it is providing services. The bastion also goes on the public subnet, but Lambda and RDS go on the private subnet.
Anything can talk to anything on any subnet within a VPC as long as security groups allow it (and Network ACLs, but don't change these unless you have a specific reason to -- if you aren't sure, then the default settings are sufficient).
We're setting up an Amazon VPC in which we will provision (for now) a single EC2 instance and one RDS instance. This is to 'extend our data center', and should only be using private subnet(s).
So actually, we have this setup, and it is working well (insert smiley face icon). For all intents and purposes, we're mirroring the VPC scenario 4 outlined by Amazon here: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenario4.html
tl;dr: A single VCP, with a VPN connecting to our corporate network. The VPN uses a Virtual Private Gateway (VPC end) and Customer Gateway (our end) to allow us access as necessary to the EC2, which contains a webserver connecting to the RDS instance as needed. Anyone on our network has access to the web server running on the EC2 via a URL. All this is working as expected.
The problem comes when the EC2 instance needs to access a resource on the Internet - The idea is for us to NOT have any public subnets, but to route all traffic from the EC2 instance through our VPN and out the 'standard' path of our corporate Internet access. However we're having trouble setting this up.
The fact that it can be done is hightlighted in Amazon's FAQ here:
https://aws.amazon.com/vpc/faqs/
Q. How do instances without public IP addresses access the Internet?
Instances without public IP addresses can access the Internet in one of two ways:
Instances without public IP addresses can route their traffic through a NAT gateway or a NAT instance to access the Internet. These instances use the public IP address of the NAT gateway or NAT instance to traverse the Internet. The NAT gateway or NAT instance allows outbound communication but doesn’t allow machines on the Internet to initiate a connection to the privately addressed instances.
For VPCs with a hardware VPN connection or Direct Connect connection, instances can route their Internet traffic down the virtual private gateway to your existing datacenter. From there, it can access the Internet via your existing egress points and network security/monitoring devices.
We are trying to avoid option #1 as there is a cost involved (along with complexity and security issues). #2 is the perfect resolution for us, but understanding the process to set it up has been eluding us for a while.
Can anyone walk us through what we need to do (or point us to the correct resources) to ensure the EC2 instance* can access the Internet by routing the traffic down the VPN, through our corporate datacenter, and our our existing Internet access point?
* and anything within the private subnet for that matter
If you are using scenario #2, then all there is to do on the AWS end is to ensure that traffic destined for the internet, 0.0.0.0/0 is routed to your Virtual Private Gateway.
Once traffic heads there, it will go to your Customer Gateway, and into your corporate datacenter. It's up to your local IT guys on that end to get Internet-destined traffic heading out, if it's even possible. But at that point, it's no longer an AWS issue.
I'm setting up a test VPC on AWS to see how I like different configurations.
Using the default Scenario 2 configuration:
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenario2.html
I'm not sure what the best practice process for accessing members of the subnet is. You can ssh into the NAT and then ssh into the instance in the subnet, but this of course requires having the pem keys for that instance on the NAT, which makes me uncomfortable.
Have I miss understood the configurations of the security groups, and should be able to ssh directly into the members of the subnet?
In general, instances in private VPC subnets do not have inbound access from the Internet and cannot be accessed directly. You can assign elastic IPs to instances in private subnets and inbound traffic would be routed to it. However, outbound return traffic would never reach the source since the NAT won't route asymmetric traffic.
There are at least two options for accessing those systems via SSH:
Use a bastion host in a public subnet to jump to the instances in the private subnet. The NAT could serve as this bastion, or you could set up another instance. You can have your private key on the bastion, or you can use the ForwardAgent option in your SSH configuration to use the private key from the bastion without requiring the key file to be present.
Establish a VPN from your network to the VPC using a VPN gateway. Allow access in the VPC security groups from your network's address space to the instances in the private subnet. You can then SSH to the private instances as if they're local to your network.
Leaving the private keys on the bastion host shouldn't make you uncomfortable if you harden the bastion sufficiently using security group rules, local firewalls, and other standard system security techniques.