I'm new to AWS VPC setup for 3-tier web application. I created a VPC with subnet 10.0.0.0/16, and what is the good best practice to do the subnet segmentation in AWS VPC for 3 tier web application? I have ELB with 2 EC2 instances, and RDS and S3 in the backend.
Please advise!! Thanks.
A common pattern you will find is:
VPC with /16 (eg 10.0.0.0/16, which gives all 10.0.x.x addresses)
Public subnets with /24 (eg 10.0.5.0/24, which gives all 10.0.5.x addresses)
Private subnets with /23 (eg 10.0.6.0/23, which gives all 10.0.6.x and 10.0.7.x) -- this is larger because most resources typically go into private subnets and it's a pain to have to make it bigger later
Of course, you can change these above sizes to whatever you want within allowed limits.
Rather than creating a 3-tier structure, consider a 2-tier structure:
1 x Public Subnet per AZ for the Load Balancer (and possibly a Bastion/Jump Box)
1 x Private Subnet per AZ for everything else — application, database, etc.
There is no need to have apps and databases in separate private subnets unless you are super-paranoid. You can use Security Groups to configure the additional layer of security without using separate subnets. This means less IP addresses are wasted (eg in a partially-used subnet).
Of course, you could just use Security Groups for everything and just use one tier, but using private subnets gives that extra level of assurance that things are configured safely.
The way we do it:
We create a VPC that is a /16, e.g. 172.20.0.0/16. Do not use the default VPC.
Then we create a set of subnets for each application “tier”.
Public - Anything with a public IP. Load balancers and NAT gateways are pretty much the only thing here.
Web DMZ - Web servers go here. Anything that is a target for the load balancer.
Data - Resources responsible for storing and retrieving data. RDS instances, EC2 database servers, ElastiCacahe instances
Private - For resources that are truly isolated from Internet traffic. Management and reporting. You may not need this in your environment.
Subnets are all /24. One subnet per availability zone. So there would be like 3 Public subnets, 3 Web DMZ subnets, etc.
Network ACLs control traffic between the subnets. Public subnets can talk to Web DMZ. Web DMZ can talk to Data. Data subnets can talk to each other to facilitate clustering. Private subnets can’t talk to anybody.
I intentionally keep things very coarse in the Network ACL. I do not restrict specific ports/applications. We do that at the Security Group level.
Pro tip: Align the Subnets groups on a /20 boundary to simplify your Network ACLs rules. Instead of listing each data subnet individually, you can just list a single /20 which encompasses all data subnets.
Some people would argue this level of separation is excessive. However I find it useful because it forces people to think about the logical structure of the application. It guards against someone doing something stupid with a Security Group. It’s not bulletproof, but it is a second layer of defense. Also, we sometimes get security audits from customers that expect to see a traditional structure like you would find in an on-prem network.
Related
Question
Can we make a route table which has both igw-id (Internet gateway ID) and vgw-id (VPN gateway ID)? If we can't/shouldn't do it, why?
Example
10.0.0.0/16 --> Local
172.16.0.0/12 --> vgw-id
0.0.0.0/0 --> igw-id
Reference
https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Scenario3.html
(I mainly refer to "Overview" section and "Alternate routing" section.)
Explanation
This webpage above shows the scenario where one VPC has one public subnet, one private subnet and one VPN gateway. In this example, the VPN gateway is always accessed from instances in the private subnet (meaning its route table doesn't have the record "igw-id"). I wonder why one route table doesn't have both "igw-id" and "vgw-id".
Yes, we can have both igw and vgw. In fact, the example above would be a perfect for a public subnet which can connect to your corporate network through direct connect or site-to-site VPN, and also have internet access and be accessible from the internet.
Now, weather you would want this or not, it is an architectural decision. In the example scenario given by AWS, they try to segregate subnets by having a public subnet (with the igw) which can contain services accessible from the internet and a private subnet for other backend services (exmaple: databases). These backend services can be accessed from the corporate network using a site-to-site VPN, this is why the subnet has the vgw.
Yes, you can have a route table with the 3 routes you specified. However, bear in mind that with a route 0.0.0.0/0 --> igw-id, hosts on the internet can initiate connections with the instances in that subnet. Typically, you would want to secure the instances in a subnet that you allow a route to your on-premise network, and not expose it to the internet. If these instances require to connect to the internet, AWS recommends NAT devices for your VPC.
While it's technically possible, the main reason not to do that is due to a concept called network segmentation. It basically follows a "defense in depth" approach that network layers should be separated in a "public" and a "private" part (more zones are also often used like third tier for data storage and a fourth for infra used to manage the other three tiers).
Since your public subnet is directly exposed to the internet, it'most likely to be breached when you misconfigure something. If you would have routes in your public subnet to your VPN gateway, a single breach enables an attacker to attack your on-prem environment as well.
This is why its best practice to add one or two network tiers to your architecture. Even when the public tier is compromised, they still need to get into an instance in the private tier before they can attack your on prem environment.
It's recommended that your public subnet only contains an elastic load balancer or similar when possible. Its a pretty good way to reduce the attack surface of your application servers (less chance that you expose an unwanted port etc.).
A nice guide on this can be found here. It doenst include vpns, but you can condisider them part of the app layer.
In AWS we have subnets tired like public and private subnets.
To simplify the question, I am only referring to the private subnet.
I know we can use single routing table for private subnets in all AZs, or we can use multiple routing table per private subnets in all AZs (One per AZ).
My question is, what is the best practice to design this? should I create different routing table for subnets in each AZs or should create single routing table for subnets in all AZs? what would be the recommended approach and what is advantage of the recommended approach?
Please note, the reason I'm asking this is, I have seen this both approaches being used in different environments and trying to understand what would be the good approach.
Ultimately it depends on your own requirements, however here are points to consider.
Public Subnets
Are any of your public subnets uniquely different? If they're all the same in a pattern many people follow (a public subnet per AZ) then you might find it simpler to have a single route table for your public subnets. As an internet gateway is highly available and a VPC can only have one attached it just adds to complexity to add a route table per subnet if they're all the same.
Private Subnets
The first factor to consider is, are all your private subnets meant to be able to reach the internet. If you have specific subnets which should and those which shouldn't you'll want to keep their routing separate. Bear in mind that internet will be required to interact with third party APIs or to patch the system.
For those private subnets that do require internet you will need to use a NAT device (either a NAT Gateway or NAT instance). Generally the best practice is to have each AZ (not subnet) have its own NAT device, this prevents devices in other AZs being unable to connect to the internet if the NATs AZ is facing issues.
In this case you would consider a route table per availability zone, then bind the subnets to the correct route table as appropriate.
VPC to Network Communication
The final thing to consider is your VPC speaking to external networks (be it in AWS, another cloud provider or even on-premise).
If you're connecting to an external source you need to understand whether that external source should be able to speak to every subnet (and vice-versa) or if there's just a specific range in which it should be able to communicate to).
If you want to have only specific subnets be able to speak to the external source then you should consider route tables for those subnets taking above sections into account to.
Summary
In summary there are a few factors that decide on the routing setup you create.
Is the subnet public or private?
Is there any unique routing requirements for specific subnets? (Such as no internet)
For private subnets are you going to use best practice of a NAT per AZ?
Is there routing between specific subnets and an external network required?
Take these all into account, above all security is generally the highest factor (don't allow communication that is not required) but also consider the management of these. If you have 100 subnets and 90 of them are private with the same routing, then create the minimum number of routes tables required. If requirements change you can change them when they are needed.
I am having trouble understanding this image:
Specifically what I want to have is the Global Accelerator attached to every region, with each region having a VPC and Load Balancer. Then in each AZ in a region, have a public and private subnet. The private subnet or database servers will do all the computation and rendering. The public subnet is the Load Balancer (or does the public subnet need to be a set of instances?).
The public subnet also has an Internet Gateway and NAT Gateway attached. The docs say a private subnet can access the internet using a NAT Gateway in the public subnet, but I don't understand why it doesn't just use the Internet Gateway.
I have a route table in the public subnet, and the private subnet. In the public subnet, the route table goes to 0.0.0.0/0 pointing to the internet gateway. In the private subnet, it goes to 0.0.0.0/0 pointing to the NAT gateway.
There are also an Elastic IP associated with the NAT Gateway, presumably so that's what the internet sees when I make a request from a private server.
My question is, what is connected to what? The docs aren't clear enough on a practical example. I would like to have a public subnet, which I think doesn't require having any instances (am I wrong?), only the load balancer. Then the private subnet is the computation/database subnet which does consist of instances which can only connect to the internet through the load balancer connected to the NAT gateway.
I am confused. I have read the docs over and over again but still don't see how this is supposed to be wired up. Any help explaining what is connected to what in this diagram (extending it to include the Global Accelerator) would be greatly appreciated.
Please don't take this the wrong way, but I don't think you really need everything you are saying that you want.
I would recommend that you start with one region and no Global Accelerator. Implement your system and either creating a test platform that can simulate user traffic, or release your system to your customers and monitor it to determine performance.
If you run in multiple regions, you have a problem with where to keep your data -- data might be kept separate in each region, or replicated. This all depends upon your needs. I suggest you start simple, then expand if you need to operate differently.
Global Accelerator does not make sense when using multiple regions, since it is designed to send traffic to one location. Instead, consider Amazon CloudFront to cache data closer to your customers to provide better performance.
If your application runs on multiple web servers, put a Load Balancer in public subnet(s) and everything else in private subnet(s).
Put simply, start simple. Get your application working in the simplest possible way. Then, if you have greater requirements (eg low latency to users), look at adding Amazon CloudFront. Very few applications span multiple regions, so make that choice carefully and have a specific reason for doing so.
When you use AWS Global Accelerator, you can keep your ALB and instance private, without exposing them to the internet.
Such architecture is explained in the recent AWS blog post:
Accessing private Application Load Balancers and EC2 instances through AWS Global Accelerator
In this architecture, internal ALB is used, and there are no public subnets. The only requirements is the pretense of internet gateway in the VPC. It should be noted, that even though there is internet gateway, no routes are configured for the subnets to access the internet:
To avoid internet traffic from inadvertently flowing into a private subnet, we require that an internet gateway is attached to the VPC that contains resources when a client address-preserving accelerator is created.
I came in late but:
One rule is "if it can stay in private subnet then keep it in private subnet". Accessing the internet will always be costlier. When that is done if you must reach the internet then use VPC endpoints (if possible) instead of NATG and IGW. Endpoints are cheaper and make cheaper data transfers.
For Accelerator and Cloudfront, if data cacheing will be a pain for you then use Accelerator, if not, simply put, CloudFront is cheaper and better.
Despite all this, your architecture and use case will always decide your best and cheapest set up.
Just starting to move our first application to AWS. We have one public subnet (web servers) and two private subnets (for our app instances and databases). My query is where should we put our log management (greylog) and monitoring instances. Should we put them into our app instances subnet or create another private subnet?
Yes you can use the same app subnet, but it is good to have separate subnet for separate services since it will helpful for scenarios like while debugging we can easily classify that requests coming from a particular CIDR range belongs to app subnet or monitoring subnet and we can also have the flexibility of blocking traffic through ACL which applies at subnet level.
You can very well place the logging/monitoring servers in the one the existing private subnets as long as you have rightly sized the subnet CIDR block with enough IP addresses.
Refer this documentation for VPC/Subnet sizing
https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html#vpc-sizing-ipv4
I am trying to evaluate private and public subnets while creating a new VPC.
Is seclusion from inbound traffic from internet the only reason to go ahead with a private subnet? These points also need to be taken into account when considering the private subnet.
NAT Gateway is chargeable - 0.045$/hour and 0.045$ for per GB of data transferred. So there is cost consideration. I would need NAT gateway for pulling code or updates from internet.
I should be able to secure instances in my public subnets by using security groups with different levels of strictness.
When launching an instance I would assign a public ip to only those instances which I want to access from outside the VPC.
I went through this question but it didn't solve my doubts with respect to above points. Any help is appreciated.
From 7 Security Measures to Protect Your Servers:
Isolated Execution Environments
How Do They Enhance Security?
Isolating your processes into individual execution environments
increases your ability to isolate any security problems that may
arise. Similar to how bulkheads and compartments can help contain hull
breaches in ships, separating your individual components can limit the
access that an intruder has to other pieces of your infrastructure.
So, IMHO, do you need private subnets? Depends. In a production environment with public and private services, VPN, databases, etc., yes; but if you have only one server, and you don't want to deal with the configuration of network ACLs, routing, NAT, and so on, maybe a public subnet with your server and a well configured security group could be enough.
To answer your questions:
Use NAT instance (t2.small or m3.medium) instead of NAT gateway. Far cheaper.
Why launch them in public subnet and then tweak the security group if there is no need to accept the incoming internet traffic. There is always a chance to make a mistake in SG rules and allow malicious traffic unintentionally. Even if you want to accept internet traffic, I suggest using a reverse proxy like haproxy
Then launch only those instances in public subnet or use a reverse proxy
The private subnet is an extremely useful feature to protect your instances from DDoS, unauthorized access etc., Do not bypass it for the sake of convenience.