I've got an AWS VPC set up with 3 subnets - 1 public subnet and 2 private. I have an EC2 instance with an associated Elastic Block Store (the EBS contains my website) running in the public subnet, and a MySQL database in the private subnets. The security group attached to the EC2 instance allows inbound HTTP access from any source, and SSH access from my IP address only. The outbound security rule allows all traffic to all destinations. The security group associated with the database allows MySQL/Aurora access only for both inbound and outbound traffic, with the source and destination being the public access security group.
This has all been working perfectly well, but when I came to setting up the NACLs for the subnets I ran into a snag that I can't figure out. If I change the inbound rule on the public subnet's NACL to anything other than 'All Traffic' or 'All TCP', I get an error response from my website: Unable to connect to the database: Connection timed out. 2002. I've tried using every option available and always get this result. I'm also getting an unexpected result from the NACL attached to the private subnets: If I deny all access (i.e. delete all rules other than the default 'deny all' rule) for both inbound and outbound traffic, the website continues to function correctly (provided the inbound rule on the public subnet's NACL is set to 'All Traffic' or 'All TCP').
A similar question has been asked here but the answer was essentially to not bother using NACLs, rather than an explanation of how to use them correctly. I'm studying for an AWS Solutions Architect certification so obviously need to understand their usage and in my real-world example, none of AWS' recommended NACL settings work.
I know this is super late but I found the answer to this because I keep running into the same issue and always try to solve it with the ALL TRAFFIC rule. However, no need to do that anymore; it's answered here. The Stack Overflow answer provides the link to an AWS primary source that actually answers your question.
Briefly, you need to add a Custom TCP Rule to your outbound NACL and add the port range 1024 - 65535. This will allow the clients requesting access through the various ports to receive the data requested. If you do not add this rule, the outbound traffic will not reach the requesting clients. I tested this through ICMP (ping), ssh (22) http (80) and https (443).
Why do the ports need to be added? Apparently, AWS sends out traffic through one of the ports between 1024 and 63535. Specifically, "When a client connects to a server, a random port from the ephemeral port range (1024-63535) becomes the client's source port." (See second link.)
The general convention around ACLs is that because they are stateless, incoming traffic is sent back out through the mandatory corresponding port, which is why most newbies (or non hands on practitioners like me) may miss the "ephemeral ports" part of building custom VPCs.
For what it's worth, I went on to remove all the outgoing ports and left just the ephemeral port range. No outgoing traffic was allowed. It seems like either the ACL still needs those ports listed so it can send traffic requested through those ports. Perhaps the outgoing data, first goes through the appropriate outgoing port and then is routed to the specific ephemeral port to which the client is connected. To verify that the incoming rules still worked, I was able to ssh into an EC2 within a public subnet in the VPC, but was not able ping google.com from same.
The alternative working theory for why outgoing traffic was not allowed is because the incoming and matching outgoing ports are all below 1024-63535. Perhaps that's why the outgoing data is not picked up by that range. I will get around to configuring the various protocol (ssh, http/s, imcp) to higher port numbers,, within the range of the ephemeral ports, to continue to verify this second point.
====== [Edited to add findings ======
As a follow up, I worked on the alternate theory and it is likely that the outgoing traffic was not sent through the ephemeral ports because the enabled ports (22, 80 and 443) do not overlap with the ephemeral port range (1024-63535).
I verified this by reconfiguring my ssh protocol to login through port 2222 by editing my sshd_config file on the EC2 (instructions here. I also reconfigured my http protocol to provide access through port 1888. You also need to edit the config file of your chosen webserver, which in my case was apache thus httpd. (You can extrapolate from this link). For newbies, the config files will be generally found in the etc folder. Be sure to restart each service on the EC2 ([link][8] <-- use convention to restart ssh)
Both of these reconfigured port choices was to ensure overlap with the ephemeral ports. Once I made the changes on the EC2, I then changed the security group inbound rule, removed 22, 80 and 443 and added 1888 and 2222. I then went to the NACL and removed the inbound rules 22, 80 and 443 and added 1888 and 2222. [![inbound][9]][9]For the NACL, I removed the outbound rules 22, 80 and 443 and just left the custom TCP rule and add the ephemeral ports 1024-63535.[![ephemeral onnly][10]][10]
I can ssh using - p 2222 and access the web server through 1888, both of which overlap with ephemeral ports.[![p 1888][11]][11][![p2222][12]][12]
[8]: https://(https://hoststud.com/resources/how-to-start-stop-or-restart-apache-server-on-centos-linux-server.191/
[9]: https://i.stack.imgur.com/65tHH.png
[10]: https://i.stack.imgur.com/GrNHI.png
[11]: https://i.stack.imgur.com/CWIkk.png
[12]: https://i.stack.imgur.com/WnK6f.png
Related
I have a Custom TCP rule in Outbound Security Configuration with Port Range as 0. Does this 0 means it is open to all ports to the ip range provided in the rule?. I have faced issues with this 0 as port when I changed this to correct port(5432) number after which it works fine.
I assume you are talking about the inbound rules (ingress) and outbound rules (egress) of a security group. A security group being a firewall around an AWS component - in your case, as your using port 5432, a PostgreSQL RDS database.
I just tested it on my side. I had a working inbound rule for accessing an EC2 server. When I changed the port to 0 - I could no longer access it. If you were to allow inbound\outbound traffic to all the ports you would specify: 0-65535
So as far as I can tell port 0 specified on its own doesn't seem to give any access.
Besides the 0-65535 port range, there is another valid port value when working with Cloudformation: -1.
This is used for the ICMP and ICMPv6 protocols. Some mention of it here.
You can not specify -1 as a port number when working with the AWS GUI.
If you are worried about security and just want to create an inbound rule for yourself - make sure to specify your /32 CIDR as a suffix to the IP address as per the previous image where TCP access is given across all ports to someone at the 1.2.3.4 IP address. Using a CIDR other than /32 will allow access to other IP addresses O_O
If you are new to working with security groups also please note the following:
Security groups are stateful - if you send a request from your
instance, the response traffic for that request is allowed to flow in
regardless of inbound security group rules. Responses to allowed
inbound traffic are allowed to flow out, regardless of outbound rules.
In other words, don't create a possible security risk by unnecessarily duplicating inbound rules to outbound rules (or vica verca) if you mistakenly thought that you had to create a inbound\outbound rule to handle the responses to the allowed inbound\outbound traffic.
We're using Windows 2016 EC2 instance and incoming HTTPS is being dropped,
Even though the Windows FW Incoming rule is set to allow inbound 443.
Looking at Windows FW logs (of the server) it seems its blocking Incoming traffic: 2018-10-31 09:54:08 DROP TCP x.x.x.x 172.30.1.110 58837 443 52 S 2608940108 0 64240 - - - RECEIVE
Only if I change "Inbound Connections" to "Allow" for the "Private Profile" (It's under Advanced Settings > Actions > Properties > State) - then traffic is received.
Is there a proper set up the allows working properly with "Inbound Connections" as "Blocked (default)"?
BTW: I'm not aware of any rule that is supposed to dominate my rule.
Welcome to SO,
You might want to look at the security groups settings you have setup in AWS on the instance to see if you are allowing traffic to the Ec2 instance on port 443.
https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html
There is also Network ACL, but most people get caught out by Security groups so I check that first.
https://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.html
and lastly you might want to check that your instance has a public ip assigned if you are trying to talk directly to the box from outside of AWS and subnet routing is correctly set for public subnet.
This link might help,
https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Scenario1.html
i got to see something and did not able to understand and so asking relating AWS NACL. I created one public subnet and associated with an NACL. I created rules in NACL where 80 and 443 allowed for both inbound and outbound. Now created an EC2 instance in the subnet. When i tried yum update it did not work. I reattached the subnet to a default NACL where it allows all and yum update worked. If i am not wrong yum does download packages by http or https. my NACL had these rules and still yum update did not work. I also tried to curl the http://packages.ap-southeast-1.amazonaws.com and did not work. Is there something i am missing in NACL rules.
your answers will clear my fundamentals. please suggest.
Thanks,
You can use a NACL to restrict Inbound ports, but you'll probably have a problem restricting Outbound ports.
The way it works is:
The remote site connects to your Amazon EC2 instance on port 80. It also includes a 'return port' identifier saying which port to send the response to.
The EC2 instance receives the request on port 80, generates a response and sends it back to the originating IP address, to the port requested (which will not be port 80).
The originating system receives the response.
Ports are one-way. You only receive content on a port. You don't send from the same port. This way, if you have made multiple requests, each request is received on a different port and can be matched back to which to the original request.
So, the problem with your NACL is that it is only allowing outbound traffic to 80 and 443, which is not the port that the originating system is requesting to receive the traffic. You would need to open up the range of outbound ports.
It's worth mentioning the the use-case for using NACLs is normally to block specific protocols. If you simply wish to limit access to ports 80 and 443 on your EC2 instance, you should use Security Groups. Security Groups are stateful, so you really only need to open the Inbound connection and outbound responses will be permitted.
Oh, and presumably you also opened Port 22, otherwise you wouldn't be able to login to the instance.
My website (korrade.com) always shows up "Connection Timed Out" error, even though in my security group "Port 80" is shown open, and SSH works fine over Port 22(No Issues). I have only ALLOW rules in my security group.
Upon debugging, I found these Network ACL rules, out of which 1 seems strange as its getting automatically added by AWS.
I believe the inclusion of this rule is causing the website to not open. Or I think something else serious is there. As of now, this is the obs from my side..
I have Security Group configured, with only ALLOW rules
/sbin/iptables -L (showing nothing)
Telnet status of Port 80 says CLOSED
Firewall Status: not Loaded
Please help anyone... Much appreciated!
Post editing NACL & SG,
Security Group Screen
SSH over WinSCP
The * rule in a Network Access Control List (NACL) is a catch-all for any packets that do not meet any of the numbered rules.
The NACL in your picture will Allow any HTTP and SSH traffic. All other traffic will be Denied.
Security Groups default to Deny all inbound traffic and Allow all outbound traffic. You should then typically add rules to the Inbound traffic based upon the needs of your application and server.
NACLs default to Allow all inbound & outbound traffic. Traditionally, there is no need to edit a NACL because the Security Group is more appropriate. However, a NACL is Stateless, which means you need to specifically permit traffic in both directions -- inbound an outbound.
I would recommend:
Unless you have a good reason, reset your NACLs to default (100, All traffic, Allow) for both Inbound and Outbound
Try connecting to web server via its Public IP address rather than DNS Name (to test connectivity)
If problems persist, you can Edit your question to show your Security Group settings.
EC2 --> RDS:
RDS (DB Engine): I have inbound and outbound open on port 3306 for the web server's security group.
EC2 (Web Server): I have inbound open for 80, 443 and 22(myIP). Outbound is open for 80,443 and 3306, and it needs all traffic as well to function properly.
My question is about the outbound rules of my web server. Why do I need all traffic to be open? Does this have any security concern?
Some people lock down outbound to prevent against data loss. It works better for immutable architecture since you've removed the ability to update packages from public sources.
Obviously you can choose your own security profile; generally speaking I consider this the levels of security:
Port 22 open to the world
Port 22 access by white listed IPs
Bastion host with white listed IPs
VPN (from here down, all using VPN)
Private IPs + NAT
Proxies server outbound access
That's my ec2 security maturity model. I'm sure I missed some- feel free to comment below.
The security group outbound rules let you to specify "destination", not source. Basically you don't need to worry being attack by Denial of Server through the outbound rules.
On the other hand, unless your Web server need to connect out to Internet without restriction, then you set 80+443 destination to 0.0.0.0/0.
Otherwise , if your web server only need to connect to OS repositories for security update (e.g. ubuntu, apache,etc), then you can explicitly specify the repositories IP address instead of using 0.0.0.0/0.
Other than that, there is little risk. Unless you load something that render webpage, e.g. load web browser in the web server that read random webpage, then it make you susceptible to browser/java engine/rendering engine exploit : if exploit can execute something like ssh reverse tunnel, then there is possibilities that attacker may gain access to your web server.