I'm having trouble understanding the source attribute on AWS security groups. My understanding is that e. g. this security group:
Would allow traffic on Port 5432 from all EC2 instances that have the attached security group sg-0a5.... I'm not sure how this is determined though. Is there an IP table with all security groups and their instances that this is checked against?
I'm also very confused because I'm currently following a guide that suggests creating a security group and attaching the source attribute to its own environment. To my understanding, this would mean that the environment allows traffic to itself, which is quite paradoxical to me. I'd love some clarification.
When traffic traverses internally inside the VPC the evaluation is actually performed on whether the private IP address is part of an ENI which has that security group attached.
To be clear as it is evaluated based on the ENI, this rule will only work traffic travels internally inside the VPC/region (private IP to private IP). If it leaves the AWS network or region then the evaluation cannot be performed and thus the rule will fail.
When you specify a security group as the source for a rule, traffic is allowed from the network interfaces that are associated with the source security group for the specified protocol and port. Incoming traffic is allowed based on the private IP addresses of the network interfaces that are associated with the source security group (and not the public IP or Elastic IP addresses).
In your example you mention referencing a security group as a source within itself, however based on the description above it is more seen as metadata when evaluated to confirm the host instance has met this requirement.
In fact the default security group starts by allowing an inbound rule where the source is itself.
Related
I have a license server that has a static IP set up inside the same security group and VPC as the EC2s I’m spinning up using terraform. Basically I want the license server and these EC2s to be able to communicate. At the moment they can’t because the security doesn’t add the EC2s as an inbound rule so that the license server also gets it as an inbound rule (same security group). I’m wondering how to add the IP of the EC2 I’m spinning up in terraform to the inbound of its security group so that it can talk to the license server?
There is no such concept as being "inside the same security group". Security Groups rules apply to each instance individually.
The correct configuration is:
Create a Security Group for the EC2 instances (EC2-SG) that permits appropriate Inbound access to use the instance, and
Create a Security Group for the licensing server (License-SG) that permits Inbound access on the licensing port from EC2-SG
That is, you need TWO security groups. License-SG will permit inbound access from any EC2 instance that has is associated with EC2-SG. You can have more security group as necessary for your instances -- simply add them as sources in License-SG.
Security Groups can also refer to themselves. In your current situation, where the instances and the licensing server are all associated with the same security group, you could add a rule that permits inbound connections from itself. However, this is also saying that the instances are permitting inbound connections from the licensing server, which would not be a correct configuration.
Please note that all communication within the VPC should take place via private IP addresses, otherwise the traffic exits the VPC and then comes in again, which does not allow Security Groups to operate correctly (and also costs you more money!).
I am trying to understand this statement:
"You can reference AWS Security Groups from other Security Groups."
What exactly does this mean?
This is how I understood this. I created a security group and call it "SG-10" and attached it to an instance "EC2-10". The SG "SG-10" has allow port 443 defined inside it.
Now, I create a security group and call it "SG-20" and attached this to an instance "EC2-20". This also has port 443 allowed. Now, if I call "SG-10" inside "SG-20" does this mean that "EC2-10" will be able to connect to "EC2-20" on port 443.?
Regards,
Nik.
if I call "SG-10" inside "SG-20"
No, you can not call a security group, this does not make any sense.
If a security group A references security group B, it does mean that the instance to which the security group A is attached allows inbound or outbound traffic to another instance to which security group B is attached. But if you want to send traffic form instance with security group A to the instance with security group B, you have to use the IP or the DNS of the instance B.
Usually the reason why we would want to reference a security group instead of an IP, is that the IP might change over time or it is not exposed at all. A fairly common example is having an application load balancer (ALB) and a group of EC2 instances which allow traffic only from the ALB. The IP address of the ALB changes over time, so in order to be able to receive traffic from the ALB, we can reference the security group attached to it.
In we want to reference a security group from another security group, we have to edit the rules of the initial security group:
As an example of referencing security groups, imagine a 3-tier architecture:
A Load Balancer receiving traffic from the Internet and sending requests to an Amazon EC2 instance
An Amazon EC2 instance receiving requests from the Load Balancer and sending queries to an Amazon RDS database
An Amazon RDS database receiving requests from the EC2 instance
This would involve 3 security groups:
A security group on the Load Balancer (LB-SG) that allows incoming traffic on port 80/443.
A security group on the EC2 instance (App-SG) that allows incoming traffic on port 80 only from the load balancer. It does this by setting the source to LB-SG.
A security group on the RDS database (DB-SG) that allows incoming traffic on the relevant port (eg 3306) only from the EC2 instance. It does this by setting the source to App-SG.
All security groups allow All Outbound Traffic.
By referencing other security groups, resources can be added/removed without having to change the security groups. For example, another EC2 instance could be launched and assigned the App-SG security group. This new instance would then be able to communicate with the database since DB-SG allows incoming traffic from App-SG, without being tied to any specific IP addresses.
If a resource is associated with multiple security groups, then all rules apply to the resource. Security Groups only say what is 'Allowed'. They do not include 'Deny' rules.
In your SG-10/SG-20 example, you do not mention the source of the traffic, so it is not possible to answer your question. If you want EC2-20 to accept connections from EC2-10, then the SG-20 security group should allow connections with the Source set to SG-10.
As an aside, I should mention that Network ACLs should normally be left at their default "Allow All" settings unless there is a specific networking requirement (eg creating a DMZ).
I've been burned on this before by trying to connect to an EC2 instance's public address... sounds very similar to your setup. Please check this question: Source Security group isnt working as expected in aws. Actually. When you wire up the inbound rule so that the source is a security group, you must communicate through the source instance's private address.
I am setting up EMR clusters on demand, and have a windows EC2 server as a workstation, and a linux EC2 server as a secondary server. All in the same VPC. I would like to avoid having to set security group rules each time an instances comes up with a new IP.
How would I simply allow any traffic to flow freely between all servers in the same VPC?
Thanks!
EDIT- Thanks for the replies, I know this is not good practice in production, but we are dealing with some issues tracking down functionality which we believe is caused by ports, this is just a exploration phase, and this will help us. Thanks!
In the security groups, assign incoming access to other security groups by specifying a security group ID instead of IP addresses. In the web console, if you start typing "sg" in the source field it will pop up a list of your security groups to choose from. Using a security group ID as the source allows all resources that belong to that security group to have access.
Alternatively, if you just want one rule that allows access to every resource in your VPC you would specify your VPC's IP range.
to Allow any traffic between ALL servers in the VPC is not a good practice.
you should rethink in your VPC purpose.
Any way, if you want a group of servers to communicate with each other you can create a Security Group
And Assign it for all servers that you want.
and in inbound rules you add one rule from type "All TCP" and the source of this rule will be the same Security Group.
if your Security Group ID is 'sg-xxxxxxxx'
then the rule will be like this:
All TPC | TCP | 0-65535 | custom | sg-xxxxxxxx
To add to the other answers here, if you really want this, you can set it up via the Subnet or VPC IP CIDR Block
For example, if your VPC is 10.0.0.0/16, then add that to your security groups and all VPC traffic should be able to flow to each other.
I have 2 AWS instances, i-1 and i-2. They are each on a different security group: sg-1 and sg-2, respectively. Both machines have elastic IPs.
sg-2 is configured to allow all traffic from sg-1, regardless of port, source IP or protocol.
When i-1 tries to talk to i-2 its traffic is being blocked. It seems AWS doesn't account for the fact that i-1's traffic is actually coming from its elastic IP.
Is this expected? Is there anything I can do to work around it, apart from manually adding i-1's elastic IP to sg-2?
sg-2 is configured to allow all traffic from sg-1
When you do this, only traffic from Private IP address is allowed. However, as you as using EIP, you explicitly need to allow traffic from that ip address.
Read this: https://forums.aws.amazon.com/thread.jspa?messageID=414060
Quoting from above link:
Out of curiosity, are you perhaps connecting using a public IP address? When you use a rule with a security group as the source, it will only match when connecting over the internal network. The private IP address can change though. If you have an Elastic IP associated with the instance, the public DNS name happens to be static and will always resolve to the current private IP address when used from within the same EC2 region. That allows you to easily connect internally without worrying about any address changes.
You haven't really provided enough information to diagnose the problem, but there are a few things to check:
Is I-1 definitely in SG-1? If you've got the instances muddled, the SG rules would be around the wrong way.
Does the machine in SG-2 have a firewall running that might be blocking incoming traffic even though the SG rules are allowing it?
You've tagged this with the VPC tag - do you have any network ACL settings that might be preventing traffic flow? Are the machines private, using a NAT appliance to get out to the Internet, or public, routing through the standard AWS gateway? Can I-1 see the Internet? If you're routing through a NAT, assigning an EIP to a machine effectively cuts it off from the Internet because EIP and NAT are mutually incompatible, and although I haven't tried it this might also screw up SG routing.
Does SG-1 have any egress rules that might be preventing traffic from leaving?
The answer to your question is likely to be found in the resolution of one of these questions if the answer to any of them is 'Yes'.
As previously stated by slayedbylucifer, you will need to explicitly allow traffic from the EIP.
Here's the reasoning from the official AWS documentation about Security Groups:
When you specify a security group as the source for a rule, traffic is allowed from the network interfaces that are associated with the source security group for the specified protocol and port. Incoming traffic is allowed based on the private IP addresses of the network interfaces that are associated with the source security group (and not the public IP or Elastic IP addresses).
So for reasons Id rather not go into, my DB is on an EC2 instance in eu-west-1 and I have created a beanstalk app on us-east-1. Id like my app to talk to that EC2 instance on a MySQL port (3306).
Can anyone assist with how Id set this up, what ingress rules I need to setup on the EC2 security group? Given that I will have multiple versions of the app in beanstalk, the IP address may change regularly (after environment rebuilds etc).
The important concept regarding Security Group Rules you might be missing is, that you do not necessarily specify IP addresses as traffic sources alone, rather regularly will refer to other security groups as well:
The source can be an individual IP address (203.0.113.1), a range of
addresses (e.g., 203.0.113.0/24), or an EC2 security group. The
security group can be another group in your AWS account, a group in
another AWS account, or the security group itself.
By specifying a security group as the source, you allow incoming
traffic from all instances that belong to the source security group.
[...] You might specify another security group in your account if you're creating a
three-tier web service (see Creating a Three-Tier Web Service).
[emphasis mine]
Consequently you'll simply need to add the Beanstalk app instances security group as a traffic source for TCP port 3306 within the MySQL instance security group.
Taking this further
An additional concept to make oneself familiar with is, that you can have multiple security groups assigned to an instance, thus enabling (possibly dynamic) composition of the resulting firewall.
For example, a recommended practice for larger architectures suggests to specify a dedicated security group per 'role' your instances have (rather than accumulating several rules within one security group as usual), e.g. we have security groups like 'role-ssh' (TCP port 22) and 'role-mysql' (TCP port 3306), which are assigned to EC2 instances as needed in turn. You can read more about this concept in e.g. Security Groups - Most Underappreciated Feature of Amazon EC2.