I am seeking some guidance on the best approach to take with EC2 security groups and services with dynamic IP's. I want to make use of services such as SendGrid, Elastic Cloud etc which all use dyanmic IP's over port 80/443. However access to Port 80/443 is closed with the exception of whitelisted IPs. So far the solutions I have found are:
CRON Job to ping the service, take IP's and update EC2 Security Group via EC2 API.
Create a new EC2 to act as a proxy with port 80/443 open. New server communicates with Sendgrid/ElasticCloud, inspects responses and returns parts to main server.
Are there any other better solutions?
Firstly, please bear in mind that security groups in AWS are stateful, meaning that, for example, if you open ports 80 and 443 to all destinations (0.0.0.0/0) in your outbound rules, your EC2 machines will be able to connect to remote hosts and get the response back even if there are no inbound rules for a given IP.
However, this approach works only if the connection is always initiated by your EC2 instance and remote services are just responding. If you require the connections to your EC2 instances to be initiated from the outside, you do need to specify inbound rules in security group(s). If you know a CIDR block of their public IP addresses, that can solve the problem as you can specify it as a destination in security group rule. If you don't know IP range of the hosts that are going to reach your machines, then access restriction at network level is not feasible and you need to implement some form of authorisation of the requester.
P.S. Please also bear in mind that there is a soft default limit of 50 inbound or outbound rules per security group.
Related
I've been trying to create and connect to a new DB on AWS for days. I made it publicly accessible.
I've done this but it can't connect:
HOST: vepo-qa-database.xxxxxxx.ap-southeast-2.rds.amazonaws.com
URL: jdbc:postgresql://vepo-qa-database.xxxxxxx.ap-southeast-2.rds.amazonaws.com:5432/
Why will it not connect?
Your Security Group inbound rule configuration is:
Allow Inbound traffic on any port as long as it comes from sg-1fefe550 (itself!)
This means that only resources associated with that same security group can communicate with the database. Therefore, it is blocking access from your own computer on the Internet.
To enable access to the database, you should add another Inbound rule that permits inbound access on port 5432 (PostgreSQL) from your own IP address.
(I'm also not sure why you have configured the security group to permit access from 'itself'. This is not a normal way to configure security groups.)
There is not enough information in your question to be certain, but here are two suggestions:
1 - verify that the subnets for the database actually are public and that there is a route to the IGW
2 - add a rule to the inbound security group rules to allow you to connect to port 5431. Generally you'd want to restrict this to just your IP address for a public instance.
Also, I'm not sure why you have 192.31.0.0 as part of your VPC range - that is a public IP range.
So right now I have 4 subnets per availability zone: The internet facing "entrypoint" subnet (associated with a load balancer), the generic "service" subnet for internal computation, the "database" subnet for all things data related, and the "external request" subnet for making requests out to the internet. This defines essentially 4 classes of EC2 instances.
I am supposed to now create security groups for these 4 classes of EC2 instances. What I'm wondering is how to do that correctly (I am using terraform).
Can I create 1 security group for "ingress" (incoming) traffic, and a 2nd security group for "egress" (outgoing) traffic, for each class, for each connection type?
So basically, I want this. I want the internet entrypoint to talk to the service. The service can only respond to requests from the internet, it doesn't make any external internet requests itself. The service can talk to the database and the external requesting class. The database can only talk to the service, and the external request can only respond back to the service. The entrypoint can come in as HTTP or HTTPS (or websockets, is that just HTTPS?). It comes in on port 443. This is the load balancer. It then converts the request to HTTP and connects to the compute with port 3000. Should I have a separate port for each different connection type? (So the service layer would have 1 port for the database to respond to like 4000, 1 port for the external request layer to respond to like 5000, etc.). Or does that part matter? Lets say we have the ports thing though.
sg1 (security group 1): ingress 443 -> 3000 (load balancer -> service)
sg2: egress 3000 -> internet? is that 0.0.0.0/0? I don't want it to make free requests out, only to connected clients.
sg3: ingress 3000 -> 4000 (service -> database), specifying the database subnet
sg3: egress 4000 -> 3000 (database -> service), specifying the service subnet, etc.
Am I on the right track? I am new to this and trying to figure it out. Any guidance would be much appreciated, I've been reading the AWS docs for the past week but there's little in terms of best practices and architecture.
You can specify upto 5 individual security groups per ENI (Elastic Network Interface). All available rules are evaluated whenever either the inbound or outbound ingress rule is established.
Regarding communication, security group rules establish a tunnel (allowing stateful communication) during any network communication allowing bi-directional communication as long as the initial connection was allowed by the security group.
Security groups are stateful — if you send a request from your instance, the response traffic for that request is allowed to flow in regardless of inbound security group rules. Responses to allowed inbound traffic are allowed to flow out, regardless of outbound rules.
For example:
Inbound rule allows SSH on port 22 from a specific IP address, no outbound rules for port 22 exist. A user can safely SSH to the server with no connection issues, but is unable to SSH to another server. Add outbound rules if the server should be able to speak outbound, by default it will be allow all.
From this above example this means if you allow no outbound rules for HTTP/HTTPS only inbound connections over HTTP/HTTPS will allow it to return. Also be aware for patching that you will not be able to download from the internet.
Regarding the source, perhaps rather than specifying subnets you can reference the logical security group name instead. This would mean if a resource in any subnet has that security group attached the target resource would allow inbound access (this only works if the connection is private host to private host).
The source of the traffic and the destination port or port range. The source can be another security group, an IPv4 or IPv6 CIDR block, a single IPv4 or IPv6 address, or a prefix list ID.
I would recommend trying to keep the resource realm within a single security group (i.e. DB server all in a single security group) primarily to reduce the overhead of management.
More information is available at the Security groups for your VPC page.
I am new to aws and ec2 interaction with traffic flow.
I have one ec2 instance which I am using as a web server and other as an application server.
how can my two ec2 interact with each other maintaining all the security required?
Both the ec2 machines are on the ubuntu image.
I tried adding All ICMP - IPv4 with source 0.0.0.0/0. I feel it's not the correct way I want only my other instance to access it.
I also tried adding source as other instance security group but didn't work. I was not able to ping from one machine to other
The recommended security configuration would be:
Create a Security Group for the web server (Web-SG) that permits Inbound traffic for HTTP and HTTPS (ports 80, 443). Leave the Outbound configuration as the default "Allow All".
Create a Security Group for the app server (App-SG) that permits Inbound traffic from Web-SG on the desired ports. Leave the Outbound configuration as the default "Allow All".
That is, App-SG should specifically refer to Web-SG in the Inbound rules. This will permit traffic from Web-SG to enter App-SG.
You might want to add additional access so that you can manage the instances (eg SSH), or you can use AWS Systems Manager Session Manager to connect.
Do not use Ping to test access since that requires additional settings and only proves that Ping works. Instead, test the actual access on the desired ports (eg port 80).
My aws instance is stuck and cannot connect using ssh client what should i do?
My hosted websites are also not working. I do not want to restart my aws instance through aws console.
Please help me in this regard.
Thanks in advance.
A recommendation to troubleshoot these kinds of problems
Always generate logs.
Always use a Cloudwatch's agent to retrieve specific logs from your instances.
Check this link to learn more about it: https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/send_logs_to_cwl.html
About your problem
I think you tried to connect to it via SSH too many times without closing the previous connections.
Your instance is out of memory, for this situation you must restart your instance.
You could get the last screenshot of your instance using the options from Console.
Follow this link for more information about troubleshooting
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesConnecting.html
Some suggestions from that link:
Check your security group rules. You need a security group rule that allows inbound traffic from your public IPv4 address on the proper port.
[EC2-VPC] Check the route table for the subnet. You need a route that sends all traffic destined outside the VPC to the internet gateway for the VPC.
[EC2-VPC] Check the network access control list (ACL) for the subnet. The network ACLs must allow inbound and outbound traffic from your local IP address on the proper port. The default network ACL allows all inbound and outbound traffic.
-If your computer is on a corporate network, ask your network administrator whether the internal firewall allows inbound and outbound traffic from your computer on port 22 (for Linux instances) or port 3389 (for Windows instances).
And more...
If the issue still continues, please create an AMI(image) of the instance and try to create a new instance from that AMI. Then try to SSH and everything went smooth the terminate the old instance.
I have two AWS Linux AMI servers.
One with a Node.JS HTTPS server and the second running my Mongo DB.
The only way to connect to my Mongo DB server is through my Node.JS server.
In my in the AWS security group for the Mongo DB server I was able to put the name of the the AWS security group for the Node.JS server in the 'Source' column in the 'Inbound' tab and it allowed connectivity, but I'm wondering what setting the security group actually does?
More specifically, what IP addresses is this allowing?
If you click the 'i' next to Source it states:
I initially resorted to using the "private IP address" for communication between AWS instances because according to AWS documentation,
A private IP address is an IP address that's not reachable over the
Internet. You can use private IP addresses for communication between
instances in the same network (EC2-Classic or a VPC).
But after some discussion (see comments below) it seems the better solution may be to utilize AWS Security Groups. My initially hesitation in using AWS security groups was unfounded.
My fear was that if I added the same security group assigned to my Node.js server to the 'Source' for the inbound traffic tab of my MongoDB server, then my MongoDB server would inherit the same inbound traffic rules (all traffic).
The aforementioned fear is unfounded because setting the inbound source to a security group does NOT inherit the rules, but rather simply allows inbound traffic from any instance that has been assigned to that security group.