I'm not sure why my browser is timing out when I try to connect to my AWS Ubuntu Instance squid proxy
I want to have my AWS Ubuntu instance act as a proxy for my python requests. The requests I make in my program will hit my AWS proxy and my proxy will return to me the webpage. The proxy is acting as a middleman. I am running squid in this Ubuntu instance. This instance is also within a VPC.
The VPC security group inbound traffic is currently set to
HTTP, TCP, 80, 0.0.0.0/0
SSH, TCP, 22, 0.0.0.0/0
RDP, TCP, 3389, 0.0.0.0/0
HTTPS, TCP, 443, 0.0.0.0/0
and outbound traffic is open to all traffic
This is my current squid configuration is the default squid.conf except that I changed one line to
http_access allow all meaning traffic is open to all.
However when I changed my mozilla browser to use the Ubuntu instance's Public IP and squid.conf default port of 3128, I cannot see any traffic going through my proxy using this command on the ubuntu instance
tail -f /var/log/squid/access.log
My browser actually times out when I try to connect to a website such as google.com. I am following this tutorial but I cannot get the traffic logs that his person is getting.
HTTP/S as shown in security group settings actually has nothing whatsoever to do with HTTP/S.
Many port numbers have assigned names. When you see "HTTP," here, it's only an alias that means "whatever stuff happens on TCP port 80." The list of values only inludes common services and the names aren't always precise compared the official port names, but the whole point is to give neophytes a word that nakes sense.
What should I change? I always thought I should be leaving HTTP/S ports to their default values.
That is not at all what this does. As already inferable from above, changing an "HTTP" rule from port 80 to something else does not change the value for the HTTP port on instances behind it. Changing the port value makes the rule no longer be an "HTTP" rule, since HTTP is just a friendly label which means "this rule is for TCP port 80."
You need a custom TCP rule allowing port 3128 from your IP, and that's it.
You need to add 3128 as custom TCP in your SG. This will allow Squid to send/ receive traffic.
Also as a best practice, make SSH accessible from your own IP rather than public.
Related
I have deployed a spring boot application in elasticbeanstalk.
In the "application.properties" file, I have set,
server.port=5000
I have added a RDS db and set the following environment properties.
I have also added an inbound rule in the security group of the environment as shown in the image below:
I am still getting the 502 Bad Gateway error when I click on the URL.
Your rule is incorrect.
0.0.0.0/32 means that you accept traffic only from the IP address 0.0.0.0 which basically doesn't exist.
What you want to do is allow traffic from 0.0.0.0/0 which means accept traffic from anywhere in the world.
In case of Elastic Beanstalk, in your instance there is an nginx reverse proxy, which accepts HTTP connections on port 80 and proxies these connections to port 5000.
In your security group inbound rule you should accept HTTP connections on port 80 from everywhere (0.0.0.0/0).
I am confused about configuring the EC2 security group settings.
There are three options (TCP, SSH, HTTPS) and each of them requires you to add an IP/port number.
For context, in my work I'm usually running Flask apps over EC2 and I only want particular people to view them. My question is understanding the difference between TCP, SSH, and HTTPs but more importantly which of these are important for me to configure.
Within the EC2 Console, under Security Groups:
SSH and HTTPS in the Type dropdown, are presets which set the port to 22 and 443 respectively.
TCP is the protocol. Both SSH and HTTPS are TCP.
If you're running a server which you want to expose on a non standard port, you can select Custom TCP Rule, then set the port acordingly.
You should probably have one security group that allows SSH traffic, then assign this security group to the EC2 instances you wish to shell into:
Then have a separate security group that allows the webserver traffic, in this case I also have one for port 80, aswell as 443:
Of course you will then need a server running on that EC2 instance to receive the traffic. This might be a reverse proxy like nginx, which then proxies traffic to the correct port for your app server (run your flask app with something like gunicorn in production).
If nginx and gunicorn are running on the same box, and say gunicorn serves on port 8000, then you wouldn't need a security group for this as it's loopback traffic. Your nginx configuration points to port 8000.
However if you have a separate EC2 instance running gunicorn, you might wish to set up a secuirty group for this to allow internal traffic from your VPC CIDR range:
I only want particular people to view them
This is probably a job for authentication on the app, as oppose to security groups, unless your certain of the public IPs from which you wish people to connect.
In the above examples above a Source of 0.0.0.0/0 is allowing traffic from anywhere to reach that port. The console has a convenient dropdown which lets you set My IP if you only want to allow traffic from the IP you're using to connect to the console. Otherwise you'd need to manually calculate the CIDR blocks.
Hope this helps. It probably raises more questions.
Https/Http are important for you. Both are used with websites. Https is http over SSL, meaning more secure than http. You just need these.
Http/https uses TCP port 80 and 443 by default.
SSH is used to securely access a Unix based server.
I've got an AWS VPC set up with 3 subnets - 1 public subnet and 2 private. I have an EC2 instance with an associated Elastic Block Store (the EBS contains my website) running in the public subnet, and a MySQL database in the private subnets. The security group attached to the EC2 instance allows inbound HTTP access from any source, and SSH access from my IP address only. The outbound security rule allows all traffic to all destinations. The security group associated with the database allows MySQL/Aurora access only for both inbound and outbound traffic, with the source and destination being the public access security group.
This has all been working perfectly well, but when I came to setting up the NACLs for the subnets I ran into a snag that I can't figure out. If I change the inbound rule on the public subnet's NACL to anything other than 'All Traffic' or 'All TCP', I get an error response from my website: Unable to connect to the database: Connection timed out. 2002. I've tried using every option available and always get this result. I'm also getting an unexpected result from the NACL attached to the private subnets: If I deny all access (i.e. delete all rules other than the default 'deny all' rule) for both inbound and outbound traffic, the website continues to function correctly (provided the inbound rule on the public subnet's NACL is set to 'All Traffic' or 'All TCP').
A similar question has been asked here but the answer was essentially to not bother using NACLs, rather than an explanation of how to use them correctly. I'm studying for an AWS Solutions Architect certification so obviously need to understand their usage and in my real-world example, none of AWS' recommended NACL settings work.
I know this is super late but I found the answer to this because I keep running into the same issue and always try to solve it with the ALL TRAFFIC rule. However, no need to do that anymore; it's answered here. The Stack Overflow answer provides the link to an AWS primary source that actually answers your question.
Briefly, you need to add a Custom TCP Rule to your outbound NACL and add the port range 1024 - 65535. This will allow the clients requesting access through the various ports to receive the data requested. If you do not add this rule, the outbound traffic will not reach the requesting clients. I tested this through ICMP (ping), ssh (22) http (80) and https (443).
Why do the ports need to be added? Apparently, AWS sends out traffic through one of the ports between 1024 and 63535. Specifically, "When a client connects to a server, a random port from the ephemeral port range (1024-63535) becomes the client's source port." (See second link.)
The general convention around ACLs is that because they are stateless, incoming traffic is sent back out through the mandatory corresponding port, which is why most newbies (or non hands on practitioners like me) may miss the "ephemeral ports" part of building custom VPCs.
For what it's worth, I went on to remove all the outgoing ports and left just the ephemeral port range. No outgoing traffic was allowed. It seems like either the ACL still needs those ports listed so it can send traffic requested through those ports. Perhaps the outgoing data, first goes through the appropriate outgoing port and then is routed to the specific ephemeral port to which the client is connected. To verify that the incoming rules still worked, I was able to ssh into an EC2 within a public subnet in the VPC, but was not able ping google.com from same.
The alternative working theory for why outgoing traffic was not allowed is because the incoming and matching outgoing ports are all below 1024-63535. Perhaps that's why the outgoing data is not picked up by that range. I will get around to configuring the various protocol (ssh, http/s, imcp) to higher port numbers,, within the range of the ephemeral ports, to continue to verify this second point.
====== [Edited to add findings ======
As a follow up, I worked on the alternate theory and it is likely that the outgoing traffic was not sent through the ephemeral ports because the enabled ports (22, 80 and 443) do not overlap with the ephemeral port range (1024-63535).
I verified this by reconfiguring my ssh protocol to login through port 2222 by editing my sshd_config file on the EC2 (instructions here. I also reconfigured my http protocol to provide access through port 1888. You also need to edit the config file of your chosen webserver, which in my case was apache thus httpd. (You can extrapolate from this link). For newbies, the config files will be generally found in the etc folder. Be sure to restart each service on the EC2 ([link][8] <-- use convention to restart ssh)
Both of these reconfigured port choices was to ensure overlap with the ephemeral ports. Once I made the changes on the EC2, I then changed the security group inbound rule, removed 22, 80 and 443 and added 1888 and 2222. I then went to the NACL and removed the inbound rules 22, 80 and 443 and added 1888 and 2222. [![inbound][9]][9]For the NACL, I removed the outbound rules 22, 80 and 443 and just left the custom TCP rule and add the ephemeral ports 1024-63535.[![ephemeral onnly][10]][10]
I can ssh using - p 2222 and access the web server through 1888, both of which overlap with ephemeral ports.[![p 1888][11]][11][![p2222][12]][12]
[8]: https://(https://hoststud.com/resources/how-to-start-stop-or-restart-apache-server-on-centos-linux-server.191/
[9]: https://i.stack.imgur.com/65tHH.png
[10]: https://i.stack.imgur.com/GrNHI.png
[11]: https://i.stack.imgur.com/CWIkk.png
[12]: https://i.stack.imgur.com/WnK6f.png
i got to see something and did not able to understand and so asking relating AWS NACL. I created one public subnet and associated with an NACL. I created rules in NACL where 80 and 443 allowed for both inbound and outbound. Now created an EC2 instance in the subnet. When i tried yum update it did not work. I reattached the subnet to a default NACL where it allows all and yum update worked. If i am not wrong yum does download packages by http or https. my NACL had these rules and still yum update did not work. I also tried to curl the http://packages.ap-southeast-1.amazonaws.com and did not work. Is there something i am missing in NACL rules.
your answers will clear my fundamentals. please suggest.
Thanks,
You can use a NACL to restrict Inbound ports, but you'll probably have a problem restricting Outbound ports.
The way it works is:
The remote site connects to your Amazon EC2 instance on port 80. It also includes a 'return port' identifier saying which port to send the response to.
The EC2 instance receives the request on port 80, generates a response and sends it back to the originating IP address, to the port requested (which will not be port 80).
The originating system receives the response.
Ports are one-way. You only receive content on a port. You don't send from the same port. This way, if you have made multiple requests, each request is received on a different port and can be matched back to which to the original request.
So, the problem with your NACL is that it is only allowing outbound traffic to 80 and 443, which is not the port that the originating system is requesting to receive the traffic. You would need to open up the range of outbound ports.
It's worth mentioning the the use-case for using NACLs is normally to block specific protocols. If you simply wish to limit access to ports 80 and 443 on your EC2 instance, you should use Security Groups. Security Groups are stateful, so you really only need to open the Inbound connection and outbound responses will be permitted.
Oh, and presumably you also opened Port 22, otherwise you wouldn't be able to login to the instance.
I am relatively new to AWS and I've been looking at quite a few tutorials for the past couple of days trying to figure out how to make my AWS ubuntu instance accessible from the browser.
What I've done:
1st: I configured security groups to accept all traffic for ssh, http, https just to see if the public DNS listed in the instance is accessible.
2nd: I changed the IP of my instance to an elastic IP
3rd: I wrote a simple node.js file that listens on port: 9000 and console.logs 'hello world'
For some reason ssh works, and I can run my node.js file, but agina I cannot access the remote instance from the browser.
Any help would be greatly appreciated since I've been on this for a couple of days
Thanks!
Thank you everyone for the quick responses!
My issue was I did not include a TCP rule to my specific port. Now I am able to access that port via ec2-DNSNAME:9123.
And, just to clarify, if I want to host that DNS for all traffic I should specify 'anywhere' for the TCP rule, correct?
I configured security groups to accept all traffic for ssh, http, https
In security groups, "HTTP" does not mean "HTTP on any port"... it means "any traffic on TCP port 80" -- 80 being the standard IANA assigned port for HTTP.
Security groups are not aware of the type of traffic you are passing, only the IP protocol (e.g. TCP, UDP, ICMP, GRE, etc.) and port number (for protocols that use port numbers) and any protocol specific information (ICMP message types).
You need a rule allowing traffic to port 9000.
Firstly go to your EC2 and see if curl http://localhost works..
Also, if you are exposing your nodejs on port 9000 ; did u open 9000 also on security groups or not ?
Few things to check:
Security groups
Subnet NACLS (these can function as a subnet level
firewall, but unless you've messed with these they should allow all
traffic.)
On the server if you run netstat -na | grep <PORT> do you see your
application listening on the correct ports?
You may also check your system for a firewalls that could be short circuiting the requests.
If the above doesn't point you towards where your issue is you can grab tcpdump and filter it just for requests coming from your web browser (e.g after installing tcpdump -vvn host 10.20.30.40 port 8000 Substitute your ip and port). This will let you know if you're running into a network issue (Packets aren't reaching the server) or if its something with the app.
I'd also recommend using IP addresses while doing your initial troubleshooting. That way we can establish it is not network/server configuration before going into DNS.