I have a web application currently running on an EC2 instance with MySQL running alongside it.
I'm building another backend batch service that needs information from the MySQL database. However, I don't want it to access the DB directly. What I want to do is build in a few API routes in the web application, i.e. /private/foo, /private/bar that are only accessible internally (e.g. within the VPC), while all other routes will continue to work as per normal.
I'm wondering how I can go about setting that up?
Run an http/s Apache reverse-proxy server in front of your web application. Use this new web-tier to control all your internal and external http/s traffic.
External Traffic:
Configure Apache to listen on 80/443 for external traffic.
Use and configure Apache module Proxy-Pass to reverse-proxy all your web-application traffic in the Apache virtualhost configuration for port 80/443.
Block access to /private using <Location > directives within your 80/443 virtualhost config
Update your DNS records to point to this web-tier instead of your web-application
How to accommodate your internal traffic:
Have Apache listen on a new port, e.g. 8080
Configure the Apache virtualhost for port 8080 to reverse proxy the internal http requests to your web-application, i.e. /private
How to secure the design:
Use AWS security groups to block any external traffic on port 8080.
Double-down on your security rules by using Apache allow,deny rules within your Apache 8080 virtualhost config to ensure traffic is only permitted from your internal ip-range
An alternative Apache config to the above:
Don't bother with port 8080, and use 80,443 for all internal and external traffic. Internal traffic would make requests against a different domain name, and your internal and external traffic can be managed/separated using Apache name-based virtual-hosting https://httpd.apache.org/docs/current/vhosts/name-based.html
Your VPC uses a private subnet (you are able to configure the address). All you need to do is make sure that traffic coming to your server originated from the same subnet.
Since you want the existing webapp to serve these private routes, you'll need to look for the originating IP address inside your code. (If you don't know how to do this, you might ask a new question about that.)
An alternative is to run a second service (or the same service but listen on a second port). If all private traffic comes in on port 8081 (for example) and all public traffic comes in on port 8080, you can just use AWS's security groups to allow only subnet-local traffic to port 8081 and all traffic to 8080.
Related
I'm looking to host multiple services on a single compute instance. I'm using docker for the one existing service, which has been configured to serve the http on the usual ports. And since I'm using docker I figured it would be easier to set a routing setting than set up a new apache/nginx server.
Could I route the traffic from one address to a specific port? Or, more specifically, is it possible to map a specific port on the server to the http/s ports for a certain domain name?
If it is possible I'm sure it must be a simple setting, but I'm not intimately familiar with GCloud so I'm also sure that I'm missing something.
Yes, you can route ports using IP Tables or setting up a container for virtual hosts which will use Apache or Nginx or similar). However, there are very good reasons to not expose Docker containers to the Internet. Deploy Apache or Nginx as your frontend or deploy a Google Cloud HTTP(S) Load Balancer.
This is not how virtual host works - only cannot simply remap :443 without breaking SSL.
Use Cloud DNS to provide name resolution & use virtual host to tell apart the host headers.
External HTTP(S) Load Balancing would be required to map different internal portswhich also requires named virtual hosts - else the backend will not know which site it is.
With named virtual hosts one can also define multiple SSL certificates.
Background
I have followed this tutorial https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-cli-tutorial-ec2.html, composed a docker compose file, made a website A (compose of 4 containers) up and run serving 1 of my client.
However, now I have another client which I need to host another web site website B using similar strategies as above.
Here is the current running service of ECS / EC2
and here are the containers up and running, serving website A now
Questions & concerns
The website A is now situated as 1 of a service in the EC2 under my only cluster, can I use the same EC2 instance and run website B (as another service of the EC2)?
If so, how are the ports / inbound / outbound traffic being managed? Now website A already occupies port 80, 443, 27017 and 3002 of the EC2 instance for inbound traffic, if website B's containers also run in the same EC2 instances, can I still use port 80, 443, 27017 and 3002 for website B. I have read the docs of ALB (Amazon Load Balancer), seems it can fulfill the requirement, am I at the right track?
And the domain name, through route 53, I have registered a domain www.websiteA.com to serve the 1st website, I have also registered another www.websiteB.com preparing to serve website B, in my case, I guess I need to configure the new domain B pointing to the same EC2 IP?
During my deployment of website B, I do not want to affect the availability of website A, can it be maintained during the process of deploying website B's containers?
I want to clear all the concepts before kick-starting to deploy the website B, appreciate for any help, thank you
Follow-up actions
I come up decided to use AWS application load balancer to solve my issue, and have the following configurations setup.
I first look into load balancer
And configured as follows
I setup a load balancer which listens for requests using HTTP protocol with incoming port 80, whenever there are users access the web server (i.e.: the frontend container), listener will forward that request to the target group (i.e.: http-port-80-access)
And here is the target group (http-port-80-access) which contains a registered target (currently my ec2 instance running the containers), the host port of the container is 32849 which in turn made used by the associated load balancer (web-access-load-balancer) for dynamic port mapping.
I have also configured 1 more rule on top of the default rule, whenever user access url of websiteA, load balancer will forward the request to the target group (http-port-80-access).
All things set, and the healthy test also passed. I then used the following ecs-cli compose service up command to wire up the load balancer with the service
ecs-cli compose --file ./docker-compose-aws-prod.yml --cluster my-ecs-cluster-name --ecs-profile my-ecs-profile --cluster-config my-cluster --project-name my-project --ecs-params ./ecs-params.yml service up --target-group-arn arn:aws:elasticloadbalancing:us-east-2:xxxxxxxxx:targetgroup/http-port-80-access/xxxxxxxx --container-name frontend --container-port 80
where frontend is the service name of the frontend container of website A
However, turn out when I access www.websiteA.com through browser, nothing but ERR_CONNECTION_REFUSED, accessing www.websiteA.com:32849 did accessible, but is not what I desired.
I am wondering which part I configured wrongly
If you are sending traffic directly to the instance then you would have to host on a different port. You should consider using an ALB, which would allow you to use dynamic ports in ECS. The ALB can accept traffic from ports 80 and 443 for different domains and route the traffic to different containers based on things like the domain.
The website A is now situated as 1 of a service in the EC2 under my only cluster, can I use the same EC2 instance and run website B (as another service of the EC2)?
Indeed. However - as you already found out, you have to split the traffic based on something (hostname, path,..). That's where the reverse-proxy comes in play (either managed - ALB, NLB or your own - nginx, haproxy,.. ) .
It's simple for the http traffic (based on the host)
If so, how are the ports / inbound / outbound traffic being managed? Now website A already occupies port 80, 443, 27017 and 3002 of the EC2 instance for inbound traffic, if website B's containers also run in the same EC2 instances, can I still use port 80, 443, 27017 and 3002 for website B.
assuming the ports 27017 and the 3002 are using own binary protocol (not http). You will have handle that.
You can in theory define the port mapping (map different public listening port to these custom ports), but then you need to either use NLB (network load balancer) or expose the ports on hosts public IP. In the latter case I'm not sure with ECS you can guarantee which IP is used (e.g. having multiple worker nodes)
I have read the docs of ALB (Amazon Load Balancer), seems it can fulfill the requirement, am I at the right track?
ALB is layer 7 reverse proxy (http), it is imho the best option for the web access, not for binary protocols.
, I guess I need to configure the new domain B pointing to the same EC2 IP?
that's the plan
During my deployment of website B, I do not want to affect the availability of website A, can it be maintained during the process of deploying website B's containers?
shouldn't be a problem
Run website B on different ports. To allow end users to interact with website B without specify port numbers use a reverse-proxy. See AWS CloudFront.
I am confused about configuring the EC2 security group settings.
There are three options (TCP, SSH, HTTPS) and each of them requires you to add an IP/port number.
For context, in my work I'm usually running Flask apps over EC2 and I only want particular people to view them. My question is understanding the difference between TCP, SSH, and HTTPs but more importantly which of these are important for me to configure.
Within the EC2 Console, under Security Groups:
SSH and HTTPS in the Type dropdown, are presets which set the port to 22 and 443 respectively.
TCP is the protocol. Both SSH and HTTPS are TCP.
If you're running a server which you want to expose on a non standard port, you can select Custom TCP Rule, then set the port acordingly.
You should probably have one security group that allows SSH traffic, then assign this security group to the EC2 instances you wish to shell into:
Then have a separate security group that allows the webserver traffic, in this case I also have one for port 80, aswell as 443:
Of course you will then need a server running on that EC2 instance to receive the traffic. This might be a reverse proxy like nginx, which then proxies traffic to the correct port for your app server (run your flask app with something like gunicorn in production).
If nginx and gunicorn are running on the same box, and say gunicorn serves on port 8000, then you wouldn't need a security group for this as it's loopback traffic. Your nginx configuration points to port 8000.
However if you have a separate EC2 instance running gunicorn, you might wish to set up a secuirty group for this to allow internal traffic from your VPC CIDR range:
I only want particular people to view them
This is probably a job for authentication on the app, as oppose to security groups, unless your certain of the public IPs from which you wish people to connect.
In the above examples above a Source of 0.0.0.0/0 is allowing traffic from anywhere to reach that port. The console has a convenient dropdown which lets you set My IP if you only want to allow traffic from the IP you're using to connect to the console. Otherwise you'd need to manually calculate the CIDR blocks.
Hope this helps. It probably raises more questions.
Https/Http are important for you. Both are used with websites. Https is http over SSL, meaning more secure than http. You just need these.
Http/https uses TCP port 80 and 443 by default.
SSH is used to securely access a Unix based server.
I have configure my VM in such a way that I have 2 application running on one VM.
First App listen on ip:80 port
Second App listen on ip:8080 port
I have enabled ports on VM instances group like this.
I have my Load Balancer configured with two front rules like this.
I want to map ip1:80 to my 80 port application and ip2:8080 to 8080 application
when I tried accessing my application using load balancers IP address it always show me 8080 port application.
I have two backend service running
help me here google team. I m newb
If you want to use IP addresses but not URLs/Domain(s) to reach to your web applications, then URL Maps cannot help to implement your design, as URL map forwards the request to the correct backend service using host values (example.com) and path values (/path) in the destination URL.
That being said, you can add one more Target Proxy to your LB resources to route incoming requests directly to the desired backend services. This will allow you to keep your minimum number of instances as one VM.
For more information, visit this article.
I had similar problem and I had to add second backend.
So I have two backends: one for 80 port, other for 8080. And I have on managed group.
There are many similar questions to this on SO, but none of the solutions I saw really solved my issue completely. I'm doing test runs for a website framework hosted on the Google Cloud Platform.
By default, the website is hosted on http://localhost:2800 And I know the external IP of the instance I'm running it on. How can I access the hosted website through a browser on my local machine? Do I use virtual hosts / port forwarding etc.?
go to your VPC firewall
https://console.cloud.google.com/networking/firewalls
and create a firewall rule to allow traffic on your desired tcp port
Create a Firewall Rule for SQL Server
Configure a firewall rule to allow traffic on port 1433 so other clients can connect to the newly created SQL Server instance over the public internet:
In the Developers Console main menu, go to the Firewall rules section.
OPEN THE FIREWALL RULES
Click the Add firewall rule button.
Name the new firewall rule allow-tcp-1433.
Set Source Filter to IP Ranges.
For Source IP Ranges enter 0.0.0.0/0. This value allows access by all IP addresses.
Warning: This configuration leaves your SQL Server instance open to traffic from everyone, everywhere. It is used only for demonstration purposes. In production environments, restrict access to only those IP addresses that need access.
For Allowed protocols and ports enter tcp:1433.
Click the Create button to create the firewall rule.
Set firewall rules for you google cloud project with following properties:
Target: all instances
Direction: ingress
Source IP ranges: 0.0.0.0/0
Ports and protocols: allow all
Then Depending on your framework, set allowed_host to externalIP or "*" .
For example - In Django, in settings.py set ALLOWED_HOSTS = ["*"]
Now run server on specific 0.0.0.0:[your_port]
For example in django - Python manage.py runserver 0.0.0.0:8000
After this note down you instance external IP address and then in your browser :- goto
[external-IP:[your_port]]
you have to go VPC network, then add firewall to allow your port. then don't forget to select [All intances in the network] (see pic). because, by default its value is [Spesified by tags].
and you can access that instance to its External IP
Try your [externalip]:[port] This worked for me in Amazom ec2.
example: 31.181.171.141:2800