Typical setups I've found on Google to run a django application on AWS all suggest a setup like
ELB -> nginx -> gunicorn -> django
I was wondering why the nginx part is really needed here? Isn't ELB sufficient as proxy?
In our case, we are running multiple Gunicorn/django instances in individual docker containers on ECS.
Without Nginx, It would work just fine and you will still be safe from the majority of DDOS attacks that can bring down an exposed gunicorn server.
I can only see Nginx helpful to add to the stack if it'll be serving your static files. However, it's much better to serve your static files by S3 (+ cloudfront as a bonus) since it's has high availability and reliability baked in.
Sources:
http://docs.gunicorn.org/en/latest/deploy.html#nginx-configuration
https://stackoverflow.com/a/12801140
I had to search a lot to get a satisfying answer :
ELB does not save you from DDoS attacks, it is more of a general purpose load balancer.
ELB directly sends the incoming request to the the Gunicorn server. It does not receive the full request before forwarding it to Gunicorn, i.e, if headers/body from the request is coming slowly because of bad internet connection from the client or whatever other reason, then the Gunicorn server will be waiting for the request to complete before it starts processing the request. In general, it's a bad practice to allow the same server to be the web server and application server, as this hogs up the resources of the application server(Gunicorn).
Nginx additionally helps serve static files and with GZIP compression, thus making it faster for sending/receiving data from both client/server.
Additionally, even in Gunicorn's documentation, it is recommended to use Nginx in front of it.
Related
For network gurus out there, I'll like to ask some questions regarding some unique setup where the server will be sending a request to a client on localhost on a certain port.
I have a cloudy understanding of some network fundamentals that I hope you'll be able to help me out.
Kindly check the image below:
Basically, there's a static website hosted in AWS s3 and at some point this website will send a request to https://localhost:8001.
I was expecting that it will connect to the nginx container listening on port 8001 in my local machine, but it results in 504 gateway error.
My questions are:
Is it possible for a remote server to directly send data to a client at a particular port by addressing it as localhost?
How is it possible for the static website to communicate to my local docker container?
Thanks in advance.
In the setup you show, in the context of a Web site, localhost isn't in your picture at all. It's the desktop machine running the end user's Web browser.
More generally, you show several boxes in your diagram – "local machine", "Docker VM", "individual container", "server in Amazon's data center" – and within each of these boxes, if they make an outbound request to localhost, it reaches back to itself.
You have two basic options here:
(1) Set up a separate (Route 53) DNS name for your back-end service, and use that https://backend.example.com/... host name in your front-end application.
(2) Set up an HTTP reverse proxy that forwards /, /assets, ... to S3, and /api to the back-end service. In your front-end application use only the HTTP path with no host name at all.
The second option is more work to set up, but once you've set it up, it's much easier to develop code for. Webpack has a similar "proxy the backend" option for day-to-day development. This setup means the front-end application itself doesn't care where it's running, and you don't need to rebuild the application if the URL changes (or an individual developer needs to run it on their local system).
Is it possible to deploy a django project without using third party tools like nginx or apache just to serve up https:// webpages? Being forced to setup a reverse proxy or some other web server just to serve https seems a bit overkill.
Using of built-in development server (manage.py runserver) is a bad idea for production environment. But, yes you can use SSL connection even with built-in server
Better idea is to use some application server. For example gunicorn. And yes again, you can serve SSL connection with gunicorn.
Apache or Nginx servers are not just for https. These allows you to effectively control other server resources like max number of processes, request/response headers, etc. WEB servers support many features that you can set without writing python code. And that will be more understandable for infra/server engineers.
I have a basic flask socket.io chat app that's built the same as this example https://github.com/miguelgrinberg/Flask-SocketIO-Chat
I can run the app with gunicorn mychat:app --worker-class eventlet -w 1 --bind 0.0.0.0:5000 --reload and then autossh -M 0 -R myappname:80:localhost:5000 serveo.net and hitting the appropriate serveo.net url.
I'm trying to deploy this on an AWS EC2 instance behind a loadbalancer with HTTPS. On the EC2 instance I installed nginx as here: https://flask-socketio.readthedocs.io/en/latest/#using-nginx-as-a-websocket-reverse-proxy (the single example, not the multiple nodes example).
When I go to the URL I defined from my loadbalancer target group, and i enter a chatroom, I see that in the Network tab of my browser, there are tons of rapid polling requests that keep giving a 200 then a 301. If I join the chatroom in another tab, the users are not able to send message to each other.
What is wrong with my configuration?
After digging around, I figured it out. My host, example.com, points to a load balancer with a bunch of listener rules. I made it so that if path=/mychatapp/*, then forward to the ec2 instance running the app. This is fine, but socket.io was making calls with host=example.com so it was hitting example.com/socket.io/etcetcetc... I added another load balancer listener rule if path=/socket.io/*, then also forward it to the ec2 instance. This works out.
This solution is pretty hacky since it means I couldn't run a second socket.io app. A better answer would be to somehow change the default call from "/socket.io" to "/mychatapp/socket.io" so that I wouldn't require another listener rule. I tried a few ways to do this, but it didn't work. Since this is a small project, I will stick with this solution.
A Django-based web forum I maintain is seeing application level DDOS attacks daily. Essentially, it's an HTTP GET flood hitting the home page, causing the server to execute a large number of internal requests and load various files to create the page. The flood peaks at ~4000 requests per minute (typical throughput being around 500 rpm), bringing down my server in the process.
My webserver is nginx serving as reverse proxy for a gunicorn application server. I know about DDoS mitigation via nginx; this question focuses on gunicorn.
Gunicorn has DDoS related security parameters such as: limit_request_line, limit_request_fields, limit_request_field_size. How do I use them to mitigate an HTTP GET flood? Illustrative examples would be great.
Say we have hosted a few webservices over over https://mycompany.com/Service
e.g.
https://mycompany.com/Service/Service1
https://mycompany.com/Service/Service2
https://mycompany.com/Service/Service3
As you can see on mycompany.com we have hosted 3 webservices each having their distinct urls.
What we have is a Jboss instance with 3 different web wars deployed in it. When someone hits the service it gets past our firewall and then teh load balancer redirects to Jboss on port 8080 on the requried path and it gets serviced.
the 3 services are consumed by 3 different clients. My question if say Client1 using Service 1 is only given out the url corresponding to it can they use some kind of scanner that can also inform them that Service2 and Service3 are alaso available on mycompany.com/Service?
Irrespective of clients - can anyone simply use some scanner tool to identify what Service Endpoints are exposed on the host?
Kindly note they are a mix of SOAP (WSDL) and Rest based services deployed on same instance of Jboss.
Yes, someone can scan for those endpoints. Their scanner would generate a bunch of 404s in your logs, because it would have to guess the other URLs. If you have some kind of rate limiting firewall, it might take them quite a long time. You should be checking the logs regularly anyway.
If you expose your URL to the public internet, relying on people not finding it is just security via obscurity. You should secure each URL using application-level security, and assume that the bad guys already have the URL.
You may want to consider adding subdomains for the separate applications (e.g. service1.mycompany.com, service2.mycompany.com) - this will make firewalling easier.