Django can be run either in Nginx (or some other server, but we are currently going to use Nginx) or with manage.py runserver Django own's server. In both cases I need data integrity and recovery.
For data integrity I need not to terminate (some of) Django requests until they finish. They should not terminate because they should finish started data modification in the DB to preserve data integrity (and no, using SQL transactions is not a solution). They should not terminate as soon as Nginx receives service nginx stop (on Debian Linux) or some other similar command (on other OSes), but finish processing before terminating. How to do this?
For data recovery I want:
create an empty file when the server starts, remove it when the server stops (how to do it both with Nginx and with manage.py runserver?)
When the server starts, if the file is found (indicating a crash of our software), before server starting we need to run my "data recovery" script. How to do this?
A couple things here. First, you should definitely never use runserver in production. Secondly, you don't run really Django in nginx—you run it in a WSGI server, such as uWSGI or Gunicorn. Often, these are run behind nginx, but they don't start and stop when it does.
If you're just trying to make sure that the server doesn't stop while views are still processing, you could definitely do that with uWSGI. There's a pretty good writeup on this subject in the uWSGI documentation called "The Art of Graceful Reloading".
As long as the server is gracefully reloaded, I don't think you need to worry as much about recovery. However, if you still wanted to try that, you'd need to do the empty file deletion in a bash script wrapper—there's no Python code that runs when the server stops.
Related
I am running django server, using gunicorn. Apart from gunicorn, I have a layer of nginx as a load balancer and using supervisord to manage gunicorn.
From the perspective of security is it fine to run my gunicorn server with sudo permission? Is there any potential security leak?
Also, does it makes any difference if I am a superuser and not running process with sudo permission as in any case I have sudo permissions as the user.
Does it need to run as root?
If it doesn't, don't run it as root.
Even better, add a separate user for the app and run it as that user.
I believe the answer to question "is it ok to run xxx with root permissions" should not be "If it doesn't, don't run it as root." but rather a clear "NO".
Every single server and framework is designed to be run without root rights.
What can go wrong? In case you have a vulnerability allowing to remotely execute code on the server you would be simply giving root rights to whoever can exploit it. In case one of your developers in team does something stupid like deleting the root directory, it will be deleted. You don't want that a single app running on the server disrupts your whole system, do you?
It is not a good practice to run any external network facing application with root user privilege.
Consider a scenario where your uploaded file is not validated or sanitized ( file upload vulnerability). If someone uploads some vulnerable file and executes it. Consider that file to have implemented reverse shell. Then it gets easier to take down your server.
Whenever I run a command like flask db migrate or flask db upgrade using the flask-migrate framework, it always starts running my application on localhost, and I have to press CTRL+C to quit before allowing the server to stop and generate the migration. How can I avoid this?
Another question I have is whenever I run, it will first run it in debug mode and after hitting CTRL+C to quit it will again run without debug mode on, on a different port. How do I only limit to running with the former? Thanks.
Somewhere within your application you have a app.run() call. Flask runs your application itself, this extra call is the one that is causing the db commands to run the server before carrying out the command, and also causes the server to run twice when you do flask run. If you find that and remove this line I think you will be fine.
In all standard django productions setup templates I've seen, gunicorn is run with supervisor, whereas nginx/postgres are not configured under supervisor.
Any reason? Is this required for a production system? If not, why not?
In this architecture, Gunicorn works as the application server which runs our Django code. Supervisor is just a process management utility which restarts the Gunicorn server if it crashes. The Gunicorn server may crash due to our bad code, but nginx and postgres remain intact. So in the basic config we only look after the gunicorn process through supervisor. Though we could do the same for nginx and postgres too.
You need supervisor for gunicorn because it's an simply server without any tools to restart it, run it at system startup, stop it at system shutdown or reload when it crashes.
Postgresql and nginx can take care of themselves in that aspect, so there is no need for them to be running under supervisor.
Actually, you can just use init.d, upstart or system.d to start, stop and restart gunicorn, supervisor is just easier way to handle such small servers like gunicorn.
Consider also that it is common to run multiple django apps on one system, and that requires multiple separated instances of gunicorn. Supervisor will handle them better than init, upstart or system.d
There is also uWSGI server that won't need supervisor, because it has built-in features to handle multiple instances, starting, stopping and also auto-reloading on code change. Look at uWSGI emperor system.
I'm working on a Django webapp that's running under nginx and uWSGI. When I deploy new Django code (e.g., settings.py), do I need to restart uWSGI? If so, why?
Background: I had a scenario where I updated settings.py and some other code and deployed it. I did not see the changes in the webapp behavior until I restarted uWSGI.
Yes, you need to restart the uWSGI process.
Python keeps the compiled code in memory so it won't get re-read until the process restarts. The django development server (manage.py runserver) actively monitors files for changes, but that won't happen by default with other servers. If you want to enable automatic reloading in uWSGI, the touch-reload and py-auto-reload uWSGI arguments might help.
Hey I use installed bitnami django 1.3.0,
but whenever I add changes to urls.py or views.py in my system due to some error. The error won't disappear after refresh.
I have to restart my bitnami Service, "stop" and then "start" it, which is time consuming, I feel like I'm coding C# apps in visual studio. Sometimes even that doesn't work, I have to sometimes restart my computer and then I suddenly realize "oh wow, the error is solved now!"
Any solution to this? Why does everything require a runserver / restart?
You can use Apache for deploy your application in production but use the Django server for development. You will need to configure your application for being served by apache later (modifying the settings.py and the apache configuration file) but during the development you won't need to restart the server for every change.
Everything requires a restart because of the way that the python process operates. It does not reload the file when it's changed (outside of runserver..which is an anomaly, and just there for convenience)
Python execution isn't like PHP execution in that way, and your code isn't dynamically loaded on every page refresh, it's loaded on every server restart.
Hope that helps.