How to keep Django always loaded in memory? - django

I come from a Java background where the web application is always resident in the memory. This allows it to perform all initialization tasks at the startup itself and, unlike PHP, it does not have to do that again and again for every request.
I see a lot of options to run Django projects but not sure which one of them will allow me to achieve the above? Furthermore I already have a Nginx running at 80 so requests to Django needs to be routed via it.

Django is run by python, and has a process which stays loaded in the memory, much like java. Unlike php, Django will not reload all of its data per request, and it has an application scope.
This is the reason why there are so many options for php hosting, but not as many for Django.
There are a few ways to use Nginx with Django, just google "nginx django" and you get a lot of results which teach you how, for example: https://code.djangoproject.com/wiki/DjangoAndNginx

Related

How to run multiple instances of a Django app?

This question doesn't involve any code. I just want to know a way to run multiple instances of a django app and if it is really possible in the first place.
I make django apps and host them on Apache. With the apps, I noticed a conflict between multiple users accessing the web app.
Let us assume it is a web scraping app. If one user visits the app and run the scraper, the other user accessing the site from a different location doesn't seem to be able to visit the app or run the scraper unless the scraping that the first user started finishes.
Is it really possibe to make it independent for all different users accessing the app?
There are a few ways you could approach this. You might consider putting your app into a container (Google search: docker {your stack})
Then implement something like Docker Swarm or Kubernetes to allocate multiple instances of your app.
That being said, you might consider how to refactor your app to allow multiple users. It sounds like your scraping process locks things. But in reality, there's no reason your server should lock up during this.
It might be better to create your app so that when it received a request, like someone visiting the site, the server pays out the requested web page. When a user asks for a scrape/task to run, the server calls your scaper service or script asynchronously.
That way your app can still function while a scrape is in progress. This would be MUCH more resource efficient (and likely simpler) than spinning up tens or hundreds of instances of your entire app.
tl;Dr: containerization for multiple instances
Refactor app so a single user can't threadlock it.

Django runserver & Firefox - caching conflict with multiple local sites running separately on same port?

I'm working on several Django projects on my local machine, following a single page application architecture. To initiate the server, I have a couple copies of a script in my /bin folder containing
#!/bin/bash
python /path/to/app/manage.py runserver 8080
and have each script with the app name. This makes the application accessible via localhost:8080. In addition, I usually have the majority of my site CSS inside main.css
My issue is that I seem to be coming across a caching issue with Firefox, regardless of which application server is running. Sometimes a page will load with almost no CSS styling, but the jQuery UI elements will be initialized and I can interact somewhat with the application, although the functionality and styling is seriously broken. Refreshing the page shows no improvement, and no errors are shown in the console.
Clearing the cache and changing the port in the scrip seem to solve the issue, but it requires me to have bookmarks for each project, whereas it is pretty convenient to have a single localhost:8080 URL for all projects.
Has anyone come across this issue, and is there a solution other than clearing cache and changing ports?
This thread discusses methods to prevent client side caching of content served by the development server in Django:
Fighting client-side caching in Django
I prefer to simply disable caching in my browser though, seeing that I spend so much time on developing that I don't want to bother with the hassle of trying to prevent it in my own code.
A simple web search for "how to disable caching in firefox" came up with this:
http://support.mozilla.org/en-US/questions/764993
I'm pretty sure that searching for the same thing for different browsers will also give you expected results.
EDIT:
These guys also seem to go pretty in depth about how to prevent the caching of static files when using the Django development server.
Turn off caching of static files in Django development server
Just add something like this to /etc/hosts:
127.0.0.1 site1.dev
127.0.0.1 site2.dev
Visit site1.dev:8080, now site1 has its own cache and cookies (session) in the browser.

Deploying first Django project

I run a small VPS with 512M memory of memory that currently hosts 3 very low traffic PHP sites and a personal email account.
I have been teaching myself Django over the last few weeks and am starting to think about deploying a project.
There seem to be a very large number of methods for deploying a Django site. Given the limited resources I have available, what would be the most appropriate option?
Will the VPS be suitable to host both python and PHP sites or would it be worth getting a separate server?
Any advice appreciated.
Thanks.
There aren't really a great number of ways to do it. In fact, there's the recommended way - via Apache/mod_wsgi - and all the other ways. The recommended way is fully documented here.
For a low-traffic site, you should have no trouble fitting it in your 512MB VPS along with your PHP sites.
Django has documentation describing possible server arrangements. For light weight, yet very robust set up, I'd recommend Nginx setup. It's much lighter than Apache.
I run several low-traffic Django sites on a 256 VPS without problem. I have Nginx setup as a reverse proxy and to serve static files (javascript, CSS, images) and Apache using mod_wsgi for serving Django as described in the documentation.
Running PHP sites as well may add a little overhead, but, if you're talking about low-traffic "fun" sites then you should be fine.

Using Django's built in web server in a production environment

I'm going to setup a simple Django app running in a production environment on a Linux box. The app will have very little traffic - less that 100 page loads per day. Is it okay to use the builtin Django webserver for this or should I install Apache and mod_wsgi? If so, what are the reasons for this? Security perhaps?
UPDATE
OK it is clear I shouldn't be using the builtin server. Some of the alternatives to Apache look interesting. Is there one that is more popular/more frequently used with Django perhaps?
Is it okay to use the builtin Django webserver for this
No.
Should I install Apache and mod_wsgi?
Yes.
If so, what are the reasons for this? Security perhaps?
Partly.
More importantly, the little toy Django server is single-threaded and any hangup in your code hangs the server. This means that when two users click almost at the same time, user one's query must go all the way through Django before user two's query can even starts.
And this will have to include the insanely slow download speed to the desktop.
Apache (like all the alternatives, lighttpd or nginx) is multi-threaded. The slowest part of the transaction is the download from Apache to the desktop. You don't want Python code (and Django) handling this in a single-threaded manner. Even for just a few users.
Also, you don't what Django serving static media (i.e., CSS and JS library files.)
A single slow spot in your application won't effect the overall system throughput if Apache and mod_wsgi are in place. One request 's output page can be slowly downloading to a PC desktop in parallel with another user's output.
DO NOT USE THIS (the builtin Django webserver) SERVER IN A PRODUCTION SETTING. It has not gone through security audits or performance tests.
http://docs.djangoproject.com/en/dev/ref/django-admin/#runserver-port-or-address-port
But you don't have use Apache if you don't want to. You could directly use Spawning, Gunicorn etc.
Cherokee is also easy to setup.
Use nginx + gunicorn.
Nginx: five lines of configuration. Gunicorn: two lines of configuration. That's easy and efficient. For better control you can spawn the gunicorn process using supervisord.
Both gunicorn and supervisord are available to install with pip, and nginx is available in almost any distribution in the default package pool.
The built in Django server was not built for production. There are many reasons why, mainly security and efficiency.
The recommended way is to use mod_wsgi which is covered in the docs here

Can multiple Django sites share memory?

I'm using Apache + mod_wsgi daemon mode for running a Django site. When another site is added (new virtualhost), a second daemon appears.
Is there any way to let those sites share the same proces / memory?
It seems to wasteful to have ~20MB persistently in use per site.
Bonus points: how does this compare to PHP hosting? (especially Drupal/Joomla)
Have a look at Django sites framework.
http://docs.djangoproject.com/en/dev/ref/contrib/sites/
Other than that, the answer is no, as Django use global variables for configuration and so not possible to have same code base dynamically switch what site it runs as on a per request basis.