Is it somehow possible to access the request object inside settings.py?
Maybe by creating a temporary settings object, modifying it and then telling the rest of the "chain" to use that instead of the normal settings.py?
I need it to decide which DB-connection to use.
As an extra question. If I were to have something like 5000 database connections, would settings.py be just as efficient as storing them in a sqlite db on the web-frontend?
And would it be just as painless to update the connections? Or does the server have to be reloaded to catch the changes in settings.py?
Edit: To clarify why I might be needing that many connections.
I am building a webapp. It's SaaS and like many others the accounts will each have a subdomain that they can create users on and will have no need to interact with any other subdomain/account.
It would then be nice to confine each account to a DB all of its own. This grants some extra security and simplifies the app. There are many more advantages to it, but this should illustrate it just fine.
This is why I might end up with that many different databases (but not that many different physical servers if that makes any difference).
If i understand this right, you could use django's new db-routing system and select database on-the-fly based on model instance (e.g. your user) without the need of using() call.
Just adding this for anyone else looking for the same.
It is not currently possible. I have created a feature request on the Django bug-tracker (#13056 i think) and submitted a prototype for a fix, but I don't think it will be included anytime soon and it probably has a lot of bugs in it.
I have moved the project to Flask as it has the g object that is perfectly suited for this.
Django's ORM is not designed to switch database credentials mid-stride. Perhaps you would be happier with something a bit more DIY, such as SQLAlchemy.
I've addressed this problem on a site I've been using recently, and decided to let Apache/mod_wsgi do the work. This solution adds a bit of memory and CPU overhead, but for my app it was the best way to keep everything flexible.
Apache .conf:
SetEnv DJANGO_TEMPLATE_DIR '/usr/local/www/apache22/data/django/templates/'
<VirtualHost *:80>
ServerName encendio.whatever.com
ServerAdmin your_admin#whatever.com
DocumentRoot "/usr/local/www/apache22/data"
SetEnv DJANGO_DATABASE_NAME monkeys
SetEnv DJANGO_DATABASE_USER root
SetEnv DJANGO_DATABASE_PASSWORD secretPass
SetEnv DJANGO_DATABASE_PORT ''
SetEnv DJANGO_DATABASE_HOST ''
WSGIScriptAlias / /usr/local/www/apache22/data/django/wsgi_handler.py
</VirtualHost>
settings.py:
DATABASE_NAME = os.environ.get('DJANGO_DATABASE_NAME', '')
DATABASE_USER = os.environ.get('DJANGO_DATABASE_USER', '')
DATABASE_PASSWORD = os.environ.get('DJANGO_DATABASE_PASSWORD', '')
DATABASE_HOST = os.environ.get('DJANGO_DATABASE_HOST', '')
This allows you to set up each site as a VirtualHost in the httpd.conf.
Related
Django's PREPEND_WWW plus the Common middleware automatically redirects all root domain traffic to the "www." domain. How do I best achieve the opposite? Under django? Apache? Ngnix?
Where, possible, you'll want to do this kind of redirection outside of django. Here are instructions for Nginx and Apache. (You'll need to adjust the config according to desired non-www or www perference.).
As for why to do it outside of django, the answer is performance. There isn't a need here to have django process the request and response, since it can be done earlier in the request lifetime. This means you can save your django processes for when you actually need them.
The only reason you should use PREPEND_WWW, or similar custom middleware, is when you don't have access to the server configuration (such as some shared hosting environments). It's used as a last resort.
As #DanielB it is better for performance to do this outside Django.
But if you want to do it in Django anyway (e.g. you want to learn, or you don't care about performance, or you want to test if your test server doesn't use Apache, or whatever), then you can use this code which I found here:
class RemoveWwwMiddleware():
def process_request( self, request ):
try:
if request.META['HTTP_HOST'].lower().find('www.') == 0:
from django.http import HttpResponsePermanentRedirect
return HttpResponsePermanentRedirect(request.build_absolute_uri().replace('//www.', '//'))
except:
pass
I have a django website that I am hosting on twisted via the django WSGIHandler as described here - http://www.clemesha.org/blog/Django-on-Twisted-using-latest-twisted-web-wsgi
All seems OK up to the point where I want to add an extra "site" configuration to my django site using the django Sites framework. Doing so, I add an extra settings.py file for the new site and that seems to work.
What I then want to do is use the twisted NameVirtualHost class to be able to direct one domain (say site1.example.com) to the first settings file, then use another domain (say site2.example.com) to use the second settings file. This works with Apache & mod_wsgi.
The problem I face is that the twisted code can only access one django environment at a time. If I call setup_environ with the first settings file and setup a host for the first domain, a subsequent call to setup_environ will replace the settings file in use so therefore only one set of settings can be used at one time.
Any ideas how to proceed?
Gave up on this in the end. Looks like you cannot easily access 2 Django environments within the same twisted instance. I think it would require multiple instances of twisted with a reverse proxy or some sort of multiprocess hacking - either way its not worth the effort for me so I'm going to try something else...
Currently I am having three sites for example let it be site1, site2 and site3 . Each site require authentication. Both site1 and site2 take the same database let it be "Portfolio" database and site3 is having a different database let it be "site3specific" database.
I am planning to have a Common Account database for keeping the login credentials of users for the all different sites available. So that each sites (i.e. site1, site2 and site3) will make use of the Common Account database for authenticating the user login. I am planning to keep the user details in a separate database since all the three sites in development, testing and live environment can share the same user credentials without redundancy. Also each site may have its own specific data that we may be having or entering differently in development, staging and live environments.
Also there is a possibility of sharing some data between sites.
Could anyone please tell me how can I achieve these task in django + Apache + mod_wsgi.
Please advice whether I need to have a globally shared settings file , model file and urls file. IF then how my globally shared settings files need to be modified . Please advice.
This is how we currently operate.
Each site has its own VirtualHost entry in the httpd.conf, and each app has its own django.wsgi config file which looks something like this (you can probably use a simpler one):
import os, sys, site, glob
prev_sys_path = list(sys.path)
root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
site.addsitedir(glob.glob(os.path.join(root_dir, 'venv/lib/python*/site-packages'))[0])
sys.path.append('/usr/local/django-apps')
sys.path.append('/usr/local/django-apps/AppName')
new_sys_path = []
for item in list(sys.path):
if item not in prev_sys_path:
new_sys_path.append(item)
sys.path.remove(item)
sys.path[:0] = new_sys_path
os.environ['DJANGO_SETTINGS_MODULE'] = 'AppName.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
The VirtualHost needs to contain entries like this:
SetEnv DJANGO_ENV ${environment
WSGIDaemonProcess appname user=apache group=apache processes=2 threads=15 display-name=%{GROUP}
WSGIProcessGroup appname
WSGIScriptAlias / /usr/local/django-apps/AppName/apache/django.wsgi
<Directory /usr/local/django-apps/AppName/apache>
Order deny,allow
</Directory>
From there, the database set up is dependent on what database engine you're using.
Hope this helps.
You have to look at your requirements, and see if all sites would perhaps require and if so respect a single sign-on (sso) service. If that is the case, then you might need to look at how sessions are transfered between sites as sessions are SITE_ID specific. So, just making it work may be a great start, but looking at the big picture before you dig too deal in might be a good idea.
I set the same session name in these sites (a.xx.com/b.xx.com/c.xx.com -> sesssion name=xx.com). In my Django project, I used three settings files for each site and used the manager.py to separate these sites. The last step, start them up separately.
I'm serving "sensitive" information in downloadable PDF's and Spreadsheets within a user registration section of a site.
Is there a way to allow the django authentication to secure this media without serving it (and not have to manually login using basic auth)?
I'm guessing theres (fingers crossed) not a way to do it with the psuedo code below, but it helps better illustrate the end goal.
#urls.py
(r'^protected_media/(?P<filename>.*)$', 'protected_media')
#views.py
from django.contrib.auth.decorators import login_required
#login_required
def protected_media(request, filename):
# #login_required bounces you out to the login url
# if logged in, serve "filename" from Apache
It seems to me that the method you outlined in your code should work. It's really no different than any other protected resource: your views can serve files from disks, records from databases, rendered templates or anything. Just as the login_required decorator prevents unauthorized access to other views, it will prevent such access to your view serving protected media.
Am I missing something from your question here? Please clarify if that's the case.
EDIT: With regard to the django doc link in your comment: that's the method for simply serving any request file from a particular directory. So, in that example URLS like /site_media/foo.jpg, /site_media/somefolder/bar.jpg will automatically look for files foo.jpg and somefolder/bar.jpg under document_root. Basically, every thing under document_root will be publicly available. That's obviously insecure. So you avoid that with your method.
It's also considered inefficient because django is just adding a lot of unnecessary overhead when all you need is something like Apache to take a URL request and map it to a file on the hard drive. (You don't need django sessions, request processing, etc.)
In your case, this may not be such a big concern. First, you've secured the view. Second, it depends on your usage patterns. How many requests do you anticipate for these files? You're only using django for authentication -- does that justify other overhead? If not, you can look into serving those files with Apache and using an authentication provider. For more on this, see the mod_wsgi documentation:
http://code.google.com/p/modwsgi/wiki/AccessControlMechanisms
see the section "Apache Authentication Provider" and search for django
There are similar mechanisms available under mod_python I believe. (Update: just noticed the other answer. Please see Andre's answer for the mod_python method.)
EDIT 2: With regard to the code for serving a file, please see this snippet:
http://www.djangosnippets.org/snippets/365/
The send_file method uses a FileWrapper which is good for sending large static files back (it doesn't read the entire file into memory). You would need to change the content_type depending on the type of file you're sending (pdf, jpg, etc).
Read this Django ticket for more info. Start at the bottom to save yourself some time. Looks like it just missed getting into Django 1.2, and I assume also isn't in 1.3.
For Nginx, I found this Django snippet that takes advantage of the X-Accel-Redirect header, but haven't tried it yet.
If I understand your question correctly you want to restrict access to files that are not being served by Django, for example, with an Apache server?
What you would then require is some way for this Apache server to use Django as an authentication source.
This django snippet describes such a method. It creates an access handler in Django which is used by Apache when a request for a static file comes in that needs to be protected:
<Location "/protected/location">
PythonPath "['/path/to/proj/'] + sys.path"
PythonOption DJANGO_SETTINGS_MODULE myproj.settings
PythonOption DjangoPermissionName '<permission.codename>'
PythonAccessHandler my_proj.modpython #this should point to accesshandler
SetHandler None
</Location>
Hope this helps, the snippet was posted a while ago, so things might have changed between Django versions :)
More efficient serving of static files through Django is being looked at currently as part of Google SOC project. For WSGI this will use wsgi.file_wrapper extensions for WSGI if available, as it is for mod_wsgi, and req.sendfile() if using mod_python. It will also support returning of headers such as 'Location', 'X-Accel-Redirect' and others, which different web hosting mechanisms and proxy front ends accept as a means of serving up static files where location is defined by a backend web application, which isn't as effecient as front end for serving static files.
I am not sure if there is a project page for this in Django wiki somewhere or not, but the code changes are being committed into the branches/soc2009/http-wsgi-improvements branch of Django source code repository.
You needn't strictly wait for that stuff. It is just putting a clean and portable interface in place across the different mechanisms. If using nginx as front end in front of Apache/mod_wsgi, you could use X-Accel-Redirect now. If using Apache/mod_wsgi 3.0 and daemon mode, you could use Location now, but do need to ensure you set up Apache correct. Alternatively, you could implement your own WSGI middleware wrapper around the Django application which looks for some response header of your own to indicate file to be returned and which uses wsgi.file_wrapper to return that instead of actual response returned from Django.
BTW, the authentication hook mechanisms listed for both mod_python and mod_wsgi by others would use HTTP basic authentication, which isn't what you wanted. This is presuming you want files to be protected by Django form based login mechanism using cookies and backend sessions.
What options are there for installing Django such that multiple users (each with an "Account") can each have their own database?
The semantics are fairly intuitive. There may be more than one User for an Account. An Account has a unique database (and a database corresponds to an account). Picture WordpressMU. :)
I've considered this:
External solution - Multiplex to multiple servers/daemons
Multiple Django installations, with each Django installation / project corresponding to an account that sets its own DATABASE_NAME, e.g.
File system:
/bob
/settings.py (contains DATABASE_NAME="bob")
/sue
/settings.py (contains DATABASE_NAME="sue")
Then having a Django instance running for each of bob and sue. I don't like this methodology- it feels brutish and it smells foul. But I'm confident it would work, and based on the suggestions it might be the cleanest, smartest way to do it.
The apps can be stored elsewhere; the only thing that need be unique to the django configuration is the settings.py (and even there, only DATABASE_NAME, etc. need be different, the rest can be imported).
(Incidentally, I'm using lighttpd and FastCGI.)
Internal solution - Django multiplexing database settings
On the other hand, I've thought of having one single Django installation, and
(a) Adding a "prefix_" to each database table, corresponding to account of the logged-in user; or
(b) Changing the database according to the account of the User that is logged in.
I'd be particularly interested in seeing the "Django way" to do these (hoping that it's something dead-simple). For example, middleware that takes a Request's User and changes the django.conf.SETTINGS['DATABASE_NAME'] to the database for this user's account.
This raises red flags, viz. Is this thread-safe? i.e. Does changing django.conf.SETTINGS affect other processes? Is there just an inherent danger in changing django.conf.SETTINGS -- would the DB connection be setup already? Is restarting the DB connection part of the public API? -- I'm going to have a look at the Django source when I look to this problem again.
I'm conscious that 2(a) and (b) could require User authentication to be stored and accessed in a different mechanism that the core.
For now, I'm going to go with the external mapping at the webserver layer- it's simplest and cleanest for now. However, I don't like the idea of FastCGI daemons running for every account- it seems to needlessly waste memory, particularly if there will be 2000+ accounts. However, I'd like to keep this discussion open as it's an interesting problem and the solution doesn't seem ideal for certain cases.
Comments duly appreciated.
Cheers
The Django way would definitely be to have separate installations with their own database name (#1). #2 would involve quite a bit of hacking with the ORM, and even then I'm not quite sure it's possible at all.
But mind you, you don't need a WHOLE new installation of all the site's models/views/templates for each user, just a new settings.py with all the appropriate paths to the common source files. Plus, to run all these installations in Apache, do it the way I do here:
<VirtualHost 1.2.3.4>
DocumentRoot /www/site1
ServerName site1.com
<Location />
SetHandler python-program
SetEnv DJANGO_SETTINGS_MODULE site1.settings
PythonPath "['/www'] + sys.path"
PythonDebug On
PythonInterpreter site1
</Location>
</VirtualHost>
<VirtualHost 1.2.3.4>
DocumentRoot /www/site2
ServerName site2.com
<Location />
SetHandler python-program
SetEnv DJANGO_SETTINGS_MODULE site2.settings
PythonPath "['/www'] + sys.path"
PythonDebug On
PythonInterpreter site2
</Location>
</VirtualHost>
assuming you've got /www/site1/settings.py, www/site2/settings.py and so on...
Of course, you now need to have a main site where people log in, that then redirects you to the appropriate site (here I've just put it as "site1.com", "site2.com", but you get the idea.)
The Django ORM doesn't provide multiple database support classes, but it is definitely possible - you'll have to write a custom manager and make a few other tweaks. Eric Florenzano has a great article with detailed code samples:
http://www.eflorenzano.com/blog/post/easy-multi-database-support-django/