I'd like to expose some (app-specific) settings to the admin interface, so users can change them comfortably and also not have to restart Django.
How should I go about this?
I checked out the applications on http://djangopackages.com/grids/g/live-setting/ (btw django-constance was the most appealing) but really what all these apps are doing is storing values in a database, providing a web interface to change them, and caching. Aren't the first two features already built into Django?
The biggest drawbacks I see are that none of the apps are drop-in replacements for the old location of these settings (settings.py), and require me to migrate to their notation, and often add another context processor to access them in templates.
Couldn't I just do this?
Create a model for my settings (this gives me the various types and validation)
Instantiate one such object to hold my settings (this allows the users to edit them in the admin interface) - I could dump defaults as fixtures like for other models
Wrap settings.py so it makes a database query for my settings - http://www.loose-bits.com/2011/04/extending-django-settings-with-derived.html
From my current, naive point of view the only drawbacks I see would be:
Adding or changing the available settings requires a schema migration (south). - I can live with that.
I have a model with possibly multiple instances but really only need a singleton. - That could actually be a useful feature at some point.
Performance/Caching: Looking at http://code.djangoproject.com/svn/django/trunk/django/conf/ I'd have to put a little bit of cleverness into the settings wrapper and/or model, so that model changes clear or update cached values. - doesn't seem to be rocket science.
Doing the same in another project would require a similar effort again. - I think a single dictionary constant in settings.py, holding model name(s) and field names for the lookups is all that would differ.
Wouldn't this be the best of both worlds - runtime admin (with all its perks), database backend, caching, and none of my settings.USED_TO_BE_IN_SETTINGS_DOT_PY would need any changing. Am I missing something?
AFAIK, the Django settings are supposed to be immutable. There are multiple reasons for this, the most obvious being that Django is not aware of the server's execution model (prefork / multi-threaded).
Also, you can't load the settings themselves from a Django model because the settings need to be loaded before you can use anything in the ORM.
So, basically, you have two solutions:
you can bootstrap the settings from the database by using any lower-level database access mechanism to load them; or
you can just define your settings in some other model and fetch them directly when you need them.
The first is an incredible hack and I don't suggest it. The second is much more direct and cleaner, but requires you to change your usual habits (from django.conf import settings).
The second approach is probably what's implemented by the 3rd-party apps you linked to.
From Django 1.8 docs:
You shouldn’t alter settings in your applications at runtime.
DATABASES is a dict. So you can manipulate how a dictionary:
import django.conf as conf
conf.settings.DATABASES['default']['NAME'] = 'novo_banco'
Take a look: https://bitbucket.org/bkroeze/django-livesettings
*Django-Livesettings is a project split from the Satchmo Project_. It provides the ability to configure settings via an admin interface, rather than by editing "settings.py".*
Maybe it can be helpful for you.
Honestly I get more Django when I analyze his code. In version 1.4.5 did it (following the module below):
myproject\manage.py
django\core\management__init__.py ## method - execute_manager
django\conf__init__.py ## class - LazySettings; attr - _wrapped
django\utils\functional.py ## class LazyObject; important method -
new_method_proxy
Functional option, but it has its risks. In the python "_" considers the attribute as protected.
from django.conf import settings
settings._wrapped.INSTALLED_APPS = () ## *really work*
In the following project: https://github.com/alexsilva/DJPlugins
you can see this variable being modified at runtime. the idea of the project is already working.
You can use recomended .configure() method of settings module:
from django.conf import settings
settings.configure(DEBUG=True)
settings module has additional handy features. Check docs.
You cannot directly modify the settings.py file
For example:
If u want change the database at runtime, you should Separate the configuration of the database
# Projecr_name/user_database.py
user_database = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'name',
'USER': 'admin',
'PASSWORD': '111111',
'HOST': '127.0.0.1',
'PORT': '3306'
},
'user_db': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'user1',
'USER': 'admin',
'PASSWORD': '22222',
'HOST': '127.0.0.1',
'PORT': '3306'
}
}
# Projecr_name/settings.py
from .user_database import user_database
...
DATABASES = user_database
...
Call in your logical view
# view.py
from ../Projecr_name/user_database import user_database
class Some(...):
def Some(request):
user_database['user_db']['NAME']='user2'
then you can change any setting at runtime use this way
If you have a variable in the settings.py and you want it to change at any time there is 2 ways the
First one is to make a table in database then make for it a serializer then make a view set and any time you want to get it you could send a http request to this viewset then retrieve the data
Second one is to use caching it is so fast and it is familiar to localStorage in angular
Related
I have a special case where i need to run certain testmethods against one database and other testmethods against another while maintaining the same alias (within a given TestCase).
I am not asking about multiple db connections, the issue is changing a db connection for a given alias within a testmethod, and then reverting it back to the default state after the test method executes.
I have tried the following (this code is placed within my TestCase):
from django.db import connections
from django.db.backends.postgresql.base import DatabaseWrapper
def testSwitchDBParamsForAlias(self):
db_wrapper = DatabaseWrapper(
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'HOST' : 'host',
'PORT' : '2342',
'NAME' : 'dbname',
'USER' : 'user',
'PASSWORD': 'password'
)
connections.__setitem__('dbalias', db_wrapper)
print(connections._databases) # This still has the old connection
In the code above, 'dbAlias' already exists with different credentials and im trying to swap the credentials out using the setitem method.
Another attempt using the context manager:
def testSwitchDBParamsUsingCM(self):
with self.settings(DATABASES = DATABASES):
# call setup_databases here
print(connections._databases) #old credentials printed here
A third approach:
def testSwitchDB(self):
connections = ConnectionHandler(LAKE_TEST_DB)
print(connections._databases)
response = self.client.get(reverse('my_url'), format='json')
The third approach will actually modify the connections object! However this still does not work since in every api call (or view) the connections object is imported again and this change is overriden. So without modifying the API code, this method will also fail.
Neither works and I am left not understanding how this connection object works at all. Any ideas/suggestions?
This may be useful for you: https://docs.djangoproject.com/en/2.1/topics/db/multi-db/#manually-selecting-a-database
As long as you register both databases in your settings file, you should be able to route specific requests to either database with the using() method.
My Django settings.py contains the following:
DATABASES = {
'default': { # several settings omitted
'NAME': 'myproject', 'HOST': 'localhost', },
'other': { 'NAME': 'other', 'HOST': 'else.where', }
}
I now want to fetch some objects from the other DB and save it to default like so:
things = Thing.objects.using('other').raw('SELECT code,created FROM other.otherapp_bigthing WHERE created>...')
# 'code' is a unique string in other and PK of the Thing model
for t in things:
t.save(using='default')
This gives me
ProgrammingError: (1146, "Table 'other.myapp_thing' doesn't exist")
which is a correct observation, however, by the documentation of the using parameter I expected the records to be saved to myproject.myapp_thing. Why is the database name still taken from the other configuration when I explicitly advised it to use default?
I eventually worked around this by making a raw insert (connections['default'].cursor().execute('INSERT INTO myproject.myapp_thing (...) ON DUPLICATE KEY UPDATE...') but still wanted to know why the question's code does not work.
The problem was that the Thing model contained another field: next to the code CharField/PK and created date (in my use case actually last_accessed) there was another date (think modified) with default=None, null=True and I assumed the default would be used if I do not explicitly include in the original raw query.
Apparently, this assumption was wrong and Django tried, on t.save, to look up a value for modified in the DB it originally came from -- and failed, since it came from a raw SQL query.
If I include ,NULL AS 'modified' in the raw SQL query the original error goes away but I still cannot reasonably use Django logic for my use case since t.save(using='default') would, for existing things, update last_accessed and overwrite the modified date with NULL, which I do not want, and t.save(using='default', update_fields=['last_accessed']) would force an update and thus fail for new things.
Well, I have a django project which works fine now.
I'd like to add a new app to it, in which I need to access multiple databases.
I know Django support multiple databases settings and know how to configure it. This is not the problem.
The issue is that, in the 90% of my project components, I don't need to maintain a multiple databases settings. The only usage for the second databases is in the new added app.
So I tried to alter the settings by calling:
django.conf.settings.configure(DATABASES = {....})
in the new app. And django said:
RuntimeError: Settings already configured.
Which makes sense, since I have the origin settings file and set the DJANGO_SETTINGS_MODULE up.
So I question is that what should be a good approach in this case.
I don't want to discard DJANGO_SETTINGS_MODULE variable.
I try to not include second database in the setting file initially, since the new app is also an independent module which should work independently outside the django project. So I want to have the similar config in new app to setup the database config.
Does anyone has any idea about this?
Thanks in advance!
Actually, I have the same issue in a current project. As you I have a totally independent app which uses an another database, and I could have other apps which could have the same behaviour.
The thing that I have done is to create a dir apps where I store my apps and then I add this at the end of my settings.py file :
DATABASE_ROUTERS = ['myproject.routers.MultiDBRouter']
import os
APPS_DIR = os.path.join(PROJECT_ROOT, 'apps')
for app_name in os.listdir(APPS_DIR):
print '\nLooking for settings in apps/%s :' % app_name
if os.path.exists(os.path.join(APPS_DIR, app_name, 'settings.py')):
print ' Settings file found...'
app = __import__('%s.settings' % app_name)
content = dir(app.settings)
if 'DATABASES' in content:
print ' Adding databases :'
for key, value in app.settings.DATABASES.iteritems():
if DATABASES.has_key(key):
print ' Can not add %s database config, because it already exists' % key
else:
DATABASES[key] = value
DATABASES[key]['APPS'] = [app_name]
print ' Added %s database config' % key
It will automatically look after settings.py file in all the apps/myapp/ directories. If it finds a new DATABASES variable in a app/myapp/settings.py file, it will add the other database configurations to your DATABASES variable (the true one).
I have also created a router to do not have to use the using command (the MultiDBRouter).
And then I add a settings.py file in all the app which requires another database :
DATABASES = {
'db': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database',
'USER': 'username',
'PASSWORD': 'mysecretpassword',
}
}
I've been evaluating django and wondered if the following is possible. I've already looked at the regular multiple database docs so please don't point me to that because this use case isn't mentioned as far as i can make out. If i'm wrong i take it back :)
I want one main database in which most of my app's models will reside, however one of the app's will need to dynamically create databases, these will be customer specific databases.
The database path (i plan to use sqlite) will be stored in primary database and so the cursor would need to be changed but the model will remain the same.
I would welcome any thoughts on ways to achieve this?
I will open with "You should not edit settings at runtime".
Having said that, I have exactly this same issue, where I want to create a unique database for each user. The reason for doing this is I am offering the ability for the user to save/access to/from a database not stored on my server, which entails having multiple databases, and thus one for each user.
This answer is NOT the recommended way to achieve the desired goal. I would love to hear from a django-guru how to best approach this problem. However, this is a solution I have been using and it has worked well so far. I am using sqlite however it can be easily modified for any of the databases.
In summary, this is the process:
Add the new database to settings (at runtime)
Create a file to store these settings for reloading when the server is restarted (at runtime)
Run a script which loads the saved settings files (whenever the server is restarted)
Now, how to achieve this:
1) Firstly, when a new user is created, I create a new database in the settings. This code lives in my view where new users are created.
from YOUR_PROJECT_NAME import settings
database_id = user.username #just something unique
newDatabase = {}
newDatabase["id"] = database_id
newDatabase['ENGINE'] = 'django.db.backends.sqlite3'
newDatabase['NAME'] = '/path/to/db_%s.sql' % database_id
newDatabase['USER'] = ''
newDatabase['PASSWORD'] = ''
newDatabase['HOST'] = ''
newDatabase['PORT'] = ''
settings.DATABASES[database_id] = newDatabase
save_db_settings_to_file(newDatabase) #this is for step 2)
This script loads the database settings 'at runtime' into the django project settings. However if the server is restarted, this database will no longer be in settings.
2) To facilitate reloading these settings automatically whenever the server is restarted, I create a file for each database which will be loaded whenever the server is started. Creating this file is performed by the function save_db_settings_to_file:
def save_db_settings_to_file(db_settings):
path_to_store_settings = "/path/to/your/project/YOUR_PROJECT_NAME/database_settings/"
newDbString = """
DATABASES['%(id)s'] = {
'ENGINE': '%(ENGINE)s', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
'NAME': '%(NAME)s', # Or path to database file if using sqlite3.
'USER': '', # Not used with sqlite3.
'PASSWORD': '', # Not used with sqlite3.
'HOST': '', # Set to empty string for localhost. Not used with sqlite3.
'PORT': '', # Set to empty string for default. Not used with sqlite3.
}
""" % db_settings
file_to_store_settings = os.path.join(path_to_store_settings, db_settings['id'] + ".py")
write_file(file_to_store_settings, newDbString) #psuedocode for compactness
3) To actually load these settings when the server is started, I add a single line to the very bottom of /path/to/your/project/YOUR_PROJECT_NAME/settings.py, which loads each file in the settings folder and runs it, having the effect of loading the database details into the settings.
import settings_manager
Then, import settings_manager will load the file at /path/to/your/project/YOUR_PROJECT_NAME/settings_manager.py, which contains the following code:
from settings import DATABASES
import os
path_to_store_settings = "/path/to/your/project/YOUR_PROJECT_NAME/database_settings/"
for fname in os.listdir(path_to_settings):
full_path = os.path.join(path_to_settings, fname)
f = open(full_path)
content = f.read()
f.close()
exec(content) #you'd better be sure that the file doesn't contain anything malicious
Note that you could put this code directly at the bottom of settings.py instead of the import statement, but using the import statement keeps the abstraction level of settings.py consistent.
This is a convenient way to load each database setting because to remove a database from the settings all you have to do is delete the settings file, and the next time the server restarts it won't load those details into the settings, and the database will not be accessible.
As I said, this works and I have had success using it so far, but this is NOT the ideal solution. I would really appreciate if someone could post a better solution.
What's bad about it:
It explicitly defies advice from django team not to modify settings at runtime. I do not know the reason for why this advice is given.
It uses an exec statement to load the data into settings. This should be OK, but if you get some corrupt or malicious code in one of those files you will be a sad panda.
Note that I still use the default database for auth and sessions data, but all the data from my own apps is stored in the user-specific database.
To augment #thedawnrider's answer, in some cases editing settings.DATABASES may not be enough. It might be more reliable to edit django.db.connections.databases, which serves as a cache and wrapper around settings.DATABASES.
e.g.
from django.db import connections
database_id = user.username #just something unique
newDatabase = {}
newDatabase["id"] = database_id
newDatabase['ENGINE'] = 'django.db.backends.sqlite3'
newDatabase['NAME'] = '/path/to/db_%s.sql' % database_id
newDatabase['USER'] = ''
newDatabase['PASSWORD'] = ''
newDatabase['HOST'] = ''
newDatabase['PORT'] = ''
connections.databases[database_id] = newDatabase
This question is pretty outdated and should be updated.
Django support multiple databases out of the box
Here is many good tutorials [ 1, 2 ] and answers how to manage connections dynamically
Also you may simply use django-dynamic-db-router
Database connection settings may be configured on the fly by using configure method of settings object:
from django.conf import settings
dbs = settings.DATABASES.copy()
dbs['some_new_db'] = {'ENGINE': 'dummy'}
settings.configure(DATABASES=dbs)
I am trying to de-couple two apps:
Locations - app containing details about some location (town, country, place etc)
Directory - app containing details of places of interest (shop, railway station, pub, etc) - all categorised.
Both locations.Location and directory.Item contain lat/lng coords and I can find items within a certain distance of a specific lat/lng coord.
I'd like to use the following URL structure:
/locations/<location_slug>/directory/<category_slug>/
But I don't want to make my directory app reliant on my location app.
How can I translate this url to use a view like this in my directory app?
items_near(lat, lng, distance, category):
A work around would be to create a new view somewhere that translates this - but where should I put that? if it goes in the Directory app, then I've coupled that with my Location app, and vice versa.
Would a good idea be to place this workaround code inside my project URLs file? Thus keeping clear of both apps? Any issues with doing it like this?
For your urlpattern to work, the view function invoked has to be aware of both Locations and Directories. The short answer is you can put this view function anywhere you want - it's just a python function. But there might be a few logical places for it, outside of your Directory or Location app, that make sense.
First off, I would not put that view code in in your top-level urls.py, as that file is intended for URLconf related code.
A few options of where to put your view:
Create a new view function in a file that lives outside of any particular app. <project_root>/views.py is one possibility. There is nothing wrong with this view calling the item_near(..) view from the Directory app.
# in myproject/urls.py
urlpatterns = (
...
(r'/locations/(?P<location_slug>[\w\-]+)/directory/(?P<category_slug>[\w\-]+)/',
'myproject.views.items_near_from_slug')
)
# in myproject/views.py
from directory.views import items_near
def items_near_from_slug(request, location_slug, category_slug):
location = get_object_or_404(Location, slug=location_slug)
distance = 2 # not sure where this comes from
# And then just invoke the view from your Directory app
return items_near(request, location.lat, location.lng, distance, category_slug)
Create a new app and put the code there, in <my_new_app>/views.py. There is no requirement that a Django app need to have a models.py, urls.py, etc. Just make sure to include the __init__.py if you want Django to load the app properly (if, for instance, you want Django to automatically find templatetags or app specific templates).
Personally, I would go with option 1 only if the project is relatively simple, and <project_root>/views.py is not in danger of becoming cluttered with views for everything. I would go with option 2 otherwise, especially if you anticipate having other code that needs to be aware of both Locations and Directories. With option 2, you can collect the relevant urlpatterns in their own app-specific urls.py as well.
From the django docs here if you're using django >=1.1 then you can pass any captured info into the included application. So splitting across a few files:
# in locations.urls.py
urlpatterns = ('',
(r'location/(?P<location_slug>.*?)/', include('directory.urls')),
#other stuff
)
# in directory.urls.py
urlpatterns = ('',
(r'directory/(?P<directory_slug>.*?)/', 'directory.views.someview'),
#other stuff
)
# in directory.views.py
def someview(request, location_slug = None, directory_slug = None):
#do stuff
Hope that helps. If you're in django < 1.1 I have no idea.
Irrespective of how much ever "re-usable" you make your app, inevitably there is a need for site-specific code.
I think it is logical to create a "site-specific" application that uses the views of the reusable and decoupled apps.