Atomic Database Transactions in Django - django

I want to have atomic transactions on all the requests in the Django app.
There is the following decorator to add an atomic transaction for one request,
#transaction.atomic
def create_user(request):
....
but then, in this case, I'll have to use this decorator for all the APIs.
I tried doing this in settings.py:
DATABASES = {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'blah',
'USER': 'postgres',
'PASSWORD': 'blah',
'HOST': '0.0.0.0',
'PORT': '5432',
'ATOMIC_REQUESTS': True
}
But adding ATOMIC_REQUESTS does not seem to work in some cases.
Is there any way so that I can apply atomic transactions for all APIs?

Not quite sure if this the case but in the documentation, explains that you need to pass 'using' param to this decorator which is what database this operation applies to.
so
can you try this
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'blah',
'USER': 'postgres',
'PASSWORD': 'blah',
'HOST': '0.0.0.0',
'PORT': '5432',
'ATOMIC_REQUESTS': True,
},
}
then like this:
#transaction.atomic(using='default')
def create_user(request):
I also realized that you used psycopg2 as your engine
Changed in Django 1.9:
from the docs:
The django.db.backends.postgresql backend is named
django.db.backends.postgresql_psycopg2 in older releases. For
backwards compatibility, the old name still works in newer versions.
fyi

Related

Django how to use separate database for default authentication

I have a database created from non-django app and have defined the database connection info like below
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': os.environ.get('POSTGRES_DB'),
'USER': os.environ.get('POSTGRES_USER'),
'PASSWORD': os.environ.get('POSTGRES_PASSWORD'),
'HOST': os.environ.get('POSTGRES_HOST'),
'PORT': os.environ.get('POSTGRES_PORT'),
}
}
Unfortunately, when I migrate the django admin related tables, they all go into the default database.
I know that it is possible to declare multiple databases and separate read and write actions to different databases; however, what I would like to do is to have all the default django admin related tables to be created into another database. Say I declare a second database like below, how do I make sure that django admin related datas get migrated to the second database and also read from that when I login to django admin?
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': os.environ.get('POSTGRES_DB'),
'USER': os.environ.get('POSTGRES_USER'),
'PASSWORD': os.environ.get('POSTGRES_PASSWORD'),
'HOST': os.environ.get('POSTGRES_HOST'),
'PORT': os.environ.get('POSTGRES_PORT'),
},
'admin':{
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': os.environ.get('ADMIN_POSTGRES_DB'),
'USER': os.environ.get('ADMIN_POSTGRES_USER'),
'PASSWORD': os.environ.get('ADMIN_POSTGRES_PASSWORD'),
'HOST': os.environ.get('ADMIN_POSTGRES_HOST'),
'PORT': os.environ.get('ADMIN_POSTGRES_PORT'),
}
}
first you must leave default dictionary as empty
then define your database for admin and non admin
for examample do this
DATABASES = {
'default':{},
'nonAdmin': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': os.environ.get('POSTGRES_DB'),
'USER': os.environ.get('POSTGRES_USER'),
'PASSWORD': os.environ.get('POSTGRES_PASSWORD'),
'HOST': os.environ.get('POSTGRES_HOST'),
'PORT': os.environ.get('POSTGRES_PORT'),
},
'admin':{
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': os.environ.get('ADMIN_POSTGRES_DB'),
'USER': os.environ.get('ADMIN_POSTGRES_USER'),
'PASSWORD': os.environ.get('ADMIN_POSTGRES_PASSWORD'),
'HOST': os.environ.get('ADMIN_POSTGRES_HOST'),
'PORT': os.environ.get('ADMIN_POSTGRES_PORT'),
}
}
then after that you can migrate database using
./manage.py migrate --database=admin
or
./manage.py migrate --database=nonAdmin
If you don’t want every application to be synchronized onto a particular database, you can define a database router that implements a policy constraining the availability of particular models.
by using this method
allow_migrate(db, app_label, model_name=None, **hints)
you can reference this document https://docs.djangoproject.com/en/3.0/topics/db/multi-db/#topics-db-multi-db-routing

What is the value of self._db by default in Django?

Please see the following code:
user.save(using=self._db)
What is the default value of self._db for Django?
Does this value default to what I've specified under default for database in my settings.py?
I've found questions on Stack Overflow that say this value will provide a database type to Django, but if I've never set it explicitly, what is it by default?
Django default managers use using parameter to define which database underlying the manager should use for operation. This will optionally use. This is used in case you have multiple databases by which you define which database you need to use for operation.
An example user.save(using=self._db) usually defined as "default" from your database configuration in settings.py. For more info click here
Behind the scene self._db set as None. If user.save(using=None), then it will use default database.
For example, your database configuration is like
DATABASES = {
'default': {
'NAME': 'app_data',
'ENGINE': 'django.db.backends.postgresql',
'USER': 'postgres_user',
'PASSWORD': '****'
},
'new_users': {
'NAME': 'user_data',
'ENGINE': 'django.db.backends.mysql',
'USER': 'mysql_user',
'PASSWORD': '****'
}
}
Then if you want to use default database then use user.save(using=self._db)
If you want to use new_users database then use user.save(using="new_users")
Yes, Its default value is specified in settings.py
as
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'mydatabase',
'USER': 'mydatabaseuser',
'PASSWORD': 'mypassword',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}
as mentioned here
Django default managers use using parameters to define which database underlying the manager should use for operation. This will optionally use. This is used in case you have multiple databases by which you define which database you need to use for operation.
An example user.save(using=self._db) is usually defined as "default" from your database configuration in settings.py.
Behind-the-scene self._db set as None. If user.save(using=None), then it will use the default database.
For example, your database configuration is like
DATABASES = {
'default': {
'NAME': 'app_data',
'ENGINE': 'django.db.backends.postgresql',
'USER': 'postgres_user',
'PASSWORD': '****'
},
'new_users': {
'NAME': 'user_data',
'ENGINE': 'django.db.backends.mysql',
'USER': 'mysql_user',
'PASSWORD': '****'
}
}
Then if you want to use default database then use user.save(using=self._db) If you want to use new_users database then use user.save(using="new_users")

Django - Adding a second database entry causes failure. Django no longer uses 'default'

My database and project work great using this database setting:
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'myprojectdb',
'USER': 'myprojectuser',
'PASSWORD': 'my_secret_password',
'HOST': 'localhost',
'PORT': '5432',
'ATOMIC_REQUESTS': True
},
}
But I want to add a 'readonly' database entry for my readonly db user, like this, so that I can run django-sql-explorer:
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'myprojectdb',
'USER': 'myprojectuser',
'PASSWORD': 'my_secret_password',
'HOST': 'localhost',
'PORT': '5432',
'ATOMIC_REQUESTS': True
},
'readonly': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'myprojectdb',
'USER': 'myprojectreadonly',
'PASSWORD': 'your_eyes_only_secret',
'HOST': 'localhost',
'PORT': '5432',
'ATOMIC_REQUESTS': True
}
}
And now django throws a couple different errors. If I try to do anything with migrations, I get:
django.db.utils.ProgrammingError: permission denied for relation django_migrations
If I try to runserver, I get:
"AUTH_USER_MODEL refers to model '%s' that has not been installed" % settings.AUTH_USER_MODEL
django.core.exceptions.ImproperlyConfigured: AUTH_USER_MODEL refers to model 'users.User' that has not been installed
It seems like when I add the second database setting, django is attempting to use that for migrations, running the server, etc, even though it's not 'default'
If I comment out the second entry, everything works great again.
Any recommendations on how to correct this?
Edit: If I change the settings to use the same username and password, everything works great, so it's not just an issue of django being confused on names. IE:
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'myprojectdb',
'USER': 'myprojectuser',
'PASSWORD': 'my_secret_password',
'HOST': 'localhost',
'PORT': '5432',
'ATOMIC_REQUESTS': True
},
'readonly': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'myprojectdb',
'USER': 'myprojectuser',
'PASSWORD': 'my_secret_password',
'HOST': 'localhost',
'PORT': '5432',
'ATOMIC_REQUESTS': True
}
}
The two configurations shares the same database names 'NAME': 'myprojectdb'
So when you try to migrates dbname seems to conflicts so the applications is having an indecision on which to use The previous or the later

Multiple databases (mongodb[mongoengine] and sql ) with django 1.8

I am using mongoengine with Django and my project needs to connect to one instances of MongoDB while another with sql .How my databse section of setting.py should be like ?
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'admin_db',
'USER': 'root',
'PASSWORD': 'root',
'HOST': 'localhost',
},
}
from mongoengine import connect
connect(
db='pom',
username='admin',
password='root',
host='mongodb://admin:root#localhost'
)
You could add multiple databases for your app in your settings.py like,
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'admin_db',
'USER': 'root',
'PASSWORD': 'root',
'HOST': 'localhost',
},
'your_desired_db_name' : {
'ENGINE' : 'django_mongodb_engine',
'NAME' : 'db_name'
}
For integration with mongodb, you may need to look up,
Django-nonrel
Django-MongoEngine
Also, you may need to look up Django documentation for multiple databases
MongoEngine does not support all Django contrib modules directly. If your projects dont need them (unlikely) you can use mongoengine directly. Otherwise you can also try
djongo
Which seems to work fine with the latest Django version.

Is there a way to tell django-nose to handle multiple databases?

(Cross-posted here: https://github.com/jbalogh/django-nose/issues/129)
In settings.py I have two databases listed:
DATABASES = {
'default': {
'ENGINE': 'mysql_pymysql',
'NAME': 'OST_DEV_1',
'USER': 'root',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '3306',
},
'umc': {
'ENGINE': 'mysql_pymysql',
'NAME': 'UMC',
'USER': 'root',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '3306',
}
}
If I run the tests without REUSE_DB, they work, but slowly (nearly 2 minutes spent just on creating/destroying the databases):
bash
./manage.py test myapp
But this fails:
REUSE_DB=1 ./manage.py test myapp
DatabaseError: (1146, u"Table 'OST_DEV_1.tblMfg' doesn't exist")
This makes sense, as tblMfg is in the UMC database, not OST_DEV_1. Is there a way to tell django-nose where to find tblMfg? Note that my tests themselves don't reference tblMfg -- I'm sticking to 1+1 == 2 for now.
We currently use 'using' manually for tables that are in UMC:
active_mfgs = Mfg.objects.using('umc').filter(status="ACTIVE")