I was beginning to write some tests that involve async tasks, and I realised these tests (using pytest-django) were failing because the async job isnt pointed at the right database. How does one point django-q /qcluster to look at the test database that pytest uses.
I am mostly using the default settings from Django.
DATABASE related settings.py looks like this
'default': { 'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db.sqlite3',
'TEST': { 'NAME': BASE_DIR / 'testdb.sqlite3'}
I am running this to check what is the db being connected to.
from django.db import connection
db_name = connection.settings_dict['NAME']
When the function is being run in non async mode - it rightly points to the testdb.sqlite3.
When run using async mode, it points to the db.sqlite3.
How do i make the async task to refer to the testdb.
Any pointers would be very helpful.
Related
Well, I had one setting file before and it was working perfectly both in local and AWS of course with both the database configuration in one setting file. When ever I use to push my code on AWS I would comment my local db configuration and then push. So for sure, that's annoying. I can't comment and uncomment constantly both the database configuration in one file.
Therefore, I decided to have 2 setting files. one for the local and the other for AWS.
Once I pulled the code into AWS server and run migrations
python manage.py migrate --settings=settings.staging
It worked and migrated. By the way, this is the setting file which resides my RDS configuration. Now the moment I hit the endpoints via postmant the output is as
OperationalError at /account/v1/login
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
means it is still reading the default settings file. How come I make the server run this particular staging setting file. Do I have to declare it on nginx, supervisor or gunicorn?
I'm using these 3 services as well.
Below is my settings file for staging.
from .base import *
# --------------- AWS RDS ---------------
DATABASES = {
'default': {
# 'ENGINE': 'django.db.backends.sqlite3',
# 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'db name here',
'HOST': 'RDS host here',
'USER': 'postgres',
'PASSWORD': 'pass here',
'PORT': '5432'
}
}
This is my complete staging.py file. which only have this while the rest of the setting is being imported from base.py which is a default setting file. Also it has the local settings.
Any recommendations?
This is also what I've tried.
import socket
hostname = socket.gethostname()
if hostname == "staging":
from settings.staging import *
Thank you
The way I solved this issue is using hostnames. We have 1 settings file with all the default settings. We then import specific files depending on the hostname. Of course you could also use things like IAM-instance roles or something.
We would have this in the default settings file:
import socket
DATABASE = {'default': {'ENGINE': 'django.db.backends.sqlite3', ...}}
hostname = socket.gethostname()
if hostname == "staging-blabla"
from staging import *
staging.py would contain the following:
DATABASE = {'default': {'ENGINE': 'django.db.backends.postgresql_psycopg2', ...}}
This question already has an answer here:
Django: Detect database backend
(1 answer)
Closed 3 years ago.
I'm hoping this will be similar to a few questions which have been previously answered.
I was wondering what the full processes are for testing the DB connection string (not the DB name) for the current connected DB in Django is? I see a few methods for printing the name, but across local development and staging, these names will be the same. I need to know more, such as the db engine (e.g., is it PostgreSQL, or is it db.sqlite3. etc etc), and various other DB parameters.
I'm currently trying to debug why a Celery beat task is failing to create database entries for the application DB I think I am connected to.
I'm performing a simple obj, created = Object.objects.get_or_create() DB method, and on logging created, some are True and some are False. All good so far.
However, the admin section of the Django CMS is not showing any more entries to the DB.
I believe testing the location and connection strings of the DB the Django application is using might be useful in debugging this ghost object creation...unless someone can advise on similar issues they have had with the Celery daemon not actually persisting a DB create/update etc...
Perhaps, something such as:
from django.db import connection
print(connection.settings_dict)
Would be sufficient?
So, the following was useful for determining the DB engine:
from django.db import connection
print(connection.settings_dict)
For useful guidance, the connection.settings_dict contains the following structure:
{
'ENGINE': 'django.db.backends.sqlite3',
'NAME': '/usr/src/app/db.sqlite3',
'ATOMIC_REQUESTS': False,
'AUTOCOMMIT': True,
'CONN_MAX_AGE': 0,
'OPTIONS': {},
'TIME_ZONE': None,
'USER': '',
'PASSWORD': '',
'HOST': '',
'PORT': '',
'TEST': {
'CHARSET': None,
'COLLATION': None,
'NAME': None,
'MIRROR': None
}
}
So, we could work with something like this in a management command:
from django.db import connection
from django.core.management.base import BaseCommand
class Command(BaseCommand):
help = 'Displays The Currently Connected DB Settings'
def handle(self, *args, **kwargs):
print(
"The current database engine is {ENGINE}".format(**connection.settings_dict)
)
print(
"Currently connected to host {HOST} on port {PORT} ".format(**connection.settings_dict)
)
I have a Django application connectinig to 2 database.
Both of them as seen by Django as PostgreSQL database but one of them is not a PostgreSQL (but uses the PostgreSQL binary protocol so it uses the same driver (CockroachDB).
Currently it looks like during test runs, Django try to run the migration on both databases.
How can I avoid that ? I still need the database to be accessible during the tests, but no to run the migration (As it's not compatible for now and the migrations are run outside from Django)
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
...
},
'livedb': {
'ENGINE': 'django.db.backends.postgresql',
...
}
}
You can add a database router that prevents migration. Assuming livedb is the DB you do not want to migrate, it would look like this:
class LiveDBRouter:
def allow_migrate(self, db, app_label, model_name=None, **hints):
"""
Do not allow migrations on livedb.
"""
return db != 'livedb'
Then you configure that router:
DATABASE_ROUTERS = ['path.to.LiveDBRouter']
I put a Django app on Heroku recently. The home page looks fine, but when I try to go to a page that involves making a query (e.g. p = Photo.objects.get(title=title)), I get this error:
could not connect to server: Connection refused
Is the server running on host "localhost" and accepting
TCP/IP connections on port 5432?
In accordance with this answer, I did $ heroku pg:promote HEROKU_POSTGRESQL_GREEN_URL
Then in my settings.py:
DATABASES = {'default': dj_database_url.config(default=os.environ['DATABASE_URL'])}
Still got the same error, so I tried looking at the results of this (as this answer suggests):
$ heroku run python manage.py shell
>>> from django.conf import settings
>>> print settings.DATABASES['default']
{'TIME_ZONE': 'UTC', 'TEST_MIRROR': None, 'NAME': 'snorthway', 'OPTIONS': {},
'HOST': 'localhost', 'TEST_NAME': None, 'PASSWORD': '******', 'ENGINE':
'django.db.backends.postgresql_psycopg2', 'PORT': '', 'USER': 'snorthway',
'TEST_COLLATION': None, 'TEST_CHARSET': None}
At which point I realized I don't know what I should even be looking for in that. I still don't understand what the error means, so I am unsure how to go about debugging it.
You have not configured your django database correctly in settings.py. It thinks your database is on localhost. Sounds like you have a heroku postgres database so your host should be something like:
df3-64-304-50-250.compute-1.amazonaws.com
Heroku exposes a special database URL through an environment variable called:
DATABASE_URL
There is a very cool python package here called dj_database_url: https://github.com/kennethreitz/dj-database-url it converts that environment variable to what django expects.
you can install it with:
$pip install dj-database-url
I use the following in my settings.py
import dj_database_url
DATABASES = {
'default': dj_database_url.config()
}
I'm writing a desktop application with PyQt where we planned to use sqlite3-databases for file storage (instead of pickles, XML, YAML, etc). The reason is that our application is likely to migrate to a centralized data store later. (which then needs to communicate with other web based services, etc etc.)
Before everyone says "use SQLAlchemy and Elixir", I'd like to point out why chose Django, namely because:
I know Django pretty well, it's neat and I like it's ORM.
When we migrate, it's easy to add a web-ui on top of it.
Having the Admin-interface makes it easy to debug/inspect the DB during development.
Anyway, my problem is that I can't select different sqlite3 databases, since Django's settings.configure throws an 'already configured' error on the second call.
Any ideas, apart from restarting the app?
(None of many Django-desktop-orm questions here on SO seem to address this...)
Paraphrasing http://docs.djangoproject.com/en/dev/topics/db/multi-db/
Define multiple DBs in the settings.py.
DATABASES = {
'default': {
'NAME': 'defaultdb',
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'USER': 'postgres_user',
'PASSWORD': 's3krit'
},
'other': {
'NAME': 'otherdb',
'ENGINE': 'django.db.backends.mysql',
'USER': 'mysql_user',
'PASSWORD': 'priv4te'
}
}
Then you can choose the database manually.
>>> # This will run on the 'default' database.
>>> Author.objects.all()
>>> # So will this.
>>> Author.objects.using('default').all()
>>> # This will run on the 'other' database.
>>> Author.objects.using('other').all()