I am having a problem getting django to recognize my heroku postgres db. Usually the following configuration work fine, but for some reason it has stopped working on my most recent application.
In settings.py, I added the following code.
import dj_database_url
DATABASES['default'] = dj_database_url.config()
print DATABASES['default']
it prints out {'default':{}}
the heroku pg:info command returns what looks to be normal:
=== HEROKU_POSTGRESQL_WHITE_URL
Plan: Dev
Status: available
Connections: 1
PG Version: 9.2.4
Created: 2013-08-06 15:40 UTC
Data Size: 6.3 MB
Tables: 0
Rows: 0/10000 (In compliance)
Fork/Follow: Unsupported
When I go to use the database it returns
settings.DATABASES is improperly configured.
Any thoughts on why dj_database_url.config() is not properly parsing the database configurations?
Thank you for the thoughts.
EDIT:
Still don't know the reason for the above behavior, but I found a work around. I had to add the postgres db to the app before the initial push. For some reason it wasn't finding it if it was added after the first push.
This should be your db settings
DATABASES['default'] = dj_database_url.config(default='postgres://user:passwprd#host/db')
DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
Related
I am running a web application using Django and Django Rest Framework on Heroku with a postgresql and redis datastore. I am on the free postgresql tier which is limited to 20 connections.
This hasn't been an issue in the past, but recently I started using django channels 2.0 and the daphne server (switched my Procfile from gunicorn to daphne like this tutorial) and now I have been running into all sort of weird problems.
The most critical is that connections to the database are being left open so as the app runs, the number of connections keep increasing until it reaches 20 and gives me the following error message: Operational Error - 'FATAL too many connections for role "usename"'
Then I have to manually go to shell and type heroku pg:killall each time, this is obviously not a feasible solution and this is production so my users cant get access to site and get 500 errors. Would really appreciate any help.
I have tried:
Adding this to my different views in different places
from django.db import connections
conections.close_all()
for con in connections:
con.close()
I also tried doing SELECT * from pg_activity and saw a bunch of stuff but have no idea what to make of it:
We figured out whats the problem. I assume that you are using dj_database_url like in heroku manual. All you have to do is to drop conn_max_age.
db_from_env = dj_database_url.config()
There is the solution:
Nowadays heroku provide the django_heroku package that deal with default django-heroku app configuration, so when you call django_heroku.config(locals()) on the end of your settings.py the default CONN_MAX_AGE database config is set to 600 seconds, so the default of django is 0 what mean all database connections are been closed after request complete, if you don't replace the value of CONN_MAX_AGE after calling django_heroku.config(locals()) the value of this field is default to 600 what mean the DB connections still alive for 600 seconds causing this trouble.
Put this line on the end of your settings.py, its mandatory to be after heroku config:
django_heroku.config(locals())
DATABASES['default']['CONN_MAX_AGE'] = 0
I think I may have solved it.
One of the changes I made was modifying how I closed my connections.
The key is to close old connections before and after various view functions.
from django.db import close_old_connections
#csrf_exempt
#api_view(['GET', ])
def search(request):
close_old_connections()
# do stuff
close_old_connections()
Postgis extension is installed:
:DATABASE=> SELECT postgis_version();
postgis_version
2.2 USE_GEOS=1 USE_PROJ=1 USE_STATS=1
I have the following buildpacks:
https://github.com/cyberdelia/heroku-geo-buildpack.git
https://github.com/heroku/heroku-buildpack-python.git
When I run manage.py migrate I get:
AttributeError: 'DatabaseOperations' object has no attribute 'geo_db_type'
I am using the hobby deb postgres which now supports postgis
https://devcenter.heroku.com/changelog-items/792
Do I need to install a different build pack or add some additional configuration? Everything works locally using postgis.
I finally had some time to go back and look at this. It turns out the issue was Heroku was not importing my settings correctly. I was using cookiecutter-django settings scheme that imports common settings into production and for some reason Heroku was not working as expected.
My common settings contained:
DATABASES['default']['ATOMIC_REQUESTS'] = True
DATABASES['default']['ENGINE'] = "django.contrib.gis.db.backends.postgis"
And my production contained:
DATABASES['default'] = env.db("DATABASE_URL")
Heroku did not import those common settings. When I checked in the django shell in heroku the production settings had
'ENGINE': 'django.db.backends.postgresql_psycopg2', 'ATOMIC_REQUESTS': False
After adding DATABASES['default']['ENGINE'] = "django.contrib.gis.db.backends.postgis" to production settings everything is working.
Does anybody know what could be going wrong with importing settings correctly from common.py? It seems to import the rest of the settings correctly, just not the database ones.
I have a django based herokuapp site. I set everything up a long time ago and am now unable to get things working with a local instance of postgresql. In my settings file, I updated:
DATABASES['default'] = dj_database_url.config()
to work with a local database:
DATABASES['default'] = dj_database_url.config(default='postgres://localhost/appDB')
When running foreman, I can view the site, but the database is not currently populated (although I did create the empty DB). Running:
heroku run python manage.py dumpdata
Returns the contents of the remote (herokuapp) database, while a syncdb command results in "Installed 0 object(s) from 0 fixture(s)". So it looks like I'm still contacting the remote database. I'm pretty sure the postgresql DB is setup correctly locally; how can I force the app to use it?
I'm sure this is simple, but I haven't seen anything useful yet. I did try
export DATABASE_URL=postgres:///appDB
but that hasn't helped.
Cheers
When I run my Django app locally everything works just fine but when I deploy to Heroku using
import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://')}
For my database settings I get the following error
TypeError at /
cannot concatenate 'str' and 'NoneType' objects
you can see the full error message here http://tulsa-staging.heroku.com
Ive removed all none essential settings and isolated the problem to this setting.
So what the heck am I doing wrong here?
Why aren't you listing a host in the config string?
DATABASES = {'default': dj_database_url.config(default='postgres://localhost')}
Im following the getting started with Django on Heroku - and it shows to set up dj_database_url...
DATABASES = {'default': dj_database_url.config(default='postgres://localhost')}
Am I supposed to leave this exactly as above? thats what I did - and my site appears to work just fine...
OR
Am I supposed to swap out something in the above with DATABASE_URL - which gets pulled from the heroku config set ups?
With not using DATABASE_URL anywhere -- it appears to be working on heroku with the above set up and no change.... but when I look at the DB with a pg:info - I get 0 connections... which makes me think there is no connection to the postgresql -- but hows it working then??
I don't quite understand yet how dj_database_url is working on Heroku with this...can anyone shed a bit more light on this?
heroku pg:info
=== HEROKU_POSTGRESQL_GOLD_URL (DATABASE_URL)
Plan: Dev
Status: available
Connections: 0
PG Version: 9.1.6
Created: 2012-10-07 16:11 UTC
Data Size: 6.6 MB
Tables: 12
Rows: 27/10000 (In compliance)
Fork/Follow: Unavailable
It is correct as-is.
When running on Heroku, there is an environment variable set (DATABASE_URL) which contains the database URL (a string like postgres://, but with a long autogenerated username/password/database-name, and the host is on amazonaws usually)
When running locally, DATABASE_URL is not set, so your default = '...' database URL is used instead (allows you to run the code locally for development, and deploy to Heroku, without changing any code).
This is based on the "12factor methodology" (the whole document pretty much describes how Heroku is structured)
The dj_database_url.config just parses the username/password/host/db-name from the URL, and splits it into the dictionary format expected by Django - the code is rather simple, if you are curious