I'm having a very strange issue.
I have my django project running in Heroku using S3 to store my static assets.
I wanted to use the Heroku enviroment variables by setting them as follows:
heroku config:add AWS_S3_TOKEN=my_s3_token
heroku config:add AWS_S3_SECRET=my_s3_secret
And using them with python's os module:
import os
token = os.getenv('AWS_S3_TOKEN')
secret = os.getenv('AWS_S3_SECRET')
But heroku keeps throwing me the following error:
NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. ['HmacAuthV1Handler'] Check your credentials
So, I ended up writing those parameters in my settings.py file and it works fine
Why is this happening?
If I run
heroku config
I can see all my seted variables and if i do
heroku run python manage.py shell
and then
import os
print os.getenv('AWS_S3_TOKEN')
For example, it prints the variable's value.
any clue on this???
Thank you in advance
I'm not familiar with python or Django, but I'm curious.
If you try your authentication with blank strings, do you get the same error?
If so, then I suspect it's about when you're doing this authentication dance. When are you doing it? At compile time, or at runtime? (If at compile time, look at this)
The app's environment variables aren't usually available while Heroku is compiling the slug and setting up the application, they're only available once the app is running. For a Django app, Heroku will run collectstatic as part of slug compilation, which is probably why you're seeing this error.
You can make the environment variables available during compilation by enabling a Heroku Labs feature:
heroku labs:enable user-env-compile
There's more information in this Heroku dev centre article: https://devcenter.heroku.com/articles/labs-user-env-compile
Related
I think this is a simple fix, but I've deployed quite a few Django apps to Heroku and I still can't figure out what's going on.
Accessing https://dundjeon-finder.herokuapp.com/ gives me a 500 error when using the browser/curl, but if I shell into the app using heroku run ./manage.py shell I can render the views no problem. My logs aren't telling me anything (just that the response is 500) despite DEBUG being set to True, and Sentry isn't receiving an error (it has previously when the database env variable was set badly), so I'm assuming it's something to do with the way the request works.
The repo is public, any help would be much appreciated! The settings file is here.
Well it was because of using asgi instead of wsgi. I'm not sure why that caused the errors, but will do some more searching.
Just deployed a Django app on Heroku. Everything works except for one page of my site which creates a Server 500 error (even though it works fine on my local development server).
The page raising the error doesn't do anything unusual. It makes some database calls, renders some forms, implements JQuery, etc. Any clue what this could be or how I can debug it?
Also, I thought this might be a data issue since my data in Dev doesn't match my data in production, but I checked and this doesn't seem to be the cause.
enable DEBUG=TRUE in your django settings.py file or type in console heroku logs --app your_app to get heroku server logs.
This was because I didn't include a runtime.txt telling Heroku to use Python 3 instead of 2 which subsequently raised an error in one of my views where I called super() with no args.
I know there are a lot of questions floating around there relating to similar issues, but I think I have a specific flavor which hasn't been addressed yet. I'm attempting to create my local postgresql database so that I can do local development in addition to pushing to Heroku.
I have found basic answers on how to do this, for example (which I think is a wee bit outdated):
'#DATABASES = {'default': dj_database_url.config(default='postgres://fooname:barpass#localhost/dbname')}'
This solves the "ENGINE" is not configured error. However, when I run 'python manage.py syncdb' I get the following error:
'OperationalError: FATAL: password authentication failed for user "foo"
FATAL: password authentication failed for user "foo"'
This happens for all conceivable combinations of username/pass. So my ubuntu username/pass, my heroku username/pass, etc. Also this happens if I just try to take out the Heroku component and build it locally as if I was using postgresql while following the tutorial. Since I don't have a database yet, what the heck do those username/pass values refer to? Is the problem exactly that, that I need to create a database first? If so how?
As a side note I know I could get the db from heroku using the process outlined here: Should I have my Postgres directory right next to my project folder? If so, how?
But assuming I were to do so, where would the new db live, how would django know how to access it, and would I have the same user/pass problems?
Thanks a bunch.
Assuming you have postgres installed, connect via pgadmin or psql and create a new user. Then create a new database and with your new user as the owner. Make sure you can connect via psql with the new user into to the database. you will then need to set up an env variable in your postactivate file in your virtualenv's bin folder and save it. Here is what I have for the database:
export DATABASE_URL='postgres://{{username}}:{{password}}#localhost:5432/{{database}}'
Just a note: adding this value to your postactivate doesn't do anything. The file is not run upon saving. You will either need to run this at the $ prompt, or simply deactivate and active your virtualenv.
Your settings.py should read from this env var:
DATABASES = {'default': dj_database_url.config()}
You will then configure Heroku with their CLI tool to use your production database when deployed. Something like:
heroku config:set DATABASE_URL={{production value here}}
(if you don't have Heroku's CLI tool installed, you need to do it)
If you need to figure how exactly what that value you need for your production database, you can get it by logging into heroku's postgresql subdomain (at the time this is being written, it's https://postgres.heroku.com/) and selecting the db from the list and looking at the "Connection Settings : URL" value.
This way your same settings.py value will work for both local and production and you keep your usernames/passwords out of version control. They are just env config values.
I'm using django-storages for static files with S3 (and S3BotoStorage). When I do collectstatic from my local machine, the behaviour is as expected, where only modified files are pushed to S3. This process needs python-dateutils 1.5 to check for modified time.
However, doing the same on Heroku results in every file being pushed regardless, although the setup is the same. I then looked into the modified time of the files on Heroku itself, and it seems like, the os.stat(static_filename).st_mtime is the same as the time of the last push.
Is this expected behaviour? Does heroku copy around files even when there is no change from git?
Try setting DISABLE_COLLECTSTATIC=1 as an environment setting for your app - that should disable it from running on every push.
See this article for details - https://devcenter.heroku.com/articles/django-assets :
> Sometimes, you may not want Heroku to run collectstatic on your behalf.
> You can disable collectstatic by enabling user-env-compile as well:
$ heroku labs:enable user-env-compile
$ heroku config:set DISABLE_COLLECTSTATIC=1
I've found that simply setting the config will do - no need to also enable user-env-compile - it may be that that this has passed from labs into production?
NB the deployment is managed by the Heroku python buildpack, which you can see here - https://github.com/heroku/heroku-buildpack-python/
EDIT 1
I've just done a bunch of tests on this, and can confirm that DISABLE_COLLECTSTATIC does indeed disable collectstatic, irrespective of the user-env-compile setting - I think that's now in the main trunk (but that's speculation). Doesn't seem to care what the setting is - if DISABLE_COLLECTSTATIC exists as a config var it is used.
I strongly recommend using the collectfast package for any django static deployment to s3, whether local or from your heroku server. It ignores modified dates and utilizes md5 hashes, which the s3 api will provides very quickly, and (optional) caching to make your static deployments zoom. It took my static deployments from ~10-15 minutes to < 2 minutes and only deploys the files that have actually changed.
I've just had that exact same issue and contacted Heroku's support to find out what is going on. My question to them was
I've run into a funky issue doing some deployments. It appears that on each push the date modified on all files is updated to the time a new deploy/git push happens. Is this intended behaviour?
When considering that Django's collectstatic command only checks the modified date on files when evaluating if the file should be copied across to the final storage backend for static assets, it means that on each new push, all files are first removed from the remote storage (in this case S3) and then re-uploaded. This is both a very slow and wasteful process in terms of bandwidth consumed and requests made.
The answer I received today from "Caio", one of Heroku's support staff, was
Hi, that's how it currently works, yes. I'm routing your feedback to our runtime team to see if we can package files with their original dates.
As confirmed by Alen, Heroku changes the modified date of the files when it deploys. However, Amazon S3 also has an attribute called etag that is an md5 hash of the file content. It's possible to use this to check if the files have changed instead of the modified date, as implemented in this Django snippet.
I took that code, packaged it and fixed some errors I found and put it on Github as django-s3-collectstatic. It includes a new management command fasts3collectstatic that only uploads new files. Check the Github page for installation instructions.
Why not run collectstatic from local machine?
python manage.py collectstatic --noinput --settings=settings.[prod]
I agree this is annoying- there's a couple things you can do. I override the collectstatic command and wire it up in my production settings. Below is the command I use:
```
from django.core.management.base import BaseCommand
class Command(BaseCommand):
args = '< none >'
help = "disables collectstatic cmd in contrib"
def handle(self, *args, **kwargs):
print 'collectstatic disabled'
```
I keep this in mysite/disablecollectstatic/management/commands
Then in production settings:
INSTALLED_APPS += ('mysite.disablecollectstatic',)
Alternatively you could use the fact that Heroku does a dry run first before actually invoking the command. If it fails, it won't run it, which means you could contrive an error (by maybe deleting the static root in your settings, for example) but this approach makes me nervous:
https://devcenter.heroku.com/articles/django-assets#detection
I am trying django on heroku, following the official tutorial and stuck at the creating celery and kombu tables locally step using python manage.py syncdb, getting following errors:
File "/my/virtual/path/django/db/backends/postgresql_psycopg2/base.py",
line 162, in _cursor
raise ImproperlyConfigured("You need to specify NAME in your Django settings file.")
django.core.exceptions.ImproperlyConfigured: You need to specify NAME in your Django settings file.
The problem is it cannot find the db I set for heroku by the config vars of heroku. When I run heroku config, it displays the DATABASE_URL of my app correctly, but when I try:
if 'DATABASE_URL' in os.environ
in python interpreter, it returns false. I also check, none of my config vars is in my os.environ. Shouldn't vars be added to it automatically? My previous steps are correct.
I searched and got a lot of solutions to detect the heroku db but they all are based on the assumption that I have the DATABASE_URL in my os.environ.
Could anyone point out where I went wrong? Any suggestion is appreciated. Thanks!
I believe the tutorial is missing some steps on how to run the app locally. If you want to sync the database locally, you should override the "default" DATABASE with the one you have on your machine. There is one similar answer in here with code sample.
visit this page
Heroku Database Settings Injection - How do I setup my dev django database?
ldiqual's answer is right.