I am trying to deploy my django app to amazon's ec2. I ran into some trouble getting my site-packages and my middleware working on the ec2 server. i am using bitnami djangostack for my AMI but am confused where on the server to put my site-packages (like registration south etc) and where to put my middleware (like pagination).
right now i am running the server and getting the following error:
Traceback (most recent call last):
File "/opt/bitnami/apps/django/lib/python2.6/site-packages/django/core/servers/basehttp.py", line 283, in run
self.result = application(self.environ, self.start_response)
File "/opt/bitnami/apps/django/lib/python2.6/site-packages/django/contrib/staticfiles/handlers.py", line 68, in __call__
return self.application(environ, start_response)
File "/opt/bitnami/apps/django/lib/python2.6/site-packages/django/core/handlers/wsgi.py", line 250, in __call__
self.load_middleware()
File "/opt/bitnami/apps/django/lib/python2.6/site-packages/django/core/handlers/base.py", line 47, in load_middleware
raise exceptions.ImproperlyConfigured('Error importing middleware %s: "%s"' % (mw_module, e))
ImproperlyConfigured: Error importing middleware pagination.middleware: "No module named pagination.middleware"
you can see the error at the following link
site-packages is not somewhere you manually add items. Let Python's installation tools - pip and easy_install do the job for you.
As for middleware, it can go anywhere that is in your PYTHONPATH. The best place to put it is in a file that is in one of your app's directory (same place you see models.py); typically this file is called middleware.py
If the middleware is part of a third-party application; first install that application. Typically this is done with sudo easy_install package-name. Use sudo to install it in the global python site-packages directory.
A better approach is to use a virtual environment.
Related
I'm trying to run a python flask based application on google app engine using cloud datastore. I'm following the bookshelf example -
https://cloud.google.com/python/getting-started/using-cloud-datastore
Everything seems to work in local machine but on app engine, I'm getting
ImportError: No module named pwd, which is being imported by oauth2client library.
Is there anything I'm missing. Any help will be appreciated.
The question referenced as duplicate is different. The application there seems to fail only locally, whereas for me everything works locally but fails to work in GAE.
Here's the stack trace (partial) -
...
1384/v1.394932573930853146/application/home/model.py", line 16, in save_user
ds = get_client()
File "/base/data/home/apps/s~bookshelf-1384/v1.394932573930853146/application/home/model.py", line 7, in get_client
return datastore.Client('bookshelf-1384')#current_app.config['PROJECT_ID'])
File "/base/data/home/apps/s~bookshelf-1384/v1.394932573930853146/lib/gcloud/datastore/client.py", line 173, in __init__
super(Client, self).__init__(credentials, http)
File "/base/data/home/apps/s~bookshelf-1384/v1.394932573930853146/lib/gcloud/client.py", line 122, in __init__
credentials = get_credentials()
File "/base/data/home/apps/s~bookshelf-1384/v1.394932573930853146/lib/gcloud/credentials.py", line 82, in get_credentials
return client.GoogleCredentials.get_application_default()
File "/base/data/home/apps/s~bookshelf-1384/v1.394932573930853146/lib/oauth2client/client.py", line 1288, in get_application_default
return GoogleCredentials._get_implicit_credentials()
File "/base/data/home/apps/s~bookshelf-1384/v1.394932573930853146/lib/oauth2client/client.py", line 1273, in _get_implicit_credentials
credentials = checker()
File "/base/data/home/apps/s~bookshelf-1384/v1.394932573930853146/lib/oauth2client/client.py", line 1226, in _implicit_credentials_from_files
credentials_filename = _get_well_known_file()
File "/base/data/home/apps/s~bookshelf-1384/v1.394932573930853146/lib/oauth2client/client.py", line 1392, in _get_well_known_file
default_config_dir = os.path.join(os.path.expanduser('~'),
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/posixpath.py", line 268, in expanduser
import pwd
ImportError: No module named pwd
The error is caused by http://github.com/google/oauth2client/issues/578.
It will be fixed in a couple of weeks with a new AppEngine SDK. In the meantime you can downgrade the version of the oauth2client library in your app.yaml file.
I had this same issue which is a known issue with the SDK. I tried the other solutions suggested on SO (see Google App Engine 'No module named pwd') but it didn't work. I found a work around by installing an older version of oauth2client-2.0.0 overwriting the oauth2client-4.0.0 that came with the pubsub pip install. Try running
pip install -t ./lib/ --upgrade --force-reinstall oauth2client==2.0.0
in the project directory and rerunning.
I have already got boto installed to my python virtual environment, as shown in the screenshot of my pycharm project.
However I got an ImportError of boto.cloudfront when I run my script from pycharm
ERROR 2015-04-20 00:17:39,590 wsgi.py:263]
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/Users/antkong/dev/test/testmain.py", line 7, in <module>
from boto.cloudfront import CloudFrontConnection
ImportError: No module named boto.cloudfront
In the python console, I can import the library just fine:
Any suggestion what can go wrong here?
My project is a Google App Engine project. So even though boto is installed locally, it is not visible to my code within my Google App Engine project
My solution is to copy the boto and all dependency into a lib subdirectory and then add it to the sys.path
After adding python social auth to my installed apps, i.e.
INSTALLED_APPS = (
...
'social.apps.django_app.default',
...
)
and then trying a
python manage.py makemigrations
I get an unsurprising permissions error
Migrations for 'default':
0002_auto_20150217_2053.py:
- Alter field user on usersocialauth
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/lib/python2.7/site-packages/django/core/management /__init__.py", line 385, in execute_from_command_line
utility.execute()
File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
output = self.handle(*args, **options)
File "/usr/lib/python2.7/site-packages/django/core/management/commands/makemigrations.py", line 124, in handle
self.write_migration_files(changes)
File "/usr/lib/python2.7/site-packages/django/core/management/commands /makemigrations.py", line 153, in write_migration_files
with open(writer.path, "wb") as fh:
IOError: [Errno 13] Permission denied: u'/usr/lib/python2.7/site-packages/social/apps/django_app/default/migrations/0002_auto_20150217_2053.py'
It makes sense that I can not write to system wide package installation directories.
There are some obvious ways around this, like changing the permissions on the site-packages/social directories. However, is this the only way of doing this, or am I missing something?
With the accepted solution above, you're effectively putting project files in your python environment. And every time you deploy to a new server, you'd have to run makemigrations to create those file(s).
How about telling makemigrations to put the social migrations inside your own project?
MIGRATION_MODULES = {
# social.apps.django_app.default
'default': 'myproject.mysocial.migrations',
}
That way, when you deploy to your server, your project is self-contained and will work without hacking your Python environment.
As stated in your question, changing the permissions on the directories would be a solution. Yet, another way to do this is to create an isolated Python environment for your Django project using virtualenv. Or, more conveniently, the extension virtualenvwrapper.
Install the later like this:
$ pip install virtualenvwrapper
The following creates an activates a virtualenv for your project:
$ mkvirtualenv django_project
$ workon django_project
After that, you are free to install Django and Python Social Auth
$ pip install django
$ pip install python-social-auth
You will notice that all this will be installed in $HOME/.virtualenvs/django_project
This is a common, recommended practice among Python and Django users. It will solve the permission issues, as well as other dependency issues that you may run into if you use your system's Python installation for all your projects.
Note that Python Social Auth does in fact require the creation of a migration before its use in a Django project, the migration adds a related_name to the foreign key 'user' in the 'UserSocialAuth' model
None of these answers addresses your issue, as far as I can tell. Simply installing a new application shouldn't require creation of new migration files; they should, one would expect, be shipped with the application package.
This looks like a bug or an oversight by the package author.
I've followed installations of Django using pip in a virtualenv since I'm learning Django. When the Django Book told me to type this in the Python shell,
from django import template
t = template.Template("My name is {{ name }}.")
I got this exception. I don't know how to solve this.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "django/template/base.py", line 123, in __init__
if settings.TEMPLATE_DEBUG and origin is None:
File "django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "django/conf/__init__.py", line 46, in _setup
% (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting TEMPLATE_DEBUG,
but settings are not configured. You must either define the environment variable
DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Can somebody walk me through? I'm using Mac OS X 10.8
It seems that you are running python shell using the command python. So the DJANGO_SETTINGS_MODULE variable is not available. Run your application specific python shell using
python manage.py shell
Read more about manage.py.
manage.py sets the DJANGO_SETTINGS_MODULE environment variable so that it points to your project’s settings.py file.
you need to run the django shell so it loads the settings
python manage.py shell
Django needs your application specific settings.
Since it is already inside your manage.py just run :
python manage.py shell
This should resolve the issue fast-enough
This is a follow-up question for Google App Engine and Django support:
The tutorial works great for an empty project, however when I try to deploy an existing Django app to Google App Engine, it starts throwing errors:
Traceback (most recent call last):
File "/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 223, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/core/handlers/wsgi.py", line 219, in __call__
self.load_middleware()
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/core/handlers/base.py", line 39, in load_middleware
for middleware_path in settings.MIDDLEWARE_CLASSES:
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/utils/functional.py", line 184, in inner
self._setup()
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/conf/__init__.py", line 40, in _setup
raise ImportError("Settings cannot be imported, because environment variable %s is undefined." % ENVIRONMENT_VARIABLE)
ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
And a second, which might be related:
Traceback (most recent call last):
File "/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 223, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/core/handlers/wsgi.py", line 219, in __call__
self.load_middleware()
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/core/handlers/base.py", line 39, in load_middleware
for middleware_path in settings.MIDDLEWARE_CLASSES:
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/utils/functional.py", line 184, in inner
self._setup()
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/conf/__init__.py", line 42, in _setup
self._wrapped = Settings(settings_module)
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/conf/__init__.py", line 95, in __init__
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'settings' (Is it on sys.path?): No module named settings
I have modified my settings.py as per the tutorial. The app deploys without problems and the syncdb works as well; the database and all required tables are in place.
I found this reference to the error message as shown in the log but its suggested fix did not help.
Any ideas what might be causing this?
EDIT:
To shorten this already lengthy question I removed the previously posted wsgi.py file. I had nothing to do with this issue.
EDIT 2:
I think I made a small improvement and GAE is now complaining:
ImportError: No module named properties
I presume there might be an entry missing in my app.yaml file but I have no clue as to which file that may be. I found some references regarding missing modules but none that reflect this error message...
After much trial and error I managed to solve this, albeit partially:
I did not manage to get my existing PyCharm project deployed (yet). As I was keen to have a version running on GAE I went for a different approach and created a new empty Django project (project 2) on my local machine.
I first deployed the blank project using this tutorial which ran fine. I then copied my models.py file from my existing PyCharm project to project 2 and added an app called properties which uses the models.py file. Initially this went wrong as GAE kept complaining it could not find my properties app. After fiddling about with it, it suddenly appeared in my admin. I'm not sure what caused the initial problem but it's working now.
Things crucial to success for me:
I added the following to the very top of my settings.py:
import os
ROOT_PATH = os.path.dirname(__file__)
... and then replaced the default STATIC_ROOT with:
STATIC_ROOT = ROOT_PATH + os.sep + 'static'
Then I ran:
python manage.py collectstatic
from the dir of project 2. This collects the required static files which Django admin uses (image files, css files etc)
So, I currently have a stripped down version of my original project running on GAE. Things that are missing:
The original project used grappelli for a more styled admin interface. I'm not sure whether this will run on GAE. EDIT: Deploying Grappelli on GAE is reasonbly easy. Just copy the Grappelli package to your project dir and run manage.py collectstatic. Also add grappelli urls as per documention. Redeploy. Done.
The original project uses a module called templated_email to send out templated emails from my properties app. Same as grappelli: I'm not sure whether this is supported by GAE. Hopefully it will run on GAE as it is crucial for my specific app. EDIT: I bumped into this little gem which makes templated_email obsolete and lets me plug into GAE mail system.
To be honest, this approach partially defies the benefits of PyCharm which should fully support Django and GAE out of the box. It will probably require some additional research and tweaking to get that up and running.
Any comments or insights re the above are obviously appreciated!