I am developing a Flask app that runs using uwsgi. I'm running into the following error ImportError: No module named google.oauth2.credentials.
When I run the Flask app using Flask's builtin dev server, the application runs without issues.
The problem seems to stem from how the google-auth module is installed. There's no __init__.py file in the google namespace folder in site-packages. The full path being /usr/local/lib/python2.7/site-packages/google.
I'm aware that python3.3 has implicit namespace packaging via PEP-420.
What I'm confused about is how the implicit namespace packaging works in python2.7. Does the installation process via pip do something special?
I'm also confused about what is different when running python2.7 via uwsgi.
Relevant versions
python 2.7.14
pip 9.0.3
uwsg 2.0.13
google-auth 1.4.1
Installed via pip (not using virtualenv)
https://github.com/GoogleCloudPlatform/google-auth-library-python
I've also included the following barebones test to reproduce the error.
test.ini
[uwsgi]
socket = 0.0.0.0:5001
plugins-dir = /usr/lib/uwsgi/
plugins = python
protocol = uwsgi
pythonpath = /usr/local/lib/python2.7/site-packages
callable = app
max-requests = 1000
master = True
lazy-apps = True
processes = 1
test.py
import google.oauth2.credentials
working command
python test.py
failing command
uwsgi --ini test.ini --wsgi-file test.py
failing result
When I run the above command, the program will fail due to the following ImportError.
Traceback (most recent call last):
File "test.py", line 1, in <module>
import google.oauth2.credentials
ImportError: No module named google.oauth2.credentials
Current workaround
My current workaround is to manually add an __init__.py file with the following (which is what is basically what is found in the google-auth repository, but for some reason is removed when installing via pip):
try:
import pkg_resources
pkg_resources.declare_namespace(__name__)
except ImportError:
import pkgutil
__path__ = pkgutil.extend_path(__path__, __name__)
Workaround 2
Another way I got the import working was by doing this:
import site
site.addsitedir('/usr/local/lib/python2.7/site-packages')
import google.oauth2.credentials
This seems to imply that uwsgi is not fully initialization python at least on initial start up.
I got the hint to try site.addistedir from this answer: https://stackoverflow.com/a/15209116/177646
Turns out the problem was a uWSGI configuration error on my part. Since I'm not using virtualenv, I needed to set PYTHONHOME to the correct value, which is /usr/local in my case. In the uwsgi configuration, this can be specified via the home parameter. Once this is set, then python seems to work normally.
Once the PYTHONHOME is correctly set, this allows the google_auth-1.4.1-py3.6-nspkg.pth file to be found.
This also explains why I had to always import /usr/local/lib/python2.7/site-packages in my PYTHONPATH. Once the PYTHONHOME is set, then I no longer needed to add site-packages to my PYTHONPATH.
Example:
[uwsgi]
socket = 0.0.0.0:5001
plugins-dir = /usr/lib/uwsgi/
plugins = python
protocol = uwsgi
home = /usr/local
callable = app
max-requests = 1000
master = True
lazy-apps = True
processes = 1
Related
I must be missing something but I look around and couldn't find reference to this issue.
I have the very basic code, as seen in flask-mongoengine documentation.
test.py:
from flask import Flask
from flask_mongoengine import MongoEngine
When I run
python test.py
...
from flask_mongoengine import MongoEngine
ImportError: cannot import name 'MongoEngine'
Module in virtual environment contain (requirements.txt):
click==6.7
Flask==1.0.2
flask-mongoengine==0.9.5
Flask-WTF==0.14.2
itsdangerous==0.24
Jinja2==2.10
MarkupSafe==1.0
mongoengine==0.15.3
pymongo==3.7.1
six==1.11.0
Werkzeug==0.14.1
WTForms==2.2.1
My interpreter is Python 3.6.5
Any help would be appreciated. Thanks.
Since your using a virtual environment did you try opening your editor from your virtual environment?
For example opening the vscode editor from command-line is "code". Go to your virtual environment via the terminal and activate then type "code" at your prompt.
terminal:~path/to/virtual-enviroment$ source bin/activate
(virtual-enviroment)terminal:~path/to/virtual-enviroment$ code
If that doesn't work I, myself, haven't used flask-mongoengine. I was nervous of any issues that would come from the abstraction of it and instead just used Mongoengine with Flask.
I'm assuming you're only using this library for connection management so if you can't solve your issue with flask-mongoengine but are still interested in using mongoengine this was my approach. ~
I would put this in a config file somewhere and import it where appropriate-
from flask import Flask
MONGODB_DB = 'DB_NAME'
MONGODB_HOST = '127.0.0.1' # or whatever your db address
MONGODB_PORT = 27017 # or whatever your port
app = Flask(__name__) # you can import app from config and it will keep its configurations
then I would connect and disconnect from the database within each HTTP request function like this-
from config import MONGO_DB, MONGODB_HOST, MONGODB_PORT
# to connect
db = connect(MONGODB_DB, host=MONGODB_HOST, port=MONGODB_PORT)
# to close connection before any returns
db.close()
Hope this helps.
I had this issue and managed to fix it by deactivating, reinstalling flask-mongoengine and reactivating the venv (all in the Terminal):
deactivate
pip install flask-mongoengine
# Not required but good to check it was properly installed
pip freeze
venv\Scripts\activate
flask run
I tried to upload a simple authentification project to pythonhosted which works and builds on my local machine without errors.
Then i tried to host this one in pythonanywhere.
Nevertheless the main page seems to run but but when i try to use the allauth login feature it crashes:
python3.4 manage.py makemigrations
or
make rebuild
throws the following error:
Here is the code of that project. It is build from this template with little modifications like python3 and some script modification.
Here is my wsgi-file (updated and working)
# +++++++++++ DJANGO +++++++++++
# To use your own django app use code like this:
import os
import sys
#
## assuming your django settings file is at '/home/MaRcMaN/mysite/mysite/settings.py'
path = '/home/MaRcMaN/sqlnoodle_django'
if path not in sys.path:
sys.path.append(path)
#
os.environ['DJANGO_SETTINGS_MODULE'] = 'allauthdemo.settings'
#
## then, for django >=1.5:
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
## or, for older django <=1.4
#import django.core.handlers.wsgi
#application = django.core.handlers.wsgi.WSGIHandler()
The stacktrace that you provided (which is almost impossible to read) suggests some possibilities:
that you haven't activated your virtualenv in the Bash shell where you're running your manage.py command
that you haven't installed the correct version of Django into the virtualenv
that you've installed Django into your home directory (not into the virtualenv) and it's a different version to the one that you're using on your local machine
I'm switching a very large multi-package, multi-app Django (1.4.5) project from using Django's native test runner to py.test (2.3.5). The tests for the lowest level package, web-framework, were discovered and run with no errors after creating a setup.cfg (to manage the py.test options) and a conftest.py (to ignore setup.py). When running py.test (with a setup.cfg and conftest.py) on any higher level package, which all require web-framework, I receive the following error:
ERROR: Could not import settings 'high_level_package.test_settings' (Is it on sys.path?): No module named web_framework.settings
I'm using a virtual environment, and web-framework is in the venv at the following location: ENV/lib/python2.7/site-packages/
I've tried with the venv built in the package's root directory and with it built outside the project path, to no avail. Also, I can import web_framework.settings from the Python interactive command line in the higher level package's root directory.
My current conftest.py is just the following line: collect_ignore = ["setup.py"]
I've tried adding the following lines above it:
import os
import sys
sys.path.append(os.path.dirname(__file__))
I also tried hardcoding in the path to the web-framework package in the above sys.path.append.
Neither worked.
In case it's relevant, my setup.cfg is:
[pytest]
python_files = *test*.py
norecursedirs = node_modules ENV
addopts = --junitxml=python_result.xml, --doctest-modules
Edit:
Forgot to mention the traceback relationship. higher_level_package.test_settings imports higher_level_package.settings, which itself imports web_framework.settings.
in order to have it work you either need to have a develop-install of the worktree, or add . to the PYTHONPATH env var
Change to the folder where your main python website module is located
django-admin runserver --settings=mysite.settings
where mysite was created by
django-admin startproject mysite
I have created a Django 1.3 application on Openshift.
I wanted to upgrade to Django 1.5. So I updated setup.py to install Django 1.5
#!/usr/bin/env python
from setuptools import setup
setup(
name='<Application name>',
version='1.0',
description='',
author='',
author_email='',
url='http://www.python.org/sigs/distutils-sig/',
install_requires=['Django>=1.5'],
)
The server returns http 500.
If setup.py has install_requires=['Django<=1.4'] it works fine.
How can I install Django 1.5 on Openshift?
Update: I can see a github commit where in the install_requires for Django is changed from >=1.3 to <=1.4 for the handling this same issue. But I still cannot figure out what caused that server 500 and how can we install Django 1.5 on openshift
It might come from your code, did you check the backwards incompatibilities mentioned in the release notes (mainly ALLOWED_HOSTS required in your settings.py)
It could also come from the {% url %} tag syntax change, see here.
When i installed Django app on OpenShift, Django version was 1.5.1. I think OpenShift install last version Django, because the condition Django >= 1.4, that is no lower this version.
That is screenshot, when i installed app
I had the same issue : from your screenshot you're using python2.6 ?
Try to use python2.7 with this configurations put on the application file:
#!/usr/bin/env python
import os
import sys
sys.path.append(os.path.join(os.environ['OPENSHIFT_REPO_DIR']))
os.environ['DJANGO_SETTINGS_MODULE'] = 'mywebsite.settings'
virtenv = os.environ['OPENSHIFT_HOMEDIR'] + 'python/virtenv/'
os.environ['PYTHON_EGG_CACHE'] = os.path.join(virtenv, 'lib/python2.7/site-packages')
virtualenv = os.path.join(virtenv, 'bin/activate_this.py')
try:
execfile(virtualenv, dict(__file__=virtualenv))
except IOError:
pass
#
# IMPORTANT: Put any additional includes below this line. If placed above this
# line, it's possible required libraries won't be in your searchable path
#
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
And as refered by #Charles L try to set the settings using allowed host
I am trying to create a dev environment for the Django 1.4 project using the following guide:
http://www.stereoplex.com/blog/a-django-development-environment-with-zc-buildout
virtualenv part of the guide runs ok with the following output:
virtualenv project
New python executable in project\Scripts\python.exe
Installing setuptools................done.
Installing pip...................done.
After that I am able to activate dev environment. Now I create directory named Source, download the bootstrap.py to it and create a buildout.cfg with the following content:
[buildout]
parts =
And run bootstrap.py for the following result:
Creating directory 'C:\\Dropbox\\XYZ\\project\\Source\\bin'.
Creating directory 'C:\\Dropbox\\XYZ\\project\\Source\\parts'.
Creating directory 'C:\\Dropbox\\XYZ\\project\\Source\\eggs'.
Creating directory 'C:\\Dropbox\\XYZ\\project\\Source\\develop-eggs'.
Generated script 'C:\\Dropbox\\XYZ\\project\\Source\\bin\\buildout'.
Here comes the problem part - Installing Django I configure the buildout.cfg to the following and run bin\buildout created by bootstrap:
[buildout]
parts = django
[django]
recipe = djangorecipe
version = 1.4
After running bin\buildout i get the following error:
(project) C:\Dropbox\XYZ\project\Source>bin\buildout.exe
Traceback (most recent call last):
File "C:\Dropbox\XYZ\project\Source\bin\buildout-script.py", line 15, in <module> import site # imports custom buildout-generated site.py
File "C:\Dropbox\XYZ\project\Source\parts\buildout\site.py", line 601, in <module> main()
File "C:\Dropbox\XYZ\project\Source\parts\buildout\site.py", line 584, in main known_paths = addsitepackages(known_paths)
File "C:\Dropbox\XYZ\project\Source\parts\buildout\site.py", line 328, in addsitepackages import pkg_resources
ImportError: No module named pkg_resources
Although if I run python directly in project environment I can import pkg_resources with no error:
(project) C:\Dropbox\XYZ\project\Source>python
Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
>>> pkg_resources
<module 'pkg_resources' from 'C:\Dropbox\XYZ\project\lib\site-packages\setuptools-0.6c11-py2.7.egg\pkg_resources.py'>
I am completely struck here. Any suggestions?
I don't have the definitive answer, but here are some brainstorm thoughts:
Why the virtualenv? Buildout itself provides isolation, so no virtualenv is needed. Could you re-try with just a bootstrap.py and your buildout.cfg? So just run bootstrap.py with your system python?
Do you have buildout installed globally, perhaps? They can interfere.
The latest 1.5.2 buildout has some problems with site.py files in some situations, which is a possible reason for it failing inside a virtualenv. Could you try the special 1.4.4 bootstrap mentioned in http://pypi.python.org/pypi/zc.buildout/1.5.2#system-python-and-zc-buildout-1-5 ?
bin/buildout -vvv gives you much more debugging info.
An additional comment: the version setting in djangorecipe is deprecated in the latest versions of djangorecipe. You can remove it. If you want to pin Django you have to pin it in your buildout's [version] list.
The guide you are following is a little bit obsolete. It will fail when processing the [django] part, specifically in the version variable. You must specify the versions in the new way, which is showed in the djangorecipe page. This is:
[buildout]
parts = django
versions = versions
[versions]
django = 1.4