Pandas is producing 'module' object has no attribute 'core' when being imported under django and mod_wsgi inside a virtual environment. It works fine running under the django development server inside the virtual environment.
Other modules e.g.: numpy have no problems so I assume this means the virtual environment is set up correctly with mod_wsgi. Any advice would be appreciated.
staging.wsgi
import os
import sys
import site
PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.dirname(__file__)))
site_packages = os.path.join(PROJECT_ROOT, 'env/openportfolio/lib/python2.7/site-packages')
site.addsitedir(os.path.abspath(site_packages))
sys.path.insert(0, PROJECT_ROOT)
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
os.environ['DJANGO_SETTINGS_MODULE'] = 'openportfolio.settings_staging'
import pandas #triggers error
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
Error
Traceback (most recent call last):
File "/usr/local/web/django/www/staging/openportfolio/apache/staging.wsgi", line 22, in <module>
import pandas
File "/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages/pandas/__init__.py", line 12, in <module>
from pandas.core.api import *
File "/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages/pandas/core/api.py", line 6, in <module>
import pandas.core.datetools as datetools
AttributeError: 'module' object has no attribute 'core'
Python Path
['/usr/local/web/django/www/staging/openportfolio',
'/usr/local/web/django/www/staging',
'/Library/Python/2.7/site-packages/pip-1.0.2-py2.7.egg',
'/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg',
'/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages/pip-1.0.2-py2.7.egg',
'/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC',
'/Library/Python/2.7/site-packages',
'/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages']
So it turns out this was a Python path order issue. By running sys.path.reverse() in my wsgi config file, the code now runs.
Due to the order of Python path, the built in OS X numpy library must have been imported first over the virtual environment one causing the issue.
'RuntimeError: module compiled against API version 6 but this version of numpy is 4' was the error line I missed in my original post which could have helped debug the answer.
Related
I've upgraded my project to latest flask (2.1.2 and now I'm getting following error:
Traceback (most recent call last):
File "/home/ff/conveyors/run.py", line 1, in <module>
from conveyors import create_app
File "/home/ff/conveyors/conveyors/__init__.py", line 4, in <module>
from flask_bcrypt import Bcrypt
File "/home/ff/.cache/pypoetry/virtualenvs/conveyors-SV5d9Vx2-py3.9/lib/python3.9/site-packages/flask_bcrypt.py", line 21, in <module>
from werkzeug.security import safe_str_cmp
ImportError: cannot import name 'safe_str_cmp' from 'werkzeug.security'
In my __init__.py I have
...
from flask_bcrypt import bcrypt
from flask_login import LoginManager
...
and then below in create_app:
def create_app(config_class=Config):
app = Flask(__name__)
app.config.from_object(Config)
db.init_app(app)
bcrypt.init_app(app)
login_manager.init_app(app)
...
I've read Werkzeug release note about safe_str_cmp being removed from Werkzeug 2.1 and needs to be replaced with equivalent hmac functions, but I'm not sure what I need to do in my code to fix this.
Any help greatly appreciated.
Thanks.
It turned out to be a poetry update problem. Whatever I tried it wouldn't upgrade flask_bcrypt to the latest version 1.0.1 and stayed with 0.7.1
I had to start kind of from scratch and do
poetry add flask
...
for every package needed for this app
Some day I lost access to Trac which is Python2.7 software. (I suppose, that deprecation of Python2.x is clue). There is Python3 unstable version, but I can't use it, because there's no plugins yes, which are key for my Trac installation.
So created virtual environment:
virtualenv -p /usr/bin/python2.7 env
And finally installation is succesful!
pip install trac
But... Stil can't run it by wsgi apache2:
Traceback (most recent call last):
File "/var/lib/trac/apache/trac.wsgi", line 13, in <module>
import trac.web.main
File "/var/lib/trac/env/lib/python2.7/site-packages/trac/web/__init__.py", line 19, in <module>
from trac.web.api import *
File "/var/lib/trac/env/lib/python2.7/site-packages/trac/web/api.py", line 18, in <module>
from BaseHTTPServer import BaseHTTPRequestHandler
I tried to install pip install http but still the same problem
my wsgi file:
import sys
sys.stdout = sys.stderr
import os
os.environ['PKG_RESOURCES_CACHE_ZIP_MANIFESTS'] = '1'
os.environ['TRAC_ENV_PARENT_DIR'] = '/var/lib/trac'
os.environ['PYTHON_EGG_CACHE'] = '/var/lib/trac/eggs'
#import trac.db.postgres_backend
#trac.db.postgres_backend.PostgreSQLConnection.poolable = False
import trac.web.main
application = trac.web.main.dispatch_request
How to change wsgi or how to install missing module?
I am getting this error message when I try to run the shell from Eclipse Neon while I can successfully run the Django shell from command window. I am using Python 3.4 and Django 1.10. Any idea where the problem is?
WSGI file:
'''
WSGI config for MyProject project.
It exposes the WSGI callable as a module-level variable named ``application``.
For more information on this file, see
https://docs.djangoproject.com/en/1.10/howto/deployment/wsgi/
'''
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "MyProject.settings")
application = get_wsgi_application()
I beleive This is a known bug in Eclipse Pydev.
https://www.brainwy.com/tracker/PyDev
Fixed for 5.6
Git: 2c8cd03 2017-03-12 Fabio Zadrozny #PyDev-752: Django version not
detected if > 1.10
And a fix has been posted for it.
You could try start a python shell as normal, then:
import sys; print('%s %s' % (sys.executable or sys.platform, sys.version))
import os; os.environ['DJANGO_SETTINGS_MODULE'] = '<Project Name>.settings'; import django
sys.path.append(os.path.expanduser('<path to your project>'))
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Replace 'Project Name' with your project name and 'path to your project' with your path
alternatively, use a pre 1.10 version of Django i.e 1.7
I am developing a Django project with PyDev in Eclipse. For a while, PyDev's Django Shell worked great. Now, it doesn't:
>>> import sys; print('%s %s' % (sys.executable or sys.platform, sys.version))
C:\Python26\python.exe 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]
>>>
>>> from django.core import management;import mysite.settings as settings;management.setup_environ(settings)
Traceback (most recent call last):
File "<console>", line 1, in <module>
ImportError: No module named mysite.settings
>>>
The dev server runs just fine. What could I be doing wrong?
The models module is also conspicuously absent:
>>> import mysite.myapp.models
Traceback (most recent call last):
File "<console>", line 1, in <module>
ImportError: No module named mysite.myapp.models
On the normal command line, outside of PyDev, the shell works fine.
Why could this be happening?
Seems like a simple path issue. What's the output of this:
import sys; print sys.path
I don't know anything about PyDev, but there's probably a setting somewhere to add paths to the PYTHONPATH setting. If not, you can do it directly in the shell:
sys.path.insert(0, '/path/to/directory/containing/mysite/')
I had a similar problem to this a while ago while moving my project from Django 1.3 and having the settings.py file at the root of my source and then moving it down into the application.
For example what happened was that I had the following:
rootOfSource/
- settings.py
- myapp
and I changed it to be:
rootOfSource/
- myapp
- myapp/settings.py
and I also changed my settings file to be the following:
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
However when I debugged into the os.eviron I found that the DJANGO_SETTINGS_MODULE was not as expected, I then changed my manage.py to be the following:
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings")
os.environ.__setitem__("DJANGO_SETTINGS_MODULE", "myapp.settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
Which then allowed me to run from PyDev.
Hope this helps.
I fixed this problem by going to the project properties -> PyDev Django and setting the Django settings module.
I am trying to have a Django script run every 5 minutes via cron on my dev laptop (Mac OS X). Here is the code in the script:
import sys
import os
def setup_environment():
pathname = os.path.dirname(sys.argv[0])
sys.path.append(os.path.abspath(pathname))
sys.path.append(os.path.normpath(os.path.join(os.path.abspath(pathname), '../')))
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
setup_environment()
from common.models import TweetCache
import datetime
def main():
print "(%s) Caching tweets..." % str(datetime.datetime.now())
previous_tweets = TweetCache.objects.filter(username='atmospherian')
for prev in previous_tweets:
prev.delete()
import twitter
api = twitter.Api()
tweets = api.GetUserTimeline('atmospherian')
for t in tweets:
tc = TweetCache(username='atmospherian', date=t.created_at, text=t.text)
tc.save()
if __name__ == '__main__':
main()
crontab:
*/5 * * * * python /absolute/path/to/tweet_cache.py
error from system mail:
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=jason>
X-Cron-Env: <USER=jason>
X-Cron-Env: <HOME=/Users/jason>
Date: Tue, 16 Feb 2010 17:45:00 -0500 (EST)
Traceback (most recent call last):
File "/Users/jason/Development/bandistry/tweet_cache.py", line 22, in <module>
from common.models import TweetCache
File "/Users/jason/Development/bandistry/common/models.py", line 1, in <module>
from django.db import models
ImportError: No module named django.db
can anyone tell me what im doing wrong?
sys.argv[0] isn't always the most reliable way to get the current file's path. I recommend this modification:
pathname = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, pathname)
sys.path.insert(0, os.path.abspath(os.path.join(pathname, '..')))
Note the use of sys.path.insert instead of sys.path.append, and the use of file. Also, using abspath on file -before- dirname reduces the chance you reduce the entire filename down to the empty string or simply '.' which may not even be accurate.
Also, is the django package installed in one of those two paths you add? If not, you sill need to add that path
Finally, a minor quip, probably unrelated to your django import issue, but you really should be doing:
os.environ['DJANGO_SETTINGS_MODULE'] = 'bandistry.settings'
It may work without, but it's better if you put your entire django application in a package. This reduces the chance of you obscuring other package names.