No module named 'django' when trying to import Django into Scrapy - django

This is my folder structure:
root:
|--Django_project
|--|-- db_app
|--|-- Django_project
|--|-- manage.py
|--Scrapy_project
|--|--Scrapy_project
|--|--|--spiders
|--|--|--settings.py
|--|--|--pipelines.py
|--|--|--items.py
|--|--|--middlewares.py
|--|--scrapy.cfg
In settings.py I have this:
import sys
import os
sys.path.append(os.path.dirname(os.path.abspath('.')))
os.environ['DJANGO_SETTINGS_MODULE'] = 'Django_project.settings'
import django
django.setup()
I've tried every possible path, including an absolute path to the project root, to the Django project, to the Django app - nothing works and I get this:
ModuleNotFoundError: No module named 'django'
Thanks for the help!
EDIT:
I should probably clarify that I'm running in a virtual environment.

Related

ModuleNotFoundError: No module named 'decouple'

I have a working Django project. If I run it - everything works fine
Here is the structure of the project
titest_project
tibrains_app
tools
__init__.py
load_django.py
countries_add.py
apps.py
models.py
load_django.py
import sys
import os
import django
sys.path.append('/mnt/HDD/tibrains/03/titest_project')
os.environ['DJANGO_SETTINGS_MODULE'] = 'titest_project.settings'
django.setup()
countries_add.py
import load_django
from tibrains_app.models import ShopCountry
When I run countries_add.py it gives the error - No module named 'decouple'. Though when I run the project itself - it works without errors.
What could be the problem?

setting up Django app to work on google app engine GAE (app.yaml & main.py)

I have a working Django app that I was able to get functioning on Heroku. The structure is project named 'untitled' and an app named 'web' such that the structure is:
project_root
static
templates
untitled
--->init.py
--->settings.py
--->urls.py
--->wsgi.py
web
--->init.py
--->admin.py
--->apps.py
--->models.py
--->tests.py
--->urls.py
--->views.py
This is a fairly basic app that I can get working outside of GAE (local and on Heroku), however, I'm getting stuck on the app.yaml and main.py requirements for GAE.
My app.yaml is:
application: seismic-interpretation-institute-py27
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.app
libraries:
- name: django
version: "latest"
and my main.py (generated from PyCharm) is
import os,sys
import django.core.handlers.wsgi
import django.core.signals
import django.db
import django.dispatch.dispatcher
# Google App Engine imports.
from google.appengine.ext.webapp import util
# Force Django to reload its settings.
from django.conf import settings
settings._target = None
os.environ['DJANGO_SETTINGS_MODULE'] = 'untitled.settings'
# Unregister the rollback event handler.
django.dispatch.dispatcher.disconnect(
django.db._rollback_on_exception,
django.core.signals.got_request_exception)
def main():
# Create a Django application for WSGI.
application = django.core.handlers.wsgi.WSGIHandler()
# Run the WSGI CGI handler with that application.
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
Finally, the output that is reported when running locally is
It seems that the error,
ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
is causing my problems. I am not exactly sure how to fix it.
try replace
from django.conf import settings
settings._target = None
os.environ['DJANGO_SETTINGS_MODULE'] = 'untitled.settings'
to
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "untitled.settings")
from django.conf import settings
settings._target = None

Django Gunicorn no module named error

My project directory structure looks like following:
My wsgi file:
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
application = get_wsgi_application()
and my manage.py file:
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
try:
from django.core.management import execute_from_command_line
except ImportError:
# The above import may fail for some other reason. Ensure that the
# issue is really that Django is missing to avoid masking other
# exceptions on Python 2.
try:
import django
except ImportError:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
)
raise
execute_from_command_line(sys.argv)
in my settings .py I have:
WSGI_APPLICATION = 'config.wsgi.application'
I added my project path to python with following:
export PYTHONPATH=/home/ec2-user/amm_inovision_backend:$PYTHONPATH
and I am trying to run gunicorn command where my manage.py file is as: gunicorn amm_inovision_backend.config.wsgi:application
But it throws me error no module named amm_inovision_backend.config.wsgi
If I run instead gunicorn config.wsgi:application it throws no module named amm_inovision_backend.config.settings
What am I doing wrong?
Note: in the screenshot it is amm_ino_backend but actually it is amm_inovision_backend in the production
when you want to use gunicorn you must run gunicorn installed on your virtual environment.
so you don't need to export any python path, you only need to find path to gunicorn on your virtual-env where Django is installed, or you can add path to virtual environment not to project files.
also you need first to edit your wsgi.py
you have to replace this
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings") with
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
your command must be something like that:
/path/to/.virtual-envs/your-env/bin/gunicorn config.wsgi:application
Use:
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
application = get_wsgi_application()
note the "CONFIG.SETTINGS"

ImportError on newaly production django + apache + mod_wsgi

I'm in the middle of deploying my django project.
got the wsgi working and now I got importError.
error:
Exception Type: ImportError
Exception Value:
No module named forms
this is the import:
from profile.forms import UserRegistrationForm
profile is a folder resides in my project dir.
this is the wsgi script I use:
import os
import sys
path = '/srv/www/project'
if path not in sys.path:
sys.path.append(path)
path = '/srv/www'
if path not in sys.path:
sys.path.append(path)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
got it to work, needed to add project name as prefix.
from projectname.profile.forms import <something>

Scrapy project can't find django.core.management

I'm trying to follow the method here to 'Scrapy' data from the web and simultaneously save that data directly to my Django database using Scrapy's item pipeline.
However, when I try to run scrapy crawl spidername, I'm getting the error:
ImportError: No module named django.core.management
At first I thought it was because my Scrapy project was outside of my Django project folder, but even after I moved the whole project into my Django project folder I kept getting the same error. If I open a python shell inside the Scrapy project folder in its new location (inside my Django project folder), import django.core.management works fine. So what's going on?
EDIT: Some additional info: I'm doing this on a Webfaction server, and the path to my Django project is /home/gchorn/webapps/django_app/django_project. I'm using Django version 1.4.1, Scrapy 0.16 and Python2.7. The layout of the Django+Scrapy project is as follows:
django_project/
__init__.py
manage.py
settings.py
urls.py
myproject #folder containing wsgi.py
app1
app2
app3
templates
ScrapyProject/
scrapy.cfg
ScrapyProject/
__init__.py
items.py
pipelines.py
settings.py
spiders/
__init__.py
my_spider.py
Try setting this in your Spider's settings.py:
import os
import sys
sys.path.append('/home/gchorn/webapps/django_app')
os.environ['DJANGO_SETTINGS_MODULE'] = 'django_project.settings'
Then you can import your model classes like:
from django_project.app1.models import some_model