Celery Cannot find a module in my Django project - django

I have a Django 2.0 project using celery 4.2.1 and redis 2.10.6. The django project has two apps, memorabilia and face_recognition. I have it all successfully running tasks with django running on my development machine. I uploaded everything to my git server, then installed the apps on my laptop from git, updated all requirements, etc. Both are Ubuntu machines. I am not using django-celery.
When I try to run celery -A MemorabiliaJSON worker -l debug,
I get an exception saying ModuleNotFoundError: No module named 'face_recognition.tasks'
I am not sure how to fix this, as the same code base is running on my development machine.
My file structure is:
├── celery.sh
├── face_recognition
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   ├── models.py
│   ├── __pycache__
│   ├── tasks.py
│   ├── tests.py
│   └── views.py
├── __init__.py
├── manage.py
├── memorabilia
│   ├── admin.py
│   ├── apps.py
│   ├── fields.py
│   ├── fixtures
│   ├── __init__.py
│   ├── logs
│   ├── migrations
│   ├── models.py
│   ├── __pycache__
│   ├── storage.py
│   ├── tasks.py
│   ├── templates
│   ├── tests
│   ├── urls.py
│   ├── validators.py
│   ├── views.py
│   ├── widgets.py
├── MemorabiliaJSON
│   ├── celery.py
│   ├── default_images
│   ├── documents
│   ├── __init__.py
│   ├── __pycache__
│   ├── settings
│   ├── static
│   ├── urls.py
│   ├── views.py
│   ├── wsgi.py
├── __pycache__
│   ├── celery.cpython-36.pyc
│   └── __init__.cpython-36.pyc
├── requirements.txt
└── tests
MemorabiliaJSON/celery.py
# http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
from django.apps import apps
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'MemorabiliaJSON.settings.tsunami')
app = Celery('MemorabiliaJSON')
app.config_from_object('django.conf:settings', namespace='CELERY')
#app.autodiscover_tasks(lambda: [n.name for n in apps.get_app_configs()])
app.autodiscover_tasks()
#app.task(bind=True)
def debug_task(self):
print('Request: {0!r}'.format(self.request))
(memorabilia-JSON) mark#octopus:~/python-projects/memorabilia-JSON
face_recognition/__init__.py
default_app_config = 'face_recognition.apps.FaceRecognitionConfig'
memorabilia/__init__.py
default_app_config = 'memorabilia.apps.MemorabiliaConfig'
INSTALLED_APPS has these two apps
'memorabilia.apps.MemorabiliaConfig',
'face_recognition.apps.FaceRecognitionConfig',

Related

List apps of a django project

I'm completely new for django and I'd want to list the apps of a django project, for example:
FeinCMS
I know that startapp creates the directory structure for an app. I wonder if there is either a function or a file to get the app list.
For example taking FeinCMS as an example, the repo contains:
feincms
├── AUTHORS
├── CHANGELOG.rst
├── CONTRIBUTING.rst
├── docs
│   ├── admin.rst
│   ├── advanced
│   ├── conf.py
│   ├── contenttypes.rst
│   ├── contributing.rst
│   ├── deprecation.rst
│   ├── extensions.rst
│   ├── faq.rst
│   ├── images
│   ├── index.rst
│   ├── installation.rst
│   ├── integration.rst
│   ├── Makefile
│   ├── medialibrary.rst
│   ├── migrations.rst
│   ├── page.rst
│   ├── releases
│   ├── settings.rst
│   ├── templatetags.rst
│   └── versioning.rst
├── feincms
│   ├── admin
│   ├── apps.py
│   ├── content
│   ├── contents.py
│   ├── context_processors.py
│   ├── contrib
│   ├── default_settings.py
│   ├── extensions
│   ├── __init__.py
│   ├── _internal.py
│   ├── locale
│   ├── management
│   ├── models.py
│   ├── module
│   ├── shortcuts.py
│   ├── signals.py
│   ├── static
│   ├── templates
│   ├── templatetags
│   ├── translations.py
│   ├── urls.py
│   ├── utils
│   └── views
├── LICENSE
├── MANIFEST.in
├── README.rst
├── setup.cfg
├── setup.py
└── tests
├── cov.sh
├── manage.py
├── requirements.txt
├── testapp
└── tox.ini
How can I use django to list the apps in the repo?
settings.INSTALLED_APPS is the list of apps. You can start a django shell (manage.py shell) to query the settings:
from django.conf import settings
print(settings.INSTALLED_APPS)
>>> ['user', 'django.contrib.auth', 'django.contrib.sites', ...]
Query django.apps ...
from django.apps import apps
for app in apps.get_app_configs():
print(app, app.name, app.label)
Results in ...
<ContentTypesConfig: contenttypes> django.contrib.contenttypes contenttypes
<AdminConfig: admin> django.contrib.admin admin

Flask Blueprint; No module found named

I'm trying to get Flask Blueprints running in Docker, but having issues with registering Blueprints correct.
I have the following structure:
├── docker-compose.yml
├── nginx
│   ├── Dockerfile
│   └── sites-enabled
│   └── flask_project
└── web
├── Dockerfile
├── __init__.py
├── app.py
├── modules
│   ├── __init__.py
│   └── page
│   ├── __init__.py
│   ├── forms.py
│   ├── models.py
│   ├── views.py
├── requirements.txt
├── static
│   ├── css
│   │   ├── bootstrap.min.css
│   │   └── main.css
│   ├── img
│   └── js
│   └── bootstrap.min.js
└── templates
├── _base.html
└── index.html
app.py contains:
from flask import Flask
from web.modules.page import simple_page
app = Flask(__name__)
app.register_blueprint(simple_page)
if __name__ == '__main__':
print app.url_map
app.run(debug=True)
views.py contains:
from flask import Blueprint
simple_page = Blueprint('simple_page', __name__,
template_folder='templates')
#simple_page.route('/')
def index():
return "Hello world"
__init__.py under page:
from web.modules.page.views import simple_page
The __init__.py files are empty.
The console gives an ImportError: No module named web.modules.page
Thanks for your time.
Look like is structure problem, you can reference from here: https://www.digitalocean.com/community/tutorials/how-to-structure-large-flask-applications
The following is my example, hope it can help for you:
├── app
│   ├── __init__.py
│   ├── main
│   │   ├── __init__.py
│   │   └── views.py
│   ├── models
│   │   └── __init__.py
│   ├── static
│   │   ├── css
│   │   ├── js
│   │   ├── img
│   │   └── file
│   ├── templates
│   │   └── index.html
└── master.py
app/__init__.py
from flask import Flask
from app.main import main
def create_app():
app = Flask(__name__)
app.register_blueprint(main)
return app
app/main/__init__.py
from flask import Blueprint
main = Blueprint('main', __name__)
from app.main import views
master.py
from app import create_app
if __name__ == '__main__':
app = create_app()
app.run(host='0.0.0.0', port=8000, threaded=True)

Can't make test execute

Folders structure:
.
├── db.sqlite3
├── homepage
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
├── manage.py
├── photoarchive
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
├── somesite
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-34.pyc
│   │   ├── settings.cpython-34.pyc
│   │   ├── urls.cpython-34.pyc
│   │   └── wsgi.cpython-34.pyc
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── tests
├── functional_test.py
├── __init__.py
├── __pycache__
│   ├── functional_test.cpython-34.pyc
│   ├── __init__.cpython-34.pyc
│   └── validators.cpython-34.pyc
└── validators.py
functional_test.py
from selenium import webdriver
from django.test import TestCase
import pdb
class HomePageTest(TestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
pdb.set_trace()
def tearDown(self):
self.browser.quit()
def test_home_page(self):
#Edith goes to home page.
self.browser.get("http://localhost:8000")
#Edith sees "Hello, world" in the browser title.
estimated_browser_title ="Hello, world"
real_browswer_title = self.browser.title
self.assertIn(estimated_browser_title, real_browswer_title)
I run the test:
(venv) michael#michael:~/workspace/mysite/somesite$ python manage.py test tests
Creating test database for alias 'default'...
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
Destroying test database for alias 'default'...
Could you help me understand why my tests are not executed. I set a pdb breakpoint. The interpreter doesn't stop at that breakpoint. Well, tests are ignored.
Could you give me a kick here?
Django's test runner will only discover tests inside apps that are included in INSTALLED_APPS. Rather than putting your code in a functional_test.py file inside a tests directory, you should put it in a file called tests.py inside one of the app directories.

Django relative import from external app

I'm a 2 Scoops of Django 1.8 reader. Chapter 29 (what about those random utilities) suggests to create a core app to store commonly used code. It also suggests that you can use this syntax to import code from it:
e.g.
from core.models import TimeStampedModel
How ever it seems that this relative import does not work. I'm using cookiecutter-django and I needed to do:
from projectname.apps.core.models import TimeStampedModel
I tried adding my APPS_DIR to the path:
sys.path.insert(str(APPS_DIR))
But that resulted in import conflicts given that now there were 2 modules with the same name, new_app and projectname.apps.new_app.
I just want to avoid explicit imports. Is there a way to include the Installed Apps in the python path without creating import conflicts? what are best practices regarding external apps imports?
edit: adds project structure
.
├── README.rst
├── manage.py
├── config
│   ├── __init__.py
│   ├── settings
│   │   ├── __init__.py
│   │   ├── common.py
│   │   ├── local.py
│   │   ├── test.py
│   │   ├── production.py
│   │   └── staging.py
│   ├── urls.py
│   ├── views.py
│   └── wsgi.py
├── projectname
│   ├── __init__.py
│   ├── apps
│   │   ├── __init__.py
│   │   ├── core
│   │   │   └── __init__.py
│   │   └── new_app
│   │ └── __init__.py
│   ├── static
│   │   └── ...
│   └── templates
│   └── ...
├── requirements
│   ├── base.txt
│   ├── local.txt
│   ├── production.txt
│   └── test.txt
└── tests
└── ...
If you want to use relative imports you need to import it this way
from .core.models import TimeStampedModel
This would take the relative path from which the code is being executed, unlike absolute imports which are not supported in Django 1.8

auto importing python classes in flask shell not working as expected

So I have a flask app structured in this manner
calvin % tree -L 3 .
.
├── README
├── alembic
│   ├── README
│   ├── env.py
│   ├── env.pyc
│   ├── script.py.mako
│   └── versions
├── alembic.ini
├── api.sublime-project
├── app
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── admin
│   │   ├── __init__.py
│   │   └── __init__.pyc
│   ├── agencies
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── models.py
│   │   └── models.pyc
│   ├── agents
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── models.py
│   │   ├── models.pyc
│   │   ├── views.py
│   │   └── views.pyc
│   ├── api.py
│   ├── api.pyc
│   ├── auth
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── constants.py
│   │   ├── constants.pyc
│   │   ├── decorators.py
│   │   ├── decorators.pyc
│   │   ├── models.py
│   │   ├── models.pyc
│   │   ├── views.py
│   │   └── views.pyc
│   ├── districts
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── models.py
│   │   ├── models.pyc
│   │   ├── views.py
│   │   └── views.pyc
│   ├── helpers.py
│   ├── helpers.pyc
│   ├── middleware.py
│   ├── middleware.pyc
│   ├── models.py
│   ├── models.pyc
│   ├── properties
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── models.py
│   │   └── models.pyc
│   ├── templates
│   │   └── index.html
│   ├── users
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── models.py
│   │   ├── models.pyc
│   │   ├── views.py
│   │   └── views.pyc
│   └── viewings
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── models.py
│   ├── models.pyc
│   ├── views.py
│   └── views.pyc
├── config.py
├── config.pyc
├── manage.py
├── requirements.txt
├── run.py
└── shell.py
I am setting up my shell so that it auto-imports all the classes located in the models.py files when I execute ./manage.py shell
And this is the script in manage.py that is intended to achieve that (reference flask-script docs)
def _make_context():
from app import models
return dict(app=app, db=db, models=models) # TODO: this is not working appropriately
manager.add_command("shell", Shell(make_context=_make_context))
In my app/models.py, I have import statements from every module, "agencies", "auth" etc etc.
However, when I enter my shell environment, I have to access my classes as models.Users instead of directly Users, which is not what I am expecting. How do I auto-import everything so that I can access the classes directly?
Below is a simplified solution that assumes you're using SQLAlchemy (db.Model). If not, you should only need to change the issubclass if-statement to match your appropriate check of what to import.
from app import db, create_app
import app.models as models
from flask.ext.script import Manager, Shell
app = create_app()
manager = Manager(app)
def make_shell_context():
return_dict = {}
# grab everything we can possibly import from the models module
module_importables = dir(models)
for importable in module_importables:
# if it isn't a class, it'll throw a TypeError exception that importable is not a class
try:
# we only want our SQLAlchemy models
if issubclass(getattr(models,importable),db.Model):
return_dict[importable] = getattr(models,importable)
except TypeError as inst:
pass
return_dict['app'] = app
return_dict['db'] = db
return return_dict
manager.add_command("shell", Shell(make_context=make_shell_context))
if __name__ == '__main__':
manager.run()
Instead of
from app import models
You can do:
from app.models import *
This will import all classes,variables from models. Note: It is usually not recommended to import * but in this case, it could work for you. Ideally, you should do something like:
from app.models import Users, ...and so on