Installed Celery but it does not implement tasks - django

Question

I,m using Amazon web services. And I'm trying to make Celery work.
Tried to do what the documentation says (http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html#configuring-your-django-project-to-use-celery), but when I start celery worker and try to call a task nothing happens, as if there were no task.
Here's my settings.py file:
import os
import djcelery
import djkombu
import sys
import tasks
sys.path.append(os.getcwd())
djcelery.setup_loader()
# Django settings for analogg project.
DEBUG = False
TEMPLATE_DEBUG = DEBUG
ADMINS = (
('Nikita', 'bodnarnikita#gmail.com'),
)
MANAGERS = ADMINS
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
'NAME': 'analoggdb', # Or path to database file if using sqlite3.
'USER': '', # Not used with sqlite3.
'PASSWORD': '', # Not used with sqlite3.
'HOST': '', # Set to empty string for localhost. Not used with sqlite3.
'PORT': '', # Set to empty string for default. Not used with sqlite3.
}
}
# Local time zone for this installation. Choices can be found here:
# http://en.wikipedia.org/wiki/List_of_tz_zones_by_name
# although not all choices may be available on all operating systems.
# On Unix systems, a value of None will cause Django to use the same
# timezone as the operating system.
# If running in a Windows environment this must be set to the same as your
# system time zone.
TIME_ZONE = 'America/Chicago'
# Language code for this installation. All choices can be found here:
# http://www.i18nguy.com/unicode/language-identifiers.html
LANGUAGE_CODE = 'en-us'
SITE_ID = 1
# If you set this to False, Django will make some optimizations so as not
# to load the internationalization machinery.
USE_I18N = True
# If you set this to False, Django will not format dates, numbers and
# calendars according to the current locale.
USE_L10N = True
# If you set this to False, Django will not use timezone-aware datetimes.
USE_TZ = True
# Absolute filesystem path to the directory that will hold user-uploaded files.
# Example: "/home/media/media.lawrence.com/media/"
MEDIA_ROOT = ''
# URL that handles the media served from MEDIA_ROOT. Make sure to use a
# trailing slash.
# Examples: "http://media.lawrence.com/media/", "http://example.com/media/"
MEDIA_URL = '/static/'
# Absolute path to the directory static files should be collected to.
# Don't put anything in this directory yourself; store your static files
# in apps' "static/" subdirectories and in STATICFILES_DIRS.
# Example: "/home/media/media.lawrence.com/static/"
SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
STATIC_ROOT = os.path.join(SITE_ROOT, 'static')
# URL prefix for static files.
# Example: "http://media.lawrence.com/static/"
STATIC_URL = 'http://analogg.info/static/'
# Additional locations of static files
STATICFILES_DIRS = (
# Put strings here, like "/home/html/static" or "C:/www/django/static".
# Always use forward slashes, even on Windows.
# Don't forget to use absolute paths, not relative paths.
)
# List of finder classes that know how to find static files in
# various locations.
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'django.contrib.staticfiles.finders.DefaultStorageFinder',
)
ADMIN_MEDIA_PREFIX = '/static/admin/'
# List of callables that know how to import templates from various sources.
TEMPLATE_LOADERS = (
'django.template.loaders.filesystem.Loader',
'django.template.loaders.app_directories.Loader',
# 'django.template.loaders.eggs.Loader',
)
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
# Uncomment the next line for simple clickjacking protection:
# 'django.middleware.clickjacking.XFrameOptionsMiddleware',
)
ROOT_URLCONF = 'analogg.urls'
# Python dotted path to the WSGI application used by Django's runserver.
WSGI_APPLICATION = 'analogg.wsgi.application'
TEMPLATE_DIRS = (
"/templates/",
"/home/ubuntu/analogg/templates",
# Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
# Always use forward slashes, even on Windows.
# Don't forget to use absolute paths, not relative paths.
)
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
# Uncomment the next line to enable the admin:
'django.contrib.admin',
# Uncomment the next line to enable admin documentation:
'django.contrib.admindocs',
)
INSTALLED_APPS += ('djcelery', )
#INSTALLED_APPS += ('djkombu', )
CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"
CELERY_SEND_TASK_ERROR_EMAILS = True
CELERY_DISABLE_RATE_LIMITS = True
CELERY_IMPORTS = ("analogg.tasks", )
#BROKER_URL = 'amqp://guest:guest#localhost:5672/'
#BROKER_BACKEND = "djkombu.transport.DatabaseTransport"
# A sample logging configuration. The only tangible logging
# performed by this configuration is to send an email to
# the site admins on every HTTP 500 error when DEBUG=False.
# See http://docs.djangoproject.com/en/dev/topics/logging for
# more details on how to customize your logging configuration.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
}
},
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}
This is my views.py file:
# -*- coding: utf-8 -*-
from django.http import HttpResponseRedirect, HttpResponse
from django.template import Context, loader, RequestContext
from django.shortcuts import render_to_response
from tasks import parsesob, add
from parsersob import parser
import os, random, string
def index(request):
html = "<html><body>Hello, world!</body></html>"
add.apply_async(args=[id, '23bnn'])
return HttpResponse(html)
This is my tasks.py file:
from celery.decorators import task
from celery.task.schedules import crontab
from parsersob import parser
#from parsersob2 import parser2
from celery import Celery
#celery = Celery('tasks', broker='amqp://rabbitmqprelaunch9176#ip-10-117-81-80//')
#task
def add(id, a1, a2):
f = open('add.txt', 'w')
g = a1 + a2
f.write(g)
f.close()
And this is my celeryconfig.py file being in the same folder with tasks.py, settings.py and views.py:
#BROKER_URL = 'amqp://'
#CELERY_RESULT_BACKEND = 'analoggdb'
#BROKER_URL = "amqp://guest:guest#localhost:5672//"
CELERY_IMPORTS = ("analogg.tasks", )
#CELERY_RESULT_DBURI = "sqlite:///analoggdb.db"
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'Europe/Moscow'
CELERY_ENABLE_UTC = True
I'm a newbie, so could anyone please tell me what to do to make it work.
Any help will be greatly appreciated.
UPD:
ubuntu#ip-10-117-81-80:/var/log/apache2$ sudo rabbitmqctl status
Status of node 'rabbit#ip-10-117-81-80' ...
[{pid,849},
{running_applications,[{rabbit,"RabbitMQ","2.7.1"},
{os_mon,"CPO CXC 138 46","2.2.7"},
{sasl,"SASL CXC 138 11","2.1.10"},
{mnesia,"MNESIA CXC 138 12","4.5"},
{stdlib,"ERTS CXC 138 10","1.17.5"},
{kernel,"ERTS CXC 138 10","2.14.5"}]},
{os,{unix,linux}},
{erlang_version,"Erlang R14B04 (erts-5.8.5) [source] [64-bit] [rq:1] [async-threads:30] [kernel-poll:true]\n"},
{memory,[{total,25494568},
{processes,11083752},
{processes_used,11077008},
{system,14410816},
{atom,1124433},
{atom_used,1120234},
{binary,89696},
{code,11134393},
{ets,752120}]},
{vm_memory_high_watermark,0.3999999990304762},
{vm_memory_limit,247544217}]
...done.
The output of running celeryd:
ubuntu#ip-10-117-81-80:~/analogg$ python manage.py celeryd worker --loglevel=info
-------------- celery#ip-10-117-81-80 v3.0.3 (Chiastic Slide)
---- **** -----
--- * *** * -- [Configuration]
-- * - **** --- . broker: amqp://guest#localhost:5672//
- ** ---------- . app: default:0x13c2a90 (djcelery.loaders.DjangoLoader)
- ** ---------- . concurrency: 1 (processes)
- ** ---------- . events: OFF (enable -E to monitor this worker)
- ** ----------
- *** --- * --- [Queues]
-- ******* ---- . celery: exchange:celery(direct) binding:celery
--- ***** -----
[Tasks]
. analogg.tasks.add
. analogg.tasks.parsesob
[2012-07-22 06:24:30,336: WARNING/MainProcess] celery#ip-10-117-81-80 has started.
UPD:
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] mod_wsgi (pid=7411): Exception occurred processing WSGI script '/home/ubuntu/analogg/apache/django.wsgi'.
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] Traceback (most recent call last):
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/wsgi.py", line 241, in __call__
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] response = self.get_response(request)
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 82, in get_response
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] urlconf = settings.ROOT_URLCONF
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 185, in inner
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] return func(self._wrapped, *args)
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] AttributeError: 'Settings' object has no attribute 'ROOT_URLCONF'
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] mod_wsgi (pid=7377): Exception occurred processing WSGI script '/home/ubuntu/analogg/apache/django.wsgi'.
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] Traceback (most recent call last):
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/wsgi.py", line 241, in __call__
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] response = self.get_response(request)
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 82, in get_response
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] urlconf = settings.ROOT_URLCONF
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 185, in inner
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] return func(self._wrapped, *args)
[Sun Jul 22 08:10:58 2012] [error] [client 37.110.0.126] AttributeError: 'Settings' object has no attribute 'ROOT_URLCONF'

Answer 1

I'm not quite sure what is the problem here, but here is an example config, that works:
This is located at the end of settings.py:
#CELERY CONFIG
import djcelery
djcelery.setup_loader()
BROKER_HOST = "Host"
BROKER_PORT = "Port"
BROKER_USER = "User"
BROKER_PASSWORD = "Pass"
BROKER_VHOST = "VHost"
CELERY_DEFAULT_QUEUE = "some_queue"
CELERY_DEFAULT_EXCHANGE = "some_exc"
The broker's user/pass/vhost are configured using the rabbitmqctl tool
You can trace the queue/exchange with
rabbitmqadmin get queue="some_queue"
NOTE: It is very useful to track the logs in /var/log/rabbitmq/rabbit#localhost.log or whatever the log file is. This will show you if there is a problem with the connection to RabbitMQ, like wrong credentials, permissions, etc.