In trying to replace django-celery and upgrade celery to 4.x from an inherited project, I'm having hard time understanding the real changes to effect.
Celery is already setup as the project uses 3.x, however in removing djcelery from the app, I come across this:
CELERY_RESULT_BACKEND = 'djcelery.backends.database:DatabaseBackend'
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
Reading the docs, I'm more confused about using result_backend or celery.backend.database or which:
CELERY_RESULT_BACKEND = 'celery.backends.database'
CELERYBEAT_SCHEDULER = 'beat_scheduler' OR
CELERY_RESULT_BACKEND: result_backend
CELERYBEAT_SCHEDULER: beat_scheduler
I'm new to Celery, still getting familiar with the details.
Celery 4 changed their settings as follows: http://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings
The major difference between previous versions, apart from the lower
case names, are the renaming of some prefixes, like celerybeat_ to
beat_, celeryd_ to worker_, and most of the top level celery_ settings
have been moved into a new task_ prefix.
Celery will still be able to read old configuration files, so there’s
no rush in moving to the new settings format.
The expectation is that you use result_backend instead of CELERY_RESULT_BACKEND. Full mapping of old upper case settings to new ones are documented here: http://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings
In other words, resut_backend is the new name of the key, NOT the new recommended value. It is the replacement for the left hand side of your assignment. These are equivalent:
CELERY_RESULT_BACKEND = 'djcelery.backends.database:DatabaseBackend'
result_backend = 'djcelery.backends.database:DatabaseBackend'
Likewise these are equivalent:
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
beat_scheduler = 'djcelery.schedulers.DatabaseScheduler'
I'm using Django served by uWSGI and NGINX.
Ubuntu 14.04.1 LTS 64-bit
Python 3.4
Django 1.7.4
uWSGI 1.9.17.1-debian (64bit)
NGINX 1.4.6
python-pdfkit 0.5.0
wkhtmltopdf 0.12.2.1
OpenLayers v3.0.0
When I try running pdfkit.from_url(...) to print a map to pdf the request times out.
More specifically it hangs in python's subprocess.py communicate, self._communicate:
with _PopenSelector() as selector:
if self.stdin and input:
selector.register(self.stdin, selectors.EVENT_WRITE)
if self.stdout:
selector.register(self.stdout, selectors.EVENT_READ)
if self.stderr:
selector.register(self.stderr, selectors.EVENT_READ)
while selector.get_map():
...
selector.get_map() always returns a valid result, ensuring an infinite loop.
If I run this in the Django development server (instead of uWSGI+NGINX) everything runs fine.
in my view:
wkhtmltopdfBinLocationString = '/usr/local/bin/wkhtmltopdf'
wkhtmltopdfBinLocationBytes = wkhtmltopdfBinLocationString.encode('utf-8')
#this fixes some leftover python2 assumptions about strings
config = pdfkit.configuration(wkhtmltopdf=wkhtmltopdfBinLocationBytes)
pdfkit.from_url(reportPdfUrl, reportPdfFile, configuration=config, options={
'javascript-delay': 1500
})
Several places I have seen answers along the line of "set the close-on-exec flag on the socket" solving similar issues.
Is this something I can set from my "from_url" options (wkhtmltopdf does not accept it by that name) or can I configure uWSGI to assume 'close-on-exec'? I have not been able to make either of these work, but maybe I just need help with changing my uWSGI customization file:
[uwsgi]
workers = 1
chdir = [...]
plugins = python34
wsgi-file = [...]/wsgi.py
pythonpath = [...]
I tried something like
close-on-exec = true
but that didn't seem to do anything.
NOTE: the wsgi.py file is simple:
"""
WSGI config for dst project.
It exposes the WSGI callable as a module-level variable named ``application``.
For more information on this file, see
https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/
"""
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "[my_project].settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Any thoughts?
On upgrading to Django 1.7 I'm getting the following error message from ./manage.py
$ ./manage.py
Traceback (most recent call last):
File "./manage.py", line 16, in <module>
execute_from_command_line(sys.argv)
File "/home/johnc/.virtualenvs/myproj-django1.7/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 427, in execute_from_command_line
utility.execute()
File "/home/johnc/.virtualenvs/myproj-django1.7/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 391, in execute
django.setup()
File "/home/johnc/.virtualenvs/myproj-django1.7/local/lib/python2.7/site-packages/django/__init__.py", line 21, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/johnc/.virtualenvs/myproj-django1.7/local/lib/python2.7/site-packages/django/apps/registry.py", line 89, in populate
"duplicates: %s" % app_config.label)
django.core.exceptions.ImproperlyConfigured: Application labels aren't unique, duplicates: foo
What's the problem and how do I resolve it?
The problem is that with the changes to apps in Django 1.7, apps are required to have a unique label.
By default the app label is the package name, so if you've got a package with the same name as one of your app modules (foo in this case), you'll hit this error.
The solution is to override the default label for your app, and force this config to be loaded by adding it to __init__.py.
# foo/apps.py
from django.apps import AppConfig
class FooConfig(AppConfig):
name = 'full.python.path.to.your.app.foo'
label = 'my.foo' # <-- this is the important line - change it to anything other than the default, which is the module name ('foo' in this case)
and
# foo/__init__.py
default_app_config = 'full.python.path.to.your.app.foo.apps.FooConfig'
See https://docs.djangoproject.com/en/1.7/ref/applications/#for-application-authors
I found simple solution for this. In my case following line is added twice under INSTALLED_APPS,
'django.contrib.foo',
Removed one line fixes the issue for me.
I had the same error - try this:
In INSTALLED_APPS, if you are including 'foo.apps.FooConfig', then Django already knows to include the foo app in the application, there is therefore no need to also include 'foo'.
Having both 'foo' and 'foo.apps.FooConfig' under INSTALLED_APPS could be the source of your problem.
Well, I created a auth app, and included it in INSTALLED_APP like src.auth (because it's in src folder) and got this error, because there is django.contrib.auth app also. So I renamed it like authentication and problem solved!
I got the same problem.
Here my app name was chat and in the settings.py , under installed apps i have written chat.apps.ChatConfig while i have already included the app name chat at the bottom. When i removed the chat.apps.ChatConfig mine problem was solved while migrations. This error may be due to the same instance that you might have defined you app name foo twice in the settings.py. I hope this works out!!
please check if anything is duplicated in INSTALLED_APPS of settings.py
This exception may also be raised if the name of the AppConfig class itself matches the name of another class in the project. For example:
class MessagesConfig(AppConfig):
name = 'mysite.messages'
and
class MessagesConfig(AppConfig):
name = 'django.contrib.messages'
will also clash even though the name attributes are different for each configuration.
In previous answer 'django.contrib.foo', was mentioned, but basically adding any app twice can cause this error just delete one (Django 3.0)
for me it was in settings.py
INSTALLED_APPS = [
...
'accounts.apps.AccountsConfig',
'accounts.apps.AccountsConfig',
...
]
just delete one of them
Basically this problem has been created due to duplication of name of installed app in the settings:
This is how I resolved the problem. In settings.py file:
Check the install app in the setting.py if the install app are duplicate
Error shown due to duplication of app name
Remove the duplicate name in the install file
After problem is resolved, you will see interface in your screen
For this I have created application name as polls instead of foo
As therefromhere said this is a new Django 1.7 feature which adds a kind of “app registry” where applications must be determined uniquely (and not only having different python paths).
The name attribute is the python path (unique), but the label also should be unique. For example if you have an app named 'admin', then you have to define the name (name='python.path') and a label which must be also unique (label='my admin' or as said put the full python path which is always unique).
Had same issue, read through the settings.py of the root folder, removed any INSTALLED APPS causing conflict... works fine. Will have to rename the apps names
Need to check in two file
1- apps.py
code something like
from django.apps import AppConfig
from django.utils.translation import ugettext_lazy as _
class ModuleConfig(AppConfig):
name = "ocb.module_name"
verbose_name = _("Module Name")
2 - init.py
code something like
default_app_config = "ocb.users.apps.ModuleConfig"
default_app_config is pointed to your apps.py's class name
in my case, in mysite settings.py , in INSTALLED_APPS array variable I put the name of the app twice by mistake.
I had almost the same issue.
```File "/Users/apples/.local/share/virtualenvs/ecommerce-pOPGWC06/lib/python3.7/site-packages/django/apps/registry.py", line 95, in populate
"duplicates: %s" % app_config.label)
django.core.exceptions.ImproperlyConfigured: Application labels aren't unique, duplicates: auth```
I had installed Django.contrib.auth twice. I removed one and it worked well.
From my experience, this exception was masking the real error. To see the real error (which in my case was an uninstalled python package) comment out the following in django/apps/registry.py:
if app_config.label in self.app_configs:
# raise ImproperlyConfigured(
# "Application labels aren't unique, "
# "duplicates: %s" % app_config.label)
pass
Check for duplicates in INSTALLED_APPS inside the settings.py...If so remove one of it and rerun the command
I had Django==3.2.9 when tried to test my existing Django app on a new environment. I had this exact issue and fixed it by downgrading to Django==3.1.13.
There seems to be an update to applications, check the Django 3.2 documentation for detailed information.
This error occurs because of duplication in your INSTALLED_APPS in settings.py file which is inside your project.
For me, the problem was that I had copy-pasted entire app instead of creating it using command line. So, the app name in the apps.py file was same for 2 apps. After I corrected it, the problem was gone.
In case if you have added your app name in settings.py
example as shown in figure than IN settings.py Remove it and Try this worked for me.
give it a try .
This Worked Because settings.py assumes installing it twice and does not allow for migration
If you want to back older version, command
pip install django==1.6.7
I am trying to use django-debug-toolbar with my django application and it worked for django v1.5. However, I am trying to migrate the system to django v1.6 and it is generating the following error when I try to load any page.
python manage.py runserver
Validating models...
0 errors found
December 21, 2013 - 22:53:18
Django version 1.6.1, using settings 'MySite.settings'
Starting development server at http://XXX.XXX.XXX.XXX:XXXX/
Quit the server with CONTROL-C.
Internal Server Error: /
Traceback (most recent call last):
File "/home/user/django-env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 90, in get_response
response = middleware_method(request)
File "/home/user/django-env/local/lib/python2.7/site-packages/debug_toolbar/middleware.py", line 45, in process_request
mod_path, func_name = func_path.rsplit('.', 1)
AttributeError: 'function' object has no attribute 'rsplit'
[21/Dec/2013 22:53:21] "GET / HTTP/1.1" 500 58172
The source has this note, but I'm not sure what they mean precisely (import_by_path):
def process_request(self, request):
# Decide whether the toolbar is active for this request.
func_path = dt_settings.CONFIG['SHOW_TOOLBAR_CALLBACK']
# Replace this with import_by_path in Django >= 1.6.
mod_path, func_name = func_path.rsplit('.', 1)
show_toolbar = getattr(import_module(mod_path), func_name)
if not show_toolbar(request):
return
Also, while we're at it. Any idea what this message means as well?
/home/user/django-env/local/lib/python2.7/site-packages/debug_toolbar/settings.py:68: DeprecationWarning: SHOW_TOOLBAR_CALLBACK is now a dotted path. Update your DEBUG_TOOLBAR_CONFIG setting.
"DEBUG_TOOLBAR_CONFIG setting.", DeprecationWarning)
My settings.py:
def custom_show_toolbar(request):
return True # Always show toolbar, for example purposes only.
DEBUG_TOOLBAR_CONFIG = {
'INTERCEPT_REDIRECTS': False,
'SHOW_TOOLBAR_CALLBACK': custom_show_toolbar,
'INSERT_BEFORE': 'div',
'ENABLE_STACKTRACES' : True,
}
The problem is described by the message you get (which should not be a deprecation warning, rather it should be a TypeError, that's probably a bug in debug_toolbar):
SHOW_TOOLBAR_CALLBACK is now a dotted path. Update your DEBUG_TOOLBAR_CONFIG setting
The SHOW_TOOLBAR_CALLBACK setting used to be a callable, but now it is a dotted path to a callable: a string. The code you quoted tries to rsplit the value of SHOW_TOOLBAR_CALLBACK, but because you can't rsplit a function object, you get an error.
To solve this, put your callback into another python file, for example your_site/toolbar_stuff.py, and adjust the setting to 'SHOW_TOOLBAR_CALLBACK': 'your_site.toolbar_stuff.custom_show_toolbar'. You can test if this works beforehand by trying in the shell:
from your_site.toolbar_stuff import custom_show_toolbar
If that works, your new setting should work, too.
I'm following the project structure as laid out by Zachary Voase, but I'm struggling with one specific issue.
I'd very much like to have a custom settings boolean variable (let's call it SEND_LIVE_MAIL) that I would be using in the project. Basically, I'd like to use this settings variable in my code and if SEND_LIVE_MAIL is True actually send out a mail, whereas when it is set to False just print its contents out to the console. The latter would apply to the dev environment and when running unittests.
What would be a good way of implementing this? Currently, depending on the environment, the django server uses dev, staging or prd settings, but for custom settings variables I believe these need to be imported 'literally'. In other words, I'd be using in my views something like
from settings.development import SEND_LIVE_MAIL
which of course isn't what I want. I'd like to be able to do something like:
from settings import SEND_LIVE_MAIL
and depending on the environment, the correct value is assigned to the SEND_LIVE_MAIL variable.
Thanks in advance!
You shouldn't be importing directly from your settings files anyways. Use:
>>> from django.conf import settings
>>> settings.SEND_LIVE_MAIL
True
The simplest solution is to have this at the bottom of your settings file:
try:
from local_settings import *
except ImportError:
pass
And in local_settings.py specify all your environment-specific overrides. I generally don't commit this file to version control.
There are more advanced ways of doing it, where you end up with a default settings file and a per-environment override.
This article by David Cramer covers the various approaches, including both of the ones I've mentioned: http://justcramer.com/2011/01/13/settings-in-django/
import os
PROJECT_PATH = os.path.dirname(__file__)
try:
execfile(os.path.join(PROJECT_PATH, local_settings.py'))
except IOError:
pass
Then you can have your local_settings.py behave as if it was pasted directly into your settings.py:
$ cat local_settings.py
INSTALLED_APPS += ['foo']
You can do something like this for a wide variety of environment based settings, but here's an example for just SEND_LIVE_MAIL.
settings_config.py
import re
import socket
class Config:
def __init__(self):
fqdn = socket.getfqdn()
env = re.search(r'(devhost|stagehost|prodhost)', fqdn)
env = env and env.group(1)
env = env or 'devhost'
if env == 'devhost':
self.SEND_LIVE_MAIL = # whatever
elif env == 'stagehost':
self.SEND_LIVE_MAIL = # whatever
elif env == 'prodhost':
self.SEND_LIVE_MAIL = # whatever
config = Config()
settings.py
from settings_config import config
SEND_LIVE_MAIL = config.SEND_LIVE_MAIL