I want to make a periodic task with Django and Celery. I have configured the celery in my project.
The project structure looks like this:
project
├── apps
│ ├── laws
│ └──tasks
│ └──periodic.py # the task is in this file
├── config
│ ├── celery.py
│ ├── settings
│ └── base.py # CELERY_BEAT_SCHEDULE defined in this file
base.py:
CELERY_BEAT_SCHEDULE = {
"sample_task": {
"task": "apps.laws.tasks.periodic.SampleTask", # the problem is in the line
"schedule": crontab(minute="*/1"),
},
}
periodic.py:
class SampleTask(Task):
name="laws.sample_task"
def run(self, operation, *args, **kwargs):
logger.info("The sample task in running...")
Here is the error I get:
The delivery info for this task is:
{'exchange': '', 'routing_key': 'celery'}
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/celery/worker/consumer/consumer.py", line 591, in on_task_received
strategy = strategies[type_]
KeyError: 'apps.laws.tasks.periodic.SampleTask'
How can I define the route of the task correctly?
I solved it.
instead of the route, just put the name of the task in settings:
CELERY_BEAT_SCHEDULE = {
"laws.sample_task": {
"task": "laws.sample_task", # put the name here.
"schedule": crontab(minute="*/1"),
},
}
Related
I have a flask app with the following structure:
/test
├── /application
│ ├── __init__.py
│ ├── routes.py
│ ├── /static
│ ├── /templates
│ └── /dashboard
│ └── __init__.py
├── db.py
├── config.py
├── requirements.txt
├── helpers.py
└── wsgi.py
I am trying to implement a dash dashboard by intitating it through calling a function in application.init.py
where the function is define in dashboard.init.py
my application.init.py
def init_app():
"""Construct core Flask application."""
app = Flask(__name__, instance_relative_config=False)
app.config.from_object('config.Config')
# Ensure responses aren't cached
#app.after_request
def after_request(response):
response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate"
response.headers["Expires"] = 0
response.headers["Pragma"] = "no-cache"
return response
# Custom filter
app.jinja_env.filters["usd"] = usd
Session(app)
assets = Environment()
assets.init_app(app)
with app.app_context():
# Import parts of our core Flask app
from . import routes
from .assets import compile_static_assets
# Import Dash application
from .dashboard import init_dashboard
app = init_dashboard(app)
# Compile static assets
compile_static_assets(assets)
return app
in dashboard.init.py file the function init_dashboard need to access the database and the user id, because the dashboard layout need to use values from the database so that it will show the specific dashboard for the user, but of course this does not work as it return the error "Working outside of request context".
If I will try to create the dashboard in routes for example:
#app.route("/dashapp/")
#login_required
def redirect_to_dashapp():
dash_app = dash.Dash(
server=app,
routes_pathname_prefix="/dashapp/")
"""Create a Plotly Dash dashboard."""
stocks = db.session.query(Records.symbol, Records.number_of_shares, Records.purchase_p, Records.execution_time).filter_by(user_id=session["user_id"]).all()
.....
it will return the error "A setup function was called after the first request was handled. "
As the dash layout need values from the database I can't initiate it in init_dashboard() without accessing the db, and I can't initiate in routes either.
Does anyone has a suggestion on how to properly do it?
i want remove file from storage when a File instance removed.trying to use django signal. here is my model File:
class File(models.Model):
orig_name = models.CharField(_('Original Name'), max_length=255)
conversation = models.ForeignKey('conversation.Conversation', on_delete=models.CASCADE)
the architecture of every app are same.means all of app have own signals directory and in signals's directory we have __init__.py and handlers.py files .here is a brief tree of my app with some detail for file's app:
apps
├── conversation
├── post
└── file
├── admin.py
├── apps.py
├── __init__.py
├── models.py
├── signals
│ ├── handlers.py
│ └── __init__.py
├── tests.py
├── urls.py
└── views.py
here is apps/file/apps.py:
from django.apps import AppConfig
class FileConfig(AppConfig):
name = 'apps.file' # exact as INSTALLED_APP in setting.py
def ready(self):
import apps.file.signals.handers
here is apps/file/signals/handlers.py :
from django.db.models.signals import pre_delete
from django.dispatch import receiver
from apps.file.models import File
#receiver(pre_delete, sender=File)
def remove_file_from_storage(sender, instance, *args, **kwargs):
print('pre_delete signal for File working')
# some code is here
and app/file/signals/__init__.py file is empty. in this project we are using multiple signals and all of them working fine but i don't know why this signal not working.other signal are from custom and build-in signal both. please notice when i moving def remove_file_from_storage function into other app into related app_name/signals/handlers.py it's working fine.the process of deleting is a simple generics.DestroyAPIView of DRF. where is the problem with my architect of using signal?thank you
The problem is because you use receiver decorator to register signals. With that you have to place your code inside ready function of your Django application. This document does mention about that:
In practice, signal handlers are usually defined in a signals submodule of the application they relate to. Signal receivers are connected in the ready() method of your application configuration class. If you’re using the receiver() decorator, simply import the signals submodule inside ready().
But in real practice, we won't put all signals code inside ready method, that might make our code harder to read and harder to maintain. So like you did, we will put the signals inside signals.py and then load it into your app via ready method, like so:
from django.apps import AppConfig
class FileConfig(AppConfig):
label = 'file'
name = 'file'
def ready(self):
from apps.file import signals # NOQA
# add # NOQA to ignore error message from Python linter when we just import but never used
and set this to __init__.py so that Django could discover your app:
apps
├── conversation
├── post
└── file
├── admin.py
├── apps.py
├── __init__.py # Add to this file
├── models.py
├── signals
│ ├── handlers.py
│ └── __init__.py
├── tests.py
├── urls.py
└── views.py
default_app_config = 'apps.file.apps.FileConfig'
You could also take a look at this article to gain more details about Django signals
Hope that helps!
Everyone.
I'm a newbie in this field. I develops web application with google app engine using django framework. I have a troubleshot about python lib dir problem... ImportError: no module named...
my appengine_config.py file is
# [START vendor]
from google.appengine.ext import vendor
vendor.add('lib') # I believes this line is to add 'lib' folder to PATH.
# vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib')) # <-- and I tried too this line.
# [END vendor]
my 'requirements.txt' file is
MySQL-python==1.2.5 #app engine django project default
Django==1.11.3 #app engine django project default
django-twilio # add i want
twilio # add i want
and I installed using pip install -t lib -r requirements.txt
ROOT
├── lib
│ ├── django
│ ├── pytz
│ ├── wanttousing_lib
│ └── ...
├── mysite
│ ├── __init__.py
│ ├── settings.py
│ ├── controllers.py
│ ├── models.py
│ ├── views.py
│ ├── templates
│ └── ....
├── test
│ ├── like
│ │ ├── models_tests.py
│ │ └── controllers_tests.py
│ └── ....
├── static
│ ├── css
│ └── js
├── app.yaml
├── manage.py
├── appengine_config.py
├── requirement-vendor.txt
└── requirements.txt
so, I installed in my project... but..compiled error.
from wanttousing_lib import example_module
importError wanttousing_lib..........
however, if I move my wanttousing_lib to ROOT dir, it works.....
ROOT
├── lib
│ ├── django
│ ├── pytz
│
│ └── ...
├── mysite
│ ├── __init__.py
│ ├── settings.py
│ ├── controllers.py
│ ├── models.py
│ ├── views.py
│ ├── templates
│ │ └── like
│ │ ├── index.html
│ │ └── _likehelpers.html
│ └── ....
├── test
│ ├── like
│ │ ├── models_tests.py
│ │ └── controllers_tests.py
│ └── ....
├── static
│ ├── css
│ └── js
├── app.yaml
├── manage.py
├── appengine_config.py
├── requirement-vendor.txt
├── requirements.txt
└── wanttousing_lib <--- moved
--> All traceback.
Unhandled exception in thread started by <function wrapper at 0x103e0eaa0>
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/autoreload.py", line 227, in wrapper
fn(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/commands/runserver.py", line 125, in inner_run
self.check(display_num_errors=True)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/base.py", line 359, in check
include_deployment_checks=include_deployment_checks,
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/base.py", line 346, in _run_checks
return checks.run_checks(**kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/checks/registry.py", line 81, in run_checks
new_errors = check(app_configs=app_configs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/checks/urls.py", line 16, in check_url_config
return check_resolver(resolver)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/checks/urls.py", line 26, in check_resolver
return check_method()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/urls/resolvers.py", line 254, in check
for pattern in self.url_patterns:
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/functional.py", line 35, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/urls/resolvers.py", line 405, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/functional.py", line 35, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/urls/resolvers.py", line 398, in urlconf_module
return import_module(self.urlconf_name)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "ROOT/mysite/urls.py", line 19, in <module>
from polls.views import index
File "ROOT/polls/views.py", line 17, in <module>
from sms_twilio.tests import send_sms_test
File "ROOT/sms_twilio/tests.py", line 13, in <module>
from twilio import twiml
ImportError: No module named twilio
ERROR SOURCE:
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.test import TestCase
# Create your tests here.
from django.conf import settings
# file: your_code.py
# import twilio # no need for 'from lib import twilio'
# do stuff with twilio...
from twilio import twiml
from twilio.rest import Client
def send_twilio_message(to_number, body):
client = Client(
#client = twilio.rest.TwilioRestClient(
settings.TWILIO_ACCOUNT_SID, settings.TWILIO_AUTH_TOKEN)
return client.messages.create(
body=body,
to=to_number,
from_=settings.TWILIO_PHONE_NUMBER
)
def send_sms_test():
client = Client(
#client = twilio.rest.TwilioRestClient(
settings.TWILIO_ACCOUNT_SID, settings.TWILIO_AUTH_TOKEN)
return client.messages.create(
body="[TEST] SEND SMS !! HELLO !!",
to="TO_SENDER",
from_=settings.TWILIO_PHONE_NUMBER
)
perhaps, Do I add library list to app.yaml ?
like
libraries:
- name: MySQLdb
version: 1.2.5
- name: twilio <-- like this
version: -
requirement-vendor.txt file is
Django==1.11.3
how can i fix it? please help...
I had a similar issue a while back, and instead of using vendor.add('lib'), I had success doing this:
vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))
I found that solution here in the docs for using third-party libraries on GAE, in step 4:
To copy a library into your project:
Create a directory to store your third-party libraries, such as lib/.
mkdir lib
Use pip (version 6 or later) with the -t flag to copy the libraries into the folder you created in the previous step. For example:
pip install -t lib/ <library_name>
(Using Homebrew Python on Mac OS X?)
Create a file named appengine_config.py in the same folder as your app.yaml file.
Edit the appengine_config.py file and provide your library directory to the vendor.add() method.
# appengine_config.py
from google.appengine.ext import vendor
# Add any libraries install in the "lib" folder.
vendor.add('lib')
The appengine_config.py file above assumes that the current working directory is where the lib folder is located. In some cases, such as unit tests, the current working directory can be different. To avoid errors, you can explicity pass in the full path to the lib folder using:
vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))
My python lib's dir are two dir.
1) /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/...
2) /usr/local/lib/python2.7/...
My project points at 1), but pip install at 2)...
I tried at 1) ./pip install twilio. and so, it works!
thanks.
Project setting
Python 3.5.3
flask 0.12.2
Directory
.
├── Core
│ ├── BackgroundProcessManager.py
│ ├── FirebaseDatabaseManager.py
│ ├── LearningManager.py
│ ├── __init__.py
│
├── FrontEnd
│ ├── HomePage.py
│ ├── __init__.py
│
├── LICENSE
├── README.md
├── Route
│ ├── RouteManager.py
│ ├── __init__.py
│
├── Settings
│ ├── DefineManager.py
│ ├── __init__.py
│
├── Utils
│ ├── LoggingManager.py
│ ├── __init__.py
│
├── index.py
└── runner.sh
Expected behavior
All the route link is in Route/RouteManager.py
Flask main source is in index.py
I want send fake request and test response using pytest-flask.
Actual behavior
I don't know how to send fake post data and testing.
Source
index.py
from flask import Flask
from Settings import DefineManager
from Route import *
import imp
import sys
imp.reload(sys)
app = Flask(__name__)
app.register_blueprint(routes)
if __name__ == '__main__':
app.debug = True
app.run(host=DefineManager.SERVER_USING_HOST, port=DefineManager.SERVER_USING_PORT)
Route/RouteManager.py
#routes.route("/")
def IndexPage():
LoggingManager.PrintLogMessage("RouteManager", "IndexPage", "web page connection!", DefineManager.LOG_LEVEL_INFO)
return HomePage.RenderIndexPage()
#routes.route("/upload/", methods=['POST'])
def UploadRawDatas():
content = request.get_json(silent=True)
LoggingManager.PrintLogMessage("RouteManager", "UploadRawDatas", "json data: " + str(content), DefineManager.LOG_LEVEL_INFO)
return BackgroundProcessManager.UploadRawDatas(content['Data'], content['Date'], content['Day'])
#routes.route("/forecast/", methods=['POST'])
def ForecastDatas():
content = request.get_json(silent=True)
LoggingManager.PrintLogMessage("RouteManager", "ForecastDatas", "json data: " + str(content), DefineManager.LOG_LEVEL_INFO)
return BackgroundProcessManager.ForecastDatas(content['ProcessId'])
Test case
/upload/
Request data
header
Content-Type application/json
body
{
"Data": [20.0, 30.0, 401.0, 50.0],
"Date": ["2017-08-11", "2017-08-12", "2017-08-13", "2017-08-14"],
"Day": 4
}
Response data
header
Content-Type application/json
body
{"Result": 39}
Expect test process
Send fake post data.
Receive response data.
Assert check Result is not equal -1
pytest-flask provide several fixtures, including a client one. With it you can make a test function similar to this:
def test_upload(client):
mimetype = 'application/json'
headers = {
'Content-Type': mimetype,
'Accept': mimetype
}
data = {
'Data': [20.0, 30.0, 401.0, 50.0],
'Date': ['2017-08-11', '2017-08-12', '2017-08-13', '2017-08-14'],
'Day': 4
}
url = '/upload/'
response = client.post(url, data=json.dumps(data), headers=headers)
assert response.content_type == mimetype
assert response.json['Result'] == 39
While the accepted answer works, it can be simplified by passing the kwarg json into the post method instead of data
def test_upload(client):
data = {
'Data': [20.0, 30.0, 401.0, 50.0],
'Date': ['2017-08-11', '2017-08-12', '2017-08-13', '2017-08-14'],
'Day': 4
}
url = '/upload/'
response = client.post(url, json=data)
assert response.content_type == 'application/json'
assert response.json['Result'] == 39
I am using django 1.5.1 and trying to get moving the management command. I have an app called patient and here is the directory structure.
patient/
├── __init__.py
├── forms.py
├── management
│ ├── __init.py__
│ └── commands
│ ├── __init.py__
│ └── recall1.py
├── models.py
├── urls.py
├── views.py
Here is what happens when I try to run the recall1 command:
$ python manage.py recall1
Unknown command: 'recall1'
Here is how my code looks like:
from django.core.management.base import BaseCommand, CommandError
from recall.model import PatientRecalls, RecallType, RecallMessage
from patient.model import Patient
class Command(BaseCommand):
args = '<patient_id patient_id ...>'
def handle(self, *args, **options):
for patient_id in args:
try:
patient = Patient.objects.get(pk=int(patient_id))
except Patient.DoesNotExist:
raise CommandError('Patient "%s" does not exist' % patient_id)
patient.opened = False
patient.save()
self.stdout.write('Successfully closed patient "%s"' % patient_id)
What wrong do I need to correct before I can run this thing? The app is running great, except for this issue..
Your filename __init.py__ should actually be __init__.py both in the directories management/ and commands/