I have a flask web app deployed in heroku. I need to schedule a background task to be scheduled at a specific time. I have tried using the apscheduler module. While it allows to define periodic tasks easily adding them from your application at runtime is what I am looking for.
I tried sharing the same jobstores in apscheduler
import time
from apscheduler.scheduler import Scheduler
from apscheduler.jobstores.shelve_store import ShelveJobStore
sched = Scheduler()
sched.add_jobstore(ShelveJobStore('jobstore.db'), 'shelve')
sched.start()
And from terminal I tried this,
Python 2.7.5 (default, May 12 2013, 12:00:47)
[GCC 4.8.0 20130502 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from apscheduler.scheduler import Scheduler
>>> sc = Scheduler()
>>> sc.add_jobstore('jobstore.db', 'shelve')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/dhananjay/git/blast/venv/lib/python2.7/site-packages/apscheduler/scheduler.py", line 168, in add_jobstore
jobstore.load_jobs()
AttributeError: 'str' object has no attribute 'load_jobs'
I have come across this question, while looking for a celery based approach. It talks about the same problem from a django perspective but I can't get it to work with my app (I am completely oblivious to django)
When you tried running it from the terminal, you gave add_jobstore a string as the first parameter, instead of a job store. It expects a job store as the first parameter, see the documentation for more info.
As for scheduling background tasks in Heroku, I would recommend reading the Worker Dynos, Background Jobs and Queueing article on the matter.
Related
I am trying to set up geoip2 for GeoDjango as per the instructions.
For some reason the wrapper isn't importing the function. It worked before I downloaded the databases and pointed to them in my settings, but for some reason now I can't load GeoIP2 (even when I comment out the line in settings.py). How should I troubleshoot this?
Python 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 17:26:49) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import geoip2
>>>
>>> from django.contrib.gis.geoip2 import GeoIP2
Traceback (most recent call last):
File "<console>", line 1, in <module>
ImportError: cannot import name 'GeoIP2'
>>>
EDIT: I can see the GeoIP2 function correctly listed in the source file (which I haven't modified). What could possibly be preventing it from loading?
I am running Django 1.11.4
If I import django.contrib.gis.geoip2 this is it's __path__ property:
>>> geoip2.__path__
['C:\\Users\\Adam\\Envs\\otherlane\\lib\\site-packages\\django\\contrib\\gis\\geoip2']
I fixed by installing it through pip package geoip2==2.9.0
pip install geoip2==2.9.0
This module is Deprecated since version 1.9 in favor of django.contrib.gis.geoip2, which supports IPv6 and the GeoLite2 database format.
If you have a django < 1.9, use instead
from django.contrib.gis.geoip import GeoIP
just run the command pip install geoip2 then it will work fine...
Double check your GEOIP_PATH. And my I remind you that Windows requires back slashes not forward slashes.
I was also facing the same error and couldn't resolve it after multiple attempts.
Since it was working on one of my system, I found one difference, When you install geoip2 via pip it also installs maxminddb. The system on which it was working, maxminddb version was 1.5.4 and the one on which it was not working it was maxminddb==2.0.0
so finally I did
pip install maxminddb==1.5.4
and it worked
You can handle it this way
from geoip2 import geolite2
I am running one software in ubuntu 12.04. It requires python. I have python 2.7.3 on ubuntu. When try to run command ./cmdsadit sim -c ./Example/SimExample.py it shows following
jayram#ubuntu:~/sadit$ ./cmdsadit sim -c ./Example/SimExample.py
[warning] socketIO_client is not found. Real-time ability is disabled
--> Use [matplotlib] as plot backend
--> [warning] cannot import sql related function, reading for sql server is not supported
WARNING:util:Cannot find [socketIO_client], real-time ability is disabled.
WARNING:util:Cannot import sql related function, reading for sql server is not supported
Traceback (most recent call last):
File "fs.py", line 10, in <module>
from networkx.algorithms.traversal.path import single_source_dijkstra_path, dijkstra_path_length
ImportError: No module named path
jayram#ubuntu:~/sadit$
Some initial line screenshot of fy.py file is given below
How do I solve this error?
The error is self explanatory, your import from path module is incorrect
It should be
from networkx.algorithms import single_source_dijkstra_path, dijkstra_path_length
I have been running into this problem for a short while now and simply can't find a solution anywhere. I am using Google App Engine to run a default Python 2.7 app with Django 1.5 (via GAE SDK) created through PyCharm. I can upload the app successfully, but upon visiting the actual page, I get a Server Error. Then, checking the logs in Google App Engine, I see this:
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
After searching the internet for a while, I was able to find a few posts which address this issue, but attempting them never seemed to solve my problem. For example: This problem was solved by replacing "application" with "app" in the following lines:
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
In fact, I had run into this same issue before and this solution provided a fix for me in the past, however at that time I was running a separate app and it was not through the GAE.
I checked the Django documentation for version 1.5 here, but the code and suggestions there don't seem to conflict with what I currently have in my project.
I read a bit more about this type of problem, and saw another post that suggested checking the app's wsgi.py file to ensure that it is named 'application' or 'app' respectively, so that one could then use that same name throughout the rest of the application. However, upon checking those settings I saw that 'application' was used there too:
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
There's even a line in settings.py which uses the same nomenclature to declare the WSGI application:
WSGI_APPLICATION = 'Chimera.wsgi.application'
I'm really having trouble debugging this. I get the feeling it's really dumb and I just can't see it, but unfortunately I'm not particularly good at this kind of stuff -- I'm still a bit of a novice in this field.
Does anyone have any idea what I could try in an attempt to fix this issue?
UPDATE: I started making line by line changes and testing things, and eventually I found that the GAE log changes depending on the input for the "script" under app.yaml. So if I change the script under "handlers" between "main.app" and "main.application", it adjusts the log output to refer to "app" or "application" respectively. So that line in the app.yaml file tells the app what to look for, but I'm still not seeing why it can't be found. Not sure what else I could change to test it out. I wish I knew a bit more about the actual inner workings so that I could figure out why the app is confused about the attribute. Is it trying to run before it even gets instantiated or something?
Source code below:
main.py
import os, sys
os.environ['DJANGO_SETTINGS_MODULE'] = 'Chimera.settings'
from google.appengine.ext.webapp import util
from django.conf import settings
settings._target = None
import django.core.handlers.wsgi
import django.core.signals
import django.db
import django.dispatch.dispatcher
def main():
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
app.yaml
application: eloquent-ratio-109701
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.application
libraries:
- name: django
version: 1.5
wsgi.py
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "Chimera.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Full log from GAE:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 302, in _LoadHandler
raise err
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
Thanks for helping me out.
In your main.py file (i.e. the main module) application is a variable inside the main() function, not an attribute of the main module. Basically you don't need a main() function.
GAE has some specific support for using Django, I'd strongly suggest going through the Django Support documentation and the Django App example.
Based on the comment made by #DanielRoseman I discovered that declaring the app inside of the main() function caused an issue because the app attribute was then only accessible at the main() function level, as it was a member variable of main() as opposed to a global variable. Although the default application files were structured this way by PyCharm, it seems that it was incorrect. I'm not sure if this is a compatibility issue, but regardless, moving the app declaration outside of the main() function adjusts the scope in a way which allows for other parts of the project to access it, solving my problem.
Thank you #DanielRoseman for the comment.
It is working fine under Django's runserver with the monkey patch:
if __name__ == "__main__":
import gevent
from gevent import monkey
monkey.patch_all()
execute_manager(settings)
However, in production we are using Apache with mod_wsgi, and a wsgi file. Putting the above in the WSGI file has no effect. It seem that when the wsgi file is called, it is not as __main__, but removing the if also does nothing.
I found gevent.wsgi.WSGIHandler() and tried to replace django.core.handlers.wsgi with it, but it requires request and application as parameters, which I don't have in my wsgi file.
This is what my wsgi file looks like:
import os,sys
import django.core.handlers.wsgi
from gevent import wsgi
sys.path.append('/app/src')
sys.path.append('/app/src/webInterface')
os.environ['DJANGO_SETTINGS_MODULE'] = 'WebInterface.settings'
#application = django.core.handlers.wsgi.WSGIHandler()
application = wsgi.WSGIHandler()
You are correct that __name__ is not '__main__' in mod_wsgi. Even without the if(), where in the WSGI script file did you place the monkey patch call? You don't show that in what the WSGI script file looks like.
Overall, using gevent monkey patching in mod_wsgi is probably a bad idea anyway. This is because using gevent usually gives people a false sense of security that they no longer have to deal with thread locking because greenlets will to some degree order execution so for simple stuff it isn't needed. It is most definitely a bad idea to rely on that under mod_wsgi because all the request handler threads will still be real threads and not greenlets because the threads are created as external threads using Apache thread APIs. Thus very much still need to handle thread locking properly.
One last thing. You may want to add to your question what you are trying to achieve in doing this, because your attempts at replacing the application with the WSGIHandler from gevent make no sense.
I am trying to create a dev environment for the Django 1.4 project using the following guide:
http://www.stereoplex.com/blog/a-django-development-environment-with-zc-buildout
virtualenv part of the guide runs ok with the following output:
virtualenv project
New python executable in project\Scripts\python.exe
Installing setuptools................done.
Installing pip...................done.
After that I am able to activate dev environment. Now I create directory named Source, download the bootstrap.py to it and create a buildout.cfg with the following content:
[buildout]
parts =
And run bootstrap.py for the following result:
Creating directory 'C:\\Dropbox\\XYZ\\project\\Source\\bin'.
Creating directory 'C:\\Dropbox\\XYZ\\project\\Source\\parts'.
Creating directory 'C:\\Dropbox\\XYZ\\project\\Source\\eggs'.
Creating directory 'C:\\Dropbox\\XYZ\\project\\Source\\develop-eggs'.
Generated script 'C:\\Dropbox\\XYZ\\project\\Source\\bin\\buildout'.
Here comes the problem part - Installing Django I configure the buildout.cfg to the following and run bin\buildout created by bootstrap:
[buildout]
parts = django
[django]
recipe = djangorecipe
version = 1.4
After running bin\buildout i get the following error:
(project) C:\Dropbox\XYZ\project\Source>bin\buildout.exe
Traceback (most recent call last):
File "C:\Dropbox\XYZ\project\Source\bin\buildout-script.py", line 15, in <module> import site # imports custom buildout-generated site.py
File "C:\Dropbox\XYZ\project\Source\parts\buildout\site.py", line 601, in <module> main()
File "C:\Dropbox\XYZ\project\Source\parts\buildout\site.py", line 584, in main known_paths = addsitepackages(known_paths)
File "C:\Dropbox\XYZ\project\Source\parts\buildout\site.py", line 328, in addsitepackages import pkg_resources
ImportError: No module named pkg_resources
Although if I run python directly in project environment I can import pkg_resources with no error:
(project) C:\Dropbox\XYZ\project\Source>python
Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
>>> pkg_resources
<module 'pkg_resources' from 'C:\Dropbox\XYZ\project\lib\site-packages\setuptools-0.6c11-py2.7.egg\pkg_resources.py'>
I am completely struck here. Any suggestions?
I don't have the definitive answer, but here are some brainstorm thoughts:
Why the virtualenv? Buildout itself provides isolation, so no virtualenv is needed. Could you re-try with just a bootstrap.py and your buildout.cfg? So just run bootstrap.py with your system python?
Do you have buildout installed globally, perhaps? They can interfere.
The latest 1.5.2 buildout has some problems with site.py files in some situations, which is a possible reason for it failing inside a virtualenv. Could you try the special 1.4.4 bootstrap mentioned in http://pypi.python.org/pypi/zc.buildout/1.5.2#system-python-and-zc-buildout-1-5 ?
bin/buildout -vvv gives you much more debugging info.
An additional comment: the version setting in djangorecipe is deprecated in the latest versions of djangorecipe. You can remove it. If you want to pin Django you have to pin it in your buildout's [version] list.
The guide you are following is a little bit obsolete. It will fail when processing the [django] part, specifically in the version variable. You must specify the versions in the new way, which is showed in the djangorecipe page. This is:
[buildout]
parts = django
versions = versions
[versions]
django = 1.4