I'm in the process of writing a scheduling application on a shared hosting server. So far it doesn't look like my scheduling will have to be any more granular than daily, but that could change later. Ideally, I'll want to use a Windows host so I write my application in .NET but I'm open to using a PHP+*nix host.
In a shared hosting environment (read: cheap) how can I write an application where notifications can occur on a scheduled basis?
On a *nix system, you're going to want to use cron. An example of its use, starting a process every day at 3:20 AM:
echo "20 3 * * * /path/to/run-my-job.sh" | crontab
edit: To answer the question asked in a comment on another response, it depends on the shared host; many of them allow you to run cron jobs.
Are you wanting something like a cron job?
Steve
Related
I am sorry if its basics but I did not find any answers on the Internet comparing these two technologies. How should I decide when to use which as both can be used to schedule and process periodic tasks.
This is what an article says:
Django-celery :
Jobs are essential part of any application that does
some processing for you in the background. If your job is real time
Django application celery can be used.
Django-cronjobs :
django-cronjobs can be used to schedule periodic_task which is a
valid job. django-cronjobs is a simple Django app that runs registered
cron jobs via a management command.
can anyone explain me the difference between when should I choose which one and Why? Also I need to know why celery is used when the computing is distributed and why not cron jobs
The two things can be used for the same goal (background execution). However, if you are going to choose wisely, you should really understand that they are actually completely different things.
Here's what I wish someone had told me back when I was a noob (instead of the novice level that I have achieved today :)).
cron
The concept of a cron job is that we want a command / process to be executed on some schedule. Furthermore, we want that process to receive x,y,z parameters, run with a,b,c environment variables, and as user id 123.
Some cron systems may facilitate a few extra features, such as:
catching up on missed tasks (e.g. the server was off for a power outage all night and as soon as we turn it on, it runs the 8 instances of the command we normally run hourly).
might help you with the type of locking you normally do using a pid file in order to avoid parallel runs of the same command.
For the most part, cron systems are meant to be dumb: "just run this command at this time, thanks!".
Celery
The concept of Celery is much more sophisticated. It works with tasks, chains & chords of tasks, error handling, and (in most cases) collection of work result. It has a queue (or many queues) of work and a worker (or many). When a task (really just a message describing requested work) enters the queue it waits there until a worker is available to handle it. Much the same way as 1 or more employees at the DMV service a room full of waiting customers.
Furthermore, Celery can facilitate distributed work. That's a bit like (if I may torture the analogy a bit) - the difference between a DMV office where every worker shares the same phone, computer, copier, etc and a DMV where workers have dedicated resources and are never blocked by other workers.
Celery for web apps
In web applications, Celery is often used when a bit of web access results in a thing to be done that should be handled out of band of the conversation with the web browser. For example:
the web user just did something which should result in an email being sent. In order to send an email, your web server will need to contact a mail server. This could take time, the server could be busy, etc - we cant make the web user just wait, seeing nothing on their browser while we do this. Well, you can but it won't work reliably. So, we do that email send as a bit of work in the queue. That way, it can happen "whenever" and the web server can get back to communicating with the browser.
the user just submitted a credit card as payment. You're going to need to contact the card processor, but that might take several seconds. You might even have to contact them multiple times (e.g. they are really busy there right now). Again, you don't want your user's web browser to just sit blankly and you don't want a web server process or thread of execution tied up. Instead, you use Celery to create a job, you tell the browser to check back in a few seconds (or use a "web socket"), and your web server moves on and talks to other web users. When the browser checks back later, you lookup the task id and find out from celery whether it is finished and what the outcome was (card declined, etc).
Using Celery as cron
When you use Celery as a "cron system" all you are really doing is saying: "hey, can someone please generate work of X type on Y schedule". A process is created that runs continuously which sleeps most of the time and wakes up occasionally to inject a bit of work into the queue on the schedule you requested.
Usually the "hey someone" that you ask to do that for you is: celery beat and beat gets the schedule you want from somewhere in the database or from your settings file.
I searched for celery vs cron and found a few results that might be helpful to you.
https://www.reddit.com/r/Python/comments/m2dg8/explain_like_im_five_why_or_why_not_would_celery/
Why would running scheduled tasks with Celery be preferable over crontab?
Distributed task queues (Ex. Celery) vs crontab scripts
We have several ColdFusion 2016 instances in production. We need the capability to remotely stop/start them as needed from a central dashboard. Can FusionReactor handle this? I wasn't able to find anything in the docs.
FusionReactor can execute a script in order to restart a service (such as CF 2016). However, the trigger mechanism for FR doing this is based upon a monitored server (instance) changing state e.g. becoming non-responsive or going down. In this case, FR would recognize this change of state and trigger a script. This may or may not solve your problem - as you wrote that you would like to remotely start/stop the server as needed! You can read about FR Enterprise Scripting here - http://docs.intergral.com/display/FR62/Using+FusionReactor+Enterprise+Scripting
I want to make a tool that runs on win32 to monitor our online game servers. The servers actually are .exe files. I need to know whether they have crashed so I can restart them. Therefore, the tool will have 2 main features:
Frequently check a list of the server programs to see whether they are running or not
Reopen the executable of any server that has crashed
Does anyone have any idea or knows an API to start with?
If you're in Win32, you can start out with C# using System.Diagnostics
using System.Diagnostics;
Then get a process list:
Process[] processlist = Process.GetProcesses();
foreach(Process theprocess in processlist){
Console.WriteLine(“Process: {0} ID: {1}”, theprocess.ProcessName, theprocess.Id);
}
And from there it's up to you what you want to do with the info.
Game servers should produce logs. You should make a service / cron job to monitor the logs. Depends on the contents of logs, your service should act respective actions, e.g. restart service, trigger alarms, etc.
I realize that this is not exactly what you asked for, but what about doing the monitoring with a fully-fledged monitoring tool such as Nagios? You would of course have to "teach" the monitoring tool about the processes that shall be monitored but you would also profit from its more advanced functions. In the case of Nagios, for example, these would comprise automatic e-mail notifications, an online dashboard of process status, notifications via SMS etc.
You should start the game servers using CreateProcess or similar in the win32 api.
You will get back a process handle in the lpProcessInformation parameter.
You can use WaitForSingleObject to WaitForMultipleObjects to wait until that handle is notified which will happen when the process terminates for any reason.
My goal is to create an application that will be able to do long-lasting mainly system tasks, such as:
checking out code from the repositories,
copying directories between various localizations,
etc.
The problem is I need to prepare it somehow independently from the web browser. I mean that for example after starting the checkout/copy action, closing the web browser will not interrupt the action. So after going back to that site I can see that the copying goes on or another action started when the browser was closed...
I was searching through various tools, like RabbitMQ + Celery, Twisted, Pyro, XML-RPC but I don't know if any of these will be suitable for me. Has anyone encountered similar needs when creating Django app? Please let me know if there are any methods/packages that I should know. Code samples also will be more than welcome!
Thank you in advance for your suggestions!
(And sorry for my bad English. I'm working on it.)
Basically you need to have a process that runs outside of the request. The absolute simplest way to do this (on a Unix-like operating system, at least) is to fork():
if os.fork() == 0:
do_long_thing()
sys.exit(0)
… continue with request …
This has some downsides, though (ex, if the server crashes, the “long thing” will be lost)… Which is where, ex, Celery can come in handy. It will keep track of the jobs that need to be done, the results of jobs (success/failure/whatever) and make it easy to run the jobs on other machines.
Using Celery with a Redis backend (see Kombu's Redis transport) is very simple, so I would recommend looking there first.
You might need to have a process outside the request / response cycle. If that is the case, Celery with a Redis backend is what I would suggest looking into, as that integrates nicely with Django (as David Wolever suggested).
Another option is to create Django management commands, and then use cron to execute them at scheduled intervals.
I have a web service which I would like to call on a scheduled basis. Currently, I have a scheduled task setup to execute the call. Here's a link Jon Galloway's post on the topic.
I'm planning to switch this out with a Win Service, simply to appease my boss, but I'm wondering what the advantages are (aside from keeping my job ;).
I personally like the script method due to its simplicity, but am curious to know what I'm missing over a Win Service approach..
Reliability is higher with a Windows Service. It can be set to start when the system starts, and to restart automatically if it fails.