I want to setup cronjob in my Django project, but I am using windows.
I tried django-cron, but it is not working with windows.
How can I use cronjob scheduling in my project?
Using django-cron is not the case and cannot be used on Windows, because Windows do not support cron job scheduling. You can use Windows analog of the Unix cron command called "schtasks" to schedule execution of your script or Windows Task Scheduler.
See more in SO Questions What is the Windows version of cron? and Schedule Python Script - Windows 7
Related
I'm trying to automate deploying code to my 3 GCE Linux VM's. I read this article Scripting with gcloud: a beginner’s guide to automating GCP tasks, it shows how to make a script. Now I assume that means saving the code as a .sh file (it even has a shebang on top), now how do I run that. Do I type the script file name in the Google Cloud SDK Shell? I tried it, it does not seem to work. can someone help me? I will really appreciate.
Here is an image of my google cloud shell where I am trying to use the script files.
You're able to install Google Cloud SDK on variety of operation systems such as Linux, macOS and Windows. After that, you'll be able to use same commands like gcloud, gsutil and bq. Meanwhile, scripting relies on the command-line interpreters: you can use bash with Linux and macOS, but for Windows you should use cmd and PowerShell. You can run examples provided at the article, you've mentioned, and at the documentation Scripting gcloud CLI commands with bash on Linux and macOS, so the error messages you've got were expected. You can't run .sh scripts on windows naively, as it was mentioned by #Pievis at the comment section.
As a possible workaround you can install Windows Subsystem for Linux (WSL) for Windows 10 (usually you can choose between WSL2 and WSL1, but it depends on build version of your Windows 10) to get some interoperability between Windows and Linux.
If you need to transfer files to you VM instances please follow the documentation Transferring files to VMs.
If you are interested in automation with GCP, please have a look on the documentation Infrastructure as code to "automate repeatable tasks like provisioning, configuration, and deployments".
I want to run my bat file on windows with the help of GCP Composer, but i am not sure if we can communicate with windows machine as composer is fully based on linux environment. Please help me if you have any solution.
There are a couple possible solutions described in this thread, basically:
Installing ssh into your Windows machine and then connecting to run commands remotely using the Airflow’s ssh operator.
Install a package like pywinrm, which allows you to run Windows commands on a target machine from Python code. Then, use the Python operator, within your DAG, to make the call. You may refer to the GCP documentation for steps on installing additional Python packages in Composer.
I am working on my graduation project and It's my first time dealing with spark and EC2
so I am following the steps in this blog
http://www.supergloo.com/fieldnotes/apache-spark-cluster-amazon-ec2-tutorial/#comment-3843
The problem is he is working on MAC and I don't know how to make these commands work on Windows (CMD).
for example this command
ec2/spark-ec2 –key-pair=courseexample –identity-file=courseexample.pem launch spark-cluster-example
Any Help ?!
try running it in the following way (from the same folder):
python -Wdefault "ec2\spark_ec2.py" –key-pair=courseexample –identity-file=courseexample.pem launch spark-cluster-example
if you don't know how to open console in windows - just press "Start->Run", type cmd and hit enter, then you would need to havigate to your Spark home folder, and execute the above command.
NOTE: I don't currently own a Windows machine, so I haven't tried this command myslef.
http://blog.cloudera.com/blog/2014/04/how-to-use-cron-like-scheduling-in-apache-oozie/
I referred this link which shows us how to schedule jobs using cron like syntax. But it shows scheduling in CDH4 and CDH5. Can anyone tell me how this can be achieved in CDH3? We have a CDH3 cluster.
Unfortunately the 'cron-like' scheduling functionality wasn't included until Oozie 4.1.0. https://issues.apache.org/jira/browse/OOZIE-1306. CDH3 is only running a 2.X.X release I believe.
However, you may be in some luck if you're willing to compile Oozie by hand an install / update your current version. Here are the only requirements for Oozie 4.1.0 :
Unix box (tested on Mac OS X and Linux)
Java JDK 1.6+
Maven 3.0.1+
Hadoop 0.20.2+
Pig 0.7+
All of which are covered in CDH3. https://oozie.apache.org/docs/4.1.0/DG_QuickStart.html
I'm a complete beginner on spark. I'm trying to run spark on Amazon EC2, but my system does not recognize "spark-ec2" or "./spark-ec2". It says "spark-ec2" is not recognized as an internal or external command.
I followed the instruction here to launch a cluster. I would like to use Scala, how do I make it work?
Add PYTHON PATH environment variable with boto.
PYTHONPATH="${SPARK_EC2_DIR}/third_party/boto-2.4.1.zip/boto-2.4.1:$PYTHONPATH"
And execute the python script
In order to run the Spark-EC2 script on Windows you need Cygwin and Python. If you don't want to install these programs, you can use the dockerized version of the script (https://github.com/edrevo/spark-ec2-docker), which only depends on Docker.