I want to set a cron that fetches some stories from an api every 5 minutes and shoot a mail if any new stories comes up. Here is my crontab file. (Used a django management command to do so). I fired the management command and its sending me the correct info, but when I am trying to set a cronjob for the same, its not firing. Here is my crontab file
vi /etc/crontab
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# m h dom mon dow user command
17 * * * * root cd / && run-parts --report /etc/cron.hourly
25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
5 * * * * root /home/anurag/projects/virtualenvs/django6/bin/python /home/anurag/Documents/HNStories/hnstories/manage.py get_stories >> /home/anurag/cron_log.txt
Here are its permission
ls -l /etc/crontab
-rw-r--r-- 1 root root 884 Aug 17 20:20 /etc/crontab
Also I am not able to see any warning and error in syslog file
cat /var/log/syslog | grep crontab
Aug 17 12:58:01 anurag cron[1257]: (*system*) RELOAD (/etc/crontab)
Aug 17 17:24:01 anurag cron[8534]: (*system*) RELOAD (/etc/crontab)
Aug 17 20:21:01 anurag cron[1139]: (*system*) RELOAD (/etc/crontab)
I also tried to restart the crontab and restart my computer. But I am not able to fix this issue.
The correct syntax for executing every 5 minutes would be
*/5 * * * * root /home/anurag/projects/virtualenvs/django6/bin/python /home/anurag/Documents/HNStories/hnstories/manage.py get_stories >> /home/anurag/cron_log.txt
Another reason why the command isn't executing might be a missing newline at the end of /etc/crontab
EDIT:
You might also want to look into django-extensions which provides a command extension (runjobs) to run scheduled jobs.
Related
In a Dockerfile, the common way to copy a directory as a non-root user (e.g $UID 1000) is the following:
COPY --chown=1000:1000 /path/to/host/dir/ /path/to/container/dir
However, I want to use variables instead. For example
ARG USER_ID=1000
ARG GROUP_ID=1000
COPY --chown=${USER_ID}:${GROUP_ID} /path/to/host/dir/ /path/to/container/dir
But this is not possible. There exist a workaround?
Note I know that a possible workaround could be to copy the directory as root and then run chown on the directory (variables works fine with RUN). However, the size of the image will grow just for the use of chown in a separate command.
You can create a user before running the --chown;
mkdir -p test && cd test
mkdir -p path/to/host/dir/
touch path/to/host/dir/myfile
Create your Dockerfile:
FROM busybox
ARG USER_ID=1000
ARG GROUP_ID=1000
RUN addgroup -g ${GROUP_ID} mygroup \
&& adduser -D myuser -u ${USER_ID} -g myuser -G mygroup -s /bin/sh -h /
COPY --chown=myuser:mygroup /path/to/host/dir/ /path/to/container/dir
Build the image
docker build -t example .
Or build it with a custom UID/GID:
docker build -t example --build-arg USER_ID=1234 --build-arg GROUP_ID=2345 .
And verify that the file was chown'ed
docker run --rm example ls -la /path/to/container/dir
total 8
drwxr-xr-x 2 myuser mygroup 4096 Dec 22 16:08 .
drwxr-xr-x 3 root root 4096 Dec 22 16:08 ..
-rw-r--r-- 1 myuser mygroup 0 Dec 22 15:51 myfile
Verify that it has the correct uid/gid:
docker run --rm example ls -lan /path/to/container/dir
total 8
drwxr-xr-x 2 1234 2345 4096 Dec 22 16:08 .
drwxr-xr-x 3 0 0 4096 Dec 22 16:08 ..
-rw-r--r-- 1 1234 2345 0 Dec 22 15:51 myfile
Note: there is an open feature-request for adding this functionality:
issue #35018 "Allow COPY command's --chown to be dynamically populated via ENV or ARG"
In my case, I used my UID and GID numbers and it works as I do have the same non-root account in the DEV and PROD environments.
COPY --chown=1000:1000 /path/to/host/dir/ /path/to/container/dir
And you can find the user and group IDs with the linux command: id
Not an expert on AWS and trying to fool around with Cron jobs. For testing, I had a sample script send me emails every minute. Now, I want to change it to once every 10 minutes (*/10 * * * *) These are the container commands I tried and none of them seems to work.
I am using a config file and a txt file to define the crons.
Config file contents (with various ideas I read from online sources)
container_commands:
00_remove_old_cron_jobs0:
command: "rm -fr /etc/cron.d/cron_job"
01_remove_old_cron_jobs1:
command: "sudo sed -i 's/empty stuff//g' /etc/cron.d/cron_job"
02_remove_old_cron_jobs2:
command: "crontab -r || exit 0"
03_cron_job:
command: "cat .ebextensions/cron_job.txt > /etc/cron.d/cron_job && chmod 644 /etc/cron.d/cron_job"
leader_only: true
cron_job.txt file contents.
# The newline at the end of this file is extremely important. Cron won't run without it.
0 * * * * ec2-user /usr/bin/php -q /var/www/html/cron1.php > /dev/null
0 * * * * ec2-user /usr/bin/php -q /var/www/html/html/cron2.php > /dev/null
*/10 * * * * ec2-user /usr/bin/php -q /var/www/html/cronTestEmailer.php > /dev/null
The test emailer script keeps firing every minute instead of every 10 mins and I dont know how I can make sure cron updates are reflected correctly.
You can achieve the same with the follow ebextensions config file.
files:
"/etc/cron.d/mycron":
mode: "000644"
owner: root
group: root
content: |
* * * * * root /usr/local/bin/myscript.sh
"/usr/local/bin/myscript.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash
date > /tmp/date
# Your actual script content
exit 0
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/*.bak"
More Details about the config file below:
files: Creates a Cron job and a file with the name myscript.sh. If a file with the same name exists already, first it moves the old file .bak and creates the file with new contents.
commands: deletes the all .bak files
There are a few solutions to configuring .ebextension container commands for cronjobs but none of them are working for me.
I am concerned that the reason they aren't working is because .ebextensions is not in the root directory. This messy code was handed over to me and I've tried to move .ebextensions to where it needs to be but that seems to break everything else.
This app is a streaming video application currently in production and I can't afford to break it so I ended up just leaving it where it is.
Can someone tell if I am doing this right and I just need to find a way to move .ebextensions or is the problem in my cronjob configuration?
app1/.ebextensions/02_python.config
container_commands:
...
cronjob:
command: "echo .ebextensions/cronjobs.txt > /etc/cron.d/cronjobs && 644 /etc/cron.d/cronjobs"
leader_only: true
...
app1/.ebextensions/cronjobs.txt
***** root source /opt/python/run/venv/bin/activate && python3 manage.py runcrons > /var/log/cronjobs.log
app1/settings.py
INSTALLED_APPS = [
...
'django_cron',
...
]
CRON_CLASSES = [
'app2.crons.MyCronJob',
]
app2/crons
from django_cron import CronJobBase, Schedule
class MyCronJob(CronJobBase):
RUN_EVERY_MINS = 1
schedule = Schedule(run_every_mins=RUN_EVERY_MINS)
def do(self):
# calculate stuff
# update variables
This deploys to AWS elastic beanstalk without error and logs show it's run but the work doesn't get done and it only runs the command once on deploy. Logs show this.
Command cronjob] : Starting activity...
[2018-02-15T12:58:41.648Z] INFO [24604] - [Application update ingest16#207/AppDeployStage0/EbExtensionPostBuild/Infra-EmbeddedPostBuild/postbuild_0_api_backend/Test for Command 05_cronjob] :
Completed activity. Result:
Completed successfully
This does the job but only once on deploy.
container_commands:
...
cronjob:
command: "source /opt/python/run/venv/bin/activate && python3 manage.py runcrons"
leader_only: true
...
This doesn't work at all.
container_commands:
...
cronjob:
command: "echo /app1/.ebextensions/cronjobs.txt > /etc/cron.d/cronjobs && 644 /etc/cron.d/cronjobs"
leader_only: true
...
Hi why using django_cron, when you only need cron ?
Here is my config .ebextensions:
container_commands:
...
0.0.1.cron.mailing:
command: "cat .ebextensions/mailing.txt > /etc/cron.d/mailing && chmod 644 /etc/cron.d/mailing"
leader_only: true
Here my mailing.txt:
Every Morning at 05:00am
#* * * * * * command
#| | | | | | |
#| | | | | | + Comande Line
#| | | | | +-- Year (range: 1900-3000)
#| | | | +---- Day of the Week (range: 1-7, 1 standing for Monday)
#| | | +------ Month of the Year (range: 1-12)
#| | +-------- Day of the Month (range: 1-31)
#| +---------- Hour (range: 0-23)
#+------------ Minute (range: 0-59)
# m h dom mon dow command
0 5 * * * root source /opt/python/run/venv/bin/activate && source /opt/python/current/env && cd /opt/python/current/app/ && python manage.py my_command >> /home/ec2-user/cron-mailing.log 2>&1
And here how to create custom command : https://docs.djangoproject.com/en/2.0/howto/custom-management-commands/#module-django.core.management
Hope this help,
You need space in your cron file between * :
Your cronfile :
***** root source /opt/python/run/venv/bin/activate && python3 manage.py runcrons > /var/log/cronjobs.log
Fix it like that :
* * * * * root source /opt/python/run/venv/bin/activate && python3 manage.py runcrons > /var/log/cronjobs.log
Relatively new to running cron jobs in Centos6, I can't seem to get this Python script to execute properly. I would like this script to execute and then email me the output. I have been receiving emails, but they're empty.
So far, in Crontab I've tried entering:
*/10 * * * * cd /home/local/MYCOMPANY/purrone/MyPythonScripts_Dev1 && /usr/bin/python ParserScript_CampusInsiders.py > /var/log/cron`date +\%Y-\%m-\%d-\%H:\%M:\%S`-cron.log 2>&1 ; mailx -s "Feedparser Output" my#email.com
and
*/10 * * * * /home/local/MYCOMPANY/purrone/MyPythonScripts_Dev1/ParserScript_CampusInsiders.py > /var/log/cron`date +\%Y-\%m-\%d-\%H:\%M:\%S`-cron.log 2>&1 ; mailx -s "Feedparser Output" my#email.com
I have run chmod +x on the python script to make the script executable and the Python script has #!/usr/bin/env python at the header. What am I doing wrong here?
The other problem might be that I shouldn't be using the log file? All I see at /var/log/cron when I open with cat cron is entires like this, for example (no actual output from the script):
Jul 23 13:20:01 ent-mocdvsmg01 CROND[24681]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Jul 23 13:20:01 ent-mocdvsmg01 CROND[24684]: (MYJOB\purrone) CMD (/home/local/MYCOMPANY/purrone/MyPythonScripts_Dev1/ParserScript_CampusInsiders.py > /var/log/cron`date +\%Y-\%m-\%d-\%H:\%M:\%S`-cron.log 2>&1 ; mailx -s "Feedparser Output" my#email.com)
There is nothing going into your mailx input; it expects the message on stdin. Try running it outside of crontab as a test until it sends a valid email. You could test with:
% echo hello |mailx -s test my#email.com
Note that cron can email you the output of its run. You just need to add a line to the top of crontab like:
MAILTO = you#email.com
Solution was to omit the redirect > and instead edit the Crontab thusly:
*/15 * * * * /home/local/COMPANY/malvin/SilverChalice_CampusInsiders/SilverChalice_Parser.py | tee /home/local/COMPANY/malvin/SilverChalice_CampusInsiders`date +\%Y-\%m-\%d-\%H:\%M:\%S`-cron.log | mailx -s "SilverChalice CampusInsiders" my#email.com
mercurial-server runs on Ubuntu 12.04 LTS
myserver#ip:/etc$ hg --version
Mercurial Distributed SCM (version 2.0.2)
myserver#ip:/etc$ dpkg -s mercurial-server
Package: mercurial-server
Version: 1.2-1
....
myserver#ip:/etc/mercurial-server/remote-hgrc.d$ ls -ltr
total 12
-rw-r--r-- 1 root root 180 Oct 10 2011 logging.rc
-rw-r--r-- 1 root root 139 Oct 10 2011 access.rc
-rw-r--r-- 1 root root 74 Mar 13 22:14 check.rc
myserver#ip:/etc/mercurial-server/remote-hgrc.d$ cat check.rc
[hooks]
pretxncommit.author_check = /SOURCE/mercurial-server/validate.sh
#manually added here too
myserver#ip:/etc/mercurial-server/remote-hgrc.d$ cat ~hg/repos/hgadmin/.hg/hgrc
# WARNING: when these hooks run they will entirely destroy and rewrite
# ~/.ssh/authorized_keys
[extensions]
hgext.purge =
[hooks]
changegroup.aaaab_update = hg update -C default > /dev/null
changegroup.aaaac_purge = hg purge --all > /dev/null
changegroup.refreshauth = python:mercurialserver.refreshauth.hook
pretxncommit.author_check = /SOURCE/mercurial-server/validate.sh
myserver#ip:/etc/mercurial-server/remote-hgrc.d$ cat /SOURCE/mercurial-server/validate.sh
#!/bin/bash
echo "REMUSR:$REMOTE_USER"
echo "ATHR:`hg tip --template "{author}\n"`b"
exit 1
myserver#ip:~$ sudo -u hg cat ~hg/.ssh/authorized_keys
no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding,command="/usr/share/mercurial-server/hg-ssh root/user1/user1.pub" ssh-rsa AAAAB3xOMN8ZiF user1#server.com
no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding,command="/usr/share/mercurial-server/hg-ssh users/user2/user2.pub" ssh-rsa AAAAB3N..0HchQQw== user2#server.com
After this from a local machine(Windows) I cloned a testproject ,changed,commited,push and it was successfull without any error or message.I tried this with both the initial user/key and a user/key added via hgadmin push
D:\hg\testproj>hg push
pushing to ssh://hg#myserver.com/testproj
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
Works with
$ cat check.rc
[hooks]
pretxnchangegroup.author_check = /SOURCE/mercurial-server/validate.sh
Not working with pretxncommit