Remove old crons when redeplying to AWS beanstack - amazon-web-services

Not an expert on AWS and trying to fool around with Cron jobs. For testing, I had a sample script send me emails every minute. Now, I want to change it to once every 10 minutes (*/10 * * * *) These are the container commands I tried and none of them seems to work.
I am using a config file and a txt file to define the crons.
Config file contents (with various ideas I read from online sources)
container_commands:
00_remove_old_cron_jobs0:
command: "rm -fr /etc/cron.d/cron_job"
01_remove_old_cron_jobs1:
command: "sudo sed -i 's/empty stuff//g' /etc/cron.d/cron_job"
02_remove_old_cron_jobs2:
command: "crontab -r || exit 0"
03_cron_job:
command: "cat .ebextensions/cron_job.txt > /etc/cron.d/cron_job && chmod 644 /etc/cron.d/cron_job"
leader_only: true
cron_job.txt file contents.
# The newline at the end of this file is extremely important. Cron won't run without it.
0 * * * * ec2-user /usr/bin/php -q /var/www/html/cron1.php > /dev/null
0 * * * * ec2-user /usr/bin/php -q /var/www/html/html/cron2.php > /dev/null
*/10 * * * * ec2-user /usr/bin/php -q /var/www/html/cronTestEmailer.php > /dev/null
The test emailer script keeps firing every minute instead of every 10 mins and I dont know how I can make sure cron updates are reflected correctly.

You can achieve the same with the follow ebextensions config file.
files:
"/etc/cron.d/mycron":
mode: "000644"
owner: root
group: root
content: |
* * * * * root /usr/local/bin/myscript.sh
"/usr/local/bin/myscript.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash
date > /tmp/date
# Your actual script content
exit 0
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/*.bak"
More Details about the config file below:
files: Creates a Cron job and a file with the name myscript.sh. If a file with the same name exists already, first it moves the old file .bak and creates the file with new contents.
commands: deletes the all .bak files

Related

Log Each Request And Output In EC2 Ubuntu or Linux

I have an AWS EC2 instance and in this instance, I have some cron jobs.
This cron jobs looks like:
0 5 * * mon curl -Ssi -X POST http://example.com
And I have some manual outputs like:
echo "output: hello..."
I want to store these actions on a log file in EC2 ubuntu or linux instance. Is it possible? Any suggestion?
Expected output:
[2021-10-10 ...] - POST http://example.com
[2021-10-11 ...] - output: hello...
Write the following script ~/bin/site-detector
#!/bin/bash
source ~/.bash_profile
log_file=/tmp/site-detector.log
curl -Ssi -X POST http://example.com >> $log_file
echo "Detected # $(date)" >> $log_file
echo " " >> $log_file
Make your script executable:
chmod a+x ~/bin/site-detector
Update your crontab script:
0 5 * * mon ~/bin/site-detector

Copy file from .ebextentions folder

I want to file from .ebextentions/td-agent.conf to /etc/td-agent/td-agent.conf. But its not working and getting below error.
if you see attached image, there are 3 files in .ebextentions. I had put copy command in 01-main.config.
---
container_commands:
01_cron_job:
command: "touch /tmp/is_leader"
leader_only: true
01_tdconfcopy_job:
command: "yes | cp .ebextensions/td-agent.conf /etc/td-agent/td-agent.conf"
error is as given below
Command failed on instance. Return code: 1 Output: cp: cannot create regular file '/etc/td-agent/td-agent.conf': No such file or directory
The issue is you're copying a file to a directory that doesn't exist. You should create the output directory first and then copy config files.
So it would be:
---
container_commands:
01_cron_job:
command: "touch /tmp/is_leader"
leader_only: true
01_create_dir:
command: "sudo mkdir -p /etc/td-agent/"
02_tdconfcopy_job:
command: "yes | cp .ebextensions/td-agent.conf /etc/td-agent/td-agent.conf"
Alternatively, you could to create a file on the server directly with files command.
files:
"/etc/td-agent/td-agent.conf":
mode: "000644"
owner: root
group: root
content: |
content of your config file that you want to copy

Chef checking log file size

I want to create a code with chef which it test the size of log file if it exceeds 30mb for example it will delete it in cron job.
Can you help me in it please?
this is my solution but it's not what i want:
execute "echo '/srv/#{app['shortname']}/current/app/log/*.log {
rotate 1
missingok
copytruncate
sharedscripts
postrotate
/bin/rm -rf /srv/#{app['shortname']}/current/app/log/*.log*
endscript
}' >> /etc/logrotate.d/#{app['name']}-app-log-rotation" do
ignore_failure true
end
You can always create a file with the shell script to delete the file and call that file via chef cron resource
file '/tmp/foo.sh' do
content "if (wc -c < /tmp/y.txt) > 3072000; then rm -f /tmp/y.txt; fi"
mode 755
end
cron 'name_of_cron_entry' do
minute '3'
command 'sh -x /tmp/foo.sh > /tmp/backup 2>&1'
end
This will create a cron entry for you with that script.
You can try something like
file '/tmp/y.txt' do
only_if { ::File.size('/tmp/y.txt').to_f / 1024000 > 30 }
action :delete
end

django_cron config when .ebextensions is not in root directory

There are a few solutions to configuring .ebextension container commands for cronjobs but none of them are working for me.
I am concerned that the reason they aren't working is because .ebextensions is not in the root directory. This messy code was handed over to me and I've tried to move .ebextensions to where it needs to be but that seems to break everything else.
This app is a streaming video application currently in production and I can't afford to break it so I ended up just leaving it where it is.
Can someone tell if I am doing this right and I just need to find a way to move .ebextensions or is the problem in my cronjob configuration?
app1/.ebextensions/02_python.config
container_commands:
...
cronjob:
command: "echo .ebextensions/cronjobs.txt > /etc/cron.d/cronjobs && 644 /etc/cron.d/cronjobs"
leader_only: true
...
app1/.ebextensions/cronjobs.txt
***** root source /opt/python/run/venv/bin/activate && python3 manage.py runcrons > /var/log/cronjobs.log
app1/settings.py
INSTALLED_APPS = [
...
'django_cron',
...
]
CRON_CLASSES = [
'app2.crons.MyCronJob',
]
app2/crons
from django_cron import CronJobBase, Schedule
class MyCronJob(CronJobBase):
RUN_EVERY_MINS = 1
schedule = Schedule(run_every_mins=RUN_EVERY_MINS)
def do(self):
# calculate stuff
# update variables
This deploys to AWS elastic beanstalk without error and logs show it's run but the work doesn't get done and it only runs the command once on deploy. Logs show this.
Command cronjob] : Starting activity...
[2018-02-15T12:58:41.648Z] INFO [24604] - [Application update ingest16#207/AppDeployStage0/EbExtensionPostBuild/Infra-EmbeddedPostBuild/postbuild_0_api_backend/Test for Command 05_cronjob] :
Completed activity. Result:
Completed successfully
This does the job but only once on deploy.
container_commands:
...
cronjob:
command: "source /opt/python/run/venv/bin/activate && python3 manage.py runcrons"
leader_only: true
...
This doesn't work at all.
container_commands:
...
cronjob:
command: "echo /app1/.ebextensions/cronjobs.txt > /etc/cron.d/cronjobs && 644 /etc/cron.d/cronjobs"
leader_only: true
...
Hi why using django_cron, when you only need cron ?
Here is my config .ebextensions:
container_commands:
...
0.0.1.cron.mailing:
command: "cat .ebextensions/mailing.txt > /etc/cron.d/mailing && chmod 644 /etc/cron.d/mailing"
leader_only: true
Here my mailing.txt:
Every Morning at 05:00am
#* * * * * * command
#| | | | | | |
#| | | | | | + Comande Line
#| | | | | +-- Year (range: 1900-3000)
#| | | | +---- Day of the Week (range: 1-7, 1 standing for Monday)
#| | | +------ Month of the Year (range: 1-12)
#| | +-------- Day of the Month (range: 1-31)
#| +---------- Hour (range: 0-23)
#+------------ Minute (range: 0-59)
# m h dom mon dow command
0 5 * * * root source /opt/python/run/venv/bin/activate && source /opt/python/current/env && cd /opt/python/current/app/ && python manage.py my_command >> /home/ec2-user/cron-mailing.log 2>&1
And here how to create custom command : https://docs.djangoproject.com/en/2.0/howto/custom-management-commands/#module-django.core.management
Hope this help,
You need space in your cron file between * :
Your cronfile :
***** root source /opt/python/run/venv/bin/activate && python3 manage.py runcrons > /var/log/cronjobs.log
Fix it like that :
* * * * * root source /opt/python/run/venv/bin/activate && python3 manage.py runcrons > /var/log/cronjobs.log

No Output for Python Script Executed via Cron Job

Relatively new to running cron jobs in Centos6, I can't seem to get this Python script to execute properly. I would like this script to execute and then email me the output. I have been receiving emails, but they're empty.
So far, in Crontab I've tried entering:
*/10 * * * * cd /home/local/MYCOMPANY/purrone/MyPythonScripts_Dev1 && /usr/bin/python ParserScript_CampusInsiders.py > /var/log/cron`date +\%Y-\%m-\%d-\%H:\%M:\%S`-cron.log 2>&1 ; mailx -s "Feedparser Output" my#email.com
and
*/10 * * * * /home/local/MYCOMPANY/purrone/MyPythonScripts_Dev1/ParserScript_CampusInsiders.py > /var/log/cron`date +\%Y-\%m-\%d-\%H:\%M:\%S`-cron.log 2>&1 ; mailx -s "Feedparser Output" my#email.com
I have run chmod +x on the python script to make the script executable and the Python script has #!/usr/bin/env python at the header. What am I doing wrong here?
The other problem might be that I shouldn't be using the log file? All I see at /var/log/cron when I open with cat cron is entires like this, for example (no actual output from the script):
Jul 23 13:20:01 ent-mocdvsmg01 CROND[24681]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Jul 23 13:20:01 ent-mocdvsmg01 CROND[24684]: (MYJOB\purrone) CMD (/home/local/MYCOMPANY/purrone/MyPythonScripts_Dev1/ParserScript_CampusInsiders.py > /var/log/cron`date +\%Y-\%m-\%d-\%H:\%M:\%S`-cron.log 2>&1 ; mailx -s "Feedparser Output" my#email.com)
There is nothing going into your mailx input; it expects the message on stdin. Try running it outside of crontab as a test until it sends a valid email. You could test with:
% echo hello |mailx -s test my#email.com
Note that cron can email you the output of its run. You just need to add a line to the top of crontab like:
MAILTO = you#email.com
Solution was to omit the redirect > and instead edit the Crontab thusly:
*/15 * * * * /home/local/COMPANY/malvin/SilverChalice_CampusInsiders/SilverChalice_Parser.py | tee /home/local/COMPANY/malvin/SilverChalice_CampusInsiders`date +\%Y-\%m-\%d-\%H:\%M:\%S`-cron.log | mailx -s "SilverChalice CampusInsiders" my#email.com