I have an AWS EC2 instance and in this instance, I have some cron jobs.
This cron jobs looks like:
0 5 * * mon curl -Ssi -X POST http://example.com
And I have some manual outputs like:
echo "output: hello..."
I want to store these actions on a log file in EC2 ubuntu or linux instance. Is it possible? Any suggestion?
Expected output:
[2021-10-10 ...] - POST http://example.com
[2021-10-11 ...] - output: hello...
Write the following script ~/bin/site-detector
#!/bin/bash
source ~/.bash_profile
log_file=/tmp/site-detector.log
curl -Ssi -X POST http://example.com >> $log_file
echo "Detected # $(date)" >> $log_file
echo " " >> $log_file
Make your script executable:
chmod a+x ~/bin/site-detector
Update your crontab script:
0 5 * * mon ~/bin/site-detector
I want to download an apk which exists into my private s3 bucket using curl command. I dont want to use awscli/boto3. I have SecretAccessKey, SessionToken, Expiration, AccessKeyId
Tried Following Code:
curl -k -v -L -o url="https://s3-eu-west-1.amazonaws.com" -H "x-amz-security-token: xxxxxxxxxxxxxxxx" -H "Content-Type: application/xml" -X GET https://xyz/test.apk
curl -k -v -L -o url="https://s3-eu-west-1.amazonaws.com" -H "Content-Type: application/xml" -X GET https://xyz/.test.apk?AWSAccessKeyId=xxxxxxxxxxxxx
Here is the script that downloads and upload file to s3, you have to export keys or can modify the script accordingly.
export AWS_ACCESS_KEY_ID=AKxxx
export AWS_SECRET_ACCESS_KEY=zzzz
Download a file
./s3download.sh get s3://mybucket/myfile.txt myfile.txt
That's it, all you need to pass s3 bucket along with file name
#!/bin/bash
set -eu
s3simple() {
local command="$1"
local url="$2"
local file="${3:--}"
# todo: nice error message if unsupported command?
if [ "${url:0:5}" != "s3://" ]; then
echo "Need an s3 url"
return 1
fi
local path="${url:4}"
if [ -z "${AWS_ACCESS_KEY_ID-}" ]; then
echo "Need AWS_ACCESS_KEY_ID to be set"
return 1
fi
if [ -z "${AWS_SECRET_ACCESS_KEY-}" ]; then
echo "Need AWS_SECRET_ACCESS_KEY to be set"
return 1
fi
local method md5 args
case "$command" in
get)
method="GET"
md5=""
args="-o $file"
;;
put)
method="PUT"
if [ ! -f "$file" ]; then
echo "file not found"
exit 1
fi
md5="$(openssl md5 -binary $file | openssl base64)"
args="-T $file -H Content-MD5:$md5"
;;
*)
echo "Unsupported command"
return 1
esac
local date="$(date -u '+%a, %e %b %Y %H:%M:%S +0000')"
local string_to_sign
printf -v string_to_sign "%s\n%s\n\n%s\n%s" "$method" "$md5" "$date" "$path"
local signature=$(echo -n "$string_to_sign" | openssl sha1 -binary -hmac "${AWS_SECRET_ACCESS_KEY}" | openssl base64)
local authorization="AWS ${AWS_ACCESS_KEY_ID}:${signature}"
curl $args -s -f -H Date:"${date}" -H Authorization:"${authorization}" https://s3.amazonaws.com"${path}"
}
s3simple "$#"
You can find more detail here
I used the exact same youtube-dl command without the playlist option to download individual audio files, and it worked. But when I use it for this playlist, I get an error: Cannot download a video and extract audio into the same file! Use "(ext)s.%(ext)s" instead of "(ext)s" as the output template
Running on windows 10. Any help would be greatly appreciated!!
PS C:\xxx\FFMPEG> .\YouTubeBatchAudioPlaylistIndexes.bat
C:\xxx\FFMPEG>call bin\youtube-dl.exe -x --audio-format "mp3" --audio-quality 3 --batch-file="songs.txt" --playlist-items 4,6,7,8,10,11,16,17,20,21,23,25,27,28,31,33,36,38,39,41,43,45,46,48,50 -o"C:\Users\xxx\Downloads\%(title)s.%(ext)s" --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-x', '--audio-format', 'mp3', '--audio-quality', '3', '--batch-file=songs.txt', '--playlist-items', '4,6,7,8,10,11,16,17,20,21,23,25,27,28,31,33,36,38,39,41,43,45,46,48,50', '-oC:\\Users\\xxx\\Downloads\\(ext)s', '--verbose']
[debug] Batch file urls: ['https://www.youtube.com/watch?v=anurOHpo0aY&index=4&list=PLlRluznmnq9f7OMI4avwFyV2xMVxlV3_w&t=0s']
Usage: youtube-dl.exe [OPTIONS] URL [URL...]
youtube-dl.exe: error: Cannot download a video and extract audio into the same file! Use "C:\Users\xxx\Downloads\(ext)s.%(ext)s" instead of "C:\Users\xxx\Downloads\(ext)s" as the output template
If you look at the output, you see that the percent signs in your output template were gobbled up:
(...) '-oC:\\Users\\xxx\\Downloads\\(ext)s', '--verbose']
That is because in a batch file, you need to write %% if you want a percent sign, and double that again for call, like this:
call bin\youtube-dl.exe -x --audio-format "mp3" --audio-quality 3 ^
--batch-file="songs.txt" --playlist-items ^
4,6,7,8,10,11,16,17,20,21,23,25,27,28,31,33,36,38,39,41,43,45,46,48,50 ^
-o "C:\Users\xxx\Downloads\%%%%(title)s.%%%%(ext)s" --verbose
This is the query I am trying to use in order to connect to elasticsearch, which is in (172.21.150.230) in order to pull out information in a csv format:
es2csv -u http://xxx.xx.xxx.xxx:5601/ -f _all -d doc -i test2 -r -q '{"query": {"match": {"NAME": "xxx"}}}' -o database.csv
However, I get SyntaxError: invalid syntax
Thanks
Which version are you using?
This error raised when your json query is not valid or when you forget to add -r argument.
ubuntu#ubuntu:~/workspace/abc$ pserve --reload development.ini
Starting subprocess with file monitor
at start of server
usage: pserve [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: pserve --help [cmd1 cmd2 ...]
or: pserve --help-commands
or: pserve cmd --help
error: option --reload not recognized
Can anyone tell me what could be causing this error?
Try to move reload behind ini file like :
../bin/pserve development.ini --reload
Probably as the first argument it needs pyramid config file