Currently, I'm trying to convert my Bash script to a C++ executable, but I am stuck with the sed command.
Here is my Bash script:
unset WIFIMAC
unset BTMAC
# Skip processing if MAC addresses are already written
if [ -f /data/.mac.info -a -f /data/.bt.info ]; then
echo "MAC addresses already found."
fi
# Wait until Samsung's RIL announces MAC addresses
until [ $(expr length "$WIFIMAC") == 17 ]; do
WIFIMAC=`getprop ril.wifi_macaddr`
sleep 1
done
until [ $(expr length "$BTMAC") == 12 ]; do
BTMAC=`getprop ril.bt_macaddr`
sleep 1
done
# Set WiFi MAC address
echo $WIFIMAC >/data/.mac.info
# Convert BT MAC address to proper format
echo $BTMAC | sed 's!^M$!!;s!\-!!g;s!\.!!g;s!\(..\)!\1:!g;s!:$!!' >/data/.bt.info
exit
Here my try to convert it to C++ (I put comments next to Bash commands):
# This script will read the MAC addresses from Samsung's RIL.
unset WIFIMAC ---> char wifimac....
unset BTMAC ---> char btmac...
# Skip processing if MAC addresses are already written
if [ -f /data/.mac.info -a -f /data/.bt.info ]; then ----> create file_exist(); function with fd = open... and put a smiply if return block
echo "MAC addresses already found."
fi
# Wait until Samsung's RIL announces MAC addresses
until [ $(expr length "$WIFIMAC") == 17 ]; do -----> while strlen(wifimac) == 17 blah blah blah....
WIFIMAC=`getprop ril.wifi_macaddr` -----> property_get function in cutils.h
sleep 1 -----> mdelay(1) if ım not wrong huh?.....
done
until [ $(expr length "$BTMAC") == 12 ]; do
BTMAC=`getprop ril.bt_macaddr` -----> SAME COMMANDS ABOVE
sleep 1
done
# Set WiFi MAC address
echo $WIFIMAC >/data/.mac.info -----> create write_string_to_path(); function with write(fd, ...)
# Convert BT MAC address to proper format
echo $BTMAC | sed 's!^M$!!;s!\-!!g;s!\.!!g;s!\(..\)!\1:!g;s!:$!!' >/data/.bt.info -----> ********HERE İS THE COMMAND "sed" *********
exit
I know a little bit about sed, but I don't know what it is doing here, thus I don't know which command in C++ does the same thing.
sed reads its input (in your example the string in $BTMAC which is echoed) and based on the expression passed to it, replaces some parts of the input data with something else (again according to the expression) and writes the result into the stdout or output file (in your case: /data/.bt.info file)
For example:
$ sed 's/replaceThis/WithThis/'
will replace word replaceThis with word WithThis in its input data and output the result. While both the above strings can be regular expressions.
If you are unfamiliar with it, you have to read the manual to first understand what's going on in your sed command. no one here is going to do that for you.
Now, in C++11, we have regex library that you can and I guess should use. If you can't use C++11 stuff, you should seek out other libraries which provide working with regex like Qt.
So in regex library of C++11, you can use regex_replace() function, or search with regex_search() and replace manually.
Related
This question already has answers here:
How to parse $QUERY_STRING from a bash CGI script?
(16 answers)
Closed 3 years ago.
All right, folks, you may have seen this infamous quirk to get hold of those values:
query=`echo $QUERY_STRING | sed "s/=/='/g; s/&/';/g; s/$/'/"`
eval $query
If the query string is host=example.com&port=80 it works just fine and you get the values in bash variables host and port.
However, you may know that a cleverly crafted query string will cause an arbitrary command to be executed on the server side.
I'm looking for a secure replacement or an alternative not using eval. After some research I dug up these alternatives:
read host port <<< $(echo "$QUERY_STRING" | tr '=&' ' ' | cut -d ' ' -f 2,4)
echo $host
echo $port
and
if [[ $QUERY_STRING =~ ^host=([^&]*)\&port=(.*)$ ]]
then
echo ${BASH_REMATCH[1]}
echo ${BASH_REMATCH[2]}
else
echo no match, sorry
fi
Unfortunately these two alternatives only work if the pars come in the order host,port. But they could come in the opposite order.
There could also be more than 2 pars, and any order is possible and allowed. So how do you propose to get the values into the
appropriate bash vars? Can the above methods be amended? Remember that with n pars there are n! possible orders. With 2 pars
there are only 2, but with 3 pars there are already 3! = 6.
I returned to the first method. Can it be made safe to run eval? Can you transform $QUERY_STRING with sed in a way that
makes it safe to do eval $query ?
EDIT: Note that this question differs from the other one referred to and is not a duplicate. The emphasis here is on using eval in a safe way. That is not answered in the other thread.
This method is safe. It does not eval or execute the QUERY_STRING. It uses string manipulation to break up the string into pieces:
QUERY_STRING='host=example.com&port=80'
declare -a pairs
IFS='&' read -ra pairs <<<"$QUERY_STRING"
declare -A values
for pair in "${pairs[#]}"; do
IFS='=' read -r key value <<<"$pair"
values["$key"]="$value"
done
echo do something with "${values[host]}" and "${values[port]}"
URL "percent decoding" left as an exercise.
You must avoid executing strings at all time when they come from untrusted sources. Therefore I would strongly suggest never to use eval in Bash do something with a string.
To be really save, I think I would echo the string into a file, use grep to retrieve parts of the string and remove the file afterwards. Always use a directory out of the web root.
#! /bin/bash
MYFILE=$(mktemp)
QUERY_STRING='host=example.com&port=80&host=lepmaxe.moc&port=80'
echo "${QUERY_STRING}" > ${MYFILE}
TMP_ARR=($(grep -Eo '(host|port)[^&]*' ${MYFILE}))
[ ${#TMP_ARR} -gt 0 ] || exit 1
[ $((${#TMP_ARR} % 2)) -eq 0 ] || exit 1
declare -A ARRAY;
for ((i = 0; i < ${#TMP_ARR[#]}; i+=2)); do
tmp=$(echo ${TMP_ARR[#]:$((i)):2})
port=$(echo $tmp | sed -r 's/.*port=([^ ]*).*/\1/')
host=$(echo $tmp | sed -r 's/.*host=([^ ]*).*/\1/')
ARRAY[$host]=$port
done
for i in ${!ARRAY[#]}; do
echo "$i = ${ARRAY[$i]}"
done
rm ${MYFILE}
exit 0
This produces:
lepmaxe.moc = 80
example.com = 80
I've had a bit of shell scripting practice reading piped input from other programs, but am unsure how to approach this problem.
THE BACKSTORY
A program robinbotter whose internals I can't really fix/modify takes its input from files equities.sym and blacklist.sym, each simple text files containing one ticker symbol per line.
When it runs okay, its output produces:
...
Downloading instruments: [ AFL KELYB LFUS ]
...
When it breaks due to internal bugs,
...
Downloading instruments: [ AFL KELYB LFUS LNVGY
and halts there, with no further output, yielding exit code 0 like in the okay case (unfortunately).
The ticker symbols are printed out with slight delay--no newlines in between--while the program is processing them.
When it hits LNVGY or unpredictably any other many possibilities, somehow it can't handle or at least skip them, instead crashing with no proper exception nor error code.
THE QUESTION
I'm trying to write a minimalistic wrapper script in BASH (eg. retryRB.sh ./robinbotter) which:
Somehow monitors the live unbuffered output of robinbotter, using a regex or other method to detect when output of a line containing "Downloading instruments: [ " doesn't end with "]" before the program ends. In which case:
Take the last symbol printed out (eg. LNVGY) which crashes the program, and append it to the bottom of file blacklist.sym. Like with
echo $lastSymbol >> blacklist.sym
Restart the program robinbotter, retaining its original command-line parameters: $#
I am familiar with tools like awk and sed, and would be open to building a short solution in Ruby if Bash doesn't cut it.
Here you have a Bash version of a code that imitates what your binary does.
then you have a wrapper which logs when the apps successfully completes, and also, when it fails. On failure, it also appends the last item printed, as you can see in the images below ( in this case Im hard-coding a failure on Bomb! and Boom! but you get the idea):
main.sh
#!/bin/bash
some=('Pera' 'Manzana' 'Frutilla' 'Durazno' 'Banana' 'Lechuga' 'Sandia' 'Papa' 'Melon' 'Milanesa' 'Bomb!' 'Boom!')
printf 'Downloading instruments: [ '
for (( i=1 ; i < 5 ; i++ )) {
item=${some[$( shuf -i 0-$(( ${#some[#]} - 1 )) -n 1 )]}
printf "$item"
[[ $item == 'Bomb!' || $item == "Boom!" ]] && exit || printf "$item"
[[ $i -lt 4 ]] && printf ' '
}
printf ' ]'
wrapper.sh
#!/bin/bash
while :
do
res=$( ./main.sh )
[[ ! "$res" =~ \[[^]]*\] ]] && printf "Failure : ${res##*[\[ ]}" || printf "Success"
printf '\n'
sleep 1
done
You can test these scripts and then put your binary in place of main.sh.
Regards!
I have this Bash script:
#!/bin/bash
rawurldecode() {
# This is perhaps a risky gambit, but since all escape characters must be
# encoded, we can replace %NN with \xNN and pass the lot to printf -b, which
# will decode hex for us
printf -v REPLY '%b' "${1//%/\\x}" # You can either set a return variable (FASTER)
echo "${REPLY}" #+or echo the result (EASIER)... or both... :p
}
echo -e "Content-type: video/x-matroska\n"
arr=(${QUERY_STRING//=/ })
ffmpeg -i "$(rawurldecode ${arr[1]})" -acodec copy -vcodec copy -map 0:0 -map 0:2 -f matroska - 2>/dev/null &
pid=$!
trap "kill $pid" SIGTERM SIGPIPE
wait
I want to change it so it can handle multiple parameters in the query string like this:
param1=value1¶m2=value2¶m3=value3
currently the arr regex split is based on = so it can only handle one parameter. I am not sure how to change this regex so I get arr[1] = value1; arr[2] = value2, etc.
Ideally I need it to be an associative array like: arr['param1'] = value1 but I am not sure if this is possible in Bash.
Solutions in other languages (PHP, Perl, Python) are acceptable as long as the behaviour of the script remains the same (i.e. it needs to take the query string and output the header + output from the stdout, and be able to kill the process it spawned when the client disconnects).
Any suggestions how to sanitize this input are also welcome.
You can just change the line:
arr=(${QUERY_STRING//=/ })
With:
arr=(${QUERY_STRING//[=&]/ })
Then you can get your values in the odd indexes.
Example
$ QUERY_STRING='param1=value1¶m2=value2¶m3=value3'
$ arr=(${QUERY_STRING//[=&]/ })
$ echo ${arr[1]}
value1
$ echo ${arr[3]}
value2
$ echo ${arr[5]}
value3
Reading your question again, I see you want the values in subsequent indices. You can do that with the extglob shell option as follows:
shopt -s extglob # with this you enable 'extglob'
arr=${QUERY_STRING//?(&)+([^&=])=/ }
Explanation:
?(&) -> matches zero or one occurrence of &
+([^&=])= -> matches 1+ occurrences of string without & or = followed by a =
I'm trying to build a secure copy protocol quick function. When I run the command it will work with a single file OR with the entire directory, but as soon as I put a /* after the local_repo it returns zsh: no matches found: hackingedu/*.
If I put the command scp hackingedu\/\* hackingedu the command works properly. I think I'm on the right track, but can't get it to work.
contains() {
string="$1"
substring="$2"
if test "${string#*$substring}" != "$string"
then
# echo '$substring is in $string'
return 1 # $substring is in $string
else
# echo '$substring is not in $string'
return 0 # $substring is not in $string
fi
}
# Quickly scp files in Workspace to Remote
function scp() {
local_repo="$1"
remote_repo="$2"
# find all the `*` and replace with `/*`
if [ contains $local_repo '*' ]; then
# replace all instances of * with \* <- HOW TO DO
fi
command scp -r $LOCAL_REPOS/$local_repo $ALEX_SERVER_UNAME#$ALEX_SERVER_PORT:$ALEX_REMOTE_ROOT_PATH/$remote_repo
# Description: $1: Local Repo | $2: Remote Repo
# Define ex: scpp local/path/to/file/or/directory/* remote/path/to/file/or/directory/*
# Live ex: scpp alexcory/index.php alexcory/index.php
# Live ex: scpp alexcory/* alexcory/*
#
# This Saves you from having long commands that look like this:
# scp -r ~/Google\ Drive/server/Dev/git\ repositories/hackingedu/* alexander#alexander.com:/home2/alexander/public_html/hackingedu/beta
}
Command trying to execute: scp -r ~/Google\ Drive/server/Dev/git\ repositories/hackingedu/* alexander#alexander.com:/home2/alexander/public_html/hackingedu/beta
Any ideas on how to find and replace an *? If there's a better way to do this please do tell! :)
If you know how to do this in bash I would like your input as well!
References:
How do you tell if a string contains another string in Unix shell scripting?
ZSH Find command replacement
ZSH Find command replacement 2
Using wildcards in commands with zsh
You can either prefix your scp call using noglob (which will turn off globbing for that command, e.g. noglob ls *) or use
autoload -U url-quote-magic
zle -N self-insert url-quote-magic
zstyle -e :urlglobber url-other-schema '[[ $words[1] == scp ]] && reply=("*") || reply=(http https ftp)'
the above should make zsh auto quote * when you use scp.
[...]
BTW, in any case, you should learn that you can easily quote special characters using ${(q)variable_name}, e.g.
% foo='*&$%normal_chars'
% echo $foo
*&$%normal_chars
% echo ${(q)foo}
\*\&\$%normal_chars
i need a bash script for mac osx working in this way:
./script.sh * folder/to/files/
#
# or #
#
./script.sh xx folder/to/files/
This script
read a list of files
open each file and read each lines
if lines ended with the same letters ('*' mode) or with custom letters ('xx') then
remove line and RE-SAVE file
backup original file
My first approach to do this:
#!/bin/bash
# ck init params
if [ $# -le 0 ]
then
echo "Usage: $0 <letters>"
exit 0
fi
# list files in current dir
list=`ls BRUTE*`
for i in $list
do
# prepare regex
case $1 in
"*") REGEXP="^.*(.)\1+$";;
*) REGEXP="^.*[$1]$";;
esac
FILE=$i
# backup file
cp $FILE $FILE.bak
# removing line with same letters
sed -Ee "s/$REGEXP//g" -i '' $FILE
cat $FILE | grep -v "^$"
done
exit 0
But it doesn't work as i want....
What's wrong?
How can i fix this script?
Example:
$cat BRUTE02.dat BRUTE03.dat
aa
ab
ac
ad
ee
ef
ff
hhh
$
If i use '*' i want all files that ended with same letters to be clean.
If i use 'ff' i want all files that ended with 'ff' to be clean.
Ah, it's on Mac OSx. Remember that sed is a little different from classical linux sed.
man sed
sed [-Ealn] command [file ...]
sed [-Ealn] [-e command] [-f command_file] [-i extension] [file
...]
DESCRIPTION
The sed utility reads the specified files, or the standard input
if no files are specified, modifying the input as specified by a list
of commands. The
input is then written to the standard output.
A single command may be specified as the first argument to sed.
Multiple commands may be specified by using the -e or -f options. All
commands are applied
to the input in the order they are specified regardless of their
origin.
The following options are available:
-E Interpret regular expressions as extended (modern)
regular expressions rather than basic regular expressions (BRE's).
The re_format(7) manual page
fully describes both formats.
-a The files listed as parameters for the ``w'' functions
are created (or truncated) before any processing begins, by default.
The -a option causes
sed to delay opening each file until a command containing
the related ``w'' function is applied to a line of input.
-e command
Append the editing commands specified by the command
argument to the list of commands.
-f command_file
Append the editing commands found in the file
command_file to the list of commands. The editing commands should
each be listed on a separate line.
-i extension
Edit files in-place, saving backups with the specified
extension. If a zero-length extension is given, no backup will be
saved. It is not recom-
mended to give a zero-length extension when in-place
editing files, as you risk corruption or partial content in situations
where disk space is
exhausted, etc.
-l Make output line buffered.
-n By default, each line of input is echoed to the standard
output after all of the commands have been applied to it. The -n
option suppresses this
behavior.
The form of a sed command is as follows:
[address[,address]]function[arguments]
Whitespace may be inserted before the first address and the
function portions of the command.
Normally, sed cyclically copies a line of input, not including
its terminating newline character, into a pattern space, (unless there
is something left
after a ``D'' function), applies all of the commands with
addresses that select that pattern space, copies the pattern space to
the standard output, append-
ing a newline, and deletes the pattern space.
Some of the functions use a hold space to save all or part of the
pattern space for subsequent retrieval.
anything else?
it's clear my problem?
thanks.
I don't know bash shell too well so I can't evaluate what the failure is.
This is just an observation of the regex as understood (this may be wrong).
The * mode regex looks ok:
^.*(.)\1+$ that ended with same letters..
But the literal mode might not do what you think.
current: ^.*[$1]$ that ended with 'literal string'
This shouldn't use a character class.
Change it to: ^.*$1$
Realize though the string in $1 (before it goes into the regex) should be escaped
incase there are any regex metacharacters contained within it.
Otherwise, do you intend to have a character class?
perl -ne '
BEGIN {$arg = shift; $re = $arg eq "*" ? qr/([[:alpha:]])\1$/ : qr/$arg$/}
/$re/ && next || print
'
Example:
echo "aa
ab
ac
ad
ee
ef
ff" | perl -ne '
BEGIN {$arg = shift; $re = $arg eq "*" ? qr/([[:alpha:]])\1$/ : qr/$arg$/}
/$re/ && next || print
' '*'
produces
ab
ac
ad
ee
ef
A possible issue:
When you put * on the command line, the shell replaces it with the name of all the files in your directory. Your $1 will never equal *.
And some tips:
You can replace replace:
This:
# list files in current dir
list=`ls BRUTE*`
for i in $list
With:
for i in BRUTE*
And:
This:
cat $FILE | grep -v "^$"
With:
grep -v "^$" $FILE
Besides the possible issue, I can't see anything jumping out at me. What do you mean clean? Can you give an example of what a file should look like before and after and what the command would look like?
This is the problem!
grep '\(.\)\1[^\r\n]$' *
on MAC OSX, ( ) { }, etc... must be quoted!!!
Solved, thanks.