Is there a cleaner (preferably gdb only) way to do the following (Setting breakpoints on every line in the code where a certain pattern appears),
grep ~/work/proj some_var_or_pattern -Irni| cut –f1,2 –d":" | sed –e 's/^/b /' > ~/moo.txt
and load the results with
(gdb) source ~/moo.txt
rbreak (unfortunately only for functions).
Unfortunately there is no build in command for setting breakpoints on all lines matching certain pattern. But there are build in commands for seting breakpoints on all functions matching the regular expressions:
rbreak regex and rbreak file:regex (short tutorial).
gdb doesn't do that by itself, but you can put your grep and break command generating code into a shell script and call it from gdb with a single command.
Put this in a file in your $PATH as, say, greptobp:
#!/bin/sh
# usage: greptobp pattern dir outputfile
pattern="$1"
dir="$2"
outputfile="$3"
if [ "${dir#/}" = "$dir" ]
then
# if dir isn't absolute, make it absolute, as requested by OP
dir="$PWD/$dir"
fi
grep -HIirn "$pattern" "$dir" |
awk -F: '{ print "break \"" $1 "\":" $2; }' > "$outputfile"
and add this to gdb:
(gdb) define patbreak
Type commands for definition of "patbreak".
End with a line saying just "end".
>shell greptobp $arg0 $arg1 /tmp/gdbtmp
>source /tmp/gdbtmp
>shell rm /tmp/gdbtmp
>end
(gdb) document patbreak
Type documentation for "patbreak".
End with a line saying just "end".
>Set breakpoints at lines matching specified pattern in a specified directory.
>Usage: patbreak pattern dir
>end
Related
I have EDI files I need to find, by using SED to search for some anomalies.
The anomaly is when I search for a "token" called SGP, and where they are on multiple consecutive lines — so one SGP on one line and another SGP on another line — regardless of what's after the token:
SGP+SEGU1037087'
SGP+DFSU1143210'
SGP+SEGU1166926'
SGP+TGHU1203545'
But I don't want to find files where there are other segment lines between each SGP line:
SGP+TGHU1643436'
GID+2+3:BAG'
FTX+AAA+++sdfjkhsdfjkhsdfjkh'
MEA+AAE+AAB+KGM:20000.0000'
MEA+AAE+AAW+MTQ:.0000'
SGP+HCIU2090577'
So I've tried this:
sed 'SGP.*\n.*SGP' < *.txt
And as probably expected, I get nothing.
Any ideas on how to feed into SED a list of files in DOS, and get a list of files that meet the above criteria?
UPDATE
I think I have the "feed the files" bit here. But I am still stuck on how to use SED properly.
for i in *.txt; do
sed -i '<<WHAT DO I PLACE HERE?>>' $i
done
UPDATE 2
Please no Unix/Bash/etc solutions.. I am in Windows only! Thank you
UPDATE 3
Tried a DOS equivalent of #tshiono's answer but I get nothing..
for %%f in (*.txt) do (
sed -ne ':l;N;$!b l;/SGP[^\n]\+\nSGP/p' %%f
}
UPDATE 4
#tshiono - I want the script to find files that have this pattern...
SGP+SEGU1037087'
SGP+DFSU1143210'
SGP+SEGU1166926'
SGP+TGHU1203545'
Not this pattern ...
SGP+SEGU1037087'
FTT+asdjkfhsdkf hsdjkfh sdfjkh sdf
FTX+f sdfjsdfkljsdkfljsdklfj
GID+sdfjkhsdjkfhsdjkfsdf
SGP+DFSU1143210'
FTT+asdjkfhsdkf hsdjkfh sdfjkh sdf
FTX+f sdfjsdfkljsdkfljsdklfj
GID+sdfjkhsdjkfhsdjkfsdf
SGP+SEGU1166926'
FTT+asdjkfhsdkf hsdjkfh sdfjkh sdf
FTX+f sdfjsdfkljsdkfljsdklfj
GID+sdfjkhsdjkfhsdjkfsdf
SGP+TGHU1203545'
Again - only lines with SGP as a token on every NEWLINE
Could you please try following.
awk '
FNR==1{
if(count){
if(fnr==count){
print prev_file " has all lines of SGP."
}
}
prev_file=FILENAME
count=fnr=""
}
/^SGP/{
++count
}
{
fnr++
}
END{
if(fnr==count){
print prev_file " has all lines of SGP."
}
}
' *.txt
The requirement is to detect which files contain consecutive lines both starting SGP.
Using standard (POSIX) sed, there's no way to get sed to print the file name. You can use this combination of shell script and sed, though, to detect which files contain consecutive lines starting with SGP:
for file in *.txt;
do
if [ -n "$(sed -n -e '/^SGP/{N;/^SGP.*\nSGP/{p;q;}}' "$file")" ]
then echo "$file"
fi
done
The shell test [ … ] checks whether the output of $(sed …) is a non-empty string, and reports the name of the file if it is. Note that the script is more flexible if, instead of using the glob *.txt, it uses the "$#" (list of arguments, preserving spaces etc). You can the write:
sh find-consecutive-SGP.sh *.txt
or use other more fanciful ways of specifying the file names as arguments.
The sed command doesn't print by default (-n). It looks for a line starting SGP and appends the next line into the 'pattern space'. It then looks to see if the result has two lots of SGP in it; one at the start (we know that will be there) and one after a newline. If that's found, it prints both lines (the pattern space) and quits because its job is done; it has found two consecutive lines both starting SGP. If the pattern space doesn't match, it is not printed (because of the -n) and more data is read. Any lines that don't start SGP are ignored and not printed.
With GNU sed, the F command prints the file name and a newline, so you could use:
for file in *.txt;
do
sed -n -e '/^SGP/{N;/^SGP.*\nSGP/{F;q;}}' "$file"
done
AFAICT from the GNU sed manual, there's no way to 'skip to the start of the next file' so you have to test each file separately as shown, rather than trying sed -n -e '…' *.txt — that will only report the first file that breaches the condition, not all the files.
If your objective is to get the list of filenames which meet the criteria,
how about:
for i in *.txt; do
[[ -n $(sed -ne ':l;N;$!b l;/SGP[^\n]\+\nSGP/p' "$i") ]] && echo "$i"
done
The sed commands :l;N;$!b makes a loop and slurps the whole lines
in the pattern space including "\n"
Then it matches the lines with the pattern of two consecutive lines
which both contain SGP.
If the sed output is non-empty, it prints the current filename.
[Update]
If your requirement is DOS platform, please try instead:
setlocal EnableDelayedExpansion
for %%f in (text*.txt) do (
set result=
for /f "usebackq tokens=*" %%a in (`sed.exe -ne ":l;N;$!b l;/SGP.\+\nSGP.\+/p" %%f`) do set result=!result!%%a
if "!result!" neq "" (
echo %%f
)
)
I've tested with Windows10 and sed-4.2.1.
The Problem
I am attempting to reuse a shell function I have defined in bash script later on in the script, within a perl cmd execution block. The call to perl cmd basically needs to to run the defined shell function after matching a piece of the regex (capture group #2). See code definitions below.
The Code
The pertinent function definition in bash shell script:
evalPS() {
PS_ARGS=$(eval 'echo -en "'${1}'"' | sed -e 's#\\\[##g' -e 's#\\\]##g')
PS_STR=$((set +x; (PS4="+.$PS_ARGS"; set -x; :) 2>&1) | cut -d':' -f1 | cut -d'.' -f2)
echo -en "${PS_STR}"
}
The definition above uses some bashisms and hacks to evaluate the users real prompt to a string.
That function needs to be called within perl in the next function:
remPS() {
# store evalPS definition
EVALPS_SOURCE=$(declare -f evalPS)
# initalize prompt
eval "$PROMPT_COMMAND" &> /dev/null
# handle args
( [[ $# -eq 0 ]] && cat - || cat "${1}" ) |
# ridiculous regex
perl -pe 's/^[^\e].*//gs' |
perl -s -0777 -e '`'"$EVALPS_SOURCE"'`; while (<>) { s%(.*?\Q$ps1\E)(?{`eval $PROMPT_COMMAND`})|(.*?\Q$ps2\E)(?{$ps2=`$\(evalPS "${PS2}"\)`})|(\Q$ps3\E)|(^\Q$ps4\E+.*?\b)%%g; } continue {print;}' -- \
-ps1="$(evalPS "${PS1}")" -ps2="$(evalPS "${PS2}")" \
-ps3="${PS3}" -ps4="${PS4:0:1}" |
perl -pe 's/(.*?)\e\[[A-Z](\a)*/\1/g'
}
The call to perl could be moved to a separate script but either way the issue is I can not find a way to "import" or "source" the remPS() function, within the perl script. I also tried sourcing the function from a separate file definition, into the perl command. Like so:
perl -s -0777 -e '`. /home/anon/Desktop/flyball_labs/scripts/recsesh_lib.sh`; while (<>) { s%(.*?\Q$ps1\E)(?{`eval $PROMPT_COMMAND`})|(.*?\Q$ps2\E)(?{$ps2=`$\(evalPS "${PS2}"\)`})|(\Q$ps3\E)|(^\Q$ps4\E+.*?\b)%%g; } continue {print;}'
...
Or using the source builtin:
perl -s -0777 -e '`source /home/anon/Desktop/flyball_labs/scripts/recsesh_lib.sh`; while (<>) { s%(.*?\Q$ps1\E)(?{`eval $PROMPT_COMMAND`})|(.*?\Q$ps2\E)(?{$ps2=`$\(evalPS "${PS2}"\)`})|(\Q$ps3\E)|(^\Q$ps4\E+.*?\b)%%g; } continue {print;}'
...
And for clarity, the final attempt was passing the function declaration into perl like so:
perl -s -0777 -e '`'"$EVALPS_SOURCE"'`; while (<>) { s%(.*?\Q$ps1\E)(?{`eval $PROMPT_COMMAND`})|(.*?\Q$ps2\E)(?{$ps2=`$\(evalPS "${PS2}"\)`})|(\Q$ps3\E)|(^\Q$ps4\E+.*?\b)%%g; } continue {print;}'
...
The Results
With no luck in any of the above cases.. It seems that the . cmd runs whereas the source cmd does not, and the syntax for passing the declaration of the function into perl may not be possible, as shown from the output of my tests:
Sourcing library definitions w/ source cmd
(1)|anon#devbox /tmp|$ remPS "${TEXT_FILE}"
sh: 1: source: not found
...
Sourcing w/ shell . cmd
(1)|anon#devbox /tmp|$ remPS "${TEXT_FILE}"
sh: 1: evalPS: not found
...
Passing declaration to perl
(1)|anon#devbox /tmp|$ remPS "${TEXT_FILE}"
sh: 3: Syntax error: ")" unexpected (expecting "}")
sh: 1: evalPS: not found
...
To summarize
Q) How to "import" and run a user defined shell command within perl?
A) 2 Possible solutions:
source the function from separate file definition
pass into perl command from bash using variable expansion
Sources & Research
Evaluating real bash prompt value:
how-to-print-current-bash-prompt echo-expanded-ps1
Note: I chose this implementation of evalPS() because using the script cmd workaround was unreliable and using call bind_variable() bash function required root privileges (effectively changing user's prompt).
Perl regex embeded code
Note: the function has to be run after every match of $PS2 to re-evaluate the new prompt and effectively match the next iteration (as it would in a real shell session). The use case I have for this is many people have (including myself) set their $PROMPT_COMMAND to iterate an integer indicating which line number (or offset from $PS1) the current line is, and displayed within $PS2.
running a shell command in perl
Sourcing shell code in perl:
how-to-run-source-command-linux-from-a-perl-script can-we-source-a-shell-script-in-perl-script sourcing-a-shell-script-from-a-perl-script
Alternatively if anyone knows how to translate my implementation of evalPS() into perl code, that would work too, but I believe this is impossible because the evaluated string is obtained using a "set hack" which as far as I know is strictly a bashism.
how-can-i-translate-a-shell-script-to-perl
Any suggestions would be much appreciated!
Edit
Some more info on the data being parsed..
The text file looks like the following (cat -A output):
^[]0;anon# - 3^G^[[1m^[[31m(^[[36m1^[[31m)|^[[33manon^[[32m#^[[34mdevbox ^[[35m/tmp^[[31m|^[[36m^[[37m$ ^[(B^[[mecho test^M$
test^M$
^[[1m^[[31m(^[[36m1^[[31m)|^[[33manon^[[32m#^[[34mdevbox ^[[35m/tmp^[[31m|^[[36m^[[37m$ ^[(B^[[mecho \^M$
^[[1m^[[31m[^[[36m2^[[31m]|^[[33m-^[[32m-^[[34m-^[[35m> ^[(B^[[m\^M$
^[[1m^[[31m[^[[36m3^[[31m]|^[[33m-^[[32m-^[[34m-^[[35m> ^[(B^[[m\^M$
^[[1m^[[31m[^[[36m4^[[31m]|^[[33m-^[[32m-^[[34m-^[[35m> ^[(B^[[mtest^M$
test^M$
^[[1m^[[31m(^[[36m1^[[31m)|^[[33manon^[[32m#^[[34mdevbox ^[[35m/tmp^[[31m|^[[36m^[[37m$ ^[(B^[[mexit^M$
exit^M$
Or similarly (less formatted):
ESC]0;anon# - 3^GESC[1mESC[31m(ESC[36m1ESC[31m)|ESC[33manonESC[32m#ESC[34mdevbox ESC[35m/tmpESC[31m|ESC[36mESC[37m$ ESC(BESC[mecho test
test
ESC[1mESC[31m(ESC[36m1ESC[31m)|ESC[33manonESC[32m#ESC[34mdevbox ESC[35m/tmpESC[31m|ESC[36mESC[37m$ ESC(BESC[mecho \
ESC[1mESC[31m[ESC[36m2ESC[31m]|ESC[33m-ESC[32m-ESC[34m-ESC[35m> ESC(BESC[m\
ESC[1mESC[31m[ESC[36m3ESC[31m]|ESC[33m-ESC[32m-ESC[34m-ESC[35m> ESC(BESC[m\
ESC[1mESC[31m[ESC[36m4ESC[31m]|ESC[33m-ESC[32m-ESC[34m-ESC[35m> ESC(BESC[mtest
test
ESC[1mESC[31m(ESC[36m1ESC[31m)|ESC[33manonESC[32m#ESC[34mdevbox ESC[35m/tmpESC[31m|ESC[36mESC[37m$ ESC(BESC[mexit
exit
My $PROMPT_COMMAND and corresponding prompts ($PS1-$PS4) for example:
PROMPT_COMMAND='TERM_LINE_NO=1'
PS1="\[$(tput bold)\]\[$(tput setaf 1)\](\[$(tput setaf 6)\]\${TERM_LINE_NO}\[$(tput setaf 1)\])|\[$(tput setaf 3)\]\u\[$(tput setaf 2)\]#\[$(tput setaf 4)\]\h \[$(tput setaf 5)\]\w\[$(tput setaf 1)\]|\[$(tput setaf 6)\]\$(parse_git_branch)\[$(tput setaf 7)\]\\$ \[$(tput sgr0)\]"
PS2="\[$(tput bold)\]\[$(tput setaf 1)\][\[$(tput setaf 6)\]\$((++TERM_LINE_NO))\[$(tput setaf 1)\]]|\[$(tput setaf 3)\]-\[$(tput setaf 2)\]-\[$(tput setaf 4)\]-\[$(tput setaf 5)\]> \[$(tput sgr0)\]"
PS3=""
PS4="+ "
The answer was to scrap this whole idea and use a better one..
Lets step back first.. Big Picture:
Goal was to make the script program output an executable shell script of the entire recorded session.
Back to Answers..
The above implementation was supposed to remove all prompts and control characters from the output of script (which is the input examples I gave) and then remove the output of each command (i.e. any line that didn't contain control characters).
Passing the evalPS function to perl to execute proved to be quite redundant and getting bash and perl to expand the parameters correctly was a nightmare..
The Final Solution
Scrapped the perl regex idea and used a combination of subshell and history redirection to grab the commands for the entire script session, while it was running.
The entire implementation looks like this:
# log cmds to script file as they are entered (unbuffered)
# spawn script cmd in subshell and wait for it to finish
wait -n
(
history -c
export HISTFILE="${SCRIPT_FILE}"
shopt -s histappend
script -q --timing="${TIME_FILE}" "${REC_FILE}"
history -a
)
...
Simple and much easier to read! :)
Hope this helps anyone trying to make their own mods to script in the future, cheers!
What I'm trying to do is check that a file has been created. The best way I can think to do this is by listing the files before hand, listing them afterwards, deleting the before list from the after list, then seeing if the after list is not zero. I ran into trouble deleting the before list from the after list. Filenames with square brackets were not being deleted from the list.
while read -r LINE
do
sed -i -- "/$LINE/d" listfilesafter.swp #without the -- I get 'sed: 1: "listfilesafter.swp": extra characters at the end of l command'
rm listfilesafter.swp--
done < listfilesbefore.swp
If I use '' then the variable doesn't get called, and the -r option on read doesn't seem to make it work like I expected. If anyone has any suggestions on alternative ways of doing this, do contribute, but I would still like to know how to use a variable in the search pattern when the value of the variable contains metacharacters. If anyone can help remove the code smell of "rm listfilesafter.swp--" then that would also be appreciated. Full code below:
cd ~/Desktop
ls >listfilesbefore.swp
#echo "balh blah" >SomeNonZeroFile.txt #comment or uncomment to test the if then statement
ls >listfilesafter.swp
sed -i -- '/listfilesafter.swp/d' listfilesafter.swp #deletes listfilesafter.swp from the list of files create after the event on line 3
while read -r LINE
do
sed -i -- "/$LINE/d" listfilesafter.swp #without the -- I get 'sed: 1: "listfilesafter.swp": extra characters at the end of l command'
rm listfilesafter.swp--
done < listfilesbefore.swp
cat listfilesafter.swp
echo "check listfiles. Enter to continue."
read dummy_variable
if [ -s listfilesafter.swp ]
then
rm listfilesbefore.swp
rm listfilesafter.swp
echo "success, the file was created"
else
rm listfilesbefore.swp
rm listfilesafter.swp
echo "failure, the file was not created"
fi
Given that you have two lists of files in sorted order (since ls lists the files in sorted order), you should probably be using a command like diff or, in this case,
comm to find the differences between the two lists of files.
If you want to know which file(s) were created, then that's the list of files (lines) in the second file that are not in the first. With no options, comm lists the lines it reads in 3 columns:
lines in the first file not in the second
lines in the second file not in the first
lines in both files
You only need the lines (file names) in the second column, and therefore you want to suppress the list of files in the first and third columns, so you'll use comm -13 to do that:
before=$(mktemp ${TMPDIR:-/tmp}/files.XXXXXX)
after=$(mktemp ${TMPDIR:-/tmp}/files.XXXXXX)
trap "rm -f $before $after; exit 1" 0 1 2 3 13 15
ls > $before
…execute command that creates file(s)…
ls > $after
comm -13 $before $after
rm -f $before $after
trap 0
Obviously, you could capture the list of files from comm in a variable for further analysis, etc.
Making sed work when the search strings contain metacharacters
I'm still confused about sed. How do I use a variable in the search pattern of sed if the value contains metacharacters? Or in this case would I be better off using something other than sed?
In the scenario you have, you're far better off not using sed, and in any case your technique is horrendously slow if there are hundreds or thousands of files in the directory (running sed once per file name is not going to be fast).
However, supposing that it was necessary to use sed and that you wanted to deal with metacharacters in the file names in the list, then you would have to escape the metacharacters (with a backslash in front). I'd probably do something like this:
sed 's/[][\/*.]/\\&/g; s%.*%/^&$/d%' listfilesbefore.swp > script.sed
sed -f script.sed listfilesafter.swp
The first script takes any metacharacters in the line (file name) and replaces it with backslash-metacharacter. In the first substitute, the [][\/*.] character class matches square brackets, two types of slashes, stars and dots. Depending on the predilections of the variant of sed you're using, you might need to protect (){} with backslashes too, but in POSIX standard sed, the {} gain metacharacter meaning when prefixed with a backslash, so they're not modified by default. The second substitute takes the possibly modified line and converts it into a 'match and delete' command. The output, therefore, is a sed script that will delete the file names found in listfilesbefore.swp. The second command applies that script to listfilesafter.swp, doing in one sed command what your outline code does with one run of sed per file name.
Using sed to generate a sed script is a powerful technique. It isn't always appropriate, but when it is, it is very useful.
Shell script demo.sh
echo "Pre-populate the directory with some random file names"
for file in $(random -n 20 -T '%W%V%C-%w%v%c%v%c-%04[0000:9999]d.txt')
do
cp /dev/null $file
done
for template in '%w%v%w(%03[000:999]d)%w%v%w.txt' \
'%w%v%w[123]%w%v%we.txt' \
'%w%v%wfile*%03[0:999]d*.txt' \
'%w%v%w%v%c\\\%d.txt' \
'%w%v%w-{%04X}-{%04X}.txt'
do
for file in $(random -n 2 -T "$template")
do
cp /dev/null "$file"
done
done
ls > listfilesbefore.swp
ls
echo
echo "Create some new files with metacharacters in the names"
for file in 'new(123)file.txt' 'new[123]file.txt' 'newfile*321*.txt' \
'newfile\\\.txt' 'newfile-{A39F}-{B77D}.txt'
do
cp /dev/null "$file"
done
ls
ls > listfilesafter.swp
echo
echo "Create sed script"
sed 's/[][\/*.]/\\&/g; s%.*%/^&$/d%' listfilesbefore.swp > script.sed
echo
cat script.sed
echo
echo "Apply it"
sed -f script.sed listfilesafter.swp
The random command I'm using is of my own devising, but it is convenient for demonstrations such as this.
Example run
Pre-populate the directory with some random file names
AIG-taral-3486.txt
COV-oipuc-9088.txt
CUG-vowan-5758.txt
FEH-ieqek-0603.txt
IUS-aaduw-7080.txt
KER-jazuc-4824.txt
MIZ-iezec-8255.txt
NIT-kupib-6873.txt
PUX-oocov-2216.txt
QAW-xonod-3937.txt
QES-wawok-4790.txt
RON-difag-1986.txt
SAD-gesug-5706.txt
SAJ-luqoj-4311.txt
TUZ-wapaw-8547.txt
VAL-zutap-8054.txt
YIP-xudeb-7397.txt
YUP-uudiv-8848.txt
ZIB-jurax-2903.txt
ZUR-xonik-8800.txt
aavfile*147*.txt
demo.sh
diman\\\7115.txt
ganur\\\8732.txt
gud-{7049}-{3103}.txt
listfilesbefore.swp
lur[123]maee.txt
rivfile*065*.txt
ueo(417)yea.txt
uoi(751)qio.txt
woi-{37E8}-{009C}.txt
xof[123]hoxe.txt
Create some new files with metacharacters in the names
AIG-taral-3486.txt
COV-oipuc-9088.txt
CUG-vowan-5758.txt
FEH-ieqek-0603.txt
IUS-aaduw-7080.txt
KER-jazuc-4824.txt
MIZ-iezec-8255.txt
NIT-kupib-6873.txt
PUX-oocov-2216.txt
QAW-xonod-3937.txt
QES-wawok-4790.txt
RON-difag-1986.txt
SAD-gesug-5706.txt
SAJ-luqoj-4311.txt
TUZ-wapaw-8547.txt
VAL-zutap-8054.txt
YIP-xudeb-7397.txt
YUP-uudiv-8848.txt
ZIB-jurax-2903.txt
ZUR-xonik-8800.txt
aavfile*147*.txt
demo.sh
diman\\\7115.txt
ganur\\\8732.txt
gud-{7049}-{3103}.txt
listfilesbefore.swp
lur[123]maee.txt
new(123)file.txt
new[123]file.txt
newfile*321*.txt
newfile-{A39F}-{B77D}.txt
newfile\\\.txt
rivfile*065*.txt
ueo(417)yea.txt
uoi(751)qio.txt
woi-{37E8}-{009C}.txt
xof[123]hoxe.txt
Create sed script
/^AIG-taral-3486\.txt$/d
/^COV-oipuc-9088\.txt$/d
/^CUG-vowan-5758\.txt$/d
/^FEH-ieqek-0603\.txt$/d
/^IUS-aaduw-7080\.txt$/d
/^KER-jazuc-4824\.txt$/d
/^MIZ-iezec-8255\.txt$/d
/^NIT-kupib-6873\.txt$/d
/^PUX-oocov-2216\.txt$/d
/^QAW-xonod-3937\.txt$/d
/^QES-wawok-4790\.txt$/d
/^RON-difag-1986\.txt$/d
/^SAD-gesug-5706\.txt$/d
/^SAJ-luqoj-4311\.txt$/d
/^TUZ-wapaw-8547\.txt$/d
/^VAL-zutap-8054\.txt$/d
/^YIP-xudeb-7397\.txt$/d
/^YUP-uudiv-8848\.txt$/d
/^ZIB-jurax-2903\.txt$/d
/^ZUR-xonik-8800\.txt$/d
/^aavfile\*147\*\.txt$/d
/^demo\.sh$/d
/^diman\\\\\\7115\.txt$/d
/^ganur\\\\\\8732\.txt$/d
/^gud-{7049}-{3103}\.txt$/d
/^listfilesbefore\.swp$/d
/^lur\[123\]maee\.txt$/d
/^rivfile\*065\*\.txt$/d
/^ueo(417)yea\.txt$/d
/^uoi(751)qio\.txt$/d
/^woi-{37E8}-{009C}\.txt$/d
/^xof\[123\]hoxe\.txt$/d
Apply it
listfilesafter.swp
new(123)file.txt
new[123]file.txt
newfile*321*.txt
newfile-{A39F}-{B77D}.txt
newfile\\\.txt
I am newbie of shell script. Correct if I am wrong.
I have one file called ignorefileslist in which it has file's names and patterns like /trunk/* means ignore all files from trunk.
I am able to get line of file which holds this pattern but unable to find out how to execute those pattern to get files matching that pattern. For example files inside /trunk folder for /trunk/* pattern.
ignorefileslist.txt :
/.settings
/.project
/branches/staging/test.php
/trunk/index.php
/trunk/*.php
/trunk/*
Here is my code :
if grep -q "*" "$HOME/$REPONAME/ignorablefileslist.txt";then
filewithpattern=`cat $HOME/$REPONAME/ignorablefileslist.txt | grep "*" `
echo "It contains pattern $filewithpattern"
fi
Output :
It contains pattern /trunk/*.php /trunk/*
How can I both /trunk/* and /trunk/*.php regex to get files accordingly?
Change your grep command like below to print the filenames which contains /trunk/* or /trunk/*.php strings.
grep -lP '/trunk/\*(\.php)?' *.*
If for some reason you need to keep your conditional statement intact then you could use a loop and process substitution to avoid a subshell.
if grep -q "*" ignorefileslist.txt; then while read filewithpattern; do echo "It contains pattern $filewithpattern"; done < <( grep "*" ignorefileslist.txt ); fi
i need a bash script for mac osx working in this way:
./script.sh * folder/to/files/
#
# or #
#
./script.sh xx folder/to/files/
This script
read a list of files
open each file and read each lines
if lines ended with the same letters ('*' mode) or with custom letters ('xx') then
remove line and RE-SAVE file
backup original file
My first approach to do this:
#!/bin/bash
# ck init params
if [ $# -le 0 ]
then
echo "Usage: $0 <letters>"
exit 0
fi
# list files in current dir
list=`ls BRUTE*`
for i in $list
do
# prepare regex
case $1 in
"*") REGEXP="^.*(.)\1+$";;
*) REGEXP="^.*[$1]$";;
esac
FILE=$i
# backup file
cp $FILE $FILE.bak
# removing line with same letters
sed -Ee "s/$REGEXP//g" -i '' $FILE
cat $FILE | grep -v "^$"
done
exit 0
But it doesn't work as i want....
What's wrong?
How can i fix this script?
Example:
$cat BRUTE02.dat BRUTE03.dat
aa
ab
ac
ad
ee
ef
ff
hhh
$
If i use '*' i want all files that ended with same letters to be clean.
If i use 'ff' i want all files that ended with 'ff' to be clean.
Ah, it's on Mac OSx. Remember that sed is a little different from classical linux sed.
man sed
sed [-Ealn] command [file ...]
sed [-Ealn] [-e command] [-f command_file] [-i extension] [file
...]
DESCRIPTION
The sed utility reads the specified files, or the standard input
if no files are specified, modifying the input as specified by a list
of commands. The
input is then written to the standard output.
A single command may be specified as the first argument to sed.
Multiple commands may be specified by using the -e or -f options. All
commands are applied
to the input in the order they are specified regardless of their
origin.
The following options are available:
-E Interpret regular expressions as extended (modern)
regular expressions rather than basic regular expressions (BRE's).
The re_format(7) manual page
fully describes both formats.
-a The files listed as parameters for the ``w'' functions
are created (or truncated) before any processing begins, by default.
The -a option causes
sed to delay opening each file until a command containing
the related ``w'' function is applied to a line of input.
-e command
Append the editing commands specified by the command
argument to the list of commands.
-f command_file
Append the editing commands found in the file
command_file to the list of commands. The editing commands should
each be listed on a separate line.
-i extension
Edit files in-place, saving backups with the specified
extension. If a zero-length extension is given, no backup will be
saved. It is not recom-
mended to give a zero-length extension when in-place
editing files, as you risk corruption or partial content in situations
where disk space is
exhausted, etc.
-l Make output line buffered.
-n By default, each line of input is echoed to the standard
output after all of the commands have been applied to it. The -n
option suppresses this
behavior.
The form of a sed command is as follows:
[address[,address]]function[arguments]
Whitespace may be inserted before the first address and the
function portions of the command.
Normally, sed cyclically copies a line of input, not including
its terminating newline character, into a pattern space, (unless there
is something left
after a ``D'' function), applies all of the commands with
addresses that select that pattern space, copies the pattern space to
the standard output, append-
ing a newline, and deletes the pattern space.
Some of the functions use a hold space to save all or part of the
pattern space for subsequent retrieval.
anything else?
it's clear my problem?
thanks.
I don't know bash shell too well so I can't evaluate what the failure is.
This is just an observation of the regex as understood (this may be wrong).
The * mode regex looks ok:
^.*(.)\1+$ that ended with same letters..
But the literal mode might not do what you think.
current: ^.*[$1]$ that ended with 'literal string'
This shouldn't use a character class.
Change it to: ^.*$1$
Realize though the string in $1 (before it goes into the regex) should be escaped
incase there are any regex metacharacters contained within it.
Otherwise, do you intend to have a character class?
perl -ne '
BEGIN {$arg = shift; $re = $arg eq "*" ? qr/([[:alpha:]])\1$/ : qr/$arg$/}
/$re/ && next || print
'
Example:
echo "aa
ab
ac
ad
ee
ef
ff" | perl -ne '
BEGIN {$arg = shift; $re = $arg eq "*" ? qr/([[:alpha:]])\1$/ : qr/$arg$/}
/$re/ && next || print
' '*'
produces
ab
ac
ad
ee
ef
A possible issue:
When you put * on the command line, the shell replaces it with the name of all the files in your directory. Your $1 will never equal *.
And some tips:
You can replace replace:
This:
# list files in current dir
list=`ls BRUTE*`
for i in $list
With:
for i in BRUTE*
And:
This:
cat $FILE | grep -v "^$"
With:
grep -v "^$" $FILE
Besides the possible issue, I can't see anything jumping out at me. What do you mean clean? Can you give an example of what a file should look like before and after and what the command would look like?
This is the problem!
grep '\(.\)\1[^\r\n]$' *
on MAC OSX, ( ) { }, etc... must be quoted!!!
Solved, thanks.