I am using Ubuntu 10.10 and using Grep to process some HTML files.
Here is the HTML snippet:
<a href="video.php?video=one-hd.mov"><img src="/1.jpg"><a href="video.php?video=normal.mov"><img src="/2.jpg"><a href="video.php?video=another-hd.mov">
I would like to extract one-hd.mov and another-hd.mov but ignore normal.mov.
Here is my code:
example='<a href="video.php?video=one-hd.mov"><img src="/1.jpg"><a href="video.php?video=normal.mov"><img src="/2.jpg"><a href="video.php?video=another-hd.mov">'
echo $example | grep -Po '(?<=video.php\?video=).*?(?=-hd.mov">)'
The result is:
one
normal.mov"><img src="/2.jpg"><a href="video.php?video=another
But I want
one
another
There is a mismatch there.
Is this because of the so-called Greedy Regular Expression?
I am sing GREP but any command line bash tools are welcome to solve this problem like sed etc.
Thanks a lot.
You want use Perl regexes for grep - why not directly perl?
echo "$example" | perl -nle 'm/.*?video.php\?video=([^"]+)">.*video.php\?video=([^"]+)".*/; print "=$1=$2="'
will print
=one-hd.mov=another-hd.mov=
Here is a solution using xmlstarlet:
$ example='<a href="video.php?video=one-hd.mov"><img src="/1.jpg"><a href="video.php?video=normal.mov"><img src="/2.jpg"><a href="video.php?video=another-hd.mov">'
$ echo $example | xmlstarlet fo -R 2>/dev/null | xmlstarlet sel -t -m "//*[substring(#href, string-length(#href) - 6, 7) = '-hd.mov']" -v 'substring(#href,17, string-length(#href) - 17 - 3)' -n
one-hd
another-hd
$
Solution using awk:
{
for(i=1;i<NF;i++) {
if ($i ~ /mov/) {
if ($i !~ /normal/){
sub(/^.*=/, "", $i)
print $i
}
}
}
}
outputs:
$ awk -F'"' -f h.awk html
one-hd.mov
another-hd.mov
But I strongly advice you to use a html-parser for this instead, something like BeautifulSoup
Related
I would like to Find in several files the empty function like
fLocalEvent(){
}
I would like to find using grep
I tried:
grep -Rlz "function fLocalEventoBotoes.*[\n\s]*?}" sob/SOB910C.js
grep -Rlz "fLocalEventoBotoes\((?:(?!\)\s*\{).)*\)\s*\{\s*\}" sob/SOB910C.js - return -bash: !\: event not found
You are better off using something like Perl for this:
echo '
fLegit1(x){ something }
empty1(){}
legit2(){ something }
fLocalEvent(){
}' | perl -0777 -lnE 'say $1 while (/(\w+\(\s*\)\s*{\s*})/g)'
Prints:
empty1(){}
fLocalEvent(){
}
With GNU grep, you can use the -Pzo switches to use that same regex, ignore line endings, and print only the matched section:
echo '
fLegit1(x){ something }
empty1(){}
legit2(){ something }
fLocalEvent(){
}' | grep -Pzo '\w+\(\s*\)\s*{\s*}\s*'
# same output...
This grep should work :
grep -Poz '\w+\(\){\s*}\n?' data
I have some basic knowledge on using regular expressions with grep (bash).
But I want to use regular expressions the other way around.
For example I have a file containing the following entries:
line_one=[0-3]
line_two=[4-6]
line_three=[7-9]
Now I want to use bash to figure out to which line a particular number matches.
For example:
grep 8 file
should return:
line_three=[7-9]
Note: I am aware that the example of "grep 8 file" doesn't make sense, but I hope it helps to understand what I am trying to achieve.
Thanks for you help,
Marcel
As others haven pointed out, awk is the right tool for this:
awk -F'=' '8~$2{print $0;}' file
... and if you want this tool to feel more like grep, a quick bash wrapper:
#!/bin/bash
awk -F'=' -v seek_value="$1" 'seek_value~$2{print $0;}' "$2"
Which would run like:
./not_exactly_grep.sh 8 file
line_three=[7-9]
My first impression is that this is not a task for grep, maybe for awk.
Trying to do things with grep I only see this:
for line in $(cat file); do echo 8 | grep "${line#*=}" && echo "${line%=*}" ; done
Using while for file reading (following comments):
while IFS= read -r line; do echo 8 | grep "${line#*=}" && echo "${line%=*}" ; done < file
This can be done in native bash using the syntax [[ $value =~ $regex ]] to test:
find_regex_matching() {
local value=$1
while IFS= read -r line; do # read from input line-by-line
[[ $line = *=* ]] || continue # skip lines not containing an =
regex=${line#*=} # prune everything before the = for the regex
if [[ $value =~ $regex ]]; then # test whether we match...
printf '%s\n' "$line" # ...and print if we do.
fi
done
}
...used as:
find_regex_matching 8 <file
...or, to test it with your sample input inline:
find_regex_matching 8 <<'EOF'
line_one=[0-3]
line_two=[4-6]
line_three=[7-9]
EOF
...which properly emits:
line_three=[7-9]
You could replace printf '%s\n' "$line" with printf '%s\n' "${line%%=*}" to print only the key (contents before the =), if so inclined. See the bash-hackers page on parameter expansion for a rundown on the syntax involved.
This is not built-in functionality of grep, but it's easy to do with awk, with a change in syntax:
/[0-3]/ { print "line one" }
/[4-6]/ { print "line two" }
/[7-9]/ { print "line three" }
If you really need to, you could programmatically change your input file to this syntax, if it doesn't contain any characters that need escaping (mainly / in the regex or " in the string):
sed -e 's#\(.*\)=\(.*\)#/\2/ { print "\1" }#'
As I understand it, you are looking for a range that includes some value.
You can do this in gawk:
$ cat /tmp/file
line_one=[0-3]
line_two=[4-6]
line_three=[7-9]
$ awk -v n=8 'match($0, /([0-9]+)-([0-9]+)/, a){ if (a[1]<n && a[2]>n) print $0 }' /tmp/file
line_three=[7-9]
Since the digits are being treated as numbers (vs a regex) it supports larger ranges:
$ cat /tmp/file
line_one=[0-3]
line_two=[4-6]
line_three=[75-95]
line_four=[55-105]
$ awk -v n=92 'match($0, /([0-9]+)-([0-9]+)/, a){ if (a[1]<n && a[2]>n) print $0 }' /tmp/file
line_three=[75-95]
line_four=[55-105]
If you are just looking to interpret the right hand side of the = as a regex, you can do:
$ awk -F= -v tgt=8 'tgt~$2' /tmp/file
You would like to do something like
grep -Ef <(cut -d= -f2 file) <(echo 8)
This wil grep what you want but will not display where.
With grep you can show some message:
echo "8" | sed -n '/[7-9]/ s/.*/Found it in line_three/p'
Now you would like to transfer your regexp file into such commands:
sed 's#\(.*\)=\(.*\)#/\2/ s/.*/Found at \1/p#' file
Store these commands in a virtual command file and you will have
echo "8" | sed -nf <(sed 's#\(.*\)=\(.*\)#/\2/ s/.*/Found at \1/p#' file)
I have a string something like this
xsd:import schemaLocation="AppointmentManagementService.xsd6.xsd" namespace=
I want to extract the following from it :
AppointmentManagementService.xsd6.xsd
I have tried using regex, bash and sed with no success. Can someone please help me out with this?
The regex that I used was this :
/AppointmentManagementService.xsd\d{1,2}.xsd/g
Your string is:
nampt#nampt-desktop:$ cat 1
xsd:import schemaLocation="AppointmentManagementService.xsd6.xsd" namespace=
Try with awk:
cat 1 | awk -F "\"" '{print $2}'
Output:
AppointmentManagementService.xsd6.xsd
sed doesn't recognize \d, use [0-9] or [[:digit:]] instead:
sed 's/^.*schemaLocation="\([^"]\+[[:digit:]]\{1,2\}\.xsd\)".*$/\1/g'
## or
sed 's/^.*schemaLocation="\([^"]\+[0-9]\{1,2\}\.xsd\)".*$/\1/g'
You can use bash native regex matching:
$ in='xsd:import schemaLocation="AppointmentManagementService.xsd6.xsd" namespace='
$ if [[ $in =~ \"(.+)\" ]]; then echo "${BASH_REMATCH[1]}"; fi
Output:
AppointmentManagementService.xsd6.xsd
Based on your example, if you want to grant, at least, 1 or, at most, 2 digits in the .xsd... component, you can fine tune the regex with:
$ if [[ $in =~ \"(AppointmentManagementService.xsd[0-9]{1,2}.xsd)\" ]]; then echo "${BASH_REMATCH[1]}"; fi
using PCRE in GNU grep
grep -oP 'schemaLocation="\K.*?(?=")'
this will output pattern matched between schemaLocation=" and very next occurrence of "
Reference:
https://unix.stackexchange.com/a/13472/109046
Also we can use 'cut' command for this purpose,
[root#code]# echo "xsd:import schemaLocation=\"AppointmentManagementService.xsd6.xsd\" namespace=" | cut -d\" -f 2
AppointmentManagementService.xsd6.xsd
s='xsd:import schemaLocation="AppointmentManagementService.xsd6.xsd" namespace='
echo $s | sed 's/.*schemaLocation="\(.*\)" namespace=.*/\1/'
I make a cat of a file and apply on it a grep with a regular expression like this
cat /tmp/tmp_file | grep "toto.titi\[[0-9]\+\].tata=55"
the command display the following output
toto.titi[12].tata=55
is it possible to modify my grep command in order to extract the number 12 as displayed output of the command?
You can grab this in pure BASH using its regex capabilities:
s='toto.titi[12].tata=55'
[[ "$s" =~ ^toto.titi\[([0-9]+)\]\.tata=[0-9]+$ ]] && echo "${BASH_REMATCH[1]}"
12
You can also use sed:
sed 's/toto.titi\[\([0-9]*\)\].tata=55/\1/' <<< "$s"
12
OR using awk:
awk -F '[\\[\\]]' '{print $2}' <<<"$s"
12
use lookahead
echo toto.titi[12].tata=55|grep -oP '(?<=\[)\d+'
12
without perl regex,use sed to replace "["
echo toto.titi[12].tata=55|grep -o "\[[0-9]\+"|sed 's/\[//g'
12
Pipe it to sed and use a back reference:
cat /tmp/tmp_file | grep "toto.titi\[[0-9]\+\].tata=55" | sed 's/.*\[(\d*)\].*/\1/'
I have a string of text in a file that I am parsing out, I almost got it but not sure what I am missing
basic expression I am using is
cat cred.txt | grep -m 1 -o '&CD=[^&]*'
I am getting a results of
&CD=u8AA-RaF-97gc_SdZ0J74gc_SdZ0J196gc_SdZ0J211
I do not want the &CD= part in the resulting string, how would I do that.
The string I am parsing from is:
webpage.asp?UserName=username&CD=u8AA-RaF-97gc_SdZ0J74gc_SdZ0J196gc_SdZ0J211&Country=USA
If your grep knows Perl regex:
grep -m 1 -oP '(?<=&CD=)[^&]*' cred.txt
If not:
sed '1s/.*&CD=\([^&]*\).*/\1/' cred.txt
Many ways to skin this cat.
Extend your pipe:
grep -o 'CD=[^&]*' cred.txt | cut -d= -f2
Or do a replacement in sed:
sed -r 's/.*[&?]CD=([^&]*).*/\1/' cred.txt
Or get really fancy and parse the actual QUERY_STRING in awk:
awk -F'?' '{ split($2, a, "&"); for(i in a){split(a[i], kv, "="); out[kv[1]]=kv[2];} print out["CD"];}'