I have data as this
tatusx2.atc?beginnum=0;8pctgRB Mwdf fgEio"text1"text4"text
tatqsx3.atc?beginnum=1;8pctgRBwsaNezxio"text2
tatssx4.atc?beginnum=2;8pctgsvMALNejkio"data2
tatksx4.atc?beginnum=1;8pctgxdfALNebfio"text3
tatzsx5.atc?beginnum=3;8pwerRBMALNetior"datac
How to get only data between ; and "
I have tried grep -oP ';.*?"' file and got output :
;8pctgRBMwdffgEio"
;8pctgRBwsaNezxio"
;8pctgsvMALNejkio"
;8pctgxdfALNebfio"
;8pwerRBMALNetior"
But my desired output is:
8pctgRB Mwdf fgEio
8pctgRBwsaNezxio
8pctgsvMALNejkio
8pctgxdfALNebfio
8pwerRBMALNetior
You need to use lookahead and lookbehind regex expressions
grep -oP '(?<=;)\w*(?=")'
I consider you play around regexr to learn more about regular expressions. Checkout their cheatsheet.
A much more readable way to write the expression you need is:
grep -oP '(?<=;).*(?=")' file
and will get you the desired result. PERL regexes are apparently experimental but certain patterns work without issues.
The following options are being used:
-o --only-matching to the print only the matched parts of a matching line
-P --perl-regexp
Using ?=; will get you the string beginning with ; but using the > you are able to start at the index after. Similarly the end string tag is specified.
Here is suggested additional reading.
I am looking to translate this regular expression into grep flavour:
I am trying to filter all lines that contain refs/changes/\d+/$VAR/
Example of line that should match, assuming that VAR=285900
b3fb1e501749b98c69c623b8345a512b8e01c611 refs/changes/00/285900/9
Current code:
VAR=285900
grep 'refs/changes/\d+/$VAR/' sample.txt
I am trying to filter all lines that contain refs/changes/\d+/$VAR/
That would be
grep "refs/changes/[[:digit:]]\{1,\}/$VAR/"
or
grep -E "refs/changes/[[:digit:]]+/$VAR/"
Note that the \d+ notation is a perl thing. Some overfeatured greps might support it with an option, but I don't recommend it for portability reasons.
inside simple quotes I cannot use variable expansion
You can mix and match quotes:
foo=not; echo 'single quotes '"$foo"' here'
with double quotes it does match anything.
It's not clear what you're doing, so we can't say why it doesn't work. It should work. There is no need to escape forward slashes for grep, they don't have any special meaning.
I'm using grep to search for text within a specific directory. I would like to return rows of text that contain stringA AND stringB.
I know that doing grep "stringA|stringB" is effectively an OR statement, is there something I can do, maybe using regex, that would allow me to run an AND statement ?
Many Thanks
If you don't know the order of the items, you could always reverse a pattern using |
grep '1st pattern.*2nd pattern|2nd pattern.*1st pattern' foofile
This works geat with two items, three or more would start slowing things down for sure...
You can pipe through two greps:
... | grep "stringA" | grep "stringB"
Note that if your patterns are actually fixed strings and not regular expressions then you can use fgrep instead of grep.
"stringA.*stringB"
This will locate stringA with any number of any characters between stringB at last
I have a regular expression for use with awk to find any of the specified words in a line of a file. It looks like this awk "/word1/||/word2/||/word3/" filename. As an alternative, I have been trying to specify the words like this WORDS="word1 word2 word3" and then use bash string substitution to form the regular expression to pass to awk.
I've tried numerous ways of doing this to no avail. awk simply dumps the contents of the entire file or spits out some complaint about the regex form.
Here's what I have:
#!/bin/bash
FILE="myfile"
WORDS="word1 word2 word3"
# use BASH string substitution to obtain the regex which should look like this:
# "/word1/||/word2/||/word3/"
REGEX=\"/${WORDS// //||/}/\"
awk ${REGEX} $FILE
I'm fairly sure it has to do with quoting and I've tried various methods using echo and back ticks and can get it look right (when echoed) but when actually trying to use it, it fails.
Try to replace:
REGEX=\"/${WORDS// //||/}/\"
with:
REGEX="/${WORDS// //||/}/"
Note that there is no need to escape double quotes since they are not really part of the regular expression.
This question already has answers here:
How do I match any character across multiple lines in a regular expression?
(26 answers)
Closed 3 years ago.
Really basic question here. So I'm told that a dot . matches any character EXCEPT a line break. I'm looking for something that matches any character, including line breaks.
All I want to do is to capture all the text in a website page between two specific strings, stripping the header and the footer. Something like HEADER TEXT(.+)FOOTER TEXT and then extract what's in the parentheses, but I can't find a way to include all text AND line breaks between header and footer, does this make sense? Thanks in advance!
When I need to match several characters, including line breaks, I do:
[\s\S]*?
Note I'm using a non-greedy pattern
You could do it with Perl:
$ perl -ne 'print if /HEADER TEXT/ .. /FOOTER TEXT/' file.html
To print only the text between the delimiters, use
$ perl -000 -lne 'print $1 while /HEADER TEXT(.+?)FOOTER TEXT/sg' file.html
The /s switch makes the regular expression matcher treat the entire string as a single line, which means dot matches newlines, and /g means match as many times as possible.
The examples above assume you're cranking on HTML files on the local disk. If you need to fetch them first, use get from LWP::Simple:
$ perl -MLWP::Simple -le '$_ = get "http://stackoverflow.com";
print $1 while m!<head>(.+?)</head>!sg'
Please note that parsing HTML with regular expressions as above does not work in the general case! If you're working on a quick-and-dirty scanner, fine, but for an application that needs to be more robust, use a real parser.
By definition, grep looks for lines which match; it reads a line, sees whether it matches, and prints the line.
One possible way to do what you want is with sed:
sed -n '/HEADER TEXT/,/FOOTER TEXT/p' "$#"
This prints from the first line that matches 'HEADER TEXT' to the first line that matches 'FOOTER TEXT', and then iterates; the '-n' stops the default 'print each line' operation. This won't work well if the header and footer text appear on the same line.
To do what you want, I'd probably use perl (but you could use Python if you prefer). I'd consider slurping the whole file, and then use a suitably qualified regex to find the matching portions of the file. However, the Perl one-liner given by '#gbacon' is an almost exact transliteration into Perl of the 'sed' script above and is neater than slurping.
The man page of grep says:
grep, egrep, fgrep, rgrep - print lines matching a pattern
grep is not made for matching more than a single line. You should try to solve this task with perl or awk.
As this is tagged with 'bbedit' and BBedit supports Perl-Style Pattern Modifiers you can allow the dot to match linebreaks with the switch (?s)
(?s).
will match ANY character. And yes,
(?s).+
will match the whole text.
As pointed elsewhere, grep will work for single line stuff.
For multiple-lines (in ruby with Regexp::MULTILINE, or in python, awk, sed, whatever), "\s" should also capture line breaks, so
HEADER TEXT(.*\s*)FOOTER TEXT
might work ...
here's one way to do it with gawk, if you have it
awk -vRS="FOOTER" '/HEADER/{gsub(/.*HEADER/,"");print}' file