I read the documentation of ?: in tcl regexp. Which says that it matches an expression without capturing it.
I tried and it worked fine.
My query is, what is the proper use case for this option, as it we do not want to use capture sequence, we won't puts brackets there.
Is it just an alternate way, or have some special condition, where we should use this? Kindly clarify.
Easy: You need to group several elements in your Regex, but you don't need them as a capturing group for reference.
a+ (b+|c+) OR (a+ b+)|c+
I need braces for grouping. But if I run it like this the engine will capture all those matches. This may need a lot of memory and cost a lot of performance. If I don't need the capturing groups later for reference, I can use ?: to get grouping without the performance impact:
a+ (?:b+|c+) OR (?:a+ b+)|c+
First, have a look at the Tcl regex reference:
(expression)
Parentheses surrounding an expression specify a nested expression. The substring matching expression is captured and can be referred to via the back reference mechanism, and also captured into any corresponding match variable specified as an argument to the command.
(?:expression)
matches expression without capturing it.
While the first part describing capturing group ability to capture subtext to be referred to with backreferences is universal, the second part dwelling on initializing variables based on the capturing group is specific to Tcl.
Bearing that in mind, Tcl regex usage can be greatly simplified with non-capturing groups in case you have a pattern with a number of capturing groups, and you want to modify it by adding another group in-between existing groups.
Say, you want to match strings like abc 1234 (comment) and use {(\w+)\s+(\d+)\s+\(([^()]+)\)}:
regexp {(\w+)\s+(\d+)\s+\(([^()]+)\)} $a - body num comment
However, you were asked to also match strings with any number of word+space+digits in-between 1234 and comment. If you write
set a1 "abc 1234 more 5678 text 890 here 678 (comment)"
regexp {(\w+)\s+(\d+)(\s+\w+\s+\d+)*\s+\(([^()]+)\)} $a - body1 num1 comment1
^^^^^^^^^^^^^^^
the $comment will hold a value you would not expect.
Turning it into a non-capturing group fixes the issue.
See IDEONE demo
For other common uses of a non-capturing group, please refer to Are optional non-capturing groups redundant post.
You can use () parentheses in regex when matching multiple word options which you then do not want to capture.
(?:one|two|three)
Related
I have a need to replace this:
fixed variable 123
with this:
fixed variable 234
In VSCode this matches fine:
fixed(.*)123
I can't find any way to make it put the capture in the output if a number follows:
fixed$1234
fixed${1}234
But the find replace window just looks like this:
I read that VSCode uses rust flavoured rexes.. Here indicates ${1}234 should work, but VSCode just puts it in the output..
Tried named capture in a style according to here
fixed(?P<n>.*)123 //"invalid regular expression" error
VSCode doesn't seem to understand ${1}:
ps; I appreciate I could hack it in the contrived example with
FIND: fixed (.*) 123
REPL: fixed $1 234
And this does work in vscode:
but not all my data consistently has the same character before the number
After a lot of investigation by myself and #Wiktor we discovered a workaround for this apparent bug in vscode's search (aka find across files) and replace functionality in the specific case where the replace would have a single capture group followed by digits, like
$1234 where the intent is to replace with capture group 1 $1 followed by 234 or any digits. But $1234 is the actual undesired replaced output.
[This works fine in the find/replace widget for the current file but not in the find/search across files.]
There are (at least) two workarounds. Using two consecutive groups, like $1$2234 works properly as does $1$`234 (or precede with the $backtick).
So you could create a sham capture group as in (.*?)()(\d{3}) where capture group 2 has nothing in it just to get 2 consecutive capture groups in the replace or
use your intial search regex (.*?)(\d{3}) and then use $` just before or after your "real" capture group $1.
OP has filed an issue https://github.com/microsoft/vscode/issues/102221
Oddly, I just discovered that replacing with a single digit like $11 works fine but as soon as you add two or more it fails, so $112 fails.
I'd like to share some more insights and my reasoning when I searched for a workaround.
Main workaround idea is using two consecutive backreferences in the replacement.
I tried all backreference syntax described at Replacement Strings Reference: Matched Text and Backreferences. It appeared that none of \g<1>, \g{1}, ${1}, $<1>, $+{1}, etc. work. However, there are some other backreferences, like $' (inserts the portion of the string that follows the matched substring) or $` (inserts the portion of the string that precedes the matched substring). However, these two backreferences do not work in VS Code file search and replace feature, they do not insert any text when used in the replacement pattern.
So, we may use $` or $' as empty placeholders in the replacement pattern.
Find What: fix(.*?)123
Replace With:
fix$'$1234
fix$`$1234
Or, as in my preliminary test, already provided in Mark's answer, a "technical" capturing group matching an empty string, (), can be introduced into the pattern so that a backreference to that group can be used as a "guard" before the subsequent "meaningful" backreference:
Find What: fixed()(.*)123 (see () in the pattern that can be referred to using $1)
Replace With: fixed$1$2234
Here, $1 is a "guard" placeholder allowing correct parsing of $2 backreference.
Side note about named capturing groups
Named capturing groups are supported, but you should use .NET/PCRE/Java named capturing group syntax, (?<name>...). Unfortunately, the none of the known named backreferences work replacement pattern. I tried $+{name} Boost/Perl syntax, $<name>, ${name}, none work.
Conclusion
So, there are several issues here that need to be addressed:
We need an unambiguous numbered backerence syntax (\g<1>, ${1}, or $<1>)
We need to make sure $' or $` work as expected or are parsed as literal text (same as $_ (used to include the entire input string in the replacement string) or $+ (used to insert the text matched by the highest-numbered capturing group that actually participated in the match) backreferences that are not recognized by Visual Studio Code file search and replace feature), current behavior when they do not insert any text is rather undefined
We need to introduce named backreference syntax (like \g<name> or ${name}).
I have this regex which scans a text for the word very: (?i)(?:^|\W)(very)[\W$] which works. My goal is to upgrade it and avoid doing a match if very is within quotes, standalone or as part of a longer block.
Now, I have this other regex which is matching anything NOT inside curly quotes: (?<![\S"])([^"]+)(?![\S"]) which also works.
My problem is that I cannot seem to combine them. For example the string:
Fred Smith very loudly said yesterday at a press conference that fresh peas will "very, very defintely not" be served at the upcoming county fair. In this bit we have 3 instances of very but I'm only interested in matching the first one and ignore the whole Smith quotation.
What you describe is kind of tricky to handle with a regular expression. It's difficult to determine whether you are inside a quote. Your second regex is not effective as it only ignores the first very that is directly to the right of the quote and still matches the second one.
Drawing inspiration from this answer, that in turn references another answer that describes how to regex match a pattern unless ... I can capture the matches you want.
The basic idea is to use alternation | and match all the things you don't want and then finally match (and capture) what you do want in the final clause. Something like this:
"[^"]*"|(very)
We match quoted strings in the first clause but we don't capture them in a group and then we match (and capture) the word very in the second clause. You can find this match in the captured group. How you reference a captured group depends on your regex environment.
See this regex101 fiddle for a test case.
This regex
(?i)(?<!(((?<DELIMITER>[ \t\r\n\v\f]+)(")(?<FILLER>((?!").)*))))\bvery\b(?!(((?<FILLER2>((?!").)*)(")(?<DELIMITER2>[ \t\r\n\v\f]+))))
could work under two conditions:
your regex engine allows unlimited lookbehind
quotes are delimited by spaces
Try it on http://regexstorm.net/tester
I have a confusion when reading regular expression written by others, my question is parentheses () could only be used for capture group? Or we could use it to make regular expression more elegant to read by group logical set of operations together, especially for long regular expression?
I ask this since I sometimes see after people write long regular expression, they write additional parentheses even if they do not need to know capture group match information -- I mean even if they just need to know if the whole regular expression match or not, they still add some parentheses.
Parentheses are used for grouping and that would make regex engine capture the sub pattern inside the parentheses.
If you don't want to capture the text inside using non-capturing group by using this syntax:
(?:...)
This will also save some memory while processing the long and complex regex.
See more on Non capturing groups
A basic & simple example on how ad where () should be placed carefully if regex is this
^\w+|\d+$
Where we are selecting word at start or digits at end in the input:
foo -a123 bar baz 123
Thus we are matching foo and 123.
If you place brackets () like this:
^(\w+|\d+)$
then your match will fail as we are asserting presence of both anchors ^ and $ around a word OR a number in input.
Correct regex with brackets would be:
(?:^\w+|\d+$)
I'm trying to use ST2's regex capability in search & replace, but can't figure out how to probably make a non-capturing group. For this example, I want to find instances of "DEAN" which are not followed by "UMBER", i.e. to distinguish "DEANCARE" from "DEANUMBER"
From what I've read and used in the past, the syntax with a non-capture should be:
DEAN(?:UMBER)
Which should match "DEANCARE" but not "DEANUMBER". Yet instead, Sublime Text only finds "DEANUMBER" as if I had typed:
DEAN(UMBER)
Using square brackets on the first (or each) of the unwanted letters does work:
DEAN[^U]
But I'd still prefer to use the group non-match as opposed for other purposes and to avoid having to explicitly not-match each individual character. Do I have a syntax mistake, or maybe a conceptual error in how ST2's regex works?
A non capturing group is the same as a group except it does not capture the matching portion of the regex in a back-reference.
If you were to use the regex DEAN(?:UMBER) on the string DEANUMBER then you would have a match, but referencing \1 in, e.g. a search and replace would give you nothing, because the group is non-capturing.
Using DEAN(UMBER) on the other hand you could do a search and replace with made of L\1 which would produce made of LUMBER because the match of the first (capturing) group is being back-referenced by \1. This of course is a very pointless example, if you want to learn more about groups and back-referencing I'd suggest you read this or some other documentation/turoial on the matter.
As suggested in the comments, what you want is a negative lookahead.
I have come across a regular expression that I don't fully understand - can somebody help me in deciphering it:
^home(?:\/|\/index\.asp)?(?:\?.+)?$
It is used in url matching and the above example matches the following urls:
home
home/
home/?a
home/?a=1
home/index.asp
home/index.asp?a
home/index.asp?a=1
It seems to me that the question marks within the brackets (?: don't do anything. Can somebody enlighten me.
The version of regex being used is the one supplied with Classic ASP and is being run on the server if that helps at all.
(?:) creates a non-capturing group. It groups things together without creating a backreference.
A backreference is a part you can refer to in the expression or a possible replacement (usually by saying \1 or $1 etc - depending on flavor). You can also usually extract them from a match afterwards when using regex in a programming language. The only reason for using (?:) is to avoid creating a new backreference, which avoids incrementing the group number, and saves (a usually negligible amount of) memory
It's a non-capture group, which essentially is the same as using (...), but the content isn't retained (not available as a back reference).
If you're doing something like this: (abc)(?:123)(def) You'll get abc in $1 and def in $2, but 123 will only be matched.
From documentation:
(?:...)
A non-capturing version of regular parentheses. Matches whatever regular expression is inside the parentheses, but the substring matched by the group cannot be retrieved after performing a match or referenced later in the pattern.
its really easy
every parentheses will create a variable in the memory so you can use the parentheses value afterward so to not save it in memory just put :? in the parentheses like this (?:) and then fill the rest as you need.
that's it and nothing else