I have a structured file with hierarchical text which describes a GUI in Delphi (a DFM-File).
Let's assume I have this file and I have to match all "Color = xxx" Lines, which are in the context of TmyButton (marked), but not those in other context. Within the TMyButton-Context there won't be a deeper hierarchical level.
object frmMain: TfrmMain
Left = 311
Top = 201
Color = clBtnFace
object MyFirstButton: TMyButton
Left = 555
Top = 301
Color = 16645072 <<<<<<MATCH THIS
OnClick = ButtonClick
end
object MyLabel: TLabel
Left = 362
Top = 224
Caption = 'a Caption'
Color = 16772831
Font.Color = clWindowText
end
object Panel2: TLTPanel
Left = 348
Top = 58
Width = 444
Height = 155
Color = clRed
object MyOtherButton: TMyButton
Left = 555
Top = 301
Color = 16645072 <<<<<<MATCH THIS
OnClick = ButtonClick
end
end
end
I tried it two days long with many, many different tries.
Here some of my incomplete pieces of the pattern:
/^[ ]{2,}object [A-Za-z0-9]+: TmyButton\r\n/mi <<<Matches the needed context
/^[ ]{4,}Color = [A-Za-z0-9]+\r\n/mi <<<Matches the needed result
/^[ ]{2,}end\r\n/mi <<<Matches the end of the context
(I don't know why, but I had to use "\r\n" instead of "$"...). I need to put this together, but ignoring the other lines except other "object xxx: yyy" and "end" Lines....
I would be glad to have some help!
Matching a line in a complex context requires a regex feature called lookaround, if you want or have to do it with a single regex. Specifically, you'd need variable-length lookbehind which PCRE doesn't offer.
So there are two possibilities:
Use a scripting approach like Rorick suggested or use a regex that matches everything from the start of your needed context until the actual match, and extract that using a capturing group. That could be done with
[ ]{2,}object \w+: TMyButton\r\n.*?^([ ]{4,}Color = \w+[ \t]*\r\n)
(brackets around the space inserted for clarity). Your match would then be in capturing group \1
Nested structures generally are not well suited for regexes (better for parsers) but if you're sure of the structure of your data as you mentioned, it might work OK.
If I understand you correctly, you try to create single regexp for this. There is no reason to do so.
Just find line with pattern object [A-Za-z0-9]+: TmyButton
Then check each next line against Color = [A-Za-z0-9]+ until you find it or reach end keyword.
Repeat steps until end of file
If you try to modify a bulk of source files, you could use some scripting for this purpose.
I know this is not PCRE, but a good alternative for software archeology.
You could at any time use AWK, if you do this from a command prompt. The script would look like this:
BEGIN { inObj = 0; } // Not really necessary
/TMyButton/ { inObj = 1; }
/end$/ { inObj = 0; }
/^[ ]{4,}Color = [A-Za-z0-9]+\r\n/ && inObj == 1
{ //do whatever you need to do
print $3;
}
AWK can be found all over the internet. I would try GAWK.
Related
Let's say I have a line,
$bagel = parser(1);
$potato = parser(3+(other var));
$avocado = parser(3-(var1+var2+var3));
$untouchedtoast = donotremove(4);
I want to print, instead of parser(1), just 1. So I want to strip function calls (matching parser(.) I guess?), but leave the innards untouched. The output would, ideally, be
$bagel = 1;
$potato = 3+(other var);
$avocado = 3-(var1+var2+var3);
$untouchedtoast = donotremove(4);
I tried %s/parser(.)//g, but it only replaced everything to the left of the innards. Tried a few other wildcards, but I think I have to somehow pass a variable from the input regex to the output regex, and I'm not sure if that's possible. If it matters, I'm doing this in vim.
Thoughts?
%s/parser(\(.\+\));/\1;/
Search parser();, extract everything inside () using \(.\+\) group, replace the entire expression with the group (\1), add a semicolon (because it was eaten by the search expression).
Try this:
search: \w\(|\)(?=;)
replace: blank
I am trying to extract the camera make & model from exifdata.
The exifdata itself is quite long, 4 lines follow:
JPEG.APP1.Ifd0.ImageDescription = ' '
JPEG.APP1.Ifd0.Make = 'Canon'
JPEG.APP1.Ifd0.Model = 'Canon PowerShot S120'
JPEG.APP1.Ifd0.Orientation = 1 = '0,0 is top left'
I use the following regexs, but they do not match. Are the patterns correct?
make := substring( meta from 'Make\\s+=\\s+(.*)');
model := substring( meta from 'Model\\s+=\\s+(.*)');
subtring([str] from [pattern]) doesn't work like you seem to think it does. You can find the details of how it works here: 9.7.2. SIMILAR TO Regular Expressions. That's the regular expression syntax your call uses.
For starters, there's this relevant bit of info:
As with SIMILAR TO, the specified pattern must match the entire data string, or else the function fails and returns null.
Your regular expressions clearly don't match the entire string.
Second is the next sentence:
To indicate the part of the pattern that should be returned on success, the pattern must contain two occurrences of the escape character followed by a double quote (")
This isn't quite the standard regex, so you need to be aware of it.
Rather than trying to get subtring([str] from [pattern]) working, I'm going to recommend an alternative: regexp_matches. This function uses standard POSIX regex syntax, and it returns a text[] containing all the captured groups from the match. Here's a quick test to show that it works:
SELECT regexp_matches($$JPEG.APP1.Ifd0.ImageDescription = ' '
JPEG.APP1.Ifd0.Make = 'Canon'
JPEG.APP1.Ifd0.Model = 'Canon PowerShot S120'
JPEG.APP1.Ifd0.Orientation = 1 = '0,0 is top left'$$, '(Make)') m;
(I'm using dollar quoting for your example string, in case you're not familiar with that syntax.)
This gives back the array {Make}.
Second, your regex actually doesn't work, as I found out in my testing. You have two problems:
The double slashes are incorrect. You don't need to escape the \ as PostgreSQL doesn't treat it as an escape character by default. You can read about escaping in strings here; the most relevant section is probably 4.1.2.2. String Constants with C-style Escapes. That section describes what you thought was happening by default, but it actually requires an E prefix to enable.
Fixing that improves the result:
SELECT regexp_matches($$JPEG.APP1.Ifd0.ImageDescription = ' '
JPEG.APP1.Ifd0.Make = 'Canon'
JPEG.APP1.Ifd0.Model = 'Canon PowerShot S120'
JPEG.APP1.Ifd0.Orientation = 1 = '0,0 is top left'$$, 'Make\s+=\s+(.*)') m;
now gives an array containing this string:
'Canon'
JPEG.APP1.Ifd0.Model = 'Canon PowerShot S120'
JPEG.APP1.Ifd0.Orientation = 1 = '0,0 is top left'
This brings us to...
The (.*) is matching everything to the end of the string, not the end of the line. You can actually fix this by doing something you probably want to do anyway: get the single quote marks out of the match. You can use this pattern to do that:
$$Make\s+=\s+'([^']+)'$$
I've used dollar quoting again, this time to avoid the ugliness of escaping all those single quote marks. Now the query is:
SELECT regexp_matches($$JPEG.APP1.Ifd0.ImageDescription = ' '
JPEG.APP1.Ifd0.Make = 'Canon'
JPEG.APP1.Ifd0.Model = 'Canon PowerShot S120'
JPEG.APP1.Ifd0.Orientation = 1 = '0,0 is top left'$$, $$Make\s+=\s+'([^']+)'$$) m;
which gives you pretty much exactly what you want: an array containing just the string Canon. You'll need to extract the result from the array, of course, but I'll leave that as an exercise for you.
That should be enough info for you to get the second expression working, too.
P.S. PostgreSQL's truly fine manual is a blessing.
I have a text file that is in a comma separated format, delimited by " on most fields. I am trying to get that into something I can enumerate through (Generic Collection, for example). I don't have control over how the file is output nor the character it uses for the delimiter.
In this case, the fields are separated by a comma and text fields are enclosed in " marks. The problem I am running into is that some fields have quotation marks in them (i.e. 8" Tray) and are accidentally being picked up as the next field. In the case of numeric fields, they don't have quotes around them, but they do start with a + or a - sign (depicting a positive/negative number).
I was thinking of a RegEx, but my skills aren't that great so hopefully someone can come up with some ideas I can try. There are about 19,000 records in this file, so I am trying to do it as efficiently as possible. Here are a couple of example rows of data:
"00","000000112260 ","Pie Pumpkin ","RET","6.99 "," ","ea ",+0000000006.99000
"00","000000304078 ","Pie Apple caramel ","RET","9.99 "," ","ea ",+0000000009.99000
"00","StringValue here","8" Tray of Food ","RET","6.99 "," ","ea ",-00000000005.3200
There are a lot more fields, but you can get the picture....
I am using VB.NET and I have a generic List setup to accept the data. I have tried using CSVReader and it seems to work well until you hit a record like the 3rd one (with a quote in the text field). If I could somehow get it to handle the additional quotes, than the CSVReader option will work great.
Thanks!
I recommend looking at the TextFieldParserClass in .Net. You need to include
Imports Microsoft.VisualBasic.FileIO.TextFieldParser
Here's a quick sample:
Dim afile As FileIO.TextFieldParser = New FileIO.TextFieldParser(FileName)
Dim CurrentRecord As String() ' this array will hold each line of data
afile.TextFieldType = FileIO.FieldType.Delimited
afile.Delimiters = New String() {","}
afile.HasFieldsEnclosedInQuotes = True
' parse the actual file
Do While Not afile.EndOfData
Try
CurrentRecord = afile.ReadFields
Catch ex As FileIO.MalformedLineException
Stop
End Try
Loop
From here:
Encoding fileEncoding = GetFileEncoding(csvFile);
// get rid of all doublequotes except those used as field delimiters
string fileContents = File.ReadAllText(csvFile, fileEncoding);
string fixedContents = Regex.Replace(fileContents, #"([^\^,\r\n])""([^$,\r\n])", #"$1$2");
using (CsvReader csv =
new CsvReader(new StringReader(fixedContents), true))
{
// ... parse the CSV
As this link says... Don't roll your own CSV parser!
Use TextFieldParser as Avi suggested. Microsoft has already done this for you. If you ended up writing one, and you find a bug in it, consider replacing it instead of fixing the bug. I did just that recently and it saved me a lot of time.
Give a look to the FileHelpers library.
You could give CsvHelper (a library I maintain) a try and it's available via NuGet. It follows the RFC 4180 standard for CSV. It will be able to handle any content inside of a field including commas, quotes, and new lines.
CsvHelper is simple to use, but it's also easy to configure it to work with many different types of delimited files.
CsvReader csv = new CsvReader( streamToFile );
IEnumerable<MyObject> myObjects = csv.GetRecords<MyObject>();
If you want to read CSV files on a lower level, you can use the parser directly, which will return each row as a string array.
var parser = new CsvParser( myTextReader );
while( true )
{
string[] line = parser.ReadLine();
if( line == null )
{
break;
}
}
I am posting this as an answer so I can explain how I did it and why.... The answer from Mitch Wheat was the one that gave me the best solution for this case and I just had to modify it slightly due to the format this data was exported in.
Here is the VB Code:
Dim fixedContents As String = Regex.Replace(
File.ReadAllText(csvFile, fileEncoding),
"(?<!,)("")(?!,)",
AddressOf ReplaceQuotes)
The RegEx that was used is what I needed to change because certain fields had non-escaped quotes in them and the RegEx provided didn't seem to work on all examples. This one uses 'Look Ahead' and 'Look Behind' to see if the quote is just after a comma or just before. In this case, they are both negative (meaning show me where the double quote is not before or after a comma). This should mean that the quote is in the middle of a string.
In this case, instead of doing a direct replacement, I am using the function ReplaceQuotes to handle that for me. The reason I am using this is because I needed a little extra logic to detect whether it was at the beginning of a line. If I would have spent even more time on it, I am sure I could have tweaked the RegEx to take into consideration the beginning of the line (using MultiLine, etc) but when I tried it quickly, it didn't seem to work at all.
With this in place, using CSV reader on a 32MB CSV file (about 19000 rows), it takes about 2 seconds to read the file, perform the regex, load it into the CSV Reader, add all the data to my generic class and finish. Real quick!!
RegEx to exclude first and last quote would be (?<!^)(?<!,)("")(?!,)(?!$). Of course, you need to use RegexOptions.Multiline.
That way there is no need for evaluator function. My code replaces undesired double quotes with single quotes.
Complete C# code is as below.
string fixedCSV = Regex.Replace(
File.ReadAllText(fileName),
#"(?<!^)(?<!;)("")(?!;)(?!$)", "'", RegexOptions.Multiline);
There are at least ODBC drivers for CSV files. But there are different flavors of CSV.
What produced these files? It's not unlikely that there's a matching driver based on the requirements of the source application.
Your problem with CSVReader is that the quote in the third record isn't escaped with another quote (aka double quoting). If you don't escape them, then how would you expect to handle ", in the middle of a text field?
http://en.wikipedia.org/wiki/Comma-separated_values
(I did end up having to work with files (with different delimiters) but the quote characters inside a text value weren't escaped and I ended up writing my own custom parser. I do not know if this was absolutely necessary or not.)
The logic of this custom approach is: Read through file 1 line at a time, split each line on the comma, remove the first and last character (removing the outer quotes but not affecting any inside quotes), then adding the data to your generic list. It's short and very easy to read and work with.
Dim fr As StreamReader = Nothing
Dim FileString As String = ""
Dim LineItemsArr() as String
Dim FilePath As String = HttpContext.Current.Request.MapPath("YourFile.csv")
fr = New System.IO.StreamReader(FilePath)
While fr.Peek <> -1
FileString = fr.ReadLine.Trim
If String.IsNullOrEmpty(FileString) Then Continue While 'Empty Line
LineItemsArr = FileString.Split(",")
For Each Item as String In LineItemsArr
'If every item will have a beginning and closing " (quote) then you can just
'cut the first and last characters of the string here.
'i.e. UpdatedItems = Item. remove first and last character
'Then stick the data into your Generic List (Of String()?)
Next
End While
public static Encoding GetFileEncoding(String fileName)
{
Encoding Result = null;
FileInfo FI = new FileInfo(fileName);
FileStream FS = null;
try
{
FS = FI.OpenRead();
Encoding[] UnicodeEncodings = { Encoding.BigEndianUnicode, Encoding.Unicode, Encoding.UTF8 };
for (int i = 0; Result == null && i < UnicodeEncodings.Length; i++)
{
FS.Position = 0;
byte[] Preamble = UnicodeEncodings[i].GetPreamble();
bool PreamblesAreEqual = true;
for (int j = 0; PreamblesAreEqual && j < Preamble.Length; j++)
{
PreamblesAreEqual = Preamble[j] == FS.ReadByte();
}
if (PreamblesAreEqual)
{
Result = UnicodeEncodings[i];
}
}
}
catch (System.IO.IOException)
{
}
finally
{
if (FS != null)
{
FS.Close();
}
}
if (Result == null)
{
Result = Encoding.Default;
}
return Result;
}
--Edit-- The current answers have some useful ideas but I want something more complete that I can 100% understand and reuse; that's why I set a bounty. Also ideas that work everywhere are better for me than not standard syntax like \K
This question is about how I can match a pattern except some situations s1 s2 s3. I give a specific example to show my meaning but prefer a general answer I can 100% understand so I can reuse it in other situations.
Example
I want to match five digits using \b\d{5}\b but not in three situations s1 s2 s3:
s1: Not on a line that ends with a period like this sentence.
s2: Not anywhere inside parens.
s3: Not inside a block that starts with if( and ends with //endif
I know how to solve any one of s1 s2 s3 with a lookahead and lookbehind, especially in C# lookbehind or \K in PHP.
For instance
s1 (?m)(?!\d+.*?\.$)\d+
s3 with C# lookbehind (?<!if\(\D*(?=\d+.*?//endif))\b\d+\b
s3 with PHP \K (?:(?:if\(.*?//endif)\D*)*\K\d+
But the mix of conditions together makes my head explode. Even more bad news is that I may need to add other conditions s4 s5 at another time.
The good news is, I don't care if I process the files using most common languages like PHP, C#, Python or my neighbor's washing machine. :) I'm pretty much a beginner in Python & Java but interested to learn if it has a solution.
So I came here to see if someone think of a flexible recipe.
Hints are okay: you don't need to give me full code. :)
Thank you.
Hans, I'll take the bait and flesh out my earlier answer. You said you want "something more complete" so I hope you won't mind the long answer—just trying to please. Let's start with some background.
First off, this is an excellent question. There are often questions about matching certain patterns except in certain contexts (for instance, within a code block or inside parentheses). These questions often give rise to fairly awkward solutions. So your question about multiple contexts is a special challenge.
Surprise
Surprisingly, there is at least one efficient solution that is general, easy to implement and a pleasure to maintain. It works with all regex flavors that allow you to inspect capture groups in your code. And it happens to answer a number of common questions that may at first sound different from yours: "match everything except Donuts", "replace all but...", "match all words except those on my mom's black list", "ignore tags", "match temperature unless italicized"...
Sadly, the technique is not well known: I estimate that in twenty SO questions that could use it, only one has one answer that mentions it—which means maybe one in fifty or sixty answers. See my exchange with Kobi in the comments. The technique is described in some depth in this article which calls it (optimistically) the "best regex trick ever". Without going into as much detail, I'll try to give you a firm grasp of how the technique works. For more detail and code samples in various languages I encourage you to consult that resource.
A Better-Known Variation
There is a variation using syntax specific to Perl and PHP that accomplishes the same. You'll see it on SO in the hands of regex masters such as CasimiretHippolyte and HamZa. I'll tell you more about this below, but my focus here is on the general solution that works with all regex flavors (as long as you can inspect capture groups in your code).
Thanks for all the background, zx81... But what's the recipe?
Key Fact
The method returns the match in Group 1 capture. It does not care at
all about the overall match.
In fact, the trick is to match the various contexts we don't want (chaining these contexts using the | OR / alternation) so as to "neutralize them". After matching all the unwanted contexts, the final part of the alternation matches what we do want and captures it to Group 1.
The general recipe is
Not_this_context|Not_this_either|StayAway|(WhatYouWant)
This will match Not_this_context, but in a sense that match goes into a garbage bin, because we won't look at the overall matches: we only look at Group 1 captures.
In your case, with your digits and your three contexts to ignore, we can do:
s1|s2|s3|(\b\d+\b)
Note that because we actually match s1, s2 and s3 instead of trying to avoid them with lookarounds, the individual expressions for s1, s2 and s3 can remain clear as day. (They are the subexpressions on each side of a | )
The whole expression can be written like this:
(?m)^.*\.$|\([^\)]*\)|if\(.*?//endif|(\b\d+\b)
See this demo (but focus on the capture groups in the lower right pane.)
If you mentally try to split this regex at each | delimiter, it is actually only a series of four very simple expressions.
For flavors that support free-spacing, this reads particularly well.
(?mx)
### s1: Match line that ends with a period ###
^.*\.$
| ### OR s2: Match anything between parentheses ###
\([^\)]*\)
| ### OR s3: Match any if(...//endif block ###
if\(.*?//endif
| ### OR capture digits to Group 1 ###
(\b\d+\b)
This is exceptionally easy to read and maintain.
Extending the regex
When you want to ignore more situations s4 and s5, you add them in more alternations on the left:
s4|s5|s1|s2|s3|(\b\d+\b)
How does this work?
The contexts you don't want are added to a list of alternations on the left: they will match, but these overall matches are never examined, so matching them is a way to put them in a "garbage bin".
The content you do want, however, is captured to Group 1. You then have to check programmatically that Group 1 is set and not empty. This is a trivial programming task (and we'll later talk about how it's done), especially considering that it leaves you with a simple regex that you can understand at a glance and revise or extend as required.
I'm not always a fan of visualizations, but this one does a good job of showing how simple the method is. Each "line" corresponds to a potential match, but only the bottom line is captured into Group 1.
Debuggex Demo
Perl/PCRE Variation
In contrast to the general solution above, there exists a variation for Perl and PCRE that is often seen on SO, at least in the hands of regex Gods such as #CasimiretHippolyte and #HamZa. It is:
(?:s1|s2|s3)(*SKIP)(*F)|whatYouWant
In your case:
(?m)(?:^.*\.$|\([^()]*\)|if\(.*?//endif)(*SKIP)(*F)|\b\d+\b
This variation is a bit easier to use because the content matched in contexts s1, s2 and s3 is simply skipped, so you don't need to inspect Group 1 captures (notice the parentheses are gone). The matches only contain whatYouWant
Note that (*F), (*FAIL) and (?!) are all the same thing. If you wanted to be more obscure, you could use (*SKIP)(?!)
demo for this version
Applications
Here are some common problems that this technique can often easily solve. You'll notice that the word choice can make some of these problems sound different while in fact they are virtually identical.
How can I match foo except anywhere in a tag like <a stuff...>...</a>?
How can I match foo except in an <i> tag or a javascript snippet (more conditions)?
How can I match all words that are not on this black list?
How can I ignore anything inside a SUB... END SUB block?
How can I match everything except... s1 s2 s3?
How to Program the Group 1 Captures
You didn't as for code, but, for completion... The code to inspect Group 1 will obviously depend on your language of choice. At any rate it shouldn't add more than a couple of lines to the code you would use to inspect matches.
If in doubt, I recommend you look at the code samples section of the article mentioned earlier, which presents code for quite a few languages.
Alternatives
Depending on the complexity of the question, and on the regex engine used, there are several alternatives. Here are the two that can apply to most situations, including multiple conditions. In my view, neither is nearly as attractive as the s1|s2|s3|(whatYouWant) recipe, if only because clarity always wins out.
1. Replace then Match.
A good solution that sounds hacky but works well in many environments is to work in two steps. A first regex neutralizes the context you want to ignore by replacing potentially conflicting strings. If you only want to match, then you can replace with an empty string, then run your match in the second step. If you want to replace, you can first replace the strings to be ignored with something distinctive, for instance surrounding your digits with a fixed-width chain of ###. After this replacement, you are free to replace what you really wanted, then you'll have to revert your distinctive ### strings.
2. Lookarounds.
Your original post showed that you understand how to exclude a single condition using lookarounds. You said that C# is great for this, and you are right, but it is not the only option. The .NET regex flavors found in C#, VB.NET and Visual C++ for example, as well as the still-experimental regex module to replace re in Python, are the only two engines I know that support infinite-width lookbehind. With these tools, one condition in one lookbehind can take care of looking not only behind but also at the match and beyond the match, avoiding the need to coordinate with a lookahead. More conditions? More lookarounds.
Recycling the regex you had for s3 in C#, the whole pattern would look like this.
(?!.*\.)(?<!\([^()]*(?=\d+[^)]*\)))(?<!if\(\D*(?=\d+.*?//endif))\b\d+\b
But by now you know I'm not recommending this, right?
Deletions
#HamZa and #Jerry have suggested I mention an additional trick for cases when you seek to just delete WhatYouWant. You remember that the recipe to match WhatYouWant (capturing it into Group 1) was s1|s2|s3|(WhatYouWant), right? To delete all instance of WhatYouWant, you change the regex to
(s1|s2|s3)|WhatYouWant
For the replacement string, you use $1. What happens here is that for each instance of s1|s2|s3 that is matched, the replacement $1 replaces that instance with itself (referenced by $1). On the other hand, when WhatYouWant is matched, it is replaced by an empty group and nothing else — and therefore deleted. See this demo, thank you #HamZa and #Jerry for suggesting this wonderful addition.
Replacements
This brings us to replacements, on which I'll touch briefly.
When replacing with nothing, see the "Deletions" trick above.
When replacing, if using Perl or PCRE, use the (*SKIP)(*F) variation mentioned above to match exactly what you want, and do a straight replacement.
In other flavors, within the replacement function call, inspect the match using a callback or lambda, and replace if Group 1 is set. If you need help with this, the article already referenced will give you code in various languages.
Have fun!
No, wait, there's more!
Ah, nah, I'll save that for my memoirs in twenty volumes, to be released next Spring.
Do three different matches and handle the combination of the three situations using in-program conditional logic. You don't need to handle everything in one giant regex.
EDIT: let me expand a bit because the question just became more interesting :-)
The general idea you are trying to capture here is to match against a certain regex pattern, but not when there are certain other (could be any number) patterns present in the test string. Fortunately, you can take advantage of your programming language: keep the regexes simple and just use a compound conditional. A best practice would be to capture this idea in a reusable component, so let's create a class and a method that implement it:
using System.Collections.Generic;
using System.Linq;
using System.Text.RegularExpressions;
public class MatcherWithExceptions {
private string m_searchStr;
private Regex m_searchRegex;
private IEnumerable<Regex> m_exceptionRegexes;
public string SearchString {
get { return m_searchStr; }
set {
m_searchStr = value;
m_searchRegex = new Regex(value);
}
}
public string[] ExceptionStrings {
set { m_exceptionRegexes = from es in value select new Regex(es); }
}
public bool IsMatch(string testStr) {
return (
m_searchRegex.IsMatch(testStr)
&& !m_exceptionRegexes.Any(er => er.IsMatch(testStr))
);
}
}
public class App {
public static void Main() {
var mwe = new MatcherWithExceptions();
// Set up the matcher object.
mwe.SearchString = #"\b\d{5}\b";
mwe.ExceptionStrings = new string[] {
#"\.$"
, #"\(.*" + mwe.SearchString + #".*\)"
, #"if\(.*" + mwe.SearchString + #".*//endif"
};
var testStrs = new string[] {
"1." // False
, "11111." // False
, "(11111)" // False
, "if(11111//endif" // False
, "if(11111" // True
, "11111" // True
};
// Perform the tests.
foreach (var ts in testStrs) {
System.Console.WriteLine(mwe.IsMatch(ts));
}
}
}
So above, we set up the search string (the five digits), multiple exception strings (your s1, s2 and s3), and then try to match against several test strings. The printed results should be as shown in the comments next to each test string.
Your requirement that it's not inside parens in impossible to satify for all cases.
Namely, if you can somehow find a ( to the left and ) to the right, it doesn't always mean you are inside parens. Eg.
(....) + 55555 + (.....) - not inside parens yet there are ( and ) to left and right
Now you might think yourself clever and look for ( to the left only if you don't encounter ) before and vice versa to the right. This won't work for this case:
((.....) + 55555 + (.....)) - inside parens even though there are closing ) and ( to left and to right.
It is impossible to find out if you are inside parens using regex, as regex can't count how many parens have been opened and how many closed.
Consider this easier task: using regex, find out if all (possibly nested) parens in a string are closed, that is for every ( you need to find ). You will find out that it's impossible to solve and if you can't solve that with regex then you can't figure out if a word is inside parens for all cases, since you can't figure out at a some position in string if all preceeding ( have a corresponding ).
Hans if you don't mind I used your neighbor's washing machine called perl :)
Edited:
Below a pseudo code:
loop through input
if line contains 'if(' set skip=true
if skip= true do nothing
else
if line match '\b\d{5}\b' set s0=true
if line does not match s1 condition set s1=true
if line does not match s2 condition set s2=true
if s0,s1,s2 are true print line
if line contains '//endif' set skip=false
Given the file input.txt:
tiago#dell:~$ cat input.txt
this is a text
it should match 12345
if(
it should not match 12345
//endif
it should match 12345
it should not match 12345.
it should not match ( blabla 12345 blablabla )
it should not match ( 12345 )
it should match 12345
And the script validator.pl:
tiago#dell:~$ cat validator.pl
#! /usr/bin/perl
use warnings;
use strict;
use Data::Dumper;
sub validate_s0 {
my $line = $_[0];
if ( $line =~ \d{5/ ){
return "true";
}
return "false";
}
sub validate_s1 {
my $line = $_[0];
if ( $line =~ /\.$/ ){
return "false";
}
return "true";
}
sub validate_s2 {
my $line = $_[0];
if ( $line =~ /.*?\(.*\d{5.*?\).*/ ){
return "false";
}
return "true";
}
my $skip = "false";
while (<>){
my $line = $_;
if( $line =~ /if\(/ ){
$skip = "true";
}
if ( $skip eq "false" ) {
my $s0_status = validate_s0 "$line";
my $s1_status = validate_s1 "$line";
my $s2_status = validate_s2 "$line";
if ( $s0_status eq "true"){
if ( $s1_status eq "true"){
if ( $s2_status eq "true"){
print "$line";
}
}
}
}
if ( $line =~ /\/\/endif/) {
$skip="false";
}
}
Execution:
tiago#dell:~$ cat input.txt | perl validator.pl
it should match 12345
it should match 12345
it should match 12345
Not sure if this would help you or not, but I am providing a solution considering the following assumptions -
You need an elegant solution to check all the conditions
Conditions can change in future and anytime.
One condition should not depend on others.
However I considered also the following -
The file given has minimal errors in it. If it doe then my code might need some modifications to cope with that.
I used Stack to keep track of if( blocks.
Ok here is the solution -
I used C# and with it MEF (Microsoft Extensibility Framework) to implement the configurable parsers. The idea is, use a single parser to parse and a list of configurable validator classes to validate the line and return true or false based on the validation. Then you can add or remove any validator anytime or add new ones if you like. So far I have already implemented for S1, S2 and S3 you mentioned, check classes at point 3. You have to add classes for s4, s5 if you need in future.
First, Create the Interfaces -
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace FileParserDemo.Contracts
{
public interface IParser
{
String[] GetMatchedLines(String filename);
}
public interface IPatternMatcher
{
Boolean IsMatched(String line, Stack<string> stack);
}
}
Then comes the file reader and checker -
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using FileParserDemo.Contracts;
using System.ComponentModel.Composition.Hosting;
using System.ComponentModel.Composition;
using System.IO;
using System.Collections;
namespace FileParserDemo.Parsers
{
public class Parser : IParser
{
[ImportMany]
IEnumerable<Lazy<IPatternMatcher>> parsers;
private CompositionContainer _container;
public void ComposeParts()
{
var catalog = new AggregateCatalog();
catalog.Catalogs.Add(new AssemblyCatalog(typeof(IParser).Assembly));
_container = new CompositionContainer(catalog);
try
{
this._container.ComposeParts(this);
}
catch
{
}
}
public String[] GetMatchedLines(String filename)
{
var matched = new List<String>();
var stack = new Stack<string>();
using (StreamReader sr = File.OpenText(filename))
{
String line = "";
while (!sr.EndOfStream)
{
line = sr.ReadLine();
var m = true;
foreach(var matcher in this.parsers){
m = m && matcher.Value.IsMatched(line, stack);
}
if (m)
{
matched.Add(line);
}
}
}
return matched.ToArray();
}
}
}
Then comes the implementation of individual checkers, the class names are self explanatory, so I don't think they need more descriptions.
using FileParserDemo.Contracts;
using System;
using System.Collections.Generic;
using System.ComponentModel.Composition;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
namespace FileParserDemo.PatternMatchers
{
[Export(typeof(IPatternMatcher))]
public class MatchAllNumbers : IPatternMatcher
{
public Boolean IsMatched(String line, Stack<string> stack)
{
var regex = new Regex("\\d+");
return regex.IsMatch(line);
}
}
[Export(typeof(IPatternMatcher))]
public class RemoveIfBlock : IPatternMatcher
{
public Boolean IsMatched(String line, Stack<string> stack)
{
var regex = new Regex("if\\(");
if (regex.IsMatch(line))
{
foreach (var m in regex.Matches(line))
{
//push the if
stack.Push(m.ToString());
}
//ignore current line, and will validate on next line with stack
return true;
}
regex = new Regex("//endif");
if (regex.IsMatch(line))
{
foreach (var m in regex.Matches(line))
{
stack.Pop();
}
}
return stack.Count == 0; //if stack has an item then ignoring this block
}
}
[Export(typeof(IPatternMatcher))]
public class RemoveWithEndPeriod : IPatternMatcher
{
public Boolean IsMatched(String line, Stack<string> stack)
{
var regex = new Regex("(?m)(?!\\d+.*?\\.$)\\d+");
return regex.IsMatch(line);
}
}
[Export(typeof(IPatternMatcher))]
public class RemoveWithInParenthesis : IPatternMatcher
{
public Boolean IsMatched(String line, Stack<string> stack)
{
var regex = new Regex("\\(.*\\d+.*\\)");
return !regex.IsMatch(line);
}
}
}
The program -
using FileParserDemo.Contracts;
using FileParserDemo.Parsers;
using System;
using System.Collections.Generic;
using System.ComponentModel.Composition;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace FileParserDemo
{
class Program
{
static void Main(string[] args)
{
var parser = new Parser();
parser.ComposeParts();
var matches = parser.GetMatchedLines(Path.GetFullPath("test.txt"));
foreach (var s in matches)
{
Console.WriteLine(s);
}
Console.ReadLine();
}
}
}
For testing I took #Tiago's sample file as Test.txt which had the following lines -
this is a text
it should match 12345
if(
it should not match 12345
//endif
it should match 12345
it should not match 12345.
it should not match ( blabla 12345 blablabla )
it should not match ( 12345 )
it should match 12345
Gives the output -
it should match 12345
it should match 12345
it should match 12345
Don't know if this would help you or not, I do had a fun time playing with it.... :)
The best part with it is that, for adding a new condition all you have to do is provide an implementation of IPatternMatcher, it will automatically get called and thus will validate.
Same as #zx81's (*SKIP)(*F) but with using a negative lookahead assertion.
(?m)(?:if\(.*?\/\/endif|\([^()]*\))(*SKIP)(*F)|\b\d+\b(?!.*\.$)
DEMO
In python, i would do easily like this,
import re
string = """cat 123 sat.
I like 000 not (456) though 111 is fine
222 if( //endif if(cat==789 stuff //endif 333"""
for line in string.split('\n'): # Split the input according to the `\n` character and then iterate over the parts.
if not line.endswith('.'): # Don't consider the part which ends with a dot.
for i in re.split(r'\([^()]*\)|if\(.*?//endif', line): # Again split the part by brackets or if condition which endswith `//endif` and then iterate over the inner parts.
for j in re.findall(r'\b\d+\b', i): # Then find all the numbers which are present inside the inner parts and then loop through the fetched numbers.
print(j) # Prints the number one ny one.
Output:
000
111
222
333
I have a list of label names in a text file I'd like to manipulate using Find and Replace in Notepad++, they are listed as follows:
MyLabel_01
MyLabel_02
MyLabel_03
MyLabel_04
MyLabel_05
MyLabel_06
I want to rename them in Notepad++ to the following:
Label_A_One
Label_A_Two
Label_A_Three
Label_B_One
Label_B_Two
Label_B_Three
The Regex I'm using in the Notepad++'s replace dialog to capture the label name is the following:
((MyLabel_0)((1)|(2)|(3)|(4)|(5)|(6)))
I want to replace each capture group as follows:
\1 = Label_
\2 = A_One
\3 = A_Two
\4 = A_Three
\5 = B_One
\6 = B_Two
\7 = B_Three
My problem is that Notepad++ doesn't register the syntax of the regex above. When I hit Count in the Replace Dialog, it returns with 0 occurrences. Not sure what's misesing in the syntax. And yes I made sure the Regular Expression radio button is selected. Help is appreciated.
UPDATE:
Tried escaping the parenthesis, still didn't work:
\(\(MyLabel_0\)\((1\)|\(2\)|\(3\)|\(4\)|\(5\)|\(6\)\)\)
Ed's response has shown a working pattern since alternation isn't supported in Notepad++, however the rest of your problem can't be handled by regex alone. What you're trying to do isn't possible with a regex find/replace approach. Your desired result involves logical conditions which can't be expressed in regex. All you can do with the replace method is re-arrange items and refer to the captured items, but you can't tell it to use "A" for values 1-3, and "B" for 4-6. Furthermore, you can't assign placeholders like that. They are really capture groups that you are backreferencing.
To reach the results you've shown you would need to write a small program that would allow you to check the captured values and perform the appropriate replacements.
EDIT: here's an example of how to achieve this in C#
var numToWordMap = new Dictionary<int, string>();
numToWordMap[1] = "A_One";
numToWordMap[2] = "A_Two";
numToWordMap[3] = "A_Three";
numToWordMap[4] = "B_One";
numToWordMap[5] = "B_Two";
numToWordMap[6] = "B_Three";
string pattern = #"\bMyLabel_(\d+)\b";
string filePath = #"C:\temp.txt";
string[] contents = File.ReadAllLines(filePath);
for (int i = 0; i < contents.Length; i++)
{
contents[i] = Regex.Replace(contents[i], pattern,
m =>
{
int num = int.Parse(m.Groups[1].Value);
if (numToWordMap.ContainsKey(num))
{
return "Label_" + numToWordMap[num];
}
// key not found, use original value
return m.Value;
});
}
File.WriteAllLines(filePath, contents);
You should be able to use this easily. Perhaps you can download LINQPad or Visual C# Express to do so.
If your files are too large this might be an inefficient approach, in which case you could use a StreamReader and StreamWriter to read from the original file and write it to another, respectively.
Also be aware that my sample code writes back to the original file. For testing purposes you can change that path to another file so it isn't overwritten.
Bar bar bar - Notepad++ thinks you're a barbarian.
(obsolete - see update below.) No vertical bars in Notepad++ regex - sorry. I forget every few months, too!
Use [123456] instead.
Update: Sorry, I didn't read carefully enough; on top of the barhopping problem, #Ahmad's spot-on - you can't do a mapping replacement like that.
Update: Version 6 of Notepad++ changed the regular expression engine to a Perl-compatible one, which supports "|". AFAICT, if you have a version 5., auto-update won't update to 6. - you have to explicitly download it.
A regular expression search and replace for
MyLabel_((01)|(02)|(03)|(04)|(05)|(06))
with
Label_(?2A_One)(?3A_Two)(?4A_Three)(?5B_One)(?6B_Two)(?7B_Three)
works on Notepad 6.3.2
The outermost pair of brackets is for grouping, they limit the scope of the first alternation; not sure whether they could be omitted but including them makes the scope clear. The pattern searches for a fixed string followed by one of the two-digit pairs. (The leading zero could be factored out and placed in the fixed string.) Each digit pair is wrapped in round brackets so it is captured.
In the replacement expression, the clause (?4A_Three) says that if capture group 4 matched something then insert the text A_Three, otherwise insert nothing. Similarly for the other clauses. As the 6 alternatives are mutually exclusive only one will match. Thus only one of the (?...) clauses will have matched and so only one will insert text.
The easiest way to do this that I would recommend is to use AWK. If you're on Windows, look for the mingw32 precompiled binaries out there for free download (it'll be called gawk).
BEGIN {
FS = "_0";
a[1]="A_One";
a[2]="A_Two";
a[3]="A_Three";
a[4]="B_One";
a[5]="B_Two";
a[6]="B_Three";
}
{
printf("Label_%s\n", a[$2]);
}
Execute on Windows as follows:
C:\Users\Mydir>gawk -f test.awk awk.in
Label_A_One
Label_A_Two
Label_A_Three
Label_B_One
Label_B_Two
Label_B_Three