Regex to match strings not enclosed in macro - regex

In a development context, I would like to make sure all strings in source files within certain directories are enclosed in some macro "STR_MACRO". For this I will be using a Python script parsing the source files, and I would like to design a regex for detecting non-commented lines with strings not enclosed in this macro.
For instance, the regex should match the following strings:
std::cout << "Hello World!" << std::endl;
load_file("Hello World!");
But not the following ones:
std::cout << STR_MACRO("Hello World!") << std::endl;
load_file(STR_MACRO("Hello World!"));
// "foo" bar
Excluding commented lines containing strings seems to work well using the regex ^(?!\s*//).*"([^"]+)". However when I try to exclude non-commented strings already enclosed in the macro, using the regex ^(?!\s*//).*(?!STR_MACRO\()"([^"]+)", it does nothing more (seemingly due to with the opening parenthesis after STR_MACRO).
Any hints on how to achieve this?

With PyPi regex module (that you can install with pip install regex in the terminal) you can use
import regex
pattern = r'''(?:^//.*|STR_MACRO\("[^"\\]*(?:\\.[^"\\]*)*"\))(*SKIP)(*F)|"[^"\\]*(?:\\.[^"\\]*)*"'''
text = r'''For instance, the regex should match the following strings:
std::cout << "Hello World!" << std::endl;
load_file("Hello World!");
But not the following ones:
std::cout << STR_MACRO("Hello World!") << std::endl;
load_file(STR_MACRO("Hello World!"));
// "foo" bar'''
print( regex.sub(pattern, r'STR_MACRO(\g<0>)', text, flags=regex.M) )
Details:
(?:^//.*|STR_MACRO\("[^"\\]*(?:\\.[^"\\]*)*"\))(*SKIP)(*F) - // at the line start and the rest of the line, or STR_MACRO( + a double quoted string literal pattern + ), and then the match is skipped, and the next match search starts at the failure location
| - or
"[^"\\]*(?:\\.[^"\\]*)*" - ", zero or more chars other than " and \, then zero or more reptitions of a \ and then any single char followed with zero or more chars other than a " and \ chars, and then a " char
See the Python demo. Output:
For instance, the regex should match the following strings:
std::cout << STR_MACRO("Hello World!") << std::endl;
load_file(STR_MACRO("Hello World!"));
But not the following ones:
std::cout << STR_MACRO("Hello World!") << std::endl;
load_file(STR_MACRO("Hello World!"));
// "foo" bar

Related

Find regex matches & remove outer part of the match

I have a string
content = "std::cout << func(some_val) << std::endl; auto i = func(some_other_val);"
and I find to find all instances with func(...), and remove the function call. So that I would get
content = "std::cout << some_val << std::endl; auto i = some_other_val;"
So I've tried this:
import re
content = "std::cout << func(some_val) << std::endl; auto i = func(some_other_val);"
c = re.compile('func\([a-zA-Z0-9_]+\)')
print(c.sub('', content)) # gives "std::cout << << std::endl; auto i = ;"
but this removes the entire match, not just the func( and ).
Basically, how do I keep whatever matched with [a-zA-Z0-9_]+?
You can use re.sub to replace all the outer func(...) with only the value like below, See regex here , Here I've used [w]+, you can do changes if you use
import re
regex = r"func\(([\w]+)\)"
test_str = "std::cout << func(some_val) << std::endl; auto i = func(some_other_val);"
subst = "\\1"
result = re.sub(regex, subst, test_str, 0, re.MULTILINE)
if result:
print (result)
Demo: https://rextester.com/QZJLF65281
Output:
std::cout << some_val << std::endl; auto i = some_other_val;
You should capture the part of the match that you want to keep into a group:
re.compile(r'func\(([a-zA-Z0-9_]+)\)')
Here I captured it into group 1.
And then you can refer to group 1 with \1:
print(c.sub(r'\1', content))
Note that in general, you should not use regex to parse source code of a non-regular language (such as C in this case) with regex. It might work in a few very specific cases, where the input is very limited, but you should still use a C parser to parse C code. I have found libraries such as this and this.

How to create a regex that matches alpha chars up to, but not including, whitespace?

With this input:
"hello the3re world"
I'm trying to create a regex that will match words containing just alpha characters and not digits.
I'm using std::regex_search with flag std::regex_constants::match_continuous.
With this regex [[:alpha:]]+ the first call to regex_search will give me back "hello". If I then advance over "hello" and any white space and try again with "the3re world", I get back: "the".
But, what I really want at this point is a failure to match as the "3" shouldn't be valid in a word.
(Adding my comment as an answer) You should use word boundaries \b for this purpose..
\b[[:alpha:]]+\b //or "\\b[[:alpha:]]+\\b" as per the syntax..
You can use the following code:
string line1 = "hello the3re world";
string regexStr1 = "\\b[a-zA-Z]+\\b";
regex rg1(regexStr1);
smatch sm1;
while (regex_search(line1, sm1, rg1)) {
std::cout << sm1[0] << std::endl;
line1 = sm1.suffix().str();
}
Output:

Regex in C++11 vs PHP

I'm new to regex and C++11. In order to match an expression like this :
TYPE SIZE NUMBER ("regina s x99");
I built a regex which looks like this one :
\b(regina|margarita|americaine|fantasia)\b \b(s|l|m|xl|xxl)\b x([1-9])([0-9])
In my code I did this to try the regex :
std::string s("regina s x99");
std::regex rgx($RGX); //$RGX corresponds to the regex above
if (std::regex_match(s, rgx))
std::cout << "It works !" << std::endl;
This code throw a std::regex_error, but I don't know where it comes from..
Thanks,
This works with g++ (4.9.2) in c++11 mode:
std::regex rgx("\\b(regina|margarita|americaine|fantasia)\\b\\s*(s|l|m|xl|xxl)\\b\\s*x([1-9]*[0-9])");
This will capture three groups: regina s 99 which matches the TYPE SIZE NUMBER pattern, while your original captured four groups regina s 9 9 and had the NUMBER as two values (maybe that was what you wanted though).
Demo on IdeOne
In C++ strings the \ character is special and needs to be escaped so that it gets passed to the regular expression engine, not interpreted by the compiler.
So you either need to use \\b:
std::regex rgx("\\b(regina|margarita|americaine|fantasia)\\b \\b(s|l|m|xl|xxl)\\b x([1-9])([0-9])");
or use a raw string, which means that \ is not special and doesn't need to be escaped:
std::regex rgx(R"(\b(regina|margarita|americaine|fantasia)\b \b(s|l|m|xl|xxl)\b x([1-9])([0-9]))");
There was a typo in this line in original question:
if (std::reegex_match(s, rgx))
More over I am not sure what are you passing with this variable : $RGX
Corrected program as follows:
#include<regex>
#include<iostream>
int main()
{
std::string s("regina s x99");
std::regex rgx("\\b(regina|margarita|americaine|fantasia)\\b \\s*(s|l|m|xl|xxl)\\b \\s*x([1-9])([0-9])"); //$RGX corresponds to the regex above
if (std::regex_match(s, rgx))
std::cout << "It works !" << std::endl;
else
std::cout<<"No Match"<<std::endl;
}

QRegExp not extracting text as expected

I am trying to extract text from between square brackets on a line of text. I've been messing with the regex for some time now, and cannot get what I need. (I can't even explain why the output is what it is). Here's the code:
QRegExp rx_timestamp("\[(.*?)\]");
int pos = rx_timestamp.indexIn(line);
if (pos > -1) {
qDebug() << "Captured texts: " << rx_timestamp.capturedTexts();
qDebug() << "timestamp cap: " <<rx_timestamp.cap(0);
qDebug() << "timestamp cap: " <<rx_timestamp.cap(1);
qDebug() << "timestamp cap: " <<rx_timestamp.cap(2);
} else qDebug() << "No indexin";
The input line is:
messages:[2013-10-08 09:13:41] NOTICE[2366] chan_sip.c: Registration from '"xx000 <sip:xx000#183.229.164.42:5060>' failed for '192.187.100.170' - No matching peer found
And the output is:
Captured texts: (".")
timestamp cap: "."
timestamp cap: ""
timestamp cap: ""
Can someone explain what is going on? Why is cap returning "." when no such character exists between square brackets
Can someone correct the regex to extract the timestamp from between the square brackets?
You are missing two things. Escaping the backslash, and using setMinimal. See below.
QString line = "messages:[2013-10-08 09:13:41] NOTICE[2366] chan_sip.c: Registration from '\"xx000 <sip:xx000#183.229.164.42:5060>' failed for '192.187.100.170' - No matching peer found";
QRegExp rx_timestamp("\\[(.*)\\]");
rx_timestamp.setMinimal(true);
int pos = rx_timestamp.indexIn(line);
if (pos > -1) {
qDebug() << "Captured texts: " << rx_timestamp.capturedTexts();
qDebug() << "timestamp cap: " <<rx_timestamp.cap(0);
qDebug() << "timestamp cap: " <<rx_timestamp.cap(1);
qDebug() << "timestamp cap: " <<rx_timestamp.cap(2);
} else qDebug() << "No indexin";
Output:
Captured texts: ("[2013-10-08 09:13:41]", "2013-10-08 09:13:41")
timestamp cap: "[2013-10-08 09:13:41]"
timestamp cap: "2013-10-08 09:13:41"
timestamp cap: ""
UPDATE: What is going on:
A backslash in c++ source code indicates that the next character is an escape character, such as \n. To have a backslash show up in a regular expression you have to escape a backslash like so: \\ That will make it so that the Regular Expression engine sees \, like what Ruby, Perl or Python would use.
The square brackets should be escaped, too, because they are used to indicate a range of elements normally in regex.
So for the Regular expression engine to see a square bracket character you need to send it
\[
but a c++ source file can't get a \ character into a string without two of them in a row so it turns into
\\[
While learning regex, I liked using this regex tool by GSkinner. It has a listing on the right hand side of the page of unique codes and characters.
QRegEx doesn't match regex exactly. If you study the documentation you find a lot of little things. Such as how it does Greedy v. Lazy matching.
QRegExp and double-quoted text for QSyntaxHighlighter
How the captures are listed is pretty typical as far as I have seen from regex parsers. The capture listing first lists all of them, then it lists the first capture group (or what was enclosed by the first set of parentheses.
http://qt-project.org/doc/qt-5.0/qtcore/qregexp.html#cap
http://qt-project.org/doc/qt-5.0/qtcore/qregexp.html#capturedTexts
To find more matches, you have to iteratively call indexIn.
http://qt-project.org/doc/qt-5.0/qtcore/qregexp.html#indexIn
QString str = "offsets: 1.23 .50 71.00 6.00";
QRegExp rx("\\d*\\.\\d+"); // primitive floating point matching
int count = 0;
int pos = 0;
while ((pos = rx.indexIn(str, pos)) != -1) {
++count;
pos += rx.matchedLength();
}
// pos will be 9, 14, 18 and finally 24; count will end up as 4
Hope that helps.

Regular expression validation fails while egrep validates just fine

I'm trying to use regular expressions in order to validate strings so before I go any further let me explain first how the strings looks like: optional number of digits followed by an 'X' and an optional ('^' followed by one or more digits).
Here are some exmaples: "2X", "X", "23X^6" fit the pattern while strings like "X^", "4", "foobar", "4X^", "4X44" don't.
Now where was I: using 'egrep' and the "^[0-9]{0,}\X(\^[0-9]{1,})$" regex I can validate just fine those strings however when trying this in C++ using the C++11 regex library it fails.
Here's the code I'm using to validate those strings:
#include <iostream>
#include <regex>
#include <string>
#include <vector>
int main()
{
std::regex r("^[0-9]{0,}\\X(\\^[0-9]{1,})$",
std::regex_constants::egrep);
std::vector<std::string> challanges_ok {"2X", "X", "23X^66", "23X^6",
"3123X", "2313131X^213213123"};
std::vector<std::string> challanges_bad {"X^", "4", "asdsad", " X",
"4X44", "4X^"};
std::cout << "challanges_ok: ";
for (auto &str : challanges_ok) {
std::cout << std::regex_match(str, r) << " ";
}
std::cout << "\nchallanges_bad: ";
for (auto &str : challanges_bad) {
std::cout << std::regex_match(str, r) << " ";
}
std::cout << "\n";
return 0;
}
Am I doing something wrong or am I missing something? I'm compiling under GCC 4.7.
Your regex fails to make the '^' followed by one or more digits optional; change it to:
"^[0-9]*X(\\^[0-9]+)?$".
Also note that this page says that GCC's support of <regex> is only partial, so std::regex may not work at all for you ('partial' in this context apparently means 'broken'); have you tried Boost.Xpressive or Boost.Regex as a sanity check?
optional number of digits followed by an 'X' and an optional ('^' followed by one or more digits).
OK, the regular expression in your code doesn't match that description, for two reasons: you have an extra backslash on the X, and the '^digits' part is not optional. The regex you want is this:
^[0-9]{0,}X(\^[0-9]{1,}){0,1}$
which means your grep command should look like this (note single quotes):
egrep '^[0-9]{0,}X(\^[0-9]{1,}){0,1}$' filename
And the string you have to pass in your C++ code is this:
"^[0-9]{0,}X(\\^[0-9]{1,}){0,1}$"
If you then replace all the explicit quantifiers with their more traditional abbreviations, you get #ildjarn's answer: {0,} is *, {1,} is +, and {0,1} is ?.