Boost regex expression capture - c++

My goal is to capture an integer using boost::regex_search.
#define BOOST_REGEX_MATCH_EXTRA
#include <boost\regex.hpp>
#include <iostream>
int main(int argc, char* argv[])
{
std::string tests[4] = {
"SomeString #222",
"SomeString #1",
"SomeString #42",
"SomeString #-1"
};
boost::regex rgx("#(-?[0-9]+)$");
boost::smatch match;
for(int i=0;i< 4; ++i)
{
std::cout << "Test " << i << std::endl;
boost::regex_search(tests[i], match, rgx, boost::match_extra);
for(int j=0; j< match.size(); ++j)
{
std::string match_string;
match_string.assign(match[j].first, match[j].second);
std::cout << " Match " << j << ": " << match_string << std::endl;
}
}
system("pause");
}
I notice that each regex search results in two matches. The first being the string matched, and the second is the capture in parenthesis.
Test 0
Match 0: #222
Match 1: 222
Test 1
Match 0: #1
Match 1: 1
Test 2
Match 0: #42
Match 1: 42
Test 3
Match 0: #-1
Match 1: -1
The documentation discourages use of BOOST_REGEX_MATCH_EXTRA unless needed. Is it required to capture a single match within parentheses, or is there another way?

If you want more speed, perhaps Boost Spirit could bring it, or other Boost Xpressive.
Both will generate code from expression templates. Meaning, among other things, that if you don't "absorb" any attribute values, no cost will be incurred.
Boost Spirit:
This solution is header-only. It can probably be made more efficient, but here's a start:
#include <boost/spirit/include/qi.hpp>
namespace qi = boost::spirit::qi;
int main()
{
std::string const tests[] = {
"SomeString #222",
"SomeString #1",
"SomeString #42",
"SomeString #-1"
};
for(auto& input : tests)
{
int value;
auto f(input.begin()), l(input.end());
if (qi::phrase_parse(f, l, // input iterators
qi::omit [ *~qi::char_('#') ] >> '#' >> qi::int_, // grammar
qi::space, // skipper
value)) // output attribute
{
std::cout << " Input '" << input << "' -> " << value << "\n";
}
}
}
See it Live On Coliru
Boost Xpressive
#include <boost/xpressive/xpressive_static.hpp>
#include <iostream>
namespace xp = boost::xpressive;
int main()
{
std::string const tests[] = {
"SomeString #222",
"SomeString #1",
"SomeString #42",
"SomeString #-1"
};
for(auto& input : tests)
{
static xp::sregex rex = (xp::s1= -*xp::_) >> '#' >> (xp::s2= !xp::as_xpr('-') >> +xp::_d);
xp::smatch what;
if(xp::regex_match(input, what, rex))
{
std::cout << "Input '" << what[0] << " -> " << what[2] << '\n';
}
}
}
See it Live On Coliru too.
I have a hunch that the Spirit solution is gonna be more performant, and close to what you want (because it parses a general grammar and parses it into your desired data-type directly).

Related

boost regex iterator returning empty string

I am a beginner to regex in c++ I was wondering why this code:
#include <iostream>
#include <string>
#include <boost/regex.hpp>
int main() {
std::string s = "? 8==2 : true ! false";
boost::regex re("\\?\\s+(.*)\\s*:\\s*(.*)\\s*\\!\\s*(.*)");
boost::sregex_token_iterator p(s.begin(), s.end(), re, -1); // sequence and that reg exp
boost::sregex_token_iterator end; // Create an end-of-reg-exp
// marker
while (p != end)
std::cout << *p++ << '\n';
}
Prints a empty string. I put the regex in regexTester and it matches the string correctly but here when I try to iterate over the matches it returns nothing.
I think the tokenizer is actually meant to split text by some delimiter, and the delimiter is not included. Compare with std::regex_token_iterator:
std::regex_token_iterator is a read-only LegacyForwardIterator that accesses the individual sub-matches of every match of a regular expression within the underlying character sequence. It can also be used to access the parts of the sequence that were not matched by the given regular expression (e.g. as a tokenizer).
Indeed you invoke exactly this mode as per the docs:
if submatch is -1, then enumerates all the text sequences that did not match the expression re (that is to performs field splitting).
(emphasis mine).
So, just fix that:
for (boost::sregex_token_iterator p(s.begin(), s.end(), re), e; p != e;
++p)
{
boost::sub_match<It> const& current = *p;
if (current.matched) {
std::cout << std::quoted(current.str()) << '\n';
} else {
std::cout << "non matching" << '\n';
}
}
Other Observations
All the greedy Kleene-stars are recipe for trouble. You won't ever find a second match, because the first one's .* at the end will by definition gobble up all remaining input.
Instead, make them non-greedy (.*?) and or much more precise (like isolating some character set, or mandating non-space characters?).
boost::regex re(R"(\?\s+(.*?)\s*:\s*(.*?)\s*\!\s*(.*?))");
// Or, if you don't want raw string literals:
boost::regex re("\\?\\s+(.*?)\\s*:\\s*(.*?)\\s*\\!\\s*(.*?)");
Live Demo
#include <boost/regex.hpp>
#include <iomanip>
#include <iostream>
#include <string>
int main() {
using It = std::string::const_iterator;
std::string const s =
"? 8==2 : true ! false;"
"? 9==3 : 'book' ! 'library';";
boost::regex re(R"(\?\s+(.*?)\s*:\s*(.*?)\s*\!\s*(.*?))");
{
std::cout << "=== regex_search:\n";
boost::smatch results;
for (It b = s.begin(); boost::regex_search(b, s.end(), results, re); b = results[0].end()) {
std::cout << results.str() << "\n";
std::cout << "remain: " << std::quoted(std::string(results[0].second, s.end())) << "\n";
}
}
std::cout << "=== token iteration:\n";
for (boost::sregex_token_iterator p(s.begin(), s.end(), re), e; p != e;
++p)
{
boost::sub_match<It> const& current = *p;
if (current.matched) {
std::cout << std::quoted(current.str()) << '\n';
} else {
std::cout << "non matching" << '\n';
}
}
}
Prints
=== regex_search:
? 8==2 : true !
remain: "false;? 9==3 : 'book' ! 'library';"
? 9==3 : 'book' !
remain: "'library';"
=== token iteration:
"? 8==2 : true ! "
"? 9==3 : 'book' ! "
BONUS: Parser Expressions
Instead of abusing regexen to do parsing, you could generate a parser, e.g. using Boost Spirit:
Live On Coliru
#include <boost/spirit/home/x3.hpp>
#include <boost/fusion/adapted.hpp>
#include <iomanip>
#include <iostream>
namespace x3 = boost::spirit::x3;
int main() {
std::string const s =
"? 8==2 : true ! false;"
"? 9==3 : 'book' ! 'library';";
using expression = std::string;
using ternary = std::tuple<expression, expression, expression>;
std::vector<ternary> parsed;
auto expr_ = x3::lexeme [+(x3::graph - ';')];
auto ternary_ = "?" >> expr_ >> ":" >> expr_ >> "!" >> expr_;
std::cout << "=== parser approach:\n";
if (x3::phrase_parse(begin(s), end(s), *x3::seek[ ternary_ ], x3::space, parsed)) {
for (auto [cond, e1, e2] : parsed) {
std::cout
<< " condition " << std::quoted(cond) << "\n"
<< " true expression " << std::quoted(e1) << "\n"
<< " else expression " << std::quoted(e2) << "\n"
<< "\n";
}
} else {
std::cout << "non matching" << '\n';
}
}
Prints
=== parser approach:
condition "8==2"
true expression "true"
else expression "false"
condition "9==3"
true expression "'book'"
else expression "'library'"
This is much more extensible, will easily support recursive grammars and will be able to synthesize a typed representation of your syntax tree, instead of just leaving you with scattered bits of string.

C++: regex matching code, printing multiple matches?

I'm using the following code which is similar to Stroustrup's C++ 4th Edition Page 127&128. Per output log below, it prints the first match, however not the match for the trailing -XXXX digits.
Does anyone know why the trailing digits are not matched and/or printed??
Thanks
#include <iostream>
#include <regex>
using namespace std;
int main(int argc, char *argv[])
{
// ZIP code pattern: XXddddd-dddd and variants
regex pat (R"(\w{2}\s*\d{5}(−\d{4})?)");
int lineno = 0;
for (string line; getline(cin,line);) {
++lineno;
smatch matches; // matched strings go here
if (regex_search(line, matches, pat)) // search for pat in line
for (auto p : matches) {
cout << p << " ";
}
cout << endl;
// cout << lineno << ": " << matches[0] << '\n';
}
return 0;
}
Output log:
$ ./a.out
AB00000-0000
AB00000
− is not -. That are two different symbols. You have − in the code and - in the input. Here I fixed the code:
#include <iostream>
#include <regex>
using namespace std;
int main(int argc, char *argv[])
{
// ZIP code pattern: XXddddd-dddd and variants
regex pat (R"(\w{2}\s*\d{5}(-\d{4})?)");
int lineno = 0;
for (string line; getline(cin,line);) {
++lineno;
smatch matches; // matched strings go here
if (regex_search(line, matches, pat)) // search for pat in line
for (auto p : matches) {
cout << p << " ";
}
cout << endl;
// cout << lineno << ": " << matches[0] << '\n';
}
return 0;
}

MSVC regular expression match

I am trying to match a literal number, e.g. 1600442 using a set of regular expressions in Microsoft Visual Studio 2010. My regular expressions are simply:
1600442|7654321
7895432
The problem is that both of the above matches the string.
Implementing this in Python gives the expected result:
import re
serial = "1600442"
re1 = "1600442|7654321"
re2 = "7895432"
m = re.match(re1, serial)
if m:
print "found for re1"
print m.groups()
m = re.match(re2, serial)
if m:
print "found for re2"
print m.groups()
Gives output
found for re1
()
Which is what I expected. Using this code in C++ however:
#include <string>
#include <iostream>
#include <regex>
int main(){
std::string serial = "1600442";
std::tr1::regex re1("1600442|7654321");
std::tr1::regex re2("7895432");
std::tr1::smatch match;
std::cout << "re1:" << std::endl;
std::tr1::regex_search(serial, match, re1);
for (auto i = 0;i <match.length(); ++i)
std::cout << match[i].str().c_str() << " ";
std::cout << std::endl << "re2:" << std::endl;
std::tr1::regex_search(serial, match, re2);
for (auto i = 0;i <match.length(); ++i)
std::cout << match[i].str().c_str() << " ";
std::cout << std::endl;
std::string s;
std::getline (std::cin,s);
}
gives me:
re1:
1600442
re2:
1600442
which is not what I expected. Why do I get match here?
The smatch does not get overwritten by the second call to regex_search thus, it is left intact and contains the first results.
You can move the regex searching code to a separate method:
void FindMeText(std::regex re, std::string serial)
{
std::smatch match;
std::regex_search(serial, match, re);
for (auto i = 0;i <match.length(); ++i)
std::cout << match[i].str().c_str() << " ";
std::cout << std::endl;
}
int main(){
std::string serial = "1600442";
std::regex re1("^(?:1600442|7654321)");
std::regex re2("^7895432");
std::cout << "re1:" << std::endl;
FindMeText(re1, serial);
std::cout << "re2:" << std::endl;
FindMeText(re2, serial);
std::cout << std::endl;
std::string s;
std::getline (std::cin,s);
}
Result:
Note that Python re.match searches for the pattern match at the start of string only, thus I suggest using ^ (start of string) at the beginning of each pattern.

This regex doesn't work in c++

It is supposed to match "abababab" since "ab" is repeated more than two times consecutively but the code isn't printing any output.
Is there some other trick in using regex in C++.
I tried with other languages and it works just fine.
#include<bits/stdc++.h>
int main(){
std::string s ("xaxababababaxax");
std::smatch m;
std::regex e ("(.+)\1\1+");
while (std::regex_search (s,m,e)) {
for (auto x:m) std::cout << x << " ";
std::cout << std::endl;
s = m.suffix().str();
}
return 0;
}
Your problem is your backslashes are escaping the '1''s in your string. You need to inform std::regex to treat them as '\' 's. You can do this by using a raw string R"((.+)\1\1+)", or by escaping the slashes, as shown here:
#include <regex>
#include <string>
#include <iostream>
int main(){
std::string s ("xaxababababaxax");
std::smatch m;
std::regex e ("(.+)\\1\\1+");
while (std::regex_search (s,m,e)) {
for (auto x:m) std::cout << x << " ";
std::cout << std::endl;
s = m.suffix().str();
}
return 0;
}
Which produces the output
abababab ab

How to match multiple results using std::regex

For example, If I have a string like "first second third forth" and I want to match every single word in one operation to output them one by one.
I just thought that "(\\b\\S*\\b){0,}" would work. But actually it did not.
What should I do?
Here's my code:
#include<iostream>
#include<string>
using namespace std;
int main()
{
regex exp("(\\b\\S*\\b)");
smatch res;
string str = "first second third forth";
regex_search(str, res, exp);
cout << res[0] <<" "<<res[1]<<" "<<res[2]<<" "<<res[3]<< endl;
}
Simply iterate over your string while regex_searching, like this:
{
regex exp("(\\b\\S*\\b)");
smatch res;
string str = "first second third forth";
string::const_iterator searchStart( str.cbegin() );
while ( regex_search( searchStart, str.cend(), res, exp ) )
{
cout << ( searchStart == str.cbegin() ? "" : " " ) << res[0];
searchStart = res.suffix().first;
}
cout << endl;
}
This can be done in regex of C++11.
Two methods:
You can use () in regex to define your captures(sub expressions).
Like this:
string var = "first second third forth";
const regex r("(.*) (.*) (.*) (.*)");
smatch sm;
if (regex_search(var, sm, r)) {
for (int i=1; i<sm.size(); i++) {
cout << sm[i] << endl;
}
}
See it live: http://coliru.stacked-crooked.com/a/e1447c4cff9ea3e7
You can use sregex_token_iterator():
string var = "first second third forth";
regex wsaq_re("\\s+");
copy( sregex_token_iterator(var.begin(), var.end(), wsaq_re, -1),
sregex_token_iterator(),
ostream_iterator<string>(cout, "\n"));
See it live: http://coliru.stacked-crooked.com/a/677aa6f0bb0612f0
sregex_token_iterator appears to be the ideal, efficient solution, but the example given in the selected answer leaves much to be desired. Instead, I found some great examples here:
http://www.cplusplus.com/reference/regex/regex_token_iterator/regex_token_iterator/
For your convenience, I've copy-pasted the sample code shown by that page. I claim no credit for the code.
// regex_token_iterator example
#include <iostream>
#include <string>
#include <regex>
int main ()
{
std::string s ("this subject has a submarine as a subsequence");
std::regex e ("\\b(sub)([^ ]*)"); // matches words beginning by "sub"
// default constructor = end-of-sequence:
std::regex_token_iterator<std::string::iterator> rend;
std::cout << "entire matches:";
std::regex_token_iterator<std::string::iterator> a ( s.begin(), s.end(), e );
while (a!=rend) std::cout << " [" << *a++ << "]";
std::cout << std::endl;
std::cout << "2nd submatches:";
std::regex_token_iterator<std::string::iterator> b ( s.begin(), s.end(), e, 2 );
while (b!=rend) std::cout << " [" << *b++ << "]";
std::cout << std::endl;
std::cout << "1st and 2nd submatches:";
int submatches[] = { 1, 2 };
std::regex_token_iterator<std::string::iterator> c ( s.begin(), s.end(), e, submatches );
while (c!=rend) std::cout << " [" << *c++ << "]";
std::cout << std::endl;
std::cout << "matches as splitters:";
std::regex_token_iterator<std::string::iterator> d ( s.begin(), s.end(), e, -1 );
while (d!=rend) std::cout << " [" << *d++ << "]";
std::cout << std::endl;
return 0;
}
Output:
entire matches: [subject] [submarine] [subsequence]
2nd submatches: [ject] [marine] [sequence]
1st and 2nd submatches: [sub] [ject] [sub] [marine] [sub] [sequence]
matches as splitters: [this ] [ has a ] [ as a ]
You could use the suffix() function, and search again until you don't find a match:
int main()
{
regex exp("(\\b\\S*\\b)");
smatch res;
string str = "first second third forth";
while (regex_search(str, res, exp)) {
cout << res[0] << endl;
str = res.suffix();
}
}
My code will capture all groups in all matches:
vector<vector<string>> U::String::findEx(const string& s, const string& reg_ex, bool case_sensitive)
{
regex rx(reg_ex, case_sensitive ? regex_constants::icase : 0);
vector<vector<string>> captured_groups;
vector<string> captured_subgroups;
const std::sregex_token_iterator end_i;
for (std::sregex_token_iterator i(s.cbegin(), s.cend(), rx);
i != end_i;
++i)
{
captured_subgroups.clear();
string group = *i;
smatch res;
if(regex_search(group, res, rx))
{
for(unsigned i=0; i<res.size() ; i++)
captured_subgroups.push_back(res[i]);
if(captured_subgroups.size() > 0)
captured_groups.push_back(captured_subgroups);
}
}
captured_groups.push_back(captured_subgroups);
return captured_groups;
}
My reading of the documentation is that regex_search searches for the first match and that none of the functions in std::regex do a "scan" as you are looking for. However, the Boost library seems to be support this, as described in C++ tokenize a string using a regular expression