Sliding window pattern match in perl or matlab regular expressions - regex

Question

I am trying to use either Perl or MATLAB to parse a few numbers out of a single line of text. My text line is:
t10_t20_t30_t40_
now in matlab, i used the following script
str = 't10_t20_t30_t40_';
a = regexp(str,'t(\d+)_t(\d+)','match')
and it returns
a =
't10_t20' 't30_t40'
What I want is for it to also return 't20_t30', since this obviously is a match. Why doesn't regexp scan it?
I thus turned to Perl, and wrote the following in Perl:
#!/usr/bin/perl -w
$str = "t10_t20_t30_t40_";
while($str =~ /(t\d+_t\d+)/g)
{
print "$1\n";
}
and the result is the same as matlab
t10_t20
t30_t40
but I really wanted "t20_t30" also be in the results.
Can anyone tell me how to accomplish that? Thanks!
[update with a solution]:
With help from colleagues, I identified a solution using the so-called "look-around assertion" afforded by Perl.
#!/usr/bin/perl -w
$str = "t10_t20_t30_t40_";
while($str =~ m/(?=(t\d+_t\d+))/g)
{print "$1\n";}
The key is to use "zero width look-ahead assertion" in Perl. When Perl (and other similar packages) uses regexp to scan a string, it does not re-scan what was already scanned in the last match. So in the above example, t20_t30 will never show up in the results. To capture that, we need to use a zero-width lookahead search to scan the string, producing matches that do not exclude any substrings from subsequent searches (see the working code above). The search will start from zero-th position and increment by one as many times as possible if "global" modifier is appended to the search (i.e. m//g), making it a "greedy" search.
This is explained in more detail in this blog post.
The expression (?=t\d+_t\d+) matches any 0-width string followed by t\d+_t\d+, and this creates the actual "sliding window". This effectively returns ALL t\d+_t\d+ patterns in $str without any exclusion since every position in $str is a 0-width string. The additional parenthesis captures the pattern while its doing sliding matching (?=(t\d+_t\d+)) and thus returns the desired sliding window outcome.

Answer 1

Using Perl:
#!/usr/bin/perl
use Data::Dumper;
use Modern::Perl;
my $re = qr/(?=(t\d+_t\d+))/;
my #l = 't10_t20_t30_t40' =~ /$re/g;
say Dumper(\#l);
Output:
$VAR1 = [
't10_t20',
't20_t30',
't30_t40'
];

Answer 2

Once the regexp algorithm has found a match, the matched characters are not considered for further matches (and usually, this is what one wants, e.g. .* is not supposed to match every conceivable contiguous substring of this post). A workaround would be to start the search again one character after the first match, and collect the results:
str = 't10_t20_t30_t40_';
sub_str = str;
reg_ex = 't(\d+)_t(\d+)';
start_idx = 0;
all_start_indeces = [];
all_end_indeces = [];
off_set = 0;
%// While there are matches later in the string and the first match of the
%// remaining string is not the last character
while ~isempty(start_idx) && (start_idx < numel(str))
%// Calculate offset to original string
off_set = off_set + start_idx;
%// extract string starting at first character after first match
sub_str = sub_str((start_idx + 1):end);
%// find further matches
[start_idx, end_idx] = regexp(sub_str, reg_ex, 'once');
%// save match if any
if ~isempty(start_idx)
all_start_indeces = [all_start_indeces, start_idx + off_set];
all_end_indeces = [all_end_indeces, end_idx + off_set];
end
end
display(all_start_indeces)
display(all_end_indeces)
matched_strings = arrayfun(#(st, en) str(st:en), all_start_indeces, all_end_indeces, 'uniformoutput', 0)