Capturing output with Perl, until a specific pattern is found - regex

Question

I feel like I am missing something very simple here, but this is the first time I've needed to do this and am having trouble finding an example.
I have a giant foreach loop that walks through output logs and extracts various bits of information based on matching regular expressions. My main problem is that a few larger types of output have a header and footer, like *** Begin bangle tracking log*** followed by several lines of gibberish and then a ***End bangle tracking log***.
Is there a way, from within a foreach loop, to have an inner loop that stores all the lines until a footer is found?
foreach my $line( #parseme )
{
if( $line =~ m/***Begin bangle tracking log***/ )
{
#Help! Push all lines into an array until bangle tracking footer is found.
}
if( $line =~ m/Other stuff I am tracking/ )
{
#Do other things
}
}

Answer 1

You could use the range operator, which acts as a flip-flop in scalar context:
foreach ( #parseme ) {
if ( /Begin bangle tracking log/ .. /End bangle tracking log/ ) {
push #array, $_;
}
# other stuff...
}
I used $_ for the foreach loop because it allows for more concise syntax. You can use another variable if you like, but then you'll have to write the condition as something like:
if ( $line =~ /Begin .../ .. $line =~ /End .../ ) {
which might be more readable with some extra parentheses:
if ( ($line =~ /Begin .../) .. ($line =~ /End .../) ) {
One issue to note about the flip-flop operator is that it remembers its state even after the loop ends. This means that, if you intend to run the loop again, you really ought to make sure that the #parseme array ends with a line that matches the /End .../ regexp, so that the flip-flop will be in a known state when the loop starts the next time.
Edit: Per DVK's comment below, if you want to process the collected lines as soon as you reach the footer line, you can do that by checking the return value of the .. operator, which will end with E0 on the last line:
foreach ( #parseme ) {
my $in_block = /Begin bangle tracking log/ .. /End bangle tracking log/;
if ( $in_block ) {
push #array, $_;
}
if ( $in_block =~ /E0$/ ) { # last line
# process the lines in #array
#array = ();
}
# other stuff...
}

Answer 2

You can do that somewhat easily by implementing a primitive state machine:
my $inside_bangle = 0; # 0=outside block, 1=inside
my #buffer;
foreach my $line( #parseme ) {
if ($line =~ m/***Begin bangle tracking log***/ ) {
$inside_bangle = 1;
next;
}
if ($line =~ m/***End bangle tracking log***/ ) {
$inside_bangle = 0;
# PROCESS #buffer somehow
next;
}
if ($inside_bangle) {
push #buffer, $line;
next;
}
if ($line =~ m/other stuff i am tracking/ ) {
#Do other things
}
}
Another option is to use flip-flop (..)

Answer 3

You're probably looking for the .. operator, which has some magical properties when applied with regular expressions. The following example is stolen from PLEAC:
while (<>) {
if (/BEGIN PATTERN/ .. /END PATTERN/) {
# line falls between BEGIN and END in the
# text, inclusive.
}
}
Within the block, append to your array variable as you see fit.

Answer 4

And now you can store multiple instances of your log snippet, if it occurs more than once (DVK's original code):
my $inside_bangle = 0; # 0=outside block, 1=inside
my %buffer;
my $index = 0;
foreach my $line( #parseme ) {
if ($line =~ m/***Begin bangle tracking log***/ ) {
$inside_bangle = 1;
next;
}
if ($line =~ m/***End bangle tracking log***/ ) {
$inside_bangle = 0;
$index++;
# PROCESS #buffer somehow
next;
}
if ($inside_bangle) {
push #{ $buffer{$index} }, $line;
next;
}
if ($line =~ m/other stuff i am tracking/ ) {
#Do other things
}
}
Here's what I wrote initially but I thought DVK's code is more readable and neat:
open FILE, "<", 'testfile.log';
#parseme = <FILE>;
my $initialize = shift #parseme;
my $startLogging = $initialize =~ m/^Log Start$/ ? 1 : 0; # test if first line of array is start of log
my %storage = ();
my $index = 0;
foreach my $line (#parseme) {
$startLogging = 1 if $line =~ m/^Log Start$/;
if ($startLogging == 1) {
push( #{ $storage{$index} }, $line ) unless $line =~ m/(Log Start|Log End)$/;
if ($line =~ m/^Log End$/) {
$startLogging = 0;
$index++;
}
}
}