Text Pattern Processing in paragraph with unix linux utilities - regex

Question

I have a file with the following pattern (please note this is a file generated using sed,
awk, grep etc processing). The part of file input is as follows.
filename1,
BASE=a/b/c
CONFIG=$BASE/d
propertiesfile1=$CONFIG/e.properties
EndOfFilefilename1
filename2,
BASE=f/g/h
CONFIG=$BASE/i
propertiesfile1=$CONFIG/j.properties
EndOfFilefilename2
filename3,
BASE=k/l/m
CONFIG=$BASE/n
propertiesfile1=$CONFIG/o.properties
EndOfFilefilename3
I want the output like
filename1,a/b/c/d/e.properties,
filename2,f/g/h/i/j.properties,
filename3, k/l/m/n/o.properties,
I could not find a solution with sed or awk or grep. So I ams tuck. Please do let me know if you know the solution with these unix utilities or any other language, platform.
Regards,
Suhaas

Answer 1

Assuming you generated the original file, and therefore it is safe to execute it as a script:
sed -e 's/^.*,/FILE=&/' \
-e 's/^.*=\$CONFIG/PROPFILE=$CONFIG/' \
-e 's/^EndOfFile.*/echo $FILE $PROPFILE/' < yourInputFile | sh
This converts each section of your file into the form:
FILE=filename1,
BASE=a/b/c
CONFIG=$BASE/d
PROPFILE=$CONFIG/e.properties
echo $FILE $PROPFILE
... and then sends it into a shell for processing.
Line-by-line explanation:
Line 1: Searches for the lines ending in a comma (the filenames), and sets FILE to the name.
Line 2: Searches for lines that set the properties file, and renames the variable to PROPFILE.
Line 3: Replaces the EndOfFile lines with a command to echo the file name and the properties file, then pipes it into a shell.

Answer 2

This is an excellent use case for structural regular expressions, which have been implemented as a python library, amongst other places. Here's an article which descibes how to emulate SREs in Perl.

Answer 3

And here is an awk script to process that input and generate what you want:
BEGIN {
FS="="
state = 0;
base = "";
config = "";
prop = "";
filename = "";
dbg = 0;
}
/^BASE=/ {
if (dbg) {
print "BASE";
print $0;
}
if (state != 1) {
print "Error base!";
exit 1;
}
state++;
base = $2;
if (dbg > 1) printf ("BASE = %s\n", base);
}
/^CONFIG=/ {
if (dbg) {
print "CONFIG";
print $0;
}
if (state != 2) {
print "Error config!";
exit 1;
}
state++;
config = $2;
sub (/\$BASE/, base, config);
if (dbg > 1) printf ("CONFIG = %s\n", config);
}
/^propertiesfile1=/ {
if (dbg) {
print "PROP";
print $0;
}
if (state != 3) {
print "Error pF!";
exit 1;
}
state++;
prop = $2;
sub (/\$CONFIG/, config, prop);
}
/^EndOfFile/ {
if (dbg) {
print "EOF";
print $0;
}
if (state != 4) {
print "Error EOF!";
print state;
exit 1;
}
state = 0;
printf ("%s%s,\n", filename, prop);
}
/,$/{
if (dbg) {
print "FILENAME";
print $0;
}
if (state != 0) {
print "Error filename!";
print state;
exit 1;
}
state++;
filename = $1;
}

Answer 4

gawk
gawk -vRS= 'BEGIN{FS="BASE[=]?|CONFIG|\n"}
{
s=$1
for(i=1;i<=NF;i++){
if($i~/\// ){ s=s $i }
}
print s
s=""
}' file
output
$ more file
filename1,
BASE=a/b/c
CONFIG=$BASE/d
propertiesfile1=$CONFIG/e.properties
EndOfFilefilename1
filename2,
BASE=f/g/h
CONFIG=$BASE/i
propertiesfile1=$CONFIG/j.properties
EndOfFilefilename2
filename3,
BASE=k/l/m
CONFIG=$BASE/n
propertiesfile1=$CONFIG/o.properties
EndOfFilefilename3
$ ./shell.sh
filename1,a/b/c/d/e.properties
filename2,f/g/h/i/j.properties
filename3,k/l/m/n/o.properties

Answer 5

A perl script that does what you want would be something like (note this is untested)
while (<>) {
$base = $1 if (m/BASE=(.+)/);
$config = $1 if (m/CONFIG=(.+)/);
if (m/propertiesfile1=(.+)/) {
$props = $1;
$props =~ m/\$CONFIG/$config/;
$props =~ m/\$BASE/$base/;
print $ARGV . ", " . $props . "\n";
}
}
you give the script the filenames as arguments.

Answer 6

Multi-steps but it works!
cat yourInputFile | egrep ',|\/' | \
sed -e "s/^.*=//g" -e "s/\$.*\(\/.*\)/\1/g" | \
awk '{if($0 ~ "properties") print $0; else printf $0}'
The egrep grabs the lines containing a "," or a "/" and so eliminates the last line:
BASE=a/b/c
CONFIG=$BASE/d
propertiesfile1=$CONFIG/e.properties
The sed reduces the output to:
filename1,
a/b/c
/d
/e.properties
The awk portion reassembles the line to:
filename1,a/b/c/d/e.properties