Find specific date match use of regular expression in shell? - regex

Question

I have bunch of file name in one consolidated file (i.e. main_file), those file name are having different naming conversions but they all have one in common that is date of format date +%Y%m%d example : 20151202 and it will come some where in middle of the name.
main_file file contain will look as :
DTC_by_PV_201511220000_raw_out.snappy
Belle_Tire_201511230000_raw_out.snappy
Goodyear_Tire_201511220200_raw_out.snappy
Sams_Club_201511230000_raw_out.snappy
eTire_All_201511230200_raw_out.snappy
I want to figure out the regular expression that i can use in shell script to read the main_file and generate the different file for each date we would found in file.
In this case we should have 2 files -
1. for date 20151122 and will contain :
DTC_by_PV_201511220000_raw_out.snappy
Goodyear_Tire_201511220200_raw_out.snappy
2. for date 20151123 and will contain :
Belle_Tire_201511230000_raw_out.snappy
Sams_Club_201511230000_raw_out.snappy
eTire_All_201511230200_raw_out.snappy
Note - Convention followed for file name is date with hour and mins (like
-201511230200 here 20151123 is date and 0200 is 2 am )

Answer 1

An awk oneliner:
awk -F_ '{i=substr($(NF-2),1,8);dates[i]=dates[i] $0 "\n"}END{for(d in dates)print(dates[d]) > d}' main_file
This will create a file with the date as the name that has only the lines with that date.
The expanded details... The first action runs on every line:
{
i=substr($(NF-2),1,8);
dates[i]=dates[i] $0 "\n"
}
what it does is find the date part of the line (which is two fields back from the end if the field separator is underscores). It only uses the date part (not the time) by cutting it with substr. Then it appends the whole line to an array element for the given date.
Then, at the end it prints to a file that has the date as the name, for each date found.
END {
for(d in dates)
print(dates[d]) > d
}