perl loop avoid print duplicate in regex match - regex

I Search, and research for a solution but without successful.
The file.txt:
Name Server: NS1.SERVER.COM
Name Server: NS2.SERVER.COM
..........................
..........................
Name Server: NS1.SERVER.COM
Name Server: NS2.SERVER.COM
Whois server: whois.directnic.com!
..........................
..........................
Name Server: NS1.SERVER.COM
Name Server: NS2.SERVER.COM
Whois server: whois.directnic.com!
..........................
..........................
When I run:
use strict;
use warnings;
my $filename = 'file.txt';
open(my $fh, '<:encoding(UTF-8)', $filename)
or die "Could not open file '$filename' $!";
while (my $row = <$fh>) {
if ($row =~ /^Name\sServer:\s+(.*+)?/) {
print "dns $1\n";
}
if ($row =~ /Whois.+server:.(.*)/) {
print "whois server: $1\n";
}
}
The output:
dns NS1.SERVER.COM
dns NS2.SERVER.COM
dns NS1.SERVER.COM
dns NS2.SERVER.COM
whois server: whois.directnic.com!
dns NS1.SERVER.COM
dns NS2.SERVER.COM
whois server: whois.directnic.com!
I want to get like this:
dns NS1.SERVER.COM
dns NS2.SERVER.COM
whois server: whois.directnic.com!
I know I can use Last
(If you’ve used the “break” operator in C or a similar language, it’s like that.)
If I set last; in the first condition I get this:
dns NS1.SERVER.COM
If I set last; in the second condition I get this:
dns NS1.SERVER.COM
dns NS2.SERVER.COM
dns NS1.SERVER.COM
dns NS2.SERVER.COM
whois server: whois.directnic.com!
I find a solution but using hash, while, increase count and grep, like this:
my %count_of;
my %count_of_two;
while (my $row = <$fh>) {
if ($row =~ /^Name\sServer:\s+(.*+)?$/) {
$count_of{$row}++;
}
if ($row =~ /Whois.+Server:.(.*)/) {
$count_of_two{$row}++;
}
}
print join "\n", grep { $count_of{$_} > 1 } keys %count_of;
print join "\n", grep { $count_of_two{$_} == 1 } keys %count_of_two;
¿How can I do this with while or foreach?
Note: I think I know why while loop print duplicate because that the functionality, but maybe exist another way.
I research about this but I can't find a solution.
Thanks for your patience.

The easiest approach is probably to use a hash (called %seen in my example below) which keeps track of the strings you have ever seen.
Making the smallest changes to your code, we get:
use strict;
use warnings;
my $filename = 'file.txt';
open(my $fh, '<:encoding(UTF-8)', $filename)
or die "Could not open file '$filename' $!";
my %seen;
while (my $row = <$fh>) {
if ($row =~ /^Name\sServer:\s+(.*+)?/) {
print "dns $1\n" unless $seen{$1}++;
}
if ($row =~ /Whois.+server:.(.*)/) {
print "whois server: $1\n" unless $seen{$1}++;
}
}

Related

How to get all info regarding the cell, based on cell_list.txt

In this case, I have two files, cell_list.txt and allcells.txt. In cell_list.txt were listed cell names that required.
For example:
cell_abc
cell_acde
c_swer
Then, i have allcells.txt which show the details of all cells which more that 100 cells details. I found that the pattern seems to be quite same which all cell details starts with ***** and ends with 'END'. For example:
*****
Lib: lib_a
Cell: cell_abc
*****
info absw ...
info swea ...
END
*****
Lib: lib_a
Cell: cell_acdd
*****
info awee ...
info awod ...
info acwe ...
END
*****
Lib: lib_b
Cell: cell_acde
*****
info wseo ...
info poee ...
info awec ...
END
*****
Lib: lib_b
Cell: c_swer
*****
info rtoe ...
info swkt ...
END
I need to get all the details based on cell listed in cell_list.txt and somehow copy to a new file for each cell, cellname.txt. Is there any way to make this works using csh or perl? Expected output as below.
Content of cell_abc.txt:
*****
Lib: lib_a
Cell: cell_abc
*****
info absw ...
info swea ...
END
Content of cell_acde.txt:
*****
Lib: lib_b
Cell: cell_acde
*****
info wseo ...
info poee ...
info awec ...
END
Content of c_swer.txt:
*****
Lib: lib_b
Cell: c_swer
*****
info rtoe ...
info swkt ...
END
This is roughly on what i have on my script now as i am not familiar with perl.
#!/usr/bin/perl
use strict;
use warnings;
my $file = 'allcells.txt';
my $list = 'cell_list.txt';
my $string;
my #matches = $file =~ m/(^\* .+? END)/g;
{
local $/=undef;
open FILE, $file or die "Couldn't open file: $!";
$string = <FILE>;
close FILE;
while(<>){
if ($string = #matches) #how to check on cell_list.txt if the cell is listed in the file or not before checking the matching string.
{
print $string; #how to extract and print the matching string to new file which will be named based on the cell name listed in cell_list.txt
}
}
}
You need to actually read in the file first, instead of trying to perform a regex match on an empty string. Iterate over the other file to populate a hash, and use hash membership to decide whether to print out a section into a new file. You can use \Q and \E inside the regex to make a literal match. The trailing /s regex flag treats the string like one long line.
#!/usr/bin/env perl
use strict;
use warnings;
my $file = 'allcells.txt';
my $list = 'cell_list.txt';
my %required_cells;
open my $fhrc, "<$list"
or die "Unable to open '$list' : $!";
while ( my $line = <$fhrc> ) {
chomp($line);
$required_cells{ $line } = 1;
}
open my $fh, "<$file"
or die "Unable to open '$file' : $!";
my $allcells_txt = do { local $/; <$fh> }; # Slurp file into a string
my #matches = $allcells_txt =~ m|\Q*****\E.+?\Q*****\E.+?END|gs;
for my $group (#matches) {
my ($cell) = $group =~ m|Cell: (\w+)|s;
if ( exists $required_cells{ $cell } ) {
print "Cell [ $cell ] is required\n";
my $out_name = "$cell.txt";
open my $out, ">$out_name"
or die "Unable to open '$out_name' for writing : $!";
print $out "$group . "\n";
close $out
or die "Unable to close '$out_name' : $!";
print "==> Created $out_name\n";
} else {
print "Skipping $cell\n";
}
}
output
Cell [ cell_abc ] is required
==> Created cell_abc.txt
Skipping cell_acdd
Cell [ cell_acde ] is required
==> Created cell_acde.txt
Cell [ c_swer ] is required
==> Created c_swer.txt

Read a file untill a string in matched

My input file contains:
Startoffile
Host status
Server1
Server2
Server3
Pending Device
Device1
Device2
..
Devic100
Endoffile
How to write a simple PowerShell script which will read the files until the line "Pending Device"?
without knowing much about your desired action, try this:
$file = Get-Content "C:\Users\user\Downloads\file.txt"
foreach ($line in $file) {
if ($line -match "Pending Device") {
Write-Host "Reached 'Pending Device'" -ForegroundColor Red
}
else {
Write-Host "No 'Pending Device' yet"
}
}

Perl: file search regex multiple infos in multiple lines

Hello I have this in a file, multiple lines and from them I want to be able to get the User name and the version he's using.
File
<W>2016-06-25 00:27:30.577 1 => <4:(-1)> Client version 1.2.10 (Win: 1.2.10)
<W>2016-06-25 00:27:30.635 1 => <4:[AAA] User1(1850)> Authenticated
<W>2016-06-25 00:27:30.635 1 => <2:(-1)> Client version 1.2.16 (Win: 1.2.16)
<W>2016-06-25 00:27:30.687 1 => <2:[AAA] User2(942)> Authenticated
Outpout wanted
4 : User1 : 1.2.10
2 : User2 : 1.2.16
So the datas for one client is on 2 lines.
The first line get the version number.
The second line the user name.
I noticed that both lines have a match ID, in my example the user1 line match ID is 4: and 2: for the second user.
So I started with something like this, but don't really work as intended and creating a second read to find the second line in the entire file is too much / not optimized.
Perl Script
#!/usr/bin/perl
use strict;
use warnings;
my $file = 'mylogfile.log';
open (my $fl, '<:encoding(UTF-8)', $file)
or die 'File not found';
while (my $row = <$fl>) {
if ($row =~ m/\<(\d+).*\>\sclient\sversion\s(\d+.\d+.\d+)\s/i) {
my $id = $1;
my $vers = $2;
while (my $row1 = <$fl>) {
if ($row1 =~ m/\<$id\:(.+)\(\d+\)\>/i) {
my $name = $1;
print "$id : $name : $vers\n";
}
}
}
}
If any perl guru have an idea, thanks! :-)
I see in your log file that timestamps of corresponding rows are different.
So, I suppose, when two users log in at the same time, log records could get interspersed, for example:
<W>2016-06-25 00:27:30.577 1 => <4:(-1)> Client version 1.2.10 (Win: 1.2.10)
<W>2016-06-25 00:27:30.635 1 => <2:(-1)> Client version 1.2.16 (Win: 1.2.16)
<W>2016-06-25 00:27:30.635 1 => <4:[AAA] User1(1850)> Authenticated
<W>2016-06-25 00:27:30.687 1 => <2:[AAA] User2(942)> Authenticated
If this is the case, I would suggest using a hash to remember ids:
use strict;
use warnings;
my $file = 'mylogfile.log';
open (my $fl, '<:encoding(UTF-8)', $file)
or die 'File not found';
my %ids;
while (my $row = <$fl>) {
if ($row =~ m/\<(\d+).*\>\sclient\sversion\s(\d+.\d+.\d+)\s/i) {
my ($id,$vers)=($1,$2);
$ids{$id}=$vers;
}
elsif ($row =~ m/\<(\d+)\:(.+)\(\d+\)\>.*authenticated/i) {
if (defined $ids{$1}) {
print "$1 : $2 : $ids{$1}\n";
delete $ids{$1};
}
}
}
I don't know much about perl, but can provide some idea:
login= map();
while( row=readrow())
{
if(match(id version))
login[$1]=$2
else
if(match(id username userid ))
{
print "user: ", $2, "version:",login[$1], "userid: $3", "sessionid: ", $1
delete login[$1]
}
}
Running your code gave me the result
4 : [AAA] User1 : 1.2.10
Your second regular expression is capturing the bracketed letters and the user name. This isn't what your desired output looks like.
The second while loop exhausts the remainder of the file. And, this isn't what you want to do.
Here is a program that will produce the output you want. (I created a file at the top of the program. You would not use this but instead, open your file 'mylogfile.log' just as you did in your code).
#!/usr/bin/perl
use strict;
use warnings;
open my $fh, '<', \<<EOF;
<W>2016-06-25 00:27:30.577 1 => <4:(-1)> Client version 1.2.10 (Win: 1.2.10)
<W>2016-06-25 00:27:30.635 1 => <4:[AAA] User1(1850)> Authenticated
<W>2016-06-25 00:27:30.635 1 => <2:(-1)> Client version 1.2.16 (Win: 1.2.16)
<W>2016-06-25 00:27:30.687 1 => <2:[AAA] User2(942)> Authenticated
EOF
while (<$fh>) {
if (/<(\d+).+?Client version (\d+\.\d+\.\d+)/) {
my ($id, $vers) = ($1, $2);
# read next line and capture name
if (<$fh> =~ /<$id\S+ ([^(]+)/) {
my $name = $1;
print join(" : ", $id, $name, $vers), "\n";
}
}
}
In my second regular expression, the piece, [^(]+, is called a negated class. It matches non 'left parens' (1 or more times). This matches "User1' and 'User2' in the line of the file.
Update: You can find info about character classes here.
Update2: Looking at wolfrevokcats reply, I see he made a valid observation and his solution is the safer one.

How do you find something in an array that would match keywords from a text file and while doing a regex in perl

I'm at a loss at trying to find out how to match the keywords in my keywords text which are:
NetworkManager
dhclient
dbus
My code currently looks like this:
#!/usr/bin/perl
use strict;
use warnings;
use File::Compare;
my $syslogFile = 'syslog';
open (my $syslogInfo, '<', $syslogFile) or die "Could not open $syslogFile";
my $keywordFile = 'keyword';
open (my $keywordInfo, '<', $keywordFile) or die "Could not open $keywordFile";
while(my #keywordArray = <$keywordInfo>)
{
print "#keywordArray\n";
}
while(my #syslogLine = <$syslogInfo>)
{
print "#syslogLine\n";
}
and now I am stuck on how to get these 2 text files to work together and get the desired result.
What Im trying to do is to have things in my syslog txt for example:
Dec 27 21:17:52 osboxes rsyslogd: [origin software="rsyslogd" swVersion="8.12.0" x-pid="695" x-info="http://www.rsyslog.com"] rsyslogd was HUPed
Dec 27 21:18:05 osboxes anacron[634]: Job `cron.daily' terminated
Dec 27 21:18:05 osboxes anacron[634]: Normal exit (1 job run)
Dec 27 21:22:29 osboxes NetworkManager[686]: <info> lease time 1800
Dec 27 21:22:29 osboxes NetworkManager[686]: <info> domain name 'localdomain'
To have it matched with the keywords and only print the lines that have the keywords in it so that it will only print:
Dec 27 21:22:29 osboxes NetworkManager[686]: <info> lease time 1800
Dec 27 21:22:29 osboxes NetworkManager[686]: <info> domain name 'localdomain'
I have tried out using IF by doing this:
if(my #syslogLine = (/^[a-zA-Z][a-zA-Z][a-zA-Z]\s(\d\d)\s(\d\d):(\d\d):(\d\d)\s[a-zA-Z]*\s(#keywordArray).*/s))
{
open(outputFile, ">>output");
print outputFile "#syslogLine\n";
}
but this gives me the errors:
Possible unintended of #keywordArray
Global symbol "#keywordArray requires explict package name
Im guessing that perl doesn't like having an array in the regex?
I just couldn't get my head around this problem.
The common approach is to build a regex from the keywords, then read the second file and match each line against the keyword:
#!/usr/bin/perl
use warnings;
use strict;
my $keywordFile = 'keyword';
open my $KW, '<', $keywordFile or die "Could not open $keywordFile";
chomp( my #keywords = <$KW> );
my $regex = join '|', map quotemeta, #keywords;
my $syslogFile = 'syslog';
open my $SYS, '<', $syslogFile or die "Could not open $syslogFile";
while(<$SYS>) {
print if /$regex/;
}

unexpected EOF while looking for matching `' '

#!/usr/bin/perl
use warnings;
while(1){
system ( "dialog --menu Customize 30 80 60 "
. "'Show rules' 'Show all the current rules' "
. "'Flush rules' 'Flush all the tables' "
. "Allow IP' 'Block all except one IP' "
. "'Block IP' 'Block all the packets from an IP' "
. "'Block MAC' 'Block using the hardware address' "
. "'Private networking' 'Allow only one network and block other networks' "
. "'Allow lo' 'Allow local network interface' "
. "'Save' 'Save customized rules' "
. "'Exit' 'Close the program' "
. "'more options' '........' 2> /tmp/customize.txt");
open FILE4, "/tmp/customize.txt" or die $!;
chomp(my $customize = <FILE4>);
#SHOW RULES
if($customize =~ /Show rules/){
`iptables -nvL | tee /tmp/nvl.txt`;
system ("dialog --textbox /tmp/nvl.txt 22 70");
}
#FLUSH RULES
elsif($customize =~ /Flush rules/){
`iptables -F`;
system ("dialog --infobox 'All tables have been flushed.' 05 35");
sleep 2;
}
#ALLOW IP
elsif($customize =~ /Allow IP/){
system ("dialog --inputbox 'Enter the IP address of the sysetm which you want to allow:' 15 40 2> /tmp/allowIP.txt");
open FILE7, "/tmp/allowIP.txt" or die $!;
chomp(my $aip = <FILE7>);
`iptables -I INPUT -s $aip -j DROP`;
system ("dialog --infobox 'IP address $aip is allowed and rest are blocked' 05 45");
sleep 2;
}
#BLOCK IP
elsif($customize =~ /Block IP/){
system ("dialog --inputbox 'Enter the IP address of the system which you want to block:' 15 40 2> /tmp/blockIP.txt");
open FILE5, "/tmp/blockIP.txt" or die $!;
chomp(my $ip = <FILE5>);
`iptables -A INPUT -s $ip -j DROP`;
system ("dialog --infobox 'IP address $ip has been blocked!' 05 35");
sleep 2;
}
#PRIVATE NETWORK
elsif($customize =~ /Private networking/){
system ("dialog --inputbox 'Enter the network address which you want to allow (eg. 192.168.0.0/24)' 15 40 2> /tmp/network.txt");
open FILE6, "/tmp/network.txt" or die $!;
chomp(my $network = <FILE6>);
`iptables -I INPUT -s $network -j ACCEPT`;
system ("dialog --infobox 'Network $network is allowed and rest networks are blocked' 05 35");
sleep 2;
}
#ALLOW LO
elsif($customize =~ /Allow lo/){
`iptables -I INPUT -i lo -j ACCEPT`;
system ("dialog --infobox 'Local interface is allowed.' 05 35");
sleep 2;
}
#SAVE
elsif($customize =~ /Save/){
`service iptables save`;
system ("dialog --infobox 'All rules have been saved successfully' 05 45");
sleep 2;
}
#EXIT
elsif($customize =~ /Exit/){
system ("dialog --infobox 'Closing application.' 05 35");
sleep 2;
exit 0;
}
else{
exit;
}
}
perl file.plx
error:
sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file
How do I resolve this error?
Missing ' here: "Allow IP'
Your forgot a ' in
. "Allow IP' 'Block all except one IP' "
before Allow IP' in line 7 of your Perl code.