I am newbie in Perl programming and currently trying to use Net::OpenSSH module in my code, my new code as below which the task is to run multiple command in remote server,
Code as below::
---
---
foreach $s (#servers) {
my $ssh = Net::OpenSSH->new("$username\#$s", timeout=>30);
$ssh->error and die "Unable to connect: " . $ssh->error;
print "Connected to $s\n";
my $fh = $ssh->capture("df -k /home/ | tail -1") or die "Unable to run command\n";
my #df_arr = split(/\s+/, $fh);
print "$s: Disk space /home/ = $df_arr[3] \n";
my $fh1 = $ssh->capture("svmon -G -O unit=GB | grep memory") or die "Unable to run command\n";
my #sv_arr = split(/\s+/, $fh1);
print "$s: Free memory = $sv_arr[3] \n\n";
close $fh;
undef $ssh;
}
----
----
This code is not so nice since I plan to make it more simplify and reduce a line as many as possible.
Does there are any technique's or methods that I can use to simplify this code?
our way to make it happen in this way:
sub ssh_connect($username, $password)
$ssh = ssh_connect($username, $password)
sub ssh_execute($ssh, $command_str)
there are too much duplicated code in your origin code, maybe you can put them into a subroutine and reuse it.
Related
When I run the following script from a newly opened PowerShell console, the loop exits so there is clearly a match, but the $matches variable (and thus $matches.PORT) is not populated the first time around. When the script is run again, it is populated.
./ssh.ps1
$BLOCK = { az webapp create-remote-connection --subscription <MY-SUBSCRIPTION> --resource-group <RESOURCE-GROUP> -n <NAME> }
$global:CONNECTION = Start-ThreadJob -ScriptBlock $BLOCK
$done = 0
$match_string = ".*Opening tunnel on port: (?<PORT>\d{1,5})\b.*"
while ($done -lt 1) {
if ($CONNECTION.HasMoreData)
{
$DATA = Receive-Job $CONNECTION 2>&1
if ($DATA -match $match_string)
{
$port = $matches.PORT
Write-Output "Connection open on port $port."
$done = 1
}
}
}
Write-Output "Loop ended."
exit
Output in the PowerShell console is:
PS <LOCAL-DIR>> ./ssh
Connection open on port .
Loop ended.
PS <LOCAL-DIR>> ./ssh
Connection open on port 63182.
Loop ended.
By contrast, when I try running the following script, $matches is populated the first time it is run.
./match.ps1
$string1 = "hello, hello, you big beautiful world of wonder!"
$match_str = ".*\b(?<gotcha>world)\b.*"
$done = 0
while ($done -lt 1)
{
if ($string1 -match $match_str)
{
write-output "Matches:"
write-output $matches
$done = 1
}
}
Output:
PS <LOCAL-DIR>> ./match
Matches:
Name Value
---- -----
gotcha world
0 hello, hello, you big beautiful world of wonder!
If anyone can fathom why the text is matched in the first script without $matches being populated I would be incredibly grateful.
P.S.
The script existing after the loop is just for investigative purposes and not what my code will actually do.
P.P.S.
For reference, the output from az webapp create-remote-connection, after a delay whilst connecting, is:
Verifying if app is running....
App is running. Trying to establish tunnel connection...
Opening tunnel on port: 63341
SSH is available { username: root, password: Docker! }
Ctrl + C to close
(The port varies each time.)
If the automatic $Matches variable isn't populated after a -match operation, the implication is that the LHS operand was a collection rather than a single string.
Therefore, loop over the value of $DATA and match each line individually:
foreach ($line in $DATA) {
if ($line -match $match_string)
{
$port = $matches.PORT
"Connection open on port $port."
$done = 1
break
}
}
By design:
$Matches is only populated if the LHS is a string (scalar).
With a collection (array) as the LHS, -match - as many comparison operators do - acts as a filter and returns the (potentially empty) sub-array of matching elements.
Any prior $Matches value is preserved if either a given string-scalar -match operation happens not to find a match or its LHS is a collection.
I am trying to remove the old files in a dir if the count is more than 3 over SSH
Kindly suggest how to resolve the issue.
Please refer the code snippet
#!/usr/bin/perl
use strict;
use warnings;
my $HOME="/opt/app/latest";
my $LIBS="${HOME}/libs";
my $LIBS_BACKUP_DIR="${HOME}/libs_backups";
my $a;
my $b;
my $c;
my $d;
my $command =qq(sudo /bin/su - jenkins -c "ssh username\#server 'my $a=ls ${LIBS_BACKUP_DIR} | wc -l;my $b=`$a`;if ($b > 3); { print " Found More than 3 back up files , removing older files..";my $c=ls -tr ${LIBS_BACKUP_DIR} | head -1;my $d=`$c`;print "Old file name $d";}else { print "No of back up files are less then 3 .";} '");
print "$command\n";
system($command);
output:
sudo /bin/su - jenkins -c "ssh username#server 'my ; =ls /opt/app/latest/libs_backups | wc -l;my ; =``;if ( > 3); { print " Found More than 3 back up files , removing older files..";my ; =ls -tr /opt/app/latest/libs_backups | head -1;my ; =``;print "Old file name ";}else { print "No of back up files are less then 3 .";} '"
Found: -c: line 0: unexpected EOF while looking for matching `''
Found: -c: line 1: syntax error: unexpected end of file
If you have three levels of escaping, you're bound to get it wrong if you do it manually. Use String::ShellQuote's shell_quote instead.
Furthermore, avoid generating code. You're bound to get it wrong! Pass the necessary information using arguments, the environment or some other channel of communication instead.
There were numerous errors in the interior Perl script on top of the fact that you tried to execute a Perl script without actually invoking perl!
#!/usr/bin/perl
use strict;
use warnings;
use String::ShellQuote qw( shell_quote );
my $HOME = "/opt/app/latest";
my $LIBS = "$HOME/libs";
my $LIBS_BACKUP_DIR = "$HOME/libs_backups";
my $perl_script = <<'__EOI__';
use strict;
use warnings;
use String::ShellQuote qw( shell_quote );
my ($LIBS_BACKUP_DIR) = #ARGV;
my $cmd = shell_quote("ls", "-tr", "--", $LIBS_BACKUP_DIR);
chomp( my #files = `$cmd` );
if (#files > 3) {
print "Found more than 3 back up files. Removing older files...\n";
print "$_\n" for #files;
} else {
print "Found three or fewer backup files.\n";
}
__EOI__
my $remote_cmd = shell_quote("perl", "-e", $perl_script, "--", $LIBS_BACKUP_DIR);
my $ssh_cmd = shell_quote("ssh", 'username#server', "--", $remote_cmd);
my $local_cmd = shell_quote("sudo", "su", "-c", $ssh_ccmd);
system($local_cmd);
I created a new file and handling the dir check and deletion logic , scp file to remote server and executing in remote server , after completion removing the file.
#!/usr/bin/perl
use strict;
use warnings;
use File::Basename;
use File::Path;
use FindBin;
use File::Copy;
my $HOME="/opt/app/test/latest";
my $LIBS_BACKUP_DIR="${HOME}/libs_backups";
my $a="ls ${LIBS_BACKUP_DIR} | wc -l";
my $b=`$a`;
my $c="ls -tr ${LIBS_BACKUP_DIR} | head -1";
my $d=`$c`;
chomp($d);
print " count : $b\n";
if ($b > 3)
{
print " Found More than 3 back up files , removing older files..\n";
print "Old file name $d\n";
my $filepath="${LIBS_BACKUP_DIR}/$d";
rmtree $filepath;
}
else
{
print "No of back up files are less then 3 .\n";
}
I need to write a perl program where I parse through an error log and output the error messages to a new file. I am having issues with setting up the regex to do this. In the error log, an error code starts with the word "ERROR" and the end of each error message ends with a ". " (period and then a space). I want to find all the errors, count them, and also output the entire error message of each error message to a new file.
I tried this but am having issues:
open(FH,"<$filetoparse");
$outputfile='./errorlog.txt';
open(OUTPUT,">$outputfile");
$errorstart='ERROR';
$errorend=". ";
while(<FH>)
{
if (FH=~ /^\s*$errorstart/../$errorend/)
{
print OUTPUT "success";
}
else
{
print OUTPUT "failed";
}
}
}
the $errorstart and $errorend are something I saw online and am not sure if that is the correct way to code it.
Also I know the printing "Success" or "Failure" is not what I said I am looking for, I added that in to help with confirmed that the code works, I haven't tried coding for counting the error messages yet.
before this snippet of code I have a print statement asking the user for the location address of the .txt file they want to parse. I confirmed that particular section of code words properly. Thanks for any help! Let me know if more info is needed!
Here is an example of data that I will be using:
Sample Data
-----BEGIN LOAD-----
SUCCESS: file loaded properly .
SUCCESS: file loaded properly .
SUCCESS: file loaded properly .
SUCCESS: file loaded properly .
SUCCESS: file loaded properly .
SUCCESS: file loaded properly .
ERROR: the file was unable to load for an unknown reason .
SUCCESS: file loaded properly .
SUCCESS: file loaded properly .
ERROR: the file was unable to load this is just an example of a log file that will span
multiple lines .
SUCCESS: file loaded properly .
------END LOAD-------
While the log may not necessarily NEED to span multiple lines, there will be some data throughout the log that will similar to how it is above. Every message logged starts with either SUCCESS or ERROR and the message is done when a " . " (whitespace-period-whitespace) is encountered. The log I want to parse through is 50,000 entries long so needless to say I would like to code so it will also identify multi-line error msgs as well as output the entire multi-line message to the output file.
update
I have written the code but for some reason it won't work. I think it has to do with the delimiter but I can't figure out what it is. The file I am using has messages that are separated by "whitespace period newline". Can you see what I'm doing wrong??
{
local $/ = " .\n";
if ($outputtype == 1)
{
$outputfile="errorlog.txt";
open(OUTPUT,">>$outputfile");
$errorcount=0;
$errortarget="ERROR";
print OUTPUT "-----------Error Log-----------\n";
{
while(<FH>)
{
if ($_ =~ m/^$errortarget/)
{
print OUTPUT "$_\n";
print OUTPUT "next code is: \n";
$errorcount++;
}
}
print OUTPUT "\nError Count : $errorcount\n";
}
}
}
There are several problems with your code to start off.
ALWAYS use strict; and use warnings;.
3 argument open is much less error prone. open ( my $fh, "<", $filename ) or die $!;
Always check open actually worked.
FH =~ doesn't do what you think it does.
range operator tests if you're between two chunks of text in code. This is particularly relevant for multi-line operations. If your error log isn't, then it's not what you need.
Assuming you've error data like this:
ERROR: something is broken.
WARNING: something might be broken.
INFO: not broken.
ERROR: still broken.
This code will do the trick:
use strict;
use warnings;
my $filetoparse = "myfile.txt";
my $outputfile = "errorlog.txt";
open( my $input, "<", $filetoparse ) or die $!;
open( my $output, ">", $outputfile ) or die $!;
my $count_of_errors = 0;
#set record delimiter
local $/ = " . \n";
while ( my $lines = <$input> ) {
$lines =~ s/^-----\w+ LOAD-----\n//g; #discard any 'being/end load' lines.
if ( $lines =~ m/^ERROR/ ) {
$count_of_errors++;
print {$output} $lines;
}
}
close ( $input );
close ( $output );
print "$count_of_errors errors found\n";
If you've multi-line error message, then you'll need a slightly different approach though.
I am trying to use Perl to create a program that will read in data for a file that is 40,000+ lines long and parse through each message to extract the error messages from it.
A sample of the data I am using looks like this:
--------All Messages---------
SUCCESS: data transferred successfully .
SUCCESS: data transferred successfully .
SUCCESS: data transferred successfully .
ERROR: there was an error transferring data .
SUCCESS: data transferred successfully .
SUCCESS: data transferred successfully .
SUCCESS: data transferred successfully .
ERROR: there was an error transferring the data and the error message spans
more than 1 line of code and may also contain newline characters as well .
SUCCESS: data transferred successfully .
SUCCESS: data transferred successfully .
SUCCESS: data transferred successfully .
---------END REPOSITORY---------
each message in the log has the following in common:
1) it starts with either SUCCESS or ERROR depending on the outcome
2) all messages will end with <whitespace><period><newline>
The following is code that I have written but for some reason I can't seem to debug it. Any help is greatly appreciated.
open(FH,$filetoparse);
{
# following line is supposed to change the delimiter for the file
$/ = " .";
# the follow statement will create an error log of all error messages in log and save it
# to a file named errorlog.txt
while(<FH>)
{
push (#msgarray, $_);
}
if ($outputtype == 1)
{
$outputfile="errorlog.txt";
open(OUTPUT,">>$outputfile");
$errorcount=0;
$errortarget="ERROR";
print OUTPUT "-----------Error Log-----------\n";
for ($i=0;$i<#msgarray;$i++)
{
if ($msgarray[$i] =~ /^$errortarget/)
{
print OUTPUT "$msgarray[$i]\n";
# print OUTPUT "next code is: \n";
$errorcount++;
}
print OUTPUT "\nError Count : $errorcount\n";
close (OUTPUT);
}
}
Add the newline character to your delimiter. Change:
$/ = " .";
to:
$/ = " .\n";
And if you want to remove the delimiter, you can chomp.
while(<FH>)
{
chomp;
push (#msgarray, $_);
}
The problem with setting $/ = " ." is that the lines you read will end at that closing dot, and the following line will start with the newline character after it. That means none of your lines except possibly the first will start with "ERROR" - they will start with "\nERROR" instead, and so your test will always fail
There are some other issues with your code that you will want to understand.
You must always use strict and use warnings, and declare all your variables with my as close as possible to their first point of use
You should always use lexical file handles with the three-parameter form of open. You also need to check the status of every open and put $! in the die string so that you know why it failed. So
open(FH,$filetoparse);
becomes
open my $in_fh, '<', $filetoparse or die qq{Unable to open "$filetoparse" for input: $!};
It is better to process text files line by line unless you have good reasons to read them into memory in their entirety — for instance, if you need to do multiple passes through the data, or if you need random access to the contents instead of processing them linearly.
It's also worth noting that, instead of writing
while ( <$in_fh> ) {
push #msgarray, $_;
}
you can say just
#msgarray = <$in_fh>;
which has exactly the same result
It is often better to iterate over the contents of an array rather than over its indices. So instead of
for ( my $i = 0; $i < #msgarray; ++$i ) {
# Do stuff with $msgarray[$i];
}
you could write
for my $message ( #msgarray ) {
# Do stuff with $message;
}
Here's a rewrite of your code that demonstrates these points
open my $in_fh, '<', $filetoparse
or die qq{Unable to open "$filetoparse" for input: $!};
{
if ( $outputtype == 1 ) {
my $outputfile = 'errorlog.txt';
my $errorcount = 0;
my $errortarget = 'ERROR';
open my $out_fh, '>>', $outputfile
or die qq{Unable to open "$outputfile" for output: $!};
print $out_fh "-----------Error Log-----------\n";
while ( <$in_fh> ) {
next unless /^\Q$errortarget/;
s/\s*\.\s*\z//; # Remove trailing detail
print $out_fh "$_\n";
++$errorcount;
}
print $out_fh "\nError Count : $errorcount\n";
close ($out_fh) or die $!;
}
}
The file handle OUTPUT is closed within the for loop which you access for every iteration after closing. Move it outside the loop and try it
I'm attempting to write a menu driven modular perl script that will capture user input and automate the network configuration process. This script has to be able to install required Arch packages, configure AP mode, configure either DHCP or a static address for the user selected interface and give an option to enable bridging. (EDIT: The script also needs to be able to enable and configure the dhcpd service)
The part I'm stuck on right now is creating a backup of the rc.conf file, reading the file and editing the lines that need to be modified if a network interface has already been statically configured. This script is for use in ArchLinux, I did some searching around and didn't find anything that met my needs specifically.
Using generic input for
$ip = 1.1.1.1; $Bcast = 2.2.2.2; $netmask = 3.3.3.3; $GW = 4.4.4.4;
I've spent about two hours reading about file I/O and tried several things that didn't work including scrapping the multiple file IO method and using something similar to: while(<IS>){s/^interface.?=(.*)$/"interface=#if[0] \n"/;} with inputs for each of the values that need to be replaced and couldn't get it to actually do anything.
if (system ("cat","/etc/rc.conf","|","grep","interface")){
use File::Copy "cp";
$filename = "/etc/rc.conf";
$tempfile = "/etc/rc.tmp";
$bak = "/etc/rc.bak";
cp($filename,$bak);
open(IS, $filename);
open(OS, ">$tempfile");
while(<IS>){
if($_ =~ /^interface.?=(.*)$/){ print OS"interface=#if[0] \n";}
if($_ =~ /^address.?=(.*)$/){ print OS "address=$ip\n";}
if($_ =~/^netmask.?=(.*)$/){ print OS "netmask=$netmask\n";}
if($_ =~/^broadcast.?=(.*)$/){ print OS "broadcast=$Bcast\n";}
if($_ =~/^gateway.?=(.*)$/){ print OS "gateway=$GW\n"; }
else {print OS $_;}
}
close(IS); close(OS);
unlink($filename); rename($tempfile, $filename);
}
rc.conf before
#
# /etc/rc.conf - Main Configuration for Arch Linux
. /etc/archiso/functions
LOCALE_DEFAULT="en_US.UTF-8"
DAEMON_LOCALE_DEFAULT="no"
CLOCK_DEFAULT="UTC"
TIMEZONE_DEFAULT="Canada/Pacific"
KEYMAP_DEFAULT="us"
CONSOLEFONT_DEFAULT=
CONSOLEMAP_DEFAULT=
USECOLOR_DEFAULT="yes"
LOCALE="$(kernel_cmdline locale ${LOCALE_DEFAULT})"
DAEMON_LOCALE="$(kernel_cmdline daemon_locale ${DAEMON_LOCALE_DEFAULT})"
HARDWARECLOCK="$(kernel_cmdline clock ${CLOCK_DEFAULT})"
TIMEZONE="$(kernel_cmdline timezone ${TIMEZONE_DEFAULT})"
KEYMAP="$(kernel_cmdline keymap ${KEYMAP_DEFAULT})"
CONSOLEFONT="$(kernel_cmdline consolefont ${CONSOLEFONT_DEFAULT})"
CONSOLEMAP="$(kernel_cmdline consolemap ${CONSOLEMAP_DEFAULT})"
USECOLOR="$(kernel_cmdline usecolor ${USECOLOR_DEFAULT})"
MODULES=()
UDEV_TIMEOUT=30
USEDMRAID="no"
USEBTRFS="no"
USELVM="no"
HOSTNAME="archiso"
DAEMONS=(hwclock syslog-ng)
interface=eth0
address=192.168.0.99
netmask=255.255.255.0
broadcast=192.168.0.255
gateway=192.168.0.1
rc.conf after
#
# /etc/rc.conf - Main Configuration for Arch Linux
. /etc/archiso/functions
LOCALE_DEFAULT="en_US.UTF-8"
DAEMON_LOCALE_DEFAULT="no"
CLOCK_DEFAULT="UTC"
TIMEZONE_DEFAULT="Canada/Pacific"
KEYMAP_DEFAULT="us"
CONSOLEFONT_DEFAULT=
CONSOLEMAP_DEFAULT=
USECOLOR_DEFAULT="yes"
LOCALE="$(kernel_cmdline locale ${LOCALE_DEFAULT})"
DAEMON_LOCALE="$(kernel_cmdline daemon_locale ${DAEMON_LOCALE_DEFAULT})"
HARDWARECLOCK="$(kernel_cmdline clock ${CLOCK_DEFAULT})"
TIMEZONE="$(kernel_cmdline timezone ${TIMEZONE_DEFAULT})"
KEYMAP="$(kernel_cmdline keymap ${KEYMAP_DEFAULT})"
CONSOLEFONT="$(kernel_cmdline consolefont ${CONSOLEFONT_DEFAULT})"
CONSOLEMAP="$(kernel_cmdline consolemap ${CONSOLEMAP_DEFAULT})"
USECOLOR="$(kernel_cmdline usecolor ${USECOLOR_DEFAULT})"
MODULES=()
UDEV_TIMEOUT=30
USEDMRAID="no"
USEBTRFS="no"
USELVM="no"
HOSTNAME="archiso"
DAEMONS=(hwclock syslog-ng)
interface=eth0
interface=eth0
address=1.1.1.1
address=192.168.0.99
netmask=3.3.3.3
netmask=255.255.255.0
broadcast=2.2.2.2
broadcast=192.168.0.255
gateway=4.4.4.4
I am not going to comment on the wisdom of the rest of your script, but you have:
if (system ("cat","/etc/rc.conf","|","grep","interface")){
system returns 0 on success.
So, you'll enter the block only if that system call fails.
If fact, I am on a Windows system right now with no /etc/rc.conf (but cat and grep thanks to Cygwin. Running the following script:
#!/usr/bin/env perl
use strict; use warnings;
if (system ("cat","/etc/rc.conf","|","grep","interface")){
print "*** it worked! ***\n";
if ($? == -1) {
print "failed to execute: $!\n";
}
elsif ($? & 127) {
printf "child died with signal %d, %s coredump\n",
($? & 127), ($? & 128) ? 'with' : 'without';
}
else {
printf "child exited with value %d\n", $? >> 8;
}
}
produces the output:
cat: /etc/rc.conf: No such file or directory
cat: |: No such file or directory
cat: grep: No such file or directory
cat: interface: No such file or directory
*** it worked! ***
child exited with value 1
That means system returned a failure code. Now, if you want to use shell piping and redirection, you should pass system a string, not a list, and check like this:
if (system ('cat /etc/rc.conf | grep interface') == 0) {
On the other hand, I would rather not trust shells propagating exit status.
The following should point you in a better direction:
#!/usr/bin/env perl
use strict;use warnings;
my %lookup = (
eth0 => {
address => '1.1.1.1',
broadcast => '2.2.2.2',
netmask => '3.3.3.3',
gateway => '4.4.4.4',
},
wlan0 => {
address => '5.5.5.5',
broadcast => '6.6.6.6',
netmask => '7.7.7.7',
gateway => '8.8.8.8',
},
);
while (my $line = <DATA>) {
if (my ($interface) = ($line =~ /^interface=(\S+)/)) {
print $line;
if (exists $lookup{$interface}) {
$line = process_interface(\*DATA, $lookup{$interface});
redo;
}
}
else {
print $line;
}
}
sub process_interface {
my ($fh, $lookup) = #_;
my $keys = join '|', sort keys %$lookup;
while (my $line = <DATA>) {
$line =~ s/\A($keys)=.+/$1=$lookup->{$1}/
or return $line;
print $line;
}
return;
}
__DATA__
#
# /etc/rc.conf - Main Configuration for Arch Linux
. /etc/archiso/functions
# stuff
interface=eth0
address=192.168.0.99
netmask=255.255.255.0
broadcast=192.168.0.255
gateway=192.168.0.1
interface=wlan0
address=192.168.0.99
netmask=255.255.255.0
broadcast=192.168.0.255
gateway=192.168.0.1
Output:
#
# /etc/rc.conf - Main Configuration for Arch Linux
. /etc/archiso/functions
# stuff
interface=eth0
address=1.1.1.1
netmask=3.3.3.3
broadcast=2.2.2.2
gateway=4.4.4.4
interface=wlan0
address=5.5.5.5
netmask=7.7.7.7
broadcast=6.6.6.6
gateway=8.8.8.8
The problem is your if/if/if/if/if/else chain, which should be an if/elsif/elsif/elsif/elsif/else chain. The else { print OS $_ } triggers on every line that doesn't match gateway=, including the ones that match interface, address, etc.