Storing strings in an array - regex

I am currently storing below lines in a file named google.txt. I want to seperate these lines and store those seperated strings in arrays.
Like for first line
#qf_file= q33AgCEv006441
#date = Tue Apr 3 16:12
#junk_message = User unknown
#rf_number = ngandotra#nkn.in
the line ends at the #rf_number at last emailadress
q33AgCEv006441 1038 Tue Apr 3 16:12 <test10-list-bounces#lsmgr.nic.in>
(User unknown)
<ngandotra#nkn.in>
q33BDrP9007220 50153 Tue Apr 3 16:43 <karuvoolam-list-bounces#lsmgr.nic.in>
(Deferred: 451 4.2.1 mailbox temporarily disabled: paond.tndt)
<paond.tndta#nic.in>
q33BDrPB007220 50153 Tue Apr 3 16:43 <karuvoolam-list-bounces#lsmgr.nic.in>
(User unknown)
paocorp.tndta#nic.in>
<dtocbe#tn.nic.in>
<dtodgl#nic.in>
q33BDrPA007220 50153 Tue Apr 3 16:43 <karuvoolam-list-bounces#lsmgr.nic.in>
(User unknown)
<dtokar#nic.in>
<dtocbe#nic.in>
q2VDWKkY010407 2221878 Sat Mar 31 19:37 <dhc-list-bounces#lsmgr.nic.in>
(host map: lookup (now-india.net.in): deferred)
<arjunpan#now-india.net.in>
q2VDWKkR010407 2221878 Sat Mar 31 19:31 <dhc-list-bounces#lsmgr.nic.in>
(host map: lookup (aaplawoffices.in): deferred)
<amit.bhagat#aaplawoffices.in>
q2U8qZM7026999 360205 Fri Mar 30 14:38 <dhc-list-bounces#lsmgr.nic.in>
(host map: lookup (now-india.net.in): deferred)
<arjunpan#now-india.net.in>
<amit.bhagat#aaplawoffices.in>
q2TEWWE4013920 2175270 Thu Mar 29 20:30 <dhc-list-bounces#lsmgr.nic.in>
(host map: lookup (now-india.net.in): deferred)
<arjunpan#now-india.net.in>
<amit.bhagat#aaplawoffices.in>

Untested Perl script:
Let's call this script parser.pl:
$file = shift;
open(IN, "<$file") or die "Cannot open file: $file for reading ($!)\n";
while(<IN>) {
push(#qf_file, /^\w+/g);
push(#date, /(?:Sat|Sun|Mon|Tue|Wed|Thu|Fri)[\w\s:]+/g);
push(#junk_message, /(?<=\().+(?=\)\s*<)/g);
push(#rf_number, /(?<=<)[^>]+(?=>\s*$)/g);
}
close(IN);
This assumes the last email on the line should be the "rf_number" for that line. Note that emails may be tricky to print, as they have an # character, and perl is more than happy to print a non-existent list for you :-)
To call this in a command line:
parser.pl google.txt
See this working here.

Related

Extract string with numbers using regex for numbers between 0 to 999

In the below phrase i need to extract /testcard-directory/100 where the number at the last can vary between 0 to 999.
[Tue Jun 27 02:40:25.207457 2020] [:error] [pid 4085] [client
23.23.23.32:34223] Server IP: 172.17.0.3\nUser agent: Amazon CloudFront\nServer time: 2020-06-27 19:40:25\nRequestURI:
/testcard-directory/100\nMessage: mongodb:27017: Read timed out after reading 0 bytes, waited for 30.000000 seconds\n\nTrace:\n#0
I tried this
\/flashcard-directory\/[1-9][0-9]
but the last number is not getting picked up.
Try :
/testcard-directory/[0-9]*
Demo :
$estcard-directory/100\nMessage: mongodb:27017: Read timed out after reading 0 bytes, waited for 30.000000 seconds\n\nTrace:\n#0' | grep -o '/testcard-directory/[0-9]* <
> '
/testcard-directory/100
$
regex: \/testcard-directory\/\d{0,3}
https://regex101.com/r/DDuKsX/1
Use: ^\/testcard-directory\/\d{1,3}$
Here is a Javascript implementation easily portable:
var test = [
'/testcard-directory/',
'/testcard-directory/0',
'/testcard-directory/123',
'/testcard-directory/1234',
];
console.log(test.map(function (a) {
return a+' : '+/^\/testcard-directory\/\d{1,3}$/.test(a);
}));
Demo & explanation

Ansible: ios upgrade router: check "spacefree_kb" prior to image copy

I'm writing a playbook for ios upgrade of multiple switches and have most pieces working with exception of the flash free check. Basically, I want to check if there is enough flash space free prior to copying the image.
I tried using the gather facts module but it is not working how I expected:
from gather facts I see this:
"ansible_net_filesystems_info": {
"flash:": {
"spacefree_kb": 37492,
"spacetotal_kb": 56574
This is the check I want to do:
fail:
msg: 'This device does not have enough flash memory to proceed.'
when: "ansible_net_filesystems_info | json_query('*.spacefree_kb')|int < new_ios_filesize|int"
From doing some research I understand that any value returned by a jinja2 template will be a string so my check is failing:
Pass integer variable to task without losing the integer type
The solution suggested in the link doesn't seem to work for me even with ansible 2.7.
I then resorted to store the results of 'dir' in a register and tried using regex_search but can't seem to get the syntax right.
(similar to this :
Ansible regex_findall multiple strings)
"stdout_lines": [
[
"Directory of flash:/",
"",
" 2 -rwx 785 Jul 2 2019 15:39:05 +00:00 dhcp-snooping.db",
" 3 -rwx 1944 Jul 28 2018 20:05:20 +00:00 vlan.dat",
" 4 -rwx 3096 Jul 2 2019 01:03:26 +00:00 multiple-fs",
" 5 -rwx 1915 Jul 2 2019 01:03:26 +00:00 private-config.text",
" 7 -rwx 35800 Jul 2 2019 01:03:25 +00:00 config.text",
" 8 drwx 512 Apr 25 2015 00:03:16 +00:00 c2960s-universalk9-mz.150-2.SE7",
" 622 drwx 512 Apr 25 2015 00:03:17 +00:00 dc_profile_dir",
"",
"57931776 bytes total (38391808 bytes free)"
]
]
Can anyone provide some insight to this seemingly simple task? I just want '38391808' as an integer from the example above (or any other suggestion). I'm fairly new to ansible.
Thanks in advance.
json_query wildcard expressions return a list. The tasks below
- set_fact:
free_space: "{{ ansible_net_filesystems_info|
json_query('*.spacefree_kb') }}"
- debug:
var: free_space
give the list
"free_space": [
37492
]
which neither can be converted to an integer nor can be compared to an integer. This is the reason for the problem.
The solution is simple. Just take the first element of the list and the condition will start working
- fail:
msg: 'This device does not have enough flash memory to proceed.'
when: ansible_net_filesystems_info|
json_query('*.spacefree_kb')|
first|
int < new_ios_filesize|int
Moreover, json_query is not necessary. The attribute spacefree_kb can be referenced directly
- fail:
msg: 'This device does not have enough flash memory to proceed.'
when: ansible_net_filesystems_info['flash:'].spacefree_kb|
int < new_ios_filesize|int
json_query has an advantage : see this example on a C9500 :
[{'bootflash:': {'spacetotal_kb': 10986424.0, 'spacefree_kb': 4391116.0}}]
yes they changed flash: to bootflash:.

parse command line arguments in a different order

I have a small C++ program that reads arguments from the bash.
Let's say that I have a folder with some files with two different extensions.
Example: file1.ext1 file1.ext2 file2.ext1 file2.ext2 ...
if I execute the program with this command: ./readargs *.ext1
it will read all the files with the ext1.
if I execute ./readargs *.ext1 *.ext2 it will read all the .ext1 files and then all the .ext2 files.
My question is how can I execute the program in a way that reads with this order: file1.ext1 file1.ext2 file2.ext1 file2.ext2 ... Can I handle that from the command line or I need to handle it in the code?
If the names of your files is really in the form file1.ext1 file1.ext2 file2.ext1 file2.ext2, then you can sort it with
echo *.{ext1,ext2}|sort -u
the above gives the output:
$ ll | grep ext
23880775 0 -rw-r--r-- 1 user users 0 Apr 29 13:28 file1.ext1
23880789 0 -rw-r--r-- 1 user users 0 Apr 29 13:28 file1.ext2
23880787 0 -rw-r--r-- 1 user users 0 Apr 29 13:28 file2.ext1
23880784 0 -rw-r--r-- 1 user users 0 Apr 29 13:28 file2.ext2
$ echo *.{ext1,ext2} | sort -u
file1.ext1 file2.ext1 file1.ext2 file2.ext2
Then you copy the output and call your program. But if you do in fact need the files with .ext1 before the files of .ext2, then you have to either make sure that the .ext1 is alphabetically inferior to .ext2 or use another sorting criterion.
Optionally you could also adapt your executable to handle command line arguments in the correct order, but if you do already have an executable I'd recommend the first solution as work-around.
edit: this command does also sort lexically:
$echo *.ext[1,2]
$file1.ext1 file1.ext2 file2.ext1 file2.ext2

How can I extract all conversations in a Postfix log from a particular relay using awk?

I am trying to extract the from address from the sending relay IP address in a postfix log file
Any ideas???
Much appreciated for any help
Ken
Nov 16 00:05:10 mailserver pfs/smtpd[4365]: 925D54E6D9B: client=client1[1.2.3.4]
Nov 16 00:05:10 mailserver pfs/cleanup[4413]: 925D54E6D9B: message-id=<11414>
Nov 16 00:05:10 mailserver pfs/qmgr[19118]: 925D54E6D9B: from=<11414#localhost>, size=40217, nrcpt=1 (queue active)
Nov 16 00:05:10 mailserver pfs/smtp[4420]: 925D54E6D9B: to, relay=[1.3.5.7]:25, delay=0.02, delays=0.02/0/0/0, dsn=5.0.0, status=bounced (host [1.3.5.7] refused to talk to me: 550 Please remove this address from your list)
Nov 16 00:05:10 mailserver pfs/bounce[4310]: 925D54E6D9B: sender non-delivery notification: 972E34E6D9F
Nov 16 00:05:10 mailserver pfs/qmgr[19118]: 925D54E6D9B: removed
Hmm, if you just want to collect the from and relay fields with their display bling, you could use this:
/: from=/ { lastFrom = $7 }
/relay=/ { print lastFrom, $8 }
If you really want to extract the core addresses, it gets slightly more complex...
/: from=/ { lastFrom = $7 }
/relay=/ {
r = $8
gsub(/from=</, "", lastFrom)
gsub(/>,*/, "", lastFrom)
gsub(/relay=\[/, "", r)
gsub(/\].*/, "", r)
print lastFrom, r
}
$ awk -f mail2.awk mail.dat
11414#localhost 1.3.5.7
As usual, these solutions work in both The One True Awk as well as gawk.
$7 ~ /^from=,$/ {
from[$6] = substr($7, 7, length($7) - 8)
}
$8 ~ /^relay=\[/ {
if (substr($8, "[1.3.5.7]"))
print from[$6]
delete from[$6]}
}
Each time a from-recording line is seen, this saves it in an associative array,
indexed by the queue ID of the message. When a relay line is seen, if it's for
the relay you're interested in the associated from line is printed. substr() is
used just so you don't have to \-escape all of the metacharacters - "[", "]", ".".
Whether it's a relay you're interested in or not, the from data is cleaned
up so that the array doesn't grow without bounds.

Perl: Deleting multiple reccuring lines where a certain criterion is met

I have data that looks like below, the actual file is thousands of lines long.
Event_time Cease_time
Object_of_reference
-------------------------- --------------------------
----------------------------------------------------------------------------------
Apr 5 2010 5:54PM NULL
SubNetwork=ONRM_RootMo,SubNetwork=AXE,ManagedElement=BSJN1,BssFunction=
BSS_ManagedFunction,BtsSiteMgr=LUGALAMBO_900
Apr 5 2010 5:55PM Apr 5 2010 6:43PM
SubNetwork=ONRM_RootMo,SubNetwork=AXE,ManagedElement=BSJN1,BssFunction=
BSS_ManagedFunction,BtsSiteMgr=LUGALAMBO_900
Apr 5 2010 5:58PM NULL
SubNetwork=ONRM_RootMo,SubNetwork=AXE,ManagedElement=BSCC1,BssFunction=
BSS_ManagedFunction,BtsSiteMgr=BULAGA
Apr 5 2010 5:58PM Apr 5 2010 6:01PM
SubNetwork=ONRM_RootMo,SubNetwork=AXE,ManagedElement=BSCC1,BssFunction=
BSS_ManagedFunction,BtsSiteMgr=BULAGA
Apr 5 2010 6:01PM NULL
SubNetwork=ONRM_RootMo,SubNetwork=AXE,ManagedElement=BSCC1,BssFunction=
BSS_ManagedFunction,BtsSiteMgr=BULAGA
Apr 5 2010 6:03PM NULL
SubNetwork=ONRM_RootMo,SubNetwork=AXE,ManagedElement=BSJN1,BssFunction=
BSS_ManagedFunction,BtsSiteMgr=KAPKWAI_900
Apr 5 2010 6:03PM Apr 5 2010 6:04PM
SubNetwork=ONRM_RootMo,SubNetwork=AXE,ManagedElement=BSJN1,BssFunction=
BSS_ManagedFunction,BtsSiteMgr=KAPKWAI_900
Apr 5 2010 6:04PM NULL
SubNetwork=ONRM_RootMo,SubNetwork=AXE,ManagedElement=BSJN1,BssFunction=
BSS_ManagedFunction,BtsSiteMgr=KAPKWAI_900
Apr 5 2010 6:03PM Apr 5 2010 6:03PM
SubNetwork=ONRM_RootMo,SubNetwork=AXE,ManagedElement=BSCC1,BssFunction=
BSS_ManagedFunction,BtsSiteMgr=BULAGA
Apr 5 2010 6:03PM NULL
SubNetwork=ONRM_RootMo,SubNetwork=AXE,ManagedElement=BSCC1,BssFunction=
BSS_ManagedFunction,BtsSiteMgr=BULAGA
Apr 5 2010 6:03PM Apr 5 2010 7:01PM
SubNetwork=ONRM_RootMo,SubNetwork=AXE,ManagedElement=BSCC1,BssFunction=
BSS_ManagedFunction,BtsSiteMgr=BULAGA
As you can see, each file has a header which describes what the various fields stand for(event start time, event cease time, affected element). The header is followed by a number of dashes.
My issue is that, in the data, you see a number of entries where the cease time is NULL i.e event is still active. All such entries must go i.e for each element where the alarm cease time is NULL, the start time, the cease time(in this case NULL) and the actual element must be deleted from the file.
In the remaining data, all the text starting from word SubNetwork upto BtsSiteMgr= must also go. Along with the headers and the dashes.
Final output should look like below:
Apr 5 2010 5:55PM Apr 5 2010 6:43PM
LUGALAMBO_900
Apr 5 2010 5:58PM Apr 5 2010 6:01PM
BULAGA
Apr 5 2010 6:03PM Apr 5 2010 6:04PM
KAPKWAI_900
Apr 5 2010 6:03PM Apr 5 2010 6:03PM
BULAGA
Apr 5 2010 6:03PM Apr 5 2010 7:01PM
BULAGA
Below is a Perl script that I have written. It has taken care of the headers, the dashes, the NULL entries but I have failed to delete the lines following the NULL entries so as to produce the above output.
#!/usr/bin/perl
use strict;
use warnings;
$^I=".bak" #Backup the file before messing it up.
open (DATAIN,"<george_perl.txt")|| die("can't open datafile: $!"); # Read in the data
open (DATAOUT,">gen_results.txt")|| die("can't open datafile: $!"); #Prepare for the writing
while (<DATAIN>) {
s/Event_time//g;
s/Cease_time//g;
s/Object_of_reference//g;
s/\-//g; #Preceding 4 statements are for cleaning out the headers
my $theline=$_;
if ($theline =~ /NULL/){
next;
next if $theline =~ /SubN/;
}
else{
print DATAOUT $theline;
}
}
close DATAIN;
close DATAOUT;
Kindly help point out any modifications I need to make on the script to make it produce the necessary output.
Your data arrives in sets of 3 lines, so one approach is to organize the parsing that way:
use strict;
use warnings;
# Ignore header junk.
while (<>){
last unless /\S/;
}
until (eof) {
# Read in a set of 3 lines.
my #lines;
push #lines, scalar <> for 1 .. 3;
# Filter and clean.
next if $lines[0] =~ /\sNULL\s/;
$lines[2] =~ s/.+BtsSiteMgr=//;
print #lines[0,2];
}
Looks like a good candidate for a little input record separator ($/) trickery. The idea is to manipulate it so that it deals with one record at a time, rather than the default single line.
use strict;
use warnings;
$^I = '.bak';
open my $dataIn, '<', 'george_perl.txt' or die "Can't open data file: $!";
open my $dataOut, '>', 'gen_results.txt' or die "Can't open output file: $!";
{
local $/ = "\n\t"; # Records have leading tabs
while ( my $record = <$dataIn> ) {
# Skip header & records that contain 'NULL'
next if $record =~ /NULL|Event_time/;
# Strip out the unwanted yik-yak
$record =~ s/SubNetwork.*BtsSiteMgr=//s;
# Print record to output file
print $dataOut $record;
}
}
close $dataIn;
close $dataOut;
Pay attention to the following:
use of the safer three-argument form of open (the two-argument form is what you've shown)
use of scalar variables rather than barewords for defining filehandles
use of the local keyword and extra curlies to modify the definition of $/ only when needed.
the second s in s/SubNetwork.*BtsSitMgr=//s allows matches over multiple lines as well.
s/^.*NULL\r?\n.*\r?\n.*\r?\n//mg;
should filter out the lines that end in NULL plus the two following lines.