I'm trying to write a tcl script basically to do following. Based on the syslog below,
LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/23, changed state to down
When that log is seen on the router, the script needs to push out a shell command which includes the interface index number, in this case it is 23. I can use regex to scrape the interface in the syslog by doing this below.
set interface ""
if {[regexp {.* (GigabitEthernet1/0/[0-9]*)} $syslog_msg match interface]}
if [catch {cli_exec $cli1(fd) "show port_diag unit 1 port 23"} result] {
error $result $errorInfo
}
}
But how can I use only the interface index number (which is 23) in the command above? Do I need to extract [0-9]* from the regexp and store it as a vairable or somethig like that?
Please just enclose the expression [0-9]* with parentheses and append
a variable name, say num, to be assigned to the second capture group.
Here is a snipped code to demonstrate:
if {[regexp {.* (GigabitEthernet1/0/([0-9]*))} $syslog_msg match interface num]} {
puts $num
}
Output:
23
If the result looks okay, modify the command within the curly braces to perform your task as:
if {[regexp {.* (GigabitEthernet1/0/([0-9]*))} $syslog_msg match interface num]} {
if [catch {cli_exec $cli1(fd) "show port_diag unit 1 port $num"} result] {
error $result $errorInfo
}
}
Related
I have the following yaml:
role::project::conf::files:
entry_01:
- 'file "/var/project/some_file.txt" 5333;'
- 'echo no;'
entry_02:
- 'file "/var/project/some_other_file.txt" 5334;'
- 'echo yes;'
entry_03:
- 'file "/var/project/extra_file.txt" 5335;'
- 'echo yes;'
then I've used the following regular expression https://regex101.com/r/pVzseA/1 in order to grab the value between those quotation marks (the regex works in regex101.com) but doesn't work in Puppet:
each($files) | $files_itm | {
if $files_itm[1] =~ /"([^"]*)"/ {
#how to get only the path here in var
}
}
Update, the class:
class role::project::conf (
$files = [],
){
each($files) | $files_itm | {
if $files_itm[1] =~ /"([^"]*)"/ {
notify { "file ${1} seen": }
}
}
}
I've used the following regular expression
https://regex101.com/r/pVzseA/1 in order to grab the value between
those quotation marks (the regex works in regex101.com) but doesn't
work in Puppet:
each($files) | $files_itm | {
if $files_itm[1] =~ /"([^"]*)"/ {
#how to get only the path here in var
}
}
As I noted in comments, Puppet uses Ruby's flavor of regular expression. This is not among those explicitly supported by Regex101, so it is not out of the question that a regex that worked there would not work with Puppet. However, the Ruby-flavor online regular expression tester at Rubular shows that your particular regular expression works just fine in Ruby, including the the matching group.
You don't show how you tried to obtain the matched group in your Puppet code, but the appropriate procedure is given in Puppet's regular expression docs. In particular:
Within conditional statements and node definitions, substrings withing [sic]
parentheses () in a regular expression are available as numbered
variables inside the associated code section. The first is $1, the
second is $2, and so on. The entire match is available as $0. These
are not normal variables, and have some special behaviors: [...]
This is also reflected in the documentation for conditional statements.
You have only one capture group, so inside the body of your conditional statement, you can refer to the captured content as $1. For example,
each($files) | $files_itm | {
if $files_itm[1] =~ /"([^"]*)"/ {
notify { "file ${1} seen": }
}
}
Note well, however, that the capture group variables are only accessible within the body of the conditional statement, and outside any nested conditionals that have a regular expression match in their condition. Also, it does not seem to be documented what the implications are if there are two or more regex-match expressions in the same conditional statement's condition.
Update
But the update to the question and the error messages you presented in comments show that the problem you're struggling with is something altogether different. The value associated with Hiera key role::project::conf::files is a hash, with string keys and array values, whereas you seem to expect an array according to the parameter default encoded into the class. When you iterate over a hash and capture the entries in a single variable, that variable will take two-element array values, where the first (element 0) is the entry's key and the second (element 1) is corresponding value.
Thus, when the Hiera data presented in the question is bound to your $files parameter, your expression $files_itm[1] evaluates to array values such as ['file "/var/project/some_file.txt" 5333;', 'echo no;']. Those are not suitable left-hand operands for the =~ operator, which is what the error message is telling you.
It's difficult to tell what you really want here, but this ought to at least avoid error (in conjunction with your YAML data):
each($files) | $itm_key, $itm_value | {
if $itm_value[0] =~ /"([^"]*)"/ {
notify { "file ${1} seen": }
}
}
This is a markdown document example.md I have:
## New language
Raku is a new language different from Perl.
## what does it offer
+ Object-oriented programming including generics, roles and multiple dispatch
+ Functional programming primitives, lazy and eager list evaluation, junctions, autothreading and hyperoperators (vector operators)
+ Parallelism, concurrency, and asynchrony including multi-core support
+ Definable grammars for pattern matching and generalized string processing
+ Optional and gradual typing
This code will be evaluated.
```{raku evaluate=TRUE}
4/5
```
Rakudo is a compiler for raku programming language. Install it and you're all set to run raku programs!
This code will be evaluated.
```{raku evaluate=TRUE}
say "this is promising";
say $*CWD;
```
This code will **not** be evaluated.
```{raku evaluate=FALSE}
say "Hello world";
```
which I want to convert into example.md as shown below with the code and output within it.
## New language
Raku is a new language different from Perl.
## what does it offer
+ Object-oriented programming including generics, roles and multiple dispatch
+ Functional programming primitives, lazy and eager list evaluation, junctions, autothreading and hyperoperators (vector operators)
+ Parallelism, concurrency, and asynchrony including multi-core support
+ Definable grammars for pattern matching and generalized string processing
+ Optional and gradual typing
This code will be evaluated.
Code:
```{raku evaluate=TRUE}
4/5
```
Output:
```
0.8
```
Rakudo is a compiler for raku programming language. Install it and you're all set to run raku programs!
This code will be evaluated.
Code:
```{raku evaluate=TRUE}
say "this is promising";
say $*CWD;
```
Output:
```
this is promising
"C:\Users\suman".IO
```
This code will **not** be evaluated.
Code:
```{raku evaluate=FALSE}
say "Hello world";
```
What I want to accomplish is:
capture the code between backticks{raku evaluate} and backticks
execute the code if evaluate is TRUE
insert the code and output back into the document
What I tried to do:
Capture multiline code and evaluate expression
my $array= 'example.md'.IO.slurp;
#multiline capture code chunk and evaluate separately
if $array~~/\`\`\`\{raku (.*)\}(.*)\`\`\`/ {
#the first capture $0 will be evaluate
if $0~~"TRUE"{
#execute second capture which is code chunk which is captured in $1
}else {
# don't execute code
};
};
create a temp.p6 file and write code chunk $1 from above into it
my $fh="temp.p6".IO.spurt: $1;
execute the chunk if $0 is TRUE
my $output= q:x/raku temp.p6/ if $0==TRUE
integrate all this into final example.md while we create intermediate example_new.md
my $fh-out = open "example_new.md", :w; # Create a new file
# Print out next file, line by line
for "$file.tex".IO.lines -> $line {
# write output of code to example_new.md
}
$fh-out.close;
# copy
my $io = IO::Path.new("example_new.md");
$io.copy("example.md");
# clean up
unlink("example.md");
# move
$io.rename("example.md");
I am stuck in the first step. Any help?
There are two ways to execute the code and capture the output:
You can write it to a tempfile and use my $result = qqx{perl6 $filename} to spawn a separate process
You can execute the code in the same interpreter using EVAL, and use IO::Capture::Simple to capture STDOUT:
my $re = regex {
^^ # logical newline
'```{perl6 evaluate=' (TRUE|FALSE) '}'
$<code>=(.*?)
'```'
}
for $input.match(:global, $re) -> $match {
if $match[0] eq 'TRUE' {
use IO::Capture::Simple;
my $result = capture_stdout {
use MONKEY-SEE-NO-EVAL;
EVAL $match<code>;
}
# use $result now
}
}
Now you just need to switch from match to subst and return the value from that block that you want to substitute in, and then you're done.
I hope this gives you some idea how to proceed.
Code that accomplishes "What I want to accomplish"
You can run this code against your data with glot.io.
use v6;
constant $ticks = '```';
my regex Search {
$ticks '{raku evaluate=' $<evaluate>=(TRUE|FALSE) '}'
$<code>=[<!before $ticks> .]*
$ticks
}
sub Replace ($/) {
"Code:\n" ~ $ticks ~ $<code> ~ $ticks ~
($<evaluate> eq 'TRUE'
?? "\n\n" ~ 'Output:' ~ "\n" ~ $ticks ~ "\n" ~ Evaluate($<code>) ~ $ticks
!! '');
}
sub Evaluate ($code) {
my $out; my $*OUT = $*OUT but role { method print (*#args) { $out ~= #args } }
use MONKEY; my $eval-result = EVAL $code;
$out // $eval-result ~ "\n"
}
spurt
'example_new.md',
slurp('example.md')
.subst: &Search, &Replace, :g;
Explanation
Starting at the bottom and then working upwards:
The .subst method substitutes parts of its invocant string that need to be replaced and returns the revised string. .subst's first argument is a matcher; it can be a string, or, as here, a regex -- &Search1. .subst's second argument is a replacement; this can also be a string, or, as here, a Callable -- &Replace. If it's a Callable then .subst passes the match from the matcher as a match object2 as the first argument to the Callable. The :g adverb directs .subst to do the search/replace repeatedly for as many matches as there are in the invocant string.
slurp generates a string in one go from a file. No need for open, using handles, close, etc. Its result in this case becomes the invocant of the .subst explained above.
spurt does the opposite, generating a file in one go from a string, in this case the results of the slurp(...).subst... operation.
The Evaluate routine generates a string that's the output from evaluating the string of code passed to it. To capture the result of evaluation it temporarily modifies Raku's STDOUT variable $*OUT, redirecting prints (and thus also says etc.) to the internal variable $out before EVALing the code. If the EVAL results in anything being printd to $out then that is returned; if not, then the result of the EVAL is returned (coerced to a string by the ~). (A newline is appended in this second scenario but not the first because that is what's needed to get the correctly displayed result given how you've "specified" things by your example.)
The Replace routine is passed a match object from a call of the Code regex. It reconstructs the code section (without the evaluate bit) using the $<code> capture. If the $<evaluate> capture is 'TRUE' then it also appends a fresh Output: section using the Evaluate routine explained above to produce the code's output.
The Code regex matches a code section. It captures the TRUE or FALSE setting from the evaluate directive into a capture named $<evaluate> and the code into a capture named $<code>.
Footnotes
1 To pass a routine (a regex is a routine) rather than call it, it must be written with a sigil (&foo), not without (foo).
2 It does this even if the matcher was merely a string!
You can try this regex:
```{perl6 evaluate=(?<evaluate>[^}]+)}\s+(?<code>[^`]+)
You will get three results from your sample text, each result contains two named groups, the first is evaluate, containing the flag and the second one codeis the code.
Have a look at the regex-demo:
https://regex101.com/r/EsERkJ/1
Since I don't know perl, i can't help you with the implementation :(
so I have little problem, because I need to print host name which is bettwen "(?# )", for example:
Apr 17 23:39:02 test pure-ftpd: (?#researchscan425.eecs.umich.edu) [INFO] New connection from researchscan425.eecs.umich.edu
And I need to print "researchscan425.eecs.umich.edu".
I tried something like:
if(my ($test) = $linelist =~ /\b\(\?\#(\S*)/)
{
print "$test\n";
}
But it doesn't print me anything.
You can use this regex:
\(\?#(.*?)\)
researchscan425.eecs.umich.edu will be captured into Group 1.
See demo
Sample code:
my $linelist = 'Apr 17 23:39:02 test pure-ftpd: (?#researchscan425.eecs.umich.edu) [INFO] New connection from researchscan425.eecs.umich.edu';
if(my ($test) = $linelist =~ /\(\?#(.*?)\)/)
{
print "$test\n";
}
How about:
if(my ($test) = $linelist =~ /\(\?\#([^\s)]+)/)
You need to remove the \b which exists before (. Because there isn't a word boundary exists before ( (non-word character) and after space (non-word charcater).
my $linelist = 'Apr 17 23:39:02 test pure-ftpd: (?#researchscan425.eecs.umich.edu) [INFO] New connection from researchscan425.eecs.umich.edu';
if(my ($test) = $linelist =~ /\(\?\#([^)]*)/)
{
print "$test\n";
}
The problem here is the definition of \b.
It's "word boundary" - on regex101 that means:
(^\w|\w$|\W\w|\w\W)
Now, why this is causing you problems - ( is not a word character. So the transition from space to bracket doesn't trigger this pattern.
Switch your pattern to:
\s\(\?\#(\S+)
And it'll work. (Note - I've changed * to + because you probably want one or more, not zero or more).
It's amazing what you can do with logging tools or with perl as part of the logging service itself (c.f. Ubic), but even if you're just writing a "quick script" to parse logs for reporting (i.e. something you or someone else won't look at again for months or years) it helps to make them easy to maintain.
One approach to doing this is to process the lines of your log file lines with Regexp::Common. One advantage is that RX::Common matches practically "self document" what you are doing. For example, to match on specific "RFC compliant" definitions of what constitutes a "domain" using the $linelist you posted:
use Regexp::Common qw /net/;
if ( $line =~ /\?\#$RE{net}{domain}{-keep}/ ) { say $1 }
Then, later, if you need you can add other matches e.g "numeric" IPv4 or IPv6 addresses, assign them for use later in the script, etc. (Perl6::Form and IO::All used for demonstration purposes only - try them out!):
use IO::All ;
use Regexp::Common qw/net/;
use Perl6::Form;
my $purelog = io 'logfile.lines.txt' ;
sub _get_ftphost_names {
my #hosts = () ;
while ($_ = $purelog->getline) {
/\(\?\#$RE{net}{IPv6}{-sep => ":" }{-keep}/ ||
/\(\?\#$RE{net}{IPv4}{-keep}/ ||
/\(\?\#$RE{net}{domain}{-keep}/ and push #hosts , $1 ;
}
return \#hosts ;
}
sub _get_bytes_transfered {
... ;
}
my #host_list = _get_ftphost_names ;
print form
"{[[[[[[[[[[(30+)[[[[[[[[[[[[[}", #host_list ;
One of the great things about Regexp::Common (besides stealing regexp ideas from the source) is that it also makes it fairly easy to roll your own matches, You can use those to capture other parts of the file in an easily understandable way adding them piece by piece. Then, as what was supposed to be your four line script grows and transforms itself into a ITIL compliant corporate reporting tool, you and your career can advance apace :-)
I need to get all macs based on a list that match checkpoint and cisco vendors. I do a "show ip arp" on Cisco Router, catch the arp output and save on file. Now, I need to check line by line what macs I have.
To be exact, I need to match only the lines containing mac addresses of the Cisco and Checkpoint vendors. To do that I have to get IP address, the first forth hex digits, the dot, and more two hex digits to define a mac vendor e.g (001c.7f or e02f.6d) and print this line on a file. After that, I need to compare the "IP-ARP.txt" and "MAC-ADDRESS.txt" (this last one contains a full mac-address vendors). If my output match, save this line on another file.
Here a piece of files:
IP-ARP.txt
Internet 172.20.14.12 0 001c.7f41.186e ARPA
Internet 172.20.14.13 57 001c.7f41.074e ARPA
Internet 172.20.14.14 0 0200.5ebd.e17d ARPA
Internet 172.20.19.11 - 7081.050f.9402 ARPA
Internet 172.20.19.12 54 7cad.7499.e602 ARPA
Internet 172.20.19.13 7 e02f.6d14.c1bf ARPA
Internet 172.20.19.14 104 e02f.6d15.1d7f ARPA
MAC-ADDRESS.txt
001c.7f
001c.ab
001b.de
001b.ff
001c.cd
001c.de
e02f.6c
e02f.7c
Thank's in advance!
(updated again due to changing spec)
What we have here is a list of identifying strings (the OUI MAC address parts written as fragments of a MAC address string) and a list of data strings that need to be checked against this list.
My solution uses the fileutil package from the Tcl library. It's not quite necessary since you could use standard Tcl commands, but it simplifies the script a lot.
package require fileutil
Define some filenames to use.
set filename(macaddr) MAC-ADDRESS.txt
set filename(iparp) IP-ARP.txt
set filename(output) output.txt
If the identifying strings list is subject to change, you may want to read it from file everytime you run your script:
set idlist [::fileutil::cat $filename(macaddr)]
Or if these addresses seldom change, you can just hard-code it in your script and edit when necessary:
set idlist {001c.7f 001c.ab 001b.de 001b.ff 001c.cd 001c.de e02f.6c e02f.7c}
Set the contents of the output file to the empty string.
::fileutil::writeFile $filename(output) {}
To select the lines in your IP-ARP.txt file that match any of these addresses, there are several ways to traverse it. My suggestion is to use the fileutil::foreachLine command. Basic invocation is like this:
::fileutil::foreachLine varName filename script
(The first parameter is an arbitrary variable name: on every iteration the current line will be stored in that variable. The second is the name of the file to traverse, and the third parameter is a script to run once for every line in that file.)
The script calls a command that matches id strings using the string match command. The regexp command could be used instead, but I think that's quite unnecessary in this case. Every line in the IP-ARP.txt file is either blank or a proper Tcl list with five elements, where the MAC address is the fourth. Also, the second element is the ip number, and only those beginning with 172 are to be used. This means that the matching command can be written like this:
proc matchid {idlist line} {
set ipAddr [lindex $line 1]
set macAddr [lindex $line 3]
if {[string match 172* $ipAddr]} {
foreach id $idlist {
if {[string match $id* $macAddr]} {
return "$ipAddr $macAddr\n"
}
}
}
}
(Matching the ip address in this way only works if the address is in dotted decimal form: if it can be in any other form the Tcllib ip module should be used to match it.)
The result of the command is either a line containing the ip address and the MAC address if the line matched, or the empty string if it didn't.
Now lets traverse the contents of the IP-ARP.txt file. For each line, match the contents against the id list and get either an output string or an empty string. If the string isn't empty, append it to the output file.
::fileutil::foreachLine line $filename(iparp) {
set res [matchid $idlist $line]
if {$res ne {}} {
::fileutil::appendToFile $filename(output) $res
}
}
And that's it. The complete program:
package require fileutil
set filename(macaddr) MAC-ADDRESS.txt
set filename(iparp) IP-ARP.txt
set filename(output) output.txt
set idlist [::fileutil::cat $filename(macaddr)]
::fileutil::writeFile $filename(output) {}
proc matchid {idlist line} {
set ipAddr [lindex $line 1]
set macAddr [lindex $line 3]
if {[string match 172* $ipAddr]} {
foreach id $idlist {
if {[string match $id* $macAddr]} {
return "$ipAddr $macAddr\n"
}
}
}
}
::fileutil::foreachLine line $filename(iparp) {
set res [matchid $idlist $line]
if {$res ne {}} {
::fileutil::appendToFile $filename(output) $res
}
}
Documentation for the Tcllib fileutil module
Documentation: foreach, if, lindex, package, proc, set, string
(Note: the 'Hoodiecrow' mentioned in the comments is me, I used that nick earlier.)
Given a partial mac address (in the form "xxxx.xx") in a variable $mac, to match a line containing a mac address starting with that value:
^.*\b$mac[0-9a-f]{2}\.[0-9a-f]{4}\b.*$
If your language matches lines that contain a pattern, you can omit the wrapping:
\b$mac[0-9a-f]{2}\.[0-9a-f]{4}\b
Based on Hoodiecrow's answer, I did it:
set Ouilist { 0000.0c 0001.42 0001.43 0001.63 0001.64 ... }
set Macs1 [open "IP-ARP.txt" r]
foreach a [split [read -nonewline $Macs1] \n] {
set macAddr [lindex $a 3]
set IP [lindex $a 1]
if { [regexp {(172)\.([0-9]+)\.([0-9]+)\.([0-9]+)} $IP RealIP] } {
regexp {([0-9a-f])([0-9a-f])([0-9a-f])([0-9a-f])\.([0-9a-f])([0-9a-f])} $macAddr OuiPart
if { $OuiPart in $Ouilist } {
puts "$RealIP $macAddr\r
}
}
That way I get all ips that start with 172 and mac-address that are Cisco and Checkpoint vendors.
If I want to match DEF_23 using the following regexp:
expect {
-re "DEF_\[0-9]*"
set result $expect_out(1,string)
}
why does it say no such element in array?
How does $expect_out work, and how can I capture the DEF using a regexp and assign it to the variable result?
You're looking for expect_out(0,string) -- the array element 1,string would be populated if you had capturing parentheses in your regular expression.
The expect manpage documents the use of expect_out in the documentation of the expect command:
Upon matching a pattern (or eof or full_buffer), any matching and previously unmatched output is saved in the variable expect_out(buffer). Up to 9 regexp substring matches are saved in the variables expect_out(1,string) through expect_out(9,string). If the -indices flag is used before a pattern, the starting and ending indices (in a form suitable for lrange) of the 10 strings are stored in the variables expect_out(X,start) and expect_out(X,end) where X is a digit, corresponds to the substring position in the buffer. 0 refers to strings which matched the entire pattern and is generated for glob patterns as well as regexp patterns.
There is an illustrative example in the manpage.
It seems that the above explication is not precise!
Check this example:
$ cat test.exp
#!/usr/bin/expect
set timeout 5
log_user 0
spawn bash
send "ls -1 db*\r"
expect {
-re "^db.*$" {
set bkpfile $expect_out(0,string)
}
}
send_user "The filename is: $bkpfile\n"
close
$ ls -1 db*
dbupgrade.log
$ ./test.exp
can't read "bkpfile": no such variable
while executing
"send_user "The filename is: $bkpfile\n""
(file "./test.exp" line 15)
$
The test result is the same when $expect_out(1,string) or $expect_out(buffer)is used.
Am I missing something or this is the expected behavior?
Aleksandar - it should work if you change the match to "\ndb.*$".
If you turn on exp_internal 1, you will see the buffer contains something like this: "ls -1 db*\r\ndbupgrade.log\r\n08:46:09"
So, the caret (^) will throw your pattern match off.