Execute module-qualified function inside perl regex substitution? - regex

I have the following perl one-liner to convert /path/to/file.txt to /path/to/
echo "/path/to/file.txt" | perl -pe 's{(.*)}{File::Basename->dirname($1)}ge'
but I'm missing something in my invocation of File::Basename->dirname(), causing the following error:
Can't locate object method "dirname" via package "File::Basename" (perhaps you forgot to load "File::Basename"?) at -e line 1, <> line 1.
What am I missing?
(I know I can just use dirname from bash but I'm trying to do something more complicated with perl than what this stripped down example shows).

Error #1:
Like the message suggests (perhaps you forgot to load "File::Basename"?), you need to load File::Basename.
perl -pe'use File::Basename; ...'
or
perl -MFile::Basename -pe'...'
Error #2:
dirname is not a method, so File::Basename->dirname is incorrect. It needs to be called as File::Basename::dirname.
perl -MFile::Basename -pe's{(.*)}{File::Basename::dirname($1)}ge'
You could also import dirname.
perl -MFile::Basename=dirname -pe's{(.*)}{dirname($1)}ge'
Fortunately, File::Basename exports dirname by default, so you can simply use
perl -MFile::Basename -pe's{(.*)}{dirname($1)}ge'

Load a module with -MModName=func
perl -MFile::Basename=dirname -pe 's{(.*)}{dirname($1)}ge'
The File::Basename module exports all its functions by default so you don't need =dirname above. But this varies between modules, and mostly you do need to import symbols. For more on how to do that in a one-liner, find the -M switch in Command Switches in perlrun.

The problem was that you had <button> in the replacement string, but you've removed that now so it should work
The replacement string must be a valid Perl expression if you're using the /e modifier, and
<button>File::Basename->dirname($1)
isn't valid Perl
The correct command would be:
echo "/path/to/file.txt" | perl -pe 'use File::Basename 'dirname'; s{([^\n]+)}{dirname($1)}ge'

Related

Sed | Variable containing regex causes invalid reference error

I'm having problems with sed and the back-referencig when using variables containing regexes.
It is a parser written in bash. At a very earlier point, I want to use sed to clean every line into the needed data: the indentation, a key and a value (colon separated). The data is similar to yaml but using an equals.
A basic example of the data:
overview = peparing 2016-10-22
license= sorted 2015-11-01
The function I'm having problems with does the logic in a while loop:
function prepare_parsing () {
local file=$1
# regex components:
local s='[[:space:]]*' \
w='[a-zA-Z0-9_]*' \
fs=':'
# regexes(NoQuotes, SingleQuotes, DoubleQuotes):
local searchNQ='^('$s')('$w')'$s'='$s'(.*)'$s'$' \
searchSQ='^('$s')('$w')'$s'='$s\''(.*)'\'$s'\$' \
searchDQ='^('$s')('$w')'$s'='$s'"(.*)"'$s'\$' \
replace="\1$fs\2$fs\3"
while IFS="$fs" read -r indentation key value; do
...
SOME CUSTOM LOGIC
...
done < <(sed -n "s/${searchNQ}/${replace}/p" $file)
}
When trying to call the function, I receive the known invalid reference error into \3: invalid reference \3 on s' command's RHS
To debug this, after the vars definition, I've printed their values using the printf and the %q option.
printf "%q\n" $searchNQ $searchSQ $searchDQ $replace
Getting these values:
\^\(\[\[:space:\]\]\*\)\(\[a-zA-Z0-9_\]\*\)\[\[:space:\]\]\*=\[\[:space:\]\]\*\(.\*\)\[\[:space:\]\]\*\$
\^\(\[\[:space:\]\]\*\)\(\[a-zA-Z0-9_\]\*\)\[\[:space:\]\]\*=\[\[:space:\]\]\*\'\(.\*\)\'\[\[:space:\]\]\*\\\$
\^\(\[\[:space:\]\]\*\)\(\[a-zA-Z0-9_\]\*\)\[\[:space:\]\]\*=\[\[:space:\]\]\*\"\(.\*\)\"\[\[:space:\]\]\*\\\$
$'\\1\034\\2\034\\3'
And maybe here's the problem, the excessive escape sequences when the shell (bash) expand the variables (for example, it seems to be escaping the *, the [], ...).
If I pass the -r option to sed, it works perfectly, but I have to avoid this since the system that will execute the script won't have this sed implementation: I have to use basic sed.
Do you have any idea on how to store the regex into variables and make them usable for the backreferencing on the RHS?
It works in these two cases:
When using a plain regex string:
sed -n "s/^\([[:space:]]*\)\([a-zA-Z0-9_]*\)[[:space:]]*=[[:space:]]*\(.*\)[[:space:]]*\$/\1:\2:\3/p" $file
And when I use just the vars s, w and fs:
sed -n "s/^\($s\)\($w\)$s=$s\(.*\)$s\$/\1$fs\2$fs\3/p" $file
Many thanks for the help!
perl that supports extended RegExps may be used instead of sed, like
perl -n -e "s/${searchNQ}/${replace}/; print"

Perl from command line: substitute regex only once in a file

I'm trying to replace a line in a configuration file. The problem is I only want to replace only one occurence. Part of the file looks like this. It is the gitolite default config file:
# -----------------------------------------------------------------
# suggested locations for site-local gitolite code (see cust.html)
# this one is managed directly on the server
# LOCAL_CODE => "$ENV{HOME}/local",
# or you can use this, which lets you put everything in a subdirectory
# called "local" in your gitolite-admin repo. For a SECURITY WARNING
# on this, see http://gitolite.com/gitolite/cust.html#pushcode
# LOCAL_CODE => "$rc{GL_ADMIN_BASE}/local",
# ------------------------------------------------------------------
I would like to set LOCAL_CODE to something else from the command line. I thought I might do it in perl to get pcre convenience. I'm new to perl though and can't get it working.
I found this:
perl -i.bak -p -e’s/old/new/’ filename
The problem is -p seems to have it loop over the file line by line, and so a 'o' modifier won't any have effect. However without the -p option it doesn't seem to work...
A compact way to do this is
perl -i -pe '$done ||= s/old/new/' filename
Yet another one-liner:
perl -i.bak -p -e '$i = s/old/new/ if !$i' filename
There are probably a large number of perl one liners that will do this, but here is one.
perl -i.bak -p -e '$x++ if $x==0 && s/old/new/;' filename

How can I disable variable interpolation with the Perl substitution operator?

I'm trying to replace a particular line in a text file on VMS. Normally, this is a simple one-liner with Perl. But I ran into a problem when the replacement side was a symbol containing a VMS path. Here is the file and what I tried:
Contents of file1.txt:
foo
bar
baz
quux
Attempt to substitute 3rd line:
$ mysub = "disk$data1:[path.to]file.txt"
$ perl -pe "s/baz/''mysub'/" file1.txt
Yields the following output:
foo
bar
disk:[path.to]file.txt
quux
It looks like Perl was overeager and replaced the $data1 part of the path with the contents of a non-existent variable (i.e., nothing). Running with the debugger confirmed that. I didn't supply /e, so I thought Perl should just replace the text as-is. Is there a way to get Perl to do that?
(Also note that I can reproduce similar behavior at the linux command line.)
With just the right mix of quotes and double quotes you can get there:
We create a tight concatenation of "string" + 'substitute symbol' + "string".
The two double quoted strings contain single quotes for the substitute.
Contrived... but it works.
$ perl -pe "s'baz'"'mysub'"'" file1.txt
That's a DCL level solution.
For a Perl solution, use the q() operator for a non-interpolated string and execute that.
Stuff the symbol into the parens using simple DCL substitution.
This is my favorite because it (almost) makes sense to me and does not get too confusing with quotes.
$ perl -pe "s/baz/q(''mysub')/e" file1.txt
As ruakh discovered when reading my comment, the problem of perl interpolation can be solved by accessing the %ENV hash, rather than using a shell variable:
perl -pwe "s/baz/$ENV{mysub}/" file1.txt
Added -w because I do not believe in not using warnings even in one-liners.
Haven't even seen a VMS system in decades, but ... escape your sigil?
$ mysub = "disk\$data1:[path.to]file.txt"
or maybe
$ mysub = "disk\\$data1:[path.to]file.txt"
?
For sh or a derivative, I'd use
perl -pe'BEGIN { $r = shift(#ARGV) } s/baz/$r/' "$mysub" file1.txt
Otherwise, you have to somehow covert the value of mysub into a Perl string literal. That would be more complicated. Find the equivalent for your shell.
Edit by OP:
Yes, this works. The VMS equivalent is
perl -pe "BEGIN { $r = shift(#ARGV) } s/baz/$r/" 'mysub' file1.txt
You have to escape the $ with \$. Otherwise Perl sees a variable reference and replaces the string $data1 with the content of $data1 before the regular expression is evaluated. As you didn't define $data1 it is, of course, empty.
You can use the following code :
$ mysub='disk\$data1:[path.to]file.txt'
$ perl -pe 's/baz/'$mysub'/' FILE
foo
bar
disk$data1:[path.to]file.txt
quux
You could try using a logical name instead of a DCL symbol:
$ define mysub "disk$data1:[path.to]file.txt"
$ $ perl -pe "s/baz/mysub/" file1.txt
Use sed.
Sed won't try to interpolate $data1 as a variable, so you should get what you want.
$ sed -e "s/baz/''mysub'/" file1.txt

vim & csv file: put header info into a new column

I have a large number of csv files that look like this below:
xxxxxxxx
xxxxx
Shipment,YD564n
xxxxxxxxx
xxxxx
1,RR1760
2,HI3503
3,HI4084
4,HI1824
I need to make them look like the following:
xxxxxxxx
xxxxx
Shipment,YD564n
xxxxxxxxx
xxxxx
YD564n,1,RR1760
YD564n,2,HI3503
YD564n,3,HI4084
YD564n,4,HI1824
YD564n is a shipment number and will be different for every csv file. But it always comes right after "Shipment,".
What vim command(s) can I use?
In one file type the following in normal mode:
qqgg/^Shipment,<CR>ww"ay$}j:.,$s/^/<C-R>a,<CR>q
Note that <CR> is the ENTER key, and <C-R> is CTRL-R.
This will update that file and recrd the commands in register q.
Then in each other file type #q (also in normal mode). (this will play back register q)
You can do this using a macro, and applying it over several files.
Here's one example. Type the following in as is:
3gg$"ayiw:6,$s/^/<C-R>a/<CR>:w<CR>:bn<CR>
Now that looks horrendous. Let me see if I can explain that a bit better.
3gg$ : Go to the end of the third line.
"ayiw : Copy the last word into the register a.
:6,$s/^/<C-R>a/<CR> : In every line from the 6th onwards, replace at the beginning whatever is in register a.
:w<CR>:bn<CR> : Save and go to the next buffer.
Now you can map this to a key, by
:nnoremap <C-A> 3gg$"ayiw:6,$s/^/<C-R>a/<CR>:w<CR>:bn<CR>
Then if you have say 200 csv files, you open vim as
vim *.csv
and then
200<C-A>
Where you type Ctrl-A there, and it should be all done.
That said, I'd definitely be more comfortable doing this in a proper scripting language, it'd be much more straightforward.
This could be done as a Perl one-liner:
perl -i.bak -e' $c = do {local $/; <>};
($n) = ($c =~ /Shipment,(\w+)/);
$c =~ s/^(\d+,)/$n,$1/gm;
print $c' shipment.csv
This will read contents of shipment.csv into $c, extract the shipment ID into $n, and prepend every CSV line with the shipment number. The file will be modified in-place with a backup saved to shipment.csv.bak.
To do this from within Vim, adapt it as a filter:
:%!perl -e' $c = do {local $/; <>}; ($n) = ($c =~ /Shipment,(\w+)/); $c =~ s/^(\d+,)/$n,$1/gm; print $c'
Well, don't bash me, but... you could consider: Don't do this in vim!!
This is a classic usage example for scripting languages.
Take a basic python, perl or ruby tutorial. The solution for this would
be in it.
The regex for this might not be too difficult and it is doable in vim.
But there are much easier alternatives out there.
And much more flexible ones.
Why vim?
Try this shell script:
#!/bin/sh
input=$1
shipment=`grep Shipment $input|awk -F, '{print $2}'`
mv $input $input.orig
sed -e "s/^\([0-9]\)/$shipment,\1/" $input.orig > $input
You could iterate through specific files:
for input in *.txt
do
script.sh $i
done
I also think this isn't well suited for vim, how about in Bash instead?
FILENAME='filename.csv' && SHIPMENT=`grep Shipment $FILENAME | sed 's/^Shipment,//'` && cat $FILENAME | sed "s/^[0-9]/$SHIPMENT,&/" > $FILENAME

How can I make this Perl one-liner to toggle character in line in a file?

I am attempting to write a one-line Perl script that will toggle a line in a configuration file from "commented" to not and back. I have the following so far:
perl -pi -e 's/^(#?)(\tDefaultServerLayout)/ ... /e' xorg.conf
I am trying to figure out what code to put in the replacement (...) section. I would like the replacement to insert a '#' if one was not matched on, and remove it if it was matched on.
pseudo code:
if ( $1 == '#' ) then
print $2
else
print "#$2"
My Perl is very rusty, and I don't know how to fit that into a s///e replacement.
My reason for this is to create a single script that will change (toggle) my display settings between two layouts. I would prefer to have this done in only one script.
I am open to suggestions for alternate methods, but I would like to keep this a one-liner that I can just include in a shell script that is doing other things I want to happen when I change layouts.
perl -pi -e 's/^(#?)(?=\tDefaultServerLayout)/ ! $1 && "#" /e' foo
Note the addition of ?= to simplify the replacement string by using a look-ahead assertion.
Some might prefer s/.../ $1 ? "" : "#" /e.